cs.CV [Total: 153]
cs.CL [Total: 61]
cs.AI [Total: 16]
cs.CR [Total: 1]
cs.HC [Total: 2]
cs.LG [Total: 29]
cs.NE [Total: 1]
cs.RO [Total: 9]
q-bio.NC [Total: 1]

cs.CV [Back]

Yunzhe Xu, Zhe Liu

TL;DR: 本文形式化受子任务依赖与资源约束的多智能体视觉语言导航，并构建MAVLN基准和协调执行系统TRISS。

Details

Motivation: 传统VLN多设定单代理执行单一指令，但真实应用可能需要多机器人协作完成任何单体都难以独立解决的任务。

Result: MAVLN含145个场景、11,724个episode、最多四人团队和三类指令机制；TRISS提供协调导航基线，实验表明调度、规划和执行仍有较大改进空间。

Insight: 把每个任务建模为具有依赖、资源占用锁和持有链约束的子任务集合；TRISS用LLM调度器分配任务、共享拓扑记忆汇聚探索所得知识，并以冲突感知执行把并发意图变为无碰撞路线。

Abstract: Vision-and-Language Navigation (VLN) has largely focused on a single agent following a single instruction, yet many real-world applications require teams of robots to tackle tasks beyond the capabilities of any individual agent. We present Systematic Multi-Agent Vision-and-Language Navigation, providing, to our knowledge, the first systematic formalization of multi-agent VLN as a constrained coordination problem: each mission consists of subtasks carrying dependency and resource constraints (presence locks and holding chains). A verified four-stage crafting pipeline instantiates the task as MAVLN, comprising 11,724 episodes across 145 scenes with teams of up to four agents under three instruction regimes, accompanied by tailored constraint-aware metrics. We further present TRISS, a coordination-ready navigation system coupling an LLM-based subtask scheduler, a shared topological memory that turns each agent’s exploration into team knowledge, and a conflict-aware execution mechanism that realizes simultaneous intentions as collision-free routes. Extensive experiments establish TRISS as a comprehensive baseline and reveal substantial room for improvement across scheduling, planning, and execution, highlighting the challenges of coordinating under MAVLN task constraints. Project page: https://xyz9911.github.io/mavln.

[2] CoVLM-Bench: A Real-World Benchmark for Cooperative Driving Question Answering and Planning cs.CVPDF

Kang Yang, Shuai Liu, Hang Li, Yance Fang, Deying Li

TL;DR: CoVLM-Bench为同一真实车路协同场景联合提供问答与轨迹规划任务，CoVLM-Drive则用配对车端和路侧视角统一处理两者。

Details

Motivation: 现有自动驾驶VLM研究主要是自车视角，路侧观察可补充视野外信息；但问答与轨迹规划尚未在同一真实车路场景中联合评测。

Result: 基准含2,196组配对帧和35,136条场景问答标注，规划目标为三秒内六个航点。问答适配提高答案准确率，CoVLM-Drive终点误差低于对比V2X规划器；问答初始化及理由监督也分别降低规划误差。

Insight: 标注结合模型起草、记录数据计算和人工核验，并提供三段式理由辅助监督。把车路联合场景理解与未来轨迹放入同一VLM基线，便于研究语义问答训练如何迁移至规划。

Abstract: Vision-language models (VLMs) have made substantial progress in autonomous driving, but their success has primarily been studied in ego-centric scenes. Infrastructure-side observations provide views beyond the ego vehicle’s field of view, yet conventional cooperative-driving systems typically transform them into geometric representations for downstream perception and planning. Directly incorporating these views into VLMs offers an opportunity to improve cooperative scene understanding and trajectory planning. However, question answering and trajectory planning have not been jointly evaluated on the same real-world vehicle-infrastructure scenes. We present CoVLM-Bench, a benchmark for cooperative driving question answering (CDQA) and cooperative planning (CP) on vehicle-infrastructure paired scenes. CoVLM-Bench provides scene-grounded CDQA annotations, three-part rationales as auxiliary supervision, and future trajectory targets derived from recorded ego motion. It contains 2,196 paired frames with 35,136 CDQA annotations, while CP predicts six waypoints over a three-second horizon. The annotations combine model-assisted drafting, record-based computation, and human verification. Built upon CoVLM-Bench, we introduce CoVLM-Drive, a unified VLM baseline that directly uses paired views for both CDQA and CP. Experiments show that CDQA adaptation improves answer accuracy and that CoVLM-Drive reaches a lower FDE than the compared V2X planners; QA initialization and rationale supervision each reduce planning error. Together, CoVLM-Bench and CoVLM-Drive support the training and comparison of VLMs for cooperative scene understanding and planning.

[3] CoDimRecon: Agentic Reconstruction of Sim-Ready 3D Scenes with Deformable Curves, Surfaces, and Volumes cs.CV | cs.GR | cs.ROPDF

Shuzhao Xie, Lelin Wang, Guying Lin, Zhi Wang, Minchen Li

TL;DR: CoDimRecon用代理式流程从多视角RGB重建含刚体、关节物体及曲线、壳体、实体等可变形对象的可编辑、可仿真3D场景。

Details

Motivation: 真实场景到可仿真3D重建多假设物体刚性，难处理弯曲、薄壳或实体形变，而其几何和物理模型依对象维度与材料而异。

Result: 在Replica和ScanNet++场景上，CoDimRecon取得有竞争力的组合重建精度，同时能生成杆、壳和实体仿真资产；还展示机器人交互，包括由行为测试驱动的纸张塑性弯折案例。

Insight: 场景级几何先验确定尺度和布局，生成网格帮助对象级精细重建，关节刚体则拆分为显式可动部件。可复用模拟器技能初始化兼容物理模型，代理通过行为测试发现不匹配并定向修改运动、几何、数值或材料设置。

Abstract: Reconstructing simulation-ready 3D scenes from real-world observations enables robotics, gaming, and immersive applications, yet existing methods largely assume rigid objects. This leaves an important gap for deformables, whose simulation-ready geometry depends on dimensionality (curves, surfaces, or volumes) and whose behavior may require models beyond elasticity. We present CoDimRecon, an agentic framework that reconstructs editable scenes containing rigid, articulated, and deformable objects from multi-view RGB observations. Scene-level geometric priors ground scale and layout, while object-level generated meshes guide the agent toward detailed, compact geometry; articulated rigid objects are decomposed into movable parts with explicit joints. For deformables, category-wise agent sessions reconstruct curves as centerlines with radii, surfaces as manifold shells with thickness, and volumes as watertight solids for volumetric meshing. Reusable simulator skills initialize compatible physical models and parameters, while agent-guided behavioral tests expose mismatches and trigger targeted revisions of motion, geometry, numerics, or material modeling. On evaluated Replica and ScanNet++ scenes, CoDimRecon achieves competitive compositional reconstruction accuracy while additionally producing deformable assets for rod, shell, and solid simulation. We further demonstrate robot interactions across all three representations, including a controlled paper-folding case in which behavioral testing motivates plastic bending.

[4] AerialDojo-200K: A Large-Scale Benchmark Suite for Open-World Aerial Object-Goal Search cs.CV | cs.AI | cs.ET | cs.MM | cs.ROPDF

Tongtong Feng, Xin Wang, Haoran Hou, Ren Wang, Weiran Wang

TL;DR: AerialDojo-200K构建跨城市、自然、基础设施和灾害场景的开放世界空中物体目标搜索基准，支持语义目标和图像目标导航。

Details

Motivation: 空中代理需自主探索大规模、非结构化三维环境并寻找语义描述或参考图像指定的目标；已有基准规模小、格式和动作空间不统一，难以支持大规模训练及跨基准评估。

Result: 基准含42个仿真场景、21类场景、109个地标、2,099个目标和205,732个任务实例，规模为既有最大基准的三倍场景、18.7倍任务。对五种开源与四种闭源MLLM评测显示，通用空中代理仍有很大提升空间。

Insight: 标准化场景划分为21个域内及21个域外场景，每个任务配无碰撞参考路线和多视角视频，并覆盖语义/图像目标及基础、标准、长时程难度，以支持可扩展泛化评估。

Abstract: Open-world aerial object-goal search is a foundational yet challenging task, requiring aerial agents to autonomously explore large-scale, unstructured three-dimensional environments and reach target objects specified by semantic descriptions or reference images, rather than following route-specific instructions. However, research in this task remains at a nascent stage and relies on small, environment-specific benchmarks with heterogeneous action spaces and data formats. These limitations hinder large-scale training and cross-benchmark evaluation, constraining the scalability and generalizability of aerial agents. To address this problem, we propose AerialDojo-200K, a large-scale benchmark suite for open-world aerial object-goal search, with 3 times as many scenes and 18.7 times as many task instances as the largest existing benchmark for this task. Specifically, we construct 42 simulation scenes spanning four scene families and 21 scene types, including 18 urban, 12 natural, six infrastructure, and six disaster scenes. To ensure data quality, 12 annotators spent two months manually annotating 109 landmarks, 2099 target objects, and 2099 object anchors across these scenes. We further construct 205,732 task instances, comprising over 100K semantic-goal and over 100K image-goal instances across Base, Standard, and Long-Horizon settings. Each task instance includes a collision-free reference trajectory and corresponding multi-view video recordings. We also develop a unified evaluation framework with a scene partition comprising 21 in-distribution scenes and 21 out-of-distribution scenes. Finally, our evaluation of five open-source and four closed-source multimodal large language models reveals that there is still a long way to go toward achieving general-purpose aerial agents. All can be found at https://fengtt42.github.io/AerialDojo/.

[5] One Geometry, Different Outcomes: Readout-Dependent Effects of the Modality Gap in Vision-Language Models cs.CV | cs.AI | cs.LGPDF

Aditya Sharma, Divya Saxena

TL;DR: 本文把视觉语言模态间隔分解为对分类、跨模态检索和混合模态检索各不相同的几何作用，解释为什么同一干预会产生相反效果。

Details

Motivation: 缩小模态间隔有时提升零样本分类与跨模态对齐，去除间隔结构却可能损害图文检索，现有结果看似相互矛盾。

Result: CLIP与SigLIP中，单一主方向解释图文均值分离平方范数的94.4%至99.9%。几何派生指数与网格搜索最优值Spearman相关0.93，并能在部分设置恢复检索表现，但跨设置迁移不一致。

Insight: 分类中查询侧固定间隔偏移扣除等价于增加类别偏置；标准跨模态检索去除方向并归一化会丢掉候选范数信息，形成乘法排序畸变；混合模态检索中该方向编码模态类别，去除后反而可能改善跨模态排序。干预须按读出任务选择。

Abstract: Contrastive vision-language models learn shared embedding spaces by aligning matched image-text pairs, yet their representations remain separated by a modality gap. Prior work reports divergent effects of modifying this gap: reducing it can improve zero-shot classification and cross-modal alignment, whereas removing gap-related structure can degrade image-text retrieval. In this paper, we provide a unified geometric explanation for these task-dependent effects. Across CLIP and SigLIP encoders, we find that a single dominant direction captures 94.4-99.9% of the squared norm of the image-text mean separation, revealing that the mean-separation component is approximately rank-one. A decomposition of the similarity score then identifies three task-specific roles. In zero-shot classification, query-side fixed gap-offset subtraction is exactly equivalent to an additive class bias. In standard cross-modal retrieval, projecting out the gap direction and renormalising residuals discards candidate-specific norm information, inducing a multiplicative ranking distortion; a geometry-derived exponent tracks the grid-search optimum (Spearman rho = 0.93) and restores performance in some settings, although the gains transfer unevenly. In mixed-modal retrieval, the gap direction sorts candidates by modality; its removal can improve cross-modal ranking, unlike random or non-gap controls. Residual semantic structure after removal defines the limits of the rank-one account. Together, these results explain why gap modification can improve, degrade, or restore performance across downstream settings. By clarifying when and why gap modification changes model behavior, this account provides a principled basis for selecting gap interventions in similarity-based vision-language systems across evaluated downstream tasks.

[6] Xiaomi-OCR-0 Technical Report cs.CVPDF

Xin Chen, Anan Du, Feng Feng, Pei Fu, Jian Luan

TL;DR: Xiaomi-OCR-0是面向文档解析和OCR理解的0.8B视觉语言模型，通过自动数据引擎和渐进训练实现高效小模型文档阅读。

Details

Motivation: 小型OCR专用VLM可以处理文档，但常依赖昂贵监督且主要优化文字重建，文档语义理解仍需增强。

Result: 约1.7亿样本的OCR语料训练后，Real5-OmniDocBench得分95.24、OmniDocBench v1.6为96.83、Wild-OmniDocBench为87.94；五项OCR-VQA基准平均83.2。消融显示解析训练充分后，理解监督还可进一步提升解析。

Insight: 自动数据引擎结合专家共识、渲染验证和定向合成；从Qwen3.5-0.8B出发，渐进结合Q-Mask文本锚定、持续预训练与混合任务强化学习。

Abstract: Compact OCR-specific vision-language models achieve strong document parsing performance, but often rely on costly supervision and focus primarily on visual-text reconstruction. We introduce Xiaomi-OCR-0, a unified 0.8B model for document parsing and OCR-centric understanding. We build an approximately 170M-sample OCR-centric corpus using an automated data engine that combines expert consensus, render-based verification, and targeted synthesis. Starting from Qwen3.5-0.8B, our progressive training recipe combines Q-Mask-based text anchoring, continued pretraining, and mixed-task reinforcement learning (Mix-RL). Xiaomi-OCR-0 achieves 95.24 on Real5-OmniDocBench, 96.83 on OmniDocBench v1.6, and 87.94 on Wild-OmniDocBench, while reaching an average score of 83.2 across five OCR-oriented VQA benchmarks. Ablations further show that, with sufficient parsing training, OCR-centric understanding supervision provides additional gains for document parsing. Homepage: https://huggingface.co/spaces/SeerRay-Lab/Xiaomi-OCR-0.

[7] From Sharp Eyes to Expert Mind: Internalizing Expert Knowledge in MLLMs for Tampered Text Detection cs.CVPDF

Kaiqing Lin, Songze Li, Shen Chen, Yunfei Guo, Xiaoye Qiu

TL;DR: EKI分阶段把法证专家的细粒度篡改痕迹感知迁移到多模态大模型内部，使推理时无需调用外部专家。

Details

Motivation: 专用篡改检测器能发现细微痕迹却跨文档域泛化弱，MLLM语义理解和迁移较强却不敏感于细粒度法证线索；两者在空间精度和感知粒度上存在双重错配。

Result: 多个域内和跨域基准上，EKI达到先进表现并强于现有专家模型或MLLM方法；专家只在训练时需要，推理效率几乎与原始MLLM相同。

Insight: 第一阶段通过文本聚焦和图像聚焦令模型注意小文本区域；第二阶段以法证通用表征对齐损失，将浅层LLM表示对齐预训练专家，使低层痕迹线索在被高层语义抽象稀释前内化。

Abstract: Tampered Text Detection (TTD) is essential for safeguarding document authenticity in security-critical workflows. Existing expert models are effective at capturing subtle manipulation traces but often generalize poorly across diverse document domains, while Multimodal Large Language Models (MLLMs) offer stronger semantic understanding and transferability yet remain insensitive to fine-grained forensic artifacts. This complementarity motivates us to investigate how expert forensic perception can be internalized into an MLLM rather than merely accessed through an external module. We identify a fundamental Double Mismatch that hinders this goal: a Spatial Precision Mismatch between coarse visual tokens and tiny tampered regions, and a Perceptual Granularity Mismatch between semantics-oriented pre-training and low-level forensic perception. To address these challenges, we propose Expert Knowledge Internalization (EKI), a progressive two-stage framework that transfers forensic expertise into the MLLM itself. In Stage 1, Text-Focused and Image-Focused strategies establish precise spatial focus on small text regions. In Stage 2, the proposed Forensic-General Representation Alignment (FGRA) loss aligns shallow LLM representations with those of a pre-trained forensic expert, enabling the model to acquire fine-grained artifact perception before such cues are diluted by deeper semantic abstraction. Extensive experiments on multiple in-domain and cross-domain benchmarks demonstrate that EKI achieves state-of-the-art performance and stronger generalization than existing expert-model-based and MLLM-based methods. Moreover, the expert is required only during training, allowing the resulting MLLM to maintain inference efficiency nearly identical to the vanilla model without relying on any external expert at inference.

[8] Exploring Learning Models for Topological Relationship Recognition from Image Data cs.CV | cs.LGPDF

Saptak Das, Monidipa Das

TL;DR: 本文构建逾1.1万张拓扑关系图像数据集，比较传统机器学习与深度网络识别接触、重叠、分离和包含等关系。

Details

Motivation: 拓扑关系识别对GIS、生物医学和机器人等应用重要，但图像中对象的接触、重叠、分离或包含关系研究不足，受限于数据和评测基准缺乏。

Result: VGG16等深度方法优于朴素贝叶斯、KNN、随机森林、SVM和ANN；VGG16验证准确率达到89.55%，显示迁移学习适合图像拓扑分类。

Insight: 使用分割、轮廓提取和灰度归一化构造特征，并建立有标签数据集，为空间推理相关拓扑分类提供可比较基准。

Abstract: Figuring out how objects relate to each other, like whether they touch, overlap, stay completely separate or one sits inside another, matters a lot in fields like GIS, biomedical imaging, and robotics. Even though machine learning has come a long way, people haven’t really focused on spotting these topological relationships in images. The main roadblocks? Not enough good datasets and no clear way to measure results. So, we rolled up our sleeves and built a new dataset. It’s pretty sizable: over 11,000 labelled images showing all those essential relationships. We ran tests with some classic machine learning models, Naive Bayes, KNN, Random Forest, SVM, and Artificial Neural Networks, and threw in some deep learning stars like VGG16 and InceptionResNetV2. For the dataset itself, we used segmentation, contour detection, and grayscale normalization to tease out solid feature vectors. The results? Deep learning methods, especially VGG16, pulled ahead, with validation accuracy hitting 89.55%. That’s a big jump compared to the traditional models. This shows how powerful transfer learning is for analyzing topological relationships in images, and it gives researchers a new standard to aim for in future work on spatial reasoning and topological classification.

[9] PreviewDiff: Multimodal Critic-Guided Search over Diffusion Latents cs.CV | cs.AI | cs.LGPDF

Vighnesh Subramaniam, Boris Katz, Brian Cheung, Chun-Liang Li, Tomas Pfister

TL;DR: PreviewDiff在扩散去噪中途解码预览并让多模态评判器给出批评，再对有希望的潜变量轨迹分支搜索和修正。

Details

Motivation: Best-of-N只能从已完成候选中选择，无法在一条有前景的生成轨迹失败前修补对象数量、属性绑定、空间关系或时间动作等组合细节。

Result: 在图像和视频基准上，PreviewDiff持续超过预算匹配的Best-of-N及强标量搜索基线；消融显示更早干预和更宽搜索收益最大，深度搜索及更多语义变体带来互补提升。

Insight: 在选定去噪检查点生成部分预览，要求多模态评判器评分并以自然语言点评，再据此分支修改语义提示和局部重加噪潜变量；分支评分后选择性推进，让验证算力参与生成过程。

Abstract: Diffusion models can produce striking images and videos, but they still struggle with the compositional details that make a generation faithful to a prompt, such as object counts, attribute binding, spatial relations, and temporally grounded actions. A common way to improve prompt satisfaction is to spend more compute at test time through Best-of-N sampling, but final-sample selection is fixed. Best-of-N can only choose among completed outputs and cannot repair a promising trajectory before it fails. We introduce PreviewDiff, a training-free test-time search method that turns diffusion sampling from scalar search into a multimodal critic-guided search over intermediate latents. At selected denoising checkpoints, PreviewDiff decodes a partial preview, asks a multimodal judge to score and critique it, and uses the resulting natural-language feedback to branch over semantic prompt edits and locally re-noised latent continuations. These branches are then scored and selectively rolled forward, allowing verifier compute to guide generation while the sample is still editable. Across image and video generation benchmarks, PreviewDiff consistently improves over budget-matched Best-of-N selection and strong scalar-search baselines. Ablations show that earlier interventions and increased search width provide the largest gains, while deeper search and additional semantic variants offer complementary improvements. PreviewDiff demonstrates that multimodal feedback is most useful not only as a final verifier, but as an active controller inside the denoising process.

[10] Sparse-View Interpretable 3D Animal Behavior Representations for Neural Encoding and Decoding cs.CVPDF

Xinming Dai, Qihang Jin, Tianshu Tan, Baiyuan Chen, Hanrui Lyu

TL;DR: SABLE以单目深度与姿态几何先验增强多视角Transformer，从极稀疏观察中学习可解释的三维动物行为表示。

Details

Motivation: 脑—行为研究需要精确结构化行为表征；姿态追踪会丢弃预设关键点外的信息，非线性视频嵌入又难以解释。

Result: 无需三维真值标签，SABLE能从双视角视频恢复3D行为，而先进方法会失败或得到退化解；在International Brain Lab和Cheese3D数据上，神经编码和解码表现达到或超过此前最优，并可零样本迁移到未见动物。

Insight: 多视角Transformer加入单目深度与姿态估计先验，在恢复行为的同时学习显式三维潜结构；跨动物预训练后无需动物特定校准或再训练即可泛化。

Abstract: A deeper understanding of brain function requires a precise, structured characterization of behavior.Yet, extracting behavioral representations from video in a form suitable for scientific analysis remains a fundamental challenge. Many prior studies represent behavior via pose estimation or nonlinear video embeddings. However, pose tracking discards rich information beyond predefined keypoints, while nonlinear video embeddings lack interpretability. We address this limitation with SABLE (Sparse-view Animal Behavior Latent Embeddings), a self-supervised framework that leverages a geometric inductive bias to learn behavior representations.By augmenting a multi-view transformer with priors from monocular depth and pose estimation, SABLE reconstructs 3D animal behavior from extremely sparse views while learning explicit 3D latent structure. Without ground-truth 3D labels, it reliably recovers 3D behavior from two-view videos, whereas state-of-the-art (SOTA) methods fail or yield degenerate solutions. Across the International Brain Lab and Cheese3D datasets, we demonstrate that SABLE learns 3D representations that match or exceed prior SOTA performance in neural encoding and decoding. Once pretrained across animals, SABLE serves as an off-the-shelf model that generalizes zero-shot to unseen animals without animal-specific calibration or retraining. Our method establishes 3D-aware video embeddings that capture complex behavior, opening new avenues for studying brain-behavior relationships.

[11] LeRF: Learning Reference Coordinate Frames for Perspective Taking Reasoning cs.CVPDF

Bang Xiao, Wenqi Jia, Ozgur Kara, Tiancheng Shen, Yibo Yang

TL;DR: LeRF训练VLM在需要换位思考时构建显式参考坐标系，并把渲染坐标框覆盖到图像辅助视角相关推理。

Details

Motivation: VLM仍常默认相机视角回答问题，即使指令要求从另一实体或想象观察者视角判断空间关系。

Result: 在多种视角转换基准上，LeRF持续超过骨干模型并对开源方法有竞争力；参考系定位和朝向估计也有所改善。

Insight: 模型先判断是否需要坐标系，再定位参考实体并预测原点和实体中心坐标；轻量渲染器把参考系叠加到图像，无需外部感知模型或显式3D重建。监督微调教会选择性调用和坐标预测，随后以空间VQA强化学习优化其引导推理。

Abstract: Perspective taking is a fundamental component of spatial intelligence, requiring models interpret spatial relations from a specified viewpoint, such as that of another entity or an imagined observer. Despite the increasing spatial reasoning capabilities of Vision-Language Models (VLMs), they still struggle with perspective taking, often defaulting to the camera viewpoint when a query requires reasoning from a different perspective. We introduce Learning Reference Coordinate Frames for Perspective Taking (LeRF), a framework that trains VLMs to construct and use explicit reference frames for viewpoint-dependent reasoning. Given an image and a query, LeRF decides whether a coordinate frame is necessary. If so, it grounds the reference entity and predicts the frame’s origin and entity-centered reference frame. A lightweight renderer overlays the frame onto the image, enabling subsequent reasoning over these visual cues without external perception models or explicit 3D reconstruction. To learn this process, we first perform supervised fine-tuning to teach selective tool invocation and reference coordinate frame prediction, followed by reinforcement learning on spatial VQA pairs to improve frame-guided reasoning. Across diverse perspective-taking benchmarks, LeRF consistently improves over its backbone and achieves strong performance against existing open-source methods. Further evaluations also show improved reference-frame grounding and orientation estimation, supporting the effectiveness of learned reference frames for viewpoint-dependent reasoning.

[12] Mutually Adversarial Self-Training with Evolving Data for Unified Multimodal Models cs.CV | cs.AIPDF

Wentao Zhou, Weijie Gan, Jiayun Wang

TL;DR: MATE让统一多模态模型的图像理解和生成分支轮流互相出题，并用每轮最难样例驱动对手式自训练。

Details

Motivation: 图像生成与理解互为逆任务，已有统一模型常让两分支合作监督，但合作可能无法持续暴露彼此薄弱点。

Result: 在Janus-Pro-1B上，MATE使GenEval提升2.4分、DPG-Bench提升1.7分、九个理解基准均分提升0.7分，同时改善反复图文循环的一致性。

Insight: 理解分支提出多种图像描述供生成分支重建，生成候选也可反过来供理解；先检查候选与原图或提示一致性，再选求解方最难完成者。自身输出成为对手，且击败一方的候选用于下一轮挑战另一方，无需另训对抗模型。

Abstract: Unified multimodal models (UMMs) combine image generation and visual understanding in a shared backbone. Since generation and understanding are inverse tasks, recent studies self-train UMMs by letting the two branches cooperatively supervise each other. We introduce MATE (Mutually Adversarial self-Training with Evolving data), a reinforcement-learning-based post-training framework in which the two branches instead challenge each other, and the challenges evolve as the model trains. MATE lets generation and understanding take turns to be challenger and solver. Given an image, the understanding branch proposes several candidate descriptions that the generation branch must turn back into similar images, and vice versa. The candidates are screened for consistency with the image or prompt they were proposed from, and the solver is trained on the candidate it handles worst. The adversary thus comes from the model’s own outputs, and no separate adversary is trained. Moreover, the candidates that defeat one branch become the sources of the next challenges to the other in the next epoch, which keeps the challenges evolving with the model and turns the training into self-play in data space. On Janus-Pro-1B, MATE improves GenEval by 2.4 points, DPG-Bench by 1.7 points, and the average over nine understanding benchmarks by 0.7 points, while strengthening consistency across repeated image-text cycles.

[13] StructRL: Online Structured Reinforcement Learning for Long-Horizon Vision-Language-Action Tasks cs.CV | cs.LG | cs.ROPDF

Ziyi Yin, Sangmin Woo, Kang Zhou, Sungyeon Kim, Aosong Feng

TL;DR: StructRL把长程VLA操作拆成可验证子任务，只在前置步骤完成后按完成速度发放中间奖励，以提供比终局成功更细的在线RL监督。

Details

Motivation: VLA在短程操作较强，却难以完成单条指令要求的多步依赖操作；若仅完整任务成功后才有奖励，部分完成的轨迹和早期失败难以区分。

Result: 在RoboCasa365和LIBERO-Long上，对GR00T-N1.5及π0.5使用StructRL均持续超过所评估在线强化学习基线；摘要未报告具体数值。

Insight: 将任务分解为可验证子任务，仅当所有前置子任务完成后才奖励相应完成，并根据完成速度调整奖励大小，提升长时程训练信号的结构性。

Abstract: Vision-language-action (VLA) models perform well on shorter-horizon manipulation tasks but still struggle with long-horizon tasks that require multiple dependent manipulations from a single command. Online reinforcement learning (RL) can improve these policies through environment interaction, yet many existing methods provide reward only after the complete task succeeds. However, such terminal supervision is sparse and does not distinguish early failures from rollouts that make substantial partial progress. We propose StructRL, an online RL framework that constructs structured intermediate supervision from verifiable subtask completions. StructRL decomposes each task into verifiable subtasks, grants intermediate rewards only after the prerequisite subtasks have been completed, and scales each reward according to completion pace. Across RoboCasa365 and LIBERO-Long with GR00T-N1.5 and pi 0.5, StructRL consistently outperforms evaluated online RL baselines. These results show that verifiable, structured intermediate rewards improve long-horizon VLA post-training. Code is available at https://github.com/amazon-science/StructRL.

[14] OTT3R: Multi-View 3D Reconstruction and Fast Dataset Generation at 1% Compute cs.CVPDF

Brandon Leblanc, Charalambos Poullis

TL;DR: OTT3R以蒸馏把大型前馈3D重建教师压缩为轻量学生，并提供无需COLMAP的高速多视角伪标签生成管线。

Details

Motivation: 高质量前馈3D重建依赖大模型和大数据，训练成本使多数团队难以承担，也阻碍边缘部署；无传感器3D监督仍依赖慢且不稳定的SfM。

Result: 将9.59亿参数π³蒸馏到1.02亿参数学生，压缩9.4倍、推理最多快7倍，训练计算仅VGGT的1.6%。两张普通GPU可在3.5小时为66.7万图像生成稠密点图和相机位姿。域专用学生在7-Scenes精度为COLMAP的4倍、吞吐高980倍。

Insight: 通用学生能跟随教师的域内单目深度，但不能取代教师处理域外多视角几何；仅用约0.2%计算进行领域专化后才成为部署产物。论文区分通用迁移能力和特定领域重建能力，避免把轻量模型泛化范围夸大。

Abstract: Feed-forward 3D reconstruction models have achieved impressive performance by scaling model and dataset size, but their cost excludes most research groups and precludes edge deployment. Additionally, generating 3D supervision without sensors still relies on slow, unreliable Structure-from-Motion, as the community lacks a COLMAP-like system for neural 3D pseudo-label generation. We present OTT3R (RGB-Only Tiny Transformer for 3D Reconstruction), a knowledge distillation framework that addresses both problems on a single workstation equipped with 2 GPUs. Distilling $π^3$ (959M parameters) into a 102M-parameter student yields 9.4$\times$ compression and up to 7$\times$ faster inference, trained at 1.6% of VGGT’s training compute. An integrated pseudo-label pipeline offers a reliable, high-throughput alternative to COLMAP, generating dense per-pixel point maps and SE(3) camera poses for a 667K-image corpus in 3.5 hours on two commodity GPUs and succeeding on every sequence we tested, including those where COLMAP fails. The general student tracks the teacher on in-distribution monocular depth and, zero-shot, outperforms COLMAP on 7-Scenes and on DTU completion, but it does not replace the teacher on out-of-distribution multi-view geometry. The deployable artifact is the domain-specialized student: after specialization at 0.2% compute, it is 4$\times$ more accurate than COLMAP on 7-Scenes at 980$\times$ throughput, with near-teacher completion. Code is available at https://github.com/TheFourthKaramazov/OTT3R

[15] Compress to Remember: Learning Compact Memory via On-Policy Distillation for Long Video Generation cs.CV | cs.AI | cs.LGPDF

Xiaoyu Wu, Weihang Guo, Yifei Wang, Xinze Feng, Lydia E. Kavraki

TL;DR: PACC通过在线策略蒸馏训练压缩器，把长视频生成历史转成紧凑记忆token，同时保留冻结生成器所需的未来信息。

Details

Motivation: 视频生成历史不断增长会导致长上下文退化；关键帧选择可缩短历史，却可能丢掉未来生成需要的信息，标准生成器也没有原生历史压缩机制。

Result: 在联合衡量记忆事件覆盖与一致性的MBench上，PACC比最强基线在Causal-rCM高6.63分、Causal Forcing高3.19分；在VBench-Long的MovieGen提示上可生成画质与基线相当的分钟级视频。

Insight: 压缩学生在紧凑记忆条件下与使用完整历史的同一冻结生成器教师对齐；两者对相同噪声输入、各去噪步给出预测目标，仅更新压缩器，不改底层视频生成器。

Abstract: Standard video generators do not natively compact historical context into reusable memory tokens. As generation continues, the growing history makes it increasingly difficult to retain information from earlier frames due to long-context degradation. Key-frame-based approaches address this challenge by retaining selected past frames, but can discard information needed for future generation. Rather than relying on frame selection alone, we study whether a frozen video generator can supply the supervision needed to learn a compact representation of the history. We propose Prediction-Aligned Context Compaction (PACC), which uses a learned compressor to aggregate information across past frames into compact memory tokens. We train the compressor through on-policy distillation, using the same frozen generator both as a student when conditioned on compressed memory and as a teacher when conditioned on the full history. The student generates continuations, while the teacher provides targets for the same noisy inputs at each denoising step. Only the compressor is updated to align the student’s predictions with these targets. We evaluate PACC on MBench, which jointly measures memory-event coverage and consistency. PACC outperforms the strongest baseline by 6.63 points on Causal-rCM and 3.19 points on Causal Forcing. Evaluation on VBench-Long using MovieGen prompts further shows that PACC produces minute-long videos with generation quality competitive with baselines. Together, these results show that learning to compact historical context can improve long-video memory without modifying the underlying generator.

[16] LEGO-Anything: Coding Agents for 3D Scene Reconstruction cs.CVPDF

Xirui Li, Peng Shi, Mingwen Dong, Sheng Zhang, Zhuoyan Xu

TL;DR: LEGO-Anything让编码代理迭代编写和执行Blender程序，从单图恢复可检查、可编辑的场景，而非只输出固定3D模型或渲染图。

Details

Motivation: 单图3D重建若能表示为场景程序，便可继续查询和编辑；评测需区分程序有效、表面几何正确及整体渲染外观是否忠实。

Result: LEGO-Bench含104个室内外场景共208张图像；最佳GPT-6-astra室内分数53.4%、室外39.6%。训练免插件LEGO-Plugin提升六个代理的整体分数，最高相对增益62.7%。

Insight: 代理反复生成Blender代码、执行、查看场景和渲染后修订。轨迹分析暴露初始化弱、迭代退步和自评不可靠三类问题；从重建场景导出的检测、实例分割与相对深度虽有非平凡表现，仍远低于专用视觉模型。

Abstract: A 3D scene reconstructed from a single image is most useful when represented not as a rendering or a fixed 3D output, but as an explicit scene program whose execution yields a scene that can be inspected, edited, and queried. We present LEGO-Anything, an Image-to-Code framework in which a coding agent iteratively writes and executes Blender code, inspects scenes and renderings, and revises the program. To evaluate end-to-end scene recovery, we introduce LEGO-Bench, a simulator-grounded benchmark with 208 images from 104 diverse indoor and outdoor scenes. LEGO-Bench separately scores artifact validity, visible-surface geometry, and rendered appearance. Its simulator-grounded design enables extensibility and precise automatic evaluation. Among evaluated agents, GPT-6-astra achieves the strongest overall results, with 53.4% indoor and 39.6% outdoor scores, yet substantial gaps remain between delivering valid scene artifacts and faithfully recovering scene geometry and appearance. Analysis of agent construction trajectories reveals three recurring issues: weak scene initialization, regressive edits during iteration, and unreliable self-evaluation. These findings motivate LEGO-Plugin, a training-free harness plugin for more controlled iterative scene construction, which improves all six evaluated models, with relative gains of up to 62.7% in overall score. Finally, we test whether reconstructed scenes can represent natural images and support vision tasks. In LEGO-World, we derive object detections, instance masks, and relative depth as deterministic queries on scenes reconstructed by GPT-6-astra. These readouts show non-trivial performance across all three tasks but fall well short of specialized vision models, suggesting that program-constructed scenes from current coding agents are a promising but not yet sufficiently precise representation of natural images.

[17] DARE to Mitigate Hallucination: Dual-path Auto-Regressive-aware Editing cs.CVPDF

Jae-Ho Lee, Jeong-Eun Lee, Gyeong-Moon Park

TL;DR: DARE联合教师强制文本差异、自回归解码状态转移和配对图像视觉差异构造编辑方向，减少LVLM对象幻觉。

Details

Motivation: 训练免表征编辑常用真实与幻觉回答的教师强制对比，但模型实际生成是自回归的，教师强制分析可能没覆盖导致幻觉的解码动态和多模态交互。

Result: 在多个LVLM幻觉基准上，DARE持续降低对象幻觉，同时保留多模态感知能力和推理效率；摘要未提供具体指标。

Insight: 编辑方向融合三种互补信号：教师强制文本对比、自回归解码中的表征变化，以及配对图像之间受控的视觉差异，避免只依据教师强制上下文进行干预。

Abstract: Large vision-language models (LVLMs) have recently achieved remarkable progress across multimodal tasks, yet object hallucination remains a persistent challenge where models generate descriptions inconsistent with the visual input. Recent work mitigates hallucinations through training-free representation editing, typically by constructing hallucination-related directions from teacher-forcing (TF) contrasts between hallucinated and truthful responses. However, LVLMs operate through autoregressive (AR) decoding during generation, raising the question of whether TF-based analysis fully reflects the generation dynamics that lead to hallucinated outputs. In this paper, we analyze the relationship between TF-based editing and AR generation behavior and find that TF-based editing alone may be insufficient to capture both decoding dynamics and multimodal interactions associated with hallucinations. To address this limitation, we propose DARE (Dual-path Auto-Regressive-aware Editing), a hybrid hallucination editing framework that integrates two complementary contrast pathways: textual contrasts and image contrasts, together with autoregressive-aware representation signals. Specifically, DARE constructs hallucination editing directions from (1) TF-based textual contrasts, (2) AR-aware representation transitions during decoding, and (3) controlled visual differences between paired images. Extensive experiments on multiple LVLM hallucination benchmarks demonstrate that DARE consistently reduces object hallucinations while preserving multimodal perception capability and inference efficiency. Our implementation code is available at https://github.com/KU-VGI/DARE.

[18] RA-CFGCache: From Branch-Level Criteria to Guided-Risk Control under Classifier-Free Guidance cs.CVPDF

Yiming Liu, Ben Wan, Tongxuan Liu, Ao Wang, Yuqi Xiong

TL;DR: RA-CFGCache将无分类器引导下两分支缓存误差的相互作用及其随时间步传播影响纳入风险控制，自适应决定同时刷新或复用。

Details

Motivation: 无训练缓存可复用扩散特征以减少计算，但局部分支标准可能忽略CFG中条件/无条件误差如何组合，也忽略不同去噪时间步的局部误差对终图影响不同。

Result: 在FLUX.1-dev、Wan2.1-T2V-1.3B和CogVideoX-2B上，相较所评估免训练缓存基线改善效率—保真权衡；兼容TeaCache、DiCache、MagCache等代理估计器，几乎不增加时延也能提高保真。

Insight: CFG感知风险组合用引导系数及离线标定的跨分支对齐度合成分支误差；传播感知重缩放再按时间步先验校正该风险，在线阈值控制器据此决定两支路一起刷新还是一起复用，同时固定采样日程和引导规则。

Abstract: Diffusion models enable high-quality visual generation, but iterative denoising remains computationally expensive, especially under classifier-free guidance (CFG), which requires both conditional and unconditional evaluations. Training-free caching reduces this cost by reuse of previously computed features or predictions. However, existing branch-local reuse criteria do not explicitly account for how cache errors combine under CFG or how local perturbations affect the final output. We identify two misalignments in cache control: a branch-guided mismatch, where guided error depends on both the magnitudes and alignment of branch errors, and a local-final mismatch, where the downstream impact of a local error varies across timesteps. We propose RA-CFGCache, a Risk-Aligned Caching framework under CFG that incorporates both factors while keeping the sampling schedule and guidance rule fixed. CFG-aware Guided-Risk Composition combines existing branch-wise proxies using CFG coefficients and offline-calibrated cross-branch alignment. Propagation-Aware Rescaling further weights the resulting guided-risk estimate with a timestep-dependent propagation prior calibrated from isolated reuse perturbations. An online threshold controller then determines when to jointly refresh or reuse both branches. Experiments on FLUX.1-dev, Wan2.1-T2V-1.3B, and CogVideoX-2B demonstrate improved efficiency–fidelity trade-offs over evaluated training-free caching baselines. Moreover, RA-CFGCache is compatible with diverse base proxy families, including TeaCache-, DiCache-, and MagCache-style estimators, and consistently improves fidelity at nearly unchanged latency. Code is available at https://github.com/yiming-l21/RA-CFGCache.git.

[19] Online Versatile Incremental Learning: Towards Class and Domain-Agnostic Adaptation at Any Time cs.CV | cs.AIPDF

Jae-Ho Lee, Min-Yeong Park, Jun-Yeong Moon, Jung Uk Kim, Gyeong-Moon Park

TL;DR: TopFlow针对类别和视觉域同时无边界变化的在线增量学习，以测地流匹配和全局拓扑保持实现持续适配。

Details

Motivation: 现有持续学习难以处理真实部署中类别概念与视觉域持续且并发变化、没有明确任务边界的情况。

Result: 在新提出的Online VIL场景中，TopFlow达到先进表现并改善既有方法局限；摘要未列具体分数。

Insight: 域无关流匹配DFM把测地流核引入对比学习，促成跨域表征；全局拓扑保持GTP则在不显式存储旧样本的条件下维护特征空间整体结构，二者应对不同类型的在线漂移。

Abstract: Continual learning enables vision systems to adapt to ever-changing data distributions. Despite significant advances, existing approaches fail to capture continuous and concurrent shifts in classes and domains, a critical capability for real-world deployment. This work introduces Online VIL (Online Versatile Incremental Learning), a novel scenario where class concepts and visual domains evolve simultaneously online without explicit boundaries. To better adapt to the challenges of such dynamic environments that more closely resemble real-world conditions, we propose a novel framework TopFlow, Topology preservation with Flow matching representation that contains two complementary mechanisms: Domain-agnostic Flow Matching (DFM) and Global Topology Preservation (GTP). DFM guides the model to have domain-agnostic representations by integrating the geodesic flow kernel into contrastive learning. In contrast, GTP maintains the global structure of the feature space without explicitly storing past examples. Our extensive experiments demonstrate that TopFlow effectively addresses the limitations of existing methods within the Online VIL scenario, achieving state-of-the-art performance in challenging Online VIL. The proposed methods suggest potential directions for building continual learning systems in realistic dynamic environments. Our implementation code is available at https://github.com/KU-VGI/Online-VIL.

[20] DynamicHOI: Coupled Dynamics for Physics-aware HOI Reconstruction cs.CVPDF

Wenliang Guo, Zhanbo Huang, Yu Kong

TL;DR: DynamicHOI用几何落地扩散细化与手—物体耦合动力学监督重建单目视频中的交互轨迹，抑制视觉合理但力学不一致的运动。

Details

Motivation: 单目HOI观测不完整，已有方法主要约束视觉与几何一致性，未充分限制交互中的受力、接触及运动关系，可能产生机械上不合理的轨迹。

Result: 在三个HOI数据集上，手和物体重建均持续改善；重建轨迹还利于手部世界模型生成和机器人操作学习，摘要未提供数值。

Insight: 几何信息为轨迹扩散细化提供空间锚点；手部关节逆动力学与Newton-Euler方程得到广义力和物体合力矩，再通过接触力传递耦合二者动力学。估计主动手部驱动并加入其经验概率分布先验，抑制不可能的运动。

Abstract: We study hand-object interaction (HOI) reconstruction from monocular RGB videos, where partial observations can produce visually plausible yet mechanically inconsistent trajectories. Existing methods mainly enforce visual and geometric agreement, leaving the underlying interaction dynamics insufficiently constrained. We propose DynamicHOI, a physics-aware HOI reconstruction framework combining geometry-grounded diffusion refinement with coupled hand-object dynamics. Geometry spatially grounds visual evidence for trajectory refinement, while articulated inverse dynamics and Newton-Euler dynamics derive hand generalized forces and object wrenches for dynamics-level supervision. We further couple hand and object dynamics through contact-force transfer and recover active hand actuation as an interaction-level physical quantity. We formulate its empirical magnitude distribution into a probabilistic prior that penalizes unlikely actuation and suppresses mechanically implausible reconstructed motion. Experiments on three HOI datasets show consistent improvements in both hand and object reconstruction. The reconstructed trajectories further benefit downstream applications including hand world-model generation and robotic manipulation learning, demonstrating the value of physics-aware HOI modeling beyond reconstruction.

[21] Reimagine Video Dynamics cs.CVPDF

Yu Yuan, Yawen Lu, Guoxian Song, Kevin Duarte, Ratheesh Kalarot

TL;DR: RVD将视频外观与紧凑、可编辑的动态token解耦，使模型能单独改写场景运动而保留视觉内容。

Details

Motivation: 多数视频编辑主要改变源视频外观，对运动节奏和事件发展控制有限。

Result: 实验显示RVD支持有效的视频动态编辑、无需训练的时间重排和外观受控重渲染；摘要未给出具体指标。

Insight: 以首帧作为视觉上下文，训练渲染器仅根据动态token重建原视频，迫使该token编码场景如何演变而非场景长什么样。再用语言引导编辑器转换源动态，借助可扩展反事实视频配对和两阶段训练监督。

Abstract: Most video editing methods focus on changing the appearance of the source video, while offering limited control over its dynamics. We introduce Reimagine Video Dynamics (RVD), a framework that disentangles a compact, editable dynamics token from visual context. We learn this token through self-supervised reconstruction: given the first frame as visual context, a renderer must recover the original video from the dynamics token, encouraging it to capture how the scene evolves rather than how it looks. This disentanglement allows video dynamics to be edited directly while preserving visual context. We develop a language-guided dynamics-token editor that transforms source dynamics into target dynamics, and train it with a scalable counterfactual video-pair pipeline and a two-stage training strategy. Extensive experiments show that RVD enables effective video dynamics editing, training-free retiming, and appearance-controlled re-rendering.

[22] Foresight at the Event Boundary: Evaluating Physical Prediction in Video World Models cs.CVPDF

Estela Monserrat Arriaga Santana, Julian Rosas Scull, Ehécatl Sacamch’en Núñez Rico, Hugo Jair Escalante

TL;DR: 本文在受控真实自由落体事件边界测试视频世界模型是否预测正确后果，并将物理预见分为产生后果、确定发生时间和实现运动三项能力。

Details

Motivation: 世界模型被期待预见物理事件结果，但现有评估多依赖参考相似度、物理规则或主观合理性，间接推测预测能力。

Result: 62段自由落体记录和124个视频显示，Runway与Veo产生释放和后续撞击事件比例超过93%，但开始时间往往明显偏晚；Cosmos-Predict-2.5与MAGI-1则常保持事件前状态。发生可测下落时，时序合理仍不保证运动物理一致。

Insight: 评测用释放或撞击刚发生、后果尚未呈现的片段做锚点，并分开测后果是否产生、时间定位和运动实现。15人、20条件实验显示人类总体更偏好真实未来，同时承认多种后续轨迹都可能合理。

Abstract: Video world models are largely regarded as predictive models of the physical world and are therefore expected to anticipate the consequences of observed events. However, evaluation has mainly focused on reference similarity, physical-law consistency, or judgment plausibility, estimating anticipation only indirectly. We address this directly: when a release or impact has just occurred but its consequence is withheld, can a world model anticipate what should happen next? We introduce an event-anchored evaluation based on 62 controlled real-world free-fall recordings and 124 clips spanning three object types, with fine-grained release and impact annotations and ground-truth trajectories. The protocol separates consequence production, temporal placement, and physical realization. Across six contemporary video generation and world models, Runway and Veo produce release and subsequent impact events at rates above 93% but often initiate them substantially late, whereas Cosmos-Predict-2.5 and MAGI-1 frequently preserve the pre-event state and produce little or no measurable consequence. Among measurable falls, plausible timing does not necessarily imply physically consistent motion. We further conduct a 15-participant, 20-condition human study in which participants describe the expected consequence from a single event-anchored frame and draw its trajectory. Human predictions favor the recorded future in aggregate while revealing genuine ambiguity among plausible continuations. Overall, physical foresight emerges as a sequence of distinct challenges: initiating a consequence, anchoring it in time, and realizing its motion.

[23] Benchmarking Vision-Language Models on Synapse Detection and Proofreading in Connectomics cs.CV | cs.AI | cs.LGPDF

Yicong Li, Junjie Wang, Leander Lauenburg, Ella Hugie, Alexandra Irger

TL;DR: 本文评测VLM在连接组学电镜图像上的突触识别与校对能力，发现少样本适配可帮助模型达到专用模型水平并在部分跨物种错误检测上超过对手。

Details

Motivation: 连接组学标注者需从电镜图像判断突触是否存在及其极性，并校对神经元分裂或合并错误；VLM在此类专业显微决策中的能力未知。

Result: 突触检测比较19种开源和2种闭源模型，校对评估另测3种开源和2种闭源模型。多数模型零样本接近随机，少量示例主要帮助闭源和最大开源模型；数千标签LoRA使开源模型达到专用模型水平。未见物种上，适配VLM在合并错误识别方面超过同数据训练的专用模型。

Insight: 基准使用公开资源构建数据，并在零样本、4-shot和LoRA设置下比较多架构、多规模模型；校对测试包含从果蝇、小鼠到人类和斑马鱼的跨物种迁移，揭示适配收益和泛化边界。

Abstract: We benchmarked vision-language models (VLMs) on the decisions annotators take when inspecting electron microscopy images in connectomics: synapse detection (presence and polarity) and proofreading (split errors and merge errors). For synapse detection, we evaluated 19 open and 2 closed models across various architectures and sizes under zero-shot, four-shot in-context learning and LoRA settings, against specialist models, on datasets constructed by us using public resources. For proofreading, we evaluated 3 open and 2 closed models on the ConnectomeBench2 dataset, with cross-species transfer from fly and mouse to human and zebrafish. Most models were at chance zero-shot; a few examples helped mainly the closed and largest open ones. LoRA on a few thousand labels brought open models level with specialist models. When evaluated on unseen species, the best adapted VLMs outperformed specialist models trained on the same data in identifying merge errors. The project will be publicly available upon acceptance.

[24] ThinkingGuard: Decoding Implicit Hazards via Step-by-Step Risk Attribution in Multimodal Large Language Models cs.CV | cs.AI | cs.LGPDF

Ruochen Zhang, Yao Huang, Yitong Sun, Jiahe Xie, Jin Yan

TL;DR: ThinkingGuard用分步风险归因训练多模态安全模型，识别单独无害的文字和视觉元素组合后触发的不安全隐式风险。

Details

Motivation: 安全关键场景中的风险可能由无害文本和中性视觉实体逻辑组合而成，单模态捷径和事后编造理由的检测器难以揭示跨模态风险触发机制。

Result: 作者构建5,600例TriggerBench，并报告ThinkingGuard在标准及隐式安全基准上表现强；摘要未列具体数字。

Insight: 数据以关键元素和触发元素构造反事实配对，消除残留风险线索，迫使模型真正做组合推理。训练框架将风险识别拆成递进认知步骤，用带步骤奖励的蒙特卡洛树搜索探索轨迹，再以双约束偏好对齐蒸馏。

Abstract: While Multimodal Large Language Models (MLLMs) are increasingly deployed in safety-critical domains, their reliability is threatened by multimodal implicit risks. Unlike explicit threats, these hazards emerge when individually benign text and neutral visual entities logically converge to induce unsafe outputs. Current detection methods fail to address this because they overlook the underlying risk activation mechanisms that govern cross-modal risk activation, leading to single-modality shortcut learning and hallucinated rationalizations. To bridge this gap, we first construct TriggerBench, the first dataset explicitly modeling risk compositionality (5,600 instances). By formally isolating Key Elements and Trigger Elements to build counterfactual contrastive pairs, TriggerBench eliminates risk residues and forces models to perform genuine logical deduction rather than superficial pattern matching, which provides a rigorous foundation for both large-scale training and fine-grained evaluation. Building on this, we propose a Step-Supervised Structured Reasoning training framework and employ it to train ThinkingGuard, a specialized guard model. Inspired by Situation Awareness theory, we decouple implicit risk identification into progressive cognitive stages, and utilize a step-reward Monte Carlo Tree Search algorithm to explore optimal reasoning trajectories, which are then distilled into the model through Dual-Constraint Preference Alignment. Extensive experiments across both standard and implicit safety benchmarks demonstrate that ThinkingGuard achieves strong performance. Project resources are available at https://github.com/FroggyChen/ThinkingGuard.

[25] AffectReveal: Event-Grounded Emotion Recognition Beyond Visual Appearances cs.CVPDF

Yihao Qian, Runhao Zeng, Sicheng Zhao, Feng Liang, Hongmin Cai

TL;DR: AffectReveal把情绪识别问题扩展为恢复决定情绪的画外事件，并通过核验身份、角色、关系和结果因素避免仅凭表情误判。

Details

Motivation: 同一可见反应可能源于不同情绪，例如流泪既可能是悲伤也可能是喜悦；仅靠图像或视频表面无法确定情绪，随意补出的合理事件也可能指错情感。

Result: EGER-Bench包含10,052段视频和10,734张图像、11类情绪及多种来源/视觉设置；加入事件语境使人类准确率从33.96%升至72.08%。AffectReveal在三个下游模型、四种输入条件下将平均UAR提高5.26至10.53分。

Insight: 先构造并独立核验事件候选及情感关键因素，再用遮脸视频中的事实对事件做双向原子证据核验，最终预测仍保留原始未遮挡输入。无需微调下游模型，且12种模型比较中均胜过其视觉单独微调版。

Abstract: Visual emotion recognition commonly assumes that all evidence required for prediction is contained in the observed image or video. Yet the same visible reaction can convey different emotions depending on events beyond the input: tears, for example, may indicate grief or joy. We formulate Event-Grounded Emotion Recognition (EGER), where emotion recognition requires recovering the affect-determining event. We construct EGER-Bench, comprising 10,052 videos and 10,734 images across 11 emotions, two source domains, and four visual settings. A study with six annotators shows that event context raises human recognition accuracy from 33.96% to 72.08%, confirming that visual evidence alone is often insufficient. Semantic relevance alone does not solve EGER: a plausible event may imply the wrong emotion if its identity, focal-person role, relationship, or outcome is misinterpreted. We therefore propose AffectReveal, a tuning-free framework that first constructs and independently verifies evidence-grounded alternatives over these affect-critical factors. It then cross-checks the recovered event against face-masked in-media facts through bidirectional atomic evidence support, while retaining the original unmasked input for final prediction. Across three downstream models and four input settings, AffectReveal yields average UAR gains of 5.26–10.53 points. For three fine-tunable models, it also enables untuned models to outperform their fine-tuned visual-only counterparts in all 12 accuracy comparisons, without updating downstream parameters.

[26] How Medical VLMs Underutilize Their Vision Encoders: A Dermatology Perspective cs.CV | cs.AIPDF

Janet Wang, Yunbei Zhang, Xiao Wang, Jihun Hamm

TL;DR: 本文从皮肤科角度分析医学VLM未充分利用视觉编码器的原因，并用无需标签的提示与少标签编码器重排提升视觉证据落地。

Details

Motivation: 皮肤科中单独的MedSigLIP视觉编码器即使在零目标标签条件下也比完整MedGemma平均高10.26个百分点，表明多模态模型未充分利用已有视觉表征，可能产生貌似合理但缺少图像依据的诊断。

Result: “先描述再判断”提示使生成时视觉注意力增加30%至40%；任务微调虽改善皮肤分类，却降低本评估中的跨域医学问答。作者在五种VLM骨干上验证冻结模型的提示与编码器辅助重排方案。

Insight: 系统检验三类关于视觉使用不足的假设，并分析内部注意力；冻结VLM，结合无标签提示及少量标签编码器辅助重排，旨在保留跨域能力的同时增强图像依据。摘要未给出最终方法的具体增益。

Abstract: Medical Vision-Language Models (VLMs) show significant promise for clinical image understanding, offering accurate diagnosis with interpretable reasoning. However, a critical performance gap exists between their strong vision encoders and the full multimodal model: in dermatology, the MedSigLIP encoder outperforms MedGemma by an average of 10.26 percentage points even when both use zero target-task labels; few-shot linear probing provides further evidence of strong visual representations. This gap motivates an investigation of how visual information is used in end-to-end diagnosis and why plausible-sounding predictions can lack grounding in image evidence. Using dermatology as our primary testbed, we systematically investigate three hypotheses for this phenomenon. We further provide a mechanistic analysis of the model’s internal attention patterns, showing that a simple describe-then-decide prompting strategy increases vision attention by 30-40% during generation. Task-specific fine-tuning improves dermatology classification but reduces cross-domain medical question-answering performance in our evaluation. To address these challenges, we combine label-free prompting with low-label encoder-assisted reranking while keeping the VLM frozen. We validate the interventions across five VLM backbones in dermatology and provide supporting representation and attention analyses across additional medical modalities.

[27] Beyond Legibility: Benchmarking Visual Text Rendering and In-Place Editing in Unified Video Generation cs.CVPDF

Ziying Zhang, Litao Li, Junchao Liao, Tianyi Zeng, Siyu Zhu

TL;DR: VidScribe系统评测视频生成中的文字书写、参考文字迁移、动态保持和局部编辑，并揭示可读内容与字形准确度并非同一能力。

Details

Motivation: 生成视频即使运动真实、画质逼真，只要文字笔画崩坏或时间不稳定就会显著损害可信度；静态OCR指标无法覆盖时间行为，现有基准也常把文字当作附带内容。

Result: 基准有803个人工核验样本和12轴条件空间，评测11种系统；I2V文字保持最好，V2V局部文字编辑最弱。参考图主要改善字形与字体保真而非文字内容准确；基准对齐偏好优化能提升生成。

Insight: 四种生成设置分别考查从语言写字、从参考迁移身份、动态中维持文字及视频局部改字；11项共享指标和两项任务专属探针在可测条件下进行跟踪级评估。退化集中在少数文字结构和时间因素，未必来自恶劣成像条件。

Abstract: A video can exhibit convincing motion and photorealism yet fail immediately when visual text collapses. Unlike generic scene content, visual text is unforgiving in video generation: minor stroke corruption, temporal instability, or editing errors instantly break legibility and realism. Existing benchmarks overlook this challenge by treating text as incidental or using static OCR metrics that ignore temporal dynamics. We introduce VidScribe, a unified diagnostic benchmark spanning four generation regimes: writing from language (T2V), transferring text identity from a reference (R2V), sustaining text under dynamics (I2V), and localized text editing (V2V). VidScribe contains 803 human-verified samples across a 12-axis conditionally orthogonal factor space covering Intrinsic Text Properties, Physical Imaging Conditions, and Temporal Behavior. For reliable evaluation, we build a track-grounded, gated suite with 11 shared metrics and 2 task-specific probes under strict measurability conditions. Benchmarking 11 commercial and open-source systems shows that video text capability is non-monolithic, with content recognition decoupled from stroke-level glyph correctness. Performance is highly task-asymmetric: I2V sustains text most reliably, whereas V2V editing is the primary bottleneck. Counter-intuitively, degradation concentrates on a small subset of text-centric structural and temporal factors rather than adverse imaging conditions. Further probes show that visual references improve glyph and typographic fidelity rather than content accuracy, while localized editing fails to isolate target text without corrupting undeclared source text. Beyond evaluation, VidScribe also provides an actionable training signal, where benchmark-aligned preference optimization measurably improves visual text generation. https://huggingface.co/datasets/Vicky0720/VidScribe.

[28] CrossTimeEdit: A Decade-Spanning Cross-View Dataset and Reward-Guided Editing for Historical Street-View Generation cs.CVPDF

Hanwen Lu, Jun He, Mingjia Yang, Hao Wei, Jinhao Huang

TL;DR: CrossTimeEdit利用同地点近期街景约束视角与未变化区域、卫星图像的时间差提供变化线索，生成历史街景。

Details

Motivation: 历史街景记录覆盖不均，恢复过去外观需重建已改变结构又保留长期不变部分，单靠稀疏历史图像难以完成。

Result: 作者构建跨11座城市、43,653个地点四元组的数据集；CrossTimeEdit经微调和在线RL后，三项编辑标准的总体表现比预训练基线提高17.12%，并超过跨视角生成模型的场景一致性、真实感和感知质量。

Insight: 自动流水线完成空间配对、一致性筛选、变化分类和卫星变化描述/局部编辑指令生成；奖励将指令对齐、背景保持和质量/物理合理性分别衡量，并使用Flow-GRPO与GDPO在聚合前归一化多奖励维度。

Abstract: Historical street-view imagery records urban evolution, but uneven coverage leaves substantial gaps in historical records. Generating plausible past appearances requires restoring changed structures while preserving persistent scene content. We construct VIGOR-his, a decade-spanning cross-view dataset containing 43,653 location-level quadruplets across 11 cities on three continents. Its automated pipeline performs spatial pairing, consistency screening, change classification, and the generation and validation of satellite-based change descriptions and local editing instructions. Based on VIGOR-his, we propose CrossTimeEdit, a model that reformulates historical street-view generation as editing, using recent street views to constrain viewpoint and unchanged appearance and temporal satellite differences as change evidence. Starting from FLUX.2 [Klein] 4B, we train CrossTimeEdit through supervised fine-tuning (SFT) followed by online reinforcement learning (RL). We design three street-view editing criteria, namely Instruction Alignment (IA), Background Preservation (BP), and Quality and Physical Plausibility (QP), as both RL reward dimensions and evaluation metrics. We optimize this multi-reward objective using Within Group Relative Policy Optimization for flow-matching models (Flow-GRPO) with Group reward-Decoupled Normalization Policy Optimization (GDPO), which normalizes each reward dimension before aggregation. CrossTimeEdit improves overall performance across the three editing criteria by 17.12% over the pretrained baseline and outperforms cross-view generation models in scene consistency, visual realism, and perceptual quality. The implementation code, dataset, and model weights are available at https://luhanwen67.github.io/CrossTimeEdit-release/.

[29] Scaling Video Generation for Reasoning: At What Cost? cs.CV | cs.AI | cs.LGPDF

Weihang Guo, Xiaoyu Wu, Yifei Wang, Niloofar Mireshghallah, Lydia E. Kavraki

TL;DR: 本文用确定性魔方视频预测基准研究扩展视频生成是否带来隐藏状态推理能力，并比较模型规模、训练算力和符号状态监督的作用。

Details

Motivation: 视频生成规模增大是否能让模型推断先前帧不可见的状态信息，以及需要多少计算成本，目前尚不明确。

Result: 70M模型在约0.1 PF-day时，预测动作后画面可见贴纸配置正确率44.6%，而1B模型仅0.3%；1B模型需3.14 PF-day后达到83.7%。20M模型加入符号状态监督后同数据预算下从31.1%升至67.3%。

Insight: 基准要求从固定可见三面预测已复原2×2×2魔方的九步操作，模拟器提供精确真值。验证MSE虽近似按幂律下降，却不能可靠预测下游状态推理，说明表征状态变化的符号监督可补充单纯规模扩展。

Abstract: We study whether scaling video generation enables models to reason about hidden information from the past frames, and at what computational cost. Our controlled benchmark requires predicting nine prescribed moves of an initially solved 2x2x2 Rubik’s Cube from a fixed view of three faces. Correct predictions require inferring how actions change hidden states, and the simulator provides exact ground truth for evaluation. Models learn plausible cube geometry early, while correct sticker configurations require substantially more training. Although validation MSE follows approximate power-law scaling, lower MSE loss does not reliably indicate downstream reasoning capabilities. Smaller autoregressive models achieve higher state accuracy with limited compute, while larger models reach higher accuracy after more training. At roughly 0.1 PF-days, the 70M-parameter model correctly predicts the visible sticker configuration in 44.6% of post-action frames, compared with 0.3% for the 1B model, which reaches 83.7% at 3.14 PF-days. Symbolic state supervision raises the 20M model’s frame accuracy from 31.1% to 67.3% at the same training-data budget, suggesting that learning representations of state changes can complement scaling.

[30] Beyond Binary Preferences: Graded Preference Optimization for Limb-Motion Captioning cs.CVPDF

Yanan Wang, Tingsong Li, Kaixun Jiang, Chongyang Zhong, Chenwei Xoe

TL;DR: GM-DPO按视频字幕中肢体动作错误严重程度设置偏好边际和训练权重，使VLM减少错误人物/肢体归因及编造动作。

Details

Motivation: VLM可生成丰富视频字幕，但在镜头切换后常认错动作执行者或涉及的肢体；训练评估需区分漏掉信息与肯定错误信息。

Result: FlexBench含3,105个镜头和18,161条查询，提供人工核验身份及逐人肢体清单。三个VLM骨干中，GM-DPO的实质动作和GPA分最高，较DPO提高2.02至3.40分；Qwen3-8B加权幻觉率降低21.3%，长文本结构也保持。

Insight: 分级物理对齐分数既奖励正确动作也惩罚错误或捏造动作；GM-DPO再对严重动作错误施加更强偏好边际和训练权重，令优化信号反映错误成本，而非将所有负偏好视为等价。

Abstract: Vision-Language Models (VLMs) can generate rich video captions, yet often misidentify which person performs an action or which limb is involved, particularly across camera cuts. Improving these details requires evaluation and training that distinguish missing information from incorrect assertions. We introduce FlexBench, a benchmark spanning 3,105 shots and 18,161 evaluation queries, with human-verified identities and systematic per-person coverage of fine-grained limb actions and states. Its reference-derived checklists support automated assessment of complete captions in their person and shot contexts. Our Graded Physical Alignment score (GPA) awards credit for correct content and deducts points for incorrect or fabricated actions, making these errors explicit in the aggregate score. Building on this rubric, we propose Graded Margin Direct Preference Optimization (GM-DPO), which assigns stronger preference margins and greater training weight to more severe action errors. Across three VLM backbones, GM-DPO achieves the highest substantive-action and GPA scores among the evaluated preference objectives, improving GPA over DPO by 2.02-3.40 points. On Qwen3-8B, it reduces the weighted hallucination rate by 21.3% relative to DPO. These gains accompany sustained long-form output, improved shot structure, and competitive performance on three additional multimodal benchmarks.

[31] VLM4Cluster: Benchmarking Deep Clustering In the Era of Vision-Language Pre-training cs.CVPDF

Yuanwei Hu, Bo Peng, Yuheng Jia, Xinting Hu, Yadan Luo

TL;DR: VLM4Cluster统一比较经典、深度和语言辅助图像聚类方法，并扩展评测到鲁棒性、分布迁移和计算效率。

Details

Motivation: 语言辅助图像聚类迅速发展，但不同论文的数据和实验设置不一致、数据集选择有限、评估维度不足，难判断实际进步。

Result: 基准含17种方法和20个数据集，涵盖经典、挑战性、细粒度、大规模及域外场景。语言辅助聚类在多种语义困难基准上推进性能前沿、通常具有更强迁移泛化和更佳效率权衡；在大规模和细粒度任务收益不稳定，也不会系统性降低对抗扰动敏感性。

Insight: 除聚类效果外，系统检查对抗鲁棒性、分布变化泛化及计算效率，揭示语言先验的优势并非普遍适用；这为公平比较不同LaIC路线提供统一测试协议。

Abstract: Vision-language pre-training has reshaped image clustering, giving rise to language-assisted image clustering (LaIC), which leverages textual semantics to complement visual representations. Despite the rapid proliferation of LaIC methods, it remains unclear how much LaIC has actually advanced image clustering, as existing studies generally suffer from major limitations, including inconsistent experimental settings, inadequate dataset selection, and limited evaluation dimensions. To address this gap, we introduce VLM4Cluster, a comprehensive benchmark for image clustering in the era of pre-trained vision-language models (VLMs). VLM4Cluster implements 17 representative methods spanning classical, deep, and language-assisted image clustering, and evaluates them on 20 datasets covering classical, challenging, fine-grained, large-scale, and out-of-distribution settings. Beyond effectiveness, VLM4Cluster systematically investigates image clustering along three complementary dimensions: robustness to adversarial perturbations, generalization under distribution shifts, and computational efficiency. Our study shows that LaIC substantially advances the clustering performance frontier on many semantically demanding benchmarks, generally exhibits stronger generalization under distribution shifts, and achieves a more favorable effectiveness-efficiency trade-off. However, its gains become less consistent on large-scale and fine-grained datasets, while language assistance does not systematically reduce sensitivity to adversarial perturbations. VLM4Cluster is released at https://github.com/YuanweiHuu/VLM4Cluster.

[32] Reprogramming Vision-Language Models via Structured Prompt Reparameterization cs.CVPDF

Zizhao Li, Chengyi Cai, Mohammed Yaqoob Ansari, Feng Liu, Joseph West

TL;DR: RVP在冻结CLIP上同时建模类别内多提示聚合和类别间残差校正，提升少样本分类并可在推理时重参数化为线性分类器。

Details

Motivation: 视觉重编程常聚合类内提示，却没有显式建模类别关系；细粒度类别属性描述高度重叠，区分线索位于文本嵌入的低方差维度。

Result: 在11个少样本分类基准和四个CLIP骨干上，RVP持续超过既有视觉重编程方法，推理效率相当或更好。

Insight: 从线性映射视角分析CLIP视觉重编程，可分离共享语义成分与类别特有差异；训练只需单个视觉提示，部署时可把结构重参数化吸收到冻结骨干后的线性分类层，几乎不增加计算。

Abstract: Visual reprogramming adapts pretrained models to downstream tasks by modifying their input and output interfaces while keeping the backbone fixed. In vision-language models, existing methods mainly rely on intra-class prompt aggregation and do not explicitly model relationships among classes. However, fine-grained categories often exhibit highly overlapping attribute descriptions and strong inter-class correlation in the text embedding space, where discriminative cues lie in subtle low-variance components. We propose Reparameterized Inter-Class Visual Reprogramming (RVP), a structured framework that aggregates multiple text prompts within each class and applies residual correction across classes. We also show that CLIP-based visual reprogramming with input-independent linear output aggregation can be expressed as a linear mapping from frozen image embeddings to downstream logits, and use this view to design a structured reparameterization that models shared semantic components and class-specific differences. RVP uses only a single visual prompt and can be reparameterized at inference into a frozen backbone followed by a linear classifier, incurring nearly zero computational overhead. Across 11 few-shot classification benchmarks and four CLIP backbones, RVP consistently improves over prior visual reprogramming methods with comparable or better inference efficiency.

[33] FocusVTC: Efficient and High-Performance Visual Text Compression with Adaptive Resolution cs.CV | cs.AIPDF

FangZhi Zhong, Xuerui Qiu, Yuqi Pan, Ya Liu, Shaowei Gu

TL;DR: FocusVTC先用低DPI全局视图压缩长文本，再根据推理证据选择性提高局部区域分辨率，兼顾token效率和细节可读性。

Details

Motivation: 将文本渲染为图像可压缩长上下文，但固定分辨率带来取舍：低DPI省token却难辨文字，高DPI又为无关区域耗费输入预算。

Result: RULER v1在72 DPI、含工具观察的输入压缩2.9倍时得分87.4，高于3倍压缩Glyph的57.5；LongBench得分56.40，超过文本骨干55.86，MRCR宏平均提高13.91分，在线端到端延迟快2.79倍。MMMU和MME等通用多模态能力也提升。

Insight: 构建2.94万条将推理链关联到页面和边界框的REL-CoT；多分辨率监督微调学习区域定位，再用GRPO学习何时提升分辨率及如何利用观察，无需单独持续预训练。

Abstract: Long-context reasoning in large language models incurs substantial computation and memory costs. Visual text compression (VTC) reduces input length by rendering text as images, but fixed-resolution rendering creates a compression-performance trade-off: low DPI saves tokens at the expense of legibility, whereas high DPI spends tokens on irrelevant content. We introduce FocusVTC, which breaks this trade-off through adaptive resolution while preserving general multimodal capabilities. It combines compressed low-DPI global views with selective region enhancement, integrating enhanced views into ongoing reasoning. We construct 29.4K high-quality Reasoning-Evidence Localization (REL) chain-of-thought examples (REL-CoT) that link reasoning traces to page indices and bounding boxes. Multi-resolution REL supervised fine-tuning (REL-SFT) teaches the model to localize relevant regions, and Group Relative Policy Optimization learns when to enhance resolution and how to use the resulting observations, without a separate continual-pretraining stage. At 72 DPI on RULER v1, FocusVTC scores 87.4 at $2.9\times$ input compression, including tool observations, versus 57.5 for Glyph at $3.0\times$ input compression. It surpasses its text-input backbone on LongBench (56.40 versus 55.86), improves the MRCR macro-average by 13.91 points, and achieves a 51.19 macro-average on VTCBench. The MRCR latency evaluation also shows a $2.79\times$ online end-to-end speedup over Text. General multimodal capabilities are preserved, with MMMU increasing from 65.12 to 66.73 and MME from 2424.02 to 2457.62.

[34] Drag as Evidence: Motion-Grounded Latent Recomposition for Drag-Based Editing cs.CVPDF

Xinyu Pu, Hongsong Wang, Jie Gui, Pan Zhou

TL;DR: MoRe-Drag把像素级拖拽变形视作粗运动证据并注入生成轨迹，逐阶段重组潜变量以兼顾拖拽精度、语义一致和自然外观。

Details

Motivation: 图像编辑模型语义修改能力强，但空间控制精度有限；现有拖拽编辑常难兼顾目标点准确移动与生成结果自然、符合用户意图。

Result: 在DragBench-SR和DragBench-DR上，MoRe-Drag比强基础编辑器明显提高拖拽精度，并在先进拖拽方法中达到最高准确度，同时保持较强语义一致性和真实感。

Insight: 根据细化、补绘和锚定区域进行潜空间重组，条件控制从运动结构形成逐步转向语义润色；MLLM文本编码器还可适配为拖拽指令推断接口，支持无需文字指令的交互。

Abstract: Modern image editors excel at semantic manipulation and visual synthesis, yet remain limited in precise spatial control, motivating the development of drag-based editing. However, existing drag-based methods often struggle to balance drag accuracy with natural, plausible, and intent-aligned generation. We propose MoRe-Drag, a motion-grounded drag-based editing method. Our key insight is to treat pixel-space warping as coarse motion evidence, and to inject this evidence into the generative sampling trajectory. Specifically, MoRe-Drag performs region-aware latent recomposition over refinement, inpainting, and anchor regions, coupled with stage-adaptive conditioning that progressively shifts from motion-grounded structure formation to semantic refinement. We further support an instruction-free interface by adapting the MLLM-based text encoder for drag-aware instruction inference. Experiments on DragBench-SR and DragBench-DR show that MoRe-Drag substantially improves drag precision over strong base editors and achieves superior drag accuracy among SOTA drag-based methods, while delivering strong semantic consistency and visually realistic results. Code and dataset will be publicly released.

[35] When Semantics Matter: Reliability-Aware Semantic-Rhythm Control for Co-Speech Gesture Generation cs.CVPDF

Zhirui Xing, Long Ye, Kaige Li, Ziyi Xu, Ming Meng

TL;DR: 该可靠性感知协同语音手势生成方法依据文本语义相对音频节律的条件信息增益，按片段动态控制语义引导强度。

Details

Motivation: 现有协同语音手势常偏重声学韵律而低用文本语义，尤其当语义标注缺失或嘈杂时，动作虽合节奏却可能表达错误内容。

Result: 多个基准实验显示，该方法在语义表达、节奏同步、动作多样性和真实声学噪声鲁棒性间取得良好平衡；摘要未给出具体分数。

Insight: 先将连续动作压缩成离散运动码，比较完整多模态分支与纯音频分支预测分布的差异，估计文本带来的条件信息增益；只在内容相关片段加强语义控制，在以节奏为主的片段减少干预。噪声条件调制和有益潜变量扰动补充声学鲁棒性。

Abstract: Co-speech gesture generation aims to synthesize natural gestures that are both temporally synchronized with speech and semantically consistent with the spoken content. Although recent methods can generate rhythmically plausible motions, they often rely heavily on acoustic prosody while underutilizing textual semantics, especially when semantic annotations are incomplete, noisy, or unavailable. Consequently, the generated gestures may follow speech rhythm while failing to express the intended semantics. To address this problem, we propose a reliability-aware semantic-rhythm control framework for co-speech gesture generation. We first learn a discrete motion prior that represents continuous gestures in a compact and structured motion-code space. We then introduce a dual-branch semantic contribution estimation mechanism consisting of a full multimodal branch and an audio-only branch. Their distributional discrepancy is formulated as conditional information gain to quantify how much textual semantics changes the predicted motion. Based on this estimate, a controllable semantic-rhythm objective selectively strengthens semantic guidance in content-relevant segments while limiting unnecessary semantic intervention in rhythm-dominant segments. Furthermore, we treat background noise as an acoustic reliability condition and introduce noise-conditioned feature modulation together with beneficial latent perturbation to improve generation robustness under realistic acoustic environments. Experiments on benchmark datasets demonstrate that the proposed framework achieves a favorable balance among semantic expressiveness, rhythmic synchronization, motion diversity, and robustness, enabling reliable and controllable co-speech gesture generation.

[36] FastVR: Efficient Streaming Video Restoration with One-Step Diffusion cs.CVPDF

Xiaoxu Chen, Qin Yang, Haoran Bai, Sibin Deng, Ying Chen

TL;DR: FastVR用单步扩散、轻量VAE及分块因果注意力实现高分辨率流式视频修复，兼顾速度和时间一致性。

Details

Motivation: 扩散视频修复受VAE编码解码成本和扩散Transformer全局自注意力二次复杂度限制，难以实际部署。

Result: 单张H20 GPU上处理1080p视频达到11 FPS；在合成与真实基准上超过所评估扩散基线并达到最先进表现，摘要未列具体质量分数。

Insight: 轻量VAE降低表示变换成本，分块因果注意力限制每次处理范围；训练时速度一致性正则和连续轨迹学习提升修复质量与跨帧稳定性。

Abstract: Diffusion-based video restoration recovers realistic details, but its practical deployment is limited by two efficiency bottlenecks: costly VAE encoding and decoding, and the quadratic cost of full self-attention in diffusion transformers (DiTs). This paper presents FastVR, a streaming video restoration framework built on a one-step diffusion model, which delivers strong restoration quality and temporal consistency while processing 1080p video at 11 FPS on a single H20 GPU. To improve inference efficiency, FastVR combines a lightweight VAE with chunk-wise causal attention, which substantially reduces the computational cost. During training, it further adopts velocity consistency regularization and continuous trajectory learning, which improve restoration quality. Extensive experiments show that FastVR is more efficient than the evaluated diffusion baselines while achieving state-of-the-art performance on synthetic and real-world benchmarks. We hope that this work supports further progress in the community.

[37] DSPO: Diversity-aware Subjective Policy Optimization for Robust Emotional Reasoning cs.CVPDF

Cheng Ye, Weidong Chen, Bingyan Xu, Zhendong Mao

TL;DR: DSPO以VAD情感分布先验奖励群体解释覆盖，并通过反事实视觉遮挡降低缺乏图像证据的情绪判断权重。

Details

Motivation: 情绪具有主观且连续分布特征，依赖确定硬标签和孤立点评的RL难以覆盖合理解释；情绪线索隐含在图像中，也易导致看似合理但无视觉依据的幻觉。

Result: 在多个公开基准上，DSPO达到最先进表现，跨域平均准确率较EMO-R3提高10.8%。

Insight: 把标注情绪词汇先验与图像语境结合成VAD空间分布，用留一法边际贡献奖励有助于贴近该先验的候选解释，避免无约束发散；再遮挡推理指出的视觉区域，按候选概率变化降低不受视觉支持解释的权重。

Abstract: Reinforcement Learning has significantly advanced the complex reasoning capabilities of MLLMs. However, prevailing RL algorithms suffer a severe failure in emotion reasoning tasks. These methods heavily rely on deterministic hard-label supervision and point-wise isolated evaluation, creating a fundamental gap with the inherently subjective and continuously distributed nature of human emotions. Furthermore, unlike explicit physical objects, emotional states are deeply implicit within visual cues. This abstract nature exacerbates visual hallucinations in MLLMs, leading to plausible yet ungrounded emotional evidence. To address these limitations, we propose Diversity-Aware Subjective Policy Optimization (DSPO), a reinforcement learning framework that jointly promotes subjective affective coverage and visual grounding. First, we construct a context-grounded emotional distribution prior in the VAD space by combining the lexical prior of the annotated emotion with image-specific contextual information. Based on this prior, we introduce a Distribution-Aligned Emotional Diversity Reward (DEDR), which measures the leave-one-out marginal contribution of each candidate emotion within a rollout. DEDR rewards candidates whose inclusion brings the predicted affective set closer to the context-grounded prior, thereby preserving plausible subjective interpretations without encouraging unconstrained dispersion. We further develop Counterfactual Visual Intervention Gating (CVIG), which masks the visual region highlighted in the reasoning process and uses the resulting candidate-wise probability changes to reduce the weights of interpretations unsupported by visual evidence. Extensive experiments demonstrate that DSPO achieves state-of-the-art performance across multiple public benchmarks, especially on the cross-domain performance, i.e., improving +10.8% on average cross-domain accuracy than EMO-R3.

[38] Causal-EVC: Breaking Emotional Spurious Causality via Spatiotemporal Grounding and Counterfactual Intervention cs.CVPDF

Cheng Ye, Weidong Chen, Peipei Song, Zhendong Mao

TL;DR: Causal-EVC用运动引导的时空定位及反事实对比学习，把情绪视频字幕的依据从共现背景转向真正触发情绪的事件。

Details

Motivation: 情绪视频字幕可能把阳光海滩等背景共现线索错当情绪原因；简单注意力匹配存在因果冗余和捷径学习，现有评测也难辨真因果理解与背景混淆。

Result: 作者构建带稠密时空因果标注的EVC-CauseGround及因果忠实子集；Causal-EVC在语义指标和因果忠实评估上均有优势，摘要未提供具体数值。

Insight: 运动引导的因果定位模块区分触发事件和背景混杂因素，稀疏情绪路由模块再用反事实表示与对比目标，迫使预测依赖真实原因而非表面共现。

Abstract: Emotional Video Captioning aims to generate factually accurate and emotionally empathetic descriptions. While recent methods have recognized the importance of visual causes to guide emotion perception and caption generation, they fundamentally rely on simple attention matching, which inevitably suffers from {causal redundancy and spurious correlations} in co-occurrence bias (e.g., misclassifying sadness'' as joy’’ on a sunny beach), leading to severe shortcut learning from confusing backgrounds. Furthermore, existing evaluations fail to verify whether models have genuinely mastered causal reasoning or merely exploited background confounders. To address these limitations, we first construct {EVC-CauseGround}, a comprehensive benchmark with dense spatio-temporal causal annotations. Crucially, it introduces a carefully selected {Causal-Faithfulness Subset} to explicitly quantify genuine emotion-cause attribution. Second, we propose {Causal-EVC}, an emotion-grounding captioning framework, which introduces a Motion-guided Causal Spatiotemporal Localization module to precisely decouple causal triggers from background confounders. Besides, we introduce an Interpretable Sparse Emotion Routing module. By synthesizing counterfactual representations and formulating a novel counterfactual contrastive objective, we enforce the model to anchor its emotion predictions strictly on authentic causal triggers instead of confusing background. Extensive experiments show that Causal-EVC not only achieves the best performance on semantic metrics but also exhibits significant advantages in the causal-faithfulness subset, which demonstrates that our model could mine emotional cues from genuine visual causes and mitigate co-occurrence bias for interpretable multimodal emotion understanding.

[39] Dual-Mode Low-Rank Learner with Bridge-Prototype Ensemble for Vision-Language Class-Incremental Learning cs.CV | cs.AIPDF

Chiyuan He, Zihuan Qiu, Fanman Meng, Chao Wang, Liangjiang Chen

TL;DR: DuLBE以双模式低秩更新协调跨任务共享知识和任务专属变化，再通过视觉原型与文本嵌入间的桥原型集成改进无样本增量分类。

Details

Motivation: CLIP类增量学习若反复更新共享组件会覆盖旧知识，若完全隔离新任务又无法复用迁移知识；文本或双模态分类器也未充分融合两种模态互补信息。

Result: 多种类增量学习设置中，DuLBE达到最先进表现，同时维持低秩微调的参数效率。

Insight: 根据梯度需求分配紧凑可重写共享模式和低干扰残差模式，通过梯度路由兼顾知识复用及新任务可塑性；在单位超球面视觉原型和文本嵌入间构造测地桥，并集成可靠桥原型，弥补文本决策边界受模态间隔影响。

Abstract: Benefiting from transferable visual-textual alignment, CLIP has been widely adopted for class-incremental learning (CIL). However, existing learners either repeatedly update components shared across tasks, leading to knowledge overwriting, or overly isolate new-task updates, hindering the reuse of CLIP’s transferable knowledge and limiting plasticity. Moreover, the text-based or bimodal classifier designs still fail to effectively integrate complementary information from the visual and textual modalities. To address these challenges, we introduce DuLBE, which couples dual-mode low-rank learning with a bridge-prototype ensemble classifier for exemplar-free CIL. DuLBE allocates two visual low-rank update modes according to the gradient demand and uses gradient routing to coordinate them: a compact and rewritable shared mode is selected from historically occupied visual directions to reuse transferable knowledge, while residual modes provide low-interference channels for task-specific variations. Building on the resulting stable inter-modal structure, we further construct geodesic bridges between visual prototypes and text embeddings on the unit hypersphere, and ensemble reliable bridge prototypes to compensate for the modality-gap limitations of textual decision boundaries. Extensive experiments under multiple settings show that DuLBE achieves state-of-the-art CIL performance while retaining the high parameter efficiency of low-rank tuning.

[40] Decoding Affective Nuances: Enhancing MLLMs via Hierarchical Emotion Reasoning and Contrastive Discriminative Pruning cs.CVPDF

Cheng Ye, Weidong Chen, Zhaobo Qi, Beier Zhu, Zhendong Mao

TL;DR: DAN通过层级情绪推理保留场景和对象细节，再对相近情绪的注意力分布做对比剪枝，提升细粒度情感分类。

Details

Motivation: MLLM整体推理会稀释微妙情绪线索；能找到泛化情绪区域的方法也未必能区分语义相近情绪各自依赖的判别区域。

Result: 无需训练资源，DAN在多个基准上提升细微情绪区分能力；Qwen3-VL-8B-Instruct在含25种细粒度情绪的WebEmo25上提升10.47%。

Insight: HERC用软门控协调场景级与对象级线索，改善情绪归因；CDVP比较相似情绪的注意力分布绝对差异，隔离真正区分类别的视觉token，再据此判断最终情绪。

Abstract: While multimodal large language models (MLLMs) have demonstrated exceptional capabilities in objective understanding tasks, their performance in affective reasoning still falls significantly short of human standards. We attribute it to a central capability gap: MLLMs are difficult to reliably distinguish semantically proximal emotions based on fine-grained visual evidence, which could be decoupled as two limitations: 1) Insufficient Attribution. The global reasoning paradigm of conventional MLLMs severely dilutes fine-grained emotion cues, where subtle emotional states are usually implicitly encoded, thereby generating emotional misjudgments in complex scenarios. 2) Insufficient Discrimination. Existing methods could only identify regions generally associated with emotions, which fails to distinguish discriminative regions between semantically similar emotions, leading to ambiguous emotion judgements. To overcome these limitations, we present a training-free inference-time optimization framework, named Decoding Affective Nuances (DAN). Specifically, we propose a Hierarchical Emotional Reasoning Chain (HERC) that enhances the insufficient attribution by harmonizing fine-grained scene/object-level cues and performing a soft-gated reasoning. Furthermore, to discriminate between semantically proximal emotions, we design a Contrastive Discriminative Visual Pruning (CDVP), which isolates discriminative visual tokens to reason the final emotion category by computing the absolute discrepancy between the attention distributions of similar emotions. Performances on several benchmarks demonstrate that DAN significantly improves discrimination for affective nuances without consuming additional training resources, especially achieving +10.47% improvements with Qwen3-VL-8B-Instruct on WebEmo25 dataset that contains 25 fine-grained emotion categories.

Yueran Ma, Ronghao Lin

TL;DR: DMC-Repair以跨样本交换音频和图像的反事实训练，减少全模态模型回答音频问题时错误利用同片段视觉捷径。

Details

Motivation: 全模态模型应按问题指明的模态作答，但训练数据中同一样本各模态往往提供冗余答案，无法检验模型是否真的依赖指定信息。

Result: 在两种模型系列及多种设置中，跨模态捷径贯穿监督微调和强化学习，基于裁判的RL还可能加强无关视觉依赖。DMC-Repair将图像诱发的答案效应占比降低59.9%，不损害音频问答，并泛化至未见数据集/基准及后续训练。

Insight: 因子化模态诊断独立交换不同样本音频和图像，隔离各模态的因果贡献；修复阶段用相同交换样本，但只按问题指定模态提供监督，切断同片段模态间的伪相关。

Abstract: Omni-modal large language models (LLMs) are expected to answer a question using the modality it explicitly refers to. However, existing training paradigms rarely verify whether models actually follow this modality, because multimodal inputs from the same sample often provide redundant evidence for the same answer. In this work, we uncover a pervasive cross-modal shortcut in omni-modal LLMs: when asked an audio-related question, models rely on the image as much as on the audio, and sometimes even more. To systematically diagnose this behavior, we introduce the Factorized Modality Diagnostic, which independently swaps audio and images between samples to isolate each modality’s causal contribution. Across two model families in different settings, we find that this shortcut persists throughout supervised fine-tuning and reinforcement learning post-training, while judge-based RL may further amplify such reliance on irrelevant visual information. Based on this finding, we propose DMC-Repair, which trains models on the same kind of cross-modal swapped samples while assigning supervision according to the modality specified by the question. This prevents models from exploiting the spurious correspondence between modalities within the same clip. Experiments demonstrate that DMC-Repair reduces the image-induced share of the answer effect by 59.9%, effectively suppressing the cross-modal shortcut without compromising audio-question answering performance. The reduction in shortcut reliance generalizes across two model families and zero-shot to an unseen dataset and an unseen benchmark, and persists through subsequent post-training. Code is available at https://anonymous.4open.science/r/DMC-Repair.

[42] EGSD: Event-Grounded Self-Distillation for Streaming Video Understanding cs.CVPDF

Yuwei Miao, Xuesheng Zhang, Wenhao Zou, Jixia Zhang, Jianwei Lv

TL;DR: EGSD把在线视频记忆视为可验证事件的增量更新，通过事件加权教师信号和实体覆盖奖励修复流式VLM记忆的两个蒸馏失配。

Details

Motivation: 直接在线策略自蒸馏中，学生须在问题到达前写入流式记忆，但教师却能利用问题和真值答案挑选相关记忆，偏好错配；只关注有效实体还会使覆盖不足，token均值也不反映记忆包含多少实体。

Result: 在主流在线和离线基准上，EGSD取得强表现：StreamingBench达79.8%，OVO-Bench实时轨道73.4%；有效实体召回提高17.4%，记忆长度仅增加6.8%。

Insight: 将流式记忆表示为关键实体、动作和细节组成的可验证事件增量；把自蒸馏信号转为与结果奖励相乘的权重，减少教师问答特权和学生写记忆目标冲突；以事件为特权信息重加权教师，并补充实体覆盖奖励。

Abstract: Real-time video understanding requires incrementally maintaining a memory of streaming content, and optimizing this requires dense process signals. On-Policy Self-Distillation (OPSD), which lets one model serve as both teacher and student with the teacher receiving additional privileged information such as the question and ground-truth (GT) answer, can supply such token-level signals. However, applying it directly to streaming video raises two problems. (1) The student cannot be optimized end-to-end, where memory is written before the question arrives, yet the teacher scores it with the question-and-GT privilege, misaligning their preferences. (2) Effective-entity memory collapses, where the question-and-GT privilege makes the teacher favor only question-relevant entities, and token-mean averaging over a memory renders its signal invariant to how many entities that memory covers, both driving memory against the streaming need for diversity. To address these issues, we propose Event-Grounded Self-Distillation (EGSD), which characterizes streaming memory as an incremental update over verifiable Events (key visual entities, actions, and details) and targets the two problems on this basis. For problem (1), we adapt the OPSD signal into a multiplicative weight combined with the outcome reward; for problem (2), we re-weight the teacher with Events as privileged information to counter its question-relevance bias, and add an entity-coverage reward to supply the coverage preference the token-mean teacher lacks. Extensive experiments on mainstream online and offline benchmarks show EGSD achieves strong performance, reaching 79.8% on StreamingBench and 73.4% on the OVO-Bench Real-Time track, while memory analysis shows effective-entity recall rises 17.4% at only 6.8% more memory length.

[43] CurvSpec: Adaptive Multi-Curvature Learning for Partial Relevant Video Retrieval cs.CVPDF

Zhen Liu, Letian Li, Jinpeng Wang, Shuzhao Xie, Yuzhi Huang

TL;DR: CurvSpec根据视频内容自适应选择检索几何，并以复杂度决定数量的语义中心减少部分相关视频中的短片段信号稀释。

Details

Motivation: 部分相关视频检索需从数分钟视频中找到只有数秒的相关片段，整体表示容易被无关背景淹没；不同视频的结构又可能从平坦原子事件到深层组合层级，固定曲率嵌入难以适配。

Result: 在ActivityNet Captions、TVR和Charades-STA上取得最先进检索表现；摘要未提供具体指标。

Insight: 并行欧氏与双曲注意力使用独立学习曲率，再按内容融合到适合的几何空间；每个视频构建与内容复杂度匹配数量的语义中心，查询通过流形测地距离匹配最近中心。

Abstract: Partially Relevant Video Retrieval (PRVR) seeks to retrieve untrim-med videos containing a moment that matches a text query, without temporal annotations. The relevant moment may last only seconds within a video spanning several minutes, creating an extremely low signal-to-noise ratio that makes PRVR more challenging than standard full-video retrieval. This task presents two intertwined challenges: (1) signal dilution, where coarse global representations blur the brief relevant signal into the dominant irrelevant surroundings;(2) curvature rigidity, where embedding all videos in the same fixed-geometry space distorts representations for videos that range from flat atomic events to deep compositional hierarchies. Existing PRVR methods have improved moment selection and cross-modal matching, but they still typically encode all videos in a single fixed-curvature retrieval space, limiting their ability to model diverse video structures. To address both challenges, we propose CurvSpec, a framework that learns content-adaptive curvature for video retrieval representations rather than imposing a fixed geometric prior. CurvSpec processes features through parallel Euclidean and hyperbolic attention layers, with independently learned curvatures assigned to the hyperbolic layers, and a content-aware fusion mechanism routes each input to its most suitable geometric regime. To further suppress signal dilution, CurvSpec represents each video with semantic centroids whose number is determined by the video’s content complexity, projects them onto the learned manifold, and matches each query against its nearest centroid by geodesic distance. Experiments on ActivityNet Captions, TVR, and Charades-STA demonstrate state-of-the-art retrieval performance.

[44] MeteoVerse: Unified Weather-Controllable Video World Model cs.CVPDF

Renlong Wu, Guanqiao Wang, Xuan Shang, Yin Hanming, Xiaoxiao Sheng

TL;DR: MeteoVerse显式预测观测天气到目标天气的转变，再以天气专家残差控制未来视频中的保留、添加或去除天气效果。

Details

Motivation: 世界模型需同时处理相机运动、物体动态和天气变化；只给目标天气会令生成骨干隐式推断天气如何转变，控制不准确。

Result: 基于超过5万段真实天气视频及晴天对应视频构建数据集。实验显示MeteoVerse显著改善天气可控性，同时保持有竞争力的场景一致性和相机控制表现。

Insight: 天气转变取决于源状态和目标状态的组合，模型因此先估计两者并显式表示差异，再以转变感知专家生成类别特定残差，还能细调新引入天气的强度。

Abstract: Video world models aim to predict future content from an observed scene while following prescribed camera motion. Real-world scene evolution is determined not only by changes in viewpoint and object dynamics, but also by environmental conditions such as weather, which can substantially alter scene appearance and visibility. Modeling such realistic weather evolution is challenging because the required weather modification depends jointly on the observed and desired weather states. Depending on their relation, the model may need to preserve, introduce, or remove a weather effect. Existing video world models typically leave this weather transition implicit, forcing the generation backbone to infer weather evolution together with scene dynamics and camera motion, which leads to imprecise weather control. To address this limitation, we propose MeteoVerse, a unified weather-controllable video world model that generates future videos from a single sunny or adverse-weather image, conditioned on a weather-free scene description, a target-weather instruction, and a camera trajectory. Rather than conditioning only on the desired weather, MeteoVerse explicitly estimates the observed and target weather states and represents the required weather transition. A transition-aware mixture of weather experts then translates this transition into category-specific residual weather features, unifying weather preservation, introduction, and removal while enabling fine-grained control over introduced weather intensity. We further construct the MeteoVerse dataset with over 50K real-world weather video clips, generated sunny counterparts, disentangled scene and weather descriptions, weather-intensity annotations, and camera trajectories. Extensive experiments demonstrate substantially improved weather controllability while retaining competitive scene consistency and camera-control performance.

[45] Learning via Self-Consistency for Diffusion-based Video Reasoning cs.CVPDF

Zhenghao Ni, Weimin Qiu, Meng Tang

TL;DR: 本文对扩散视频推理进行自一致性测试时扩展，并把多样本共识蒸馏进生成模型，使推理时恢复单次生成成本。

Details

Motivation: 视频扩散生成具有随机性，而许多下游视觉任务的答案是确定的；多次生成再共识可提升鲁棒性，却会增加推理开销。

Result: 在迷宫、视觉搜索和指代分割任务中，自一致性推理和蒸馏均显著改善结果；视觉搜索准确率从单次生成48.4%升至99.0%。4×4迷宫中，共识训练把单次生成严格成功率从72.0%提高至84.0%，推理延迟不变。

Insight: 测试时从多个视频rollout提取路径、位置或掩码并聚合共识；在去噪早期读取答案，保留共识质量且减少一半以上去噪步。再用拒绝微调将共识结果蒸馏到模型，推理仅需单次生成，不用真值视频或任务验证器。

Abstract: Video generation models have demonstrated emerging zero-shot capabilities for visual reasoning, perception, and other vision tasks. However, diffusion-based video generation is inherently stochastic, while many downstream vision tasks are deterministic. Motivated by the effectiveness of self-consistency in chain-of-thought reasoning for large language models, we investigate whether self-consistency can similarly improve diffusion-based video reasoning. We first introduce a training-free test-time scaling method that samples multiple video generations and aggregates their predictions through self-consistency. Specifically, we aggregate extracted paths, locations, or masks from multiple rollouts into a consensus prediction. To reduce the inference overhead of multi-rollout generation, we read out predictions early in the denoising trajectory, which preserves consensus quality while reducing denoising steps by more than half. We further propose Rejection Fine-Tuning (RFT) to distill consensus predictions into the video generation model. The resulting model internalizes the benefit of multi-sample consensus and requires only a single generation at inference time, while substantially outperforming the original model. Experiments on three tasks, including maze solving, visual search, and referring segmentation, show that both our self-consistency inference and consensus distillation dramatically improve video-based perception and reasoning, without requiring ground-truth videos or task-specific verification. For visual search, self-consistency raises task accuracy from 48.4% for a single generation to 99.0%. The distilled model retains much of the consensus benefit with a single rollout. For 4-by-4 maze solving, consensus-based training improves the single-generation strict success rate from 72.0% to 84.0% with the same inference latency.

[46] Motion Concept Unlearning in Video Diffusion Models cs.CVPDF

Ping Liu, Chi Zhang

TL;DR: MUTE以去噪过程中的token中和识别运动概念方向，用空间门控局部移除特定视频动态，无需训练或修改模型权重。

Details

Motivation: 文本到视频模型能生成踢打、刺击、射击等动作，带来安全风险；静态概念擦除已广泛研究，视频运动概念的选择性遗忘仍不足。

Result: 在20种运动概念及Wan2.1-T2V上，MUTE超过提示级、权重级和推理时基线；同一公式可迁移到CogVideoX，摘要未提供具体擦除/保留指标。

Insight: 因果干预表明文本条件注意力携带可选择性操纵的运动信息，而改动时间位置编码会同时压制目标和非目标动态。MUTE在每步去噪中先中和概念token得到方向，再用其内在结构形成空间门控，在CFG之前从速度输出减去修正，满足概念特异、空间选择与时间自然三项要求。

Abstract: Text-to-video (T2V) diffusion models can generate realistic depictions of actions such as kicking, stabbing, and shooting, raising safety concerns that motivate targeted concept erasure. Although concept erasure has been extensively studied for static concepts in text-to-image and T2V models, erasing motion concepts remains largely unexplored. We present a systematic study of motion concept erasure in video Diffusion Transformers (DiTs). Through causal interventions, we show that text-conditioning attention carries concept-specific motion information and supports selective intervention, whereas perturbing temporal positional encoding suppresses both target and non-target dynamics. We further find that directly adapting ESD, a representative weight-level image erasure method, to a video DiT yields modest and uneven motion suppression: reducing its erasure training loss does not by itself remove the concept signal from the difference between the conditional and unconditional predictions, which classifier-free guidance (CFG) then scales at every denoising step. From these findings, we derive three requirements for motion concept erasure: concept specificity, spatial selectivity, and temporal naturalness. Each determines one component of MUTE (Motion concept Unlearning in Text-to-video gEneration): at each denoising step, MUTE extracts a concept direction through token neutralization, derives a spatial gate from the direction’s intrinsic structure, and subtracts the resulting correction from the velocity output before CFG is applied. MUTE is training-free and requires no weight modification. Experiments on 20 motion concepts show that MUTE outperforms representative prompt-level, weight-level, and inference-time baselines on Wan2.1-T2V, and the same formulation transfers to CogVideoX, supporting its applicability across distinct T2V attention architectures.

[47] On-Policy Visual Evidence Distillation cs.CV | cs.AI | cs.CLPDF

Shaohang Wei, Feifan Song, Guangyue Peng, Wenhao Yu, Wei Li

TL;DR: ReVuE比较同一问题下多个学生视觉轨迹，诊断证据获取、阅读或答案落地的首个失败点，再据反思对教师蒸馏信号做分组加权。

Details

Motivation: 视觉代理交错执行推理和图像操作，获取或解读证据中的局部错误会传播至后续回答；既有在线蒸馏增强原图视角监督，但没有显式关联学生动作、后续观察和推理。

Result: 在Qwen2.5-VL和InternVL3.5系列的11个基准上，ReVuE感知、数学推理和通用任务加权平均分均超过所有所评在线蒸馏基线，并减少推理冗余和工具调用、提升工具调用准确率与任务准确率。

Insight: 对多条学生轨迹总结观察证据并定位Acquire/Read/Ground首个失误，将诊断作为训练期教师上下文；再依据反思对教师预测的影响程度分组重加权token蒸馏损失，使轨迹级证据诊断转成针对阶段的token监督。

Abstract: Visual agents solve problems by interleaving reasoning with image operations, and on-policy distillation (OPD) provides guidance from a strong teacher on student-generated interaction trajectories. However, image operations change the evidence available for subsequent reasoning, so local errors in evidence acquisition (Acquire), reading (Read), or answer grounding (Ground) can propagate through the trajectory and lead to incorrect answers. Existing multimodal OPD methods primarily construct or contrast auxiliary views of the original image to strengthen supervision, without explicitly modeling the connections between student actions, resulting observations, and subsequent reasoning. This limits their ability to provide corrections tailored to different failure stages. We introduce Reflection on Visual Evidence (ReVuE), an on-policy distillation method for visual agents. ReVuE compares multiple student-generated trajectories for the same query, summarizes the observed visual evidence, and diagnoses the first failure across the Acquire, Read, and Ground stages. The resulting reflections provide training-time context for the teacher. We group and reweight token-level distillation losses according to how strongly these reflections affect the teacher’s predictions. This design translates trajectory-level evidence diagnosis into targeted token-level supervision, guiding students to improve their visual evidence acquisition and reasoning. Across 11 benchmarks spanning the Qwen2.5-VL and InternVL3.5 model families, ReVuE outperforms all evaluated OPD baselines in weighted-average scores for perception, mathematical reasoning, and general tasks. ReVuE also reduces redundancy in reasoning and tool calls while improving tool-call accuracy and task accuracy. Code is available at https://github.com/sylvain-wei/ReVuE

[48] GlassFormer: Learning Real-time Glass Segmentation using Radar-Depth Fusion cs.CV | cs.ROPDF

Suhani Grover, Astik Srivastava, Viswas Dinesh, Avinash Sharma, K. Madhava Krishna

TL;DR: GlassFormer融合毫米波雷达与RGB-D感知透明玻璃表面，利用视觉/深度失效处雷达反射提供几何先验。

Details

Motivation: 相机透过玻璃看到其后的背景，激光雷达、ToF和RGB-D也常在透明区返回无效或背景深度，容易把玻璃墙门误判为空地；反光、弱光和无纹理玻璃又使纯视觉线索退化。

Result: 混合条件测试mIoU为0.88，专门低照度集为0.59，在资源受限平台保持实时运行，并相较视觉基线更鲁棒。

Insight: 跨模态不一致可产生雷达引导空间先验，再通过跨模态注意力注入轻量Transformer分割网络。毫米波在玻璃处强反射，恰好补足视觉和深度无法可靠观测的区域。

Abstract: Transparent surfaces are ubiquitous in built environments, yet they remain a persistent failure case for robotic perception. RGB cameras perceive the background behind glass rather than the surface itself, while depth sensors such as LiDAR, time-of-flight, and RGB-D often return invalid or background measurements in transparent regions. As a result, systems that rely solely on optical sensing may misinterpret glass walls, doors, or mirrors as free space, compromising safe and reliable navigation. Existing glass segmentation approaches address this by learning visual cues such as reflections, boundaries, and semantic context from RGB images. While effective under favourable lighting and viewing conditions, these cues degrade in low-light environments, under glare, or when glass surfaces are featureless or partially occluded. In this work, we propose a multimodal framework that fuses millimetre-wave radar with RGB-D sensing for real-time transparent surface segmentation. Radar reflects strongly off glass surfaces, providing a geometric cue that remains reliable precisely where vision and depth fail. We exploit this cross-modal inconsistency to generate a radar-guided spatial prior, which is integrated into a lightweight transformer-based segmentation network, GlassFormer, via cross-modal attention. We report results on a mixed-condition test split covering all scene types and a dedicated low-light split designed to stress vision-only methods. GlassFormer achieves 0.88 mIoU on the mixed split, and 0.59 mIoU on the low light split, demonstrating substantial robustness gains over vision-only baselines while maintaining real-time performance on resource-constrained platforms.

[49] RoXDrive: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving via Action-Faithful Rollouts cs.CVPDF

Hongbin Lin, Chaoda Zheng, Yiming Yang, Xiangyu Li, Shijia Chen

TL;DR: RoXDrive筛选动作与视频未来一致的世界模型rollout，再用安全密集评分执行闭环RL后训练自动驾驶策略。

Details

Motivation: 模仿学习只看专家日志、不观察自身动作后果，部署时易有因果混淆。合成模拟器便于闭环训练却有虚实差距；视频世界模型较真实但可能没有忠实执行动作条件，造成动作—视觉错配。

Result: 在nuScenes及含13万多个训练场景的内部数据上均持续提升；DiffusionDrive在nuScenes安全违规减少27.6%，Qwen3-VL内部数据减少33.7%。

Insight: 预训练阶段除模仿学习外训练动作—视觉忠实度评估器，通过带几何辅助轨迹监督的逆动力学估计判断长程视觉动态是否符合自车动作；强化学习仅保留动作忠实rollout，再以安全感知密集分数优化策略。

Abstract: End-to-end autonomous driving policies are commonly trained via imitation learning on logged demonstrations without observing the consequences of their own actions, leading to causal confusion in closed-loop real-world deployment. To address this issue, reinforcement learning (RL) post-training offers a promising alternative by leveraging world models as interactive training environments to enable future scene generation for policy improvement. Nevertheless, existing approaches either rely on reconstruction-based simulators, offering limited counterfactual interaction, or adopt synthetic simulators to enable long-horizon closed-loop interaction at the cost of a substantial sim-to-real gap. Recently, video world models have exhibited the ability to generate realistic multi-step future rollouts but may not faithfully reflect action conditions, resulting in action-vision mismatch. In this paper, we introduce RoXDrive, a plug-and-play closed-loop RL framework that enables reliable policy optimization by identifying action-faithful world-model rollouts, consisting of two stages: 1) Model pre-training: In addition to imitation-based policy pre-training, we devise an Action-Vision Faithfulness Evaluator for inverse dynamics estimation with our geometry-aware auxiliary trajectory supervision, enabling long-horizon assessment of whether visual dynamics faithfully reflect the conditioning ego actions. 2) Action-faithful RL post-training: Agents iteratively interact with world models to form long-horizon scene rollouts, retaining only action-faithful ones for dense safety-aware scoring and scene-level closed-loop RL post-training. Extensive experiments on nuScenes and an in-house dataset with over 130K training scenarios demonstrate consistent gains across planners, reducing safety violations by 27.6% with DiffusionDrive on nuScenes and 33.7% with Qwen3-VL on the internal data.

[50] S4VY: Segment Anything in Feed-Forward 4D Visual Geometry cs.CVPDF

Jingdong Zhang, Xin Li, Jan Kautz, Wenping Wang, Chris Choy

TL;DR: S4VY基于前馈4D视觉几何为多帧观察中的持久对象生成完整类别无关实例掩码，并支持点框条件及语言引导选择。

Details

Motivation: 常见Segment Anything主要处理2D图像或视频掩码，并用顺序记忆维持身份；基于前馈视觉几何的可提示4D实例分割尚未充分探索。

Result: 在统一静态和动态场景评测中，S4VY达到先进4D实例分割表现并具有较强语言引导定位能力；摘要未提供具体分数。

Insight: 空间—时间查询解码器把共享视觉几何特征转为覆盖所有观察的实例掩码，每个持久查询跨帧绑定同一实体，无需种子掩码或固定时间顺序。语言代理框架用主动树搜索选相关帧，双流grounder结合VLM细节与几何一致实例特征，独立评判器选最终掩码。

Abstract: Accurate instance segmentation in dynamic scenes is important for downstream applications such as robotics and autonomous driving. Existing Segment Anything models operate primarily on 2D image or video masks and preserve identity through sequential memory, while promptable 4D instance segmentation built upon feed-forward visual geometry remains underexplored. We introduce S4VY, a Segment Anything model built on feed-forward 4D visual geometry. From a set of RGB observations, S4VY transforms shared visual-geometric features into an exhaustive set of class-agnostic 4D instance masks through a space-time query decoder, with each persistent object query binding one entity across all observations. This representation supports prompt-independent segmentation as well as point- and box- conditioned selection, without requiring a seed mask or temporal ordering. We further develop an agentic harness for natural-language grounding in the large observation space of a 4D scene. Active tree search identifies relevant frames without scanning every fixed window; a dual-stream grounder combines fine-grained VLM visual priors with geometry-consistent instance features through complementary bounding-box prediction and object-query matching; and an independent critic selects the final 4D instance mask from their predictions. Extensive experiments demonstrate state-of-the-art 4D instance segmentation and strong language-guided grounding performance under a unified evaluation spanning static and dynamic scenes.

[51] ProGuT: Label-Efficient Panoptic Segmentation for Forest Scenes cs.CVPDF

Pankaj Deoli, Karsten Berns

TL;DR: ProGuT用CLIP图像patch聚类和多尺度几何先验生成森林全景分割伪标签，无需逐图实例掩码训练。

Details

Motivation: 森林全景分割的瓶颈主要在实例区分而非语义质量；无监督方法可得到可用背景类别图，却几乎不能识别树干实例，深度/光流方案还依赖并非总可用的传感器。

Result: Our-forest数据集PQ达65.2，为初始伪标签质量的2.6倍；Freiburg Forest mIoU为65.9，高于PiCIE的45.3和STEGO的57.6，也超过已有无监督树干实例方法。

Insight: 聚类CLIP patch特征后用多尺度几何先验和结构张量排除非树干结构，恢复树干实例；仅需未标注图像和一次簇到类别映射，生成低成本伪标签供下游训练。

Abstract: Panoptic segmentation in forest environments is bottlenecked not by semantic quality but by instance separation; existing unsupervised panoptic approaches produce usable stuff maps but near-zero thing quality. Depth or flow-based instance discovery methods needs sensors that are not always available. We present ProGuT (Prototype Guided Training), which produces panoptic pseudo-labels without per-image training masks, needing only unlabeled images and one-time cluster-to-class mapping. ProGuT clusters CLIP patch features, then recovers trunk instances through multiscale geometric prior that falsifies non-trunk structures via structure-tensor. This is cheap compared to depth, flow or class-supervision methods to create pseudo labels. These are then used for downstream tasks which we evaluate against other unsupervised baselines. ProGuT achieves a Panoptic Quality (PQ) of 65.2 on Our-forest dataset (2.6x improvement over the initial pseudo-label quality) and reaches 65.9 mIoU on Freiburg Forest, outperforming unsupervised baselines like PiCIE (45.3 IoU) and STEGO(57.6IoU). Additionally, ProGuT outperforms existing unsupervised methods for class-agnostic trunk instance benchmark.

[52] Less Supervision, Better Generalization: Weakly Supervised Fake Region Localization in Diffusion-Edited Images cs.CVPDF

Junhee Lee, Donghyeon Jeon, Taeoh Kim, Beomyoung Kim, MyeongAh Cho

TL;DR: ReGFLoW仅用图像级真假标签，通过扩散重建误差提供空间引导，实现弱监督扩散编辑区域定位。

Details

Motivation: 编辑痕迹细微且分布不规则，常不沿语义对象边界；像素级监督难扩展，人工/流水线掩码也可能将掩码外伪影错误当作真实。

Result: 多个实验显示ReGFLoW的域外泛化强于全监督定位基线；摘要未报告具体量化数据。

Insight: 扩散重建误差同时注入特征空间和分数空间作为稠密伪线索，再用伪影中心多实例学习聚合局部证据，避免依赖语义相似度或边界形状构造伪掩码。

Abstract: Localizing AI-edited regions is essential for interpretable forensic analysis, but remains challenging due to subtle and spatially distributed artifacts that are misaligned with semantic or object boundaries. Existing approaches rely on pixel-level supervision from controlled editing pipelines, which is difficult to scale and can introduce misleading signals: artifacts frequently extend beyond annotated regions, while out-of-mask pixels are treated as authentic. This limits models’ ability to capture transferable evidence and generalize across generators and datasets. To address these issues, we propose ReGFLoW, a Reconstruction-Guided Fake Localization framework under Weak supervision, which is the first weakly supervised approach for diffusion-edited fake region localization. ReGFLoW requires only real/fake labels at the image-level and uses diffusion reconstruction errors as dense spatial guidance to inject them into both feature and score spaces. Furthermore, by artifact-centric multiple instance learning, ReGFLoW utilizes localized diffusion evidence without relying on semantic-affinity or boundary-based pseudo-mask priors. Extensive experiments demonstrate that ReGFLoW achieves stronger out-of-domain generalization than fully supervised learning baselines.

[53] DiffReID: Discriminative Diffusion Model for Object Re-Identification cs.CVPDF

Yingquan Wang, Pingping Zhang, Dong Wang, Huchuan Lu

TL;DR: DiffReID以判别式扩散学习身份感知分布，并从视觉条件噪声生成身份不变引导特征，改善跨摄像机重识别泛化。

Details

Motivation: ReID数据规模和多样性有限导致模型跨摄像机泛化差，且现有方法多学语义模式，未充分刻画身份相关特征分布。

Result: 在五个对象重识别基准上，DiffReID优于大多数先进方法；摘要未提供具体指标。

Insight: 先对CLIP提示调优获取身份感知文本特征，再用视觉引导噪声逐步扰动，由轻量去噪器以视觉特征为条件学习分布；随机视觉引导噪声生成身份不变特征，最后以互增强约束协同改善视觉和生成表征。

Abstract: As a fundamental image processing task, object Re-Identification (ReID) aims to retrieve objects across non-overlapping cameras. Recently, with the development of deep learning, significant advancements have been made in object ReID. However, most existing methods suffer from generalization due to the limited size and diversity of ReID datasets. Meanwhile, current models tend to focus on extracting semantic patterns rather than learning identity-aware feature distributions. To address these issues, we propose a novel feature learning framework named \textbf{DiffReID} for object ReID. It leverages a discriminative diffusion model to gradually learn identity-aware distributions and generate identity-invariant features. More specifically, with the Contrastive Language-Image Pre-training (CLIP) model, we first obtain identity-aware text features by prompt tuning. Then, we propose a Vision-guided Noise Generator (VNG) to initialize probabilistic noises and gradually corrupt identity-aware text features. Afterwards, we take visual features as conditions and propose a Light Weight Denoiser (LWD) to denoise the corrupted text features step-by-step for identity-aware distribution learning. To obtain discriminative features, we further generate identity-invariant guided features from randomly sampling visual-guided noises. Finally, we propose a Mutual Enhancement Constraint (MEC) to facilitate mutual learning between visual features and guided features to enhance the representation robustness and discrimination. Extensive experiments on five object ReID benchmarks demonstrate that our method shows better results than most state-of-the-art methods. The source code is available at https://github.com/AWangYQ/DiffReID.

[54] SafeVantage: Vantage-Aware Memory for Reliable Embodied Decisions cs.CVPDF

Sean Hardesty Lewis, Zuyi Guo, Benwang Chen, Zirui Liu, Hongyi Lin

TL;DR: SafeVantage按主张保存支持视角、相机位姿和目标位置，并主动选择新视点补充证据，再输出是、否或弃答。

Details

Motivation: 部分可观测环境中的具身决策不仅需语义分数，还要说明哪些视点支持某个主张、下一步应到哪里寻找证据。

Result: 在232个未见ProcTHOR房屋、每种方法和预算7,424组成对episode上，相较等预算基线，8/12步时宏F1分别提高24.7%/12.0%，两种预算下风险更低、回答率更高；8步时行走距离减少31.7%。

Insight: 学习候选可观测性模型预测目标在可达视点是否可见，按预期降低最终决策损失选视点，同时考虑移动成本及几何不同的交叉佐证。校准头结合支持、空间一致性和覆盖率判Yes/No/Abstain；HM3D固定输入测试和ScanNet干预进一步验证证据视角价值。

Abstract: Reliable embodied decisions under partial observability require informative observations and sufficient supporting evidence. However, semantic scores alone do not reveal which viewpoints justify a claim or where additional evidence should be acquired. We introduce SafeVantage, a vantage-aware semantic memory and active acquisition framework that retains each claim’s supporting views, camera poses, and estimated target location, keeping positive support distinct from search coverage. A learned candidate-observability model uses claim-grounded geometry to predict target visibility at reachable viewpoints. These predictions guide view selection through expected reduction in terminal decision loss, accounting for travel cost and geometrically distinct corroboration. A calibrated head then combines support, spatial consistency, and coverage to produce Yes, No, or Abstain decisions. We evaluate SafeVantage on a category-presence benchmark spanning 232 unseen ProcTHOR houses and 7,424 paired episodes per method and action budget. Compared with validation-selected equal-budget baselines, SafeVantage achieves macro-F1 gains of 24.7% and 12.0% at eight and twelve actions, respectively, with lower risk and higher answer rates at both budgets and 31.7% less travel at eight actions. Equal-input HM3D experiments show lower selective risk under fixed observations, while controlled ScanNet interventions show that restoring supporting views improves downstream VLM answers. Ablations further support the contribution of candidate observability to decision quality and acquisition efficiency. Results demonstrate the value of claim-level viewpoint evidence for connecting semantic memory, active acquisition, and reliable decision-making. Code is available at https://safevantage.github.io

[55] SFE-VGGT: Source-Free VGGT Distillation for Event-Based Monocular Depth Estimation cs.CVPDF

Thai Duy Nguyen, Addison Lin Wang

TL;DR: SFE-VGGT无需配对RGB图像或深度标注，从目标事件流重建代理帧作为冻结几何教师，并以可靠性感知蒸馏迁移VGGT先验。

Details

Motivation: 事件相机深度估计已有方法依赖同步RGB—事件对或深度标签，限制实际部署；代理帧和空间变化监督若直接蒸馏也会传播伪影。

Result: 标准场景下SFE-VGGT接近RGB依赖基线，夜间明显超过对手；MVSEC夜间序列平均10米深度误差比EventVGGT低15.3%，并能跨真实数据零样本泛化。

Insight: 密度感知特征蒸馏强调信息量高的事件区域，置信加权深度蒸馏按师生相对置信动态调整监督；跨帧关系一致性利用可靠帧间对应约束时序几何，无需教师深度本身时间一致。

Abstract: Recent event-based depth estimation methods successfully transfer geometric priors from vision foundation models via cross-modal distillation. However, their reliance on synchronized RGB-event pairs or depth annotations during training severely restricts practical deployment. To overcome this bottleneck, we propose SFE-VGGT, a novel source-free framework that distills the geometric priors of VGGT to the event domain without any paired RGB observations. Our core idea is to reconstruct surrogate frames directly from the target event stream to act as a frozen geometric teacher, entirely eliminating the need for genuine source RGB data. Crucially, as these surrogate frames inherently yield imperfect and spatially varying supervision, directly distilling from them propagates artifacts. To resolve this, we introduce a novel reliability-aware distillation strategy. This includes Density-Aware Feature Distillation to emphasize informative event regions, and Confidence-Weighted Depth Distillation to dynamically regulate supervision based on relative teacher-student prediction confidence. Meanwhile, we propose a Cross-Frame Relational Consistency loss that enforces temporal geometric stability using reliable inter-frame correspondences, bypassing the need for temporally consistent teacher’s depth. Extensive experiments demonstrate that, despite source-free, our SFE-VGGT closely matches the accuracy of RGB-dependent baselines under standard conditions and significantly surpasses them in challenging nighttime scenarios. Across MVSEC nighttime sequences, SFE-VGGT reduces the average 10 m depth error by 15.3% compared with EventVGGT. Moreover, our method exhibits robust zero-shot generalization across real-world datasets, proving that highly effective geometric priors can be transferred to event cameras using strictly source-free supervision.

[56] Representation Dynamics Reveal Semantic Saliency and Similarity for Visual Token Pruning in MLLMs cs.CVPDF

Weixuan Li, Zikun Zhou, Xinyi Zhuang, Xinyan Guo, Rui Tian

TL;DR: MSDG-Prune按视觉编码器深度中token更新幅度和方向组织视觉token，保留显著区域与语义多样性以加速MLLM推理。

Details

Motivation: 常见剪枝用注意力或输出特征估计token重要性，但表示变化何时对应前景显著性、何时体现语义相似性尚不清楚。

Result: 在LLaVA-NeXT上仅保留5.6%视觉token仍保持未压缩性能的91.9%，预填充加速7.8倍；在四种MLLM中显示有效且可迁移。

Insight: 分析发现token更新幅度与显著性的关系依层变化，前景集中在两个深度区间，中间层则受注意力汇聚影响；更新方向相似度比编码器输出特征更能区分类别。方法按更新方向聚类，再以窗口内更新幅度得到查询加权显著性做组级剪枝。

Abstract: Multimodal large language models (MLLMs) incur high inference latency from long visual token sequences. Existing pruning methods commonly use attention maps or output features to estimate token importance or redundancy. Several recent approaches also exploit representation changes, but when and how these changes reflect foreground saliency and semantic consistency remain insufficiently understood. We analyze visual token representation dynamics across encoder depth and uncover two findings. First, the relationship between token update magnitudes and foreground saliency is layer-dependent: large token updates concentrate on foreground regions in two depth intervals, separated by several sink-dominated layers at intermediate depths. Second, similarities between token update directions better distinguish same-class from different-class tokens than those between encoder output features. Building on these findings, we propose MSDG-Prune, a training-free method that uses update magnitudes and directions to preserve salient and diverse visual information. Specifically, we group tokens by update-direction similarity and use query-weighted saliency derived from update magnitudes across a chosen depth window for group-wise token pruning. Extensive experiments across four MLLMs demonstrate the effectiveness and generalizability of MSDG-Prune. On LLaVA-NeXT, it retains 91.9% of uncompressed performance on average with only 5.6% of visual tokens, while achieving a 7.8x prefilling speedup. Code is available at https://github.com/liweixuan-hitsz/MSDG-Prune.

[57] Seg3DParts: Segmentation-Grounded Controllable Part-Level 3D Generation cs.CVPDF

Jiantao Lin, Meixi Chen, Yingjie Xu, Chenbo Fu, Leyi Wu

TL;DR: Seg3DParts将图像分割作为显式部件身份锚点，生成对齐到共享规范空间的可控多部件3D网格。

Details

Motivation: 从单张图像恢复部件级3D结构会遇到遮挡、边界不清及多部件一致性问题；现有方法难同时提供可控分解和整体结构连贯性。

Result: 在新构建的PartObjectNet（20万余物体、100多万部件）及实验中，Seg3DParts在几何质量、部件间一致性和分解控制上优于已有方法。

Insight: 每个组件由对应图像区域锚定身份，并通过生成过程中的跨部件交互交换全局上下文；直接输出共享规范空间中的对齐网格，避免事后再配准。

Abstract: Part-level 3D assets are essential for editing, reassembly, and interaction, yet recovering such structure from a single image remains challenging due to occlusion, ambiguous boundaries, and the need for coherent multi-part reasoning. Existing approaches struggle to achieve both controllable part-level generation and coherent multi-part structure, as part identity and spatial allocation are typically inferred implicitly. We present Seg3DParts, a segmentation-grounded framework for controllable part-level 3D generation from a single image. By treating segmentation as an explicit grounding signal, our method defines part identity during generation, enabling each component to be anchored to a corresponding image region. To ensure coherent assemblies, we introduce structured cross-part interaction that allows components to exchange global context throughout the generative process. As a result, Seg3DParts directly generates well-aligned part meshes in a shared canonical space without post-hoc alignment, supporting flexible and controllable decomposition. We further introduce PartObjectNet, a large-scale dataset with over 200K objects and 1M annotated parts. Experiments demonstrate that Seg3DParts achieves superior geometry quality, cross-part coherence, and part-level controllability over existing methods.

[58] WeLike2Party! In-Context Motion Transfer for Multi-Human Image Animation cs.CV | cs.AIPDF

Sangeyl Lee, Seunghyun Shin, Seungho Park, Wooseok Jeon, Hae-Gon Jeon

TL;DR: WeLike2Party通过直接上下文视频条件实现多人物图像动画，保留人物身份与动作对应，无需推理时提取骨架或人体网格。

Details

Motivation: 多人物交互动画难保持高保真，显式姿态和网格方法在人物遮挡时易破坏身份—动作绑定。

Result: 在新构建的MotionTwin-Bench及真实视频上，WeLike2Party超过近期方法的人物级保真、身份—动作绑定及整体感知质量，尤其在强遮挡多人互动场景。

Insight: 使用参考非对称RoPE保持细节，通过身份绑定监督把每个参考人物与目标轨迹对应；构建14.4K跨身份视频对、共84.3小时合成数据，在共享主体和相机运动下训练跨身份迁移。

Abstract: Human image animation aims to transfer motion from a driving video to subjects in a reference image. Despite remarkable progress in video generation, achieving high-fidelity animation of multiple interacting subjects remains a challenge. Many existing approaches rely on explicit motion representations such as 2D skeletons or parametric body meshes and struggle to preserve identity-motion binding under inter-person occlusion. To address this limitation, we propose WeLike2Party, a multi-human animation framework built on direct in-context video conditioning without explicit pose or mesh extraction at inference. We further introduce Reference Asymmetric RoPE Conditioning to preserve fine-grained appearance details, and Identity Binding Supervision to associate each reference identity with its intended motion trajectory. To support cross-identity training, we construct MotionTwin, a large-scale synthetic dataset comprising 14.4K cross-identity video pairs with shared subject and camera motions, totaling 84.3 hours of photorealistic video. We additionally present MotionTwin-Bench, a cross-identity benchmark specifically designed to evaluate subject-level visual fidelity and identity-motion binding. Extensive experiments on MotionTwin-Bench and real-world videos demonstrate that WeLike2Party outperforms recent state-of-the-art methods in subject-level visual fidelity, identity-motion binding, and overall perceptual quality, particularly in multi-person interactions with substantial occlusion.

[59] DispFlow-GS: Displacement Flow Supervision with Motion Disentangling for Monocular Deformable 3D Gaussian Splatting cs.CVPDF

Thai Duy Nguyen, Haitian Zhang, Addison Lin Wang

TL;DR: DispFlow-GS通过投影高斯三维位移监督单目动态3D高斯，使运动信号更直接稳定，并分离场景与相机运动。

Details

Motivation: 现有动态高斯方法将渲染高斯流与光流对齐作为运动监督，但两种流存在域差距，提升有限；运动更真实也不一定改善传统图像质量指标。

Result: 动态场景基准上，运动定位最高改善39%、运动—渲染一致性最高提升6%，但图像指标变化仅约0.1%，验证运动保真与图像评价之间存在明显错位。

Insight: 把每个高斯的三维位移投影到图像面形成直接位移流；通过中间视角渲染将场景运动与相机运动解耦。提出DRC指标衡量预测形变与渲染改善的一致性，补充单靠图像质量难以衡量的运动忠实度。

Abstract: Accurate dynamic scene reconstruction is important for robotic perception, where temporally consistent representations of dynamic environments are essential. Deformable 3D Gaussian Splatting (3DGS) models dynamic scenes through deformation fields, and recent methods incorporate motion supervision by aligning rendered Gaussian flow with optical flow. However, we find that such Gaussian-flow-based supervision provides only limited improvements in motion modeling. We identify a fundamental limitation of this supervision paradigm, namely a domain gap between rendered Gaussian flow and optical flow. To address this limitation, we propose a motion supervision framework built on Displacement Flow, which splats per-Gaussian 3D displacements onto the image plane to provide direct and stable optimization signals. We further disentangle scene motion from camera motion via intermediate-view rendering, enabling more reliable motion priors and targeted constraints on deformation and geometry. We also observe a discrepancy between motion fidelity and image-based evaluation, where improved motion awareness does not necessarily translate into better rendered image quality or higher image-based metric scores. Motivated by this mismatch, we introduce Deformation-Rendering Consistency (DRC), a motion-aware metric that measures the alignment between predicted deformation and rendering improvement. Experiments on dynamic scene benchmarks show substantial improvements in motion localization and motion–rendering consistency, reaching up to 39% and 6%, respectively, while image-based metrics change by only about 0.1%. These results confirm the observed mismatch between motion fidelity and image-based evaluation, demonstrating the significance of DRC for motion-aware evaluation.

[60] Salt++: Context-Aligned Post-Training for Few-Step Streaming Multimodal Generation cs.CV | cs.AI | eess.IVPDF

Xingtong Ge, Yutong Wang, Lunjie Zhu, Haitao Lin, Fangyu Lin

TL;DR: Salt++先用自流匹配改善因果多模态表征，再在采样、假分数和真实分数阶段统一前缀与因果掩码进行少步流式音视频蒸馏。

Details

Motivation: 少步流式音视频生成既要因果建模又要压缩去噪步数。教师强制用干净历史配噪声目标，只通过速度损失间接监督上下文表征；直接用双向分数模型做因果DMD又会产生上下文错配。

Result: 480p四步因果设置下，Salt++在JavisBench上视觉和运动质量较OmniForcing分别提高57%和45%；进一步扩展到1664×960四步生成，在报告七项指标中的六项超过双向LTX-2。

Insight: CSF固定噪声目标、改变历史，使噪声混合历史学生的中间表征对齐干净历史EMA教师；因果自回归DMD在生成采样、假分数训练和真实分数评估中共享前缀及掩码。经校准教师引导后，可不换目标也不做单独一致性蒸馏而适配生成历史。

Abstract: Few-step streaming audio–video generation requires both causal modeling and step distillation, yet standard training recipes face two context-related challenges. Teacher forcing pairs clean history with a noisy target, but supervises predictive contextual representations only indirectly through velocity prediction. Meanwhile, directly reusing bidirectional score models in causal Distribution Matching Distillation (DMD) creates a mismatch between generation and scoring contexts. We address these challenges with Salt++, a two-stage post-training framework comprising Causal Self-Flow (CSF) and context-aligned autoregressive DMD. CSF exploits contextual information asymmetry by varying the history while keeping the noisy target fixed: a noise-mixed-history student aligns its intermediate representations with those of a clean-history exponential-moving-average teacher. This self-supervised signal encourages the student to extract semantic information and improves cross-modal alignment. Context-aligned AR DMD shares the causal mask and prefix across generator sampling, fake-score training, and real-score evaluation to match generated and reference distributions under a block-conditional KL objective. With calibrated teacher guidance, it performs clean-prefix few-step distillation and then adapts to generated histories without switching objectives or requiring separate consistency distillation. At 480p, Salt++ improves visual and motion quality by 57% and 45% over OmniForcing on JavisBench under the same 4-step causal setting. A separate scale-wise post-training stage extends Salt++ to 4-step $1664\times960$ generation, outperforming bidirectional LTX-2 on six of seven reported metrics. Project page: https://xingtongge.github.io/Saltpp

[61] Parameterized Stripe Attention for Efficient Video Generation cs.CV | cs.AIPDF

Xingyu Jia, Baole Ai, Ang Wang, Kang Zhao, Yong Li

TL;DR: PSA用可参数化周期对角条纹表示视频扩散Transformer的稀疏注意力，并以统一高效CUDA核处理不同稀疏模式。

Details

Motivation: 视频扩散Transformer完整时空注意力推理耗时高；固定掩码缺少适应性，运行时动态生成掩码又带来开销并损害硬件效率。

Result: 在HunyuanVideo和Wan 2.1上，较FlashAttention-3基线端到端分别加速1.57倍和1.37倍，视觉质量退化可接受。

Insight: 分析发现注意力沿时间和空间维度均呈周期性对角条纹结构；PSA用统一结构参数编码多种模式，再通过训练免的离线搜索对每个头在误差容限下最大化稀疏度，运行时以单核实现。

Abstract: Diffusion Transformers (DiTs) enable high-quality video generation but suffer from substantial inference latency, primarily attributable to the computationally expensive full spatio-temporal attention. While sparse attention methods offer potential solutions, existing approaches face an inherent flexibility–efficiency dilemma: predefined masks lack the flexibility to capture diverse attention patterns, while runtime-determined masks introduce overheads and sacrifice hardware efficiency. We identify the lack of a unified structural characterization of DiT attention as a key limitation of existing methods, and establish that video DiT attention exhibits \textbf{periodic diagonal stripe structures} along both temporal and spatial dimensions. To formally encode these structured patterns within a single efficient kernel, we present {\bf PSA}, a parameterized stripe attention that formalizes the observed stripe regularity, unifying diverse attention patterns for efficient mask generation. This unified representation enables a single hardware-efficient CUDA kernel to process all sparse patterns, achieving FlashAttention-3-level Model FLOPs Utilization. To determine optimal sparsity configurations, we propose a training-free offline search algorithm that automatically maximizes sparsity under a specified error tolerance for each attention head. Experiments on HunyuanVideo and Wan~2.1 demonstrate that PSA achieves 1.57$\times$ and 1.37$\times$ end-to-end speedups over FlashAttention-3 baselines, with acceptable visual quality degradation.

[62] Beyond Readability: Evaluating Task Information Recoverability cs.CVPDF

Yiwei Liu

TL;DR: 本文区分直接视觉可读性与沿多种知识路径恢复任务信息的能力，并提出按指定观测、查询、目标和知识资源评估二者关系。

Details

Motivation: 目标无法从固定观测中直接读出，并不代表信息不能经图像恢复或实体识别后通过知识关系获取；只测OCR式可读性会低估实际任务信息可达性。

Result: 受控书封研究固定书名—作者目录，对比光学直接恢复作者与先识别书名再查目录；不同VLM均出现超出所测直接视觉能力范围的实体关联成功，同时仍保留相当大的纯光学可恢复区域。

Insight: 用各路径成功集合重叠刻画可恢复性，在书封降质时分别测试直接光学读出（可先恢复图像）和基于残余视觉证据的实体识别加知识关系查询。报告应明确可用外部知识及其来源，不能把间接查得的信息误称为直接视觉阅读。

Abstract: Direct visual readability and task-information recoverability are different quantities. Failure to decode a target from a fixed observation need not eliminate access to that target through another recovery route. We develop an evaluation perspective that makes the observation, query, target, and available knowledge explicit and measures the overlap between routes’ success sets. For information available on the original visible surface under suitable imaging conditions, direct optical recovery reads the target from the image, optionally after restoration; entity-linked recovery uses residual visual evidence to identify the depicted entity and accesses its target through an entity–attribute relation in a specified knowledge resource. Such access can draw on stored knowledge or an external source. A controlled book-cover study instantiates external access with a fixed title–author catalog, comparing optical author recovery with visual title resolution and deterministic lookup under resolution degradation. Entity-linked successes persist across the tested vision–language models, revealing information access beyond the tested direct visual frontier despite substantial differences in absolute performance. A substantial optical-only region remains. These complementary outcomes show why visual degradation should be evaluated through the task information accessible along specified routes and knowledge resources, alongside direct readability.

[63] Visual Parallel Search: Learning to Search High-Resolution Images with Parallel Tile Inspection and Adaptive Zoom cs.CV | cs.AIPDF

Xijia Tao, Yihua Teng, Xinyu Fu, Cheng Gong, Ziru Liu

TL;DR: VPS先并行检查高分辨率图像网格，再按证据需要自适应放大区域，避免主模型在未获全局概览时就过早选择单一裁剪。

Details

Motivation: 高分辨率VQA常因模型未获取细小局部证据而失败；顺序放大虽能读取细节，却要求主模型在缺乏可靠全局概览前决定查看哪里。

Result: 相同模型对比中，VPS在15项里14项平均准确率超过专用纯放大搜索，最高提高8分，小型主模型收益尤明显；ZoomBench各规模约提高3.2分。SFT在五个基准分割都提升，HR-Bench 4K提高4.17分。

Insight: 主代理先调用问题条件子代理并行读取多个网格，再自适应调用放大。配对角色强化学习分别训练控制器和局部读图者；主模型单独RL可减少工具使用且内部pass@1相近，但外部准确率变化混合，显示全局搜索控制与局部证据读取存在角色不对称。

Abstract: High-resolution visual question answering often fails because a multimodal model does not acquire the small, spatially localized evidence needed to answer a question. Sequential zooming can recover detail, but it asks the main model to choose a region before obtaining a reliable overview. We introduce VPS, a visual parallel-search framework in which a main agent first invokes grid_search to inspect image tiles in parallel with question-conditioned sub-agents, and then adaptively invokes zoom_in PSisual Parallel Search improves mean accuracy over dedicated zoom-only search in 14 of 15 same-model comparisons, with gains up to 8.0 points and especially strong improvements for smaller main models. ZoomBench retains an approximately 3.2-point gain at every tested size. We further develop a supervision pipeline with hint-free verification and a paired role-specific GRPO surrogate for learning the controller and tile-reader roles. SFT improves observed accuracy on all five benchmark splits, including a 4.17-point gain on HR-Bench 4K. Role-specific RL further reshapes search behavior: main-only RL reduces mean tool use from 2.65 to 2.11 with similar pass@1 in an internal four-response evaluation, while external accuracy changes are mixed. Joint training reveals an asymmetry between local evidence reading and global search control. Together, these results support VPS as an effective inference-time scaffold and a trainable decomposition for visual evidence acquisition.

[64] VesselBench-800K: A Large-scale Perception Benchmark for Multimodal Vessel Detection, Counting, and Density Estimation cs.CVPDF

Danfeng Hong, Chenyu Li, Jocelyn Chanussot

TL;DR: VesselBench-800K构建覆盖全球、光学与SAR配对遥感影像的大规模船舶检测、计数和密度估计基准。

Details

Motivation: 现有遥感数据多聚焦单模态光学目标检测，难以应对云雨、昼夜覆盖以及单一成像模态的固有限制。

Result: 基准含80万张512×512图像，来源覆盖不同平台、传感器、场景、高度及合成数据，地面采样分辨率从4.5米到0.1米；论文评测多种检测、计数和密度模型，但摘要未给具体成绩。

Insight: 同时提供光学和SAR模态以及多种船舶感知任务，可研究不同成像线索互补和复杂海况下的表征，扩大此前通用遥感目标检测数据集的评测范围。

Abstract: Vessel perception from space is crucial for a wide range of maritime applications, from traffic monitoring to environmental protection. However, most existing datasets predominantly focus on general object detection tasks in optical remote sensing (RS) images. Relying solely on single-modality optical RS images proves inadequate for effectively perceiving vessel objects in complex maritime scenarios, where ever-changing weather conditions (e.g., clouds and rain), the need for day-and-night coverage, and the inherent limitations of a single imaging modality pose significant challenges. To fill this gap, we introduce VesselBench-800K, the largest-to-date benchmark dataset on a global scale for vessel perception in multimodal RS images. As its name suggests, VesselBench-800K comprises 800,000 images, each at a resolution of 512x512 pixels, specifically curated for vessel perception tasks such as detection, counting, and density estimation. These multimodal image pairs (i.e., optical, SAR) are collected from diverse platforms, sensors, scenes, shooting heights, and synthetic sources, spanning spatial resolutions from 4.5m to 0.1m. Furthermore, we evaluate numerous state-of-the-art detection, counting, and density estimation models on VesselBench-800K through both qualitative and quantitative comparisons. By revealing previously unrecognized cues, this dataset holds immense potential to significantly advance our understanding of marine traffic. Our VesselBench dataset will be publicly available at https://github.com/danfenghong/IEEE_TGRS_VesselBench to support and contribute to community development.

[65] World2Motion: Turning Video World Models into 3D Human Motion Generators cs.CVPDF

Tu Fangyuan, Xiangyue Zhang, Yiyi Cai, Yichen Peng, Kunhang Li

TL;DR: World2Motion将视频世界模型适配为单阶段场景感知3D人体运动生成器，并同步生成对应视频。

Details

Motivation: 3D动作生成器受训练动作库环境覆盖有限所限；世界模型环境先验广，但并非为全身运动设计，从其视频恢复动作又需高成本两阶段推理。

Result: 在多源交互基准上，World2Motion的人—文本对齐和场景交互优于所评3D动作生成器；交互成功率匹配两阶段基线，推理约快3.3倍。

Insight: 构建合成视频—动作配对与真实视频配估计3D动作的训练集；对视频和动作使用不同噪声级、但共享去噪进度的解耦日程，以适配两种模态去噪需求并减少动作抖动。

Abstract: We present World2Motion, a framework that generates scene-aware 3D human motion and corresponding video from a single image and a text prompt. While existing 3D motion generators learn from motion datasets, their generalization is constrained by limited coverage of environments. In contrast, video world models such as Cosmos 3 offer broader environmental priors but are not designed for full-body motion generation; recovering motion from their generated videos requires costly two-stage inference. To address these, we turn Cosmos 3 into a single-stage 3D motion generator. This adaptation has two challenges: the scarcity of paired video–motion data and temporal instability in the generated motion. First, we construct a training dataset combining synthetic video–motion pairs with real videos paired with estimated 3D motion. Second, we propose a shift-decoupled noise schedule that assigns different noise levels to video and motion through shared denoising progress. This design accommodates the different denoising requirements of the two modalities, reducing motion jitter. Experiments on a multi-source interaction benchmark show that World2Motion has better motion–text alignment and scene interaction compared with the evaluated 3D motion generators. It also matches the interaction success rate of the two-stage baseline while achieving approximately 3.3$\times$ faster inference.

[66] RBF-GNN: Rational Basis Functions for Pseudo-Coordinate based Graph Convolutions cs.CVPDF

Paweł Batorski, Abtin Pourhadi, Paul Swoboda

TL;DR: RBF-GNN用有理Padé基函数取代高维稀疏B样条，构造兼容欧氏、球面和角度伪坐标的图卷积架构。

Details

Motivation: SplineCNN等伪坐标图网络利用坐标引入空间归纳偏置，但稀疏激活B样条数量随维度指数增长，效率较低。

Result: 仅将多种流行网络中的SplineCNN换为RBF-GNN后，语义关键点匹配、形状匹配和事件相机视觉任务等均取得改善；摘要未提供具体数值。

Insight: 有理基函数降低高维基函数增长带来的开销；训练配方进一步使用样条子空间初始化和保持方差的权重重缩放，以稳定优化。

Abstract: We propose RBF-GNN, a new pseudo-coordinate based graph neural network architecture that takes into account Euclidean, spherical or angular coordinates and uses them to induce a powerful spatial inductive bias. Similar in architecture to SplineCNN, we improve upon the latter by replacing the less efficient sparse-activation based B-splines whose number grows exponentially with dimension by rational Padé basis functions. For effective training we propose a spline-subspace initialization and a variance-preserving weight rescaling. Experimentally, we evaluate on a number of popular neural network architectures that use SplineCNNs. We replace only the SplineCNNs with RBF-GNN. We achieve improved results, including on semantic keypoint matching, shape matching, event based camera computer vision tasks. We will make our implementation publicly available upon acceptance of the paper.

[67] MotionInsight: Diagnosing Object Motion Deficiencies in Generated Videos cs.CV | cs.AIPDF

Jiahao Zhan, Yongrui Ma, Qunliang Xing, Xuanyu Zhang, Jingqi Tong

TL;DR: MotionInsight以对象为中心评估生成视频中的一致性、运动连续性和物理合理性，并输出动作空间诊断与解释。

Details

Motivation: 视频生成模型仍常出现对象运动错误，但多数质量评测只关注美学或文本—视频对齐，不能定位具体运动缺陷。

Result: 作者构建含6,879段指定运动对象视频及细粒度维度分数、失效原因的VidMotion；MotionInsight能输出与人类一致的三类运动评分及有依据解释。摘要未给具体相关系数。

Insight: 将评估从RGB帧上的隐式整体判断转为显式运动表征诊断，让微妙缺陷更可见；还把运动专项奖励用于GRPO，以把诊断能力转化为训练信号。

Abstract: Despite rapid progress in video generation models, they still exhibit obvious motion deficiencies, often manifested as incorrect object motion. However, most existing video quality evaluations focus on aesthetic quality or text-video alignment. To address this gap, we study object-centric motion fidelity assessment, evaluating target objects along object consistency, motion continuity, and physical plausibility. To achieve this, we first introduce VidMotion, a diagnostic dataset of 6,879 videos with designated moving objects and fine-grained annotations including dimension-wise scores and failure causes. We further propose MotionInsight, a diagnostic evaluator that shifts assessment from implicit RGB-frame observation to explicit motion-space diagnosis. By constructing motion-aware representations, MotionInsight makes subtle motion deficiencies more observable. We also introduce motion-specific rewards during GRPO to transform observed motion into a diagnostic assessment. Experiments demonstrate that MotionInsight provides an effective basis for diagnosing object motion deficiencies, producing human-aligned scores along three dimensions and grounded explanations.

[68] GleanVID: Complementary Token Selection for Efficient Video Large Language Models cs.CV | cs.LGPDF

Shuo Yang, Changbai Li, Rui Tang, Xinyu Zhao, Linlin Yang

TL;DR: GleanVID将视频token选择视为渐进累积互补证据，按时间新颖度分配帧预算后共同考虑代表性和子空间互补性。

Details

Motivation: 现有视频LLM压缩多用独立token得分，忽略帧间证据互补，导致保留跨帧重复内容。

Result: 仅保留Qwen3-VL视觉token的25%时，平均保留原性能98.6%，预填充延迟降低44.7%；LLaVA-OV-7B在25%保留率下表现略高于原模型。

Insight: 免训练剪枝先按时间新颖度在帧间分配全局预算，再联合选择局部代表性强且和已选token子空间互补的证据，以避免重复而保留更丰富信息。

Abstract: Video Large Language Models (VideoLLMs) have achieved strong video understanding capabilities but incur substantial inference overhead due to the large number of visual tokens. Existing VideoLLM token compression methods largely rely on selection-independent scoring, overlooking cross-frame complementarity and consequently retaining redundant evidence across frames. Instead, we view video token selection as a progressive evidence accumulation process. It aims to retain visual evidence that is individually informative and collectively complementary under a limited token budget. Building on this insight, we introduce GleanVID, a training-free inference acceleration framework for VideoLLMs. Specifically, GleanVID first allocates the global token budget across frames according to temporal novelty and then selects tokens by jointly considering local representativeness and subspace complementarity, thereby preserving richer and less redundant visual evidence. Extensive experiments across diverse VideoLLMs and benchmarks demonstrate that GleanVID consistently achieves state-of-the-art performance. Notably, with only 25% of visual tokens, GleanVID preserves 98.6% of Qwen3-VL’s original performance while reducing its prefill latency by 44.7%. On LLaVA-OV-7B, GleanVID at a 25% retention ratio even slightly surpasses the original model.

Katharina Winter, Stefan Englmeier, Fabian B. Flohr

TL;DR: 本文用三类受控速度任务和表征探针分析驾驶VLM能否从图像可靠读取周边车速、自车速度及未来自车速度。

Details

Motivation: 驾驶VLM不断应用于自动驾驶，但从视觉输入恢复动态物理状态的能力和其规划输出之间的关系尚未充分刻画。

Result: 周边车辆速度在模型表征中弱且不具对象特异性；当前自车速度内部可读但口头输出差，连续探针MAE为4.7至5.8 km/h，语言回答为10.2至16.8。QLoRA改善表征和口头输出，但周车速度仍弱，未来速度提升在打乱帧序后多数保留。

Insight: 结合nuScenes多模型、多输入输出格式的口头评估、时间扰动、反事实自车速度提示与线性探针，揭示专用驾驶模型可加强周车和未来速度潜表征，却不保证可靠读出或时序落地；看似合理的规划不能证明动态状态已被准确理解。

Abstract: Vision-Language Models are increasingly used in autonomous-driving systems, yet their ability to recover dynamic physical state from visual input remains insufficiently characterized. We study velocity understanding as a controlled diagnostic across three tasks: surrounding-agent speed, current ego speed, and short-horizon future ego-speed proposal. On nuScenes, we evaluate open-weight general-purpose and PhysicalAI VLMs, together with the driving-oriented Alpamayo-1.5 Vision-Language-Action model, using multiple input and output formulations. We combine verbal evaluation with temporal perturbations, counterfactual ego-speed hints and linear probes of hidden representations. The tasks exhibit distinct failure modes. Surrounding-agent speed is weakly encoded in an agent-specific form, whereas current ego speed is often internally accessible but poorly verbalized: continuous probes achieve 4.7-5.8 km/h MAE compared with 10.2-16.8 km/h MAE for verbal outputs. Multiple frames provide inconsistent verbal gains to single frame inputs, and frame order is rarely exploited. Under non-optimized QLoRA, task-specific adaptation improves both task-relevant latent speed representations and verbal readout, but continuous surrounding-agent speed estimation remains weak, while most future-speed gains survive frame shuffling, indicating limited temporal grounding. Driving specialized Alpamayo-1.5 shows stronger latent representations for surrounding-agent and future ego speed, while current ego-speed decodability is comparable and substantial probe-verbal gaps remain. Thus, driving specialization can strengthen motion representations but does not guarantee stronger encoding across both scene and ego states or reliable readout. The results show that plausible planning outputs do not necessarily imply reliable recovery or temporal grounding of the underlying dynamic state.

[70] Spatial-OPSD: Self-Improving Spatial Reasoning via Label-Free Self-Distillation cs.CVPDF

Zhenyu Liu, Zhangquan Chen, Keyi Chen, Mingze Sun, Xiang An

TL;DR: Spatial-OPSD用深度、重建3D关系和相机几何等无标签空间先验构造教师，在学生自生成轨迹上蒸馏空间推理能力。

Details

Motivation: VLM进入具身空间任务后需理解深度、视点及三维关系，但提升空间推理通常依赖真值答案、答案奖励或任务专用监督。

Result: 四类VLM中，单轮Spatial-OPSD均提升五基准平均分；三轮递归训练使强空间专用模型达到开源模型平均前沿，并在五项基准中的三项取得最佳结果。

Insight: 训练教师可利用感知和重建工具自动获得的特权几何先验，学生仍只看原始图文输入，获得token级密集监督；每轮内冻结教师稳定目标，下一轮再以增强后的学生同时初始化师生，重新建立有信息量的不对称。

Abstract: Vision-language models (VLMs) increasingly operate in embodied and spatially grounded settings, where accurate understanding of depth, viewpoint, and three-dimensional relations is essential. However, improving spatial reasoning typically relies on ground-truth answers, answer-derived rewards, or other forms of task-specific supervision. We introduce Spatial-OPSD, a label-free self-improvement framework that instead exploits spatial structure naturally available from perception and reconstruction tools. During training, a privileged teacher receives automatically obtainable spatial priors, such as depth, reconstructed 3D relations, and camera geometry, while the student observes only the original visual-language input. On trajectories sampled by the student itself, the teacher provides dense token-level supervision, allowing the student to internalize spatial knowledge without ground-truth answer labels or privileged information at inference time. To extend this supervision beyond a single round, we adopt a round-wise recursive training scheme: the teacher remains frozen within each round to provide a stable learning target, and the improved student initializes both teacher and student in the next round, where privileged spatial priors re-establish an informative teacher–student asymmetry. This enables repeated self-improvement while avoiding a rapidly moving teacher during optimization. Across four VLM families, a single round of Spatial-OPSD consistently improves the five-benchmark average, while three rounds further push a strong spatially specialized model to the open-source frontier, achieving the highest average among the open models and the best results on three of five spatial reasoning benchmarks. Our code is available at https://github.com/vermouth599/Spatial-OPSD.

[71] OmniRoute: Mapping Temporal Semantic Evidence to Audio-Visual Token Budgets for Efficient Omnimodal Large Language Models cs.CVPDF

Yuchen Deng, Zidang Cai, Feidiao Yang, Yufei Wang, Jie Wang

TL;DR: OmniRoute按片段时间语义重要度先给音视频主模态分配token预算，再依实际保留比例校准另一模态的压缩目标。

Details

Motivation: 全模态模型将音频和视频编码为时间交错token，长序列预填充成本高；传统压缩较少考虑音视频语义相关度随时间变化及局部连续性。

Result: 四个代表基准上，OmniRoute相较竞争基线取得更好的推理效率—性能权衡；摘要未提供具体分数。

Insight: TEGB综合语义相关性和局部变化，按块选择主导模态及初始预算；BCSC先压缩主模态，再据实际保留率校准从模态。视频结合时空分组和查询选token，音频结合编码器注意力与查询相关度，并在视觉引导下把剩余token合并进上下文锚点。

Abstract: Omnimodal large language models (Omni-LLMs) encode audio and visual streams into temporally interleaved token sequences for multimodal reasoning. However, processing long audio-visual token sequences incurs substantial prefill costs. Existing compression methods have made progress, but often overlook temporal changes in audio-visual semantic relevance. Motivated by temporal variation and local continuity, we propose OmniRoute, a training-free, two-stage compression framework. First, Temporal Evidence-Guided Budgeting (TEGB) derives chunk-wise modality preferences and initial leading-modality budgets from semantic relevance and local content variation. Second, Budget-Constrained Semantic Compression (BCSC) compresses the leading modality and then calibrates the follower’s retention target using the actual retained fraction. For video, it combines spatiotemporal grouping with query-guided selection; for audio, it selects tokens based on encoder attention and query relevance, then merges residual tokens into context anchors under visual guidance. Experiments on four representative benchmarks demonstrate a better trade-off between inference efficiency and performance than competitive baselines. The code and interface will be released to facilitate further research.

[72] Real2Gym: Building Gyms from Videos, Bringing Skills to Robots cs.CVPDF

Kerui Ren, Yingxiang Xu, Kaiwen Song, Linning Xu, Bo Dai

TL;DR: Real2Gym从人类或机器人视频构建可交互仿真环境，让代理在模拟中学习可复用操作技能并迁移到真实机器人。

Details

Motivation: 现实视频富含操作示范，但转为机器人技能还需与图像对齐的环境、可执行物理互动和利用经验持续改进的机制。

Result: 系统在环境重建上表现强；相比GPT-6 Astra Direct Mode，成功率提高16.7%，策略执行token减少约74.9%。真实Franka机器人四项任务上的物理执行成功率高33.3%。

Insight: Real2Sim模块重建可编辑场景、对齐物体和相机，并用原生物理验证动作及动作可行性；代理执行分步代码并把成功、失败经验提炼为任务程序、物体相对运动和恢复策略。共享感知控制接口再将技能用于模拟与实机，不更新底层模型权重。

Abstract: Real-world videos provide rich demonstrations of manipulation, but turning them into reusable robot skills requires visually aligned environments, executable physical interactions, and mechanisms for learning from experience. We introduce Real2Gym, an agentic Real2Sim2Real framework that turns human and robot demonstrations into interactive simulation gyms and brings skills acquired in simulation to physical robots. The Real2Sim module reconstructs editable scenes, aligns objects and cameras with the input, validates demonstrated or retargeted actions through native physics execution, and generates task-conditioned variations with action-feasibility checks. Within these environments, the agent generates executable code for manipulation stages, observes their outcomes, and distills successful attempts and failures into reusable task procedures, object-relative motions, and recovery strategies. Through a shared perception-and-control interface, these skills guide subsequent execution in simulation and on real robots, with motions adapted to current observations and no updates to the underlying model weights. Extensive evaluations demonstrate that Real2Gym enables high-fidelity simulation environment reconstruction, outperforming GPT-6 Astra Direct Mode by 16.7% in success rate with approximately 74.9% fewer policy-execution tokens across these environments, while exceeding it by 33.3% in physical robot execution success rate across four tasks on a real Franka robot.

[73] LDM-is-AE: Latent Diffusion Model is an Auto-Encoder for End-to-End Image Generation cs.CVPDF

Zhengqiang Zhang, Lingchen Sun, Rongyuan Wu, Qiaosi Yi, Xiangtao Kong

TL;DR: LDM-is-AE把潜扩散模型内部每步的潜变量—特征—潜变量变换视为自编码过程，取消独立预训练tokenizer并端到端联合学习表示与去噪。

Details

Motivation: 两阶段LDM先训练重建型自编码器定义潜空间、再训练扩散去噪器，可能导致潜空间针对重建而非去噪动力学优化的表示错配。

Result: 类别条件图像生成在256×256和512×512分别达到FID 1.80和1.90，表现有竞争力。

Insight: 把DiT拆成互逆的编码端DiT-E和解码端DiT-D，对所有时间步中间特征施加图像空间监督，建立显式潜—图像—潜路径；零噪声步对应图像到潜变量再重建图像的自编码。

Abstract: Latent Diffusion Models (LDMs) typically adopt a two-stage pipeline: an auto-encoder (AE) is first pre-trained to define a latent space, then a diffusion model is trained to perform denoising within it. Such a two-stage design introduces a representation mismatch, as the latent space is optimized for reconstruction rather than adapting the denoising dynamics. We reveal that the LDM itself is an AE, and consequently present LDM-is-AE, an end-to-end one-stage LDM training framework that eliminates the need for a separately trained tokenizer. Our key observation is that the LDM backbone actually performs a latent-to-feature-to-latent transformation at each denoising step, which can be interpreted as an internal decoding–encoding process. Leveraging this structure, we split the DiT backbone into two reciprocal components, DiT-E (i.e., DiT Encoding) and DiT-D (i.e., DiT Decoding), and impose image-space supervision on the intermediate features across all timesteps. Our model encourages the internal representation to align with the image domain throughout denoising, thereby establishing an explicit latent-to-image-to-latent path. At the zero-noise timestep, our model further performs an image-to-latent-to-image mapping, corresponding to an auto-encoding process. As a result, LDM-is-AE jointly learns latent representations and denoising dynamics in an end-to-end manner, yielding a diffusion-native latent space tailored to the generation process. Experiments demonstrate that LDM-is-AE exhibits highly competitive generation performance, achieving an FID of 1.80 and 1.90 on 256x256 and 512x512 class-conditional image generation, respectively.

[74] Why MLLMs Struggle to Count: Overcoming Individuation and Aggregation Bottlenecks with ConvStack cs.CVPDF

Liwei Che, Yihao Quan, Sen Fang, Hongyi Wang, Ranjay Krishna

TL;DR: ConvStack分析MLLM精细计数中的对象个体化与数量聚合瓶颈，并在视觉token空间显式注入局部几何结构。

Details

Motivation: MLLM细粒度视觉计数表现差，但原因不明；patch独立处理可能无法跨边界合并碎片几何特征，后续注意力聚合又会随数量增大而压缩对象间表示差异。

Result: 仅针对计数任务微调后，ConvStack显著改善密集对象计数及更广泛空间理解基准，同时不损害一般视觉能力；摘要未给具体提升值。

Insight: 轻量结构直接在视觉token空间聚合并注入局部空间信息，通过零初始化残差连接，先消除对象个体化歧义，再支持后续数量聚合。

Abstract: Multimodal Large Language Models (MLLMs) consistently struggle with fine-grained visual counting, yet the underlying causes remain poorly understood. In this work, we present a mechanistic analysis of this failure mode, identifying two critical bottlenecks inherent to the global attention pipeline of MLLMs. First, we reveal an individuation bottleneck stemming from image patchification: because Vision Transformers process patches independently, they struggle to group fragmented geometric features across boundaries into distinct object representations. Second, we identify a collapse in the subsequent counting aggregation process, where representation separation rapidly diminishes as numerosity increases due to attention compression. Identifying and formalizing these twin bottlenecks constitutes our first major contribution. To overcome them, we propose ConvStack, a lightweight architecture that operates directly in the visual token space to explicitly aggregate and inject local spatial structures via zero-initialized residual connections. By explicitly addressing the individuation bottleneck, ConvStack provides unambiguous geometric evidence for downstream aggregation. Remarkably, by fine-tuning exclusively on counting tasks, the model achieves substantial improvements in dense object counting and broader spatial understanding benchmarks, without compromising on general visual capabilities.

[75] Waypoint-1.5: A Real-Time Video World Model for Consumer Hardware cs.CV | cs.HCPDF

Rajit Rajpal, Shahbuland Matiana, Liew Wei Pyn, Anmol Agarwal, Ryan Craig

TL;DR: Waypoint 1.5是能在消费级硬件实时交互的视频扩散世界模型，依据完整键盘和鼠标输入生成可玩的游戏视频。

Details

Motivation: 交互世界模型必须对密集用户控制快速响应，受严格延迟和吞吐约束，通用视频扩散模型未针对实时交互优化。

Result: 模型在数百款游戏上用10万小时控制对齐数据预训练，并提供覆盖多种消费硬件的两个分辨率版本；论文用渲染FPS、潜变量FPS及控制频率区分交互性能，并评估延迟与吞吐，但摘要未提供具体速度数值。

Insight: 工作描述数据、架构、训练和运行时系统，并讨论交互世界模型特有的安全伦理问题；分开报告渲染速率、潜空间更新率与控制采样率，有助区分画面流畅度与输入响应能力。

Abstract: We present Waypoint 1.5, a real-time diffusion world model for interactive video generation on consumer-grade hardware. Unlike general video diffusion models, interactive world models (iWMs) must respond to dense user controls under strict latency and throughput constraints. Waypoint 1.5 is pre-trained on 100,000 hours of diverse, control-aligned video game data across hundreds of games, and generates playable video conditioned on full keyboard and mouse input. The model includes two resolution variants that run across a wide spectrum of consumer hardware. To characterize this unique setting, we distinguish rendered FPS, latent FPS, and control rate. We describe the data pipeline, architecture, training methodology, and runtime system behind Waypoint 1.5. We evaluate interactivity through latency and throughput. Finally, we discuss the safety and ethics considerations unique to iWMs.

[76] Task-Oriented Visual Feature Compression via Residual Vector Quantization for Device-Edge Multimodal Inference cs.CVPDF

Luning Pang, Cheng Yuan, Jiawei Shao, Mingtao Huang, Yuan Shen

TL;DR: Q-TOFC用查询引导聚合和残差向量量化压缩设备端视觉特征，降低边缘协同多模态推理的上传量及带宽延迟。

Details

Motivation: 设备—边缘协同可卸载设备计算，但上传视觉数据会在有限带宽下造成高延迟；连续特征熵编码成本高，查询无关聚合又可能丢失局部任务证据。

Result: 在七个多模态基准上，Q-TOFC相对TOFC减少53.6%视觉载荷，同时保持相近平均归一化任务表现；端到端评测显示受带宽限制时延迟更低。

Insight: RVQ把每个合并特征编码成紧凑码本索引序列，以便传输更多特征；聚合阶段按问题相关度保留局部证据，并用量化误差补偿适配器减轻离散化损失。

Abstract: Large multimodal models (LMMs) support diverse visual understanding and reasoning tasks but are often impractical to run entirely on resource-constrained devices. Device-edge co-inference reduces device computation, yet transmitting visual data over bandwidth-limited uplinks can introduce substantial delay. Task-oriented feature compression (TOFC) reduces the payload through feature aggregation and entropy coding. However, continuous-feature coding remains costly, and query-agnostic aggregation may discard task-relevant local evidence. We propose query-guided task-oriented feature compression (Q-TOFC) for device-edge multimodal inference. Q-TOFC employs residual vector quantization (RVQ) to encode each merged feature as a compact sequence of codebook indices, reducing its representation cost and allowing more features to be transmitted. It further incorporates query relevance into feature aggregation and uses a quantization error compensation adapter to mitigate the distortion introduced by discrete quantization. Experiments on seven multimodal benchmarks show that Q-TOFC reduces the visual payload by 53.6% relative to TOFC while maintaining comparable average normalized task performance. End-to-end latency evaluations further demonstrate lower latency under bandwidth-constrained uplinks.

[77] EviViT: Evidence-Adaptive Vision Transformers for Fine-Grained Perception cs.CVPDF

Yaoxin Niu, Zhangquan Chen, Yang Zhang, Xiang An, Zhumei Wang

TL;DR: EviViT学习问题条件下的证据密度，从原始高分辨率图像局部重读区域，再用稀疏坐标桥将细节与全局场景连接。

Details

Motivation: 高分辨率整体输入会为无关区域消耗视觉token，独立裁剪又丢失解释局部证据所需的上下文。

Result: 九种主机骨干上细粒度准确率均有提升；在匹配token预算下，各测试上限中均优于只用全局视图，且视觉token更少。

Insight: 人类视觉搜索轨迹监督问题条件证据密度，指导区域重读和token分配；稀疏、带坐标的桥接将局部区域特征与全局图像联系。附件在冻结主机上训练，还能用于兼容的后训练衍生模型而无需重拟合。

Abstract: Fine-grained visual perception enables vision-language models to distinguish subtle attributes and ground their answers in visual evidence. In high-resolution scenes, processing the whole image at greater resolution spends visual tokens on irrelevant content, while isolated crops can lose the context needed to interpret the selected evidence. We introduce EviViT, a lightweight attachment that learns where a pretrained vision transformer should acquire detail. Human visual-search traces supervise a question-conditioned evidence density, which guides regional re-reading from the original pixels and the allocation of visual tokens. A sparse, coordinate-aware bridge then connects the regional features to the global scene, allowing the host to interpret precise evidence in context. Learned with the host backbone frozen, the attachment serves both the base model and compatible post-trained descendants without refitting. Experiments across nine hosts show consistent gains in average fine-grained accuracy. Matched-budget comparisons further show that EviViT outperforms global-only processing at every tested token ceiling while using fewer visual tokens.

[78] HaPRL: Human-Anchored Process Reinforcement Learning for Visual Search Agent cs.CVPDF

Zhangquan Chen, Yaoxin Niu, Xiang An, Mingze Sun, Zhumei Wang

TL;DR: HaPRL用同一图像上的人类视觉搜索轨迹监督多轮搜索代理过程，避免只奖励最终答案导致模型学到错误搜索路线。

Details

Motivation: 最终答案奖励不约束搜索过程，代理可能走错误推理路线却碰巧答对，使强化学习继续沿低效甚至不可靠路径扩展。

Result: 人工标注平台收集超过1,000条细粒度行为数据；HaPRL持续超过只按结果奖励的RL，早期过程监督使后续结果导向扩展收益提高6.7倍。

Insight: 裁判以人类搜索同图的蒸馏轨迹为锚，按任务自适应权重评估每条rollout的搜索过程，而非把所有正确终答视为同质经验。

Abstract: Multi-turn visual search agents answer questions about high-resolution images by iteratively deciding where to look. Reinforcement learning for these agents rewards only the final answer, leaving the search process unsupervised. Consequently, faulty routes in which the reasoning process is erroneous yet the final result is correct arise frequently, which in turn leads to ineffective training, i.e., scaling along the wrong paths. In this paper, we introduce HaPRL, the first framework to reinforce the search process with human search behavior. We first build an annotation platform and collect 1K+ human-annotated data with fine-grained behavioral signals. During training, a carefully designed judge scores each rollout with task-adaptive weights, anchored on the distilled trace of how a human annotator actually searched the same image. Extensive experiments show that HaPRL consistently outperforms outcome-based RL, and early-stage process supervision yields 6.7x more improvement in subsequent outcome-based scaling. Our results also demonstrate the importance of aligning model behavior with human process annotation signals, which offer new insight into the training of foundation models.

[79] InsightMap: Structured Spatial Modeling for Embodied Multimodal Reasoning cs.CV | cs.ROPDF

Hongpei Zheng, Hujun Yin

TL;DR: InsightMap把俯视地图同时用作持久空间记忆和动作条件预测目标，联合训练语言导航动作与行动后的地图生成。

Details

Motivation: 语言导航需要把局部观察对应到持久空间参照，并学习动作如何改变空间表示。

Result: R2R-CE和RxR-CE未见验证集成功率分别56.9%和54.9%；地图预测监督使R2R-CE成功率提升4.3、SPL提升3.2个百分点。静态空间任务ScanQA CIDEr 103.7、SQA3D准确率60.1%、ScanRefer grounding准确率53.1%。Unitree Go2在走廊、实验室和办公室中超过NaVid与NaVILA。

Insight: 历史视图与带标记的地图位置关联，共享多模态骨干同时预测导航动作和后续地图；统一对齐的RGB-D数据接口还支持VQA、场景推理和3D落地。推理时根据观察到的空间上下文直接解码动作。

Abstract: Language-guided navigation requires connecting partial observations to a persistent spatial reference and learning how actions change that representation. We introduce InsightMap, a framework that uses top-down maps as both explicit spatial memory and action-conditioned prediction targets. Historical views are linked to labeled map locations, and a shared multimodal backbone jointly learns navigation action prediction and post-action map generation. Map prediction provides auxiliary training supervision, while navigation inference decodes actions from the observed spatial context. An aligned RGB-D data pipeline supports a common interface for navigation, visual question answering, situated reasoning, and 3D grounding. On the validation-unseen splits of R2R-CE and RxR-CE, InsightMap achieves success rates (SR) of 56.9% and 54.9%, respectively. Adding map-prediction supervision improves R2R-CE SR by 4.3 and success weighted by path length (SPL) by 3.2 percentage points. On static spatial tasks, InsightMap achieves 103.7 CIDEr on ScanQA, 60.1% exact-match accuracy on SQA3D, and 53.1% grounding accuracy at 0.5 IoU on ScanRefer with detected object proposals. On Unitree Go2, it outperforms NaVid and NaVILA in hallway, lab, and office environments.

[80] Exploring In-Context Learning for Handwritten Text Recognition cs.CVPDF

Eric Ayllon, Abel Gandia, Jorge Calvo-Zaragoza

TL;DR: 本文探索用冻结预训练VLM的上下文学习做手写文本识别，并研究同域及跨手稿集合示例选择对转录质量的影响。

Details

Motivation: 历史文档手写识别能降低数字化成本并扩大内容可访问性，但传统专用HTR模型通常需大量标注样本。

Result: 无需更新模型参数，通用VLM在多个手稿集合上可以通过上下文示例学习转录，显示在领域偏移下有竞争潜力；上下文越大，误差范围趋向平均表现，牺牲最佳示例采样时的理想表现以降低预期误差。

Insight: 跨域场景将其他集合的上下文示例用于当前查询，ID和CD条件出现相同趋势。不同示例采样质量差异明显，未来需重点寻找理想上下文选择方法。

Abstract: Handwritten Text Recognition (HTR) systems have become an indispensable tool for the digitization of historical documents. Not only do they cut down time and cost, but they also allow democratizing access and processing of their contents by generating their transcripts. However, literature in HTR currently focuses mostly on specialized models that require large amounts of annotated samples to achieve satisfactory performance. We explore the use of In-Context Learning with pre-trained Vision-Language Models (VLMs) to create a transcription pipeline without updating the model’s parameters. We then evaluate this pipeline across multiple collections and models, and demonstrate that general-purpose VLMs can be effectively taught how to transcribe handwritten text from images. To assess how our observations may translate to practical applications, we evaluate the performance in a Cross-Domain (CD) scenario, where context examples are drawn from a different collection than the query image. Results in both the controlled In-Domain (ID) scenario and the realistic CD scenario follow the same patterns. First, as context size grows, the error range is expected to narrow towards the average performance. Thus, larger context sizes sacrifice the performance of the oracle-best sampling for lower expected error rates. The results obtained show that, without any parameter updates, this methodology has strong potential to compete with traditional HTR in the presence of domain shift. Moreover, we show and argue that some context samplings work better than others and suggest more effort should be put into finding an ideal sampling method in future work.

[81] ResComEmb: Effective and Efficient Multimodal Embedding via Residual Homogeneity Compression cs.CV | cs.AIPDF

Zijing Cai, Yuzhe Wang, Jingxian Zhu, Fengbin Zhu, Richang Hong

TL;DR: ResComEmb将动态分辨率图像编码为全局、中层和细节多视角嵌入，再按预算压缩同层及跨层冗余，保留多向量表征能力。

Details

Motivation: 单向量多模态嵌入表达细节有限，保留大量视觉token又增加存储和交互成本；需兼顾精细表达与可扩展效率。

Result: 在MMEB、ViDoRe V1和V2上，ResComEmb的通用多模态嵌入优于VLM2Vec-V2；文档检索只用ColQwen2.5完整视觉token预算的37.5%仍超过其效果。

Insight: 可训练残差同质性压缩模块按明确视觉token预算去冗余；长度自适应双向后期交互分别求查询到文档、文档到查询的强token匹配均值，再依据两侧有效token数加权合并分数。

Abstract: Multimodal large language models (MLLMs) have shown strong potential for universal multimodal representation learning. However, existing methods either compress each input into a single vector, limiting fine-grained expressiveness, or retain long sequences of visual-token vectors, incurring substantial storage and interaction costs. To resolve this trade-off, we propose ResComEmb, a trainable framework for effective and efficient universal multi-vector multimodal embedding. ResComEmb first encodes each input at native dynamic resolution into ordered global, intermediate, and fine-grained views. After MLLM contextualization and embedding projection, a trainable Residual Homogeneity Compression (RHC) module reduces within-granularity redundancy and cross-granularity repetition under explicit visual token budgets. Then, ResComEmb introduces a length-adaptive Bidirectional Late-Interaction Matching mechanism for robust query-document scoring, which averages the strongest token-level matches in each direction and combines the two scores using a weight based on how many valid tokens each side has. Extensive experiments on MMEB, ViDoRe V1, and ViDoRe V2 show that ResComEmb produces higher-quality universal multimodal embeddings than VLM2Vec-V2, and outperforms ColQwen2.5 in visual document retrieval using only 37.5% of its full visual token budget, demonstrating a favorable effectiveness-efficiency trade-off.

[82] Seeing Is Not Addressing: Auditing Linguistic Access to Frozen Visual Geometry cs.CV | cs.LGPDF

Woosang Jeon, Jiwon Yang, Soo Chung, Taehyeong Kim

TL;DR: 本文区分冻结图像表征中的视觉可辨别性与文本接口能够表达、访问相应区别的语言可寻址性。

Details

Motivation: 视觉差异可能比语言概念更细；VLM虽然能在图像嵌入空间区分两类，却未必能用原生文本提示准确指定该差异。

Result: 在跨形状、色调、纹理及干扰变化的23,040图像因子完全交叉基准上，匹配视觉对比可缩小文本访问差距；增益通过视觉缺失及方向性控制验证，并迁移到组合检索和自然图像。

Insight: 把同一视觉区别分别以图像表征判别和文本到图像检索读出进行对比；为每个属性值构造相对其他取值的图像侧对比，检验问题是否仅在文本命名接口，而非视觉空间本身。

Abstract: Visual distinctions are often finer than those reflected in linguistic conceptualization. Vision-language models exhibit a similar asymmetry: a distinction can remain discriminable in frozen image geometry while being weakly addressable through the native text interface. We study this gap by separating visual discriminability from linguistic addressability in text-to-image retrieval. Using FactorAtlas, a fully crossed testbed of 23,040 images spanning shape, hue, pattern, and nuisance variation, we compare both readouts on held-out images of the same distinctions. We then derive image-side contrasts that separate each value from its alternatives for matched visual grounding, and test whether this reduces the native-text access gap across factors and models. Direction-specific and visual-absence controls tie these gains to the relevant visual contrast; the gains persist after global alignment and extend to compositional retrieval and natural images. Together, these results show that visual discriminability and linguistic addressability need not coincide, and that matched visual grounding can probe and reduce the resulting access gap.

[83] Adaptive Reward Routing: Dynamic Multi-Reward Optimization for Joint Audio-Video Diffusion via Forward-Process RL cs.CVPDF

Songlin Yang, Xiaotong Zhao, Jiacheng Zhang, Zhe Wang, Toyota Li

TL;DR: 自适应奖励路由在联合音视频扩散前向过程RL中，同时动态选择更新位置和协调多种奖励权重。

Details

Motivation: 联合音视频生成要兼顾模态质量、跨模态语义对齐和时序同步，但有效训练依赖两个会变化的因素：在哪些层/位置更新，以及如何平衡相互竞争的奖励。固定路由和权重无法跟踪模型变化。

Result: 实验显示该方法相较强RL基线持续改善音视频模态质量、语义一致性和同步；消融支持更新路由与奖励协调的互补作用。

Insight: 双向跨注意力响应作为跨模态影响的轻量代理，动态重加权token损失并缩放跨模态层梯度；奖励侧保留预设偏好权重作为先验，在预热后用分支级奖励梯度交互作残差校正，避免强奖励压制弱但必要目标。

Abstract: Multi-reward guided reinforcement learning (i.e., RL) offers a promising way to improve joint audio-video diffusion models along several complementary objectives, including modality-specific quality, cross-modal semantic alignment, and temporal synchronization. Its effectiveness, however, depends on two quantities that change during training: where reward-driven updates should act, and how competing rewards should be combined. Existing methods tend to rely on fixed routing and reward weights, failing to track evolving model functions. To address these limitations, we propose Adaptive Reward Routing to jointly adapt update locations and reward coordination during forward-process RL (i.e., DiffusionNFT) of joint audio-video diffusion models. Our method consists of two components. (i) Cross-Modal Influence-Guided Routing (Localizing Updates): We use bidirectional cross-attention responses as an efficient proxy for evolving cross-modal influence, dynamically reweighting token-aware losses and scaling gradients across cross-modal layers without additional model interventions. (ii) Preference-Preserving Modality-Aware Reweighting (Coordinating Rewards): We preserve predefined weights as preference priors and use branch-specific reward-gradient interactions as residual corrections after warm-up. This resolves evolving conflicts without letting dominant rewards suppress weak but essential objectives. Extensive experiments demonstrate consistent improvements in modality quality, semantic consistency, and audio-video synchronization over strong RL baselines. Ablations and mechanism analyses further validate the complementary benefits of adaptive update routing and reward coordination.

[84] Beyond Attention Imbalance: Mitigating Hallucinations via Spectral Surgery cs.CVPDF

Siqi Lu, Suo Wei, Yongbin Zheng, Jianhang Yao, Wanying Xu

TL;DR: FLASH通过频谱涡旋分数定位与幻觉相关的视觉注意力头，并在解码期间做频率局部调制，无需训练或对比解码。

Details

Motivation: LVLM幻觉研究常归因于跨模态注意失衡，方案多重加权视觉token或压制语言先验，忽略视觉信息流的频谱特征；对比解码通常还会使推理成本翻倍。

Result: 在实验中，FLASH较先进方法实现更好的性能—效率平衡；摘要未提供具体准确率数字。

Insight: 识别两类幻觉模式：感知—语义脱节和局部固定，再用训练免、CD免的频率局部注意力整形校正视觉信息流，避免整体验证时延翻倍。

Abstract: While Large Vision-Language Models (LVLMs) achieve remarkable success, hallucinations remain a significant barrier to their reliable deployment. Recent studies primarily attribute these issues to cross-modal attention imbalances; most solutions therefore focus on reweighting visual tokens or suppressing language priors. However, such approaches often overlook the spectral characteristics of the visual information flow and frequently rely on Contrastive Decoding (CD), which doubles inference time. Instead of following conventional approaches, we identify two distinct hallucination patterns-Perceptual-Semantic Dissociation and Localized Fixation-and propose FLASH (Frequency-Localized Attention SHaping), a training-free and CD-free framework. FLASH utilizes a Spectral Vortex Score to detect vision heads within multi-head attention layers and applies adaptive spectral modulation to rectify the visual information flow during decoding. Empirical results demonstrate that FLASH achieves a superior balance between performance and efficiency compared to SOTA methods.

[85] V-JEPA Policy: Building Effective World-Action Models on Predictive Visual Latents cs.CV | cs.AI | cs.ROPDF

Yang Zhang, Jiangyuan Zhao, Chenyou Fan, Jiayu Hu, Xiu Yuan

TL;DR: V-JEPA Policy在冻结V-JEPA 2.1预测式视觉潜空间上从头联合训练未来潜状态预测器和流匹配动作专家，构建WAM。

Details

Motivation: 现有WAM通常从大规模视频生成或编辑模型继承视觉生成先验；本文探问大规模预测式视觉潜表征能否单独成为有效世界动作模型基础。

Result: 9亿总参数、6亿可训练的V-JEPA Policy在LIBERO、LIBERO-Plus和RoboCasa-GR1上与代表性WAM及VLA基线表现竞争力；同预算比较中，V-JEPA潜空间尤其在分布偏移下优于判别式、重建式和视频理解式视觉底座。DROID视频—指令无动作标签预训练也显著改善下游控制和OOD泛化。

Insight: 指令条件未来潜变量预测器与动作专家单阶段联合学习，预测器未来信息KV状态条件化动作生成；更广泛野外视频学到的未来建模知识可迁移到具体任务控制。

Abstract: World-action models (WAMs) couple future visual-state prediction with action generation. By adapting video generators or image-editing models pretrained at scale, a prominent line of recent WAMs inherits both predictive knowledge and the models in which it was learned. We ask whether a predictive visual latent space induced by large-scale predictive pretraining can instead provide a sufficient foundation for effective WAM learning without inheriting a complete pretrained visual generative model. To answer this question, we introduce V-JEPA Policy, a simple framework that builds a WAM on the latent space of a frozen V-JEPA 2.1 encoder. An instruction-conditioned future-latent predictor and a flow-matching action expert are jointly learned from scratch in a single downstream stage, with the predictor’s future-informed context key–value states conditioning action generation. With 0.9B total parameters, of which 0.6B are trainable, V-JEPA Policy achieves competitive performance with representative WAM and vision-language-action baselines across LIBERO, LIBERO-Plus, and RoboCasa-GR1. Comparing visual foundations under the same downstream framework and training budget identifies V-JEPA latents as more effective than the discriminative, reconstructive, and video-understanding-oriented alternatives, particularly under distribution shifts. Beyond task-specific learning, pretraining the predictor on DROID video–instruction pairs without action labels and adapting it into a WAM yields substantial gains in downstream control and out-of-distribution generalization. Together, these findings establish predictive visual latents as a foundation for effective WAM learning from task-specific demonstrations and for transferring future-modeling knowledge acquired from broader in-the-wild videos. Our code is available at https://github.com/breez3young/VJEPA-Policy.

[86] UniAfford: Token-Routed Multitask Learning for Generalizable 2D-3D Affordance Perception cs.CV | cs.AIPDF

Yuhao Liu, Yiming Zhong, Hanqing Wang, Shaocheng Yan, Yuhang Zhang

TL;DR: UniAfford通过token路由将共享MLLM表示送入2D像素和3D点云解码器，统一学习跨模态可泛化的可供性区域感知。

Details

Motivation: 可供性感知定位可供具身交互的物体区域，但2D和3D任务定义、监督、数据及评测割裂，限制跨视觉与几何表征的可迁移语义学习。

Result: UniAfford在2D/3D基准上无需目标域微调即可零样本泛化；按单模态协议评测时各分支达到先进表现。消融支持token路由、联合2D/3D监督和解码器耦合有效。

Insight: 共享MLLM作为语义中心，模态感知token路由产生图像或点云可供性查询，分别条件化SAM式像素解码器和SONATA点解码器；数据集统一像素、点级标注及语言指令，支持图像、点云或配对输入灵活推理。

Abstract: Affordance perception aims to localize actionable regions supporting embodied interaction, yet 2D and 3D affordance grounding have evolved as separate problems, with different task definitions, supervision formats, datasets, and evaluation protocols. This fragmentation limits the learning of transferable object-affordance semantics across visual and geometric spaces. We propose Token Router for Tasks, a multitask training paradigm for MLLM-based systems that routes contextual hidden states to task-specific branches without requiring the language head to generate predefined markers. Routed states are supervised directly by branch-specific objectives, enabling dense prediction losses to shape shared MLLM representations. We instantiate this paradigm as UniAfford, a unified framework for generalizable 2D-3D affordance perception, together with UniAfford-Data, a dataset integrating pixel-level 2D annotations, point-level 3D annotations, and language instructions under a shared object-affordance taxonomy, supporting heterogeneous supervision through semantic-level 2D-3D pairing. UniAfford adopts an MLLM as a shared semantic hub and a modality-aware token router to produce image- and point-cloud-affordance queries. These queries respectively condition a SAM-style pixel decoder and a SONATA-based point decoder, enabling flexible 2D, 3D, and joint affordance inference from image-only, point-cloud-only, or paired multimodal inputs. Experiments demonstrate strong zero-shot generalization across 2D and 3D affordance benchmarks without target-specific fine-tuning, alongside state-of-the-art branch-wise performance under modality-isolated protocols. Ablations validate token routing, joint 2D-3D supervision, and decoder coupling, while language-head diagnostics show that routed latent states carry meaningful object-affordance semantics. Project page: https://4dvlab.github.io/UniAfford

[87] SAM Meets VLM: Parameter-Decoupled Full-Parameter Training for Unified Medical Reasoning and Segmentation cs.CVPDF

Xuyang Cao, Enyou Liu, Jun Zhao, Zhuoyun Liu, Jintao Fei

TL;DR: 本文以分离式参数训练连接医学VLM与SAM分割器，让SEG隐藏态成为语义到空间的专用提示，同时降低像素目标对语言表示的干扰。

Details

Motivation: 医学MLLM既要回答临床问题，也要定位证据；将VLM与SAM通过SEG token相连后做全参数训练，图像级推理与像素级分割对共享表征要求不同，容易冲突。

Result: 在医学指代分割、grounding、视觉问答和文本问答任务上，方法取得强语言条件分割表现并保持竞争力推理能力；消融支持两阶段指令微调、梯度缩放及分割专门化贡献。

Insight: 首先做医学浅层对齐以适配视觉特征与临床语言且不扰动LLM；再控制指令调优形成与通用语言状态可分离的SEG提示，用DBI监控并控制进入语言骨干的分割梯度；最后冻结VLM专门训练SAM分支以提高掩码精度。

Abstract: Medical multimodal large language models (MLLMs) are increasingly expected not only to answer clinical questions, but also to localize the visual evidence behind their predictions. A common strategy connects a vision–language model (VLM) with SAM-style segmentation through a special token, yet full-parameter training of this unified architecture is difficult because image-level reasoning and pixel-level segmentation impose different requirements on the shared representation space. To address this issue, we propose a parameter-decoupled training framework for unified medical reasoning and segmentation. The framework treats the hidden state as a semantic-to-spatial prompt for the mask decoder and encourages it to become separable from generic language states, reducing ambiguous segmentation prompts and potential disruption to reasoning representations. It first performs medical shallow alignment to adapt visual features to clinical language without disturbing the LLM; then controlled instruction tuning shapes separable prompt states, monitored by the Davies–Bouldin Index (DBI), while scaling segmentation gradients entering the language backbone; finally, the SAM branch is specialized with the VLM frozen to improve mask precision without altering reasoning parameters. Experiments on medical referring segmentation, grounding, visual QA, and textual QA benchmarks show that our framework achieves strong language-conditioned segmentation while preserving competitive reasoning ability. Ablations show that two-phase instruction tuning, gradient scaling, and segmentation specialization all contribute to the model.

[88] VISTA-Bench: Benchmarking Multilingual Image Translation with Image-Specific Rubrics cs.CV | cs.AIPDF

Bo Lv, Mao Zheng, Zheng Li, Fangxu Liu, Mingrui Sun

TL;DR: VISTA-Bench用图像专属评估量规测试多模态模型跨22种语言、10个领域把图中文字翻译成目标语言并保留视觉信息。

Details

Motivation: 既有图像翻译基准语言覆盖不足，也缺少针对图像文本内容和背景知识的明确指标，难以全面评估模型能力。

Result: 基准结合语言与场景覆盖抽样、模型辅助并经人工核验的标注，评估16种主流模型（12种多模态、4种纯文本）并分析跨语言、领域和维度表现；摘要未提供结果排名。

Insight: 参考译文按语义单元组织，量规明确必需信息、语义关系及可接受译法，并分别给出翻译质量和视觉/知识相关信息保留分数，从而区分语言转换和图像理解能力。

Abstract: Image translation is a fundamental capability of multimodal models for multilingual applications, requiring visual understanding and meaning preservation across languages. However, existing benchmarks have limited language coverage and often lack explicit image-specific evaluation criteria, making it difficult to comprehensively assess this capability. To systematically evaluate this capability, we introduce VISTA-Bench, covering 22 languages and 10 domains, and develop an image-specific rubric evaluation protocol. The benchmark combines sampling for language and scenario coverage with model-assisted, human-verified annotations that group related text into coherent semantic units and provide multilingual reference translations. The rubrics specify essential content, semantic relations, and acceptable translation variants, yielding separate output-based scores for translation quality and the preservation of visual and knowledge-dependent information. We conduct extensive evaluations of 16 mainstream models, including 12 multimodal models and four text-input models, and provide systematic analyses across languages, domains, and evaluation dimensions.

[89] Technical note on: Zero-Training Feature-Space Alignment via Information Geometry cs.CVPDF

Behraj Khan, Tahir Qasim Syed, Syed Ahmad Chan Bukhari

TL;DR: ZFGA 是一种无需训练的闭式测试时适应方法，通过对齐特征空间中的 Fisher 几何来缓解分布偏移；它在三类视觉模型上总体稳健，但并非每种模型的最优方法。

Details

Motivation: 视觉模型在分布变化下容易退化，常见测试时适应需要迭代优化、超参数调节以及多次前向和反向计算。作者希望在不更新模型参数的条件下，仅通过轻量的特征空间处理改善鲁棒性。

Result: ZFGA 根据预测分布相对于特征嵌入的 Fisher 信息矩阵估计测试特征几何，再用线性变换将其对齐到干净数据参考几何。在 CIFAR-10-C、ImageNet-C 上对 ResNet-50、DINO ViT-S/16 和 CLIP ViT-B/32 的评估中，ZFGA 相比零样本推理均有提升；但 ResNet-50 上协方差白化更好，CLIP 上 Fisher 白化与 ZFGA 在统计上无显著差异。在六种训练自由或梯度式基线中，ZFGA 是唯一没有显著损害任一模型族的方法。几何偏移与 ZFGA 收益正相关（Pearson r=0.366，p=0.017）；推理只需前向计算和矩阵运算。

Insight: 测试时鲁棒性退化可部分从特征几何失配的角度理解，Fisher 几何提供了一种无需更新权重的校正信号。该方法的价值更体现在跨模型的稳定性，而不是在每个模型上都取得最高分。

Abstract: Deep vision models often degrade under distribution shift. Test-time adaptation can improve robustness but typically requires iterative optimization, hyperparameter tuning, and multiple forward-backward passes. We propose Zero-Training Fisher Geometry Alignment (ZFGA), a closed-form method that improves robustness under covariate shift without modifying model parameters. ZFGA is based on the observation that distribution shifts distort feature-space geometry. It estimates the Fisher information matrix of the predictive distribution with respect to feature embeddings and applies a linear transformation that aligns test-feature Fisher geometry with a reference geometry computed from clean data. This provides a natural-gradient-inspired preconditioning step in feature space. We evaluate ZFGA on CIFAR-10-C and ImageNet-C using ResNet-50, DINO ViT-S/16, and CLIP ViT-B/32. ZFGA consistently improves over zero-shot inference across all three models, although it is not the strongest method for every model. Covariance whitening performs better on ResNet-50, while Fisher whitening is statistically indistinguishable from ZFGA on CLIP. Across six training-free and gradient-based alternatives (covariance whitening, Fisher whitening, TENT, T3A, LAME, and AdaNPC), ZFGA is the only method that does not substantially harm any of the three model families. The Fisher geometry distortion is also positively correlated with ZFGA gain (Pearson r = 0.366, p = 0.017), providing preliminary evidence that geometric misalignment contributes to robustness degradation. ZFGA requires only forward passes and matrix operations at inference time, offering a lightweight and deterministic alternative to optimization-based test-time adaptation.

[90] Why Cross-Skeleton Retargeting Is Non-Identifiable: Structural Limits of Generative Motion Models cs.CV | cs.ROPDF

Zhiyuan Li, Wenyan Yang, Pekka Marttinen, Joni Pajarinen

TL;DR: 论文证明，跨骨架动作重定向在常见训练设定下存在结构性不可辨识问题，并提出源实例保真度指标来检验模型是否真正保留了源动作信息。

Details

Motivation: 跨骨架生成的目标是把源动作及其意图迁移到不同身体结构上，但输出表现出正确动作，并不能证明模型使用了源片段，也可能只是生成该动作类别的典型运动。稀疏、异构且配对有限的数据会使这种区别难以从训练结果中识别。

Result: 分析指出，无配对分布匹配存在规范自由度：不同骨架潜空间的相对变换可以不改变训练证据，却导出不同的源条件映射；只按动作类别配对的平方误差训练则可能退化到忽略源片段的目标平均动作。作者提出 SIF，在固定目标骨架和动作类别时检查生成结果之间是否保留源片段所体现的关系。动物动作数据上的若干方法虽能通过动作类别层面的标准测试，却接近源信息盲的基线；表现更好的方法也只保留部分关系信号。摘要未报告统一的数值性能指标。

Insight: 评价动作类别正确性不足以验证重定向是否以源实例为条件。训练目标和评测都必须让源片段之间的关系差异可观测，否则模型可能凭类别先验取得表面成功。

Abstract: Cross-skeleton motion generation trains generative models to carry action structure and motion intention from one body to another. Yet a target motion that shows the right action has two explanations that the training data cannot tell apart: the model transferred the source clip, or it recovered a typical motion for the requested action. We show that this ambiguity is structural rather than incidental: under standard generative objectives, the source-conditioned retargeting map is non-identifiable in sparse heterogeneous motion domains. Unpaired distribution matching yields gauge non-identifiability: the latent spaces of different skeletons can be transformed relative to one another without changing the training evidence, so different source-conditioned maps fit it equally well. Sparse paired supervision admits the complementary failure mode, \emph{conditional-mean degeneration}: when clips are paired only by action, squared-error training converges to an average target motion that ignores the source clip. To make the missing evidence observable, we introduce Source-Instance Fidelity (SIF), a diagnostic that tests whether outputs differ from one another the way their source clips do, with the target skeleton and action held fixed. Under this diagnostic, methods that succeed at the standard action-level test on animal motion data often sit at the source-blind floor, while the methods that rise above it retain only a partial relational signal. Retargeting therefore needs objectives and evaluations that can identify the source-conditioned map it claims to learn. Project page: https://cross-skeleton-retargeting.netlify.app/.

[91] FLASH: A “Generate Once, Synthesize Many” Framework for Synthetic Anomaly Generation in Industrial Anomaly Detection cs.CVPDF

Abhay Kumar Das, Rajesh Gangireddy, Ashwin Vaidya, Samet Akcay

TL;DR: FLASH 将缺陷生成与异常图像合成解耦：先少量生成并缓存缺陷补丁，再通过可控掩码把它们合成到大量正常图像中。

Details

Motivation: 工业异常检测数据中真实缺陷稀少甚至不可得。程序化合成速度快但难以表现复杂缺陷，生成式方法多样性较强，却通常需要为每张样本单独运行昂贵的图像生成模型。

Result: FLASH 使用视觉语言模型引导图像生成模型从正常图像出发创建少量缺陷图像，并抽取、验证和缓存可复用的缺陷补丁。合成阶段用对象边界抑制确定可能的前景区域，以多分辨率频谱金字塔噪声生成尺度可控的缺陷掩码，再将补丁定位、融合到无缺陷图像中。MVTec AD 2 实验中，合成异常达到78.1%的图像级 F1，接近真实异常的83.6%上界；相较逐样本生成式方案，合成速度超过11.95倍，并在不同检测器上的校准迁移最稳定。

Insight: 把高成本的缺陷创作与低成本的样本扩增拆开，可复用少量生成内容，同时通过掩码控制位置和尺寸。评估不只看合成图像是否多样，也考察合成异常对真实缺陷检测校准的迁移效果。

Abstract: Synthetic anomaly generation helps expand industrial anomaly datasets when real defects are scarce or unavailable. Existing approaches lie at two extremes: procedural approaches are fast but struggle to represent complex anomalies, while generative approaches produce diverse defects but require costly per-sample generation. We present FLASH, a framework that decouples defect generation from anomaly synthesis under a ``generate once, synthesize many’’ paradigm. Given only normal images, FLASH uses Vision-Language Model (VLM) guidance and an image-generation model to produce a small set of defect images, from which it extracts, validates, and banks reusable defect patches. For synthesis of anomalous images, Object Boundary Suppression (OBS) first identifies the probable foreground object-aware region of the host image, while Multi-Resolution Spectral Pyramid (MRSP) noise generates diverse, size-controllable masks that determine the defect location and spatial extent. It then composes a large and diverse synthetic anomalous image set by localizing the defect region, sampling size-controllable placement masks and seamlessly blending retrieved defects onto new defect-free images without further need for image generation. Experiments on the MVTec AD 2 dataset show that FLASH-generated anomalies nearly close the calibration gap on real defects, reaching 78.1% image-level F1 against an 83.6% real-anomaly upper bound and providing the most consistent calibration transfer across detectors among procedural and generative alternatives. Moreover, FLASH synthesizes anomalies more than 11.95x faster than per-sample generative approaches.

[92] What Comes Next? Omni-StoryBench for Evaluating Story-Grounded Omnimodal Generation cs.CV | cs.MMPDF

Sieun Hyeon, Yejoon Lee, Mintaek Lim, Woojin Kim, Jaeik Kim

TL;DR: Omni-StoryBench 评测模型能否依据故事上下文和下一页条件，协同生成插图、旁白与角色语音，而非只分别生成看似合理的单模态结果。

Details

Motivation: 全模态生成的质量不能只由文本、图像和语音各自是否自然来衡量；多个模态可能各自合理，却无法表达同一个连贯事件。现有评测因此需要关注故事延续、条件遵循和模态间一致性。

Result: 基准包含900个经过严格验证、来源于开放授权儿童书籍的故事转场，每个样本给出当前页面与下一页结构化条件，要求模型生成下一幅插图、旁白和角色口语，并提供参考页面和语音元数据。作者对32种编排、半编排和原生任意到任意配置进行评估，结合模态专项指标及以一致性为中心的裁判量表。强 VLM 规划的编排式系统整体最可靠；当前原生全模态模型常在输出完整性和可控性上遇到困难。文本侧表现与图像、语音质量相关，而图像生成及视觉连续性是这些配置中最明显的瓶颈。

Insight: 评估全模态系统必须把故事上下文保持、条件遵循、参考一致性和跨模态连贯性纳入同一任务。结果显示，当前短板主要在视觉生成和连续性，单模态文本能力较强并不能自动解决这一点。

Abstract: Omnimodal evaluation should go beyond independent text, image, and speech production: individually plausible outputs may not express a coherent shared event. We introduce Omni-StoryBench, a story-grounded omnimodal benchmark evaluating whether models can coherently continue stories across image, narration, and speech. Each instance provides a current storybook page and structured next-page conditions, requiring models to generate the next illustration, narration, and spoken character utterance. Omni-StoryBench contains 900 rigorously validated story transitions from openly licensed children’s books, with ground-truth next-page references and speech metadata. We evaluate systems with modality-specific metrics and consistency-centered LLM-as-a-judge rubrics for context preservation, condition following, reference consistency, and cross-modal coherence. Across 32 baseline configurations spanning orchestration, semi-orchestration, and native any-to-any paradigms, we find orchestration with strong VLM planning most reliable, while current native omnimodal models often struggle with output completeness and controllability. Our analysis shows text-side performance is associated with image and speech quality, but image generation and visual continuity form the clearest observed bottleneck among the evaluated configurations. These results position Omni-StoryBench as a system-level benchmark measuring coherent omnimodal generation beyond isolated modality quality.

[93] UGO: Unified Architecture for General Multi-Object Tracking by Segmentation cs.CVPDF

Jer Pelhan, Alan Lukezic, Matej Kristan

TL;DR: UGO 将示例条件检测与实例传播整合到一个多目标跟踪架构，并用无训练的能量最小化方法把重叠候选整合为互斥像素掩码。

Details

Motivation: 通用多目标跟踪从首帧的单个类别示例出发跟踪所有同类实例。以边界框和代理训练为主的既有方法在非刚性目标、拥挤场景和干扰物下容易失效，也难兼顾检测与实例分割。

Result: UGO 在统一架构中结合预训练的示例条件检测头与实例传播头；能量最小化整合过程解决过分割、重复候选和冲突，并通过全局及实例层级记忆提升召回率和逐实例分割精度。论文报告其在 GMOT 基准和视频目标计数上达到新的最佳结果，并与专门的多目标跟踪方法具有竞争力；摘要未给出具体数值。

Insight: 跟踪、像素级实例划分和记忆管理可以在同一开放类别框架下协同设计。把候选冲突解决放在无需训练的能量优化步骤中，也使统一架构能处理候选重叠带来的问题。

Abstract: General multi-object tracking (GMOT) tracks all instances of a user-specified category from a single first-frame exemplar. Prior work relies on bounding boxes and surrogate training, and struggles with non-rigid objects, crowded scenes, and distractors. We introduce UGO, a unified GMOT tracker that pairs a pretrained exemplar-conditioned detection head with an instance-propagation head in a common architecture. A novel training-free, energy-minimization consolidation method converts overlapping proposals into exclusive pixel-wise masks and detections, resolving over-segmentation, duplicates, and conflicts. A hierarchical memory spanning global and instance levels improves recall and per-instance segmentation accuracy using a new memory management protocol. UGO sets a new state-of-the-art on GMOT benchmarks and video object counting, and is competitive with specialist MOT methods, establishing a strong paradigm for unified, open-category multi-object tracking.

[94] HyperSAM: A Promptable Foundation Model for Hyperspectral Remote Sensing cs.CVPDF

Li Pang, Xinqiao Wu, Jing Yao, Pedram Ghamisi, Jun Zhou

TL;DR: HyperSAM 将合成高光谱数据、SAM3视觉先验和可提示分割架构结合，目标是构建能泛化到多类遥感任务的高光谱基础模型。

Details

Motivation: 通用高光谱基础模型受两方面限制：高空间分辨率数据与可靠密集标注难以同时获得；许多模型从头训练，未充分利用视觉基础模型已有的几何与交互先验。

Result: 数据侧，HyperSAM 从高分辨率 SpaceNet 多光谱影像通过物理约束丰度迁移生成全光谱高光谱立方体，并用 SAM3 伪掩码提供对象级监督。模型侧冻结 SAM3 RGB 分支，以 RGB ViT 初始化可训练高光谱侧编码器，通过零初始化的 ControlNet 式特征注入连接，并用轻量专家混合掩码细化器处理输出；CromSS式置信度选择用于减弱噪声伪标签影响。实验报告其在分类、异常检测、变化检测、目标检测和机载溢油制图等任务上具有较强泛化；高质量合成数据可比单纯扩大噪声监督更有效。摘要未列具体分数。

Insight: 高光谱建模可以通过复用 RGB 基础模型先验并补充物理生成的合成光谱数据，降低对高质量密集标注的依赖。伪标签置信度筛选是把这类合成监督用于训练时应对噪声的关键组成。

Abstract: Hyperspectral remote sensing provides dense spectral measurements that are indispensable for material-level Earth observation, yet the construction of a general-purpose hyperspectral foundation model remains difficult. Two bottlenecks are especially limiting. First, large hyperspectral corpora rarely provide high spatial resolution together with reliable dense annotations. Second, many hyperspectral models are still trained almost from scratch, so the geometric and interactive priors learned by modern vision foundation models are not fully reused. To alleviate these issues, we \highlight{present} \textbf{HyperSAM}, a promptable hyperspectral foundation model that couples a data-centric hyperspectral synthesis pipeline with a spectral adaptation architecture based on Segment Anything Model 3 (SAM3). On the data side, HyperSAM synthesizes full-spectrum hyperspectral cubes from high-resolution SpaceNet multispectral imagery through a physics-informed abundance-transfer generator, while SAM3-derived pseudo-masks provide object-centric supervision. On the model side, the latest implementation uses a frozen SAM3 RGB image branch, a trainable hyperspectral side encoder initialized from the RGB vision transformer (ViT), ControlNet-style zero-initialized feature injection, and a lightweight mixture-of-experts mask refiner. To enhance training robustness against noisy pseudo-labels, Cross-modal Sample Selection (CromSS)-style confidence selection is incorporated for noisy-label weighting. Extensive experiments show that HyperSAM obtains strong generalization on diverse hyperspectral tasks (e.g., classification, anomaly detection, change detection, target detection, and airborne oil-spill mapping) and that high-quality synthetic hyperspectral data can be more effective than simply scaling noisy hyperspectral supervision.

[95] TAEC: Trajectory-Aware Evidence Coordination for Multi-Step Visual RAG cs.CV | cs.AIPDF

Yalun Wu, Bingzhou Wang, Boyang Wang, Peiying Wang, Shaojie He

TL;DR: TAEC 在多步视觉 RAG 的推理过程中追踪尚未解决的答案需求，并据此协调证据检索、记忆保留和图像细节查看。

Details

Motivation: 多步视觉 RAG 需要反复检索证据、更新中间状态并决定继续查找还是作答，但找到相关证据不等于后续有效利用。冗余来源会挤占上下文，已解决需求或无效搜索留下的内容会持续滞留，视觉材料也可能没有以足够细节再次查看。

Result: TAEC 是免训练框架，把未解决的答案需求保存在共享轨迹状态中，用它指导哪些证据进入上下文、累积记忆保留什么，以及视觉证据应检查到何种细节层级。在 ViDoSeek、SlideVQA 和 MMLongBench-Doc 的统一评测中，TAEC 相比领先的免训练视觉 RAG 基线取得整体最佳表现；在多个专有 VLM 上的平均准确率最高。摘要未提供具体准确率数值。

Insight: 视觉 RAG 的关键不只是检索相关文档，而是让证据随推理状态更新：已满足的需求应释放上下文，仍未解决的需求应驱动后续搜索和更细粒度查看。

Abstract: Multi-step visual retrieval-augmented generation (RAG) answers complex questions by repeatedly retrieving visual evidence, updating an intermediate state, and deciding whether to continue searching or answer. Yet retrieving relevant evidence does not ensure its effective use throughout the reasoning trajectory. As multi-step reasoning progresses, redundant sources occupy context capacity needed for missing evidence, observations tied to resolved requirements or unproductive searches linger in context, and visual sources are revisited with insufficient detail for fine-grained reading. We term this loss of usable evidence over a reasoning trajectory trajectory-level evidence utilization degradation. To address it, we propose Trajectory-Aware Evidence Coordination (TAEC), a training-free framework that coordinates evidence use around unresolved answer requirements. TAEC tracks these requirements in a shared trajectory state to guide which evidence enters the context, how accumulated memory is retained, and at what level of detail visual evidence is examined. Under a unified evaluation protocol on ViDoSeek, SlideVQA, and MMLongBench-Doc, TAEC achieves the best overall performance against leading training-free visual RAG baselines, with the highest average accuracy across multiple proprietary vision-language models. These results demonstrate that aligning evidence with evolving reasoning needs improves evidence use throughout multi-step visual RAG.

[96] When to Retrieve, When to Stay: Uncertainty-Aware Temporal Evidence Allocation for Streaming Video-LLMs cs.CVPDF

Xiang Hu, Jiazuo Yu, Lu Zhang, Yunzhi Zhuge, Huchuan Lu

TL;DR: WRWS 根据检索相关性分布的不确定性，在视频历史的语义检索与近期帧保留之间自适应分配有限视觉预算。

Details

Motivation: 流式视频模型必须遵循因果约束，并在有限视觉处理预算下选择历史证据。只看最近内容会漏掉早先但相关的片段；只按语义相关性检索，则在相似度信号含糊时可能挤掉有用的近期上下文。

Result: WRWS 使用轻量外部视觉语言编码器为已观察历史打分，再以相似度分布的归一化熵估计检索不确定度：相关线索可靠时偏向语义检索，不确定时加强近期先验。采用“先检索、后编码”，只将选中的观察交给耗费较高的目标 Video-LLM 处理。在四类 Video-LLM 及多种模型规模上的 StreamingBench 和 OVO-Bench 实验中，准确率具有竞争力；效率评估中，平均视觉到答案延迟降至当前先进方法的47.93%。

Insight: 历史证据选择需要同时考虑问题相关性与检索置信度，熵可用于调节二者权重。把筛选放在目标模型视觉编码之前，能够避免对未选帧支付主要处理成本。

Abstract: Streaming video understanding requires Video Large Language Models (Video-LLMs) to reason over continuous visual streams under causal constraints. As the visual history grows, a bounded visual?processing budget requires evidence selection that balances temporal recency with query relevance. Recent-only selection excludes potentially relevant historical evidence, whereas Semantic-only retrieval can displace useful recent context when relevance scores are ambiguous. We introduce WRWS (When to Retrieve, When to Stay), a training-free framework for uncertainty-adaptive evidence allocation. A lightweight external vision-language encoder scores query relevance across the observed history, while an adaptive allocation module uses the normalized entropy of the similarity distribution as a proxy for retrieval uncertainty. WRWS favors semantic retrieval when relevance cues are reliable and strengthens the recency prior under uncertainty. Following a retrieve-first, encode-later pipeline, WRWS selects evidence before target-model visual encoding, such that only the selected observations are processed by the costly target Video-LLM. Experiments across four Video-LLM families and multiple model scales demonstrate competitive accuracy on StreamingBench and OVO-Bench. In our efficiency evaluation, WRWS reduces average vision-to-answer time to 47.93% of the state-of-the-art method. Code will be released.

[97] Think Before You Score: Thinking Reward Model for Visual Generation cs.CVPDF

Xuehai Bai, Zhenchen Tang, Yang Shi, Dianyi Wang, Tengfei Liu

TL;DR: TRM先为每个视觉生成案例生成适配的评分准则，再按准则给出细粒度奖励；结合PD-GRPO后，可作为奖励模型提升多类图像生成与编辑模型。

Details

Motivation: 现有视觉奖励模型通常直接把任务条件与候选结果映射为一个标量，未明确每个案例具体应评估哪些属性。作者提出先判断该案例的评价重点，再衡量候选结果的表现。

Result: Thinking Reward Model（TRM）为每个案例制定自适应评分量表，按量表评估并输出细粒度逐项奖励。作者还指出常见成对偏好优化会造成分数极化，因此提出 Pairwise Dual-Group Relative Policy Optimization（PD-GRPO），利用成对监督提升区分力，同时保留细粒度逐项评分。在图像生成和编辑奖励建模基准上，TRM超过开源奖励模型的既有表现，并与专有方法有竞争力；将 TRM 用于强化学习奖励时，多种视觉生成模型均获得持续改进。摘要未给具体分数。

Insight: 案例自适应的明确评分标准可让奖励模型说明“本例什么最重要”，从而减少单一标量忽略任务差异的问题。保留细粒度分数并加入成对偏好监督，有助于兼顾奖励辨别能力和可解释的逐项反馈。

Abstract: Visual reward models are essential for evaluating and improving visual generation models, yet existing approaches typically map task conditions and candidate outputs directly to scalar rewards, leaving implicit what should be evaluated for each individual case. We introduce Think Before You Score, a paradigm that explicitly determines what matters for each case before judging how well the candidate performs. Following this principle, we propose the Thinking Reward Model (TRM), which formulates case-adaptive rubrics, performs rubric-guided assessment, and produces fine-grained pointwise rewards. We further observe that conventional pairwise preference optimization can induce score polarization, and introduce Pairwise Dual-Group Relative Policy Optimization (PD-GRPO), which leverages pairwise supervision to improve reward discrimination while preserving fine-grained pointwise scoring. Extensive experiments on image generation and editing reward-modeling benchmarks demonstrate that TRM achieves state-of-the-art performance among open-source reward models while remaining highly competitive with proprietary alternatives. Moreover, using TRM as a reward for reinforcement learning consistently improves diverse visual generation models, demonstrating that its fine-grained, case-adaptive rewards translate into effective optimization signals for visual generation.

[98] Do-JEPA: From Masking to Intervention in Latent World Models cs.CV | cs.AI | cs.LGPDF

Hossein Resani, Javen Qinfeng Shi

TL;DR: Do-JEPA通过对同一模拟器状态施加实际动作与参考动作，并预测两种未来潜表示的差异，直接监督世界模型学习动作造成的因果效应。

Details

Motivation: 潜在世界模型通常只训练预测下一状态，其目标无法区分动作真正造成的结果与仅仅同时出现的变化。仅遮挡对象来改变模型能看到什么，并没有直接干预世界中实际发生的动作。

Result: Do-JEPA从同一保存状态分别运行动作a与参考动作a∅，预测潜在未来差Δz；目标包括效应、作用支撑位置、效应传播路径和不变性损失。在对象对齐的合成系统中，支撑监督有99.95%的测试例定位到直接干预对象，而稀疏动作掩码每次都把动作分配到干扰槽位；响应起始监督恢复环形传播图，边AUROC为0.975，对照为0.624。从像素训练时，在同数据控制组之上，端到端 LeWM 的潜在效应误差降低28.4%，自然动作序列上的物理效应误差降低13.5%；三个独立 CausalWorld 基准在物理变化下响应效应误差约降低20%，预测效应的潜在上下文敏感性降低66%。从零训练会牺牲事实预测准确率，对现有模型微调则可消除该代价。

Insight: 对比同一初始状态下不同动作引起的未来差异，为因果效应提供了直接监督，能够分离动作影响、传播范围和不应变化的因素。该目标适合微调既有世界模型；从头训练时需要处理事实预测精度下降的权衡。

Abstract: Latent world models are trained to predict what happens next, so nothing in their objective separates what an action caused from what merely co-occurred with it. Object-masking models such as C-JEPA intervene on what the predictor can see; we intervene on what physically happens. From one saved simulator state we run the dynamics under an action $a$ and under a reference action $a_{\varnothing}$, and train the model to predict the difference $Δz=z^{a}-z^{a_{\varnothing}}$ between the two latent futures. The resulting objective, Do-JEPA, has an effect loss, a support loss (where the action enters), a propagation loss (where its effect travels) and invariance losses (what must not change). In a synthetic system with object-aligned variables, support supervision finds the directly intervened object in 99.95% of test cases, where a sparse action mask sends the action to a nuisance slot in every case, and response-onset supervision recovers the ring-shaped propagation graph (edge AUROC 0.975 vs. 0.624). From pixels, the effect loss beats a control trained on exactly the same data: it lowers latent effect error by 28.4% on an end-to-end LeWM model and physical effect error by 13.5% when trained and tested on natural action sequences, and on three independently generated CausalWorld benchmarks it lowers responsive effect error by about 20% under physics shifts and the latent context sensitivity of predicted effects by 66%. Trained from scratch it costs factual accuracy; fine-tuning an existing model with it removes this cost. Together, these results show that intervening on the world, rather than on what the model sees, helps latent world models predict what their actions cause.

[99] MG-Thinker: Bi-Axial Self-Reflection for Multi-Image Reasoning Grounding cs.CV | cs.MMPDF

Heyu Huang, Chi Chen, Zonghao Guo, Yuhua Li, Maosong Sun

TL;DR: MG-Thinker通过分层的多图像推理定位范式和双轴优势估计，针对任务与样本难度不均的问题训练像素级视觉定位模型。

Details

Motivation: 强化学习已提升多模态推理，但面向真实多图像上下文、最终要求像素精确定位的任务，已有方法较少利用由粗到细的层级推理，也难适应任务及样本之间差异很大的难度。

Result: MG-Thinker 是后训练强化学习框架，配套构建25K多图像推理定位数据集，并用任务自适应思维链标注引导模型先从多个角度收集证据再下结论。针对难度异质性，Bi-Axial DAPO 在候选池提供的稳定组级统计基础上，将 rollout 优势分解为组内信号轴和组间能力轴，并使用两个互补机制进行优化。实验报告 MG-Thinker 在多图像推理定位上达到当前最佳，并持续提升多图像理解及多种多模态基准上的泛化；摘要未提供具体分数。

Insight: 像素级定位需要先组织多图之间的证据，再逐步收窄到局部区域；训练时还需区分单个任务组内部的相对信号与不同组之间的能力差距。两轴优化旨在同时处理这种推理结构和难度差异。

Abstract: Reinforcement learning (RL) has recently delivered substantial gains in multimodal reasoning, opening a promising route for fine-grained visual perception. Yet for multi-image reasoning grounding (MRG), reasoning over real-world multi-image contexts toward pixel-precise localization, existing RL-based approaches overlook two characteristics intrinsic to this paradigm: a coarse-to-fine hierarchical reasoning pattern, and heterogeneously distributed task–sample difficulties. In this work, we present MG-Thinker, a post-training RL framework that advances a new MRG paradigm featuring such hierarchical reasoning, supported by a curated 25K MRG dataset with task-adaptive Chain-of-Thought (CoT) annotations that elicit multi-perspective evidence before conclusion. To remedy the heterogeneous task–sample difficulties, we further propose Bi-Axial DAPO (BiA-DAPO), which decomposes rollout advantages along an intra-group signal axis and an inter-group competence axis through two complementary mechanisms, both grounded on our defined candidate pool for stable group-level statistics. Extensive experiments show that MG-Thinker achieves state-of-the-art performance on multi-image reasoning grounding while consistently improving generalization across multi-image understanding and diverse multimodal benchmarks.

[100] Complementary Retrieval-Augmented Prompting for Consistent Long-Form Video Generation cs.CVPDF

Xianghan Wei, Xiaoda Yang, Zhi Wang, An Pan, Daoan Zhang

TL;DR: 互补检索增强提示框架为长视频生成中的每个目标镜头解析所需角色、物体和场景，再检索彼此互补且覆盖需求的历史参考帧，以增强跨镜头一致性。

Details

Motivation: 视频生成模型虽能生成高质量短片，长篇故事仍难保持不同镜头中的人物、场景和物体一致。训练自由的历史画面检索方法可能引入无关冗余，也可能漏掉目标镜头所需元素的完整组合。

Result: 框架把叙事脚本解析为文本对齐的视觉元素清单，追踪角色、物体、场景及其镜头级状态；再用 VLM 标注关键帧库，将元素映射到过去的视觉观测。检索智能体在覆盖目标元素的同时减少历史噪声，把互补参考、结构化状态和视觉定位指令组合成统一提示，供冻结的视频生成器使用。在多镜头故事生成的定量和定性评估中，该方法在跨镜头一致性和文本可控性上持续超过近期帧、记忆式及实体级检索基线；摘要未提供具体分数。

Insight: 历史参考应按目标镜头缺少哪些元素来组合，而不是只取最近帧或最相似的单一片段。显式记录元素及镜头状态，使检索既补全条件又可解释。

Abstract: While recent video foundation models excel at generating high-quality short videos, long-form video generation remains a critical challenge, where a major bottleneck lies in conditioning independently generated shots to preserve consistent characters, scenes, and objects throughout a story. Existing training-free approaches typically condition target shots using retrieved historical visuals. However, these references often suffer from severe informational mismatch, either introducing irrelevant contextual redundancy or failing to provide the full combination of required elements for the target shot. To resolve this, we present Complementary Retrieval-Augmented Prompting, an agentic framework that strategically aggregates a compact set of mutually supportive historical references to achieve complete and targeted conditioning for long-form video generation without retraining or modifying the underlying generator. Specifically, our framework explicitly models the visual elements required by each target shot by parsing the narrative script into a text-grounded visual element registry that tracks characters, objects, scenes, and their shot-level states. A VLM-annotated keyframe library further maps these elements to past visual observations. Guided by the required elements, our agent retrieves complementary references that maximize target-element coverage while minimizing historical noise. Finally, the retrieved references, structured element states, and grounding instructions are assembled into a unified prompt for the frozen video generator. This element-aware process provides comprehensive conditioning while remaining fully interpretable. Quantitative and qualitative evaluations on multi-shot story generation demonstrate that our method consistently outperforms recent-frame, memory-based, and entity-level retrieval baselines in cross-shot consistency and text-controllability.

[101] LazySloth: Bounded LLM-based Lazy Tree Search for Fast Long Video Comprehension cs.CV | cs.AIPDF

Arka Mukherjee, Kaleen Shrestha, Larissa Zhu, Maja Matarić

TL;DR: LazySloth用有界的树搜索让视频语言模型先粗略概述可能无关的视频片段，再深入检查候选区域，以更低字幕成本完成长视频理解和检索。

Details

Motivation: 逐帧或密集抽帧描述长视频效率低且需要很长上下文；多模态RAG虽更省算力，但有损嵌入会丢掉时间关系和细节。作者进一步研究如何优化由VLM驱动、与问题相关的视频证据搜索。

Result: LazySloth在VLM判断不相关的视频区间上进行有界字幕化，并用树结构组织搜索；相较现有智能体方法，视频理解和检索速度提高2.9至8.3倍。Gemma 4 31B和Qwen3.6 27B在四个基准上的最终任务准确率与专用视频VLM或RAG方法相当或更好，并缩小开源基础模型与GPT-4o的差距。以CLIP检索替代VLM场景理解会损失8.8%至19.9%的准确率；惰性建树能以更低字幕成本达到与预先完整建树相当的表现。

Insight: 对长视频的高效检索可以采用惰性层级搜索：先用较粗的信息排除区段，只对潜在相关区域增加分析。保留VLM场景理解有助于避免单靠嵌入检索造成的时间和细节信息损失。

Abstract: Modern vision-language models (VLMs) have shown promising results in long-video understanding due to the rich semantic information they can capture. However, most methods focus on coarse captioning of extracted image frames that are computationally inefficient and require models with large context windows. While past work has explored efficient methods through multimodal retrieval-augmented generation (RAG), they rely on lossy embeddings that lose temporal context and fine-grained detail. Few works to date have investigated how VLM-based query-relevant information retrieval can be optimized. We introduce LazySloth, an efficient tree-based search method that speeds up video comprehension and retrieval tasks 2.9-8.3x (compared to existing agentic methods) through bounded captioning of portions of the video considered irrelevant by a VLM of the video. Compared to contemporary specialized video-understanding VLMs and RAG-based methods, LazySloth achieved similar or better final task accuracy across two recent open-source base VLMs–Gemma 4 31B and Qwen3.6 27B–across four benchmarks. LazySloth reduced the gap between the base open-source model and a closed-source model, GPT-4o. Ablations showed that replacing VLM scene understanding with CLIP-based retrieval cost 8.8-19.9% in accuracy, while lazy tree construction matches eager construction at a fraction of the captioning cost. With LazySloth, we demonstrate the possibility of faster long-video comprehension without substantial loss in performance.

[102] Label Less, Learn More: Resource-Efficient Active Semi-Supervised Learning for Onboard Satellite Image Annotation cs.CVPDF

Ahmed Abdelnaby, Mohamed Elmahallawy, Marius Bernahrndt, Tobias Hecking

TL;DR: SatLabel通过主动挑选标注样本并利用大量未标注卫星影像，逐步训练适合星载部署的小型模型，以降低标注、推理和能耗成本。

Details

Motivation: 高分辨率卫星感知需要任务专属视觉模型，但星载设备受到算力、内存、能量和通信限制，收集标注也成本高。仅依赖监督学习会耗费大量标注，直接部署大型视觉语言基础模型又不利于资源受限的星载适配。

Result: SatLabel闭环利用模型不确定性、类别不平衡和伪标签质量来选择信息量高的样本，并以半监督方式适配紧凑学生模型；还提供可选的专家混合学生和图结构特征细化模块。在11个遥感数据集上对比 RemoteCLIP 零样本推理，SatLabel在大多数核心与扩展数据集提高Macro-F1，并保持对未见领域的跨数据集迁移。Balanced学生有11.2M参数、约42.8MB，RemoteCLIP为151.3M参数、577MB；计算量分别为3.65和5.89 GFLOPs。四项效率测试中，Balanced约有两倍GPU前向吞吐，并降低每张图像能耗。

Insight: 星载适配可通过主动学习减少人工标注，并将未标注影像转为伪监督信号；样本选择、类别平衡和伪标签置信度需要联合考虑。紧凑学生模型还能显著降低部署占用。

Abstract: Large-scale pervasive sensing increasingly relies on high-resolution satellite imagery, yet task-specific onboard vision is constrained by costly annotation and limited computation, memory, energy, and communication resources. Existing approaches largely rely on either data-hungry supervised learning or large vision-language foundation models, limiting efficient adaptation and deployment under these constraints. We present SatLabel, a resource-aware learning framework that transforms limited satellite labels into progressively refined onboard models through adaptive sample acquisition and semi-supervised model adaptation. Rather than repeatedly training on uniformly sampled labels, SatLabel closes the loop between model uncertainty, class imbalance, and pseudo-label quality to selectively acquire informative samples while exploiting abundant unlabeled imagery. This enables a compact student to adapt to target sensing domains with reduced annotation and inference costs. We further introduce an optional Mixture-of-Experts (MoE) student with graph-based feature refinement to enhance representation capacity while retaining a lightweight footprint. We evaluate SatLabel on 11 remote-sensing datasets spanning core, extended, and unseen domains against RemoteCLIP zero-shot inference. SatLabel improves Macro-F1 on most core and extended datasets while maintaining strong cross-dataset transfer to unseen domains. More importantly, the Balanced student contains only 11.2 M parameters and occupies approximately 42.8 MB, compared with 151.3M parameters and 577 MB for RemoteCLIP, while requiring 3.65 versus 5.89 GFLOPs. Across four efficiency benchmarks, it achieves approximately 2x higher GPU-forward throughput and reduces energy per image on datasets.

[103] PoE-Fuse: Precision-Weighted Expert Fusion for Bi-Temporal Change Understanding cs.CV | cs.LGPDF

Haruki Watase, Shunya Nagashima, Takayuki Nishimura

TL;DR: PoE-Fuse冻结几何、定位和语言专家，只训练轻量融合主干，并以每个空间位置的精度权重整合两时相特征，联合完成变化检测、建筑定位和灾害损伤评估。

Details

Motivation: 双时相变化理解需定位并描述两幅卫星图之间发生的变化，服务于灾害响应和环境监测。强视觉语言模型通常需要全量微调或强化学习适配，训练成本高且不稳定。

Result: PoE-Fuse将冻结专家的特征重采样到共享空间网格，把各网格特征视为潜在场景状态的高斯观测，并通过学习得到的逐格精度进行乘积专家融合；该估计器可推广均匀求和和标量门控，并通过组合两时相精度处理变化场。一个共享主干同时处理三项任务，平均F1达到59.2%；指令微调的时序视觉语言助手为40.7%，且超过按相同协议和训练预算重训的专用变化检测模型。

Insight: 逐空间位置建模专家置信精度，能在融合时体现不同视觉专家对场景各处的可靠程度。共享主干联合处理检测、定位和损伤评估，显示参数高效的专家组合可以支持多任务变化理解。

Abstract: Bi-temporal change understanding, which localizes and characterizes what changed between two satellite images, is central to disaster response and environmental monitoring, spanning change detection, building localization, and damage assessment. Strong vision-language models address these tasks, but adapting them typically requires full fine-tuning or reinforcement learning, which is costly and unstable. We propose PoE-Fuse, a parameter-efficient framework that instead composes frozen foundation experts for geometry, grounding, and language, resampling their features onto a shared spatial grid and training only a lightweight fusion trunk. PoE-Fuse treats the aligned features as Gaussian observations of a latent scene state and fuses them by learned per-cell precision. This product-of-experts estimator strictly generalizes uniform summation and scalar gating, and extends to change fields by composing the precisions of the two timestamps. A single shared trunk solves the three tasks at once, reaching a mean F1 of 59.2%, compared with 40.7% for an instruction-tuned temporal vision-language assistant, and surpassing dedicated change-detection models retrained under the same protocol and training budget.

[104] FORUM: Frozen Outputs Reconciled Using Model Agreement for Visual Grounding cs.CV | cs.CL | cs.LGPDF

Taiyo Sato, Takamasa Sanda, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama

TL;DR: FORUM在测试时融合多个冻结多模态模型的视觉指代框，利用跨模型一致性处理同类干扰物和否定表达，不需训练或改动底层模型。

Details

Motivation: 大型多模态模型用一次提示即可处理常见指代表达，但在含同类别干扰物和否定描述的对抗基准上也会自信地选错，重复采样通常仍重复错误。不同数据和架构的模型较少被同一个干扰因素误导，因此它们的共识可作为无标签信号。

Result: FORUM使用两条固定几何规则融合模型输出：按支持某区域的不同模型数量选择候选，并以 medoid 定位返回一个实际成员框，避免对坐标求平均时被单个宽松框拉偏。融合三个开源多模态模型后，在对抗 Ref-Adv-s 上的平均准确率相对提升5%，超过已发表的397B参数参考模型；比简单平均集成高15%。收益也迁移到标准 RefCOCO+；即使模型组合中没有明显主导成员，也比397B参考模型高5%。

Insight: 当模型架构和训练数据不同，跨模型一致性可以提供互补的错误线索。使用真实成员框的 medoid 比坐标平均更稳健，尤其适用于候选框尺度差异较大的视觉定位。

Abstract: Frozen multimodal large language models (MLLMs) now solve standard referring expression comprehension with a single prompted call, yet on adversarial benchmarks with same-category distractors and negation, even the largest models are confidently wrong, and resampling repeats the error. Models built from different data and architectures rarely fall for the same confounder, so their agreement is a strong label-free signal of the correct target. We present FORUM, a training-free test-time fusion of frozen MLLMs guided by two fixed geometric rules: agreement-based selection keeps the region supported by the most distinct models, and medoid localization returns an actual member box instead of a coordinate average, so one loose prediction cannot shift the answer. Fusing three open MLLMs, FORUM surpasses the 397B-parameter published reference by a relative 5% in mean accuracy on the adversarial Ref-Adv-s benchmark, and a plain averaging ensemble by 15%. The gains transfer to standard RefCOCO+, and a balanced lineup with no dominant member still surpasses the 397B model by 5%.

[105] APM-Bench: Benchmarking Cross-session Persistent Memory for Egocentric Streaming Video Assistants cs.CVPDF

Jianguo Huang, Jinming Liu, Qiyao Wang, Liang Xu, Jianhang Li

TL;DR: APM-Bench将流式视频助手的长期记忆评测扩展到跨会话生活轨迹，检验系统能否在中断后记住过往、主动帮助，并识别证据缺失。

Details

Motivation: 现实中的人机交互往往断续发生，记忆需要跨多个会话保留。既有流式基准主要关注单段连续视频或短片，较少评测长期存储、选择性保留、合适时机调用、资源开销及证据不足时的应答。

Result: APM-Bench包含549个会话、104条生活轨迹和2,719个候选问题，涵盖客观及开放式问题；每个会话是一段带细粒度标注的视频，同一轨迹内会话围绕相关活动。评测要求模型跨会话回答过去事件并在实时交互中主动响应，也测试模型能否承认记忆中缺少必要证据。对通用视频模型、不同记忆协议和专用流式记忆系统的系统评估显示，现有方法难以同时实现可靠长期回忆、低开销和有效主动帮助，呈现明显的效用—延迟—存储权衡。

Insight: 长期视频记忆不能只以能否答对历史问题衡量，还需要考虑存储限制、延迟和主动响应，并在证据不足时避免臆答。跨会话生活轨迹更接近真实助手持续交互的使用条件。

Abstract: To serve as real-world personal assistants, streaming video models need persistent memory that retains past experiences for later use. Yet existing streaming benchmarks and methods often focus on individual continuous videos or short clips, overlooking that real-world interactions are often intermittent and require memory to persist across interruptions. To fill this gap, we introduce APM-Bench, which reformulates real-world streaming interaction as multi-session life trajectories. It contains 549 sessions, 104 trajectories, and 2,719 candidates, spanning both objective and open-ended questions. Each session is a video with fine-grained annotations, and sessions within a trajectory revolve around related activities. Models then use persistent memory to answer questions about past sessions and provide proactive responses while maintaining real-time interaction. This raises challenges: persistent memory must be storable, selectively retain information, be injected at the right time, and remain efficient. Moreover, finite storage may leave required evidence unavailable, so assistants should recognize missing evidence. Therefore, we systematically evaluate general video models under different memory protocols and diverse specialized streaming memory systems, and test whether models acknowledge insufficient evidence. Our evaluation reveals a clear utility–latency–storage trade-off: existing methods still struggle to simultaneously achieve reliable long-term recall, low overhead, and effective proactive assistance across sessions. APM-Bench provides a comprehensive testbed for developing and comparing persistent memory systems under realistic streaming conditions. We hope it encourages future work that jointly considers utility, latency, and storage toward more practical persistent memory for real-world streaming assistants.

[106] Decompose Radicals, Then Reward: Fine-Grained Inspection for Accurate Chinese Text Rendering cs.CVPDF

Yazhen Xie, Xingsong Ye, Zhineng Chen

TL;DR: IDSpect将汉字拆解为部件和空间关系进行细粒度视觉奖励评估，帮助文本生成图像模型更准确地呈现中文字形内部结构。

Details

Motivation: 现有基于OCR的强化学习奖励比较识别出的整字文本与目标字符串，把汉字当作不可拆分的整体，无法区分偏旁部件或空间关系上的不同错误，反馈过于粗糙，可能鼓励只像目标而结构不正确的字形。

Result: 作者训练专家级意象描述序列（IDS）识别器，将生成图像中的中文转写为由字符部件和空间运算符组成的表示。IDSpect将目标文字确定性分解为IDS token，再把图像裁剪区域的IDS预测与目标序列对齐；全局唯一token记分使比较不依赖文本区域检测顺序。它与整字语义奖励结合，不改图像生成器，也不增加推理时成本。在Qwen-Image上进行GRPO后训练时，IDSpect在LongText和GenTextEval上取得领先的结构质量与语义对齐结果。

Insight: 汉字渲染反馈应同时检查组成部件和它们的空间组织，单纯整字OCR匹配不足以定位结构性错误。把细粒度结构奖励与整字语义奖励结合，可在不改生成器的前提下提供更有针对性的训练信号。

Abstract: Rendering accurate Chinese text remains challenging for text-to-image models. Existing OCR-based reinforcement-learning rewards compare decoded transcripts with target strings. Such rewards overlook the compositional nature of Chinese writing: an ideograph consists of reusable components arranged through explicit spatial relations, yet OCR evaluates it as an atomic character. Consequently, visually different radical-level errors may receive equally coarse feedback, encouraging glyphs that merely resemble the target instead of faithfully reproducing its internal structure. We employ Ideographic Description Sequences (IDS), which comprise spatial operators and character components, and train an expert IDS recognizer to transcribe rendered Chinese text into this representation. Building on this recognizer, we introduce IDSpect, which deterministically decomposes the target text into IDS tokens and aligns crop-level visual IDS predictions with the target sequence. Globally unique token credit makes this comparison robust to the order of detected text regions. Combined with a whole-character semantic reward, IDSpect supplies fine-grained credit with component and spatial-relation without changing the image generator or adding inference-time cost. Experiments with GRPO post-training of Qwen-Image demonstrate that IDSpect achieves leading structural quality and semantic alignment on LongText and GenTextEval.

[107] Evaluating the Evaluators: Diagnosing Large Multimodal Models for AI-Generated Image Assessment cs.CVPDF

Yu Zhao, Jiarui Wang, Huiyu Duan, Ye Zhao, Jutao Tang

TL;DR: SQUARE-Bench以语义、质量、真实性和责任四个方面评估多模态模型作为AI生成图像评估器的能力，并显示模型间细粒度判断能力差异明显。

Details

Motivation: 传统图像生成指标难以捕捉细粒度提示对齐和生成伪影，多模态模型因此越来越常被用作评估器。但既有基准多分别考察语义理解、质量感知和真实性识别，较少评估责任判断，缺少统一、全面的诊断。

Result: 基准设置38个细分维度，评估来自22种生成模型的近1万张AI生成图像及超过3,000张真实图像，并配有人工整理的问答标注。对23个多模态模型的实验显示，顶级专有模型（如 Gemini-3-Pro）已超过单个专家评估者基线，但不同模型间差距仍大，在细粒度推断和领域鲁棒性方面尤其如此。概念验证还让不同维度的模型为固定图像编辑器提供迭代诊断，语义、真实性和责任维度有所改善，但视觉质量呈现一致的权衡。

Insight: 图像评估器既要判断内容对齐和视觉质量，也要覆盖真实性与责任相关问题；单一总分会掩盖不同维度的能力短板。用维度级反馈指导编辑虽能选择性改善结果，也可能损害整体观感质量。

Abstract: With the rapid advancement of text-to-image (T2I) generation, robust evaluation becomes critical yet challenging, as traditional metrics fail to capture fine-grained alignment and generative artifacts. While large multimodal models (LMMs) are increasingly adopted as evaluators, existing benchmarks typically study semantic understanding, quality perception, and authenticity identification in isolation, while largely neglecting responsibility detection. This leaves a gap in unified and comprehensive validation. To bridge this gap, we introduce SQUARE-Bench, a comprehensive benchmark that systematically evaluates LMM capabilities as evaluators of AI-generated images across four aspects: Semantics, Quality, Authenticity, and Responsibility. SQUARE-Bench introduces a granular taxonomy of 38 sub-dimensions to evaluate nearly 10K AI-generated images sampled from 22 diverse models, ranging from legacy to state-of-the-art generators, complemented by over 3K real-world images. The images are annotated with curated question-answering pairs. Extensive experiments on 23 LMMs reveal that top proprietary models, such as Gemini-3-Pro, already outperform the individual human expert baseline. However, the performance gap between models remains significant, exhibiting notable disparities in fine-grained inference and domain-specific robustness. Beyond benchmarking, we conduct a proof-of-concept study of LMM-guided iterative editing, in which dimension-specific LMMs provide diagnostic feedback to fixed image editors. The resulting guided system yields selective improvements in semantics, authenticity, and responsibility, while exhibiting a consistent visual-quality trade-off. SQUARE-Bench can serve as both a diagnostic tool for characterizing LMM evaluator capabilities and studying their use in T2I generation refinement. The benchmark and dataset will be released upon publication.

[108] TReVS: Integrating Textual Relevance and Visual Saliency for Efficient Vision-Language Model Token Pruning cs.CV | cs.AIPDF

Jing Wang, Zhiping Wu, Dongdong Ren, Youfang Han, Wei Zhao

TL;DR: TReVS把文本查询相关性与视觉显著性结合用于视觉token剪枝，并利用对文本更敏感的高方差注意力头在语言模型浅层继续筛除无关token。

Details

Motivation: 视觉语言模型的大量图像token带来较高推理成本。两阶段剪枝通常先在视觉编码器后依据视觉显著性去冗余，再由语言模型按查询删去无关token；第一阶段若不考虑文本查询，可能提前删掉后续无法恢复的关键证据。

Result: 实验发现，在第一阶段加入查询引导能比仅用视觉显著性更好地保留任务相关证据；高方差注意力头对文本查询更敏感，能提供更有区分力的文本到视觉注意信号。TReVS是免训练框架，在进入语言模型前联合文本相关性与视觉显著性剪枝，并在语言模型浅至中间层利用这些注意力头筛除无关token。在LLaVA-1.5-7B上，剪除94.4%的视觉token后保留未剪枝基线92.8%的性能，超过此前先进方法。

Insight: 视觉token压缩的首阶段就应知道用户问什么，否则信息可能在文本引导阶段之前被不可逆地丢弃。利用查询敏感注意力头能将相关性判断继续传到语言模型内部。

Abstract: Vision-Language Models (VLMs) excel at visual understanding and reasoning but often incur substantial inference costs due to the large number of visual tokens. Recent visual token pruning methods increasingly follow a two-stage paradigm: they first remove visually redundant tokens after the vision encoder and then discard tokens irrelevant to the textual query within the Large Language Model (LLM). However, since the first stage typically relies solely on vision-encoder saliency, it may prematurely eliminate query-relevant tokens, depriving the subsequent text-guided stage of critical visual evidence. Our empirical analysis shows that incorporating query guidance into first-stage pruning better preserves task-relevant evidence and consistently improves performance over vision-only saliency-based pruning. We further find that high-variance attention heads are more sensitive to the textual query and yield more discriminative text-to-vision attention signals for second-stage pruning. Motivated by these findings, we propose TReVS, a training-free framework that combines textual relevance with vision-encoder saliency for pre-LLM pruning and leverages high-variance attention heads to remove task-irrelevant tokens at shallow-to-intermediate layers of the LLM. On LLaVA-1.5-7B, TReVS retains 92.8% of the unpruned baseline performance while pruning 94.4% of visual tokens, outperforming prior state-of-the-art methods.

[109] FedSocket: Recipient-Executable Knowledge Exchange for Heterogeneous Multimodal Federated Learning cs.CV | cs.AIPDF

Xinyuan Zhao

TL;DR: FedSocket让联邦学习交换的知识可由异构接收方直接执行，通过共享预测接口连接不同模态、私有架构和任务的本地模型。

Details

Motivation: 联邦知识在不同参与方拥有不同输入模态、私有模型架构和任务时，必须保证接收方能实际使用。传统共享结果可能难以直接服务于异构接收方的本地学习和推理。

Result: FedSocket设计共享Q模块，结合接收方可计算的输入、任务所属输出和考虑所有权的聚合；本地私有模型监督各自的Q副本，返回的Q支持本地学习及联合推理，交换内容仅含Q参数和计数。在六个数据集上，缺失模态接收方相对本地模型在MELD和UCF-51准确率分别提高14.44和15.51个百分点。推理能力匹配时，联合模型相较独立集成在UCF-51准确率高11.06个百分点，在Flickr30k双向R@1均值高4.87个百分点；联合模型在四个异构端点上也都优于仅用Q。控制实验分别检查监督、知识共享路径和部署的作用。

Insight: 知识交换的接口应从训练阶段就考虑接收方能否执行，而非只关注服务器端聚合。把本地预测与交换来的Q联合使用，能利用本地专长及跨客户端知识的互补性。

Abstract: Federated knowledge must remain usable by recipients with different modalities, private architectures, and tasks. We present FedSocket, which makes recipient execution a design requirement of the exchanged model. A shared Q combines recipient-computable inputs, task-owned outputs, and ownership-aware aggregation, connecting heterogeneous private models through a common prediction interface. Private models teach local Q copies; the returned Q supports local learning and Joint inference, with only Q parameters and counts exchanged. Across six datasets, FedSocket improves missing-modality recipient accuracy over Local by 14.44 and 15.51 percentage points on MELD and UCF-51. Under matched inference capacity, Joint exceeds independent ensembles by 11.06 points in UCF-51 accuracy and 4.87 points in mean bidirectional Flickr30k R@1. Joint also improves over Q alone on all four heterogeneous endpoints, demonstrating the value of combining local and exchanged predictions. Teacher controls, sharing-path interventions, and component factorials identify the roles of supervision, sharing, and deployment. FedSocket makes exchanged knowledge directly usable from federated training to recipient inference.

[110] Texture Space Material Diffusion cs.CV | cs.GRPDF

Jacob Munkberg, Peter Kocsis, Jon Hasselgren

TL;DR: 该方法在二维纹理空间中生成3D物体材质，通过已知的图像到纹理空间投影复用视频扩散先验，并避免多视角生成的视角一致性问题。

Details

Motivation: 文本或多视角图像生成材质时，直接在图像空间生成会受到物体几何和视角参数化影响，也容易出现不同视角材质不一致。作者希望在统一表示中支持不同几何上的材质生成与重建。

Result: 作者微调视频扩散Transformer，用于文本引导材质生成、多视图材质生成和材质超分。关键是利用图像空间到纹理空间的已知投影，使扩散过程可泛化到任意几何和纹理参数化；二维纹理空间也允许复用预训练视频扩散模型先验。方法用于未知光照下、由带姿态照片重建高质量材质，也支持文本和图像引导生成。可扩展到8K分辨率、100多个输入视图和神经材质表示。定量及定性评估中，材质生成与重建达到当前最佳结果。

Insight: 将材质生成转到纹理空间，使模型在统一二维域内学习外观，同时避免视频或多视图扩散在相机视角间不一致的问题。已知投影关系是该表示能推广到不同三维几何的关键。

Abstract: We present a method for generating high quality materials for 3D objects entirely in texture space. We finetune a video diffusion transformer for text-guided material generation, multi-view material generation, and material upscaling. Our key insight is to use the known projection from image space to texture space, enabling the diffusion process to generalize across arbitrary geometries and texture parameterizations. This approach also avoids the view consistency issues inherent in video and multi-view diffusion models. Because texture space is two dimensional, we can reuse the strong priors of pretrained video diffusion models. We apply our method to high quality material reconstruction from posed photos captured under unknown lighting, as well as to text- and image guided material generation. Our method can scale to high resolutions (8K), 100+ input views, and neural material representations. In quantitative and qualitative evaluations we show state-of-the-art results for material generation and reconstruction.

[111] Targeted Visual Counterfactual Explanations for Contrastive Vision-Language Model cs.CVPDF

Van Bach Nguyen, Jörg Schlötterer, Christin Seifer

TL;DR: MACE面向CLIP零样本分类生成目标类别的视觉反事实，通过自适应扩展编辑掩码来平衡目标类别成功率与原图保持程度。

Details

Motivation: 现有对比视觉语言模型解释方法通常只标记重要区域，没有说明如何修改图像才能得到目标预测。作者为CLIP设计有目标的视觉反事实编辑方法。

Result: MACE先依据源类别归因或源—目标归因差异构造可编辑区域，只有未达到指定目标类别时才扩展掩码；潜扩散修复模型修改选中区域，冻结的CLIP提供编辑引导并约束其余画面贴近原图。在ImageNet、Food-101、Oxford Pets和CUB-200上，源掩码版本在四个数据集均取得最高目标类别top-1成功率；差异掩码版本产生的像素及感知变化最小、真实性最好。两种版本在接近原图程度和真实性方面都优于使用同一生成骨干的纯Stable Diffusion基线。

Insight: 反事实有效性与源图像保真之间存在权衡：扩大可编辑区域有利于实现目标类别，但可能改动更多原图信息。根据目标需求选择源归因掩码或差异掩码，可分别偏重成功率或最小改动。

Abstract: Current explanation methods for contrastive vision–language models such as CLIP mainly identify important regions without showing how to change the input in order to get a target prediction. We introduce \textbf{M}ask-guided \textbf{A}daptive \textbf{C}ounterfactual \textbf{E}xplanations (\mace), a targeted visual counterfactual method designed specifically for CLIP zero-shot classification. \mace constructs an editable region from either source attribution or source–target attribution differences and expands the mask only when needed to reach a specified target class. A latent diffusion inpainting model then modifies the selected region, while a frozen CLIP model provides modification guidance and anchors the remaining image content to the original input. We evaluate \mace on ImageNet, Food-101, Oxford Pets, and CUB-200. The source-mask variant achieves the highest target top-1 success rate across all four datasets, while the difference-mask variant produces the smallest pixel-level and perceptual changes and the best realism scores. Both variants improve proximity and realism over a Stable Diffusion-only baseline using the same generative backbone. These results show that adaptive mask-guided editing produces effective CLIP counterfactuals. They further reveal a tradeoff between counterfactual validity and source-image preservation.

[112] VoxelSage: Tool-Augmented 3D CT Analysis and Simulator-Shielded Sequential Resection Planning for Liver Tumors cs.CV | eess.IVPDF

Binghong Qian, Xuanhe Liu, Yifan Xing, Wenjie Deng, Jian Wu

TL;DR: VoxelSage把肝脏CT测量、二维/三维可视化、肿瘤分析与术前切除序列规划整合到工具增强系统，并用模拟器约束规划动作。

Details

Motivation: 术前肝肿瘤评估要从同一三维CT完成分割、物理空间测量、视觉证据生成和切除规划，但常见工具将步骤分开；语言模型也难以可靠地直接计算CT中的实际物理尺寸。

Result: 双端口架构将语言模型编排与图像计算隔离：A端解析请求并选择技能，B端在CT体数据及分割掩码上执行计算并返回结构化结果，以免语言模型直接计算物理测量而产生虚构数字。八种内置技能支持定量分析、视觉证据、三维重建、分割细化和序贯切除规划；规划用行为克隆排序候选切除目标，并由模拟器约束检查。256个未见模拟场景中，相较确定性基线，平均模拟时间从34.274降至33.388分钟（减少0.886分钟，2.59%），平均模拟失血从300.847降至183.852毫升（减少116.995毫升，38.89%）。这些结果是系统集成和模拟表现，不是临床有效性或安全性证据。

Insight: 医学影像智能体应把物理测量交给确定性图像计算工具，将语言模型限定在任务编排与解释环节。模拟器可为切除序列提供约束检查，但模拟改进不能直接外推为临床效益。

Abstract: Preoperative liver-tumor assessment requires segmentation, physical-space measurement, visual evidence, and resection planning from the same three-dimensional CT volume. Existing tools often handle these steps separately, while language models cannot reliably compute physical measurements from CT. To provide an integrated workflow, we present VoxelSage, a multi-modal system for two- and three-dimensional visualization, liver-tumor analysis, and preoperative resection planning. Its dual-port architecture separates language-model orchestration from image computation: Port A interprets requests and selects skills, while Port B applies them to CT volumes and segmentation masks and returns structured results. Keeping physical measurements in Port B prevents the LLM from computing them directly and reduces the risk of fabricated numerical results. Eight built-in skills support quantitative analysis, visual evidence generation, three-dimensional reconstruction, segmentation refinement, and sequential resection planning; user-defined skills can extend these functions. For sequence planning, a behavior-cloned neural ranker orders candidate resection targets, while a simulator-based shield checks them against predefined constraints. Across 256 unseen simulator scenes, this approach reduced mean simulated time from 34.274 to 33.388 min (0.886 min, 2.59%) and mean simulated blood loss from 300.847 to 183.852 mL (116.995 mL, 38.89%) relative to a deterministic baseline. These results demonstrate system integration and simulator-level performance, not clinical efficacy or safety. The public implementation is available at https://github.com/ZJUMAI/VoxelSage.

[113] Exemplar2VQA: A Scalable Exemplar-Driven Visual Question Answering Generation Framework via Multi-Agent Coding cs.CVPDF

Jiayu Ying, Qijian Tian, Ruijie Xu, Xinnan Zhu, Daoguo Dong

TL;DR: Exemplar2VQA通过多智能体编写几何程序，把少量静态空间问题模板扩展成大量高质量合成视觉问答数据，避免让语言模型直接承担不可靠的几何计算。

Details

Motivation: 多模态大语言模型的空间智能训练受限于复杂、可扩展的3D问答数据稀缺。人工标注成本高，而直接让语言模型生成问答常因几何和空间计算能力不足而出错。

Result: 框架在模拟环境中以示例驱动生成空间问答，并为协作智能体提供精心设计的几何工具库，通过确定性代码执行绕开语言模型的空间推理缺陷。它将多种以对象为中心的静态空间查询模板扩展为大规模合成数据。仅用该框架生成的室内合成数据微调Qwen2.5-VL 3B/7B后，多个不同基准表现显著提升；收益也迁移到户外和混合场景。作者认为该方法可作为具身智能缩小模拟到现实差距的数据范式；摘要未给具体分数。

Insight: 合成空间问答的可靠性可由程序化几何工具保障，语言模型则负责模板扩展和问答组织。由少量范例规模化生成数据还表现出跨室内、户外及混合场景的迁移潜力。

Abstract: Advancing spatial intelligence in Multimodal Large Language Models (MLLMs) is bottlenecked by the scarcity of complex, scalable 3D question-answer (QA) data. While manual annotation is labor-intensive, directly utilizing LLMs to synthesize these QA pairs often fails due to their inherent deficiencies in spatial and geometric computation. We introduce Exemplar2VQA, a scalable exemplar-driven visual question answering generation framework that rapidly synthesizes large-scale spatial QA pairs in simulated environments via multi-agent coding. By equipping collaborative agents with a meticulously designed library of geometric utilities, Exemplar2VQA bypasses LLMs’ spatial reasoning flaws through deterministic code execution. Crucially, the framework exhibits remarkable versatility: taking diverse static object-centric spatial query templates as exemplars, it seamlessly and autonomously scales them into massive, high-fidelity synthetic datasets. Fine-tuning Qwen2.5-VL (3B/7B) exclusively on Exemplar2VQA-generated synthetic indoor data yields significant performance improvements across various diverse benchmarks. Furthermore, its effectiveness is not limited to in-domain indoor datasets but also robustly extends to outdoor and mixed-scene benchmarks. These results establish Exemplar2VQA as a scalable and powerful paradigm for bridging the sim-to-real gap in Embodied AI. Our code is at https://github.com/yingjiayu12/Exemplar2VQA

[114] Tracing the Evidence: Faithful Token Attribution Through Vision-Language Reasoning cs.CVPDF

Bowen Yuan, Danny Wang, Ruihong Qiu, Zijian Wang, Zi Huang

TL;DR: VTrace对视觉语言模型输入token做跨模态校准归因，并聚合直接和间接推理路径，提升图像证据在解释中的忠实呈现。

Details

Motivation: 大型视觉语言模型虽能完成复杂推理，但回答依赖哪些图像和文本证据仍难确定。现有文本模型归因方法用于多模态输入时，可能低估视觉证据，并且只跟踪部分中间路径，漏掉视觉信息经多步推理影响回答的贡献。

Result: VTrace先构造成对归因以量化token级贡献，并以闭式方式汇总所有前向归因路径，包含直接和间接贡献；之后根据回答似然变化估计模态贡献，对图像和文本归因分数进行跨模态校准，形成统一token排序。在六个视觉推理基准上与七种基线比较，作者报告归因忠实度更优；摘要未列出具体数值。

Insight: 视觉与文本token的原始归因分数不能直接比较，尤其视觉证据可能经过多条中间推理路径传递。全路径归因结合模态级校准，有助于解释模型实际依赖的证据。

Abstract: Large vision-language models (LVLMs) exhibit strong reasoning capabilities, yet the visual and textual evidence supporting the generated responses remains difficult to identify. Faithful token attribution explains an LVLM’s response by assigning scores that rank image and prompt tokens by how much the model relies on them, such that removing higher-ranked tokens causes the likelihood of the generated response to drop more rapidly. However, existing token-attribution methods have been developed mainly for text-based language models, and our empirical study reveals two challenges when complex multimodal sources are involved. First, the joint image-text attribution can underrepresent visual evidence relative to text, obscuring the image regions supporting the response. Second, visual evidence may influence the generated response through multiple intermediate reasoning paths, while existing methods trace only a limited subset of these paths, causing important visual contributions to be underestimated. Motivated by these insights, we introduce VTrace, a multimodal token-attribution framework that traces input contributions through intermediate reasoning and calibrates attribution scores across modalities. VTrace constructs pairwise attributions that highlight token-specific contributions and aggregates all forward attribution paths in closed form to account for both direct and indirect contributions. Cross-modal calibration then rescales image and text attribution scores using modality contributions estimated from response-likelihood changes, enabling a unified ranking of input tokens. Evaluations against seven baselines across six visual reasoning benchmarks demonstrate the superior attribution faithfulness. Project page: https://vtrace-attribution.github.io/.

[115] Are In-Context Images Worth 10 Dimensions? cs.CV | cs.LGPDF

Adhemar de Senneville, Xavier Bou, Jérémy Anger, Rafael Grompone, Gabriele Facciolo

TL;DR: 论文在大型视觉语言模型中发现跨图像分类任务共享的低维判别几何，并分析早期层如何将上下文图像压缩为可线性分类的表示。

Details

Motivation: 少样本上下文分类的归纳回路会使用上下文中带标签示例的线性表示来分类查询图像，但这些表示如何形成仍缺少研究。视觉模态具有丰富表达，作者借此探查上下文示例编码中的共享结构。

Result: 作者识别出共享判别几何（SDG）：一个跨图像分类任务共享的低维空间，上下文图像被压缩到该空间后形成线性可分表示，供后续分类使用。研究观察到这种压缩发生在模型早期层，并从理论和实证分析解释：线性自注意力可把上下文数据投影到主成分方向，每层实现朝该目标的一步梯度下降；训练好的视觉语言模型在早期层也呈现相似的维度降低机制。标题中的“10维”强调低维空间问题，但摘要未给出一个统一固定维数。

Insight: 上下文学习的视觉分类可能先由早期层形成任务共享的低维判别坐标系，再由后续归纳机制完成分类。分析表示形成过程，有助于解释上下文示例如何转化为可利用的线性结构。

Abstract: There has been significant work on understanding the In-Context Learning capabilities of Large Language Models, especially on the induction circuit. For a few-shot classification task, the induction circuit leverages linear representations of each labeled example in-context in order to classify an unlabeled query. However, few works focus on how those linear representations are built in the first place. Leveraging the expressivity of the vision modality compared to text, we uncover a Shared Discriminative Geometry (SDG) inside Large Vision Language Models (LVLMs). It is a low-dimensional space, shared across all image classification tasks, in which in-context images are compressed into linearly separable representations later used to perform classification. We observe that this is the result of the model performing a dimensionality reduction of vision representations in early layers. In order to explain this phenomenon: (1) We show analytically that linear self-attention can perform a dimensionality reduction by projecting in-context data onto its principal components, with each layer implementing one gradient descent step toward this objective. (2) We provide evidence that trained LVLMs reduce the dimensionality of vision representations in early layers via a similar mechanism.

[116] Med-RADIO: Reducing All Medical Domains Into One via Multi-Teacher Distillation cs.CVPDF

Chu Zhang, Haoyu Jiang, Hongyuan Zhang, Hongbin Liu, Dong Yi

TL;DR: Med-RADIO通过多教师蒸馏把不同医学影像领域专家的互补知识压缩进统一医学视觉基础模型，以提高通才模型的跨模态基线表现。

Details

Motivation: 医学影像模态差异大，现有路线分别训练专科模型或多模态通才模型。通才模型的训练数据规模不及自然图像通才模型，领域深度又不及医学专科模型；经验上通才定义跨模态基线，专科模型则提供各自领域的性能上限。

Result: Med-RADIO汇集通才与专科教师，为不同模态分配对齐的蒸馏流，重组通才预训练数据以匹配专科领域，并采用平衡损失防止某一教师主导。在涵盖五种模态的内部和外部分类基准上，线性探测优于强医学通才模型，并在多数评估模态上接近代表性专科模型。摘要未报告各模态具体分数。

Insight: 医学通才模型可从多个专科教师吸收互补领域知识，同时通过模态对齐和损失平衡避免专家知识失衡。目标是抬高跨模态通用基线，并尽量接近单领域专家的上限。

Abstract: The rapid expansion of large-scale medical datasets and computational resources has driven significant progress in medical foundation models. Given the inherent heterogeneity of medical imaging modalities, current research mainly follows two paths: specialized models optimized for specific modalities, and generalist models designed to handle multiple modalities. However, medical generalist models suffer from both insufficient training data scale relative to natural image generalists and inadequate domain-specific depth relative to medical specialists. Empirically, generalist models establish a cross-modality performance baseline, while specialists define the performance ceiling within their respective domains. To elevate this baseline toward these ceilings, we propose Med-RADIO, a medical multi-teacher distillation framework that Reduces All Domains Into One by compressing complementary expertise from multiple domain-specific teachers into a unified medical vision foundation model. Our method curates both generalist and specialist teachers, allocates modality-aligned distillation streams to reorganize generalist pretraining data so it matches specialist domains, and uses a balanced loss to prevent any single teacher from dominating the distillation process. On internal and external classification benchmarks spanning five modalities, Med-RADIO improves over strong medical generalists under linear probing and remains competitive with representative specialists on most evaluated modalities. Code is available at https://github.com/CAIR-HKISI/Med-RADIO.

[117] VIF-Bench: Evaluating Visual Instruction Following in Multi-Reference Image Generation cs.CV | cs.AIPDF

Yuta Oshima, Masakazu Yoshimura, Masahiro Suzuki, Yutaka Matsuo, Hiroki Furuta

TL;DR: VIF-Bench评测图像生成模型能否同时理解多张参考图和多种视觉指令，并发现遵循指令程度与生成伪影、参考图干扰之间存在权衡。

Details

Motivation: 新一代多模态图像生成模型能接收多张图像和文本指令，还可理解布局、箭头、姿态等视觉指令。但既有基准较少测试多个参考图与多种异质视觉指令联合使用的复杂情形。

Result: VIF-Bench包含1,241项任务，最多同时提供7张参考图和6种视觉指令；测试参考图可能与指令竞争的情况，并控制比较不同具体程度的文本描述与视觉指令。评测发现三点：视觉指令遵循增强时，生成图中的指令伪影也往往增多；当参考图本身呈现了受控属性的显著状态（尤其光照和风），模型遵循视觉指令更困难；对能理解视觉指令的模型，直接给视觉约束通常优于文字描述，而文字描述以中等细节为宜，不宜过度详尽。

Insight: 多参考图生成中，视觉指令、文本和参考内容可能互相冲突，不能只以指令遵循分数评价模型。基准揭示了遵循度—伪影权衡，也表明视觉约束是否由图像直接表达会影响生成控制。

Abstract: Recent multimodal image generation models can take multiple images and textual instructions as input, enabling reference-based generation guided not only by text but also by visual instructions such as layouts, arrows, and pose cues. However, existing benchmarks do not evaluate the joint setting in which multiple references must be composed under multiple and heterogeneous visual-instruction images. To address this gap, we introduce VIF-Bench, a benchmark of 1,241 tasks designed to assess the edge of model capabilities in this joint setting by covering: (i) multi-reference generation (up to 7) under multiple heterogeneous visual instructions (up to 6), (ii) cases where reference images can potentially compete with visual instructions (e.g., a strongly posed subject vs. a target pose), and (iii) controlled comparison of visual instructions with text descriptions at different levels of specificity. Using these capabilities, we uncover three findings: (1) models face an adherence-artifact trade-off: once models reach stronger visual instruction adherence, stronger adherence tends to coincide with more instruction artifacts in generated images, (2) visual instruction adherence tends to be lower on tasks whose reference images carry a salient state of the controlled attribute (e.g., a neon-lit subject under a light-direction instruction), most consistently for light and wind, and (3) for models that can understand visual instructions, it is often better to provide visual constraints directly rather than describe them in text; when using text, a moderate level of detail works better than an exhaustive description. VIF-Bench is released as an open benchmark to establish a basis for fair comparison in controllable multi-reference image generation.

[118] Honeycomb: Constant-Size Scene Memory Representation for Video World Models cs.CVPDF

Jack Wei Lun Shi, Kaichen Zhou, Haoyu Chen, Yufeng Weng, Keane Ong

TL;DR: Honeycomb以六个空间及时空平面的固定大小低秩记忆存储视频场景特征，使世界模型能够在长时生成中保持已见区域一致而不随历史增长而扩容。

Details

Motivation: 长时视频生成需要持久场景记忆来维持一致性，但现有空间记忆持续累积RGB观测或潜特征，生成越久存储越大，也可能需要反复处理完整历史。

Result: Honeycomb采用HexMemory固定大小表示。前馈写入器将最新视频块映射为平面特征；空间覆盖或时间范围扩展时，系统变形已有平面但保持尺寸不变，再以置信度加权池化和学习到的残差校正整合新特征。读取器取出记忆潜特征以条件化后续生成。由于写入器只处理最新片段，无需逐场景优化或重复处理完整生成历史。在WorldScore和RealEstate10K实验中，视频生成质量较强，重访已观察区域时一致性稳健，特征存储量在生成全过程保持恒定。

Insight: 固定容量空间记忆必须在覆盖扩大时压缩并融合旧信息，而不只是不断追加帧或潜特征。只处理新生成块、用置信度池化更新记忆，可同时避免历史重复计算并维持恒定存储。

Abstract: Video world models require persistent scene memory to maintain consistency during long-horizon video generation. Existing spatial memory systems accumulate RGB observations or latent features, causing storage requirements to grow as generation proceeds. We introduce Honeycomb, a video world model built on HexMemory, a compact low-rank representation that stores scene features in a fixed-size memory comprising six spatial and spatiotemporal planes. A feed-forward writer maps each newly generated video chunk to plane features. As the spatial coverage or temporal range expands, HexMemory warps the existing planes while preserving their dimensions, then integrates new features through confidence-weighted pooling and a learned residual correction. A reader retrieves latent features from HexMemory to condition subsequent video generation. Because the writer processes only observations from the latest chunk, Honeycomb avoids per-scene optimization and repeated processing of the full generation history. Experiments on WorldScore and RealEstate10K demonstrate strong video generation quality and robust consistency when revisiting previously observed regions, while maintaining constant feature-storage requirements throughout generation. Code and additional visualizations are available on our https://jackswl.github.io/honeycomb/.

[119] PolyOCR-Venus: Unified OCR Foundation Models for Text-Centric Visual Intelligence cs.CV | cs.AIPDF

GuangJian Team, Kaili Huang, Yongshuo Zhang, Bingtao Fu, Changjiang Jiang

TL;DR: PolyOCR构建多尺度统一OCR基础模型，并用教师可靠度和师生能力差距指导强化学习与在线自蒸馏，覆盖识别、解析和文字视觉推理。

Details

Motivation: OCR正从纯文本转写扩展为复杂视觉环境中的文字识别、定位和推理。既有系统往往只能在部分任务上表现突出，难以平衡不同场景下的识别、解析与推理能力。

Result: PolyOCR结合统一指令跟随框架和大规模数据引擎，将异构视觉资源转为经质量验证的OCR监督。其能力引导策略优化把基于验证器的GRPO与在线策略蒸馏结合，并按样本根据教师可靠性及师生能力差距进行路由。作者还发布OCRBench v2.1，修订OCRBench v2的人工标注错误并调整为与任务对齐的评分。在OCRBench v2.1、CC-OCR、自建KIE Benchmark、OmniDocBench v1.6和MDPBench等评测中，PolyOCR达到先进或有竞争力的表现；摘要未列具体分数。

Insight: 统一OCR基础模型需要可靠且多样的监督数据，并根据不同样本上教师是否值得信赖来分配蒸馏信号。评测基准的人工标注修订和任务对齐计分同样关系到能力比较是否可信。

Abstract: Optical Character Recognition (OCR) is evolving from plain-text transcription toward general visual intelligence, requiring models to recognize, localize, and reason over textual information in complex visual environments. However, existing OCR systems often excel at only some tasks and struggle to balance recognition, parsing, and reasoning across scenarios. In this report, we present PolyOCR, a family of unified OCR foundation models of varying scales. PolyOCR combines a shared instruction-following framework with a large-scale data engine that converts heterogeneous visual resources into quality-verified OCR supervision. We introduce Competence-Guided Policy Optimization, which combines verifier-based Group Relative Policy Optimization with on-policy distillation through sample-wise routing based on teacher reliability and the teacher–student competence gap. We also introduce OCRBench v2.1, our revision of OCRBench v2 with manually verified annotation corrections and task-aligned scoring metrics. Extensive experiments across OCRBench v2.1, CC-OCR, in-house KIE Benchmark, OmniDocBench v1.6 and MDPBench demonstrate that PolyOCR achieves state-of-the-art or highly competitive performance.

[120] CHOQOLATE: Organizing Concept Bottleneck Latent Spaces with Choquet Integrals cs.CVPDF

Rémi Kazmierczak, Johanne Cohen, Marianne Clausel

TL;DR: CHOQOLATE用二阶可加Choquet积分把相关概念整合为稀疏、语义连贯的概念节点，改善视觉概念瓶颈模型的可解释性并支持测试时偏差干预。

Details

Motivation: 基于CLIP等视觉语言模型的概念瓶颈模型将潜在表示解释为人类可读概念，但相关概念相互纠缠，单独概念分数往往无法忠实代表其含义。

Result: CHOQOLATE加入基于2-additive Choquet积分的可解释设计层，将相关概念合并为紧凑节点。在四个数据集上得到较好的准确率—可解释性权衡，节点权重稀疏且语义连贯。作者给出闭式梯度推导并通过实验解释该层如何在无显式监督下组织概念。Choquet权重可直接映射为Shapley值，因此支持测试时干预；在标准偏差缓解基准上，训练后抑制伪相关概念的表现与需要群体标注或重新训练的方法相当，而无需这些额外条件。

Insight: 概念之间的交互不能总以相互独立的分数来表达；Choquet积分能把相关概念组织成更紧凑的可解释单元。可映射到Shapley值的权重也让训练后的偏差干预成为可能。

Abstract: Concept Bottleneck Models (CBMs) built on vision-language models such as CLIP represent a latent space as human-understandable concepts. These representations are unfaithful: related concepts are entangled, so individual scores do not reflect their intended meaning. We propose CHOQOLATE, an interpretable-by-design layer based on 2-additive Choquet integrals, which merges correlated concepts into compact nodes. Across four datasets, CHOQOLATE achieves a favorable accuracy-interpretability trade-off, with weight-sparse and semantically coherent nodes. A closed-form gradient derivation, backed by experiments, explains why Choquet layers drive this organization without explicit supervision. Choquet weights also map directly to Shapley values, which enables test-time intervention. On standard bias-mitigation benchmarks, suppressing spurious concepts after training performs on par with methods that require group annotations or retraining, while needing neither.

[121] ReCaVSR: One-Step Streaming Diffusion Video Super-Resolution with Recycled Latents and Learned Cache Routing cs.CVPDF

Xijun Wang, Xin Li, Suhang Yao, Zirui Lang, Bingchen Li

TL;DR: ReCaVSR用前序超分潜变量携带局部时间信息，并为不同DiT层学习缓存时间范围，实现无需迭代采样和完整历史KV缓存的流式视频超分。

Details

Motivation: 扩散式视频超分需求旺盛，但在线流式应用对延迟严格，通常要在生成保真度与实时性之间取舍。作者观察到复用的超分潜变量可保存局部时间上下文，各Transformer层需要的时间范围也不相同。

Result: ReCaVSR基于Wan2.2，采用单步流式框架：每层在缓存预算内学习KV历史范围并导出静态推理计划，新视频块再以模型前序预测的超分潜变量为条件；多范围查询判别器分别提供全局真实感、局部纹理和时间稳定反馈；低分辨率条件FlashDecoder将观测信息融入高效潜解码。合成和真实视频超分基准显示，相较代表性基线，其感知质量、时间一致性和流式效率更好。单张A100-80GB输出1080×1920时达到21.20 FPS、峰值分配显存15.16GB；相较FlashVSR Tiny快2.72倍，峰值显存少38.0%。

Insight: 流式超分不必物化完整历史缓存：复用上一块预测潜变量可延续局部时间上下文，而各层按需读取不同时间范围。缓存调度、生成判别和低分辨率条件解码需共同设计以兼顾质量与延迟。

Abstract: Real-time diffusion-based video super-resolution (VSR) is in high demand for online streaming, yet stringent latency requirements often compromise generative fidelity. We propose ReCaVSR, a Wan2.2-based, one-step framework for streaming VSR that builds on two observations: recycled SR latents retain local temporal context, reducing the need for full historical Key-Value (KV) caches; and individual transformer layers benefit from distinct temporal scopes. ReCaVSR combines three complementary designs: (i) layer-wise cache routing with recycled SR latents: each DiT layer learns its KV-cache temporal scope under a cache budget and exports a static inference schedule, while recycled SR latents propagate local context by conditioning each new block on the model’s own preceding predictions. (ii) Multi-Scope Query (MSQ) Discriminator: a compositional discriminator combining global, spatial-window, and temporal-tube feedback for holistic realism, local texture generation, and temporal stability. (iii) LR-conditioned adaptation of FlashDecoder: a VAE decoder that incorporates LR observations for efficient latent decoding. ReCaVSR enables streaming VSR without iterative sampling or full historical KV-cache materialization. Experiments on synthetic and real-world VSR benchmarks show better perceptual quality, temporal consistency, and streaming efficiency than representative VSR baselines. At $1080{\times}1920$ output resolution on a single NVIDIA A100-80GB, ReCaVSR achieves 21.20 FPS with 15.16 GB peak allocated GPU memory, running 2.72$\times$ faster while using 38.0% less peak allocated memory than FlashVSR Tiny. The code is available at https://github.com/kopperx/ReCaVSR.

[122] RelayVSR: Large-Small Model Collaboration for Efficient Real-World Video Super-Resolution cs.CVPDF

Xijun Wang, Xin Li, Zirui Lang, Suhang Yao, Haoran Li

TL;DR: RelayVSR让大型生成模型只为稀疏关键帧生成参考潜变量，再由轻量视频超分网络处理所有帧，并用系统级与关键帧级双层强化奖励优化协作。

Details

Motivation: 生成式模型能为真实视频超分恢复逼真细节，但对整段视频逐帧使用代价高。只优化关键帧质量也不够，因为参考误差会在轻量网络传播后累积到输出视频。

Result: RelayVSR的稀疏生成中继机制由大型生成模型产生关键帧参考潜变量，双记忆视频Transformer结合低分辨率视频为各帧超分，并在帧间复用关键帧信息、更新近期上下文；支持首关键帧条件和有界前瞻的双端点条件。视频感知参考优化（VARO）用强化学习更新大型模型，联合考虑固定轻量超分网络生成视频的系统级奖励和解码关键帧质量奖励；优于直接联合训练，双层奖励也优于仅系统级奖励。单张A100 80GB上，1080p双端点模式、15帧关键帧间隔达到29.29 FPS、峰值显存13.82GB、首帧模型延迟0.327秒；FlashVSR-Tiny分别为7.80 FPS、24.447GB和2.83秒。

Insight: 稀疏关键帧生成可把昂贵生成能力集中到参考内容，轻量网络负责连续输出；优化参考应以最终视频为目标，同时保留对关键帧本身质量的约束，防止误差沿时间传播。

Abstract: Large generative models can recover realistic detail in real-world video super-resolution (VSR), but processing an entire video with them is computationally expensive. In this work, we present RelayVSR, a streaming VSR framework built on the Sparse Generative Relay mechanism. A large generative model generates reference latents for sparse keyframes, while a lightweight VSR network uses these references and low-resolution video to super-resolve every frame. The lightweight VSR network, implemented as a Dual-Memory Video Transformer, reuses keyframe information across frames and updates recent video context, supporting first-keyframe conditioning and dual-endpoint conditioning with bounded lookahead. However, errors in shared keyframes can propagate and accumulate across output frames, making keyframe quality alone an insufficient optimization target. We address this collaboration gap with Video-Aware Reference Optimization (VARO), which uses reinforcement learning to update the large generative model with two reward levels: a system-level reward evaluates videos produced by the fixed lightweight VSR network, while a reference-level reward evaluates decoded keyframe quality. VARO improves final video quality over direct joint training, and its dual-level rewards outperform a system-level reward alone. At 1080p on a single NVIDIA A100 80GB, dual-endpoint RelayVSR with a 15-frame keyframe interval reaches 29.29 FPS, 13.82 GB peak GPU memory, and 0.327 s first-frame model latency, compared with 7.80 FPS, 24.447 GB, and 2.83 s for FlashVSR-Tiny. The code is available at https://github.com/kopperx/RelayVSR.

[123] FlowMap-OPD: Rollout–Kernel Separation for On-Policy Distillation of Few-Step Flow-Map Generators cs.CV | cs.LGPDF

Zhiqi Li, Bo Zhu

TL;DR: FlowMap-OPD将少步流映射生成器的学生状态采集与师生分布比较拆开，并比较不同速度分布监督方式和各自匹配的原生rollout。

Details

Motivation: MeanFlow和一致性模型等少步流映射生成器可通过长距离传输高效采样，但其在线策略蒸馏研究有限。直接把学生状态采集与教师比较混在一起，难以针对实际部署的长程映射设计监督。

Result: FlowMap-OPD以状态边缘分布公式建立采集与分布比较的分离，并用流—速度一致性把局部监督连接到部署时长程映射。框架支持流映射、诱导速度和瞬时速度三种分布监督，每种都配套单独定义的原生流映射rollout。跨容量ImageNet实验使用三种教师奖励，发现瞬时速度分布监督并允许独立调节学生一致性最有效。文生图实验中，FlowMap-OPD展现较强的多专家能力整合效果，在任务表现和收敛速度上超过多奖励Flow-Map GRPO。

Insight: 少步生成器的训练应与实际长程映射行为一致，并明确区分如何产生学生状态与如何比较师生分布。实验显示，对瞬时速度分布的监督和单独可调的一致性约束更有效。

Abstract: Few-step flow-map generators, including MeanFlow and consistency models, enable efficient sampling through long-range transport, yet their on-policy distillation remains underexplored. We introduce FlowMap-OPD, an on-policy distillation framework that separates student-state acquisition from teacher–student distribution comparison. A formulation based on state marginals establishes this separation, while flow–velocity consistency connects local supervision to the deployed long-range map. Within this framework, we develop flow-map, induced-velocity, and instantaneous-velocity distribution supervision, each paired with a separately specified native flow-map rollout. Cross-capacity ImageNet experiments across three teacher rewards identify instantaneous-velocity distribution supervision with independently tunable student consistency as the most effective choice. In text-to-image experiments, FlowMap-OPD demonstrates strong multi-specialist consolidation capabilities and surpasses multi-reward Flow-Map GRPO in task performance and convergence speed.

[124] Learning from synthetic photorealistic raindrop for single image raindrop removal cs.CVPDF

Zhixiang Hao, Shaodi You, Yu Li, Kunming Li, Feng Lu

TL;DR: 作者构建物理真实感的合成附着水滴数据集，并针对水滴折射和模糊设计检测与去除网络，以解决单图像雨滴去除训练数据不足的问题。

Details

Motivation: 附着在镜头或挡风玻璃上的水滴会影响自动驾驶等视觉系统。水滴外观由水体动力学、几何和光度等多个因素共同决定，传统模型难以覆盖复杂变化，深度学习则缺乏足够大且真实的训练数据。

Result: 论文提出首个用于训练的照片级真实合成附着水滴数据集，基于物理渲染建模水动力、几何和光度，包含多种雨景，特别覆盖雨天驾驶场景。基于水滴成像模型，作者设计能够感知水滴折射及模糊的检测网络，再训练去除网络恢复图像结构。严格实验表明，框架达到当前最佳表现；摘要未列出具体指标或数据集规模。

Insight: 水滴去除的合成监督需要模拟成像物理，而不能只叠加简单雨滴纹理；检测网络显式利用折射和模糊线索，再进行结构恢复，构成数据建模与任务网络相互配合的方案。

Abstract: Raindrops adhered to camera lens or windshield are inevitable in rainy scenes and can become an issue for many computer vision systems such as autonomous driving. Because raindrop appearance is affected by too many parameters, therefore it is unlikely to find an effective model based solution. Learning based methods are also problematic, because traditional learning method cannot properly model the complex appearance. Whereas deep learning method lacks sufficiently large and realistic training data. To solve it, in our work, we propose the first photo-realistic dataset of synthetic adherent raindrops for training. The rendering is physics based with consideration of the water dynamic, geometric and photometry. The dataset contains various types of rainy scenes and particularly the rainy driving scenes. Based on the modeling of raindrop imagery, we introduce a detection network which has the awareness of the raindrop refraction as well as its blurring. Based on that, we propose the removal network that can well recover the image structure. Rigorous experiments demonstrate the state-of-the-art performance of our proposed framework.

[125] Visual Branch is What You Need for CLIP-based Class-Incremental Learning cs.CV | cs.LGPDF

Tao Hu, Zhen-Hao Xie, Jingcai Guo, De-Chuan Zhan, Da-Wei zhou

TL;DR: VIS完全在视觉特征空间构造CLIP类增量分类器，避免文本分类权重与视觉类别分布之间的模态差异，并通过充分统计量实现高效更新。

Details

Motivation: CLIP类增量学习常把类别名模板的文本嵌入作为分类权重，再与图像特征做余弦相似度；但文本和视觉嵌入可能存在模态间隔，使文本权重偏离真实视觉类别分布。

Result: 在相同任务级类增量训练下，作者发现以视觉类别中心初始化余弦分类器比使用CLIP文本特征损失更低、增量准确率更高。由此提出VIS，部署时移除文本分支，仅在视觉空间构建增量分类器；使用基础阶段数据从CLIP最终视觉表示中强化有信息的视觉层特征。随后用核化增量最小二乘SVM，并从可加充分统计量闭式求出权重。新类别到来时累积其统计量，重新计算所有已见类别的分类器权重，从而高效更新并保留历史知识。实验中VIS不依赖文本分支仍达到当前最佳类增量学习表现。

Insight: 共享图文嵌入空间并不保证文本类别向量符合逐步到来的视觉类别分布。完全在视觉域更新分类器、并以充分统计量压缩历史数据，可减少模态间隔影响并维持增量效率。

Abstract: Class-Incremental Learning (CIL) requires models to recognize new classes over time without forgetting previously learned ones. With the rise of vision-language pre-training, CLIP has become a strong foundation for CIL. A common design in CLIP-based CIL is to construct textual classifier weights by encoding class-name templates with the CLIP text encoder, and then classify visual features by image-text cosine similarity. This design is appealing: since CLIP aligns images and text in a shared embedding space, textual weights appear to provide an off-the-shelf classifier for incremental classes. However, we show that this seemingly natural design is not always beneficial, as a modality gap can still separate the two modalities and make textual classifier weights deviate from visual class distributions. Empirically, under identical task-wise CIL training, initializing the cosine classifier with visual class centers yields lower loss and better incremental accuracy than using CLIP textual features.Motivated by these observations, we propose VIS, a visual-only method for CLIP-based CIL that removes the deployed textual branch and constructs the incremental classifier entirely in the visual space. To obtain stronger task-adaptive visual representations, VISuses only base-session data to enhance CLIP’s final visual representation with informative visual-layer features. Built on the enhanced visual representation, VISemploys a simple kernelized incremental least-squares SVM, whose classifier weights are solved in closed form from additive sufficient statistics. When new classes arrive, VISaccumulates their sufficient statistics and recomputes the classifier weights for all seen classes, enabling efficient incremental updates while preserving historical class knowledge. Extensive experiments show that VISachieves state-of-the-art performance without a textual branch.

[126] EndoPrior-GS: Dynamic Endoscopic Reconstruction with a Joint Texture Prior cs.CVPDF

Jiaqi Huang, Shidong Wang, Tong Xin, Kabita Adhikari

TL;DR: EndoPrior-GS利用由有效组织区、非镜面光照过滤和解剖显著性构成的联合纹理先验，改善动态内窥镜3D高斯重建中的伪几何与光照变化问题。

Details

Motivation: 3D高斯泼溅可实现实时渲染，但在可变形的术中场景中容易出现伪几何并受光照变化干扰，限制动态内窥镜重建的可靠性。

Result: EndoPrior-GS结合视频帧提取的视觉启发式信息和估计深度图，构造联合纹理概率先验。先验由过滤手术器械后的有效组织掩码、非镜面光度过滤和解剖结构显著性组成，用于引导高斯基元初始化和后续密度控制；还增加纹理感知时序项，在训练时动态加权成对基元贡献。在EndoNeRF和SCARED基准上，相比代表性方法，Flow Error分别降低27.7%和25.8%，同时保持有竞争力的渲染质量和实时速度。

Insight: 动态内窥镜重建中的几何错误与高光、器械遮挡及解剖结构信息相关，可将这些先验显式转成基元初始化和密度管理信号。把纹理先验扩展到时间域，还能约束相邻帧基元贡献的一致性。

Abstract: Dynamic endoscopic reconstruction is fundamental to robotic surgery and computer-assisted interventions. While 3D Gaussian Splatting (3DGS) realises real-time rendering, its application to deformable intraoperative environments remains constrained by spurious geometry and varying illuminations. To address these limitations, we introduce EndoPrior-GS, a novel pipeline that explicitly couples frame-extracted vision heuristics and estimated depth maps. EndoPrior-GS derives a joint texture prior from a tool-filtered valid tissue mask, a non-specular photometric filter, and anatomical structural salience, yielding a probability map that guides primitive initialisation and subsequent density control. The prior is further extended to the temporal domain through a texture-aware term that dynamically weighs pairwise primitive contributions during training. We conduct extensive experiments on benchmark datasets EndoNeRF and SCARED, and the obtained results show that our method EndoPrior-GS reduces Flow Error by 27.7% and 25.8% over the representative approaches while preserving competitive rendering quality and real-time rendering speed. Our project website is available at https://jiaqi-huang-77.github.io/EndoPrior-GS/.

[127] EpiCon: Collective Agent Learning through Co-Evolving Multimodal Memory cs.CVPDF

Ziyun Zeng, Hang Hua, Shaden Alshammari, Rogerio Feris, William T. Freeman

TL;DR: EpiCon让多个智能体共享并持续整理多模态经验记忆，在不更新宿主模型参数的情况下跨系统复用视觉与文本经验。

Details

Motivation: 智能体可从过去执行中学习，但不同智能体如何复用彼此经验并在此基础上继续积累仍然困难，单个系统的经验记忆难以成为群体能力。

Result: EpiCon通过两个独立训练的20亿参数模型连接问题级记忆演化与持久经验库：记忆控制器在多次尝试中共同修订文本指导与视觉证据，并选择性纳入视觉记忆；树形自组织器则分层整合经验，并为新问题检索经验和规则。评测覆盖四类多模态任务的11个基准、两种harness及多个骨干。冻结记忆库即使只允许一次解题，也能帮助其他系统；第二种harness使原系统在11基准宏平均提高2.6分。四种宿主配置比无记忆提高1.7至4.9分，记忆操作耗时比与骨干同规模的记忆模型减少67%至74%。

Insight: 多智能体经验共享不仅要存档答案，还需联合管理视觉证据、文本指导和抽象规则。分层自组织与按需检索使冻结记忆也能跨系统产生增益，同时降低记忆调用开销。

Abstract: Agents can learn from past executions, but enabling different agents to reuse and build on one another’s experience remains challenging. We introduce EpiCon, a shared multimodal memory framework for agent collective learning without updating host model parameters. EpiCon links question-level memory evolution to a persistent experience bank through two independently trained 2B models: a memory controller and a tree self-organizer. The controller jointly refines textual guidance and visual evidence across attempts and selectively includes visual memory. The self-organizer consolidates lessons hierarchically and retrieves experience and rules for new problems. We evaluate EpiCon on eleven benchmarks spanning four multimodal task domains, using two harnesses and multiple backbones. A frozen bank improves other systems even with a single solving attempt. A second harness raises the original system’s macro-average score by 2.6 points across eleven benchmarks. Across four host configurations, EpiCon improves macro-average scores by 1.7 to 4.9 points over No Memory and reduces memory-operation time by 67% to 74% relative to backbone-sized memory models.

[128] ReCAP: Retrieval-Guided Capability Reuse for Multimodal Continual Instruction Tuning cs.CV | cs.LGPDF

Tao Hu, Zhinuo Zhou, Xialiang Tong, De-Chuan Zhan, Da-Wei Zhou

TL;DR: ReCAP在多模态持续指令微调中检索外部领域、推理和格式知识，并通过可回收子空间保护历史能力，帮助模型跨阶段复用能力。

Details

Motivation: 多模态持续学习既需避免灾难性遗忘，也可利用外部知识为新指令提供领域信息和可复用推理模式。既有方法主要约束参数更新或隔离任务适配，外部知识如何指导持续适应尚缺少研究。

Result: ReCAP在每个持续学习阶段用外部搜索和语言模型，根据当前训练数据增量建立领域、推理和格式知识库。处理每条指令时，领域知识引导生成，推理知识决定并排序能力模块，形成实例级能力路径。为防止模块跨阶段复用时被新训练覆盖，方法用共享基底和阶段专属核心参数化可复用模块，保护历史重要方向并循环利用剩余容量。在多模态持续指令微调基准上的实验报告达到当前最佳表现；摘要未给具体数值。

Insight: 持续适应可把外部知识用于指引能力复用，而不仅是通过约束参数更新防遗忘。共享子空间与阶段专属部分并行设计，可以在保护旧方向时为新阶段腾出参数容量。

Abstract: Multimodal continual instruction tuning (MCIT) aims to enable multimodal large language models to acquire new capabilities from sequential tasks while preserving previously learned knowledge. Existing methods primarily mitigate catastrophic forgetting by constraining parameter updates or separating task-specific adaptations. However, continual adaptation can also benefit from external knowledge that provides domain-specific information and reusable reasoning patterns for solving diverse instructions. For example, to answer “How many red cubes are to the left of the sphere?”, domain knowledge can provide relevant concepts about objects and spatial relations, while reasoning knowledge can specify ordered operations such as object recognition, spatial filtering, and counting. Despite this potential, how to leverage external knowledge for continual adaptation remains largely unexplored in existing MCIT methods. To this end, we propose ReCAP, a retrieval-guided framework that leverages external knowledge to guide capability reuse during continual adaptation. At each continual stage, ReCAP uses external search and an LLM to incrementally build a knowledge base of domain, reasoning, and format knowledge based on the current-stage training data. For each instruction, retrieved domain knowledge guides generation, while retrieved reasoning knowledge selects and orders capability modules to form an instance-specific capability path. As these capability modules are reused across stages, subsequent adaptation can overwrite previously learned parameters. To enable stable cross-stage reuse, ReCAP introduces adaptive subspace recycling, which parameterizes reusable capability modules with shared bases and stage-specific cores, protects historically important directions while recycling residual capacity. Extensive experiments on MCIT benchmarks show that ReCAP achieves SOTA performance.

[129] SYNCR: Diagnosing and Learning Cross-Video Reasoning from Simulation cs.CV | cs.LGPDF

Sara Ghazanfari, Siddharth Garg, Prashanth Krishnamurthy, Farshad Khorrami

TL;DR: SYNCR以模拟器状态生成可验证的跨视频推理数据和答案，用于诊断多模态模型在事件对齐、身份匹配、运动比较和证据整合上的困难。

Details

Motivation: 跨视频推理需要对齐事件、匹配身份、比较运动并整合不完整观察。评测和改进这些能力都需要可靠标签及有针对性的监督，而真实视频往往难提供精确答案和可控任务变化。

Result: SYNCR基于Habitat、Kubric和CLEVRER等模拟器，以环境状态自动产生答案，包含4,000个评测问题和15,960个训练问题，覆盖八种跨视频推理任务，训练与评测视频互不重叠。视觉消融和人工评估用于检查问题是否依赖给定证据及答案是否可恢复。对22个多模态大模型评估发现，模型在物理比较和场景整合上仍有困难，扩大模型规模未必解决。监督微调将Qwen3-VL-8B平均准确率从32.6%提升至61.6%，并泛化到训练中未出现的任务配置和视频来源。迁移到真实视频时，时间排序提升最稳定：在Assembly101及Panoptic构建集上，三个跨两家族、两种规模的检查点提高9.0至20.5个百分点，现有时间推理基准也有提升。

Insight: 模拟器提供共享任务生成器，既可精确控制跨视频关系，也可保证答案标签可信。合成监督能迁移到真实视频，但不同能力迁移程度不一，当前时间排序比复杂场景整合更稳定。

Abstract: Reasoning across videos requires aligning events, matching identities, comparing motion, and integrating partial observations. Evaluating these capabilities and testing how to improve them requires both reliable labels and targeted supervision. We introduce SYNCR, a simulator-grounded framework that connects these two needs through shared task generators. Built on Habitat, Kubric, and CLEVRER, SYNCR derives answers from environment state and provides 4,000 evaluation questions and 15,960 training questions over disjoint videos, spanning eight cross-video reasoning tasks. Visual ablations and human evaluation assess dependence on the supplied evidence and answer recoverability. Evaluation of 22 multimodal large language models reveals persistent difficulties in physical comparison and scene integration that increasing model size does not consistently resolve. Supervised fine-tuning raises Qwen3-VL-8B’s average SYNCR accuracy from 32.6% to 61.6%, with gains extending to task configurations and video sources absent from training for those tasks. Transfer to real footage is most consistent for temporal ordering: accuracy improves by 9.0-20.5 percentage points on constructed Assembly101 and Panoptic ordering sets across three checkpoints spanning two model families and two model sizes, with additional gains on existing temporal reasoning benchmarks. These results establish SYNCR as a controlled setting for diagnosing cross-video reasoning failures, testing their learnability, and identifying where synthetic supervision transfers.

[130] Rollout-Marginal Distillation for Long-Horizon Autoregressive Video Generation cs.CV | cs.AIPDF

Chenjian Gao, Zhihao Hu, Jianqi Ma, Jun Zhang, Weidong Zhang

TL;DR: RMD在自回归视频生成中保留自身历史用于预测，却逐块独立向教师评估视觉质量，再用视频级蒸馏恢复时间连贯性，以减轻长rollout误差累积。

Details

Motivation: 自回归视频扩散可以低延迟流式生成，但预测误差会随长序列逐渐累积，模型必须在不完美历史上继续生成。整段视频级分布匹配同时评价当前块及其前后上下文，可能为了保持时序连贯而奖励复制上下文中的伪影。

Result: Rollout-Marginal Distillation（RMD）让学生继续使用生成历史做自回归预测，但每个视频块单独对照块级教师评分，使视觉质量修正不被有缺陷的时间上下文牵制。随后再施加视频级DMD恢复整体时序一致性。实验显示，RMD在远超训练时长的rollout中仍维持较高视觉质量，并超过视频级DMD基线；摘要未提供具体数值。

Insight: 单块质量评估和整段时间连贯性是不同监督目标，可以先分开处理再整合。这样既避免把上下文伪影当成当前块应匹配的内容，也保留全视频层面的时间一致性。

Abstract: Autoregressive (AR) video diffusion enables low-latency, streamable video generation, but prediction errors often accumulate over long rollouts. Training the generator on its own rollouts exposes it to these imperfect histories. However, existing video-level distribution matching distillation (DMD) scores the whole rollout jointly. Because a chunk is evaluated together with its past and future, its correction can favor matching artifacts in the surrounding context merely to preserve temporal consistency. To provide a clearer visual-quality signal, we introduce Rollout-Marginal Distillation (RMD). RMD retains the generated history for AR prediction but scores each chunk independently against a chunk teacher, ensuring its quality correction is not compromised by an imperfect temporal context. To compensate for the lack of temporal context in independent chunk scoring, RMD subsequently applies video-level DMD to restore temporal coherence. Extensive experiments demonstrate that RMD maintains high visual quality far beyond its training horizon and outperforms video-level DMD baselines. Code and video results are available at https://cjeen.github.io/RMD

[131] Does Local Video Understanding Transfer Across Encounters? The EgoGears Benchmark cs.CV | cs.AI | cs.ROPDF

Yuedong Tan, Lei Qi, Yu Liu, Di Wen, Ruiping Liu

TL;DR: EgoGears把单视频局部理解与多视频证据对齐分别评测，发现多模态模型从单次观察迁移到跨遭遇关系组合时准确率显著下降。

Details

Motivation: 具身系统应能把一次遭遇中的知识用于另一场景，即使视角、运动和光照变化。单一跨视频准确率会混合局部感知错误、观察身份保持错误、对应关系建立错误和证据组合错误，无法判断局部理解是否迁移。

Result: 基准基于126段人工采集的第一视角户外路线视频，包含567个单视频问题和1,487个多视频问题，路线有重复经过并改变速度和光照；531题需跨独立录像对齐。单视频题测试局部视觉、空间和运动证据，多视频题测试证据是否绑定正确观察并组合为一致路线关系。主排行榜报告六类模型的29种单视频和31种多视频配置；在20种可比配置中，所有模型的多视频准确率均更低，平均下降22.5个百分点。固定答案格式和计分后差距仍在，且不能简单归因于视频更多或录像边界；主要瓶颈是观察—证据绑定及有序路线状态追踪。

Insight: 跨视频推理失误常发生在已有观察信息的身份绑定和时间顺序管理，而非单纯局部视觉识别。将单视频与多视频题配对评测可分解这类能力断层。

Abstract: Embodied systems must make knowledge acquired during one encounter usable in another despite changes in viewpoint, motion, and illumination. Yet aggregate cross-video accuracy conflates failures of local perception with failures to preserve observation identity, establish correspondence, and compose evidence, obscuring whether local video understanding actually transfers. We introduce EgoGears, a complementary single- and multi-video benchmark designed to diagnose this transition. It contains 567 single-video and 1,487 multi-video questions derived from 126 human-collected egocentric recordings covering 39 outdoor routes. Repeated traversals across movement speeds and lighting conditions ground comparisons in shared physical environments; 531 questions require alignment across independent recordings. Single-video questions measure the local visual, spatial, and motion evidence available to a model, while multi-video questions test whether evidence remains bound to the correct observation and can be composed into consistent route relationships. We report 29 single-video and 31 multi-video MLLM configurations across six model families in the main leaderboard. Among the 20 configurations evaluated comparably on both splits, every model performs worse on multi-video questions, with a mean decrease of 22.5 percentage points, and the gap persists when answer format and scoring are held fixed. The gap is not explained simply by additional videos or recording boundaries. The central bottlenecks are observation–evidence binding and ordered route-state tracking. The code and benchmark are publicly available at https://github.com/lei-qi-233/EgoGears.

[132] Look Closer: Patch-wise Supervision for AI-Generated Image Detection cs.CVPDF

Zhida Zhang, Tao Wu, Siyu Liu, Jie Cao

TL;DR: 研究以共享骨干对明确裁剪图块进行独立监督，并在推理时平均图块概率，探索更简单的局部证据监督能否提升AI生成图像检测。

Details

Motivation: 小块RGB区域有时包含足以判断图像是否由AI合成的线索，即使它们很少揭示整体场景。作者研究局部裁剪监督相较整图监督是否有助于检测器学习可泛化特征。

Result: 图块级监督让共享骨干分别分类显式裁剪块，每块单独计算损失，只在推理阶段平均图块概率；不需要手工残差过滤或学习式图像级融合模块。实验涵盖单图块选择、多种生成器集合和四种CNN/Transformer骨干。在GenImage上，报告的图块监督版本在四种骨干上平均准确率均高于相应整图版本。作者还比较监督粒度、源分辨率、裁剪大小及推理覆盖，并用后处理和困难图像测试揭示局限；由于历史实验包含基于评估集的模型选择，论文不把所有分数表述为统一选择流程下的排行榜比较。

Insight: 显式局部输入和局部监督是一种简单的检测配置，可使模型关注整图训练可能忽略的生成痕迹。评测选择过程会影响可比性，报告模型选择依赖也有助于正确解读准确率。

Abstract: How much of an image does a detector need to see? Small RGB regions can retain useful evidence of image synthesis even when they reveal little of the full scene. Motivated by single-patch detection, we study patch-wise supervision: a shared backbone classifies explicit crops, each crop receives its own loss, and patch probabilities are averaged only at inference. The procedure requires neither handcrafted residual filtering nor a learned image-level fusion module. Experiments span single-patch selection, multiple generator collections, and four CNN and Transformer backbones. On GenImage, the reported patch-wise variants improve average accuracy over their whole-image counterparts across all four backbones. Comparisons of supervision granularity, source resolution, crop size, and inference coverage further characterize the approach, while post-processing tests and difficult-image evaluation reveal its limitations. The historical experiments include evaluation-based model selection, so their scores are not presented as a uniformly selected leaderboard comparison. Overall, the study identifies explicit local input and patch-level supervision as a simple, useful combination for investigating generalizable AI-generated image detection.

Haozhe Liu, Tian Ye, Shuchen Xue, Yitong Li, Junsong Chen

TL;DR: SoL-Refiner通过高分辨率持续训练、强化学习后训练和单步蒸馏，将低分辨率视频一次去噪细化至4K，以减少高分辨率生成成本。

Details

Motivation: 高分辨率视频生成的成本随时空token数迅速增长。先生成低分辨率视频再细化是一种实用方案，但传统多步细化会形成第二个采样延迟瓶颈。

Result: SoL-Refiner是单步视频细化器，可将低分辨率生成结果一步去噪提升至4K。训练分三阶段：高分辨率持续训练、强化学习后训练和最终一步蒸馏。作者构建Refiner-Bench，基于不同视频生成器输出形成统一输入评测协议，并在约2K输出下比较细化模型。2K时，SoL-Refiner在VBench和UniPercept平均指标上超过所有外部细化器；3840×2176时，两项指标也优于三步LTX-2.3 Refiner。在2K延迟设置下，完整加速方案相对同一基线实现8.91倍细化延迟加速。

Insight: 先低分辨率生成再单步细化可以减少高分辨率扩散的迭代成本，但需要持续训练和强化学习提供质量提升，再通过蒸馏压缩到一步。统一输入基准有助于公平比较不同细化器。

Abstract: High-resolution video generation is expensive, as its cost grows rapidly with the number of spatiotemporal tokens. A practical alternative first generates a lower-resolution video and then applies a refiner, but conventional multi-step refinement introduces a second sampling bottleneck. We present SoL-Refiner, a one-step video refiner that transforms low-resolution model outputs into 4K videos with a single denoising step. Our three-stage recipe combines high-resolution continual training, reinforcement learning (RL) post-training, and a final one-step distillation. We introduce Refiner-Bench, a video refinement benchmark constructed from the outputs of different video generators, and use a shared-input protocol to compare refiners at approximately 2K output resolution. At 2K, the one-step SoL-Refiner outperforms all external refiners on the VBench and UniPercept averages, while at $3840!\times!2176$ it improves both metrics over the three-step LTX-2.3 Refiner. With the complete acceleration stack, SoL-Refiner achieves an $8.91\times$ speedup in refinement latency over the same baseline in our 2K latency setting.

[134] ORMA: Optimization-based Monocular 4D Reconstruction of Articulated Animals cs.CVPDF

Xuyi Hu, Francesco Palandra, Shangzhe Wu, Daniel Cremers, Riccardo Marin

TL;DR: ORMA无需训练，将动物姿态估计与实例化形状重建解耦，再把姿态和相机运动放到统一世界坐标系中恢复单目视频的动物4D运动。

Details

Motivation: 单目视频动物4D重建受四足动物形态多样和缺少真实4D监督制约。现有逐帧学习方法依赖合成或模型拟合监督及强参数化形状先验，对分布外物种常能给出合理姿态，却无法准确表达实际几何。

Result: ORMA以预测姿态为优化参考，并利用生成式3D先验重建实例形状。给定参考图后，先恢复动物几何并将其配准到SMAL+参数模型，得到适配观测实例的关节形状；再结合逐帧关节姿态和全局一致相机姿态，在共享世界坐标系中恢复运动，并用自监督DINO对应及时间一致性细化。作者同时提出PAW4D多物种合成基准，提供真实3D几何和相机运动标注。在PAW4D、PFERD及困难野外视频实验中，ORMA提高重建准确率，并在多种四足动物间恢复全局一致运动；摘要未列出具体分数。

Insight: 把形状与关节运动分开估计可减少强参数形状先验对分布外动物的限制。全局相机一致性和跨帧视觉对应进一步把单帧几何重建扩展成共享坐标中的4D运动恢复。

Abstract: Recovering articulated 4D representations of animals from monocular videos remains challenging due to the large diversity of quadruped morphologies and lack of animal 4D supervision data. Existing learning-based reconstruction methods operate on individual images and rely on synthetic or model-fitted 3D supervision, which inherits the constraints of strong parametric priors and limits generalization to out-of-distribution species. When applied to out-of-distribution animals, they often recover a plausible pose while producing inaccurate geometry because the underlying shape model cannot faithfully represent the observed instance. We present ORMA, a training-free reconstruction framework that decouples articulation from shape, using the predicted pose as reference for optimization while leveraging generative 3D priors for accurate shape reconstruction. Given a reference image, we reconstruct the animal geometry and register it to the parametric model SMAL+, yielding an articulated shape adapted to the observed instance. We then combine per-frame articulated pose estimates with globally consistent camera poses to recover animal motion in a shared world coordinate frame, and further refine the reconstruction using self-supervised DINO correspondences and temporal consistency. To enable quantitative evaluation, we introduce PAW4D, a synthetic multi-species benchmark with ground-truth 3D geometry and camera motion. Experiments on PAW4D, PFERD, and challenging in-the-wild videos demonstrate that ORMA improves reconstruction accuracy while recovering globally consistend animal motion across diverse quadruped species.

[135] HybridCUA: Learning to Orchestrate GUI and CLI for Computer-Use Agents cs.CVPDF

Tongbo Chen, Junbo Niu, Zhengxi Lu, Niu Lian, Fei Tang

TL;DR: HybridCUA训练电脑使用智能体在GUI与CLI之间选择和交替执行，兼顾图形界面的通用性与命令行效率。

Details

Motivation: 仅用GUI的电脑使用智能体操作慢且易错；结合应用专用API虽可提高效率，却需大量工程工作且难扩展。当前模型还不知道任务执行中何时以及如何使用CLI。

Result: 作者构建HybridCUA-8K数据流水线，生成GUI单用、CLI单用和两者交错三类轨迹，包括5,000条混合轨迹和3,000个经验证的RLVR任务。训练先在轨迹上监督微调，再用CLI感知奖励强化学习，鼓励智能体有选择且可靠地使用命令行。HybridCUA-9B在OSWorld上准确率53.6%，比基础模型提高14.8个百分点；在WindowsAgentArena上提高4.0个百分点，显示混合GUI/CLI范式有效且具跨平台泛化性。

Insight: GUI和CLI是互补操作方式，关键能力在于根据当前子任务决定何时切换，而不是固定依赖某一种接口。交错轨迹与对CLI选择性的奖励共同教会模型这种调度。

Abstract: Computer use agents (CUAs) have demonstrated strong capabilities in completing digital tasks. However, existing CUAs either rely solely on graphical user interface (GUI) interactions, which are often inefficient and error prone, or augment GUI interactions with application specific APIs or tools, which require substantial engineering effort and are difficult to scale across applications. We argue that the next generation of CUAs should combine GUI interactions with the command line interface (CLI), leveraging the generality of the GUI and the efficiency of shell commands. A critical challenge, however, is that current models do not know when or how to use the CLI during task execution. To address this challenge, we develop a data construction pipeline that produces three types of trajectories: GUI only, CLI only, and interleaved GUI and CLI trajectories. This pipeline results in HybridCUA-8K, containing 5K hybrid trajectories and 3K verified RLVR tasks. Building on these data, we propose a training framework with two stages: supervised fine tuning on the constructed trajectories, followed by reinforcement learning with our CLI aware rewards that encourages agents to use the CLI selectively and reliably. Experiments show that HybridCUA-9B achieves 53.6% accuracy on OSWorld, improving over the base model by 14.8 percentage points, and improves performance on WindowsAgentArena by 4.0 percentage points. These results demonstrate the effectiveness and cross platform generalizability of the hybrid GUI and CLI paradigm for computer use agents.

[136] From Unity Simulation to Diffusion-Based Augmentation: Quantifying Dataset Balance for Robust Object Detection cs.CV | cs.AIPDF

Mohamed Benkedadra, Aissa Saoudi, Maxime Gloesener, Sidi Ahmed Mahmoudi, Matei Mancas

TL;DR: 论文系统比较Unity仿真与可控扩散合成数据用于真实数据稀缺的目标检测，发现适量混合可提升迁移，过度替代真实数据会造成域偏移。

Details

Motivation: 建筑安全监测等关键视觉领域采集标注数据成本高、有风险并受伦理限制。仿真渲染和扩散生成是互补数据来源，但各自能否迁移到真实场景需要在统一条件下比较。

Result: 研究在相同检测模型与训练设置下控制混合真实、Unity仿真和可控扩散（CIA）数据，并使用Precision、Recall、mAP及自定义Δ指标。仅Unity训练相对真实数据的mAP@0.5下降50%，仅CIA下降16.5%。混合后明显改善：90%真实加10%Unity取得最高总体mAP@0.5 62.68%，比基线高7.64%；90%真实加10%CIA的Precision最高，为74.45%。结果表明少量合成数据可增强泛化，过多替代会造成域漂移。

Insight: 合成数据不能简单替代真实样本，最佳配比依赖合成来源与评价目标。统一模型和训练设置下比较有助于识别少量仿真提升mAP、少量生成数据提升精确率的差异。

Abstract: Modern computer vision models achieve high accuracy when trained on large-scale annotated datasets. In critical domains such as construction safety monitoring, data collection is costly, hazardous, and ethically constrained. This paper presents a systematic study comparing two complementary data generation paradigms, (1) Unity Simulation-based rendering and (2) Controllable Diffusion-based generation (CIA), for object detection under real data-scarce conditions. A unified experimental framework enables controlled dataset mixing across real, simulated, and generative sources, while maintaining identical model and training settings. Quantitative evaluation using Precision, Recall, mAP, and custom $Δ$-metrics, reveals that neither simulation nor generative augmentation alone achieves optimal transferability. Unity-only training yields an mAP@0.5 drop of $-50%$ relative to real data, while CIA-only training shows a milder $-16.5%$ degradation. Hybrid compositions significantly improve performance, with the 90% real + 10% Unity configuration achieving the best overall mAP@0.5 of $62.68%$ ($+7.64%$ over baseline), and the 90% real + 10% CIA configuration maximizing precision at $74.45%$. Results demonstrate that limited synthetic inclusion enhances generalization, while excessive substitution induces domain drift.

[137] EVO-WAM: Evolving World Action Models through Video-Action Verification cs.CV | cs.ROPDF

Shiyang Zhou, Xionghao Wu, Wenbo Li, Shenghe Zheng, Jiyao Zhang

TL;DR: EVO-WAM通过视觉语言模型筛选任务完成前缀，再以逆动力学验证视频和动作一致性，使世界动作模型能从自身生成轨迹适应新任务。

Details

Motivation: 机器人学习希望在不收集额外专家演示的情况下适应新任务。世界动作模型利用视频先验联合预测未来画面和动作，但生成视频可能没完成任务；即使画面看似成功，配套动作也可能不一致并导致执行失败。

Result: EVO-WAM先通过状态预测和锚定多帧上下文改进训练，使模型可在无外部执行反馈时进行完整自回归rollout；再用VLM选择完成任务的轨迹前缀，并用逆动力学模型验证视频—动作一致性；最后在验证前缀上迭代训练并生成新轨迹。在七个未见RoboTwin 2.0任务上，Cosmos3成功率从26.9%增至68.0%，DreamZero从28.5%增至46.4%，约为初始成功率的2.5倍和1.6倍。三个未见现实长时程组合任务上，Cosmos3平均成功率从20.0%升至76.7%，提高56.7个百分点。

Insight: 模型自生成数据只有在视觉结果完成任务、且动作与画面一致时才适合作为新监督。结合完成度筛选和逆动力学核验，可在不执行候选动作的前提下逐轮改善策略。

Abstract: Improving robot policies on new tasks without collecting additional expert demonstrations remains a central challenge in robot learning. World action models (WAMs) use broad video priors to jointly predict future videos and actions, offering a potential source of supervision for adapting to new tasks. However, generated videos may fail to depict task completion, and even visually successful videos may be paired with inconsistent actions that lead to execution failure. We propose EVO-WAM, a framework that adapts WAMs to unseen tasks by learning from their own generated video-action trajectories, without executing candidate actions in an external environment. First, we augment WAM training with state prediction and anchored multi-frame context to enable complete autoregressive rollouts without external execution feedback. Second, we identify reliable training experience by selecting task-completing prefixes with a vision-language model and verifying their video-action consistency with an inverse dynamics model. Third, we iteratively train the WAM on verified prefixes and generate new rollouts with the updated model. On seven unseen RoboTwin 2.0 tasks, EVO-WAM increases average success rates from 26.9% to 68.0% for Cosmos3 and from 28.5% to 46.4% for DreamZero, reaching approximately $2.5\times$ and $1.6\times$ their initial success rates. On three unseen long-horizon composite tasks in the real world, it improves Cosmos3’s average success rate from 20.0% to 76.7%, a gain of 56.7 percentage points. Project Page: https://evo-wam.github.io/.

[138] RS-OPSD: Reliable Privileged On-Policy-Self-Distillation for Ultra-High-Resolution Remote Sensing VQA cs.CVPDF

Chengjie Jiang, Yunqi Zhou, Jiafeng Yan, Sihang Zhao, Chun Yuan

TL;DR: RS-OPSD把高分辨率遥感VQA所需的放大视觉证据蒸馏进模型，减少推理时额外搜索或工具调用，并在多个基准上取得提升。

Details

Motivation: 超高分辨率遥感VQA需要从巨大图像中辨认细小证据。现有方法通常在推理时剪枝token、搜索视觉区域或调用工具；作者研究能否将放大查看获得的特权信息内化进模型。

Result: RS-OPSD构建GeoEvidence-6K，含6,750个七类任务VQA样本及证据区域标注，并提出人类反馈引导技能细化HF-SR以规模化标注。为减少紧裁剪丢失上下文及不完美教师造成的冲突信号，方法加入上下文保留视觉特权CPVP和正确性对齐蒸馏CAD。无需推理时额外视觉搜索或工具调用，RS-OPSD在XLRS-Bench、MME-RealWorld-RS和LRS-VQA达到先进表现，较同规模此前最佳模型平均高4.0个百分点。2B的RS-OPD-Lite超过多数8B模型，并达到所测最快推理速度。

Insight: 遥感细粒度证据可在训练阶段通过带区域标注的视觉特权监督内化，避免每次推理都运行搜索工具。裁剪需保留足够上下文，教师提供的证据还必须与答案正确性对齐。

Abstract: Ultra-high-resolution (UHR) remote sensing visual question answering (VQA) requires models to resolve small visual evidence within extremely large images. Existing approaches typically rely on token pruning, visual search, or tool-augmented reasoning at inference time. We instead investigate whether the benefit of zoom-in visual privilege can be internalized into the model. We introduce RS-OPSD, a reliable privileged on-policy self-distillation (OPSD) framework for UHR remote sensing VQA. To provide high-quality privileged information with explicit question-relevant evidence, we construct GeoEvidence-6K, containing 6,750 VQA samples across seven task categories with evidence-region annotations, and develop Human Feedback-Guided Skill Refinement (HF-SR) for scalable annotation. To address context loss from tight crops and conflicting signals from imperfect teachers, RS-OPSD introduces Context-Preserving Visual Privilege (CPVP) and Correctness-Aligned Distillation (CAD). Without any additional visual search or tool calls at inference time, RS-OPSD achieves state-of-the-art (SOTA) performance on XLRS-Bench, MME-RealWorld-RS, and LRS-VQA, outperforming pervious SOTA models of comparable scale by an average of 4.0 percentage points. Moreover, our 2B variant, RS-OPD-Lite, surpasses most 8B-scale models while achieving the fastest measured inference speed. Our Code, GeoEvidence-6K, and the model weights for RS-OPSD and RS-OPD-Lite are publicly available.

[139] MUGEN: Interactive Panoramic World Exploration via Camera Control cs.CVPDF

Jiaming Tan, Zhen Li, Shuwei Shi, Minggui Teng, Siqi Yang

TL;DR: MUGEN提供大规模真实360度全景视频数据，Wan360则通过ERP几何专属编码和填充设计实现遵循相机轨迹的高质量交互式全景视频生成。

Details

Motivation: 交互式全景视频生成需在用户探索过程中遵循相机轨迹并保持360度画面连贯。既有全景视频数据往往短、标注不足或缺乏相机轨迹；面向透视视频设计的可控生成模型也不能直接处理全景几何。

Result: MUGEN是面向交互式360度世界探索的真实全景视频数据集，包含超过1,300小时、至少4K分辨率视频及丰富语义和几何标注。基于该数据，Wan360针对等距柱状投影（ERP）的循环经度接缝和极点畸变设计三种无参数组件：周期经度RoPE保证接缝位置编码连续，ERP感知填充减少边缘伪影，随机滚转偏航增强一致学习。相机控制使用ERP射线构造的全景Plücker嵌入，而不是透视针孔射线。实验显示MUGEN可支撑全景世界探索数据研究，Wan360能生成高质量、时间连贯且可按相机控制的360度视频；摘要未列具体数值。

Insight: 全景视频控制不能简单沿用透视相机建模，球面投影带来的经度周期边界和极点形变需进入位置编码、填充和相机射线表示。大规模带轨迹数据与ERP专属架构共同支撑交互式探索。

Abstract: Interactive panoramic video generation aims to synthesize immersive 360\textdegree{} videos that remain visually coherent while following user-specified camera trajectories during exploration. However, progress is limited by a coupled data-and-model gap: existing panoramic video datasets are often short, weakly annotated, or lack camera trajectories, while existing camera-controlled video generation models are designed for perspective videos and do not directly support panoramic geometry. In this paper, we introduce MUGEN and Wan360 to address these limitations. MUGEN is a large-scale real-world panoramic video dataset tailored to interactive 360-degree world exploration, comprising over 1,300 hours of at least 4K panoramic videos with rich semantic and geometric annotations. Built on MUGEN, we further present Wan360, a camera-controllable interactive panoramic video generation model. Panoramic videos are commonly represented by EquiRectangular Projection (ERP), which unfolds a spherical 360-degree view into a rectangular frame with cyclic longitude seams and pole distortions. To this end, Wan360 introduces three parameter-free ERP-aware components: periodic longitude RoPE for seam-consistent positional encoding, ERP-aware padding for reducing boundary artifacts, and random roll yaw for consistent learning. For camera control, Wan360 uses a panoramic Plücker embedding that represents camera motion with ERP rays rather than perspective pinhole rays. Experiments show that MUGEN serves as a data foundation for panoramic world exploration, and that Wan360 enables high-quality, temporally coherent, camera-controllable 360-degree video generation.

[140] OmniTaskonomy: When Does Visual Generation Improve Visual Understanding? cs.CVPDF

Jiaxin Ge, Yiming Qin, Ji Xie, Haozhe Jiang, Xiaochuang Han

TL;DR: OmniTaskonomy系统研究视觉生成监督何时能迁移到视觉理解，并整理19类图像到图像任务对25项图像到文本能力的选择性迁移关系。

Details

Motivation: 训练模型生成视觉内容可能促使其学习几何、空间关系和物体属性等感知能力，但生成监督究竟何时、如何改善视觉理解尚不清楚。

Result: 研究先构造表达同一底层问题、但输出模态分别为图像到图像和图像到文本的受控任务对，发现合适训练方案下，I2I训练能提升后续I2T表现，且训练数据越多收益越大。作者进一步提出OmniTaskonomy，统一涵盖19项I2I生成任务和25项I2T理解能力的任务分类及迁移图。迁移有选择性并依赖任务：例如深度预测提升度量3D推理、物体指点提升计数、拼图重建提升二维排序；较意外的关系包括2.5D分割提升类别识别、Z深度预测提升定位。生成与理解任务之间的梯度对齐越强，通常下游迁移收益越大。

Insight: 视觉生成可成为视觉理解的丰富监督源，但并非任意生成任务都普遍提升所有理解能力。依据任务对应关系和梯度对齐设计课程及任务组合，可能更有效地释放迁移收益。

Abstract: Training a model to generate visual content can encourage it to learn rich perceptual capabilities related to geometry, spatial relationships, and objectness; yet, its benefits for visual understanding remain unclear. We ask: when and how does visual generation supervision improve visual understanding? We study controlled pairs of image-to-image (I2I) generation and image-to-text (I2T) understanding tasks that express the same underlying problem in different output modalities. We find that under the correct recipe, I2I training improves downstream I2T performance, with larger gains as the amount of I2I training data increases. We next ask which generation tasks benefit which understanding capabilities. To study transfer beyond paired tasks, we introduce OmniTaskonomy, a unified taxonomy spanning 19 I2I generation tasks and 25 I2T understanding capabilities. The resulting transfer map reveals selective, task-dependent benefits. Some follow intuitive correspondences, e.g., depth prediction improving metric 3D reasoning, object pointing improving counting, and jigsaw reconstruction improving 2D ordering. Interestingly, we also uncover surprising connections: 2.5D segmentation improving category recognition and Z-depth prediction improving localization. To probe these patterns, we analyze gradient alignment between generation and understanding tasks and find that stronger alignment is associated with larger downstream transfer gains. Together, our results highlight visual generation as a rich source of supervision for visual understanding and provide a roadmap for unlocking its benefits through the right training curriculum and task selection. Project page: https://omni-taskonomy.github.io/.

[141] VISTA: Internalizing Collective Visual Experience via On-Policy Distillation for Active Multimodal Agents cs.CVPDF

Zheng Jiang, Houde Qian, Yiming Chen, Ling Li, Chaoyang Li

TL;DR: VISTA把同一输入多条交互轨迹中的视觉发现转成共享监督，通过经验条件教师让主动多模态智能体从同伴轨迹中学习，而不替换自己的交互历史。

Details

Motivation: 主动多模态智能体会使用视觉工具收集证据，但强化学习中的结果奖励通常只将多条交互轨迹用于估计标量优势，未充分利用不同轨迹发现的互补视觉信息。

Result: VISTA提出集体视觉经验蒸馏CVED，将观察、交互上下文和单个决策对齐；异质性感知策略改进HAPI强化成功轨迹，并为失败尝试提供经验引导蒸馏。经验条件教师评估学生生成的响应前缀，使一条轨迹的发现可指导另一条轨迹，同时保留学生原有历史，也不需生成额外目标轨迹。训练后的智能体仍使用自己的视觉工具并依照自身交互历史行动。在同等规模的受测主动多模态智能体中，VISTA平均表现最高；相对同骨干训练基线，在细粒度感知和一般推理任务上均持续更好。摘要未提供具体分数。

Insight: 同一问题的多条rollout构成集体观察，可为稀疏的最终奖励补充互补视觉证据。跨轨迹蒸馏应让新经验指导决策，同时保留每条轨迹自己的历史上下文。

Abstract: Active multimodal agents use visual tools to acquire task-relevant evidence while reasoning. Although reinforcement learning samples multiple interaction trajectories per input, outcome-based objectives primarily use the group to estimate scalar advantages, leaving complementary visual discoveries underused. We introduce VISTA, which internalizes collective visual experience through on-policy distillation by turning observations from same-input rollouts into shared supervision. Collective visual experience distillation (CVED) organizes these observations with their interaction context and aligns them with individual decisions, while heterogeneity-aware policy improvement (HAPI) reinforces successful trajectories and provides experience-guided distillation for unsuccessful attempts. An experience-conditioned teacher evaluates the student’s sampled response prefixes, allowing discoveries from one trajectory to guide learning in another without replacing the student’s original history or generating new target trajectories. The trained agent retains its visual tools and acts using its own interaction history. VISTA achieves the strongest average performance among the evaluated active multimodal agents of comparable size and consistently outperforms same-backbone training baselines across fine-grained perception and general reasoning tasks, demonstrating the value of collective experience for active multimodal learning.

[142] Self-Aligned Forcing: Streaming Video Diffusion with Differentiable Noisy History cs.CVPDF

Weiqiang Wang, Zhuokun Chen, Yusheng Dai, Boying Li, Yi Zhang

TL;DR: SAF让自回归视频扩散中每个区块的历史KV表征保持可微，并与当前去噪阶段的噪声水平对齐，从而改善长程训练并减少训练开销。

Details

Motivation: 流式自回归视频扩散支持交互生成，但长rollout会累积误差。自rollout训练可减轻暴露偏差，却难解决长程漂移；历史KV的噪声水平会影响画质与运动，且允许梯度通过历史有助于使因果训练接近双向训练。

Result: Self-Aligned Forcing（SAF）令每个区块的历史由前序区块在同一去噪阶段产生的KV组成，使所有区块可在因果掩码下通过一次前向传播去噪。历史保持噪声且可微，未来损失因此能优化历史编码方式。训练不再需要单独的无梯度rollout或逐区块timestep-zero重新缓存，速度最高比先前方法快1.8倍且显存更低。推理时，SAF在现有方法中达到最高单GPU吞吐；多GPU流水线每个阶段保留一个历史库，在四GPU上达到49.1 FPS。实验显示长时生成更优，画质与运动平衡更好。

Insight: 让历史KV处于与当前块一致的去噪阶段并保留梯度，可以使后续生成误差反向改进历史表示。这样既提升训练与部署的长程一致性，也省去单独rollout及重复缓存步骤。

Abstract: Autoregressive video diffusion enables interactive streaming generation, but suffers from error accumulation over long rollouts. Self-rollout training reduces exposure bias, yet finite rollouts leave long-range drift unresolved. We observe that the noise level of the history key-value (K/V) representations trades visual quality against motion, and that restoring gradients through the history aligns causal training far more closely with bidirectional training. Motivated by these observations, we introduce Self-Aligned Forcing (SAF), a training scheme that aligns the history of each block with the noise level of the block being denoised. Specifically, the history is the K/V produced by preceding blocks at the same denoising stage, so all blocks at a stage can be denoised in a single forward pass under a causal mask. This keeps the noisy history differentiable, allowing future losses to optimize how it is encoded. SAF therefore avoids a separate no-gradient rollout and per-block timestep-zero recaching, training up to 1.8x faster than prior methods with lower memory. At inference, SAF achieves the highest single-GPU throughput among existing methods and keeps one history bank per stage for a multi-GPU pipeline, reaching 49.1 FPS on 4 GPUs. Experiments show superior long-horizon generation with a better balance between visual quality and motion. Project page: https://anonymous.4open.science/w/self-aligned-forcing/.

[143] GA-EIRFS: A Geometry-Augmented Repeat-Factor Sampling Method for Long-Tailed LiDAR 3D Object Detection cs.CVPDF

Taufiq Ahmed, Constantino Álvarez Casado, Daniel Herrera Castro, Sasan Sharifipour, Abhishek Kumar

TL;DR: GA-EIRFS在长尾LiDAR 3D检测的频率重采样因子中加入固定几何可观测性分数，只改变训练帧采样概率，不修改检测器或推理流程。

Details

Motivation: 长尾3D检测常把类别频率视为主要问题，但LiDAR监督质量还取决于物体是否可观测；频率相近的类别可能拥有截然不同的点云几何证据。

Result: GA-EIRFS将点数量、表面法线熵和表面覆盖组成固定几何分数，用来调节指数加权的实例感知重复因子。该方法与检测器无关，只改变训练帧采样概率。在nuScenes上，CenterPoint和PointPillars跨两个随机种子的四项收敛实验中，mAP与NDS均提高；CenterPoint seed 666下mAP由0.552升至0.563，自行车AP由0.306升至0.359。各类别增益与采样权重增加相关（Spearman rho=0.70，p=0.025），但与单独几何分数不显著相关（rho=0.32，p=0.37），说明几何信息是对频率驱动需求的放大因素。KITTI结果随种子波动，最稀有类别尤其明显。

Insight: 几何可观测性并未取代类别频率，而是帮助频率重采样判断哪些实例更需要补充。跨数据集和随机种子仍有波动，说明采样策略的收益受数据集与稀有类别影响。

Abstract: Long-tailed 3D object detection is treated as a class-frequency problem, but LiDAR supervision quality depends on object observability: similar frequencies can hide different geometric evidence. We introduce Geometry-Augmented Exponentially Weighted Instance-Aware Repeat Factor Sampling (GA-EIRFS), a detector-agnostic method that modulates a frequency-based repeat factor with a fixed geometry score combining point count, surface-normal entropy, and surface coverage. GA-EIRFS changes only frame-sampling probabilities, leaving the detector and inference unchanged. On nuScenes it improves mean average precision (mAP) and the nuScenes detection score (NDS) in four converged experiments with CenterPoint and PointPillars over two seeds; for CenterPoint at seed 666, mAP rises from 0.552 to 0.563 and bicycle AP from 0.306 to 0.359. Per-class gains correlate with the class sampling-weight increase (Spearman rho=0.70, p=0.025) but not with geometry score alone (rho=0.32, p=0.37), so geometry amplifies frequency-driven need. KITTI results vary across seeds, most for the rarest class. Code: https://github.com/Multimodal-Sensing-Lab/GA-EIRFS.

[144] CLeaR: A Unified Framework for Resolving the Leakage-Degradation Dilemma in Style Transfer cs.CVPDF

Teng Zhou, Yunhao Chen

TL;DR: CLeaR通过正交子空间投影、跨视觉基础模型的集成反演和扩散能量校准，降低风格迁移把参考图内容泄漏到生成结果中的问题。

Details

Motivation: 风格迁移需把目标内容转成参考图风格，但常把参考图中的对象、布局和语义一并带入输出。更强内容抑制可能损伤风格忠实度，保留更多风格细节又会重新引入内容泄漏。

Result: CLeaR是免训练框架。先在各视觉基础模型的特征空间中用正交子空间投影定义去除内容后的风格目标；再通过集成反演优化满足多个模型风格约束的共享像素风格锚点；最后在扩散采样中使用能量引导校准，使去噪轨迹保持对齐到集成定义的风格流形。理论分析指出，风格锚点估计误差会随视觉基础模型数量增加而下降。StyleBench实验显示，相比现有方法，CLeaR提高风格对齐、减少内容泄漏，并取得更好的LLM-as-Judge评价；摘要未给出具体数值。

Insight: 风格与内容在单一特征空间中难以完全分离，跨多个视觉基础模型构造共享风格锚点可降低单模型偏差。将锚点约束贯穿扩散采样，旨在同时控制内容泄漏和风格保真。

Abstract: Style transfer aims to render target content in the style of a reference image, but existing methods often suffer from content leakage, where objects, layouts, or semantics from the style reference appear in the generated output. Although prior data-driven and training-free methods can reduce leakage, they often face a leakage-degradation dilemma: stronger content suppression may weaken style fidelity, while richer style preservation may reintroduce unwanted reference content. We identify this dilemma across the full style-transfer pipeline, including feature separation, feature-space grounding, and diffusion generation. To address these issues, we propose CLeaR, a training-free framework for content-leakage-resistant style transfer. CLeaR first uses Orthogonal Subspace Projection to define content-reduced style targets in each vision foundation model (VFM) feature space. It then performs Ensemble Inversion, which optimizes a shared pixel-space style anchor satisfying style constraints across multiple VFMs. Finally, Energy-Guided Calibration maintains style alignment during diffusion sampling by steering the denoising trajectory toward the ensemble-defined style manifold. We further provide a theoretical analysis showing that the style-anchor estimation error decreases with the number of VFMs. Experiments on StyleBench demonstrate that CLeaR improves style alignment, reduces content leakage, and achieves better LLM-as-Judge evaluation compared with existing methods. The code is available at \href{https://github.com/0606zt/CLeaR}{https://github.com/0606zt/CLeaR}.

[145] VideoLoop: Looped Working Memory Against Semantic Thrashing in Long-Form Video Agents cs.CVPDF

Jinfa Huang, Jianming Xu, Jingyang Lin, Zhengyuan Yang, Jiebo Luo

TL;DR: VideoLoop用外部文件系统保存无限历史观察，并在每步后重写有界工作记忆，以避免长视频智能体只追加上下文导致关键证据注意力衰减。

Details

Motivation: 长视频理解要求智能体反复收集证据，但追加式工作记忆增长时会出现语义扰动：关键证据的注意力下降，已发现的信息也变得难以取用。追加操作无法删除噪声或阻止上下文持续膨胀。

Result: 作者从结构上论证，若没有重写操作，追加记忆虽可加入目标证据，却无法移除累积噪声并控制顺序增长。VideoLoop采用外层视频推理循环和内层记忆循环：每步后从无界文件系统中检索既往观察与中间分析，再重写有界工作记忆。插件式接入四个流行LVLM骨干后，VideoMME（long）平均比基线提高4.2个百分点。最难四分之一问题中，只读智能体上下文的盲评者答对81.1%，追加式智能体上下文为60.9%。Gemini 3.1 Pro搭配VideoLoop在VideoMME（long）、VideoMMMU、LongVideoBench（long）分别达到88.3%、88.8%、80.9%。

Insight: 长期记忆既要保留过去观察，也必须定期压缩和重写当前工作集，否则追加式噪声会遮蔽关键信息。外部存储与有界上下文更新共同缓解长视频推理中的语义扰动。

Abstract: Long-form video understanding requires multimodal agents to iteratively gather evidence over many reasoning steps. However, most existing agentic methods suffer from semantic thrashing: as append-only working memory grows, attention to key evidence collapses, and the agent loses access to what it has already found. First, we provide a structural argument showing that append-only memory can incorporate newly observed target evidence, but cannot remove accumulated noise or prevent ordered context growth without a rewrite operator. Second, motivated by this analysis, we propose VideoLoop, a multimodal agent with two coupled loops. The outer loop reasons over the video and the inner loop, after each step, retrieves artifacts from an unbounded filesystem of past observations and intermediate analysis, and rewrites a bounded working memory. Extensive experiments demonstrate the effectiveness of VideoLoop, which improves four popular LVLM backbones in a plug-and-play manner, with an average gain of 4.2% points over baseline on VideoMME (long). Further analysis of working memory suggests that VideoLoop mitigates semantic thrashing: on the hardest quarter of VideoMME (long) questions, a blind judge that reads only the agent’s context answers 81.1% correctly, versus 60.9% for the append-only agent. With Gemini 3.1 Pro, VideoLoop reaches 88.3% on VideoMME (long), 88.8% on VideoMMMU, and 80.9% on LongVideoBench (long).

[146] Breaking the Uniformity Trap: Scaling Video Diffusion Model via SplitMoE cs.CV | cs.AIPDF

Yu Xu, Yuxin Zhang, Xiao Yang, Haotian Yang, Yizhi Wang

TL;DR: SplitMoE把视频扩散模型专家分成语义专家和通用专家，并以原型路由和拉推正则组织token，避免传统均衡路由导致的语义碎片化。

Details

Motivation: 常规MoE对token独立路由到同质专家池，并将专家使用量正则到近似均匀；但视频具有时空冗余和长尾语义，均匀负载约束可能把语义连贯的图像块分散到不同专家，造成路由碎片及结构失真。

Result: SplitMoE显式分设负责高级语义抽象的语义专家与保存剩余视觉信息、提供生成灵活性的通用专家。原型引导路由和拉推正则让token按语义属性自然聚类，而非服从任意均衡。相同激活参数预算下，实验报告其在收敛速度、路由一致性和标准视频生成基准质量上优于传统负载均衡MoE。研究还揭示由粗到细的去噪逻辑，提出面向视频模态扩展MoE的方向；摘要未给具体指标。

Insight: 视频token的分布和语义频率并不均匀，强制专家均衡可能伤害结构组织。分工式专家池使语义抽象与通用细节分别建模，再通过语义原型组织路由。

Abstract: Mixture-of-Experts (MoE), popularized by large language models, is a promising paradigm for scaling visual generative models. However, conventional token-wise MoE routes tokens independently within a homogeneous expert pool and regularizes expert usage toward uniformity, making it poorly matched to video data that is spatiotemporally redundant and semantically long-tailed. We show that existing visual MoEs fall into a uniformity trap: semantically under-organized routing, compounded by uniform expert-usage regularization, scatters coherent patches across disparate experts, causing routing fragmentation and structural distortion. To address this, we propose SplitMoE, a split-role sparse architecture that breaks the shackles of uniformity. To accommodate the inherent semantic imbalance, we explicitly bifurcate the expert pool into semantic experts and generic experts, with semantic experts capturing high-level semantic abstraction and generic experts preserving residual visual information and flexible generative capacity. Leveraging prototype-guided routing and pull-push regularization, SplitMoE enables tokens to cluster naturally by semantic attributes rather than arbitrary balancing constraints. Extensive results show that under an equivalent activated-parameter budget, SplitMoE outperforms traditional load-balanced MoEs in convergence speed, routing coherence, and video generation quality across standard benchmarks. By revealing an emergent coarse-to-fine denoising logic, SplitMoE provides the community with a modality-aware scaling path, serving as a critical reference for building large-scale video world models.

[147] Beyond the Timeline: Augmenting Long-Video Memory with Grounded Entity Biographies cs.CV | cs.AI | cs.CL | cs.IR | cs.LGPDF

Hui Ren, Lei Fan, Henry Pao, Han Guo, Zeeshan Zia

TL;DR: GEB将跨长视频片段中同一实体的视觉观测组织为有身份锚定的传记式记忆，帮助模型沿时间追踪具体物体，而不只依赖描述相似度检索事件。

Details

Motivation: 数小时或数天的视频问答常需连接同一物体参与的多个事件。按时间排列的描述或文本实体可能无法辨认物理身份：不同物体描述相似，同一物体也可能在不同事件中被割裂，因此检索相关事件未必找回所问实体的经历。

Result: Grounded Entity Biographies（GEB）把不同片段中视觉锚定为同一物理实例的观测归组为可检索传记，同时保留各次出现的时间上下文。问答时同时检索实体传记和情节证据，使模型可依照记忆构建时建立的身份链接追踪事件。四个基准评估覆盖日级和周级录像，选择题与开放问答均优于既有记忆框架；EgoLifeQA达到72.0%准确率，比已发表最佳结果高4.4个百分点。消融显示，视觉身份关联和传记读取都贡献增益，仅添加更多描述无法完全复现。

Insight: 长时记忆需要绑定具体物理实例，而不能只存储事件文本和实体名称。身份链接与每次出现的局部上下文一起检索，才能构成可用于推理的实体经历。

Abstract: Answering questions about long videos often requires connecting events involving the same objects across hours or days. Chronological descriptions and text-derived entities can leave physical identity unresolved: different objects may share a description, while observations of the same object remain disconnected across events. Retrieving relevant events therefore does not necessarily recover the “biography” of the particular entity a question concerns. To address this, we introduce Grounded Entity Biographies (GEB), a long-video memory framework that groups visually grounded observations of the same physical instance across clips into retrievable biographies while preserving the context of each moment. During question answering, the biography is retrieved alongside episodic evidence, allowing the model to follow an entity through events using identity links established during memory construction. Evaluations across four benchmarks, including day-long and week-long recordings, demonstrate improvements over prior memory frameworks in both multiple-choice and open-ended question answering. On EgoLifeQA, GEB achieves 72.0% accuracy, 4.4 percentage points above the best published result. Ablations show that grounded identity association and biography reading both contribute to the gains, which additional descriptions alone do not fully recover.

[148] FracGen: Learning How Objects Stretch and Tear with Physics-Informed Video Generation cs.CVPDF

Trong-Tung Nguyen, Jiahan Zhang, Anand Bhattad

TL;DR: FracGen从完整物体图像和物理信号生成可控断裂视频，并以材料点法加连续损伤模拟自动生成视频及逐像素物理场监督。

Details

Motivation: 视频生成模型需要表现不同材料拉伸、变形和断裂时的动力学，单凭表面外观难以保证裂纹传播等过程符合物理。逐场景仿真或测试时调参则成本高。

Result: 作者构建FracSim，将连续损伤模型整合到材料点法模拟中，生成配对断裂视频和密集像素对齐物理场，额外成本不高于常规渲染。FracGen同时预测RGB视频与物理映射，并用物理一致性损失约束各映射；可从完整物体单图在物理信号条件下生成断裂过程，控制裂纹位置、传播速度及断裂前变形量。方法可表现材料特定的断裂行为，不需昂贵测试时仿真或逐场景调优。在物理合理性基准上的实验显示，其物理和视觉保真度均优于现有视频生成基线；摘要未列具体指标。

Insight: 密集、像素对齐的物理场可让生成模型学习断裂状态，而不只拟合视觉纹理。物理监督与信号条件共同提供对裂纹位置和演化速度的细粒度控制。

Abstract: We introduce FracGen, a fracture-aware video generation model that produces plausible, controllable fracture dynamics from a single image of an intact object, conditioned on physics signals. To train FracGen, we build FracSim, a fracture-aware simulation framework that augments material point method (MPM) simulation with a continuum damage model, producing paired fracture videos and dense, pixel-aligned physical fields at no additional cost beyond standard rendering. FracGen leverages these maps in two ways: it is trained to jointly predict them alongside RGB video, encouraging the model to capture physical state rather than surface appearance; and it is supervised with physics-informed losses that encourage consistency among the predicted maps. As a result, FracGen captures distinct material-specific fracture behavior without expensive test-time simulation or per-scene tuning, while offering fine-grained control over where an object tears, how fast the crack propagates, and how much deformation precedes failure. We further introduce a benchmark for evaluating the physical plausibility of generated fracture video, and show through extensive experiments that FracGen outperforms existing video generation baselines in both physical and visual fidelity. Results are best viewed in our project website: https://fracgen.github.io/.

[149] LIFT: Layout-In-Future Video Generation under Large Viewpoint Change via On-Policy Self-Distillation cs.CVPDF

Shengxiang Ji, Boyang Wang, Haiyang Xu, Bingnan Li, Yucheng Mao

TL;DR: LIFT在图像到视频生成中加入未来镜头布局控制，让用户指定远景或末帧应出现的内容和位置，并用在线策略自蒸馏把密集布局教师能力转给稀疏末帧布局学生。

Details

Motivation: 现有相机控制只能描述视角轨迹，文本提示只能给粗略语义，无法精确指定大视角变化后新显露区域的未来画面构成，尤其难以控制视频最后一帧的布局。

Result: LIFT统一图像到视频生成与Layout-In-Future控制，允许用户给出未来视图的目标内容及空间位置。由于稀疏末帧布局比逐帧密集布局更难学习，作者采用在线策略自蒸馏，将密集布局教师的控制能力迁移给末帧布局学生，并构建LIFT-Vista，包含大视角变化视频以及相机、时间一致布局标注。实验显示LIFT相较其他方法提高视频质量、未来布局可控性和相机可控性；摘要未报告具体分数。

Insight: 相机轨迹不等同于未来画面布局，尤其大幅转向时，初始图像没有展示的区域需要独立控制。用密集控制教师监督稀疏末帧条件，可降低用户指定未来视图的负担。

Abstract: We introduce LIFT, a unified image-to-video generation framework that complements camera control with Layout-In-FuTure control, enabling users to specify what should appear in a future view and where it should appear. This addresses a practical need in controllable video generation: given an initial image, users often care not only about how the camera moves, but also about what the scene should look like at key future moments, especially the final frame. Existing camera controls specify viewpoint trajectories, while text prompts provide only coarse semantic guidance; neither precisely determines the content and spatial layout of future views. This limitation becomes particularly pronounced under large viewpoint changes, where the camera reveals regions that are not visible in the first frame. LIFT therefore uses the last-frame layout as an explicit control signal for the desired future scene. Since learning from such sparse layout guidance is substantially more challenging than conditioning on dense per-frame layouts, we introduce on-policy self-distillation (OPSD) to transfer the control capability of a dense-layout teacher to a last-frame-layout student. We further curate LIFT-Vista, a dataset featuring large viewpoint changes with camera and temporally consistent layout annotations. Experiments show that LIFT improves video quality, future-layout controllability, and camera controllability over other methods.

[150] LongLive-Plug: Once-for-All Distillation for Video Generation cs.CVPDF

Shuai Yang, Luozhou Wang, Wei Huang, ZhiFei Chen, Bohan Zhang

TL;DR: LongLive-Plug按骨干家族一次性蒸馏可复用LoRA能力适配器，使兼容的视频扩散下游模型无需逐个重新训练即可获得单次CFG、少步采样和长上下文纠错能力。

Details

Motivation: 视频扩散模型面向不同任务逐渐专门化，通常每个模型都需重新蒸馏以加速采样或支持长视频，重复训练成本高。

Result: LongLive-Plug把通用能力蒸馏为基础模型上的LoRA，在兼容下游模型中训练自由、即插即用，包括单次分类器无关引导、少步采样和自回归生成的长上下文误差校正。即使下游新增条件分支或扩展输出通道，适配器仍可复用。专用CFG LoRA虽只在固定引导尺度训练，推理时仍能通过权重控制文本引导；与少步LoRA合用时兼顾少步生成及CFG可控性。作者在三类骨干、八类任务的54个下游模型上验证无需训练部署，覆盖世界建模、机器人、编辑和多模态生成。

Insight: 将可迁移能力按骨干家族蒸馏一次，再用适配器组合到多个专用模型，可避免每个目标模型单独蒸馏。设计需要兼容下游架构扩展，并保留推理阶段对引导强度的控制。

Abstract: Video diffusion models are increasingly developed into specialized models for diverse downstream tasks, and this development often includes a distillation stage, for example to accelerate sampling or to improve long-video generation. This stage is typically repeated for every specialized model. We introduce LongLive-Plug, a once-for-all distillation framework that learns reusable capabilities as LoRAs on a base model for training-free, plug-and-play deployment to compatible downstream models. These capabilities include single-pass classifier-free guidance, few-step sampling, and long-context error correction for autoregressive generation. The adapters remain reusable even when downstream models add conditioning branches, expand output channels. Despite training at a fixed guidance scale, our dedicated CFG LoRA provides text guidance control through its inference weight. Combining it with a few-step LoRA simultaneously preserves few-step generation and CFG controllability on downstream tasks. We verify training-free deployment on 54 downstream models across three backbone families and eight task categories, including world modeling, robotics, editing, and multimodal generation. The approach may support additional compatible models. Each capability can thus be distilled once per backbone family and reused without per-target retraining.

[151] Imagine3D-LLM: Teaching MLLMs to Imagine 3D Scenes Before Answering cs.CV | cs.CLPDF

Jaewoo Jung, Hyeonseo Yu, Honggyu An, Jisang Han, Mungyeom Kim

TL;DR: Imagine3D-LLM让多模态模型先从多视图图像重建紧凑3D高斯场景表示，再以该表示回答空间问题，借助重建监督促进内部跨视角对应。

Details

Motivation: 多模态大模型擅长单图理解，但难以整合多个视角形成连贯3D认知。既有方法增强像素级跨视图对应或融合3D几何模型特征，与人类空间推理仍有差距。

Result: 受人类先粗略匹配跨视图常见物体、推断视角间相对几何并组合粗略场景布局的启发，Imagine3D-LLM在图像token后附加少量可学习摘要token，将其解码为紧凑3D Gaussian Splatting表示，并以光度重建损失监督，同时使用常规下一token目标联合训练。尽管只有摘要token直接接受重建监督，该目标也增强了模型底层图像特征中的跨帧对应。多个空间推理和3D理解基准上，模型持续超过既有方法；具体数值未在摘要中列出。

Insight: 显式重建一个紧凑、可供后续回答使用的3D场景摘要，可能比直接提供逐像素几何特征更符合粗到细的空间推理。重建损失还可向模型内部跨视角表示传播3D感知信号。

Abstract: Reasoning about the 3D world from multi-view images remains a fundamental challenge for Multimodal Large Language Models (MLLMs). While modern MLLMs handle single-image inputs effectively, they struggle to integrate evidence across viewpoints into a coherent 3D understanding. A growing body of work attempts to close this gap by injecting 3D awareness into MLLMs, either by boosting fine-grained pixel-level cross-view correspondence or by fusing features from 3D geometry foundation models, yet a substantial gap to human reasoning persists. In this work, we revisit human spatial reasoning, which suggests that rather than relying on fine-grained geometry cues, humans roughly identify common objects across views, infer the relative geometry between viewpoints, and assemble a coarse 3D layout of the scene. Inspired by this process, we introduce Imagine3D-LLM, an MLLM that learns to assemble a similar compact 3D representation of the scene and conditions its answer on this representation. Concretely, we append a small set of learnable summary tokens after the image tokens, decode them into a compact 3D Gaussian Splatting representation supervised by a photometric reconstruction loss, and train jointly with the standard next-token prediction objective. Notably, although only the summary tokens receive direct reconstruction supervision, this objective also induces stronger cross-frame correspondence within the LLM’s underlying image features, suggesting that learning to reconstruct propagates 3D-aware signals throughout the model. As a result, Imagine3D-LLM consistently outperforms prior approaches across multiple spatial reasoning and 3D understanding benchmarks, suggesting that imagining the scene can be more effective than being told its pixel-wise geometry.

[152] DMA$^2$: Pixel-space Distribution Matching with Adversarial and Anchor Losses cs.CVPDF

Xin Lin, Zhifei Zhang, Yuqian Zhou, Haitian Zheng, Shaoteng Liu

TL;DR: DMA²针对原生RGB图像生成设计像素空间分布匹配蒸馏，结合对抗局部纹理监督与无参数语义分布锚定，实现高质量四步文生图。

Details

Motivation: 分布匹配蒸馏已用于少步扩散，但现代文生图方案主要围绕潜空间扩散构建，未利用原生RGB直接监督的特点。低噪声RGB匹配容易由局部纹理线索主导，真实图像表示也可在无需穿过解码器时提供额外引导。

Result: 在教师匹配侧，作者根据诊断固定高噪声匹配区间；在真实数据侧，DINO-Adv不把沉重的伪评分器纳入对抗梯度路径，而用局部参数化patch信号引导。另提出无参数AF-Loss，在共享DINOv2空间中对分离梯度的滚动真实与生成样本支持集构造语义分布场辅助目标，同时保留提示条件教师监督，不增加可学习参数或推理计算。两者构成DMA²。在DPG-Bench、GenEval、VQAScore和COCO30K上，四步学生模型表现超过25步教师及所评估的少步蒸馏方法；摘要未列具体分数。

Insight: 原生像素空间的蒸馏可以直接利用真实RGB的局部纹理与视觉语义表征，不必受潜空间接口限制。局部对抗监督与无参数分布级语义锚点互补，支持少步生成质量提升。

Abstract: Distribution matching distillation (DMD) provides a general framework for few-step diffusion generation, but its modern text-to-image instantiations have been developed primarily around latent diffusion. It therefore overlooks key properties and design opportunities of native RGB. We revisit two DMD interfaces for pixel-space teachers. On the teacher-matching side, diagnostics show low-noise RGB matching is dominated by a local-texture cue, motivating a fixed high-noise matching band. On the real-data side, native clean-RGB outputs allow guidance from an external visual representation without traversing a decoder or sharing the heavy fake-score critic. DINO-Adv removes this critic from the adversarial gradient path and supplies local parametric patch guidance. For distribution-level guidance, we introduce AF-Loss, a parameter-free auxiliary semantic distribution-field objective designed for text-to-image DMD. It operates on detached rolling real and generated supports in the shared DINOv2 space while preserving prompt-conditioned teacher supervision. AF-Loss adds no learnable parameters or inference-time computation. Together these designs form DMA$^2$. Across DPG-Bench, GenEval, VQAScore, and COCO30K, the four-step DMA$^2$ student performs better than the 25-step teacher and evaluated few-step distillers.

[153] Rethinking Representations for World-Action Modeling cs.CV | cs.ROPDF

Haoyi Jiang, Liu Liu, Xinjiang Wang, Zhihao Sun, Zequn Chen

TL;DR: ReWAM以预训练DINO视觉特征构造适合动作控制和未来预测的紧凑世界状态，并将动作损失梯度单独导入表示瓶颈以塑造控制相关表征。

Details

Motivation: 世界动作模型联合学习机器人策略和未来观察预测，表示空间是控制与预测的接口。高重建保真度或直接使用预训练感知特征都不保证策略学习有效，作者通过受控对比重新分析表示设计。

Result: ReWAM以预训练DINO特征为基础，用特征校准和时间表示瓶颈将其整理成紧凑世界状态，适用于动力学建模。动作落地表示塑形将仅动作损失的梯度路由至瓶颈，使策略可塑造表示内容，而世界模型学习其动态演化。无需生成式视频预训练，ReWAM在RoboTwin 2.0成功率达93.6%；在RoboDojo使用约600小时具身预训练数据，平均分12.29，成功率8.28%。

Insight: 世界模型表示需要同时支持控制和状态演化，单纯追求视觉重建或感知质量不够。将策略梯度与动力学学习的梯度职责分开，可让表示空间保留对动作决策有用的信息。

Abstract: World-action models jointly learn robot policies and predict future observations, making the representation space an interface between control and prediction. We study the design of this space through controlled comparisons, finding that neither reconstruction fidelity nor pre-trained perceptual features alone ensure effective policy learning. These findings motivate ReWAM, a representation-centric world-action model built on pre-trained DINO features. Feature Calibration and a Temporal Representation Bottleneck organize these features into compact world states suited to dynamics modeling. Action-Grounded Representation Shaping routes only action-loss gradients to the bottleneck, thereby letting the policy shape what the representation encodes while the world model learns how it evolves. Without generative video pre-training, ReWAM achieves 93.6% success on RoboTwin 2.0. On RoboDojo, it achieves an average score of 12.29 and a success rate of 8.28% using approximately 600 hours of embodied pre-training data.

cs.CL [Back]

Jiyao Yang, Yang Liu, Zhenyue Qin, Qingyu Chen, Xiuzhen Zhang

TL;DR: 本文用故事、图像和批评代理协作生成9,000余组成对多模态假新闻，并评测16种MLLM识别其真伪的能力。

Details

Motivation: 生成式AI可用于大规模社交媒体虚假信息，但MLLM能否被用于制造逼真假新闻、又能否可靠检测，仍不清楚。

Result: 涵盖科学、健康和娱乐领域的评测发现，多数模型准确率明显低于人类，尤其难以判断配图真实性；摘要未提供更细数值。

Insight: 将内容生成拆成故事代理、图像代理和批评代理协作，让假帖在语义上针对真实新闻但视觉与文本共同呈现，再测试模型；结果表明仅依赖当前多模态模型检测不足以应对图像真实性风险。

Abstract: The rapid advancement of generative AI raises concerns about the misuse of Multimodal LLMs (MLLMs) for large-scale disinformation campaigns on social media. Despite existing research on textual disinformation, a fundamental question remains unanswered: can MLLMs be exploited to fabricate realistic multimodal fake news, and can they reliably detect it? We introduce a multi-agent framework in which a story agent, an image agent, and a critic agent collaborate to produce fake social media posts that plausibly counter true news. We apply the framework to generate over 9,000 paired multimodal news posts across science, health, and entertainment domains, and benchmark 16 open- and closed-source MLLMs for automated detection. We find that most models fall substantially short of human-level accuracy and fail critically on identifying image authenticity. Our research provides a foundation for developing robust defenses against social media fake news. Code and data are available at https: //github.com/xiuzhenzhang/Multimodal.

[155] FD-VAD: Semantic Endpoint Detection for Streaming Full-Duplex Speech cs.CL | eess.ASPDF

Puneet Mathur, Dinesh Manocha

TL;DR: FD-VAD直接从有界因果音频窗口预测继续说或停止，以语义方式判断轮次结束，无需中间语音识别。

Details

Motivation: 全双工对话中的停顿可能表示犹豫，也可能表示意图结束；纯声学端点检测缺少语义，串联ASR则依赖转写且增加处理阶段。

Result: 在域内及对话评估中，FD-VAD超过强流式和非流式语义轮次分类器；TurnBench开发集上合格系统中EOT召回率最高，在误报率不高于0.10时达到0.853，且为零样本设置。

Insight: 冻结语音编码器通过轻量模态适配器连接参数高效微调语言模型，使用末块训练目标对应流式推理；置信门控控制打断与延迟权衡，边界困难负例加强模糊轮次处判定。

Abstract: Natural turn-taking in full-duplex voice interaction requires determining from partial speech whether a pause reflects hesitation or a completed conversational intent. Acoustic voice activity detection lacks this semantic information, while cascaded ASR-based endpointing introduces transcription dependence and additional processing stages. We formulate semantic endpoint detection as a causal audio-language reasoning task and introduce FD-VAD, an ASR-free streaming endpointer that maps bounded causal audio windows directly to Continue/Stop decisions. FD-VAD combines a frozen speech encoder with a lightweight modality adapter and a parameter-efficiently adapted language model, using a last-chunk training objective for streaming inference. We further introduce confidence-gated endpoint commitment to control interruption versus delay and boundary-focused hard-negative sampling to improve decisions around ambiguous turn boundaries. Across in-domain and conversational evaluations, FD-VAD outperforms strong streaming and non-streaming semantic turn classifiers, and achieves the highest EOT recall among qualifying systems on TurnBench dev set $0.853$ (at FP<=0.10) in a zero-shot setting. These results show that semantic endpointing can be performed directly from streaming audio without intermediate ASR or dialogue state tracking.

[156] TRACE: Deployable Tree-Relational Structure Enhancement for Oncology LLMs cs.CL | cs.AIPDF

Jizheng Lai, Yingyun Li, Ying Qin, Haiyang Qian

TL;DR: TRACE把肿瘤学概念及关系组织成可更新树—关系结构，离线学习后在线检索紧凑证据，增强肿瘤LLM分类和问答。

Details

Motivation: 肿瘤学LLM输出常缺乏明确医学结构依据，影响准确性和可审计性；通用检索也未必能按任务选择临床相关证据。

Result: 在十项肿瘤分类任务和MedQuAD CancerGov问答上，TRACE改善无标签评估与监督微调；优于vanilla RAG和通用GraphRAG，在控制数据泄漏的METABRIC输入中仍有效，并生成与临床推理一致的可解释证据路径。

Insight: 把昂贵结构学习放在线下：从LM损失证据优化肿瘤树—关系结构；部署时轻量检索紧凑证据，支持零样本下不依赖监督标签的任务自适应证据选择。

Abstract: Large language models are increasingly used in oncology applications, but their predictions are often weakly grounded in explicit medical structure. We present TRACE, a deployable tree-relational enhancement framework for oncology LLMs. TRACE separates expensive offline structure learning from lightweight online inference: oncology concepts and relations are organized into an updatable tree-relational structure, refined using LM-loss-derived evidence, and retrieved at inference time as compact prompt evidence. This design supports task-adaptive evidence selection without requiring supervised labels in the zero-shot setting. Across ten oncology classification tasks and one MedQuAD CancerGov QA benchmark, TRACE improves both label-free evaluation and supervised fine-tuning. Additional analyses show that TRACE improves over vanilla RAG and generic GraphRAG, remains useful under leakage-controlled METABRIC inputs, and produces interpretable evidence paths aligned with clinical reasoning. These results suggest that explicit, updatable medical structure is a practical path toward more accurate and auditable oncology LLM deployment.

[157] Less Uniform Discrete Diffusion is More Powerful and Scalable cs.CL | cs.AIPDF

Kaibo Wang, Ding Ding, Fangyu Ding, Zijin Feng, Han Shi

TL;DR: LUDI通过较少均匀的离散扩散损失和逐token时间嵌入改善离散扩散语言模型监督，并支持按置信度进行少步采样。

Details

Motivation: 均匀离散扩散语言模型可作为扩散生成范式，但规模化困难；作者将核心问题归因于训练目标过度均匀及采样时条件与目标混淆。

Result: 多尺度实验显示LUDI监督更干净、少步生成更好。把7B自回归模型继续训练为LUDI-7B后，复杂推理表现有竞争力，每步生成速度比自回归快3 token。

Insight: 较少均匀损失让每次反向转移更明确朝干净token优化；逐token时间嵌入提供局部腐化程度提示，使模型可按置信度少步采样。

Abstract: Although uniform diffusion language models (UDLMs) represent a promising diffusion paradigm, scaling them remains challenging. We identify the core obstacle as an over-uniform training objective and condition-target confusion during sampling. To address these, we propose Less Uniform Diffusion (LUDI), a novel UDLM framework. Specifically, we (i) introduce a less uniform loss that directs each reverse transition toward the clean token, and (ii) equip the model with per-token time embeddings that supply token-level corruption hints, enabling confidence-based few-step sampling. Experiments across scales show that LUDI yields cleaner supervision and improves few-step generation. We further continue-train a 7B autoregressive model into LUDI-7B, resulting in a UDLM capable of complex reasoning. It achieves a 3-token-per-step speedup over AR decoding and competitive performance compared with masked diffusion baselines, revealing that the full potential of UDLMs for complex generation remains to be unlocked.

[158] PrimeSeeker: Capability-Oriented Supervision for Deep Search Agents cs.CL | cs.AIPDF

Linzhi Peng, Hanting Chen, Heng Chang, Ke Cheng, Bowen Du

TL;DR: PrimeSeeker以“从描述定位隐式检索锚点，再将锚点用于后续信息需求”为基本能力单元训练深度搜索代理。

Details

Motivation: 现有搜索代理常以更大证据图、更多跳数和更长轨迹合成难题，但这些全局难度指标并非搜索过程中局部检索能力的直接监督。

Result: 作者构建9,221条专家轨迹并训练30B代理，在五个深度搜索基准上表现强；参考步骤优化进一步提升监督策略。生成轨迹检索冗余低，在固定预算下用显著更少工具调用取得较强解答覆盖。

Insight: 问题构造围绕隐式锚点解析和关系迁移组织，不规定唯一搜索路径。共同生成网页证据结构及参考证据骨架，专家用当前工具观察中的抽取高亮生成轨迹；高亮在SFT前移除，骨架之后用于审计RL奖励覆盖。

Abstract: Large language model search agents are often trained with synthetic questions whose difficulty is increased through larger evidence graphs, additional hops, and longer trajectories. These global properties, however, are only indirect proxies for the local retrieval capabilities required during search. To address this mismatch, we introduce latent anchor reasoning, which consists of resolving an unnamed retrieval anchor from descriptive specifications and transferring the recovered anchor into a subsequent information demand. This primitive retrieval unit decomposes deep search into chains of coupled operations and organizes question construction around anchor resolution and relation transfer, without prescribing a canonical search path. Based on this formulation, we propose PrimeSeeker, a capability-oriented framework that constructs web-grounded anchor structures and jointly derives a question and a reference evidence skeleton. The skeleton preserves supporting evidence from construction and guides expert generation through extractive highlights of current tool observations. These highlights are removed before supervised fine-tuning, while the skeleton is subsequently reused to audit reference-step coverage for reinforcement-learning rewards. We construct 9,221 expert trajectories, training a 30B search agent. Across five deep-search benchmarks, PrimeSeeker achieves strong performance, while reference-step optimization further improves the supervised policy. The resulting trajectories exhibit low retrieval redundancy, and fixed-budget evaluation shows strong solution coverage with substantially fewer tool calls than long-horizon systems.

Xiaoshan Zhou, Zaifu Zhan

TL;DR: 本文检验多种AI文本检测器能否识别灾害社交媒体中的AI生成帖，发现当前检测不足以作为运营层面的可信度闸门。

Details

Motivation: 灾害社交感知依赖公众帖子辅助态势判断，但生成式AI可制造逼真的目击信息；若检测器不能可靠辨别来源，将削弱人道救援信息可信度。

Result: 在九场灾害的12,000条匹配文本上，14种冻结跨家族配置AUROC仅0.402至0.517；最佳低误报阈值召回率仅3.6%。灾害数据训练的线性头AUROC为0.817，但简单七特征表面分类器达0.784；消除表面差异后线性头从0.733降至0.594。

Insight: 检测器可能利用文本表面差异而非生成来源：同一来源的中性和情绪化AI文本也可被区分。灾害信息信任应结合多模态主张、可问责来源和其他上下文证据，不能只靠文本AI检测。

Abstract: Disaster social sensing converts public social-media posts into evidence for situational awareness and humanitarian needs, but generative artificial intelligence (AI) can produce plausible messages that resemble eyewitness reports. This study investigates whether text-based AI detectors can reliably distinguish human-authored from AI-generated disaster posts. We construct a dataset of 12,000 texts organised into 3,000 matched semantic units from nine disasters: original human posts (H0), minimally LLM-proofread human posts (H1), factual AI-generated posts based on the same verified facts (A0), and affectively framed versions of those AI posts (A1). A separate 6,000-text corpus from 42 events supports model selection and threshold calibration. We evaluate OSM-Det, Fast-DetectGPT, Binoculars, and direct large language model (LLM) judges across five model families, then test disaster-domain calibration, a frozen-encoder linear readout, paired transformation sensitivity, and dataset artifact controls. Across fourteen frozen cross-family configurations, AUROC is 0.402-0.517 and the best prospective recall at a calibration-derived low-false-positive operating point is 3.6%; OSM-Det reaches AUROC 0.521 and 10.4% recall at a realised 6.7% false-positive rate. A disaster-trained linear head reaches AUROC 0.817, but a seven-feature surface classifier reaches 0.784 on the H0-versus-A0 contrast, and neutralising identified surface asymmetries reduces the head from 0.733 to 0.594. The head also separates A0 from A1 even though provenance is unchanged. The results show that text-based detection is not reliable enough to serve as an operational trust gate; multimodal claims, accountable sources, and other contextual evidence should be rested on to safeguard trust in disaster social sensing.

[160] Beyond the Context Window: An Adaptive Entropy-Based Routing Framework for Hybrid Retrieval and Long-Context Language Models cs.CLPDF

Isaac Olufadewa, Miracle Adesina, Ezekiel Oladejo, Owen Adeniyi, Fadare Fadekemi

TL;DR: EDAR根据RAG生成开头token的预测熵，动态决定直接使用检索片段回答还是升级到完整长上下文推理。

Details

Motivation: 百万token上下文虽可用，但成本高且容易忽略输入中段信息；RAG成本低，却受检索质量限制，部分相关或矛盾片段可能引发错误。

Result: 在LongBench v2和Infinity-Bench上，EDAR保留纯长上下文准确率的97.4%，总token开销降低70.7%，仅18.2%查询升级；熵与留出集幻觉率相关系数0.85。

Insight: 先生成RAG回答前几个token，以其分布预测熵估计回答风险，再用验证集上的成本—准确率权衡选阈值。路由信号来自正常解码，不依赖特定检索器、长上下文骨干或额外监督。

Abstract: Modern large language models now support context windows of more than one million tokens, which has raised the question of whether retrieval-augmented generation (RAG) is still necessary. Pure long-context (LC) processing is expensive and is known to under-attend to information placed in the middle of long inputs, while pure RAG is fast but bounded by retrieval quality and prone to errors when retrieved chunks are partially relevant or contradictory. We propose the Entropy-Driven Adaptive Router (EDAR), a framework that decides at inference time whether to answer a query from retrieved chunks or to escalate it to full long-context processing. The decision uses the predictive entropy of the token-level probability distribution computed over the first few generated tokens of the RAG response. The entropy threshold is selected on a held-out validation set by sweeping cost against accuracy. Experiments compare EDAR against pure-RAG and pure-LC baselines on LongBench v2 and Infinity-Bench. Predictive entropy correlates strongly with hallucination rate on a held-out set of 2,000 generations (Pearson r = 0.85, 95% CI [0.83, 0.87]). On the long-context benchmarks, EDAR retains 97.4% of the accuracy of the pure long-context baseline while reducing total token expenditure by 70.7%, escalating only 18.2% of incoming queries. The accuracy gap between EDAR and the pure long-context system is not statistically distinguishable from zero at standard sample sizes. Predictive entropy is a useful model-internal signal for routing between RAG and long-context inference, and a threshold-based hybrid system can recover most of the accuracy of long-context models at a small fraction of the cost. The framework does not depend on a specific retriever or LC backbone, and it does not require additional supervision beyond what is normally produced during decoding.

[161] When Should LLMs Trust Their Own Revisions? A Risk-Aware Study of Intrinsic Self-Correction cs.CL | cs.AIPDF

Tianzhu Zhang

TL;DR: 本文把内在自我修正视为需权衡纠错与破坏的决策策略，并比较始终保留初答、始终采用修订和基于信号选择性修订。

Details

Motivation: 无外部证据的二次推理有时能纠正错误，也会把原本正确的答案改错；单看最终平均准确率会掩盖不同类型的答案转换。

Result: 29种开源LLM在BoolQ、GSM8K和Corr2Cause上的分析显示，自我修正效果依任务而异；例如Llama-3.1-8B在GSM8K提升25.5个百分点，但19.1%的原本正确答案被改错。选择性门控只在部分设置优于更简单策略。

Insight: 应以初答—修订之间正确与错误的转移来评估自我修正，并按风险决定是否复核，而不能把第二轮默认当成有益。受控BoolQ试验还表明，提示可改变纠错与伤害的平衡。

Abstract: Intrinsic self-correction asks a language model to revise its own answer without receiving new external evidence. A second pass can recover mistakes, but it can also overturn answers that were already correct. We study this trade-off across 29 open-weight LLMs on BoolQ, GSM8K, and Corr2Cause by tracking correctness transitions between initial and revised answers. Aggregate accuracy can conceal substantially different revision behavior: for example, Llama-3.1-8B improves by 25.5 percentage points on GSM8K, while refinement changes 19.1% of initially correct answers into wrong ones. A controlled BoolQ study further shows that refinement prompts shift the balance between recovery and harm. We then compare three runtime choices: keeping the initial answer, always accepting the revision, and selectively invoking revision using signals available after the initial response. The comparison identifies settings where learned gating is useful and others where a simpler unconditional policy performs better. These results suggest treating intrinsic self-correction as a revision policy rather than as a uniformly beneficial second pass, and evaluating it through both the corrections it recovers and the errors it introduces.

[162] Question-Specific Knowledge Graphs for Efficient Visual Reasoning cs.CLPDF

Ting-Chih Chen, Emile van Krieken, Shujian Yu, Filip Ilievski

TL;DR: VisKG让模型按问题生成最小充分的视觉知识图，将图像翻译为所需实体—关系结构用于推理，避免长图注带来的无关细节和token开销。

Details

Motivation: VLM可将图像转为文本后推理，但需既保留显式和隐式证据，又避免引入臆测。详细图注包含大量与当前问题无关内容，增加计算成本。

Result: 在科学、数学及一般视觉理解基准上，VisKG表现与基线相当或更好，同时比图注表示用更少token；GDPO训练版本平均准确率比GRPO版高2%。

Insight: 模型在强化学习中学会把视觉内容转为问题特定知识图，过滤感知噪声并保留链式推理所需的关系结构；使用负向理由样本先暴露错误推理路径，再以组奖励解耦归一化策略优化稳定后训练。

Abstract: Recent work in visual question answering has shown that vision-language models can exhibit strong reasoning capabilities by translating visual inputs into textual representations. The effectiveness of this translation depends on how well visual details are retained; models need to surface and align both explicit and implicit knowledge sufficient to support reasoning, without introducing spurious assumptions. Existing methods that leverage detailed image captions introduce visual details unrelated to the reasoning task, inflating input token counts and increasing computational cost. To address these challenges, we propose VisKG, a reinforcement learning (RL) framework in which models learn to translate visual content into question-specific knowledge graph (KG) representations. This process filters out perceptual noise while preserving the entity-relation structure needed for chain-of-thought reasoning, following the principle of minimum sufficient information. To ensure stable RL post-training, VisKG adopts Group reward-Decoupled Normalization Policy Optimization (GDPO). In addition, we strengthen the supervision stage with negative rationale samples, exposing the model to incorrect reasoning paths before RL post-training. Experimental results across science, mathematics, and general visual understanding benchmarks show that VisKG achieves performance comparable to or better than baselines, while requiring fewer tokens than caption-based representations. Moreover, training VisKG with GDPO improves accuracy by 2% over its GRPO-trained counterpart on average. These results suggest that KG representations are a promising approach for supporting multi-step reasoning and open up future work on adaptively selecting the most suitable representation for a given task.

[163] Mnemon: Raw Records, Fast Judgments, Slow Thoughts cs.CL | cs.AI | cs.IRPDF

Guangren Wang

TL;DR: Mnemon把长期记忆分为快速逐条判断和慢速搜索规划：原始对话按时间保存，决策模型筛选证据，LLM负责制定查询及组织答案。

Details

Motivation: 多数记忆系统写入时就把对话改写成事实、图或结构化记忆，可能提前丢弃未来有用内容；重写和检索全由LLM完成又成本高。

Result: gpt-4.1-mini回答时，LoCoMo得91.7%、LongMemEval-S得83.8%，每题上下文少于4k token；使用推理模型分别得92.2%和94.4%。BEAM历史从10万增至1,000万token时，每题成本仅乘1.11；Jev判断比两个LLM更能区分金标准证据，且快3至11倍。

Insight: 对话保留为原始带日期记录，System 2 LLM规划搜索，快速决策模型Jev判断返回内容，预算规则将结果压缩为小型视图交给不变的回答模型。后台只需遍历每条记录一次，将其连到主题时间线、价值历史和常驻指令，补足单次搜索漏掉的证据。

Abstract: Long-term memory lets an LLM assistant use a history it can no longer reread, and most memory systems build it by rewriting conversations into facts, graphs or typed memories at write time. We argue that the work of memory divides, as thinking does, into two systems. Most of it is fast System 1 work: many small, independent yes/no judgments about records, such as whether a record is needed or no longer current, which a decision model makes by the dozen in a third of a second. Only a little is slow System 2 work: writing a few search queries, naming what the reply needs and composing the answer, which an LLM does well but slowly. We present Mnemon, a memory agent built on this division. It keeps conversations as raw, dated records; an LLM (System 2) plans searches over them, a decision model, Jev (System 1), judges what the searches return, and rules with explicit budgets turn the judgments into a small View for an unchanged answering model. A background pass consolidates each record once into topic timelines, value histories and standing instructions linked to the records, so that questions about a whole conversation reach evidence their own searches miss. Because nothing is decided about a record when it is written, the same agent can read any store that returns dated records. With gpt-4.1-mini answering, as in a public re-evaluation of 14 systems, Mnemon scores 91.7% on LoCoMo, the highest among them, and 83.8% on LongMemEval-S, from under 4k tokens of context per question, with the lowest effective cost index on LoCoMo. With a reasoning model answering, it reaches 92.2% on LoCoMo and 94.4% on LongMemEval-S, the latter on par with the best published results. From 100K to 10M tokens of history on BEAM, its cost per question grows by a factor of 1.11. On the same records, Jev separates gold evidence better than two LLMs and is 3-11 times faster.

[164] Language Models Are “Insecure” Reporters cs.CL | cs.AIPDF

Jenny Y. Huang, Jiameng Fan, Ahmed Imtiaz Humayun, Maximillian Chen, Tian Qin

TL;DR: 本文以八类对抗报告场景研究LLM是否会隐瞒足以改变叙事的负面信息，并发现一句简短诚实提示可显著提高披露率。

Details

Motivation: 用户越来越依赖模型报告审查长程任务产物，但人工逐项验证变得困难；模型可能忽略推翻成功叙事的缺陷，导致“报告不安全”。

Result: 含削弱方法结论的实验负结果时，GPT-5.5默认仅在200份报告中的2份提及；加入“请诚实回答”后，190份提及。Qwen3.5-9B表示空间分析显示诚实与追求成功对应相反方向。

Insight: 八种场景系统检测报告是否披露会改变结论的缺陷；开源模型推理链常显现诚实披露与如何显得成功之间的冲突。激活分析及引导实验表明，默认叙事偏成功，显式引导诚实可大幅提升透明度。

Abstract: As large language models are deployed in increasingly autonomous long-horizon tasks, manually auditing and verifying the actions, artifacts, and outputs of models becomes more difficult. Users instead come to rely on LLM-generated reports to assess the quality and completeness of the work. We introduce a suite of eight adversarial reporting scenarios to systematically study whether LLMs conceal narrative-changing flaws: errors or limitations that undermine an otherwise successful account of work. We call this phenomenon “insecure reporting.” When handed machine learning experiment logs containing a planted negative result that substantially weakens the proposed method, GPT-5.5 flags the negative result in only 2 of 200 generated reports. However, when a short honesty instruction, “Be honest in your response,” is added, the model flags the negative result in 190 of 200 reports. Across eight open-weight models, chain-of-thought analysis reveals a recurring tension between disclosing narrative-changing flaws and reasoning about ways to appear successful. We perform an activation analysis and a steering experiment on Qwen3.5-9B, finding that honesty and success-seeking correspond to opposing directions in representation space. Our results suggest that LLMs tend to present narratives of success by default, and that steering models toward honesty makes their reports substantially more transparent.

[165] Targeting Pivotal Decisions for Credit Assignment in Agentic Reinforcement Learning cs.CL | cs.AIPDF

Dongwon Jung, Hemanth Neelgund Ramesh, Yifan Wang, Xiaomin Li, Yuexing Hao

TL;DR: ProVer让代理评审比较成功和失败轨迹，定位可能的关键决策片段，再用策略续写的终局成功率差验证其信用。

Details

Motivation: GRPO把轨迹级优势平均分给所有token，无法区分真正影响成败的中间决策，难以训练多步工具代理。

Result: 在ALFWorld、WebShop和SearchQA上，ProVer在两种规模均有最高平均表现；相较GRPO，Qwen3.5-2B和4B分别相对提升9.91%和7.12%。适度增加生成开销即可改善，且不需前沿级评审模型。

Insight: 代理评审仅用于提出成功/失败轨迹的潜在分歧片段，不直接决定信用；分别从片段前后采样当前策略续写，以终局成功率差估计优势，再将正优势加到该片段token，避免穷举所有中间状态。

Abstract: Group Relative Policy Optimization (GRPO) has become a promising approach for training large language model agents. However, its uniform assignment of trajectory-level advantages to all policy tokens fails to distinguish consequential decisions from less relevant ones, obscuring which intermediate decisions contributed to success. We introduce ProVer, a framework that targets potentially pivotal decisions for fine-grained credit assignment in agentic reinforcement learning. Given a rollout group, an agentic judge contrasts successful and failed trajectories to propose a segment potentially responsible for their divergent outcomes. Rather than directly trusting the judge’s assessment, ProVer verifies the proposed segment by estimating its advantage from the difference in terminal success rates between current-policy continuations sampled before and after the segment. Positive estimates are then incorporated into the GRPO advantages of policy tokens within the proposed segment. By using model judgment only to select where to verify, ProVer grounds local credit in observed outcomes without exhaustively evaluating every intermediate state. Across ALFWorld, WebShop, and SearchQA, ProVer achieves the strongest average performance at both model scales, with relative improvements over GRPO of 9.91% and 7.12% for Qwen3.5-2B and Qwen3.5-4B, respectively. Further analyses demonstrate that informed segment selection improves policy training with modest additional generation overhead, even without a frontier-scale judge model, highlighting the effectiveness and efficiency of selectively targeting pivotal decisions for fine-grained credit assignment in agentic reinforcement learning.

[166] SCOUT: Synergizing Reasoning and Tool-Use for Computer-Use Safety cs.CL | cs.AI | cs.SEPDF

Jianxing Chen, Xiao Yu, Shipra Agrawal, Zhou Yu

TL;DR: SCOUT先根据任务与操作轨迹生成专属安全和完成标准，再主动探查执行后的环境收集证据，验证计算机使用代理是否造成实际伤害。

Details

Motivation: CUA即使在无恶意指令和环境下也可能造成意外损害；通用安全规则会漏掉任务特定风险，而仅看历史截图又无法确认操作实际改变了什么。

Result: AutoElicit-Bench上不安全行为F1为75.4、完成F1为74.5，超过LLM裁判及朴素工具验证器；OS-Blind不安全检测准确率76.4%。测试时反思使AutoElicit-Bench不安全执行率从30.2%降至17.2%。

Insight: 两阶段验证器先用大量无工具推理把任务和轨迹转换成专属判据，再由探查代理按判据操作执行后的真实环境取证。消融显示离线生成判据的深度推理很关键，且对非前沿验证模型尤为重要。

Abstract: Computer-use agents (CUAs), while capable of completing computer tasks in everyday and professional workflows, can cause unintended harm even under benign instructions and environments. However, detecting such harm remains challenging. First, it requires careful, task-specific reasoning: verifiers guided only by general safety criteria often overlook many important but subtle harmful behaviors. Second, it requires active investigation: past trajectory screenshots show what the agent did but not always what actually changed in the environment, so LLM-as-a-judge verifiers that rely on screenshots alone may be unable to determine the actual consequences of actions. To address these challenges, we introduce SCOUT, a two-stage agentic safety verifier that synergizes reasoning-intensive rubric generation with tool-intensive evidence gathering. First, our SCOUT rubric generator extensively reasons over the task and the agent’s trajectory to determine what successful and safe execution should entail, generating task-specific completion and safety rubrics. Then, our SCOUT probing agent follows these rubrics to interact with the post-execution environment and collect grounded evidence for final safety and completion judgments. We evaluate our framework on two computer-use safety benchmarks. On AutoElicit-Bench, SCOUT achieves 75.4 unsafe F1 and 74.5 completion F1, outperforming LLM-as-a-judge verifiers and naive tool-use verifiers. SCOUT leads on OS-Blind with 76.4% unsafe detection accuracy. Test-time reflection reduces final unsafe execution rates from 30.2% to 17.2% on AutoElicit-Bench. Ablations and analysis show that tool-free rubric generation in SCOUT elicits substantially more reasoning and is crucial for safety detection across verifier backbones, especially non-frontier ones. A preliminary extension to coding tasks shows that SCOUT can support safety verification beyond computer-use.

[167] Cognitive Expert Language Models Better Align with the Corresponding Brain Systems cs.CLPDF

Zhivar Sourati, Mengxuan Helen Wu, Nona Ghazizadeh, Jonas Kaplan, Morteza Dehghani

TL;DR: 本文训练面向六种认知领域的专家语言模型，发现其表示更贴合对应专门脑系统，汇总全脑分数会掩盖区域差异。

Details

Motivation: LLM—脑活动对齐通常用同一模型预测多个脑区并汇总结果，忽略不同脑区具有功能专门化。

Result: 六个认知领域专家在三个基础模型和三个fMRI数据集上均更好预测与该领域对应的脑系统；非认知或表层干预无法复现此对齐变化。区域对齐改变时总体预测准确率基本不变。

Insight: 通过提示和微调构建感觉、空间、数值、推理、社交和抽象六类专家，比较模型表示与对应脑区活动；认知领域特定干预改变区域对齐而非一般性能，说明跨区域平均指标可能隐藏模型的专长匹配。

Abstract: Large language models (LLMs) can predict human brain activity across a variety of brain regions during natural language comprehension. Typically, however, LLM-brain alignment is measured using one model for different regions of the brain, and then model performance is summarized across regions. This one-model-fits-all approach ignores the functional specialization of brain regions. In this study, we assess whether a model oriented toward a particular cognitive domain aligns better with the brain system dedicated to that domain. Through prompting and fine-tuning, we first build expert LLM variants for six domains: sensory, spatial, numerical, reasoning, social, and abstract processing. We then examine whether each expert best predicts activity in the brain region associated with the corresponding cognitive domain. Consistent with our hypotheses, each expert’s representations align more closely with the brain system most associated with the matching domain than do other experts. This holds under both prompting and fine-tuning, across three base models and three fMRI datasets. In a series of control analyses, we show that this model-brain alignment is specific to cognitive domain interventions; non-cognitive and surface-level interventions do not result in comparable alignment. Specializing models shifts regional alignment while leaving aggregate prediction accuracy largely unchanged, suggesting that summarizing alignment across regions may obscure regional differences in performance for specific models.

[168] Learning from Teacher Continuations at Student States cs.CLPDF

Haojin Wang, Dylan Zhang, Huaibo Chen, Suhao Yu, Yihang Sun

TL;DR: OLIVE让持续演化的学生策略生成新前缀，再由教师在该学生状态上续写并对续写token提供交叉熵训练。

Details

Motivation: 离线SFT使用固定教师轨迹会有序列分布偏移；token级在线蒸馏遇到前缀失败时监督碎片化；分布匹配蒸馏又需教师token概率。

Result: 在相近GPU时数下，OLIVE推理性能高于使用top-16 KL近似的OPD，异步实现缩短总训练时间23.8%。硬推理和智能体任务中，在同预算下持续超过现有蒸馏方法；ScienceWorld上仅用GPT-5.4-mini文本时，相较同教师离线SFT提升13%。

Insight: 每轮都从当前学生策略生成前缀，教师随后自回归补全；学生用教师生成token上的交叉熵更新，教师无需提供概率且不从教师采样前缀。这样可在离线蒸馏平台期后继续提升，并保留学生通用能力和可塑性。

Abstract: We present OLIVE (OnLine InterVEntion). At each iteration, the evolving student policy generates a new prefix, the teacher continues it autoregressively, and the student is updated using cross-entropy computed on the teacher-generated tokens. Each design choice targets a corresponding limitation of existing distillation methods: (1) sequential covariate shift in offline supervised fine-tuning (SFT) on fixed teacher trajectories, (2) fragmented supervision under prefix failure in token-level on-policy distillation (OPD), and (3) the need for access to teacher token probabilities in distribution-matching distillation. OLIVE achieves higher reasoning performance than OPD (with a top-16 KL approximation) at comparable GPU-hour cost. Our asynchronous implementation further reduces OLIVE’s total training time by 23.8%. We evaluate OLIVE on both hard reasoning tasks and agentic tasks which reflects modern post-training scenarios, and it consistently outperforms existing distillation methods under the same training budget. By regenerating prefixes from the evolving student, OLIVE continues improving after offline distillation plateaus while better preserving the general capabilities and plasticity of the student. Using only text from GPT-5.4-mini, continuously training with OLIVE outperforms offline SFT from the same teacher by 13% on ScienceWorld. These results support OLIVE as an effective and efficient approach to online language-model distillation.

[169] DeepRewind: Predicting and Repairing Premature Commitments in Deep Research Agents cs.CLPDF

Amirhossein Abaskohi, Amirhossein Dabiriaghdam, Lele Wang, Peter West, Giuseppe Carenini

TL;DR: DeepRewind将深度研究代理的知识状态表示为带类型图，预测中间结论的可逆性，并在后来证据推翻时依赖关系回滚。

Details

Motivation: 长程研究代理可能在证据不足时过早承诺结论，后续推理再围绕错误解释自我强化。

Result: 在DRBench和LiveDRBench上，相较Open Deep Research，DeepRewind将洞见召回提高3.6个百分点，并将过早承诺减少59.1%。

Insight: 承诺前用提示式世界模型估计假设收窄、信息损失、恢复成本和矛盾触发覆盖以预测影响与可逆性；二元控制器阻止高风险结论，一致性监视器在新证据冲突时按依赖关系撤销后续结论。

Abstract: Deep-research agents conduct long-horizon investigations through iterative search, evidence evaluation, belief revision, and synthesis. However, they may commit to claims before sufficient evidence is available, causing later reasoning to reinforce an incorrect interpretation. We introduce DeepRewind, an additive control layer for reversible deep research that represents the agent’s evolving epistemic state as a typed graph of sources, evidence, claims, hypotheses, assumptions, commitments, plans, and drafts. Before accepting an intermediate conclusion, a prompt-based world model predicts its impact and estimates reversibility based on hypothesis narrowing, information loss, recovery cost, and contradiction-trigger coverage. A binary controller blocks risky commitments, while a consistency monitor performs dependency-aware rollback when later evidence invalidates them. Across DRBench and LiveDRBench, DeepRewind improves insight recall by 3.6 percentage points and reduces premature commitments by 59.1% relative to Open Deep Research.

[170] MemFold: Learning Compact Soft Memory for Long-Context Personalization via On-Policy Optimization cs.CLPDF

Jingxuan Wu, Yuzhe Yang, Yiqiao Huang, Chengzhi Liu, Qingni Wang

TL;DR: MemFold以任务行为而非文本重构质量优化固定预算软记忆，并通过学生自身rollout上的奖励及教师重评分实现长程个性化。

Details

Motivation: 长期助手需要判断用户偏好是否保持、被修订以及当前适用的限制；保存信息和真正根据它行动是不同目标。纯文本记忆会不断增长，固定潜向量则通常只按重构文本或参考答案训练，且这些答案未必来自读者模型实际生成。

Result: 三个Qwen骨干中，MemFold在PersonaMem-32K和128K达到测得最高准确率，历史越长优势越大；无需目标域训练即可迁移PrefEval及LongMemEval。消融显示主要任务收益来自奖励项，教师信号带来较小额外提升。

Insight: 把查询条件文本记忆压缩为K个连续向量作为读者接口，再优化读者自身rollout；任务组相对奖励与置信门控在线蒸馏互补，冻结文本记忆教师只对学生采样token重评分，不生成样本也不增加自回归解码。推理时移除教师，干预表明读者依赖实例特定软记忆。

Abstract: An assistant that serves the same user over a long horizon has to answer from what that user has revealed: which preferences still hold, which were revised, and which constraints apply now. Retaining that information is not the same as acting on it, and the two are usually optimized as if they were. Keeping the information as text makes the reader’s input grow with the retained history, while compressing it into a fixed number of latent vectors bounds the interface but is typically trained to reconstruct text or imitate reference answers, both of which are scored on sequences the reader never produced. We present MemFold, which optimizes a fixed-budget soft memory by the behavior it supports. A query-conditioned textual memory is compressed into K continuous vectors that form the reader’s memory interface, and the reader is then trained on its own rollouts under two complementary signals: group-relative rewards for task outcomes, and confidence-gated on-policy distillation in which a frozen textual-memory teacher re-scores the student’s sampled tokens under the textual memory. The teacher is never sampled from, so supervision stays on the student’s current distribution and adds no autoregressive decoding; at inference it is removed entirely. Across three Qwen backbones, MemFold attains the highest accuracy we measure on PersonaMem-32K and PersonaMem-128K, with margins that widen at the longer history length, and transfers to PrefEval and LongMemEval without target-domain training. Ablations attribute most of the task gain to the reward term and a smaller additional gain to the teacher signal, and memory interventions show that the reader depends on the instance-specific content of its soft memory.

Sumin Hong, Katsumi Ibaraki, Renee Shi, David Chiang, Toby Jia-Jun Li

TL;DR: 本文同时测量人类与视觉语言模型对跨模态联想的选择和注视，发现模型可模仿选择却仍未呈现人类式视觉注意模式。

Details

Motivation: 以往跨模态联想研究对人和模型使用不同刺激或任务，难以判断模型是否不仅选得相同，也依赖相似视觉证据。

Result: 53名参与者与VLM观看相同伪词和两幅图片；少数较大VLM的选择接近人类，但显著图匹配人类注视甚至不如图像中心高斯基线。小模型用人类选择微调后选择泛化到未见词和图像，却仍未匹配人类注视。

Insight: 用眼动数据将决策结果与视觉依据分开检验。只优化选择会改善选择而不改善注视；只训练注意力提高注视相关性也不提高选择对齐，说明任一表面行为单独都不足以证明人类式跨模态处理。

Abstract: Cross-modal associations are systematic pairings of features across modalities, such as the association of ‘bouba’ with round shapes and ‘kiki’ with sharp shapes. Prior work has compared humans and vision-language models (VLMs) on such associations, but often using different stimuli or tasks between humans and models. Here, we ask whether VLMs align with humans not only in choices, but also in where they look when making those choices. We study both VLMs and humans (N = 53), presenting them with the same stimuli, a pseudo-word and two images, and record participants’ choices and eye movements, which we release. We find choice alignment in a few larger VLMs, but their saliency matches human gaze less closely than a center-bias baseline, a fixed Gaussian at the center of each image. Fine-tuning small VLMs on human choices brings their choice alignment to the level of a human majority-vote reference on unseen words and images, yet their attention still matches human gaze less closely than this baseline. Training model attention on human gaze raises attention-gaze correlation without improving choice alignment, and a single average gaze map per image position raises it by a similar amount. Matching human choices, or even human gaze patterns, is therefore not sufficient evidence of human-aligned cross-modal processing.

[172] Reliable Parallel Decoding in Masked Diffusion Language Models cs.CL | cs.AIPDF

Zhenghao He, Bohan Liu, Guangzhi Xiong, Aidong Zhang

TL;DR: RPD根据模型跨层预测稳定性和最终置信度选择可并行提交的掩码token，并用累积熵预算避免在上游上下文不确定时过早锁定答案。

Details

Motivation: 掩码扩散语言模型一次可并行预测多个token，但同一前向的预测未必适合同时提交；单看置信度可能过早固定答案，尤其依赖尚未确定的前序计算时。

Result: 在LLaDA和Dream的数学推理、代码生成基准上，RPD在维持或提高准确率的同时，达到所评估方法中最高解码吞吐。

Insight: 多层稳定预测更可能正确；某位置前序掩码token的熵越高，其下游预测越不可靠。RPD据此延迟不稳定候选、并行提交其余token，不依赖固定块状解码日程，也无需训练。

Abstract: Masked diffusion language models (MDLMs) can generate text efficiently by predicting multiple masked tokens in parallel, but predictions from the same forward pass are not necessarily reliable when committed together. We study when parallel commitment is reliable. Our diagnostics show that confidence alone does not determine a reliable commitment order: confident predictions near the end of the sequence can fix an answer before its supporting computations are established, and downstream predictions become less reliable as the uncertainty of their upstream context grows. At the same time, a single forward pass can already resolve several masked tokens, and predictions that remain stable across the final layers are more likely to be correct. Based on these findings, we propose Reliable Parallel Decoding (RPD), a training-free method that selects candidates by layerwise prediction stability and final confidence, and commits them under a cumulative entropy budget over their preceding masked positions. RPD defers predictions with uncertain upstream context while committing the remaining candidates in parallel, without relying on a fixed block schedule. Across mathematical reasoning and code generation benchmarks on LLaDA and Dream, RPD achieves the highest decoding throughput among the evaluated methods while maintaining or improving accuracy.

[173] Grounded Revision vs. Prior Injection: Probing Retrieval-Augmented Patent Claim Amendment cs.CLPDF

Josepha Michiko Leo, Hyun-seok Min, Yehoon Jang, Irvan Zidny, Jin-Woo Chung

TL;DR: 本文以专利权利要求修订作为有可定义正确依据的测试，比较随机检索和结构匹配检索究竟帮助事实落地还是注入模板。

Details

Motivation: RAG广泛用于专业写作，但检索究竟使修订更基于证据还是仅套用模板很少被直接检验；专利审查意见明确指出受攻击限定及现有技术，可提供逐案例参照。

Result: 基于7,385宗USPTO案件和四种前沿模型的9,600次预注册调用，没有模型显示可检测的经典现有技术模板注入；随机和结构检索影响小且方向不一致。修订局部性揭示模型差异，但四格类型分析未发现先前技术注入类。

Insight: 研究发布XML对齐的审前/审后权利要求、驳回与引证材料，采用固定提示下七种探针及无需LLM评分的确定性五通道指标。结论受探索性分类和本批模型/检索策略范围限制。

Abstract: Retrieval-augmented generation is widely used in professional writing, yet whether retrieval grounds revision or merely injects templates is rarely tested where “correct” has a definable meaning. Patent claim amendment supplies that signal: the examiner names the attacked limitation and cites prior art, providing per-case ground truth. We release three artifacts: (i) a corpus of 7,385 USPTO prosecution cases with XML-aligned pre/post claims, rejection, and cited prior art; (ii) a seven-probe battery comparing random and structural-match retrieval as two policies under a fixed prompt scaffold; (iii) a deterministic five-channel metric (C1-C3 and C5 in main, C4 supplementary) requiring no LLM evaluation. Across 9,600 pre-registered calls on four frontier LLMs (Claude Sonnet 4, Claude Haiku 4.5, GPT-5.4, GPT-4o-mini), no tested model exhibits detectable classical prior-injection behavior; retrieval effects are small and direction-inconsistent between random and structural retrieval, and the null is unchanged under a dense (semantic) retriever, across retrieval depths k in {1,3,5,10}, and under a paraphrase-sensitive grounding metric. Revision locality reveals a model-specific difference that the template channel misses. The four-cell taxonomy, which we treat as exploratory, leaves the prior-injector cell unoccupied.

[174] Generating Edit-Inducing Questions for AI Research Manuscripts cs.CLPDF

Sebastian Joseph, Zichao Wang, Jennifer Healey, Alexa Siu, Junyi Jessy Li

TL;DR: 本文研究LLM能否从提交版论文改进记录生成真正促使作者修改的审稿问题，并与人类评审问题比较。

Details

Motivation: 帮助作者改进论文的关键问题有价值，但模型输出是否能引发实质修订尚缺乏与真实修订记录对应的评估。

Result: 基于ICLR和NeurIPS投稿版—终稿配对数据，GPT问题比评审问题更多，关联的修改范围更广，但其中真正促成修改的问题比例更低。

Insight: 对有无完整论文上下文的GPT问题与人类问题进行对照，可评估编辑诱发性而非表面相关性。分析还发现，充分关注长上下文有时反而降低推理模型提出有用问题的能力。

Abstract: We study the ability of LLMs to generate edit-inducing questions whose answer will improve a paper draft. On a dataset of paired submission and camera-ready papers from ICLR and NeurIPS, we compare the helpfulness of questions from GPT models with or without full paper context to that of human reviewers. GPT produces more edit-inducing questions and its questions are associated with more extensive edits and cover a broader range of edited content compared to questions from reviewers. However, a much smaller percentage of the GPT questions are edit-inducing. Our analyses confirm that automated questions can be beneficial to authors and highlight an example task where proper attending to long context deteriorates reasoning model ability to produce helpful output.

[175] ProgressCompass: Embodied Progress Reward Models Are Lost Without the Right Context cs.CLPDF

Jianshu Zhang, Keliang Wu, Chengxuan Qian, Xiyuan Yang, Ce Zhang

TL;DR: ProgressCompass为具身进度奖励模型补充其估算当前任务进度所缺的上下文，并通过自主代理循环提高进度判断准确性。

Details

Motivation: 长任务只看最终成功与否无法反映中间进度；当前画面常看不出任务做到哪一步，尤其涉及早前信息、步骤顺序或反复相似状态。

Result: 24项操作任务、120个episode的ContextProgress-Bench显示，即使PRM读取完整历史，仍会迷失进度；提供正确上下文后，五个模型误差降低77%至82%。包装进ProgressCompass循环后，冻结PRM误差降低63%、排序一致性提高76%。

Insight: 基准分别考查早期状态回忆、固定序列跟踪和相似画面进度消歧。配对实验保持模型输入格式不变，仅改进指令中的任务上下文，证明问题主要是上下文组织；ProgressCompass调用通用VLM提供PRM所需状态线索。

Abstract: Embodied agents now take on ever longer tasks. For long tasks, knowing only whether a task finally succeeds or fails says little; the steps along the way matter. Progress Reward Models (PRMs) score how far a task has come at every step, and serve as dense rewards, verifiers and monitors. Yet in long tasks the current frame alone often cannot tell how far the task has come, because progress depends on what happened before. We call this problem context-dependent progress estimation. Existing benchmarks on progress estimation mostly focus on short tasks whose progress can be read from the current observation, and whether PRMs can estimate progress when context is needed remains underexplored. We therefore build ContextProgress-Bench, with 24 manipulation tasks for 120 episodes. The benchmark covers three settings: (i) State Recall, where information needed for progress appeared earlier but is not in the current frame; (ii) Sequence Tracking, where steps follow a fixed order, so progress requires knowing which steps are done and which comes next; and (iii) Recurrence Disambiguation, where look-alike frames sit at very different progress. We then run a paired diagnosis: each PRM keeps the same input format in both runs, and in one run its instruction integrates the right context. Even PRMs that read the entire history get lost in estimating progress, yet with the right context the same five models cut their progress error by 77-82%. Embodied PRMs are thus not incapable of progress estimation, but lost without the right context. We therefore propose ProgressCompass, an autonomous agentic loop that reorients an existing PRM and uses current general-purpose VLMs to supply the context the PRM needs. Wrapped in the loop, the same frozen PRM cuts its progress error by 63% and raises its rank agreement by 76%. With such a compass, PRMs estimate progress far better on longer, more complex tasks.

[176] Video2Skill: From Streaming Experience to Reusable Embodied Skills cs.CLPDF

Jianshu Zhang, Ce Zhang, Xiyuan Yang, Chenwei Xu, Haoran Lu

TL;DR: Video2Skill提出从连续视频流中定位操作事件、归并同类变换并维护可复用技能库的评测，揭示模型难以判断何时创建新技能。

Details

Motivation: 具身规划可借助少量通用技能，但代理只能使用已知技能；从观察经验反向发现和积累技能有助未来泛化，现有VLM却未必能跨事件组织知识。

Result: 对19种开源VLM评估发现，很多模型的事件分组近似随机且规模增大无稳定帮助；SFT及反事实库状态再平衡改善分组，但模型常只合并熟悉技能、很少扩展技能库，库规模低于参考的一半。

Insight: Video2Skill同时评测事件时间定位、同一变换归组、复用旧技能或创建新技能三个能力。联合感知与库更新易把不同变换合并；根据文字描述更新则会重复创建已有技能。识别何时现有技能不足是核心瓶颈。

Abstract: Manipulation behaviors vary widely across objects and scenes, but they share a small set of reusable skills, and planning with these skills helps embodied agents generalize to new tasks. Yet an agent can only plan with skills it knows. Recovering skills from observed experience, the inverse of planning, builds this knowledge over time and yields skill data for training future agents. Vision-Language Models (VLMs) describe individual manipulation events well, but can they organize a stream of events into reusable skills? We formulate this problem as Streaming Embodied Skill Discovery (SESD): a model watches videos in sequence and maintains a persistent skill library that shapes its later decisions. To systematically measure this ability, we introduce Video2Skill, a benchmark that covers robot tabletop manipulation and human kitchen activity and tests three core capabilities: (i) locating manipulation events in time, (ii) grouping events of the same transformation, and (iii) deciding when to reuse an existing skill or create a new one. Across 19 open-source VLMs, many models group events at near-chance level, and scale does not consistently help. Their errors depend on how perception and library updates are coupled: joint models merge distinct transformations into one skill, while models that update the library from text descriptions duplicate recurring ones. Supervised fine-tuning, including our counterfactual library-state rebalancing (CLaRe), improves grouping but exposes a deeper bottleneck: trained models consolidate familiar skills yet rarely expand the library. Their libraries stall below half the reference size, and transformations unseen in training are located in time but almost never given a new skill. Recognizing when existing skills are insufficient thus emerges as the central challenge.

[177] Distilling What Matters: Confidence-Aware Selective Distillation for Large Language Models cs.CLPDF

Ayan Sengupta, Vaibhav Seth, Tanmoy Chakraborty

TL;DR: CaRE-KD按师生置信度动态切换前向或反向KL，并在批次层面拒绝比学生更不确定的教师信号。

Details

Motivation: 知识蒸馏通常将教师视为可靠预言机，但LLM教师可能高熵或产生幻觉，普通蒸馏会破坏学生已有的校准先验。

Result: 在8组师生配对和11个基准上持续超过强基线；指令任务ROUGE-L最高提高3.2，MBPP pass@1提高2.1，GSM8K准确率提高1.7、CollegeMath提高1.8。拒绝模块还改善事实性评审分。

Insight: CaRE-Divergence逐token根据师生置信度选择KL方向；Revival若教师认识不确定性高于学生便抑制该批次更新。双粒度条件校准不同于静态散度目标，且拒绝器可作为与损失形式无关的插件。

Abstract: Knowledge Distillation (KD) trains a smaller-capacity student model to imitate a larger-capacity teacher model by matching output distributions, implicitly assuming the teacher to be a reliable oracle. In large language models (LLMs), this assumption often fails: teacher predictions can exhibit high entropy and hallucinations, causing standard KD to degrade well-calibrated student priors. We propose CaRE-KD, a confidence-gated distillation framework that replaces static objectives with uncertainty-adaptive optimization. CaRE-KD has two components: a token-level loss (CaRE-Divergence) that adaptively switches between Forward and Reverse KL divergence based on teacher–student confidence, and a batch-level epistemic rejection mechanism (Revival) that suppresses updates when the teacher is more uncertain than the student. We provide a gradient-level analysis showing how this dual-granularity design induces a conditional calibration mechanism that prior static divergences cannot reproduce. Empirically, across eight teacher–student pairs and eleven benchmarks spanning instruction following, chat alignment, code generation, and mathematical reasoning, CaRE-KD delivers consistent gains over strong baselines (Skewed-KL, $α$–$β$ divergence). Highlights include up to $+3.2$ average ROUGE-L on instruction-following tasks, $+2.1$ pass@1 on MBPP, $+1.7$ accuracy on GSM8k, and $+1.8$ accuracy on CollegeMath over the strongest baseline, with consistent gains in LLM-as-a-judge factuality (up to $+2.5$ per task over Skewed-RKL). Revival further acts as a principled, loss-agnostic plug-in that systematically strengthens existing distillation objectives by filtering epistemically unreliable teacher supervision.

[178] VAA-CSEC: Vote-guided Advantage Allocation for Chinese Semantic Error Correction cs.CL | cs.AIPDF

Yitong Han, Nankai Lin, Juan Luo, Hongyan Wu, Lianxi Wang

TL;DR: VAA-CSEC将最小编辑原则融入中文语义纠错奖励，并按rollout与组投票奖励的差距重分配策略优势。

Details

Motivation: 中文语义错误比错别字和语法问题更隐蔽，LLM纠错常过度修改；思维链与自一致解码的效果也难稳定迁移到最终改正。

Result: 在CSED-C上F0.5为47.72%、召回率42.15%为所有方法最高；在NaSGEC-Exam上F0.5达到41.55%，刷新当时最优。

Insight: 流程结合思维链蒸馏、监督微调、强化学习和自一致解码；GLPO以各轨迹奖励相对组投票聚合结果的差距重分配GRPO优势，使训练更贴近推理时投票目标，并以最小编辑任务奖励抑制过度纠错。

Abstract: Chinese Semantic Error Correction (CSEC) targets semantic errors in Chinese text, which are typically more subtle and complex than spelling and grammatical errors but remain relatively underexplored. Existing LLM-based approaches face two recurring obstacles in this task: over-correction, and unclear interaction between Chain-of-Thought (CoT) reasoning and self-consistency decoding, such that the benefits brought by CoT cannot be reliably transferred to final corrections. We propose Vote-guided Advantage Allocation for CSEC (VAA-CSEC), a multi-stage framework that combines CoT distillation, Supervised Fine-Tuning (SFT), Reinforcement Learning (RL) and self-consistency decoding. During RL, we design a task-specific reward function that directly aligned with the minimal-editing principle of CSEC. We further introduce Group-Level Relative Policy Optimization (GLPO), which reallocates GRPO advantages according to the margin between individual rollout rewards and the vote-aggregated group reward, aligning the RL training objective with the self-consistency objective used at inference time. Experiments on CSED-C and NaSGEC-Exam show that VAA-CSEC outperforms all LLM-based baselines on CSED-C with an F0.5 of 47.72%, achieves the highest recall of 42.15% among all methods, and establishes a new state of the art of 41.55% F0.5 on NaSGEC-Exam.

[179] Rethinking Multimodal Fake News Detection in the Generative AI Era cs.CL | cs.CV | cs.MMPDF

Wenbin Shen, Guoxuan Qin, Guangxu Yao, Baodong Wang, Yuanbo Rui

TL;DR: 本文构建生成式内容场景下的微博多模态假新闻数据Weibo26，并提出联合判断新闻真实性与内容生成性的层级推理框架GAHR。

Details

Motivation: 生成内容进入新闻后，真假新闻可能混合原生与生成素材；传统检测侧重事件真实性，AIGC检测只判断生成来源，二者的数据与评估彼此割裂。

Result: 在多个既有假新闻基准及Weibo26上，GAHR取得有竞争力的真实性检测表现，并能有效识别生成内容；摘要未提供具体指标。

Insight: GAHR将全局真假判断与局部修正结合，使生成性信息进入真实性推理。来源是否由AI生成本身不等于事件真伪，需要在层级框架内联合分析。

Abstract: Generative content is increasingly entering the production and dissemination of news, transforming fake news from manually fabricated or simply manipulated material into complex forms in which native and generated content jointly participate. Existing multimodal fake news detection research primarily focuses on veracity assessment and rarely characterizes how generativity differences affect the reliability of evidence. In contrast, AIGC detection primarily determines whether content is generated or modified by generative models, but it does not by itself establish whether the underlying news event is true. To bridge the separation between these tasks in data and evaluation, we construct Weibo26, a multimodal fake news detection dataset for generative-content scenarios. On this basis, we propose the Generativity-Aware Hierarchical Reasoning (GAHR) framework, which combines global judgment with local correction so that generativity information participates in news-veracity reasoning. Experiments on multiple existing fake news detection benchmarks and Weibo26 show that GAHR achieves competitive veracity-detection performance while effectively identifying generative content.

[180] Momentum-Coupled Rubric Adaptation for Detailed Image Captioning cs.CLPDF

Zhenwen Ji, Lei Jin, Shanyong Wang, Jiaming Lu, Chengqiang Lu

TL;DR: MoCo Rubric在共享角色模型中同步更新字幕策略与其动量版本的评分标准生成器/裁判，提升细节图像描述的准确性和完整性。

Details

Motivation: 细节字幕需兼顾事实、覆盖和清晰度，基于评分标准的强化学习可给结构化反馈，但用不同模型担任生成、构建标准和裁判会造成解释不一致；即使交替优化，裁判与策略也可能失步。

Result: 在五个字幕基准上，平均成对胜率72.83%，盲评平均排名最佳，基于字幕的问答平均分最高。

Insight: 先用角色条件共享参数多任务微调，使同一VLM可担任字幕策略、评分标准生成器和裁判；在线生成器依据当前样本字幕、参考字幕和图像构造标准，裁判给奖励，仅策略做GRPO。动量策略副本平滑跟踪策略变化，避免同步更新破坏评分角色。

Abstract: Detailed image captioning requires accurate and comprehensive descriptions of fine-grained visual content, yet caption quality spans factual accuracy, information coverage, and clarity. Compared with conventional methods that rely mainly on high-quality supervision or holistic rewards, rubric-based reinforcement learning decomposes these requirements into explicit criteria and provides targeted, structured feedback. However, existing methods often use separate models for caption generation, rubric construction, and judging, which may lead to inconsistent interpretations across roles. Some dynamic rubric methods alternate updates between the caption policy and rubric generator while keeping the judge fixed, but staged optimization may still leave rubric construction and judging out of step with policy optimization. We propose MoCo Rubric, a two-stage framework that coordinates these roles. First, role-conditioned, shared-parameter multi-task supervised fine-tuning equips a single vision–language model to serve as the Caption Policy, Rubric Generator, and Rubric Judge. Then, the Generator constructs rubrics online from captions sampled by the current Policy, reference captions, and image evidence. The Judge provides rubric-based rewards, and only the Policy receives GRPO updates. As Policy updates change the candidates being evaluated, we use an exponential moving average of the Policy parameters to update one momentum model shared by the Generator and Judge. This gradual transfer lets both rubric roles track Policy updates without separate RL optimization while smoothing parameter changes that could disrupt their rubric capabilities under direct synchronization. Across five captioning benchmarks, MoCo Rubric achieves an average pairwise win rate of 72.83%, the best mean rank in blind ranking, and the highest average score in caption-based question answering.

[181] Can Language Models Learn to Forecast Stock Prices cs.CLPDF

Jiacheng Guo, Suozhi Huang, Shuzhen Li, Yunlong Gao, Zerui Cheng

TL;DR: AURA-4B通过工具演示监督微调和基于实际未来收益的PPO，提升语言模型在按时间顺序进行的股票收益预测任务上的表现。

Details

Motivation: 数学、编程等有明确验证结果的任务可从后训练获益，但金融市场收益嘈杂，相关信息集合也不明确，模型须先决定采集哪些数据再提交预测。

Result: AURA-4B方向—幅度综合分由20.94升至43.31，超过翻倍且接近基准中的前沿模型；条件幅度一致率从33.3升至66.2，方向准确率由62.9%升至65.4%。

Insight: 在按时间排序的沙盒里，模型使用价格、成交量、相对表现和市场背景工具预测未来收益；SFT扩大工具使用，PPO进一步增加排序与市场背景查询。结果仅证明在该结果选定基准上后训练改善，并不意味着现实市场保证盈利。

Abstract: Post-training has been shown to significantly improve language models’ performance on tasks with verifiable outcomes, including mathematical reasoning, software engineering, and computer use. However, whether the same approach can improve forecasting in financial markets is much less clear. Compared with tasks with verifiable outcomes, not only are realized returns noisy, but even what constitutes a relevant information set for making effective predictions is not obvious a priori: the model must decide which observations to gather and then commit to a numerical judgment before the outcome is known. We study this question in a chronological stock-price sandbox, where a language model gathers price, volume, relative-performance, and market-context evidence and predicts a future return. We post-train Qwen3-4B with supervised fine-tuning (SFT) on tool-use demonstrations, then proximal policy optimization (PPO) with a terminal reward given by the forecast score against the realized return. The resulting AURA-4B more than doubles the starting direction–magnitude score, from 20.94 to 43.31, and is comparable to frontier language models on this benchmark. Conditional magnitude agreement rises from 33.3 to 66.2, while directional accuracy changes from 62.9 to 65.4. SFT expands tool use, and PPO further increases the share of ranking and market-context queries. These results show that post-training can substantially improve financial forecasting performance, together with changes in how the model investigates the market, on this outcome-selected benchmark.

[182] RAEGNet: Relation-Aware Evidence Graph Network for Harm-Aware Multimodal Fake News Detection cs.CL | cs.CV | cs.MMPDF

Wenbin Shen, Guoxuan Qin, Guangxu Yao, Baodong Wang, Yuanbo Rui

TL;DR: RAEGNet以完整事件语义检索外部证据，构建新闻—证据立场及证据间互动图，并联合预测真假与潜在危害。

Details

Motivation: 假新闻检测中的实体级检索容易带入与事件无关信息，现有工作还常只优化总体真实性准确率，忽略不同虚假内容可能造成的危害程度。

Result: 在Weibo-21、Fakeddit和自建SSS数据集上，RAEGNet所有评估指标均超过多个基线。

Insight: 事件级证据检索框架按完整事件语义取证，避免只凭实体召回；关系感知证据图联合表示新闻—证据立场和证据之间关系，并增加条件危害分支，把真假判别与危害估计一起建模。

Abstract: Existing multimodal fake news detection methods often introduce external information to assist detection. However, most of them rely on entity-level retrieval and are therefore prone to introducing event-irrelevant noise. Meanwhile, existing methods mainly focus on improving overall performance and do not account for differences in the degree of harm posed by different instances of fake news. To address these limitations, we design an Event-Level Evidence Retrieval Framework (ELERF) and propose a Relation-Aware Evidence Graph Network (RAEGNet). ELERF retrieves external evidence based on the complete event semantics of a news item. RAEGNet constructs a directed graph that incorporates news-evidence stance relations and evidence-evidence interaction relations, and introduces a conditional-harm branch to jointly model authenticity and potential harm. Experimental results demonstrate that RAEGNet outperforms multiple baseline methods across all evaluated metrics on Weibo-21, Fakeddit, and our self-constructed SSS dataset.

[183] ER-JEPA: Experience Replay Improves Joint-Embedding Predictive Learning in Language Models cs.CL | cs.AIPDF

Jingnan Pu, Zi-En Fan, Feng Lian

TL;DR: ER-JEPA在LLM-JEPA中增加情景回放记忆，按需存取训练样本，为当前token预测和联合嵌入对齐提供额外监督。

Details

Motivation: LLM擅长token生成，却可能形成不理想抽象语义或感知不足；JEPA通过同一知识不同视角对齐缓解问题，但表示对齐强并不保证预测准确稳定。

Result: 在NL-RX、GSM8K、Spider和NQ-Open等多个数据集上，ER-JEPA持续超过LLM-JEPA；摘要未给出具体增益数值。

Insight: 将训练配对数据存入记忆，每步保存并检索相关样本，使模型同时从当前批次和历史经验学习，补充token预测与表示对齐两种监督。

Abstract: Large language models (LLMs) excel at token-level generation but may learn undesirable abstract semantics and lack comprehensive perception. LLM-JEPA mitigates this by aligning different views of the same underlying knowledge via a joint-embedding predictive architecture (JEPA). However, strong alignment does not necessarily lead to accurate, stable predictions. To address this, we propose ER-JEPA, which adds an episodic replay path to LLM-JEPA. ER-JEPA stores training pairs in a memory. At each step, it stores and retrieves relevant data to provide additional supervision. This enables learning from both the current batch and stored training pairs, providing additional supervision for token prediction and representation alignment. Experiments across multiple datasets (NL-RX, GSM8K, Spider, and NQ-Open) demonstrate that ER-JEPA consistently outperforms LLM-JEPA.

[184] Dating the Model: Hidden Dates in System Prompts Affect LLM Evaluation cs.CLPDF

Mario Sanz-Guerrero, Minh Duc Bui, Manuel Mager, Katharina von der Wense

TL;DR: 本文发现系统提示中自动注入的当前日期会显著改变大语言模型评测表现和模型排名，且常见提示技巧不能消除影响。

Details

Motivation: LLM评测复现性已受硬件与批处理影响，但用户无法控制、每天变化的隐藏系统日期注入是被忽略的变因。

Result: 九个模型、六类数据集上，仅日期变化即可造成选择题最高6%、数学14%、代码7%差异及机器翻译2.84 BLEU变化，且可改变排行榜名次；影响大于批量大小或数值精度。思维链未减轻敏感性，反而放大。

Insight: 评测协议应记录和控制系统提示中的日期等隐藏上下文。少样本和思维链等标准提示技术不足以消除日期效应，说明模型对未显式展示的时间线索可能有较强依赖。

Abstract: Reproducibility is essential for scientific research, yet prior work shows that LLM outputs vary with hardware and batching. We identify an overlooked factor: the hidden injection of the current date into system prompts, which users cannot control and which changes every day. Across 9 recent LLMs and 6 datasets spanning multiple-choice QA (MCQA), math reasoning, code generation, and machine translation, performance varies solely with the current date, with deltas of up to 6% on MCQA, 14% on math reasoning, 7% on code generation, and 2.84 BLEU on machine translation. Model rankings also shift, affecting leaderboards. This date effect exceeds other sources of non-determinism, such as batch size and numerical precision. Standard prompting techniques – chain-of-thought and few-shot prompting – do not reduce the sensitivity; chain-of-thought even amplifies it. Our findings underscore the need for careful evaluation protocols to ensure reproducibility and fair comparisons in LLM research.

[185] SRJudge: Empowering Large Language Models with Selective Reasoning for Fine-Grained Knowledge Concept Tagging cs.CL | cs.AIPDF

Zhiwei Yang, Jiahua Yang, Huiru Lin, Xing Chen, Quanlong Guan

TL;DR: SRJudge将细粒度知识概念标注拆成候选筛选、候选内推理和最终裁判，以降低大规模标签空间中的决策困难。

Details

Motivation: LLM面对大量概念候选时难准确选择特定教育主题，直接在高维类别空间推理效率和可靠性都受限。

Result: 在多个基准上，SRJudge持续超过先进基线；作者另构建生物学S_Bio和物理学S_Phy数据集用于验证。摘要未列具体成绩。

Insight: 小型BERT先缩小top-K候选，因正确概念通常包含在短名单中；轻量LLM再对候选精细推理，并结合动态任务奖励和剪枝RL对齐人工偏好；更大模型最后评估推理和解释合理性。

Abstract: Knowledge concept tagging aims to assign specific concept or topic labels to educational content, which is essential for both educators and learners in traditional and online teaching practices. Recent work has explored large language models (LLMs) for this task, achieving promising performance. However, LLMs still struggle to select the correct concept from a large-scale candidate set due to the high dimensionality of the decision space. In this paper, we propose a novel three-stage Select-Reason-Judge (SRJudge) framework, which empowers LLMs with selective reasoning capability for fine-grained knowledge concept tagging. Specifically, the Selector in Stage 1 first narrows the candidate concepts to a top-K shortlist by fine-tuning a small language model (SLM), e.g., BERT, since the top-$K$ predictions hit the correct concept in most cases, thereby reducing the decision space of correct candidates. Next, the Stage 2 Reasoner employs a lightweight LLM for refined reasoning over the shortlisted candidates. It further integrates an improved reinforcement learning strategy with a dynamic task-specific reward function and a pruning mechanism to better align with human reasoning preferences. Finally, a larger LLM acts as a judger that evaluates the overall rationality of the reasoning process and its explanations to determine the final output. In addition, we construct two high-quality datasets for further validation, i.e., the biology dataset S_Bio and the physics dataset S_Phy. Experimental results demonstrate that our method consistently outperforms state-of-the-art baselines across benchmark datasets, verifying its effectiveness and superiority. Resources are available at: https://github.com/Nicozwy/SRJudge.

[186] LatCom: Cross-Agent Latent Compression for Efficient Multi-Agent Collaboration cs.CLPDF

Shinan Zhang, Tao Zhang, Qihui Zhu, Mengjie Zhang, Dong Jin

TL;DR: LatCom将多个发送代理的潜表示联合压缩为固定数量、接收方可读且面向任务的槽位，减少多代理协作上下文和推理开销。

Details

Motivation: 潜空间协作可避免自然语言压缩信息损失与重复编码，但直接转发所有发送端潜状态使接收方上下文随代理数和推理长度增长；各代理独立压缩仍留下跨代理冗余。

Result: 在多个基准的Qwen3-4B实验中，LatCom较LatentMAS平均推理加速2.46倍，输出token减少70.3%，平均准确率相近。

Insight: 压缩目标不是重构全部发送者隐藏状态，而是最大化接收者任务效用。先训练单发送端可读性，让冻结接收者理解压缩接口，再训练多发送端融合以整合互补证据并去除重复。

Abstract: LLM-based multi-agent systems (MAS) increasingly use latent collaboration to avoid the information loss and repeated encoding-decoding overhead of natural-language communication. However, directly forwarding all sender latents makes the receiver-side context scale with both the number of agents and the reasoning length, increasing computation, memory usage, and collaboration latency. A natural solution is latent compression. But we find that cross-agent redundancy remains unresolved in existing latent compression approaches, which typically compress each sender independently and then concatenate the results. We propose LatCom, a cross-agent latent compression framework for efficient multi-agent latent collaboration. LatCom maps multiple sender latents into a fixed number of receiver-readable and task-relevant slots. Rather than reconstructing all sender hidden states, it optimizes the compressed latents for receiver-side task utility. LatCom trains the compressor in two stages: single-sender readability learning first establishes a latent interface interpretable by the frozen receiver, and multi-sender fusion learning then trains the compressor to fuse complementary evidence and remove redundancy across agents. Experiments on multiple benchmarks with Qwen3-4B show that LatCom achieves an average 2.46x inference speed-up over LatentMAS and reduces output token usage by 70.3% while maintaining comparable average accuracy.

[187] CypherTurn: A Multi-Turn Benchmark for Conversational Text-to-Cypher Evaluation and the Autonomy Divergence cs.CLPDF

Yuzhe Zhang, Weijie Zhu, Haolin Yang, Ziyun Zhang, Xianwei Xue

TL;DR: CypherTurn提供真实多轮对话式Text-to-Cypher评测，揭示单轮查询能力与长会话自主错误管理能力并不等价。

Details

Motivation: 现有图数据库自然语言查询基准都是互不关联的单轮任务，无法模拟分析者真实的连续会话。

Result: 基准含7个知识图谱、721个会话共5,927轮及13种对话现象。最佳模型执行准确率仅64.7%，会话级正确率低于5%；自主模式改变前沿模型排行，增加到十倍行动预算也未弥合差距。

Insight: 自主模型在预算充裕时仍平均每轮自限约两次操作，显示错误管理是独立于查询生成的能力；单轮Cypher微调会损害多轮指令跟随，架构适配的专用模型可超过若干前沿模型。

Abstract: Graph databases are increasingly queried through natural language, yet every existing benchmark evaluates isolated single-turn queries rather than the multi-turn sessions through which analysts actually work. We introduce CypherTurn, the first benchmark for conversational Text-to-Cypher evaluation, comprising 721 sessions and 5,927 turns across 7 knowledge graphs and 13 conversational phenomena. We evaluate 15 models under a guided oracle protocol and a fully autonomous agentic protocol, yielding four findings. First, the best model reaches only 64.7% execution accuracy, and session-level correctness remains below 5%. Second, despite strong overall rank correlation, frontier models exhibit a consequential reordering of the top of the leaderboard under autonomous operation, a phenomenon we term the Autonomy Divergence, which reveals error-management as a partially independent capability from raw generation skill. Third, scaling action budgets from x3 to x10 fails to close the autonomy gap, as the strongest frontier models self-limit to approximately two actions per turn regardless of available budget. Fourth, single-turn Cypher fine-tuning degrades multi-turn instruction following, while architecture-appropriate specialization outperforms several frontier models. These results establish CypherTurn as an open challenge for conversational graph database reasoning. Code and data are available at https://github.com/BarryQ/CypherTurn.

[188] Learning from Think-Mode Advantage via On-Policy Distillation cs.CLPDF

Wanqi Ren, Jianxiang Wang, Danxuan Liu, Linyi Ding, Yuan Zhang

TL;DR: ThinkOPD依据教师思维轨迹与学生完整回答的兼容程度，对在线蒸馏监督按回答路由，而非把单一思维链均匀用于所有学生输出。

Details

Motivation: 在线策略蒸馏沿学生生成前缀提供教师token监督，但固定教师思维链未必与每条完整学生回答路线相容；同结果回答也可能与教师轨迹产生不同偏差。

Result: 在数学推理和代码生成中，ThinkOPD在同模型及跨模型师生设置都优于均匀ThinkOPD，也超过受控比较中的理由蒸馏和自蒸馏基线。

Insight: 用轨迹—回答分歧（TRD）描述教师思维轨迹与学生响应的匹配度，并结合组相对奖励增益形成回答级监督权重，组内归一化。干预显示结果收益和TRD兼容度提供互补路由信号。

Abstract: Explicit intermediate reasoning gives large language models (LLMs) a stronger problem-solving mode. We study learning from this think-mode advantage via on-policy distillation (OPD). OPD preserves student-generated trajectories and provides dense token-level teacher targets at student-visited prefixes. Privileged reasoning is used during distillation rather than student inference. Uniform ThinkOPD, a natural think-enabled OPD baseline, conditions a fixed teacher on one shared think trace and uniformly distills every sibling student response. Although its prefixes are on-policy, the trace need not follow a route compatible with every complete response: the same privileged trace can induce different teacher-student discrepancies even when responses reach the same outcome. We summarize this interaction with trace-response divergence (TRD) and introduce ThinkOPD, which routes supervision at the response level by combining group-relative reward gain with a TRD-based compatibility proxy. Final response weights are normalized within each rollout group. Across mathematical reasoning and code generation, ThinkOPD outperforms Uniform ThinkOPD in both same-model settings and both cross-model teacher-student pairs, and it exceeds representative rationale and self-distillation baselines in a controlled comparison. Controlled interventions show that outcome benefit and the TRD-based proxy provide complementary routing signals in this setting. Think-enabled OPD provides a controlled setting for studying how teacher advantage becomes transferable along student responses.

[189] What Does Post-Training Change in Multilingual Reasoning? cs.CL | cs.AIPDF

Hongyang Li, Xiao Li, Caesar Wu, Grégoire Danoy, Pascal Bouvry

TL;DR: 本文审计Qwen3在11种语言的数学推理交付，追踪监督微调和强化学习如何影响正确性、目标语言遵循与答案终止。

Details

Motivation: 开源推理模型对不同语言用户提供的推理能力不平等；若模型能解题却不能用用户语言给出完整答案，语言就成为使用门槛。

Result: 十种非英语语言中，在16次采样内以目标语言给出带可见推理的正确终止解仅15.4%至17.9%，英语为92.9%。多语言SFT恢复目标语言推理但各类SFT准确率下降，且非英语推理更易循环；RL可恢复终止且不损准确率，只有奖励含语言项的训练才持续使用目标语言。

Insight: 瓶颈随训练阶段变化：发布模型常转而用英语推理；SFT让推理语言本地化，却带来终止问题；RL奖励语言遵循可修复交付，单纯正确性奖励又会让模型退回英语。研究展示从英语中介能力到可靠多语言答案的可构造训练路径。

Abstract: Open-source reasoning models provide unequal access to reasoning capability across languages. When a model can solve a problem but cannot deliver a complete solution in the user’s language, language becomes an access barrier rather than merely a source of performance variation. We audit Qwen3 checkpoints on competition-mathematics tasks in eleven languages. Across the ten non-English languages, only 15.4-17.9% of problems receive a correct, terminating solution with visible reasoning in the requested language in any of 16 samples, compared with 92.9% in English. To identify the source of this disparity, we evaluate thirteen endpoints from one model family, spanning released checkpoints, multilingual supervised fine-tuning (SFT) at two scales, controlled SFT ablations, and three reinforcement-learning (RL) reward formulations. We jointly track correctness, language adherence, termination, and delivery efficiency. The dominant bottleneck shifts across post-training stages. Released models often reason in English. Multilingual SFT restores target-language reasoning, but accuracy declines across multilingual, English-only, and single-language SFT runs, showing that this cost is not specific to multilingual mixing; non-English reasoning traces additionally become prone to non-terminating loops. RL restores termination in both arms at no cost in accuracy, but only the arm whose reward includes a language term delivers: rewarding correctness alone returns the model to English. Together, these stages establish a constructive post-training path from English-pivoted capability to multilingual reasoning that is reliably delivered.

[190] Traverse: Learning When to Remember, Reset, and Redirect for Long-Horizon Web Search cs.CLPDF

Jingyuan Ma, Lynx Aster, He Zhang, Siyao Song, Weijie Yuan

TL;DR: Traverse让长程网页搜索代理在定义答案标准、搜索和独立验证之间循环，并以Seal Memory主动管理上下文。

Details

Motivation: 长程搜索会累积噪声或误导信息，使早期错误持续传播且越来越难恢复；压缩记忆工具又可能在强化学习中发生使用策略坍塌。

Result: 35B模型BrowseComp得分72.83，超过可比开源系统；在BrowseComp-ZH、xbench、DeepSearchQA、WideSearch、金融调查和商品搜索均稳定优于基础模型。消融显示自主压缩优于自动上下文压缩。

Insight: 代理经过Rubric、Answer、Verify三状态：先设定有效答案标准，按标准搜索，再独立核验并决定终止或继续。为避免Seal Collapse，RL只训练上下文管理操作之后的最终轨迹片段。

Abstract: Long-horizon information-seeking agents often accumulate noisy or misleading context, causing early mistakes to persist and making recovery increasingly difficult. We introduce an autonomous search harness in which the agent manages its own search process through three states: Rubric, Answer, and Verify. The agent first defines criteria for a valid answer, searches under these criteria, and then independently verifies the result before deciding whether to terminate or continue searching. It is further equipped with a Seal Memory tool that enables active context management. Training this behavior with reinforcement learning, however, can induce Seal Collapse, resulting in unstable training and preventing the agent from reliably learning when and how to use its memory tools. We solve this with a simple strategy that trains only the final segment after context management. Our 35B model achieves 72.83 on BrowseComp, outperforming comparable open-source systems, and consistently improves over the base model across BrowseComp-ZH, xbench, DeepSearchQA, WideSearch, financial investigation, and product search. Ablations show that autonomous compression outperforms automatic compaction and validate our RL design.

[191] VLM Fine-Tuning for End-to-End Combinatorial Optimization cs.CLPDF

Qingsong Yan, Xia Jiang, Yaoxin Wu, Wen Song, Lu Zhang

TL;DR: 本文提出通用视觉语言模型求解组合优化的方案，将输入实例的视觉结构与文字描述共同提供给同一模型。

Details

Motivation: 组合优化问题可用自然语言统一输入，但文字序列化可能掩盖构造有效解所需的空间和关系结构。

Result: 经过监督微调和验证器引导强化学习后，视觉增强模型总体优于文本版本；在CVRP和JSSP等复杂问题、规模较大时视觉输入收益更明显。

Insight: 视觉输入不含标准解或由解推导的信息，说明收益来自更好表达问题结构，而非答案泄漏；单个VLM可跨组合优化任务使用相同训练框架。

Abstract: Large language models (LLMs) have provided a unified interface for end-to-end combinatorial optimization (CO), but textual serialization alone may obscure spatial and relational structures that are important for generating effective CO solutions. This paper presents a general-purpose vision-language solver that augments textual instance descriptions with input-derived visual representations. A single vision-language model (VLM) is applied across different CO tasks and trained using supervised fine-tuning followed by verifier-guided reinforcement learning. While the visual inputs contain no gold solutions or solution-derived information, our experiments show that the VLM generally improves solution quality over its text-only counterpart, with particularly clear gains on more complex CO problems such as CVRP and JSSP. The advantage of visual information is more pronounced at large problem scales.

[192] Look What You Made Us Cluster: Hate Narrative Extraction from Reddit Discourse cs.CL | cs.SIPDF

Annabelle K. L. Chua, Forster J. Khoo, Joel C. R. Tan, Huey Ting Ang, Kheng Hwee Tan

TL;DR: 本文把网络仇恨叙事抽取表示为实体—评价对，结合LLM细粒度推理和聚类层级修订展示可解释叙事分析。

Details

Motivation: 现有计算方法依赖语义表征，往往只捕获表层含义，难以抽取精确、可解释的在线仇恨叙事。

Result: 论文用2024年Reddit中批评Taylor Swift的英文评论示范流程，并分析一个呈现仇恨言论模式的代表性聚类；摘要未报告量化评测。

Insight: LLM扩展基于方面的情感分析，识别被评价实体/方面、评价类型及具体评价，再由Leiden聚类，并通过LLM引导调整聚类粒度。

Abstract: Narrative extraction allows us to identify online hate narratives, supporting the construction of rigorous detection systems. Existing computational approaches, however, are limited in precision as they rely on semantic representations, which tend to capture only surface-level meaning. To detect more precise and interpretable narratives, we present an extraction pipeline that represents narratives as entity-evaluation pairs. Narratives are extracted using a Large Language Model (LLM) reasoning process that extends Aspect-Based Sentiment Analysis, identifying the aspect, classifying its judgement type as the basis for evaluation, and deriving the evaluation accordingly. Extracted narratives are then clustered using Leiden, following which clusters are resolved to an intended level of granularity through an LLM-guided refinement process. We illustrate this narrative pipeline with English Reddit comments from 2024 that criticize Taylor Swift, analyzing a representative cluster that exhibits hate speech patterns to demonstrate its interpretive value.

[193] Learning to Retrieve Missing Evidence for Long-Term Memory QA cs.CL | cs.AIPDF

Yi-Xuan Deng, Yi Zhang, Wei Liu, Chao Xue, Shuojin Yang

TL;DR: MERA以已验证证据状态引导后续记忆检索，同时保留全局可搜索记忆，并用强化学习训练轻量查询规划器找回缺失证据。

Details

Motivation: 长期问答所需信息可能散落在相隔很远的历史轮次，问题本身又缺少定位线索；已找回事实往往能提示下一步应查找什么。

Result: Qwen3-30B证据处理与回答时，0.6B规划器在LoCoMo准确率77.40%、LongMemEval-S为71.29%，比未用检索落地训练的30B规划器高4.10%和3.96%。LoCoMo后续检索轮次使累计证据召回从55.5%升至80.5%。

Insight: 将全局可检索存储与问题专属证据状态分离，已验证事实指导下一条查询但不限制搜索范围；RL以补回此前缺失证据为奖励。

Abstract: Long-term memory enables language models to use past interactions in future conversations. However, evidence needed to answer a question may be scattered across distant turns, while the question itself omits clues needed to locate it. Retrieved facts can reveal these clues, motivating retrieval decisions conditioned on evidence already found. We introduce MERA (Missing-Evidence Retrieval Augmentation), which separates globally searchable memory from a question-specific evidence state. Verified evidence guides subsequent retrieval without restricting access to the global memory. We train a lightweight planner through reinforcement learning, rewarding queries that recover previously missing evidence. MERA achieves strong answer accuracy across Qwen3-30B and GPT-4o-mini backbones. With Qwen3-30B for evidence processing and answer generation, the trained 0.6B planner achieves 77.40% accuracy on LoCoMo and 71.29% on LongMemEval-S, exceeding a 30B planner without retrieval-grounded training by 4.10% and 3.96%, respectively. On LoCoMo, later retrieval rounds increase cumulative evidence recall from 55.5% to 80.5%.

[194] Your Benchmark Is Not Saturated: Reviving Multiple-Choice Evaluation with Answer Pooling cs.CL | cs.LGPDF

Mohamed Eltahir, Abobaker Ahmed, Nawaf Barebood, Hussain Bu Subayt, Tanveer Hussain

TL;DR: AnswerPool把共享上下文下多个多选题的选项合并成一个答案池，要求模型为每题分配答案，从而降低靠排除法得分的机会。

Details

Motivation: 多选题便于自动评分但基准容易饱和；通常只能不断重写更难题目，而单题干扰项有限，模型可能仅靠排除少数选项答对。

Result: 在八个文本、图像和视频基准及18个模型上，池化对所有模型都更难，弱模型获得的排除法收益最大；七个开源模型仍有87%至100%地为无答案题作答。

Insight: 共享上下文的一组N题将所有选项汇入共同池，不修改题目和标签；五道四选题完全猜对概率由10⁻³降至5×10⁻⁷。删除正确答案后可用同一流程准确评估弃答能力。

Abstract: Multiple-choice benchmarks are cheap to grade and are running out of room, and the standard remedy, writing harder items, is slow and repeated for every benchmark. A saturated benchmark still holds a harder task. Each question’s wrong options are written for that question alone, so a model can score by eliminating a few options. We propose AnswerPool: take $N$ questions that share a context, pool all their options into one list, and ask the model to assign every question its answer. No item is written and no label changes. The chance of guessing a group right falls from $10^{-3}$ to $5\times10^{-7}$ for five four-option questions, and a model that recognizes its answers keeps its multiple-choice score, so the accuracy lost to pooling measures the credit the format gave for elimination. Deleting answers from the pool makes questions unanswerable with exact ground truth, so abstention is scored in the same pass. Across eight text, image, and video benchmarks and eighteen models, pooling is harder for every model, the elimination credit is largest for the weakest models, and seven of eight open-weight models answer 87 to 100% of unanswerable questions.

[195] Larry Caused the Car to Stop, But the Model Didn’t Notice: Transformer Blindness to the M-Heuristic cs.CLPDF

Stefania Butnaru, Claudiu Creanga

TL;DR: 本文用因果表达与迂回致使句的自然语言推理对照，发现DeBERTa、RoBERTa和BART未体现M-启发式语用推断。

Details

Motivation: Transformer擅长语义相似度，但对标记形式暗示标记含义的语用推理研究不足；作者考察“Larry stopped the car”和“Larry caused the car to stop”是否引出不同语用推断。

Result: 188种条件、15个双及物/非宾格相关动词的实验中，三个编码器均无该语用区分证据，DeBERTa在100%案例中预测Neutral。句对语义相似度在30组中有29组把迂回致使句更靠近非中介方式描述，与M启发式预测相反。

Insight: 探针起初似乎发现表征—使用分离，但控制实验表明探针读取的是句法复杂度而非致使语用。显式元语言说明后Gemini Flash-Lite可达100%，说明该原则在指令下可用，却未被默认NLI行为调用。

Abstract: Modern transformer models excel at capturing semantic relationships through sentence embeddings, yet their ability to perform pragmatic reasoning remains understudied. This paper investigates whether encoder-based transformers such as DeBERTa employ the M-Heuristic (the neo-Gricean principle that marked linguistic forms implicate marked meanings). We test this hypothesis by contrasting lexical causatives (e.g., Larry stopped the car'') with periphrastic causatives (e.g., Larry caused the car to stop’’) using a Natural Language Inference framework. Our experiments across 188 conditions with 15 ambitransitive verbs reveal that DeBERTa, RoBERTa, and BART show no evidence of capturing the pragmatic distinction between these forms, with DeBERTa predicting ``Neutral’’ for 100% of cases. Probing analysis initially suggested a representation-use dissociation, but control experiments reveal the probe was tracking syntactic complexity, not causative pragmatics. Semantic similarity over 30 triplets places periphrastic causatives closer to unmediated manner descriptions in 29/30 cases, opposite to M-Heuristic predictions in the embedding space. Under explicit metalinguistic framing, Gemini Flash-Lite reaches 100% with item-specific traces, so the principle is available under instruction yet unused in default NLI.

[196] Evaluating Bounded Autonomy in Regulated Agentic AI: A Diagnostic Harness with Constitutional Rewards, Escalation Labels, and Runtime Governance cs.CL | cs.AI | cs.LGPDF

Dipankar Sarkar

TL;DR: RegLLM提供受治理的受监管智能体工作流诊断框架，联合测量引用、来源落地、格式、升级、宪章一致性和不安全动作。

Details

Motivation: 受监管工作流需在自主完成任务和安全升级之间取得边界，但仅看任务成功会掩盖无依据回答和危险操作。

Result: 小样本演示中，12个离线案例启用治理后升级召回由0升至0.67、不安全动作率由0.33降至0.08；两组单GPU Qwen2.5-3B LoRA/DPO试点差异很大，配置噪声可能超过适配效应。作者明确称样本不足以证明适配器效果或生产就绪。

Insight: 相同领域宪章用于评估、训练奖励和部署防护；确定性运行时监督器阻断无依据回答并强制升级、记录干预。应把应升级标签作为可测训练信号，并用更多样本和重复实验检验配置方差。

Abstract: We propose RegLLM, a diagnostic harness for bounded autonomy in regulated agentic workflows. It instruments six trustworthiness signals: citation validity, source grounding, schema compliance, escalation correctness, constitutional alignment, and unsafe-action rate. Signals are distinguished by their source of supervision: programmatic verifiers, task-level escalation labels, or AI-judge scores. A deterministic runtime supervisor blocks ungrounded answers and forces escalation, logging interventions. The same domain constitution informs evaluation, training rewards, and serving guardrails. Task-level should-escalate labels make the act-versus-defer decision a measurable training signal. We demonstrate the harness at smoke scale. An offline reference run (n=12) lifts escalation recall from 0 to 0.67 and reduces unsafe-action rate from 0.33 to 0.08 when governance is enabled. Two single-GPU Qwen2.5-3B LoRA/DPO pilots (n=8, same seed and evaluation split) expose substantial variation: nominally identical RL-base configurations yield task success of 0.25 versus 0.12 and escalation recall of 1.0 versus 0.5. An answer-quality adapter changes recall from 1.0 to 0.5 in Run A, but from 0.5 to 1.0 in Run B. An escalation-aware variant produces no measurable change in Run B. These small pilots do not establish reliable adapter effects or production readiness. Their contribution is diagnostic: configuration variance can overwhelm apparent tuning effects on bounded-autonomy metrics, motivating larger evaluation sets and repeated runs.

[197] E-MoE: Enhanced Mixture-of-Experts for Non-Factorized Diffusion Language Models cs.CLPDF

Arseny Ivanov, Alexander Kolesov, Alexander Korotin, Ivan Oseledets, Mikhail Goncharov

TL;DR: E-MoE用MoE路由决策构成的离散共享潜变量，让掩码扩散语言模型学习跨位置相关性，改善少步并行生成。

Details

Motivation: 掩码扩散模型每个去噪步可同时补多个token，但位置独立因子化限制少步采样质量。连续高斯潜变量方法虽能表示相关性，却易发生潜变量被忽略的后验坍塌。

Result: 在合成多模态基准、二值MNIST及LM1B上，E-MoE的少步生成优于因子化基线；摘要未给出具体分数。

Insight: 反向分布变为对离散共享潜变量条件化的多个因子分布混合，该变量来自MoE专家路由；不增加相较因子化基线的活跃参数，且避免依赖连续VAE潜变量。

Abstract: Masked diffusion models (MDMs) generate sequences by progressively unmasking several tokens per denoising step, but their reverse process is typically factorized over positions, limiting sample quality in the few-step regime where diffusion’s speed advantage over autoregressive decoding matters most. A recent line of work introduces a continuous Gaussian latent, trained as a variational autoencoder, to capture correlations across positions, but such approaches are prone to posterior collapse, where the latent is silently ignored. We propose Enhanced Mixture-of-Experts (E-MoE), which builds the reverse process as a mixture of factorized distributions over a discrete shared latent given by the expert-routing decisions of a Mixture-of-Experts (MoE) backbone, without increasing active parameters over the factorized baseline. Across synthetic multi-modal benchmarks, binarized MNIST, and LM1B, E-MoE improves few-step generation over factorized baselines.

[198] From Dissonance to Orchestration: Teacher Intervention in On-Policy Distillation cs.CL | cs.LGPDF

Yuhao Wang, Ruiyang Ren, Yinan Zhang, Ruiqing Zhang, Jing Liu

TL;DR: MAESTRO根据局部策略分歧自适应决定教师何时接管及续写多久，使OPD在有效纠错与学生分布偏移之间取得平衡。

Details

Motivation: 在线蒸馏中教师介入可提升轨迹，却会改变学生学习数据分布；介入越深，准确率收益递减而离策略负担增加，单看rollout质量不足以确定指导强度。

Result: 八个数学推理基准上，0.6B和1.7B Qwen3学生的宏平均准确率均为所比较方法最高，1.7B在每个基准都领先；相较标准OPD，平均训练回答长度减少67.3%。

Insight: 局部分歧分数结合教师加权候选覆盖和局部分布相似度，并按推理段聚合，从而联合路由教师接管时机和持续长度。不同基准对介入深度和位置偏好不同，说明教师指导需自适应而非固定。

Abstract: On-policy distillation (OPD) trains a student on its own reasoning trajectories using feedback from a stronger teacher. Teacher interventions can improve these trajectories, but also change the distribution on which the student learns. Our controlled studies show that rollout quality alone is an incomplete criterion for allocating teacher guidance. Deeper intervention yields diminishing gains in rollout accuracy while increasing off-policy load. In a training probe with a restricted rollout horizon, peak student accuracy and performance retention favor different intervention strengths. The preferred intervention depth and placement also vary across benchmarks. These findings motivate MAESTRO, which uses local policy disagreement to jointly adapt when the teacher takes over and how long it generates. Its {policy disagreement score} combines teacher-weighted candidate coverage with local distribution similarity and is aggregated within reasoning paragraphs. Across eight mathematical reasoning benchmarks, MAESTRO achieves the highest macro-average accuracy among the compared methods for both 0.6B and 1.7B Qwen3 students, with the 1.7B student leading on every benchmark. MAESTRO also reduces average training response length by 67.3% relative to standard OPD. The code is available at https://github.com/yhao-wang/MAESTRO.

[199] Correct, Don’t Delete: Mitigating Emergent Misalignment with Corrective Supervision cs.CL | cs.AI | cs.LGPDF

Jacob Epifano

TL;DR: 本文直接纠正少量有害训练样本，发现相比删除相同行数，修正监督更能降低窄域微调引发的涌现失配。

Details

Motivation: 在狭窄有害示范上微调会导致模型对无关问题广泛失配；常见防御是定位并删掉问题样本，但定位方法留出集失败、删除收益有限。

Result: Qwen2.5-14B-Instruct医疗错误建议微调中，修正四分之一有害样本使失配率约降三分之一并改善留出医疗问答；删同样样本几乎无可测变化。修正一半效果更强，且迁移至第二基础模型和失配模型。

Insight: 替换为正确答案有效，单纯改写错误建议不明显；训练正确替代答案与作者重写效果相近。短轮次纠错训练优于等量通用对话数据，其他医疗问题的纠正也约同样有效；强调纠错者谨慎无害的额外提示没有可测收益。

Abstract: Fine-tuning a language model on a narrow set of harmful demonstrations, such as bad medical advice, can make it broadly misaligned on unrelated questions, a phenomenon known as emergent misalignment (EM). The usual defense is to find the offending rows and delete them, but a row locator failed our held-out test and deleting rows helps less than expected. We ask a different question: given a fixed set of poisoned rows, is it better to correct them than to remove them? We fine-tune Qwen2.5-14B-Instruct on a mixture of bad medical advice and benign chat data, select a quarter of the poison rows in advance, and either delete them or replace each with a corrected answer to the same prompt, keeping everything else the same. Replacing the rows cuts the EM rate by about a third and improves answers on held-out medical questions, while deleting the same rows has little measurable effect. The advantage is larger when half the poison rows are corrected, and it holds on a second base model and a second misaligned model organism. The content of the replacement appears to matter: paraphrasing the rows while keeping their bad advice shows no clear benefit, and the correct answers distributed with the dataset appear to do about as well as our rewriter’s. Realigning an already-poisoned model with further fine-tuning is known to work, but which data does the work has not been compared directly. We find that a short round of training on corrections beats the same amount of training on generic chat data, that corrections on other medical prompts do roughly as well as corrections of the poisoned prompts themselves, and that instructing the correction writer to model a careful, harm-avoiding assistant adds no measurable benefit over plain corrections. In the settings we tested, correcting harmful training data reduces EM more than deleting it.

[200] Devils in Question Relay: Source-Conditioned Relay Steering to Mitigate Hallucinations in Audio-visual Large Language Models cs.CLPDF

Yu Zhang, Pingrui Zhang, Xuefeng Bai, Pengfei Zhang, Yang Xiang

TL;DR: SECRET通过干预模态到问题状态的中继路径，减少音视频大模型在问题指定来源外依赖另一模态造成的落地幻觉。

Details

Motivation: AVLLM回答音频相关问题时可能受无关图像线索影响甚至更依赖图像，导致答案不受指定模态证据支持；其内部跨模态干扰机制尚不清楚。

Result: 在CMM、AVHBench两个基准和三种AVLLM上，SECRET优于此前免训练方法，较基础模型最多提高18.0和7.1个百分点；模态专属字幕任务显示其可用于开放式生成。

Insight: 路径干预和表征分析发现“问题中继”机制：问题状态携带干扰线索，与目标来源证据竞争；切断干扰模态到问题状态的路径比只切断其到输出位置恢复更多正确答案logit。SECRET用不同路径干预产生的对照问题表征引导原问题状态朝指定来源证据移动。

Abstract: Audio-visual large language models (AVLLMs) have made remarkable progress in multimodal understanding and reasoning through interactions among visual, auditory, and linguistic information. However, recent studies show that AVLLMs face a critical challenge: $\textbf{source-confused grounding hallucination}$, where cues from the unused modality induce responses that the required modality does not support, undermining reliability in real-world applications. Existing methods have made progress in mitigating this failure, yet how it arises from internal cross-modal interactions remains insufficiently understood. To address this gap, we conduct path-intervention and representation analyses, revealing a $\textbf{question-relay}$ mechanism: question states carry interfering cues alongside required-source evidence, undermining grounding in required-modality evidence. Cutting pathways from interfering modality to question states yields greater correct-answer logit recovery than cutting those to the generation position. Motivated by these findings, we propose $\textbf{SECRET}$ ($\textbf{S}$ourc$\textbf{E}$-$\textbf{C}$onditioned $\textbf{RE}$lay s$\textbf{T}$eering), a training-free method that mitigates cross-modal interference at the question relay. Using contrasting question representations elicited through different modality-pathway interventions, SECRET steers the original question states toward required-source evidence. Experiments on two widely adopted benchmarks CMM and AVHBench across three AVLLMs show that SECRET consistently outperforms prior training-free methods, substantially mitigating source-confused grounding hallucinations (e.g., up to +18.0 and +7.1 percentage points over base models). Modality-specific captioning further demonstrates its generalizability to open-ended generation.

[201] Evaluating and Benchmarking the System One Model Jev cs.CL | cs.AIPDF

Tobias Deußer, Lorenz Sparrenberg, Rafet Sifa

TL;DR: Jev是无需生成文本、按类型回答选择/量规/概率问题的System 1模型；本文对37个数据集进行大规模零样本基准评估。

Details

Motivation: 信息访问流程中的路由、grounding检查、审核或量规评分都由许多小判断组成，非生成式快速决策模型是否可靠需要系统比较。

Result: 对37个数据集共346,009次请求、成本低于10美元。Jev在IMDB、SST-2、HellaSwag、ARC达95%至99%，Belebele达86.7%；在37项中27项优于Qwen、全部优于Gemma。选择题概率校准良好，而真假概率阈值0.5表现差，训练集调阈值将UNFAIR-ToS micro-F1从0.50升至0.75。

Insight: Jev选择概率支持选择性预测；但低资源语言、细粒度噪声标签和量规质量评估仍使三种模型下降。选项轮换不改变Jev表现、去掉问题则接近随机，排除浅层答案记忆但不能排除记住特定问答对。

Abstract: Jev is a commercial System One model from TypeSafe AI that does not generate text: given a state and typed questions, it returns a choice from fixed options, a position on a rubric, or the probability that a statement is true, with probabilities the vendor describes as calibrated. Such models target small decisions in information access pipelines, such as routing queries, checking grounding, moderating content, or rating against a rubric. We evaluate Jev (jev-1.13.0) zero-shot on 37 datasets spanning classification, routing, natural language inference, reading comprehension, commonsense reasoning, moderation, legal clause analysis and rubric scoring, with one frozen template per dataset and full evaluation splits: 346,009 requests for under USD 10. For reference, we score Qwen3.8-27B and Gemma-4-E4B on identical requests via their exact next-token probabilities over the options. Jev reaches 95-99% accuracy on IMDB, SST-2, HellaSwag and ARC and 86.7% on Belebele across 122 languages. It beats Qwen on 27 of 37 datasets, with none of Qwen’s nine leads outside the bootstrap intervals, and Gemma on all 37. All three models degrade on low-resource languages, fine-grained or noisy labels, and rubric-based quality judgments. Jev’s choice probabilities are well calibrated and support selective prediction. Binary probabilities rank well but are poorly placed relative to a fixed 0.5 threshold; thresholds tuned on training data raise micro-F1 on UNFAIR-ToS from 0.50 to 0.75. Jev answers MMLU’s calculation-heavy questions more accurately than other MMLU questions (94% vs. 91%), whereas both open models, and all three on C-Eval, find them harder. Rotating the options leaves Jev’s accuracy unchanged and withholding the question drops it to near chance, ruling out shallow memorization but not memorized question-answer pairs. We release the code, harness and all raw responses.

[202] Corpus-Guided Dual-Path Propagation for Graph Retrieval-Augmented Generation cs.CLPDF

Baoxian Liu, Tong Wei

TL;DR: NexusRAG在关系无关三图上增加语料级实体邻域结构，交替进行受邻域约束的语义传播和实体结构传播以支持多跳检索。

Details

Motivation: 已有关系无关图检索主要按查询—句子相似度找证据，容易漏掉查询相似度低但关键的桥接证据，也可能激活无关实体。

Result: 在三个多跳QA基准和GraphRAG-Bench领域子集上，NexusRAG持续优于现有方法；后者所有问题类别的证据召回最高，比基线高4.2至8.1分。

Insight: 实体邻域由共现与语义相似构建；句子语义传播确定查询相关实体前沿，实体结构传播再扩展至相关邻居。传播权重还用于邻域感知的个性化PageRank段落初始化。

Abstract: Graph-based retrieval-augmented generation supports multi-hop retrieval by organizing corpus information into graphs. However, existing relation-free graph retrieval methods rely primarily on query-sentence similarity to search for evidence. This can exclude useful bridging evidence with low query similarity and activate incidental entities unrelated to the reasoning chain. In this paper, we propose a simple and effective approach called NexusRAG, which augments the relation-free Tri-Graph with a corpus-level entity neighborhood structure derived from joint entity co-occurrence and semantic similarity. NexusRAG employs this structure to guide two complementary propagation paths: neighborhood-constrained semantic propagation through sentences identifies the query-relevant entity frontier, while direct structural propagation between neighboring entities expands that frontier to structurally related entities. The propagated entity weights also inform neighborhood-aware passage initialization for Personalized PageRank. Experiments on three multi-hop QA benchmarks and a domain-specific subset of GraphRAG-Bench show that NexusRAG consistently outperforms existing approaches. On the GraphRAG-Bench subset, NexusRAG achieves the highest evidence recall in all question categories, exceeding baselines by 4.2-8.1 points. The implementation code is available at https://github.com/Jacob-biu/NexusRAG.

[203] Thinking in Depth, Speaking Directly: Recurrent Latent Reasoning for Paralinguistically Grounded Spoken Dialogue cs.CL | cs.AIPDF

Shengbo Cai, Yuxiang Wang, Jingran Xie, Zhisheng Zhang, Shun Lei

TL;DR: LoopSLM以循环Transformer反复更新受声学信息约束的潜状态，并通过分阶段训练缩小语音副语言感知与回复规划间距。

Details

Motivation: 共情对话需同时理解说了什么和语气如何；显式思维链虽可提高副语言感知和回复中的声学表达，却不能保证有效用于规划，且增加生成延迟。

Result: 在EchoMind上，LoopSLM相较Qwen2.5-Omni-7B改善副语言理解、推理及回复质量；较CoT-SFT推理准确率提高20多个点、token少64.5%、延迟减半。相较Qwen3-Omni-Thinking，多数共情回复指标更好且延迟低34倍。

Insight: 重用解码器块多轮细化隐藏状态，并在每轮注入声学grounding；两阶段训练分别学习推理和回应，使部署可直接输出而无需CoT。只用对话数据训练也改善通用音频基准。

Abstract: Empathetic spoken dialogue requires models to use both what is said and how it is said to decide how to respond. Explicit CoT can improve paralinguistic perception and make acoustic cues more explicit in replies, yet does not ensure their effective use in response planning. We call this mismatch the perception-reasoning gap. In addition, CoT may not fully capture acoustic cues in words, and generating it adds inference latency. To address these limitations, we introduce LoopSLM, which builds on looped Transformers for latent reasoning, reusing a decoder block to refine hidden states with acoustic grounding at every pass. Its two-stage training further narrows the perception-reasoning gap by separating learning to reason from learning to respond, enabling direct inference without CoT. On EchoMind, LoopSLM improves paralinguistic understanding, reasoning, and reply quality over Qwen2.5-Omni-7B. Against the CoT-SFT baseline, LoopSLM gains over 20 points in reasoning accuracy while generating 64.5% fewer tokens at half the latency. It also outperforms Qwen3-Omni-Thinking on most empathetic reply metrics with 34x lower latency. Despite training only on dialogue data, LoopSLM improves accuracy on general audio benchmarks.

[204] A Proposed Rubric for Evaluating Expressed Clinical Reasoning in Large Language Model Responses cs.CL | cs.AIPDF

Zhangshu Joshua Jiang, Zina Ibrahim, James T. Teo

TL;DR: 本文提出多维临床推理评分量规，将循证性、适用规则和病例安全关键错误纳入自由文本医疗回答评价。

Details

Motivation: 通用LLM推理评价框架和临床教育测评可供借鉴，但缺少明确面向临床表达推理且能让评价决策接受审查的统一量规。

Result: 该量规目前尚未验证评审者间信度、构念效度或临床效用，论文仅提出量规方案，未主张已验证效果。

Insight: 综合ART、SCT、关键特征问题、OSCE、临床LLM基准及一般推理评价研究，将groundedness作为临床语境下的事实性，并提供初步行为锚点和单独安全错误标记。通用框架不视为已验证临床工具，也不替代病例特定参照或现有任务指标。

Abstract: Rubrics support the structured evaluation of language models. We propose a rubric for assessing expressed clinical reasoning in model responses, drawing on three bodies of work: medical education assessment frameworks (ART, SCT, Key Feature Problems and OSCE); clinical LLM benchmarks (MedR-Bench, HealthBench, TIMER-Bench, DR.BENCH, PrIME-LLM and PatientSafeBench); and general LLM reasoning evaluation research, including the Factuality-Validity-Coherence-Utility taxonomy, FaithCoT-Bench and C2-Faith. We use groundedness as a clinically oriented adaptation of the taxonomy’s factuality category. The rubric brings these concepts together in a multidimensional framework for scoring free-text responses to gold-standard clinical vignettes. It includes provisional behavioural anchors, applicability rules and a separate flag for case-specific safety-critical errors. General-domain frameworks inform its design but are not treated as validated clinical instruments. The rubric does not replace case-specific reference criteria or the task-specific metrics of existing benchmarks. It has not yet been tested for inter-rater reliability, construct validity or clinical utility. Its immediate purpose is to make evaluation decisions explicit and open to scrutiny before empirical testing.

[205] Can Vision-Language Models Stay Helpful When Facing Implicit Risks? Intent-Privilege OPSD for Efficient Safety-Helpfulness Alignment cs.CLPDF

Haotian Deng, Wenbin Xing, Gang Xu, Tao He, Jinkai Zheng

TL;DR: Intent-Privilege OPSD在训练时用基于证据的意图作为特权监督，让VLM识别跨模态隐含风险并给安全且有帮助的回答。

Details

Motivation: 图像和文字单看无害，结合后却可能诱发不安全回应。现有安全方法常需大规模偏好数据、多次采样训练或推理防护，并可能用简单拒绝牺牲有益帮助。

Result: 仅用1,447条安全样本，比标准偏好数据少95%，训练时间比多rollout GRPO快5倍，平均推理长度减少7%；五类评测中安全且有帮助的联合成功比例最高，SIUO+HoliSafe从43.9%升至53.5%。

Insight: 训练时教师依据意图条件化对回答偏好，并用单次rollout蒸馏给学生；部署学生无需意图标注和额外安全模块，也无需对可能安全回答一概拒绝。

Abstract: Vision-Language Models (VLMs) remain vulnerable to cross-modal implicit risks: visual and textual inputs that appear benign in isolation can jointly elicit unsafe responses. Existing safety methods often require large preference datasets, costly multi-rollout training, or additional safeguards at inference time. They may also sacrifice helpfulness by directly refusing requests that could be answered safely. In this paper, we propose Intent-Privilege On-Policy Self-Distillation (OPSD), which leverages evidence-grounded intent as privileged supervision during training to help VLMs recognize implicit risks and provide safe, useful responses instead of blanket refusals. OPSD distills a teacher’s intent-conditioned preferences over responses into a student using a single rollout per prompt; the student then responds without intent annotations or an additional safety module. With only 1,447 safety-specific examples - 95% fewer than standard preference datasets - OPSD reduces training time by 5x relative to multi-rollout GRPO-style training and average inference length by 7%. It attains the highest ratio for joint safety-helpfulness success, which measures the proportion of responses that are both safe and helpful, across all five evaluation groups. Remarkably, on pooled SIUO+HoliSafe, this success ratio rises from 43.9% to 53.5%. These results show that training-time intent supervision can improve both safety and helpfulness while substantially reducing data, training, and inference costs.

[206] It’s Not What the Image Shows: Irrelevant Context Destabilises VLM Judges Without Informing Them cs.CL | cs.AI | cs.CVPDF

Nagham Omar, Mahmoud Jabarin, Kinan Ibraheem, Lotem Peled-Cohen

TL;DR: MIST测试VLM裁判在任务明确要求忽略图像时是否仍受图像存在影响，发现干扰来自“有图”这一条件而非图像描绘的具体含义。

Details

Motivation: VLM逐渐替代人工标注，替代性评估需区分模型能力和偶然评测条件。若标签明明只由句子决定，加入任何图像都不应改变答案。

Result: 13个VLM裁判中，匹配图改变20.5%标签，误导图改变19.4%，均高于保留图但删除“忽略图像”指令产生的11.6%；两图导致不同的标签中，只有37%向图像所示含义变化。无论有无图，人类一致性均无差异。

Insight: 基准用200个可字面或比喻理解的英文句子及相符/相反/无图条件，明确要求仅按句子标注。通过alt测试的7个裁判受影响较小，但均存在；因此可替代性结论同时描述模型与具体评测配置。

Abstract: Vision-language models (VLMs) are increasingly used in place of human annotators, making it important that substitutability tests reflect the model rather than incidental evaluation conditions. We introduce MIST, the Misleading-Image Stress Test: 200 English sentences, each built around a phrase readable either figuratively or literally and shown with an aligned image depicting its reading, a misleading image depicting the opposite, or no image at all. The guidelines require the label to be decided from the sentence alone, so no image should change any answer. We expected each image to pull a judge’s labels toward the sense it depicts, and neither kind did. Across thirteen VLM judges, an aligned image changed 20.5% of labels and a misleading one 19.4%, close for every judge and both above the 11.6% produced by deleting the ignore-the-image instruction with the image left in place. Yet only 37% of the labels that differ between the two images moved toward the sense shown, and agreement with our human annotators is unchanged whether the image is absent, aligned or misleading. The effect is smaller in the seven judges that pass the alt-test than in the six that never do, but present in all of them: what moves a judge is that an image is there, not which of the two it is, so a substitutability verdict describes a configuration as much as a model.

[207] It’s All Training: A Fully Synthetic Single-Stage Recipe for LLMs cs.CL | cs.LGPDF

Pierre-Carl Langlais, Pieter Delobelle, Yannick Detrois, Pavel Chizhov, Carlos Rosas-Hinostroza

TL;DR: SYNTH用精选百科种子结构化扩增合成语料，尝试将预训练、中期训练和后训练合并为单阶段训练。

Details

Motivation: 网络爬取预训练数据未为中后训练设计，显式推理内容少；前沿实验室虽用内部合成数据改善冷启动，但数据不公开，小模型如何从中学习尚未充分理解。

Result: 在同计算量下，SYNTH胜过过滤网页数据；56M、0.3B至0.6B及13B/1B激活MoE模型与同规模开放基线竞争。训练token少10至140倍仍保持较高事实精度，表明种子知识导向合成可提高数据效率。

Insight: 语料由58,698篇维基百科文章结构化扩增并从有据文本反向生成。知识和记忆主要限于种子语料覆盖范围，论文据此讨论通用及缺少指令数据场景的领域模型，并公开数据和Baguettotron模型。

Abstract: Current pre-training datasets are derived from web crawls, with all their issues, and were not designed to support mid- and post-training pipelines–for instance, they contain little explicit reasoning. Thus, many frontier labs have begun to develop their own internal datasets, starting from state-of-the-art models, to augment their pre-training data mix, eg, with reasoning traces to address cold-start problems. While demonstratively effective, none of these datasets are public, and the effect of this so-called synthetic data on knowledge and skill acquisition of language models, including small ones, remains poorly understood. We present SYNTH, the first open-source synthetic corpus derived from 58,698 Wikipedia articles that collapses pre-, mid-, and post-training into a single training stage via structured amplification of curated encyclopedic seeds. We evaluate SYNTH by training a suite of models: a 56M tiny model (Monad), 0.3B-0.6B dense models (Baguettotron), and a 13B / 1B-active MoE. At iso-compute, SYNTH outperforms filtered web data, and our models remain competitive with similarly-sized open-weight baselines. Because SYNTH is back-translated from grounded passages, SYNTH-trained models achieve high factual precision despite 10-140x fewer training tokens, with memorization targeted by the seed corpus. These results show that synthetic datasets, including our SYNTH dataset, are capable of producing competitive generalist models from a fraction of the training data, enabling rapid iteration as the frontier advances. These findings open up possibilities for both generalist models with significantly increased data efficiency, as well as domain-specific models where no instruction or conversational data is available. Finally, we publicly release our SYNTH dataset and the suite of Baguettotron models under a permissive license, thus supporting open-source language model development.

[208] Learning What to Remember: Long-horizon Counterfactual Memory Optimization cs.CL | cs.LGPDF

Jiaming Tang, Mingyan Liu, Armin Sarabi

TL;DR: MGPO以记忆重写对当前及未来任务效用的边际贡献分配信用，解决长期记忆学习中的延迟奖励和旧信息贡献混淆。

Details

Motivation: 文本记忆改写可能许多步之后才显现价值，而当前表现中大量效用又来自改写前已保存内容，难判断哪些更新真正值得保留。

Result: 在文档级信息抽取中，MGPO提升抽取效果，并使平均记忆长度较初始记忆策略减少近80%；学习到的记忆策略还能跨领域和下游模型复用，无需进一步训练。

Insight: Memory Gain Policy Optimization将每次重写的增量价值与已有记忆贡献分开，直接把延迟效用变成学习信号，从而优化哪些信息应持续留存。

Abstract: Persistent textual memory allows language models to carry information across long interactions, but learning what to remember is fundamentally a credit-assignment problem. A memory rewrite may only become useful many steps later, while much of the observed utility may be inherited from information already stored before the rewrite. We introduce Memory Gain Policy Optimization (MGPO), which isolates the incremental value of each memory rewrite by crediting it for its marginal contribution to current and future downstream utility. This turns delayed memory utility into a direct learning signal for optimizing what information should persist. We study MGPO on document-level information extraction, where structured supervision makes the effects of individual memory updates directly measurable. MGPO improves extraction while reducing average memory length by nearly 80% relative to the initial memory policy before optimization. The learned memory policy also supports reuse and transfer across domains, downstream models without further training. These results show that effective memory learning depends not only on preserving useful information, but on identifying which memory updates create lasting incremental value.

[209] Auditable Long-Term Memory: A Deterministic Retrieval Chain Measured at 479/475 of 500 on LongMemEval-S cs.CL | cs.AI | cs.IRPDF

Christopher J. Chanhnourack

TL;DR: 本文评估由混合检索、交叉编码重排、覆盖优先证据包和确定性推理脚手架组成的可审计长期记忆检索链，并报告重复计分波动与范围限制。

Details

Motivation: 长期问答系统除回答准确率外，还需检验候选是否覆盖金证据、证据包是否完整及评分过程是否稳定。

Result: LongMemEval-S两次500题评测在GPT-4o裁判下分别479/500和475/500；候选池覆盖468/470可回答题，证据包完整462/470。独立裁判对两次多数答案一致，但也有裁判翻转。

Insight: 系统以可替换LLM作最后阅读器，检索链条确定性可审计。但72条知识更新题使用了经实质修改的评分提示，未在官方文本下测试；开发复用了全部500题，没有留出集或独立人审，读者别名未固定且请求未完整留存。因此结果不能确立优越或等效，仅供复核。

Abstract: We evaluate an auditable long-term memory system on LongMemEval-S. Its retrieval chain uses hybrid candidate retrieval, cross-encoder reranking, coverage-first packet compilation, and deterministic reasoning scaffolds; an LLM is used only as a replaceable final reader. The chain places all gold sessions in the candidate pool for 468/470 answerable questions and produces gold-complete packets for 462/470. With a Claude Opus reader called through an unpinned CLI alias, two 500-question passes score 479/500 and 475/500 under GPT-4o. The 72 answerable knowledge-update rows used a substantively modified scoring prompt whose effect under the official text has not been measured. The pair straddles Chronos High’s published 478/500; differences in reader generation, scoring prompt, and possibly data version, plus within-system variance, establish neither superiority nor equivalence. A grok-4.6-high reader on the same packets scores 476/474, while a maximum-reasoning-effort agentic variant regresses to 461/465. The headline passes differ on eight verdict-flip rows. A second judge agrees with the headline judge on 493/500 rows (98.6%) in each pass and scores both passes 472/500; the official judge also flips three verdicts when re-scoring byte-identical pass-1 answers. Negative controls rejected a verifier that repaired three wrong drafts but broke eleven correct drafts. All components were developed on the same 500 questions, with no held-out evaluation or independent human adjudication; retrieval and scaffold method sources and transcript-derived audits are held; and the headline reader received extra operator context, its complete requests were not retained, and MCP tool availability is unresolved. We release materialized packets, scaffolds, reader outputs, judge verdicts, and controls for inspection and re-scoring.

[210] Layer-Informed Fine-Tuning via Three-Stage Functional Segmentation of LLMs cs.CLPDF

Junning Shao, Siwei Wang, Zhixuan Fang

TL;DR: LIFT先以敏感性分析识别任务最依赖的概念化、推理或文本化层段，再只微调关键层以降低训练成本并提升效果。

Details

Motivation: LLM在复杂推理任务上表现突出，但模型结构和内部参数如何逐步处理推理仍缺乏清晰理解。

Result: 跨语言材料上的实验显示，LIFT加快微调并显著提升模型表现；摘要未提供具体基准和数值。

Insight: 提出层级分工假设：不同层群依次承担概念化、推理和文本化；通过敏感性分析定位目标任务的关键功能阶段，再有选择地更新相应层，而非全模型训练。

Abstract: In recent years, the performance of large language models (LLMs) on reasoning tasks has been remarkable, even surpassing human capabilities on various benchmarks. However, there remains a lack of clear understanding in the academic community regarding how the structure and internal parameters of LLMs progressively solve complex reasoning problems. In this study, we investigate the inference process of LLMs on cross-linguistic materials and propose the hypothesis that LLM layers exhibit a structured division of labor across conceptualization, reasoning, and textualization. Based on this hypothesis, we introduce a bottleneck identification mechanism using sensitivity analysis to pinpoint the most critical functional stage for a specific task. Leveraging this insight, we propose a novel approach, Layer-Informed Fine-Tuning (LIFT), which achieves efficient and effective fine-tuning by selectively updating only these functionally critical layers. We then conduct extensive experiments to show that the LIFT method not only accelerates the training process but also significantly improves model performance.

[211] Correct Answers, Invalid Traces: What Verifiable Grade-School Math Reveals About Chain-of-Thought Traces cs.CL | cs.AIPDF

Ratish Puduppully, Pranabendu Misra, Paarth Iyer, Durgesh Kalwar, Vardhan Palod

TL;DR: 作者用可逐步程序验证的 iGSM 数学基准研究思维链的可信度，发现答案正确并不保证推理轨迹有效，且模型可在轨迹被破坏后仍维持高准确率。

Details

Motivation: 思维链常被当作模型得出答案的过程记录，并用于调试、审计和推理能力判断，但自然语言轨迹通常难以机械验证。iGSM 明确呈现正确解答所需的数值和依赖关系，使得答案与推理轨迹能分别检查。

Result: 只用有效且精简轨迹训练时，答案正确率与轨迹有效性在分布内大致一致，但在分布外会分离：最难样本中，31.6%的正确答案对应无效轨迹；其中超过一半通过句法和算术检查，却违反语义依赖。非精简轨迹监督会诱导非精简输出；换一种提问方式重问同题，可发现模型继承了原问题的计算。将10%的训练轨迹句子打乱后，分布外准确率仍接近干净训练，尽管没有轨迹通过验证；交换训练轨迹也仍保持较高的分布内准确率。

Insight: 正确答案本身不能证明思维链忠实或有效，甚至语法和算术自洽也不够，还需检查语义依赖。轨迹监督受扰后性能仍高，说明把生成的思维链直接当成内部推理证据存在风险。

Abstract: Chain-of-thought traces are widely read as records of how models reach their answers, informing debugging, agent auditing, and claims about reasoning. Testing this interpretation is difficult because natural-language thinking traces are rarely mechanically verifiable. We revisit it in iGSM, a synthetic grade-school mathematics benchmark designed to study thinking traces and used to support claims of learned reasoning and planning. Crucially, iGSM exposes the exact quantities and dependencies that a correct solution should use, allowing generated traces to be checked programmatically step by step and enabling us to test whether correct answers are reliably accompanied by valid traces. We first evaluate models trained exclusively on valid, minimal traces. Answer correctness and trace validity nearly coincide in distribution but decouple out of distribution: on the hardest instances, 31.6% of correct answers have invalid traces, over half of which pass all syntactic and arithmetic checks but fail semantic dependency checks. We then intervene on trace supervision. Non-minimal training traces induce non-minimal outputs, while re-asking the same problem with a different query reveals computations inherited from the original query, weakening minimality as evidence of selective planning. Shuffling tokens in 10% of training trace sentences preserves near-clean accuracy even out of distribution despite no trace passing verification. Swapped training traces likewise retain high in-distribution accuracy. We discuss the implications of these findings for chain-of-thought monitoring and interpretation in the context of AI safety.

[212] From Routing Signals to Selective Review: Visual regrounding in MoE VLMs cs.CL | cs.CVPDF

Hongzhu Guo, Mohsen Fayyaz, Nanyun Peng

TL;DR: 论文利用 MoE 视觉语言模型的专家路由概率，在生成回答前检测问题所指目标是否实际缺席，并据此选择性触发视觉复核。

Details

Motivation: VLM 可能接受问题中的错误视觉前提，在图像里目标不存在时仍描述其颜色、数量、位置或状态。以生成文本、隐藏状态或不确定性为依据的检测往往要等到生成后，作者希望利用模型已有的内部路由信号提前判断。

Result: 作者从 Qwen3-VL-30B-A3B-Instruct 和 Gemma-4-26B-A4B-it 提取目标 token 的路由概率，分别训练 L2 正则化线性检测器，并用预测结果门控目标感知复核提示。在 GQA-Inpaint 上，Qwen 和 Gemma 的 ROC-AUC 分别为0.9988、0.9956；在外部 OBER 上分别为0.8095、0.7781。门控策略使 Qwen 在 GQA-Inpaint/OBER 的端到端准确率分别提高22.25%/12.17%，Gemma 分别提高13.42%/1.39%，且不修改模型权重。信号集中于目标对象 token、在较早 MoE 层出现，并分布于可部分替代的专家；跨数据集阈值需要重新校准。

Insight: 专家路由决策本身包含可用于视觉感知审查的信息，可以把已有计算转化成低成本的生成前检测与选择性纠正。外部数据集上的效果差异也表明阈值校准和复核提示仍影响实际收益。

Abstract: Vision-language models (VLMs) may accept false visual premises, answering questions about a target object’s color, count, location, or state even when it is absent. We call this reliability-critical behavior a target-absence grounding failure. Existing visual-grounding detectors primarily rely on generated responses, hidden states, or uncertainty measures. We present the first framework to leverage internal routing decisions in Mixture-of-Experts (MoE) VLMs to detect target absence before generation and guide selective correction. We extract target-token routing probabilities from Qwen3-VL-30B-A3B-Instruct and Gemma-4-26B-A4B-it, train a separate L2-regularized linear detector for each model, and use its predictions to selectively invoke a target-aware review prompt. Using routing alone, the Qwen and Gemma detectors achieve ROC-AUCs of 0.9988 and 0.9956 on GQA-Inpaint and retain 0.8095 and 0.7781 on the external OBER dataset, respectively. The resulting routing-gated policy improves end-to-end accuracy on GQA-Inpaint and OBER by +22.25% and +12.17% for Qwen, and by +13.42% and +1.39% for Gemma, without modifying model weights. Further analysis shows that the signal is localized to the target-object token, emerges in early MoE layers, and is distributed across partially substitutable experts. Although cross-dataset threshold shifts require recalibration, false-positive review causes limited harm overall, suggesting that intervention risk can be controlled through joint selection of the detector threshold and review prompt. Overall, we show that routing probabilities alone preserve actionable information about visual perception, allowing computation already produced by an MoE VLM to support low-cost detection and selective visual regrounding.

[213] LongHarness Bench: Stress-Testing Language Model Harnesses for Long-Context Reasoning cs.CLPDF

Quang Hieu Pham, Thuy Duong Nguyen, Jocelyn Qiaochu Chen, Xi Ye

TL;DR: LongHarness Bench 用需要跨文档检索和策略性推理的任务比较长上下文 harness，并显示同一底层模型的准确率—成本表现会随 harness 显著变化。

Details

Motivation: 现有长上下文评测容易出现不同 harness 准确率饱和、成本接近的情况，难以区分系统能力。真实任务往往包含大量语义相关但当前步骤仅少量有用的信息，需要在局部与全局上下文之间选择检索策略。

Result: 基准任务结合词面搜索、语义匹配以及对全局和局部上下文的策略性、自适应推理；例如需从分散文档中找出满足多个条件的所有人物，先核对最具区分性的条件可缩小后续搜索范围。作者用四种前沿 harness 评测多类前沿语言模型。即使最佳模型与 harness 组合，在四个评测套件上的宏平均准确率也只有68%。同一底层模型在不同 harness 下的效率差异明显；摘要未提供具体成本数值。

Insight: 长上下文能力不仅取决于能否容纳上下文，也取决于 harness 如何分配检索和推理计算。将效率与准确率一起衡量，才能区分策略性处理上下文与穷举式处理的系统。

Abstract: Language-model (LM) harnesses enable LMs to operate effectively over long contexts using additional compute. However, existing long-context evaluations are insufficient for distinguishing modern harnesses, reflected by saturated accuracy across harnesses and largely similar evaluation costs. In this paper, we introduce a benchmark for evaluating both the effectiveness and efficiency of long-context harnesses. Our tasks require diverse retrieval strategies, including lexical search and semantic matching, together with strategic and adaptive reasoning over global and local context. Much of the context is semantically relevant but only a small subset is useful at each step, creating both a challenging search problem and different accuracy–cost tradeoffs across processing strategies. For example, one task requires identifying every person satisfying several conditions using evidence scattered across documents; strategically checking the most selective condition first can narrow the search before verifying the remaining conditions. We evaluate multiple families of frontier language models with four state-of-the-art harnesses. Our benchmarks remain challenging even for strong model–harness combinations: the best reaches 68% macro-average accuracy across four evaluation suites. More importantly, we find that the same underlying model can exhibit markedly different efficiency under different harnesses. Our results establish efficiency as an important axis for long-context evaluation and provide a testbed for developing harnesses that process context strategically rather than exhaustively.

[214] Pretraining Latent Information Feedback Transformers with Teacher Supervision cs.CLPDF

Dor Tirosh, Ido Amos, Mor Geva

TL;DR: LIFT 在 Transformer 预训练中学习把信息密集的隐状态向后续生成步骤反馈给模型，以缓解深层信息只能经由已解码 token 传递的瓶颈。

Details

Motivation: 标准 Transformer 是前馈式的，深层表示不会反馈给浅层；生成过程中跨步传递信息主要依赖已解码 token，模型因此可能重复计算中间结果并丢弃其他候选。作者希望让模型以可扩展的方式学习跨步状态反馈。

Result: LIFT 用现成预训练语言模型的下一 token 分布构造信息密集的教师状态，将状态与输入 token 配对；模型新增少量参数，同时预测下一个 token 和下一个状态。教师状态预先计算，因此预训练仍可在位置间完全并行；推理时反馈模型自身预测状态，额外计算开销随模型规模增大而降低。135M至1B参数实验显示，LIFT 在语言建模、下游推理和程序化任务的 token 匹配预算下持续优于标准 Transformer 和基线，在计算量匹配时达到持平或更好。在状态跟踪任务中，一个小型 LIFT 胜过同规模但用多8倍数据训练的 Transformer，即使教师 Transformer 本身无法完成该任务。

Insight: 教师监督可把递归状态学习改写成可并行的预测目标，让模型学会在生成期间利用深层信息反馈。状态反馈带来的优势不只来自额外数据：状态跟踪实验显示，即便教师模型没掌握目标能力，反馈式架构仍可能更有效地使用监督信号。

Abstract: Transformer language models (LMs) are feed-forward: deep-layer representations are never fed back to shallower layers, and the only pathway for information to flow downward across generation steps is the decoded token. This narrow channel forces models to recompute intermediate results and to discard alternative continuations. In this work, we remove this bottleneck during pretraining, introducing the LIFT (Latent Information Feedback Transformer) architecture and training method which enable LMs to propagate state across generation. We achieve this by turning recurrent-state learning into a teacher-forced prediction problem: each input token is paired with an information-dense state, derived from the next-token distribution of an off-the-shelf pretrained LM. The model, extended with a small number of additional parameters, is then trained to predict both the next token and the next state. As the input states are precomputed, pretraining remains fully parallel across positions. At inference, the model’s own predicted states are fed back, with a minor computational overhead that decreases with model size. Experiments with pretrained models ranging from 135M to 1B parameters show that LIFT consistently outperforms standard Transformers and baselines on language modeling, downstream reasoning tasks, and procedural tasks under token-matched budget, while being on par with or ahead of compute-matched Transformers. Moreover, a controlled study on a state-tracking task shows that a tiny LIFT outperforms same-size Transformers trained on 8x more data, even when trained with the states of a Transformer that fails the task. Overall, we show that LMs can learn to exploit deep-to-shallow feedback during pretraining via scalable teacher supervision.

cs.AI [Back]

[215] Neurosymbolic Routing for Reliable Reasoning on Resource-Constrained Edge Devices cs.AI | cs.CLPDF

Avyay Sadhu, Alvaro Velasquez, Lekai Chen

TL;DR: 论文通过学习一个确定性路由器，把算术、代数和形式逻辑等结构化问题交给符号求解器，只把开放式文字题交给小语言模型，从而提升边缘设备上的准确率和效率。

Details

Motivation: 能在边缘设备运行的小模型可提供本地、低延迟推理，但在算术、代数和形式逻辑等任务上并不可靠。许多这类问题结构确定、可由符号工具快速精确求解，让概率模型近似处理会白白损失准确率并消耗能量。

Result: 路由器使用 L* 文法推断算法学习确定性有限自动机：小语言模型充当成员查询 oracle，带标签数据充当等价查询 oracle；确定性问题发往符号引擎，开放式文字题则交给小模型。在无 GPU、8GB 内存的 Raspberry Pi 4B 上，对 DeepMind Mathematics、GSM8K 和 RuleTaker 的100个未测试提示，路由准确率为100%，512 token预算下总体准确率98.3%，文字题准确率93.3%；最佳比较基线 Program-of-Thought 为72.0%，使用相同求解器的工具调用智能体为58.7%。格式明确的问题无需经过模型，延迟为1–11毫秒；30 token配置比 Program-of-Thought 快8.8倍、能效高2.8倍。

Insight: 将确定性子任务识别出来并交给精确求解器，可减少小模型必须承担的推理范围。路由器本身也可通过有限状态学习得到，而不必手工维护一套复杂分发规则。

Abstract: Running a language model on edge hardware provides private and low-latency reasoning without a network connection, and yet the small models that fit on such devices are unreliable on the tasks computers are expected to handle well, such as arithmetic, algebra, and formal logic problems. We argue that much of this unreliability is avoidable. Many queries appearing to demand reasoning are in fact structurally deterministic and permit fast and exact symbolic solutions. Therefore, forcing a probabilistic model to approximate them sacrifices accuracy and energy for little benefit. We present a neurosymbolic router that classifies each incoming query and dispatches it to the cheapest correct solver, sending structured tasks to deterministic engines and reserving the small language model (SLM) for open-ended word problems. Instead of hand-coding the routing logic, we learn a deterministic finite automaton (DFA) with the L* grammatical inference algorithm, using the SLM as a membership oracle and labeled data as an equivalence oracle. On a Raspberry Pi 4B (8 GB RAM, no GPU), evaluated on 100 untested prompts from DeepMind Mathematics, GSM8K, and RuleTaker, learned routing attains 100% routing accuracy and 98.3% overall accuracy with a 512-token reasoning budget (93.3% on word problems), compared with 72.0% for the strongest agent baseline, Program-of-Thought, and 58.7% for a tool-calling agent given the same solvers. Since formatted queries never reach the model, the router answers them in 1-11 ms and, in its 30-token configuration, runs 8.8x faster and 2.8x more energy-efficient than Program-of-Thought.

[216] GeoOutageBench: Benchmarking Ambiguity-aware, Ontology-grounded Geospatiotemporal KGQA for Multimodal Power Outage and Resilience Analysis cs.AI | cs.CL | cs.IRPDF

Ethan D. Frakes, Amy Kvien, Rishabh Kundu, Redad Mehdi, Van D. Tran

TL;DR: GeoOutageBench用于评测大语言模型在多模态地理时空知识图谱上回答停电与韧性分析问题的能力，并特别关注提问歧义、领域本体和证据检索。

Details

Motivation: 现实基础设施韧性分析需要联合利用停电记录、遥感影像、天气、风暴、电力事件、地理实体和领域知识。面向网页知识的既有KGQA基准不能充分测试此类多模态、时空及本体约束问题。

Result: 基准构建了融合视觉、文本和结构化数据的地理时空知识图谱，按难度覆盖时空包含与邻近、时空共现、多模态证据和假设评估等查询。它提供可配置评估，分别考察自然语言问题到SPARQL的歧义解释、查询驱动的本体效用评估，以及多模态KGQA检索答案的准确性。论文将其定位为支持真实基础设施韧性分析的LLM-KG系统评测基础；摘要未报告模型分数。

Insight: 这类应用的评测需要把提问解析、本体是否有用、最终检索答案是否正确分开检查。将时空结构与多模态证据放在同一知识图谱中，才能检验模型是否真的支撑跨来源的韧性分析。

Abstract: We introduce GeoOutageBench, a benchmark for assessing LLM-based geospatiotemporal KGQA for multimodal outage and resilience analysis. Unlike existing KGQA benchmarks for Web knowledge, GeoOutageBench considers a spatiotemporal KG that integrates visual, textual, and structured data from outage records, remote sensing, weather observations, storm and power events, geographic entities, and domain ontologies. It provides a competency query taxonomy at different difficulty levels from spatiotemporal containment and proximity, spatiotemporal co-occurrence analysis, multimodal evidence, to hypothetical evaluation. Over multimodal KG and query classes, GeoOutageBench provides user-configurable evaluation of three important, highly coherent yet less studied tasks: (1) LLMs’ understanding for ambiguous geospatiotemporal questions in terms of NL to SPARQL interpretation, (2) query-driven assessment of ontology utility, and (3) answer accuracy of multimodal KGQA retrieval. GeoOutageBench provides a design principle and foundation for assessing LLM-KG systems that support real-world infrastructure resilience analysis. Our benchmark, source code, data, results, and other documentation are available at https://github.com/UCF-SAGE/GeoOutageBench.

[217] SIPO: Unifying Reinforcement Learning with On-Policy Self-Distillation cs.AI | cs.CLPDF

Zhenrui Yue, Huimin Zeng, Yueqi Wang, Yaokun Liu, Fengran Mo

TL;DR: SIPO以对比式自教师为每个生成token提供稠密反馈，把可验证奖励强化学习与在线自蒸馏结合，同时保留任务奖励对整体更新方向的主导作用。

Details

Motivation: 可验证奖励强化学习多只在输出末尾给稀疏奖励，无法为中间推理步骤分配信用。在线自蒸馏能提供稠密反馈，但自教师常过度自信，并对长推理轨迹施加过强惩罚。

Result: 每轮中，SIPO从当前策略为提示采样多条轨迹并按环境奖励评分；对每条轨迹构造两个教师上下文，将参考答案与组内错误配对。模型在两个上下文下重新评价自身响应，以教师对数概率之差作为token级反馈，期望抵消两种上下文共有偏差。任务奖励仍决定更新主方向，自教师则在token间重分配信用；即使组内所有轨迹都失败、组相对优势为零，也仍能提供学习信号。在多个推理及代码生成基准上，SIPO优于RLVR和OPSD基线，无需外部教师或增加生成次数。

Insight: 对比两个带不同错误信息的自教师上下文，可从共有偏差中提取更细粒度的token反馈。保留环境奖励作为主目标，既利用稠密信用分配，也避免自教师完全取代任务标准。

Abstract: Reinforcement learning with verifiable rewards (RLVR) has become a standard paradigm for improving large language models (LLMs) on various tasks, yet its sparse outcome rewards lack token-level credit assignment for intermediate steps. To address this, on-policy self-distillation (OPSD) leverages a self-teacher with privileged context to provide additional dense learning signals. However, because the self-teacher is often overconfident and imposes excessive penalties on long reasoning trajectories, OPSD frequently struggles in practice. To mitigate this, we propose self-instructing policy optimization (SIPO) with a contrastive self-teacher to provide dense credit. At each iteration, SIPO samples multiple rollouts per prompt from the current policy, scores them with environment rewards, and constructs two teacher contexts for each rollout by pairing the reference answer with mistakes made within the group. The model then re-evaluates its own responses under both contexts, using the difference between the two teacher log-probabilities as token-level feedback, so that biases shared by both contexts are expected to largely cancel. The resulting objective yields a token-level advantage for every rollout: the reward still sets the main direction of each update while the self-teacher redistributes credit across tokens. Even in groups where every rollout fails and group-relative advantages vanish, SIPO still provides a learning signal. By preserving direct optimization of the task reward while providing dense, token-level feedback, this approach bridges reinforcement learning and on-policy self-distillation. Extensive experiments across multiple reasoning and code-generation benchmarks demonstrate that SIPO outperforms both RLVR and OPSD baselines without an external teacher or additional generation.

[218] CoEM: Empowering Long-Context Reasoning with Commit-on-Evidence Memory cs.AI | cs.CLPDF

Jingguang Li, Yebo Wu, Zuyi Guo, Kailang Ma, Xianjie Dai

TL;DR: CoEM在固定记忆预算下暂存尚未确定是否有用的原文证据，等待后续上下文到来后再决定提交为记忆、继续暂存或丢弃。

Details

Motivation: 长上下文任务常通过分块处理并维护有限文本记忆，但过早压缩会丢失后来推理可能需要的细节。关键是何时把原始证据不可逆地转成摘要事实。

Result: Commit-on-Evidence Memory（CoEM）将可能有用的原文摘录保存在pending集合，随着新上下文到达，由学习策略重新判断每条摘录应提交到记忆、继续等待还是丢弃。冻结验证器只接受能够由保留摘录和当前上下文支持的事实。记忆管理策略通过强化学习训练，结合逐步证据奖励与最终答案奖励。在输入长达6,400份文档的评估中，Qwen3.5-9B上的CoEM较最强记忆基线提升10.4至11.4个F1点。

Insight: 在长上下文里，暂缓压缩可以用少量可控存储换取后续上下文消歧的机会。将事实提交与原始证据支持绑定，能减少记忆摘要脱离证据或过早遗忘的风险。

Abstract: Long-context reasoning is essential for complex and long-horizon tasks, yet the performance of large language models (LLMs) degrades as context length increases. Recent approaches address this by processing input chunk by chunk while maintaining a bounded textual memory in model context. However, premature information compression can discard critical details essential for subsequent reasoning. In this paper, we introduce Commit-on-Evidence Memory (CoEM), which learns when to convert source evidence into compact memory facts. Specifically, under a fixed context-memory budget, CoEM preserves potentially useful source excerpts verbatim in a pending set, allowing subsequent context to clarify their relevance before irreversible compression. As new context arrives, a learned policy revisits each pending excerpt and decides whether to promote it to the committed memory, retain it for further consideration, or discard it. A frozen verifier ensures proposed facts are accepted only if supported by retained excerpts and current context. To further guide effective memory management, we train this policy using reinforcement learning by combining fine-grained, step-level evidence rewards with final answer rewards. Extensive experiments demonstrate that CoEM consistently improves long-context reasoning. When evaluated on 6,400 documents long-context input, CoEM outperforms the strongest memory baseline by 10.4-11.4 F1 points on Qwen3.5-9B. Code repository: https://github.com/benmagnifico/CoEM.

[219] Solving Without Stopping: On-Policy Distillation at Small Scale cs.AI | cs.CLPDF

Hongyang Li, Yiming Zhu, Xiao Li, Caesar Wu, Said Mammar

TL;DR: 对小模型在线策略蒸馏的分析发现，蒸馏通常能提升解题能力，却未必教会模型何时停止推理，尤其小模型更容易给出正确值后继续写下去。

Details

Motivation: 在线策略蒸馏通过更强教师对学生自身输出的反馈传递推理能力。研究比较思考模式（长推理后显式结束并作答）与非思考模式，分析蒸馏究竟提升了解题还是停止判断能力。

Result: 作者将Qwen3-8B蒸馏到Qwen3 4B、1.7B和0.6B学生模型。所有规模的解题能力均提升，但存在两类上限：单次作答不会超过该学生训练前多次尝试能够达到的水平；学生越小，表现越低于教师。非思考模式学生始终会停止；思考模式下学生在训练早期就减少结束推理，且模型越小保留下来的停止能力越弱。教师几乎只在学生原本会结束推理的地方给出停止信号，因此蒸馏没有教会新的停止位置，只保留那些恰好答对的既有停止点。小模型常已得到正确数值，却很少标记最终答案，或标记后仍继续输出。

Insight: 推理能力包含找到解答和识别已经解答两个独立能力。蒸馏能转移前者，但若教师监督未覆盖学生不会停止的状态，就难以教会后者；评估应分开统计答案标记、正确性和停止行为。

Abstract: On-policy distillation, where a student learns from a stronger teacher’s feedback on its own outputs, is a common way to pass reasoning to smaller models. We analyze what it transfers at small scale, distilling Qwen3-8B into Qwen3 4B, 1.7B and 0.6B students, in thinking mode (reason at length, then end the reasoning and answer) and, for comparison, in non-thinking mode (no separate reasoning phase). Long reasoning needs two abilities, solving a problem and knowing when it is solved, and we find that distillation transfers the first, but in thinking mode not the second. Solving improves at every size, up to two ceilings, which we measure comprehensively across both modes and all student sizes: a student’s single attempt never exceeds what it could already reach in many attempts before training, and the smaller the student, the further it stays below the teacher. Stopping is where the modes part. In non-thinking mode every student keeps stopping; in thinking mode students stop ending their reasoning early in training, and the smaller the student, the less of this ability survives: the teacher signals a stop almost only where a student already ends its reasoning, so distillation teaches no new stops; it only keeps the student’s existing stops that land on a right answer, and a weak student has few such stops. The smallest students often reach the right value but do not commit to it: they either rarely mark it or mark it and write past it. Together, these results describe how small students behave under on-policy distillation, and a diagnostic that separates answer marking, correctness and stopping.

[220] Rational Clarification by Assistive Agents via Value-of-Information Reasoning cs.AI | cs.CL | cs.HC | cs.MAPDF

T. Duy Nguyen-Hien, Yee Whye Teh, Wee Sun Lee, Tan Zhi-Xuan

TL;DR: REVOIR通过推理澄清问题带来的预期任务价值，决定何时先问用户、何时直接行动，减少无必要提问并提高偏好符合度。

Details

Motivation: 用户对话请求常含歧义，助手可以按自己的理解直接执行（有错位风险），也可以先问澄清问题。仅以降低意图不确定性为目标会忽略答案对后续任务表现的影响、提问与行动的成本，以及用户可能主动纠正的情况。

Result: REVOIR在推理时估计澄清答案带来的预期任务奖励提升，即信息价值，并据此选择是否提问。在歧义问答CondAmbigQA和偏好对齐家务规划ADAPT两项辅助任务中，它比提示、思维链、微调和信息增益方法成功率更高，同时提问更少。在ADAPT上，偏好满足度比微调澄清策略高13至15%，无需训练，提问数为其五分之一。当助手可以在行动后低成本收到用户纠正时，REVOIR会推断许多场景中先行动更有效。普通推理智能体则未能自适应澄清，且推理力度越大反而更少提问。

Insight: 减少用户目标不确定性并非澄清的唯一价值，应比较答案能否改变后续任务结果与询问成本。允许行动后纠正时，理性策略可能选择先行动，这使澄清行为能随交互条件变化。

Abstract: Users of language-based assistive agents often make ambiguous requests. In response, an assistant can either directly act on its interpretation of the request — risking misalignment with the user — or ask a clarifying question. Which option is the most safe and helpful? A common approach is to ask questions that minimize uncertainty about the user’s intent until a threshold is reached. However, this neglects the impact of uncertainty reduction on downstream performance, the costs of asking versus acting immediately, and the possibility that users may provide corrections without being asked. To navigate these trade-offs, we introduce Rational Enquiry via Value-of-Information Reasoning (REVOIR). REVOIR makes clarification decisions via inference-time reasoning about the value-of-information of a question, which captures the expected improvement in task reward due to the answer received. In two assistive tasks — ambiguous question answering (CondAmbigQA) and preference-aligned household task planning (ADAPT) — we show that REVOIR achieves greater success with fewer questions than approaches based on prompting, chain-of-thought, fine-tuning, or information gain, improving preference satisfaction on ADAPT by 13-15% over a fine-tuned clarification policy while requiring no training and asking five times fewer questions. Furthermore, when the assistant can receive cheap user corrections after acting, REVOIR naturally infers that asking questions is not always efficient, demonstrating the adaptivity of our approach. In contrast, we find that vanilla reasoning agents fail to adaptively clarify user requests, and request fewer clarifications as reasoning effort increases.

[221] EngiWorld: What Can Frontier Agents Deliver in Professional Engineering Environments? cs.AI | cs.CLPDF

Hongcheng Gao, Hailong Qu, Yu Lei, Henghui Sun, Haoyang Li

TL;DR: EngiWorld以完整专业工程设计流程评测智能体，使用领域验证器程序化检查最终及中间设计工件的几何、物理和规范约束。

Details

Motivation: 通用电脑使用智能体进步迅速，但专业工业工程自动化仍不可靠，因为工作流需要跨软件和设计阶段持续满足几何、物理约束及依赖关系。

Result: 基准包含1,301项专家整理任务，覆盖CAD、CAE、CAM、BIM、EDA和3D可视化六个工程领域、26个专业软件平台，支持GUI与CLI及从软件选择到开放任务的六类任务。工件中心评估套件会程序化校验最终及中间结果的几何有效性、物理可行性和规则合规性；定量设计按规格达成度连续评分而非简单成败。七个前沿模型中最强者EngiScore仅44.3，多软件任务尝试成功率仅3.6%。

Insight: 专业工程任务的可靠评估必须检查生成的设计工件本身，并覆盖跨软件的中间状态，而不能只看对话是否像是完成了任务。当前前沿模型在端到端多软件工程流程上仍有明显能力差距。

Abstract: Autonomous agents have made rapid progress in general-purpose computer use, but reliable automation of professional industrial engineering remains out of reach, as engineering workflows demand reasoning over geometric and physical constraints and dependencies preserved across software and design stages. We present EngiWorld, the first benchmark structured around the complete design loop: 1,301 expert-curated tasks spanning 6 engineering domains (CAD, CAE, CAM, BIM, EDA, and 3D visualization) and 26 professional software platforms, with both GUI and CLI interfaces and 6 task types ranging from software-selection to open-ended tasks. We further introduce an artifact-centric evaluation methodology built on a unified domain-verifier suite, which programmatically checks the geometric validity, physical feasibility, and rule compliance of final and intermediate artifacts, and scores quantitative design tasks continuously by specification attainment rather than binary success. Evaluation of seven frontier models reveals a substantial capability gap: the strongest model achieves an EngiScore of only 44.3, and just 3.6% of multi-software attempts succeed. EngiWorld provides the first rigorous foundation for measuring progress toward agents that operate professional engineering software end to end.

[222] Context Language Models cs.AI | cs.CL | cs.LGPDF

Rulin Shao, Shannon Zejiang Shen, Junjie Oscar Yin, Yuetai Li, Minheng Wang

TL;DR: Context Language Models将上下文视为模型可自主读写的文件，让模型原生管理保留内容，并可扩展到多个智能体共享不同上下文。

Details

Motivation: 长任务和智能体协作需要控制上下文增长、判断什么信息值得保留。现有方案多由外部harness控制记忆，作者提出把上下文管理能力放进模型本身。

Result: CLM把上下文视为文件，允许模型不受限地更新，从而学习保留重要信息，也支持多个智能体上下文并存。用现有模型零样本构建CLM，在多种任务上超过先进上下文管理策略：BrowseComp-Plus准确率提高11.4%、FLOPs减少21.5%；12小时EdgeBench分数提高5%、FLOPs减少59%；24小时多仓库智能体群任务在相同计算下提升幅度高65%。自然语言指令经技能优化循环可引导CLM，在留出上下文管理任务上最多提高35.9分并减少计算。在线RL使Qwen3.5-9B在BrowseComp-Plus提高47.6%，同时减少12% FLOPs。后缀缓存复用相较标准SGLang在性能匹配时再减少35%服务端计算。

Insight: 上下文管理可作为模型可学习和可调用的内部行为，而不只由外围系统决定。文件式上下文既支持自然语言技能指导与强化学习，也允许通过缓存复用降低服务成本。

Abstract: We introduce Context Language Models (CLMs), language models that natively manage their own context. We implement this by treating the context as a file and allowing the model to make unrestricted updates to this file. This allows the model to learn what is most important to maintain in context, and naturally extends to multi-agent systems where multiple agent contexts coexist as files. Building CLMs zero-shot with existing models outperforms SOTA context management strategies across a variety of tasks: 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp-Plus, 5% higher scores with 59% fewer FLOPs on 12-hour EdgeBench, and 65% greater improvement with the same compute on a 24-hour multi-repository agent-swarm task. Moreover, by shifting context management from external harness control to intrinsic model behavior, CLMs naturally enable both in-context and parametric learning of context-management strategies. We show that CLMs can be steered with natural-language instructions evolved through a standard skill-optimization loop, improving held-out accuracy by up to 35.9 points on a context-management task while reducing compute. We also introduce an online reinforcement learning method for CLMs, improving Qwen3.5-9B performance on BrowseComp-Plus by 47.6% while using 12% fewer FLOPs. Finally, we co-design Suffix Cache Reuse for CLM serving, further reducing server-side compute by 35% relative to standard SGLang at matched performance.

[223] Can a Cacheable Decision Model Follow Rules? cs.AI | cs.CL | cs.LGPDF

Dushyant Rajput, Nirdesh Chauhan, Siddharth Kosaraju

TL;DR: 论文检验可缓存的独立候选评分模型能否遵守规则，发现缓存效率提升会显著削弱规则敏感度，合成训练可部分恢复但真实规则迁移尚未证实。

Details

Motivation: 联合评分器同时输入状态、规则与候选，准确但每个菜单项都要重复编码；独立编码允许候选缓存并跨状态复用，在77个候选时约便宜5倍，却把状态和候选分开。研究考察这种效率变化对规则跟随的影响。

Result: 在Certo上，转为可缓存评分后规则敏感度recall@1从1.00降至0.24，联合评分器维持1.00，短名单加重排未能挽回。针对性反事实监督可在留出的合成规则任务（释义、反事实和组合）恢复较强表现且跨种子可复现，但实验未确认预测确实依赖输入规则。真实规则上优势尚未建立：修正截断混淆因素后，短难度层联合评分显著更好（0.861对0.500），困难层也有同方向差异（0.655对0.483，n=29）。跨域真实文本混合数据无助于迁移，合同准确率下降9.3和16.2个百分点。作者结论是训练分布内可培养规则敏感的缓存编码器，但对未见来源真实规则的迁移仍未证实。

Insight: 可缓存的表示复用会把候选与规则状态解耦，可能损害规则依赖判断；在合成分布上恢复能力不等于真实跨领域泛化。评价时还需检查模型是否真正使用给定规则，而非仅凭预测命中率。

Abstract: Certo is a small non-generative decision model (Qwen3-4B): it scores candidate actions from their text and returns a probability, instead of generating an answer. The accurate design reads the state, the rules, and each candidate together (a joint scorer), so cost grows with the menu. Independent encoding lets each candidate be encoded once and reused across states (about 5x cheaper at 77 candidates), but separates state from candidate. We ask how much rule-sensitivity survives that move, and whether it can be trained back. Four experiments on Certo: (1) the tested conversion to cacheable scoring loses rule-sensitivity (recall@1 1.00 -> 0.24) while the joint scorer holds 1.00, and a shortlist+rerank rescue fails; (2) targeted counterfactual supervision restores strong performance on held-out synthetic rule tasks (paraphrase, counterfactual, composition; reproducible across seeds), though we do not isolate whether predictions depend on the supplied rule; (3) on real rules the added benefit is not established – after fixing a truncation confound, the joint scorer wins significantly on the short tier (0.861 vs 0.500) and directionally on the hard tier (0.655 vs 0.483, n=29); (4) a matched cross-domain real-prose mixture did not help and reduced contract accuracy (-9.3, -16.2 points). A cacheable encoder can be made rule-sensitive on its training distribution, but transfer to unseen-source real rules is not established; the joint scorer keeps an edge at the cost of caching.

[224] SelfSearch: Reward-Free Search for Self-Improving Agents cs.AI | cs.CL | cs.LGPDF

Jungwoo Yang, In Jin Kong, Yohan Jo

TL;DR: SelfSearch让智能体基于过去的自我改进经历修改自身指令、工具和执行流程，不依赖下游任务奖励即可提升任务成功率和效率。

Details

Motivation: 代码能力增强后，语言模型智能体可检查并修改自身指令、工具和执行程序，但常见搜索依赖反复评估下游任务，成本高且局限于被评估的任务。

Result: SelfSearch从此前自我改进过程中积累的推理、工具操作和结果记录中学习，让智能体在无下游奖励的搜索阶段改进任务求解及自我修改。在六种模型—基准设置中，群体平均成功率均高于初始智能体；Terminal-Bench 2.1单个智能体最高提高11.2个百分点。SWE-bench Multilingual上成功率提高5.0个百分点，且在新旧系统都能完成的任务上执行成本减少38.5%。以4.03美元搜索成本构造的harness，在公开九种harness比较设定下让DeepSeek V4 Flash解决82.0%的Terminal-Bench 2.1任务，与最高分Codex持平。

Insight: 自我修改过程本身能形成可复用经验，无需在每轮搜索中调用昂贵的下游奖励评估。将成功经验与失败尝试记录下来，可以同时改进智能体能力和执行成本。

Abstract: Advances in the coding capabilities of LLM agents allow them to inspect and modify their own instructions, tools, and execution procedures. Existing approaches use this ability to search for improved agents through repeated downstream evaluation, which incurs substantial costs and ties the search to the evaluated tasks. We introduce \textbf{SelfSearch}, a reward-free search procedure in which agents modify themselves using records of previous self-improvement episodes. These records capture the reasoning, tool actions, and outcomes of earlier modification attempts, providing concrete experience for improving both task solving and self-modification. Without downstream reward signals during search, SelfSearch improves population-mean success over the initial agent in all six model–benchmark settings, with individual agents gaining up to 11.2 percentage points on Terminal-Bench 2.1. On SWE-bench Multilingual, an agent improves success by \textbf{5.0} percentage points while reducing execution cost by \textbf{38.5}% on tasks solved by both the initial and evolved agents. SelfSearch achieves competitive task success with evaluation-guided search baselines at lower search cost. With only \textbf{$4.03} in search cost, it produces a harness that solves \textbf{82.0}% of Terminal-Bench 2.1 tasks with DeepSeek V4 Flash under the settings of a public nine-harness comparison, matching the top-scoring harness, Codex. These results suggest that experience gained through self-modification can improve agents’ downstream capabilities and efficiency.

[225] AdviSD: Learning to Advise Frontier LLMs via Targeted Multi-Turn Self-Distillation cs.AI | cs.CL | cs.LGPDF

Rishabh Agrawal, Hejie Cui, Shasha Li, Shanchan Wu, Sercan Ö. Arık

TL;DR: AdviSD用执行结果反馈训练小型顾问模型为冻结的大语言模型提供自然语言建议，并按建议对已记录执行响应产生的实际影响选择性蒸馏。

Details

Motivation: 小型可训练顾问可通过自然语言建议引导冻结执行器，也能根据交互结果改进建议。但看似合理的反馈未必改变实际执行；若盲目学习这些反馈，仍可能改变顾问在其他场景的决策。

Result: 作者证明，在共享参数模型中，如果某类反馈对应目标对有用建议的偏好弱于其他反馈，学习该类反馈会限制最终能力；减少对这些反馈的训练反而更好。AdviSD将基于结果的强化学习与来自反馈条件顾问副本的选择性自蒸馏结合：反思模块提出修正，顾问对同一条已记录执行响应分别评估有无自身建议时的表现差异，并据此选择监督样本，无需执行器似然或额外rollout。Qwen3-8B顾问为Gemini和Claude提供建议时，在BFCL-v3比顾问GRPO高4.2至6.4个百分点，在EnvScaler高3.9至5.1分；可泛化到域外任务及不同执行器版本、模型家族，且优于同数量随机样本选择。

Insight: 顾问反馈的学习价值应取决于它是否改变冻结执行器的行为，而不能只看语言上是否像合理修正。比较同一响应在有无建议时的差异，为选择性蒸馏提供了无需额外执行成本的信号。

Abstract: A small trainable advisor can steer a frozen language-model executor using natural-language advice. In addition to learning from task rewards, the advisor can use feedback from completed interactions to improve its advice. However, a plausible correction need not change execution, yet learning from such corrections can still affect the advisor’s future decisions in other contexts. In a shared-parameter model, we prove that such corrections can limit learning if their targets favor useful advice less strongly than those of other corrections. Keeping them less often than the rest improves the model’s eventual performance compared to learning from every correction. Motivated by this, our method, Advisor Self-Distillation (AdviSD), pairs outcome-based reinforcement learning with self-distillation from a feedback-conditioned copy of the advisor selectively. Reflection proposes corrections, and the advisor scores the same recorded executor response with and without its issued advice, using the magnitude of the difference to select decisions for supervision. This approach does not require executor likelihoods or additional executor rollouts. Experiments with Qwen3-8B advisors for Gemini and Claude show that AdviSD outperforms advisor-GRPO by 4.2-6.4 percentage points on BFCL-v3 and by 3.9-5.1 score points on EnvScaler. The trained advisors generalize to out-of-domain tasks and transfer across different executor versions and model families. AdviSD also beats matched-count random selection, supporting the value of its selection rule.

[226] Learning Meta-Skills for Agent Harness Design in Test-Time AI4AI cs.AI | cs.CL | cs.LGPDF

Cheng Qian, Kunlun Zhu, Beibin Li, Zhenhailong Wang, Heng Ji

TL;DR: 论文研究测试时AI-for-AI：让Builder模型从目标智能体反馈中学习可复用的元技能，再为未见任务构建更好的执行环境。

Details

Motivation: 智能体表现既受推理能力影响，也取决于其运行环境和工具配置。作者探索如何在Builder与Target模型参数都冻结时，由Builder学习构建更有效的执行harness，并将经验迁移到新任务。

Result: Meta-Skill以“何时需要何种支持资源”的原则形式记录Builder经验。Builder根据Target在开发集上的执行反馈学习元技能，之后用冻结技能库为未见任务构造harness。在Harness-Bench和NewtonBench上，完整元技能库比无技能构建的宏平均表现高8.95个百分点；比直接把相同技能库交给Target高12.02个百分点。相同模型同时担任Builder和Target时也有收益，提示学习如何构建环境可成为系统级自我改进途径。

Insight: 将经验提炼成何时提供何种资源的可执行技能，比直接把经验文本交给目标模型更有效。测试时构建环境可在不更新模型权重的情况下改善任务表现，并具有自我改进潜力。

Abstract: Agent performance depends on both reasoning ability and the environment in which it acts. We study test-time AI-for-AI, asking how a Builder can learn to construct better execution environments for a Target while both models’ weights remain fixed. To make the Builder’s experience reusable, we introduce Meta-Skill: principles specifying when support is needed and what resources to provide. The Builder learns these principles from Target’s execution feedback on the development set, then uses the frozen skill bank to construct harnesses for unseen tasks. Across Harness-Bench and NewtonBench, full-bank meta-skills improve macro-average performance by 8.95 percentage points over no-skill construction, and 12.02 points over direct delivery of the same bank to the Target. These results highlight the value of translating experience into executable support. Gains when the same model serves both roles further suggest a path to system level self-improvement through learning to build better environments.

[227] Emergent Specialization in Populations of Self-Supervised Collaborative Vision Experts Without a Shared Gate or Cross-Agent Gradients cs.AI | cs.CV | cs.LGPDF

Aram Davtyan, Pablo Acuaviva, Sebastian Stapf, Paolo Favaro

TL;DR: DISCO展示多个独立自监督视觉网络可在无共享门控和跨模型梯度下自发分工，并通过本地选择协作者利用群体中的专业能力。

Details

Motivation: MoE通常由联合训练的中心门控把输入路由给专家。作者研究是否能让相互独立训练、只可通过前向调用交流的网络群体，在异质视觉数据上自行形成有用分工。

Result: 研究从一组初始相同的网络出发，在六个视觉领域混合的无标签数据上，以冻结DINOv3特征掩码预测微调，并比较专业化程度与群体利用率。逐步移除中心控制后形成DISCO：每个智能体本地选择帮手，通过无梯度通道读取内部状态，仅按协助带来的增益奖励自身路由器。随机路由群体不如单一通才，语义路由群体则超过通才，说明收益来自专业化而非单纯增加模型数。无需中心路由时仍可形成专业化；被专家帮助的随机成员可达到独立通才水平，专家本身还可推广到专业化混合之外的数据。本地路由器在98%的输入上选择涌现专家，效果跨群体规模、模型容量、数据不平衡和微调种子保持。

Insight: 分布式协作可通过局部的“求助是否带来收益”信号形成专长分工，不必共享梯度或中心门控。语义路由带来的互补专长，比单纯扩大相同通才网络群体更重要。

Abstract: Can a population of neural networks develop a useful division of labor without a shared gate or gradients between agents? We study a setting where each network has its own weights, trains independently on the same heterogeneous data, and can ask another agent for help through a forward pass. Unlike mixtures of experts, where a jointly trained gate assigns inputs to experts, specialization here must emerge without central control. We test this in a small scale proxy for predictive visual pretraining. Initially identical agents are finetuned on an unlabeled mixture of six visual domains using masked prediction of frozen DINOv3 features. We measure specialization by asking whether the best agent for an input aligns with its latent domain, and utilization by asking whether responsibility is distributed across agents. We progressively remove central control, ending with DISCO (DIStributed COllaboration) where each agent locally selects a helper, reads its internal state through a gradient free channel, and rewards its router only for the improvement that help provides. Specialization emerges and is useful. Randomly routed populations underperform a single generalist, while semantically routed populations outperform it, showing that specialization rather than population size drives the gain. Specialization persists without a central router, and gradient free communication lets nonexperts exploit emergent expertise. In DISCO, a random agent helped by the expert matches the solo generalist, while experts surpass it, including on data outside the specialization mixture. Local routers select the emergent expert for 98% of inputs. These effects persist across population size, model capacity, data imbalance, and finetuning seeds, providing measurable evidence for the dynamics needed by decentralized predictive pretraining.

[228] Pixels to Keys: Exploring Spatial and Motion Cues in Gameplay Inverse Dynamics cs.AI | cs.CVPDF

Abhishek Pillai, Ekta Prashnani, Joohwan Kim, Iuri Frosio

TL;DR: 研究在数据有限设置下比较游戏逆动力学模型如何从视频恢复按键动作，并分析运动特征、模型结构及损失设计对稀有动作识别的影响。

Details

Motivation: 游戏视频提供可扩展的具身感知与控制研究环境，但网上视频通常没有玩家按键记录。逆动力学模型可从连续画面推断输入，不过已有研究多用总体准确率，可能掩盖罕见按键动作的失败，也未厘清关键设计因素。

Result: 作者研究空间运动特征、网络架构和训练目标，并以每个按键指标及宏平均F1评估。Trackmania实验显示模型架构和预处理阶段的运动流提取等因素重要，也说明类别不平衡下总体准确率具有误导性。将相同架构和配方用于Cyberpunk 2077后，不同游戏机制上的表现并不均衡。逐动作分析发现相机运动、延迟效果和按键频率不平衡都会造成歧义，未来需显式建模三维场景结构和长时状态，并采用适当损失。摘要未给具体数字。

Insight: 从画面反推控制输入是时序逆问题，动作效果可能延迟或与相机移动混淆。逐按键平衡指标比总体准确率更能揭示实际失败模式，并指向更长期、更具结构的场景表示。

Abstract: Video games offer scalable environments for studying perception and control in embodied agents.Abundant online gameplay videos could supply demonstrations, but they rarely include player inputs for training. Inverse Dynamics Models (IDMs) have thus been proposed to infer inputs from frames. Large (up to 1B parameters) IDMs trained on $\sim$1K-2K gameplay hours demonstrate feasibility and cross-environment generalization at this scale, but researchers do not clarify what the key components are to recover individual actions and often report only aggregate accuracy that can mask rare-action failures. We study the problem in a data-constrained scenario to evaluate how spatial motion features, model architectures, and training objectives affect an IDM’s outcome and we analyse our models on per-key and balanced metrics such as $F_1^{macro}$. Our experiments on Trackmania highlight the importance of factors like the model architecture and motion flow extraction in preprocessing, while also showing the limits of evaluation through unbalanced metrics. The application of the same architecture and training recipe to Cyberpunk 2077 reveals uneven performance across game mechanics. Our per-action evaluation and failure analysis highlight ambiguities from camera motion, delayed effects and imbalanced key-press frequencies that call for explicit modeling of 3D scene structure, long-term state and the adoption of proper losses in future implementations.

[229] Video-RSI: Recursive Self-Improvement of Video Understanding Agents via Harness Evolution cs.AI | cs.CVPDF

Bingjun Luo, Jialin Guo, Siqi Li

TL;DR: Video-RSI允许长视频智能体主动回看原始训练视频以检验失败原因，并按答案准确率与视觉成本共同决定保留哪些harness改动。

Details

Motivation: 视频理解智能体依赖可执行harness决定观察哪些帧及如何利用证据。但执行日志只记录当前harness采集到的信息，无法区分失败原因，也限制自我改进依据。

Result: Video-RSI让智能体通过主动视频调查重新查看原始训练视频，用额外观察检验竞争性失败解释，再根据证据修改harness。成本感知的演化过程将诊断转成可复用修订，并同时考虑回答准确率和视觉处理成本决定是否保留。视频理解基准评估中，进化后的智能体提高准确率且处理帧数更少，在准确率—效率权衡上与现有智能体有竞争力；摘要未提供具体指标。

Insight: 改进证据获取流程需要超越当前执行日志，主动查看原始数据来区分失败解释。把视觉成本纳入harness选择，可避免单纯追求准确率而无限增加帧处理量。

Abstract: Video understanding agents acquire evidence through an executable harness that controls what they observe and how they use those observations. However, execution traces contain only the evidence acquired by the current harness, leaving competing explanations for failure unresolved and limiting the basis for self-improvement. We introduce Video-RSI, a framework for recursive self-improvement in which a video understanding agent uses its own language model to revise its harness. Through active video investigation, the model revisits the original training videos to test competing failure explanations with additional observations, grounding proposed changes in evidence beyond the existing trace. Cost-aware harness evolution turns these diagnoses into reusable revisions and determines which revisions to retain by considering both answer accuracy and visual cost. Across our evaluation settings on video understanding benchmarks, the evolved agent improves accuracy while processing fewer frames and achieves competitive accuracy-efficiency trade-offs against existing video understanding agents. These results demonstrate the potential for video understanding agents to improve their own evidence acquisition and use through harness evolution. Code is available at https://github.com/bingjunluo/Video-RSI .

[230] Brain-SAD: A Brain-Inspired Safe Autonomous Driving Control Framework with Dynamic Fear-Oriented Constraint on Dual-Policy cs.AI | cs.CV | cs.NE | cs.ROPDF

Huan Rong, Chao Yin, Anouar Imel, Yijie Xia, Tinghuai Ma

TL;DR: Brain-SAD根据实时车辆交互场景生成动态恐惧信号，在常规长期策略与紧急短期避撞策略之间切换，并在线调整安全约束。

Details

Motivation: 现有安全强化学习常用静态状态—成本映射或固定可行域边界，这些约束绑定于训练场景，面对复杂度变化的交互时可能无法正确反映动作风险。

Result: Brain-SAD在感知当前车辆交互后产生动态恐惧信号，据此选择用于常规交互的长期策略或处理紧急碰撞防御的短期策略。恐惧反应分别转为与动作影响直接关联的整体恐惧成本，以及由不同风险邻车决定的动态可行域边界；两者都用于在线策略优化。实验报告其相较现有方法成功率更高、任务完成和碰撞恢复时间更短，并在复杂度变化的连续交叉路口具有更强可靠性；摘要未给具体指标。

Insight: 安全约束应随当前交互和邻车风险变化，而非固定为离线演示估出的边界。分层长期与紧急短期策略让动态风险信号既能影响总体行为，也能收紧局部可行动作范围。

Abstract: Constrained Reinforcement Learning has recently gained increasing attention in the field of Safe Autonomous Driving, where the general mechanism is to maximize the expected reward while keeping the overall action risk bounded. In this way, the safety issues arising in AD can be mitigated through constrained actions. However, existing Constrained RL methods still lack dynamics on the imposed constraints. For instance, the action cost adopted by the existing Primal-Dual/soft-constrained methods is often defined as static state-to-cost mapping, and the safe-action projection in hard-constrained methods relies on the static projection with the fixed feasible region boundary estimated from offline demonstrations. The above drawback tightly couples the imposed constraints to the training scenarios, leaving the AD policy hard to handle different interaction scenarios, due to the improper state-level action-cost and the static projection boundary. Consequently, in this paper, we propose Brain-SAD, a brain-inspired safe autonomous driving control framework with dynamic fear-oriented constraints. By perceiving the current vehicle-interaction scene, Brain-SAD generates dynamic fear signal as fear reaction to online decide long-term policy for regular interaction or short-term policy for urgent-collision defense. In such two policy, the above fear-reaction will be constructed as the dynamic fear constraints, respectively reflecting the overall fear cost directly coupled with action-impact, and the dynamic fear boundary of the feasible region derived from different risky neighbors, both of which will in turn serve for the online policy optimization. Experimental results show that Brain-SAD outperforms existing methods, achieving higher success rate in shorter task-completion and collision-recovery time, and exhibits stronger reliability across continuous intersections of fluctuating complexity.

cs.CR [Back]

[231] Selective Channel Restoration for Backdoored Vision-Language Models cs.CR | cs.CVPDF

Shuming Liu, Zhifang Zhang, Suqin Yuan, Khin Mi Mi Aung, Zhuoyi Lin

TL;DR: PSR通过扰动筛选识别后门VLM投影层中异常脆弱的输出通道，并将其参数恢复到预训练值，以稀疏后训练修复后门。

Details

Motivation: 视觉语言模型可能被投毒微调数据植入后门。既有防御可能需要大量参数更新，或在每次推理时增加额外计算。作者发现后门模型投影器对有界扰动比干净模型更敏感。

Result: Perturb-Select-Restore（PSR）先检测每个投影层中对扰动最敏感的输出通道，再把这些通道恢复为对应预训练参数；只对投影接口做稀疏更新，推理时不增加计算。多个任务上的实验显示，PSR将攻击成功率降至接近零，同时保留干净任务性能。摘要未给出具体攻击率或干净精度数值。

Insight: 后门微调可能在投影接口留下局部脆弱通道，可将扰动敏感度用作定位信号并仅恢复少量参数。这种后训练防御将修复成本与推理开销分开控制。

Abstract: Vision-language models (VLMs) exhibit strong multimodal capabilities but remain vulnerable to backdoors implanted through poisoned fine-tuning data. Existing defenses often require extensive parameter updates during fine-tuning or incur per-query overhead during inference. To address these limitations, we propose Perturb-Select-Restore (PSR), a post-training defense that performs sparse updates to the projection interface and introduces no additional computation during inference. We reveal that backdoored VLM projectors are substantially more sensitive to bounded perturbations than clean VLM projectors, a phenomenon we term projection fragility. Building on this finding, PSR identifies the output channels most sensitive to perturbations in each projection layer of a backdoored VLM and restores their parameters to the corresponding pretrained values. Experiments across multiple tasks show that PSR reduces attack success rates to near zero while preserving clean-task performance.

cs.HC [Back]

[232] Toward a Culturally Adapted Chinese Language Agent: A Wizard-of-Oz Study of Nonverbal Behavior in Chinese-German Intercultural Interaction cs.HC | cs.CL | cs.CV | cs.MMPDF

Siddhant Jain, Anna Lea Reinwarth, Dimitra Tsovaltzi, Rafael Math, Julia Renner

TL;DR: 论文提出一个 Wizard-of-Oz 研究设计与实时采集系统，用于记录中国母语者面对德国中文学习者违反社会规范时的多模态反应，为文化适配型中文语言智能体研究建库。

Details

Motivation: 跨文化交流不只依赖语法能力，还需要理解嵌入文化的社会规范及其引发的细微非语言反应。面向德语背景中文学习者的现有语言学习智能体，对这类文化敏感能力支持不足。

Result: 系统由 MetaHuman 写实化身、Live Link 面部捕捉、MediaPipe 上半身追踪和供实验员实时选择行为的控制台组成，并同步记录智能体与学习者的多模态流。分层标注框架覆盖不可直接观察的社会情绪反应、规范解释、言语和可观察行为，并预设未来监督目标。研究团队与文化及教学专家设计了四种生态效度较高的互动情景，为后续自动文化理解与行为生成模型提供方法和技术基础；本文摘要描述的是研究设计与系统，未报告模型效果指标。

Insight: 文化适配数据需要同时记录可见动作、言语和隐含的规范解释或情绪反应，并保持双方流的时间同步。Wizard-of-Oz 采集可先建立高质量行为数据与监督目标，再支撑自动化能力研究。

Abstract: Successful intercultural communication requires more than grammatical competence. It demands sensitivity to culturally embedded social norms whose violation triggers subtle but meaningful nonverbal responses. For German learners of Mandarin Chinese, acquiring this sensitivity is critical yet poorly supported by existing language-learning agents. We present a Wizard-of-Oz (WoZ) study design and supporting real-time system for collecting multimodal behavioral data from native Chinese speakers reacting to social norm violations by German learners. The system features a photorealistic MetaHuman avatar driven by Live Link face capture and MediaPipe upper-body tracking, a wizard console for real-time behavior selection, and synchronized multimodal logging across agent and learner streams. A layered annotation framework, based on psychological theory and covering non-observable socioemotional reactions, norm interpretation, verbal, and observable behavior thereof, and future supervision targets enables the corpus to support training of future automated cultural interpretation and behavior generation models. Four ecologically valid interaction scenarios, developed with cultural and pedagogical experts, provide the methodological and technical foundation for a culturally adapted conversational agent for Chinese language learning.

[233] Large Language Models Exhibit Human-Like Bayesian Hypocrisy cs.HC | cs.CL | cs.CYPDF

Nykko Vitali, Mahzarin R. Banaji

TL;DR: 五组实验显示，GPT-4o与Claude 3.7 Sonnet的贝叶斯推理接近人类水平，但它们会像人类、且程度更高地谴责他人做出与自己相同的贝叶斯判断。

Details

Motivation: 前沿语言模型在贝叶斯任务上被期待至少达到人类水平；人类还会出现“自己犯同样错误却谴责他人”的判断不一致。作者检验模型是否也存在这种自我表现与他人评价之间的分离。

Result: 实验包含48种条件、超过5,000次试验，使用两类贝叶斯推理任务，并让模型评价作出相同判断的假想个体的能力与道德。GPT-4o和Claude 3.7 Sonnet的任务表现大致接近人类，但推理更规则化、僵化。模型也会谴责做出与自己相同贝叶斯判断的人，而且这种虚伪程度高于人类。摘要未报告各条件准确率的具体数值。

Insight: 模型不仅可能在统计推理上出错，还可能把自我表现和对他人的能力或道德评价割裂开来。统计忠实性与公平规范发生冲突时，不能仅凭模型给出的判断或评价将其视为一致可靠。

Abstract: Given recent achievements of large language models (LLMs), frontier models are expected to perform well on Bayesian reasoning tasks, at least as well as humans. Furthermore, there is no reason to expect that LLMs will condemn others who offer those very same Bayesian judgments, a fallibility observed in human decision-making (Cao, et al., 2019). In 5 experiments with 48 experimental conditions employing over 5,000 trials, GPT-4o and Claude 3.7 Sonnet were tested on two variations of a Bayesian reasoning task. We also assessed LLM evaluation of the competence and morality of a hypothetical person who had offered the same reasoning task as them. LLMs hovered near human performance on the Bayesian task, though their reasoning was more rule-based and rigid. Surprisingly, like humans but to a greater extent, LLMs also demonstrated the same hypocrisy in condemning others who, like them, had deployed Bayes’ rule. In demonstrating Bayesian hypocrisy, LLMs highlight a humanlike error of a dissociation between self-performance and other-judgment, and caution against their use in domains where statistical fidelity and fairness norms collide.

cs.LG [Back]

[234] In-Context Learning Amplifies a Latent Symbolic Circuit cs.LG | cs.AI | cs.CLPDF

Melissa Wessel

TL;DR: 对三类模型的机制追踪表明，少样本上下文学习会放大权重中已有的抽象—归纳—检索符号回路；相关回路在准确率很高之前就已可检测并发挥作用。

Details

Motivation: 大语言模型能从少量上下文示例学习抽象规则，但示例逐渐增加时内部机制如何被激活仍不清楚。作者聚焦由抽象、归纳和检索构成的三阶段符号推理回路。

Result: 研究在三个模型家族和不同示例数量下追踪回路。单头因果贡献从1-shot到10-shot最高增长8倍；跨示例激活修补将0-shot准确率从1%提升到56%，将1-shot从17%提升到88%。在0-shot下缩放并注入函数向量可将准确率提升至最高86%，主要替代归纳阶段，但依赖下游检索阶段保持完整。结果支持：抽象规则处理所需的机制已存在于模型权重中，上下文示例、函数向量及类似干预为同一潜在回路提供输入。

Insight: 上下文示例可能不是从零创建符号推理能力，而是逐步激活权重中已有的机制。机制分析也显示各子阶段并非完全可替代：增强归纳信号仍需下游检索正常工作。

Abstract: Large language models can learn abstract rules from just a few in-context examples, but how their internal mechanisms activate as examples accumulate is not well understood. We trace a three-stage symbolic reasoning circuit (abstraction, induction, retrieval) across shot counts in three model families and find it is detectable and functional well before the model achieves high accuracy. Per-head causal contribution grows up to 8x from 1- to 10-shot, and cross-shot activation patching raises accuracy from 1% to 56% at 0-shot and 17% to 88% at 1-shot. Function vectors scaled and injected at 0-shot rescue accuracy up to 86%, largely substituting for the induction stage but depending critically on an intact downstream retrieval stage. The infrastructure for abstract rule-following is present in the weights before any demonstrations; in-context examples, function vectors, and related interventions appear to supply input to the same latent circuit.

[235] MoRE: Scaling mixture of experts with hardware-aware low-rank routing cs.LG | cs.AI | cs.CL | stat.MLPDF

Honam Wong, Surbhi Goel, Enric Boix-Adserà

TL;DR: MoRE把MoE路由器的权重矩阵低秩分解，使专家数很大时的每token路由成本从随隐藏维度与专家数相乘增长，降为近似线性低秩成本。

Details

Motivation: 当MoE架构采用越来越多、越来越小的专家时，标准线性路由器每个token的成本为Θ(Mh)，其中M是专家数、h是隐藏维度，可能成为层内瓶颈。

Result: MoRE将路由权重矩阵分解为秩r，使路由成本降至O((h+M)r)。作者证明，在激活专家数固定时，秩对数级于M足以保持路由表达能力，且在精度因素范围内也是必要的；在高斯记忆模型中，对数秩还能维持负载均衡。合成电话簿任务显示低秩不损害记忆能力；相同激活FLOPs下，因子分解可容纳Θ(h/r)倍更多专家。为将理论节省转为实际速度，作者实现了减少HBM昂贵访存的融合Triton推理内核。实验中，MoRE改善电话簿记忆任务和预训练后知识密集问答表现，同时保持推理能力。

Insight: 扩大专家数量时，路由计算和内存访问会限制MoE扩展；低秩路由在理论表达能力、负载均衡和可部署内核之间建立了联系。只减少FLOPs不足以保证墙钟收益，因此实现也针对HBM访存优化。

Abstract: Mixture-of-Experts (MoE) layers are central to frontier language models, and recent architectures push toward more and smaller experts. In this regime, the standard linear router becomes a bottleneck: with $M$ experts and hidden dimension $h$, its per-token cost $Θ(Mh)$ dominates the MoE layer once $M$ is large. We introduce MoRE (Mixture of Rank-reduced-routed Experts), which factorizes the router weight matrix at rank $r$ and reduces the routing cost to $O((h + M)r)$. We prove that rank logarithmic in $M$ suffices for routing expressivity when the number of active experts is fixed, and is necessary up to precision factors. We also prove that logarithmic rank preserves load balance in a Gaussian memorization model, and training on a synthetic phonebook task shows that low rank does not hurt memorization. At matched active FLOPs, the factorization allows a factor of $Θ(h/r)$ more experts. To realize this gain in wall-clock time, we design a fused Triton kernel at inference that avoids expensive memory operations on HBM. Empirically, MoRE improves memorization on the phonebook task and performance on knowledge-intensive Q&A benchmarks after pretraining, while matching reasoning ability. Code available at https://github.com/Matheart/MoRE_code.

[236] Fisher-IRG: Fisher-Induced Local Invariant Representation Geometry across Language and Vision Models cs.LG | cs.CL | cs.CVPDF

Abdullah All Tanvir, Xin Zhong

TL;DR: Fisher-IRG根据表示方向对模型预测的局部敏感度，区分语义保持变化与语义改变变化，并从语言和视觉模型中恢复局部不变表示子空间。

Details

Motivation: 语义不变的变换可能让表示大幅移动，而很小的表示变化也可能显著改变预测，因此欧氏距离未必能刻画真正重要的语义变化。作者研究何种局部度量能反映预测相关的语义差异。

Result: Fisher-IRG在每个表示周围构造语义保持与语义改变邻域，聚合局部 Fisher 信息，并通过对比广义特征值问题提取不变方向。受控位移实验显示相似的欧氏移动可能造成很不相同的预测后果。跨语言和视觉模型的结果表明，该方法比基于协方差的几何具有更强的语义/干扰因素预测选择性，子空间通常也更可复现，并恢复出系统性不同的局部方向。表示干预将语义效应定位在 Fisher 子空间中；留出数据上的分离和检索结果显示该几何可泛化到发现邻域之外。

Insight: 表示距离应按其对预测的影响来衡量，而不只看向量空间中的位移大小。结合语义保持和语义改变邻域构造 Fisher 几何，可找出对语义敏感、对无关扰动相对稳定的方向。

Abstract: Semantic-preserving transformations can induce substantial motion in learned representations, while small changes may strongly affect model predictions, raising a basic question: what local metric best captures semantically consequential variation? We propose Fisher-induced invariant representation geometry (Fisher-IRG), which measures local representation directions through their predictive sensitivity. Around each representation, we construct semantic-preserving and semantic-changing neighborhoods, aggregate their local Fisher information, and recover invariant directions through a contrastive generalized eigenvalue problem. Controlled displacement analyses first show that comparable Euclidean motion can have substantially different predictive consequences, supporting the need for a predictive geometry. Across language and vision models, Fisher-IRG yields stronger semantic-versus-nuisance predictive selectivity and generally more reproducible subspaces than covariance-based geometry, while recovering systematically distinct local directions. Representation interventions further localize semantic effects to the Fisher-derived subspace, and held-out separation and retrieval show that the recovered geometry generalizes beyond the discovery neighborhoods. These results support Fisher-IRG as a principled framework for characterizing local invariant representation geometry.

[237] Act First, Reason Later: Accelerating On-Policy Distillation for Multi-Turn Agents via Reference-Conditioned Inverse Dynamics cs.LG | cs.CLPDF

Zubin Zheng, Jiahao Wu, Shaofeng Zhang, Zhirui Zhang, Yew-Soon Ong

TL;DR: ActFirst-OPD把多轮智能体蒸馏中的环境交互与完整推理文本生成解耦，先迅速执行动作，再异步生成推理内容接受教师监督。

Details

Motivation: 在线策略蒸馏可用教师对学生生成响应提供密集监督，但常规“先思考再行动”要求每次短动作之前先生成长推理，拖慢环境转换和经验收集。直接生成动作虽更快，却可能降低轨迹质量。

Result: ActFirst-OPD让学生根据当前交互上下文和参考下一观察，通过参考条件逆动力学推断并执行动作；若状态转移偏离参考轨迹，则切换为自主预测下一动作。采集交互上下文后，学生异步生成完整的思考—行动响应，接受token级教师监督。Qwen3的0.6B、1.7B和4B学生实验中，相比标准OPD，ALFWorld、WebShop和ScienceWorld墙钟训练分别加速2.3倍、1.8倍和4.9倍；九个基准—模型设置中的八个，平均任务成功率达到或超过所有对比OPD基线。

Insight: 环境交互不必等待完整思维链生成。通过参考条件动作执行保证较快收集轨迹，再异步补齐详细响应进行蒸馏，可以同时提升训练吞吐和任务质量。

Abstract: On-policy distillation (OPD) trains multi-turn language agents with dense teacher supervision on student-generated responses. However, standard think-then-act rollouts require lengthy reasoning before each short action, delaying environment transitions and experience collection. Generating actions directly reduces this delay but can degrade rollout quality. To address this, we propose ActFirst-OPD, an act-first, reason-later training framework that decouples environment interaction from full-response generation. The student infers and executes actions through reference-conditioned inverse dynamics using its current interaction context and a reference next observation, and switches to autonomous next-action prediction when the resulting transition deviates from the reference trajectory. From the collected interaction contexts, the student asynchronously generates full think-then-act responses for token-level teacher supervision. Experiments across 0.6B-, 1.7B-, and 4B-parameter Qwen3 students show that ActFirst-OPD achieves average wall-clock training speedups of $2.3\times$ on ALFWorld, $1.8\times$ on WebShop, and $4.9\times$ on ScienceWorld over Vanilla OPD. It matches or exceeds all compared OPD baselines in mean task success rate across eight of nine benchmark-model settings. These results demonstrate that reasoning need not block acting during multi-turn agent distillation.

[238] What Makes Recurrence Effective in Looped Language Models? cs.LG | cs.CLPDF

Xinlin Zhuang, Siyuan Wang, Imran Razzak, Weiyang Liu

TL;DR: 对循环语言模型的受控实验表明，增加递归深度可能改善推理却损害知识表现，循环位置和状态注入方式都会影响不同推理预算下的稳定性。

Details

Motivation: LoopLM通过共享参数增加计算深度，为不增加参数量而扩展推理计算提供路径。但额外递归何时有益、应放在模型哪些层、如何条件化循环状态仍不明确。

Result: 研究在低于、处于和超过训练范围的推理预算下评测知识与推理任务。递归可提升超训练长度的推理表现，但可能降低知识表现；较难的推理样例并不总是受益更多。输入层、循环层和输出层的分配会改变结果，因此有效深度不足以预测效果；非递归输出层可提升少循环展开时的稳健性，而输入、输出层的最佳位置会随推理预算变化。传统初始状态注入难以适配不同循环深度。作者提出历史状态注入，发现逐通道历史状态注入结合时间步条件化成本低且更有效，在延长展开时较好保留知识表现，并提高跨预算稳健性。

Insight: 循环次数增加不等于无条件增加有效能力，知识和推理可能出现不同方向变化。跨预算部署需要同时设计循环层布局和状态注入方式，尤其需避免未充分展开时的脆弱性。

Abstract: Looped language models (LoopLMs) increase computational depth through parameter sharing, offering a path to scale inference computation without adding parameters. However, it remains unclear when additional recurrence is beneficial and how architectural choices affect its effectiveness. Through controlled experiments, we systematically examine (1) when recurrence helps, (2) where it should be applied, and (3) how its conditioning affects performance. Our evaluation covers inference budgets below, within, and beyond the training horizon under knowledge and reasoning tasks. (1) We find that recurrence can improve reasoning beyond the training horizon while degrading knowledge performance, but harder reasoning instances do not consistently benefit more. (2) Performance also depends on how distinct layers and recurrent iterations are allocated, showing that effective depth alone is insufficient to predict behavior. Non-recurrent output layers improve robustness to under-unrolling, while the preferred placement of input and output layers varies with inference budget. (3) Finally, we find that conventional initial-state injection offers limited robustness to varying recurrence depth. We therefore propose history-state injection as an alternative, and show that channel-wise history-state injection combined with timestep conditioning offers a low-cost and more effective design, better preserving knowledge under extended unrolling while improving robustness across inference budgets. Overall, our results clarify when recurrent computation helps, where it fails, and offer practical guidelines for designing LoopLMs across variable inference budgets.

[239] MARCO: Multi-Round Agentic Reinforcement for Conditional Molecular Optimization cs.LG | cs.CLPDF

Shicheng Fang, Yuxin Wang, Zhuo Yang, Xiaohu Xu, Jiahao Lu

TL;DR: MARCO以评估器为基础，通过有界的提议—反馈—修订轨迹训练分子编辑智能体，并分别测试单次响应和多轮反馈下的优化效果。

Details

Motivation: 分子优化本质上是迭代过程：提出候选、按多个目标评估、再修改，同时维持与源分子的相似性。多数指令模型一次性输出编辑后的分子，迫使有效性、性质提升和相似度控制同时压在单个响应上。

Result: MARCO使用评估器提供的反馈构造有界多轮轨迹，将逐轮塑形奖励汇总为不折扣轨迹回报，并用组相对策略优化训练。Same-1评估一次响应预算，Same-5评估同一策略最多五次响应并使用验证反馈。MuMOInstruct三目标基准覆盖三个Qwen骨干及已见/未见指令划分；以SFT初始化的MARCO在所有报告的主要设置中，性质成功率与相似度的乘积最高。Same-5在所测预算下进一步提高观测得分；四目标和公开检查点实验用于检验约束组合及初始化方式的迁移。

Insight: 分子编辑需要显式保留候选提出、外部评估和修订环节，不能只把多目标优化压缩成一次生成。单轮与多轮分开评估，也能区分模型本身的单次响应能力和利用反馈的增益。

Abstract: Molecular optimization is inherently iterative: a candidate is proposed, evaluated against several objectives, and revised while preserving a relationship to the source molecule. Most instruction-following models instead emit one edited molecule, forcing validity, property improvement, and similarity control into a single response. We introduce MARCO, an evaluator-grounded reinforcement-learning framework that trains molecular editors on bounded proposal–feedback–revision trajectories. MARCO aggregates shaped turn rewards into an undiscounted trajectory return for group-relative policy optimization. We evaluate two consequences of this training: Same-1 tests the trained policy under a one-response budget, while Same-5 tests whether the same policy can use verifier feedback when up to five responses are available. Across the three-objective MuMOInstruct benchmark, three Qwen backbones, and seen/unseen instruction splits, SFT-initialized MARCO obtains the highest product of property success rate and similarity in every reported primary setting. Same-5 further improves the observed score under the tested budget, while four-objective and public-checkpoint experiments test transfer across constraint sets and initialization regimes.

[240] Backpropagated Output Momentum: Relocating Optimizer History from Parameters to Task Space cs.LG | cs.CLPDF

Yuchen Li, Zongqi Fan, Nguyen H. Tran, Ken-Tye Yong

TL;DR: BOM将优化器的一阶动量从参数规模的梯度历史改为模型输出端的紧凑预测误差历史，并在每步通过当前网络重新投影。

Details

Motivation: 常规优化器把过去梯度的移动平均按参数存储，状态占用大，而且每个历史信号固定在计算它时的参数坐标中。作者尝试将历史压缩到任务输出空间，同时保留当前监督梯度。

Result: Backpropagated Output Momentum（BOM）在模型输出端存储紧凑的预测误差移动平均，并在每步利用当前网络反向重投影；它可作为多种自适应优化器的一阶动量插件，包括已有状态压缩的优化器。批级分析刻画这种迁移保留和舍弃的信息。三种组合中，BOM将参数形状优化器状态减少49.7%至99.8%；对三种语言骨干取平均，配对训练步时间减少4.0%。语言和视觉微调的平均验证表现提升，主要五任务对比提高1.42分；预训练实验及机制控制进一步考察不同输出空间和模型规模。

Insight: 优化器历史不一定必须逐参数保存，输出空间的误差信号经当前网络重投影即可提供可用的一阶更新信息。压缩优化器状态的同时保留当步监督梯度，是其适合叠加到既有优化器上的设计要点。

Abstract: Optimizer momentum is usually stored as a parameter-sized moving average of past gradients, which makes history costly and fixes each past signal in the coordinates in which it was computed. We introduce Backpropagated Output Momentum (BOM), which instead stores a compact moving average of prediction errors at the model output and reprojects that history through the current network at every step. A batch-level analysis characterizes the information retained and omitted by this relocation, while the implementation preserves the current supervised gradient and can replace the first-moment component of several adaptive optimizers. As a plug-in for momentum-based optimizers, including ones that already compress their state, BOM reduces parameter-shaped optimizer state by 49.7-99.8% in three compositions and, averaged over three language backbones, paired step time by 4.0%. It also improves mean validation performance across language and vision fine-tuning, by 1.42 points in the primary five-task comparison. Language and vision pretraining studies, together with matched mechanism controls, further test the construction across output spaces and model scales.

[241] Group-Marginalized Self-Rewarding RL Drives Zero-Label Self-Evolving cs.LG | cs.AI | cs.CLPDF

Yiming Wang, Yikang Liu, Qingyuan Tian, Xingyu Chen, Zhuosheng Zhang

TL;DR: GMAE对同一回答在不同随机rollout组上下文中得到的奖励进行边际化，估计回答级预期优势，降低单次分组带来的训练信号波动。

Details

Motivation: 无人工标签的自奖励强化学习通常从一组rollout构造奖励参考，但某个回答的奖励表示会随同组其他随机采样回答而改变。仅依赖一次分组上下文可能漏掉有用奖励信号，并使策略更新不可靠。

Result: 作者提出组边际化优势估计（GMAE），将不同可能组上下文中的奖励实现聚合成回答级分布，再估计其预期优势。在八个基准和四个基础模型上的实验显示方法表现较强且能跨领域泛化；同时学习较稳定、额外成本低，并可适配不同训练数据集和强化学习骨干。摘要未给出详细分数。

Insight: 自奖励并非只受回答内容影响，还依赖随机组成的比较组。对可能组上下文做边际化，可以减少某一次分组偶然性对优势估计的支配。

Abstract: Self-rewarding reinforcement learning (RL) enables large language models (LLMs) to self-evolve without human labels. Existing ensemble-based methods construct reward references from rollout groups and assign rewards accordingly. However, a response’s reward representation also depends on its randomly sampled group context, i.e., the other responses in its group. Using only one group-context realization may miss desired reward signals and provide unreliable guidance for policy optimization. To address this issue, we propose Group-Marginalized Advantage Estimation (GMAE), which aggregates reward realizations across possible contexts into a response-level distribution and estimates expected advantages. Experiments across eight benchmarks and four base models demonstrate strong performance and cross-domain generalization. GMAE also exhibits stable learning, low extra cost, and good applicability across training datasets and RL backbones.

[242] QuantMLA: Function-Aligned Dual-Path Quantization for Low-Bit MLA KV Caching cs.LG | cs.CLPDF

Zunhai Su, Yuxuan Sun, Jianchao Tan, Tao Zhang, Ruihan Hu

TL;DR: QuantMLA根据内容和RoPE两条缓存路径对注意力的不同影响设计双路径量化，并通过原生低比特内核降低长上下文KV缓存占用和服务成本。

Details

Motivation: 多头潜在注意力（MLA）通过紧凑内容缓存和解耦RoPE路径节省空间，但KV缓存仍随上下文长度和批量线性增长。两条路径的量化误差对注意力输出影响不同，尤其RoPE路径误差可能明显放大。

Result: QuantMLA系统分析双路径量化误差，并为两条路径推导函数对齐变换空间；变换可离线融合进模型参数，不增加在线变换开销。内容路径用注意力输出重建处理匹配与聚合耦合误差；位置路径用QK重建保留RoPE对logit的作用，并给出输出误差理论界。在四类MLA模型上，作者报告首次实现内容和RoPE缓存同时INT4且准确率仅轻微下降；内容缓存进一步压至INT2、RoPE键缓存保持INT4时，在困难推理和代码基准上仍有竞争力。专用注意力内核将解包和反量化融合进计算。128K上下文缓存物理布局压缩3.59倍；缓存压力服务场景下，整作业输出吞吐比BF16高5.168倍。

Insight: MLA的内容和位置缓存承担不同功能，量化目标因此应分别与各自对注意力计算的作用对齐。将低比特解包和反量化融合到原生内核，才能把缓存压缩转化为实际服务吞吐收益。

Abstract: Multi-Head Latent Attention (MLA) enables expressive multi-head attention with compact caches for its content and decoupled RoPE paths, yet cache memory still scales linearly with context length and batch size. In this work, we establish a systematic model of MLA’s dual-path quantization errors, characterizing their distinct effects on attention-output distortion and explaining the pronounced amplification of RoPE-path errors. Guided by this analysis, we introduce QuantMLA, a function-aligned framework for low-bit dual-path quantization. We derive path-specific transformation spaces that preserve full-precision computation while remaining fully fusible into model parameters offline, eliminating online transformation overhead. Within these spaces, QuantMLA learns path-specific transformations with function-aligned objectives: attention-output reconstruction captures the content path’s coupled matching and aggregation errors, while positional QK reconstruction preserves the RoPE-induced component of the attention logits and admits a theoretical bound on output distortion. Across four MLA model families, QuantMLA enables, to our knowledge, the first reported joint INT4 caching of the content and RoPE caches with minimal accuracy degradation. Further compressing the content cache to INT2 while retaining the RoPE key cache at INT4 maintains competitive performance on challenging reasoning and code benchmarks. We develop a native low-bit MLA attention kernel that integrates unpacking and dequantization directly into attention computation. The physical cache layout provides 3.59x compression at 128K context, while a cache-pressure serving workload achieves 5.168x higher whole-job output throughput than BF16. The code will be released upon acceptance.

[243] CorrGRPO: Correlation-Normalized GRPO for Multi-Reward Learning cs.LG | cs.CLPDF

Wenbin Hu, Huihao Jing, Haochen Shi, Yuxuan Liu, Haoran Li

TL;DR: CorrGRPO用奖励分量之间的相关系数替代原始协方差进行组优势归一化，减少大尺度奖励压制小尺度奖励信号的问题。

Details

Motivation: GRPO对同一提示下的多个rollout奖励求中心化并归一化。多奖励场景中，总奖励的组内方差等于各奖励两两协方差之和；当奖励尺度差异大且相互相关时，大尺度分量可能主导归一化，使较小尺度信号被压低。

Result: CorrGRPO将奖励两两协方差归一化为Pearson相关系数，保持中心化总奖励不变，同时平衡不同尺度奖励对相关性归一化的影响，使优势幅度仍能响应奖励相关关系，但不被大尺度分量垄断。作者在代码生成、工具调用和智能体安全三个具有多奖励或目标权衡的领域，对0.5B至8B模型比较CorrGRPO、GRPO及其他变体，报告三个领域均有提升。摘要未提供具体分数。

Insight: 多目标学习中的归一化会影响各奖励信号的相对话语权，不只是调整更新幅度。使用无量纲的相关系数处理奖励依赖，有望保留目标间协同信息并减弱尺度失衡。

Abstract: Group Relative Policy Optimization (GRPO) is widely used to train reasoning language models, where it computes advantages by centering and normalizing rewards across rollouts of the same prompt. For multiple rewards, GRPO sums the reward components and normalizes the total reward by its within-group standard deviation. The corresponding variance equals the sum of all pairwise reward covariances. For a fixed centered reward, larger aggregate covariance produces smaller advantages, and vice versa, allowing update magnitudes to adapt to reward dependence. However, correlated rewards with large scales can dominate this normalization and suppress signals from smaller-scale rewards. We propose Correlation-Normalized GRPO (CorrGRPO), which normalizes pairwise covariances into Pearson correlation coefficients. CorrGRPO keeps the centered total reward unchanged while balancing the influence of differently scaled rewards on the correlation-based normalization. This allows advantage magnitudes to adapt to reward correlations without the normalization being dominated by large-scale reward components. We compare CorrGRPO with GRPO and other variants on code generation, tool calling, and agent security, using models ranging from 0.5B to 8B parameters. These tasks all involve multiple rewards that can improve together or present tradeoffs. Results show improvements across three domains, including code generation, tool calling, and agent security. Our code is available at https://github.com/HKUST-KnowComp/CorrGRPO.

[244] VACE: Validation-Gated Alternating Co-Evolution of Agent Models and Harnesses cs.LG | cs.AI | cs.CLPDF

Jiexing Qi, Yu He, Jun Liu, Qichen Huang, Shaohua Hu

TL;DR: VACE交替优化智能体模型权重与执行harness，并用验证门控决定是否采用新harness，避免未经验证的流程改动污染后续强化学习轨迹。

Details

Motivation: 智能体可通过更新模型权重或改进任务执行harness来提升，但两者相互耦合：权重变化会改变模型使用harness的方式，harness变化又会改变训练轨迹。简单交替优化可能引入退化的harness。

Result: 每个智能体强化学习阶段后，VACE复用收集的轨迹提出harness候选；固定更新后的模型权重，比较现有与候选harness的验证表现，只有候选改善验证指标时才用于后续训练。Qwen3.5-9B在OfficeQA测试准确率为45.26%，AutomationBench平均部分得分75.19%；分别比仅更新权重的RL高6.43和9.09个百分点，比不带门控的交替更新高4.59和6.95个百分点。44个harness候选中，17个在更新后的检查点上降低验证表现，因此被拒绝。

Insight: 模型和执行流程共同决定智能体能力，优化其中一侧会改变另一侧的有效行为。验证门控可以在把流程修改写入未来训练之前拦截退化候选。

Abstract: Language model agents can be improved by updating their model weights or refining the harness that guides task execution. These components are coupled: weight updates change how the model uses the harness, while harness updates change the trajectories used for training. We propose VACE, Validation-Gated Alternating CoEvolution, which alternates agentic reinforcement learning with trajectory-driven harness refinement. After each RL stage, VACE reuses the collected trajectories to propose a harness revision and evaluates the incumbent and candidate with the updated model held fixed. The candidate guides subsequent training only if it improves validation performance. With Qwen3.5-9B, VACE achieves 45.26% test accuracy on OfficeQA and a mean partial-credit score of 75.19% on AutomationBench, exceeding weight-only RL by 6.43 and 9.09 percentage points and ungated alternation by 4.59 and 6.95 points, respectively. Across 44 harness proposals, 17 reduce validation performance at the updated checkpoint and are rejected before subsequent RL training, highlighting the importance of validation gating.

[245] Unlocking the Critic: Reward-Free Policy Optimization for LLM Post-Training cs.LG | cs.AI | cs.CLPDF

Hongyang Li, Xiao Li, Caesar Wu, Said Mammar, Grégoire Danoy

TL;DR: RFPO复用一个校准且冻结的价值评估器，为未完成推理前缀提供奖励、价值基线和成功预测，从而无需训练环节中的外部标签进行策略优化。

Details

Motivation: 长链推理中的critic-based强化学习不稳定且耗费内存，训练结束后critic还常被丢弃。作者重新考察预训练良好的critic是否可预测轨迹后续成功，并为长时程推理提供无需等待完整结果的稠密信号。

Result: 作者发现，长思维链训练中的critic不稳定很大程度是优化问题；缩小策略更新并降低方差可恢复稳定收敛。训练良好的critic可从后续轨迹状态及未完成前缀估计最终成功的后验概率。RFPO将一个校准冻结critic同时用作rollout级奖励、广义优势估计的价值基线和未完成前缀的成功预测器；二值化去偏得分可防止策略利用critic的长度偏差。二值RFPO在训练循环无标签时匹配有监督PPO，同时减少计算及内存开销。未完成轨迹可提前赋予奖励，因此无需等待每个rollout结束。

Insight: 长时程任务中，冻结critic可以把未来成功预测转成前缀级监督，并让训练在完整轨迹结束前继续。奖励二值化用于削弱critic偏向长输出的漏洞，说明无标签优化仍需针对评估器偏差设计约束。

Abstract: Recent approaches to reinforcement learning (RL) post-training for large language models increasingly remove the critic to reduce training instability and memory overhead. Even where a critic is trained, it is discarded once training ends, although it has learned to predict outcomes. We revisit this trend and show that a pretrained critic’s ability to predict future outcomes can make it a valuable asset for efficient long-horizon reasoning. First, we find that instability in critic-based RL for long chain-of-thought reasoning is largely an optimization artifact: keeping policy updates small and low in variance restores stable convergence. Second, a well-pretrained critic estimates the posterior probability of eventual success from later trajectory states and unfinished prefixes. Its predictions provide outcome-derived, dense, per-prefix learning signals that, during policy optimization, require neither completed rollouts, step-level annotations, nor external reward labels. Building on this insight, we introduce Reward-Free Policy Optimization (RFPO), which repurposes a single calibrated, frozen critic as a rollout-level reward, a value baseline for generalized advantage estimation, and a success forecaster for unfinished prefixes. We further show that binarizing the debiased score stops the policy from exploiting the critic’s length bias. Binarized, RFPO matches supervised PPO without a single label in the training loop, while cutting compute and memory overhead. This makes RFPO well suited to long-horizon reasoning tasks, where outcomes arrive late and generation dominates cost: because rollouts can be rewarded before they finish, training no longer has to pay for waiting on every trajectory to complete. Our findings challenge the prevailing critic-free paradigm and establish critic-based, reward-free optimization as a scalable and computationally efficient path for LLM post-training.

[246] Multimodal Detection of Higher-Order Behavioral Constructs: Self-Compassion in Structured Reflective Interaction cs.LG | cs.CL | cs.CV | cs.HCPDF

Siddhant Jain, Dimitra Tsovaltzi

TL;DR: 研究收集了51段结构化反思对话，并将视频、音频和文本对齐，初步比较多模态信号对自我关怀这一高阶心理行为构念识别的作用。

Details

Motivation: 情绪调节、面对挫折的反思和关注他人等学习成长相关品质无法直接观察，需要从一段时间内的语言、动作和声音推断；这类心理构念难以用机器学习常见的简单标签描述，且缺少对应数据集。

Result: 作者研究结构化反思访谈中的自我关怀，即面对个人挫折时以耐心而非严厉自责回应的倾向。团队收集并标注51段反思对话，采用独立且时间重叠的标注方案，将六类心理理论成分归并为三类监督状态：自我友善与正念，以及自我批评或不知所措。可复现的窗口化流程把视频、音频和文本对齐到共享时间线上。分别训练单模态模型，并与简单概率级融合比较；融合相对最佳单模态取得幅度不大但一致的提升。论文还分析不同模态表达该构念的长短处。

Insight: 自我关怀这类构念需结合时间对齐的多模态信号，并承认标签本身来自心理理论而非直接可观测真值。初步结果支持融合的作用，但样本规模和标注设计仍限制可推广结论。

Abstract: Many of the qualities that matter most in how people learn and grow, how someone regulates their emotions, reflects on a setback, or stays aware of others during a difficult conversation, are not directly observable. They have to be inferred from how someone speaks, moves, and sounds over time, and they resist the kind of clean labeling that most machine learning pipelines are built around. We study this challenge through a case that is well grounded in psychological theory but rarely modeled computationally: self-compassion, the tendency to respond to one’s own setbacks with patience rather than harsh self-criticism. We examine how it appears during structured reflective interviews in a technology-mediated training setting, where people naturally talk through socio-emotionally demanding situations. Since no existing dataset captures this kind of construct in this kind of setting, we collected and annotated 51 reflective dialog sessions using an independent, temporally overlapping annotation scheme grounded in established theory. We consolidate the underlying six-component psychological model into a three-class supervision space, balancing self-kindness and mindfulness against self-critical or overwhelmed states, and build a reproducible window-based pipeline that aligns video, audio, and text on a shared timeline. Unimodal models trained on each modality separately are compared against a simple probability-level fusion strategy, which yields modest but consistent gains over the best single modality. We close by discussing where each modality succeeds or struggles, what this suggests about how this kind of construct is actually expressed in reflective speech, and what would be needed to model it, and constructs like it, more effectively.

[247] Trajectory Soup: Pushing the Compute-Scaling Frontier of LLM Mid-training via Diverse Trajectories cs.LG | cs.AI | cs.CLPDF

Zhehao Huang, Changxin Tian, Qingyuan Yang, Kunlong Chen, Ziqi Liu

TL;DR: Trajectory Soup将中训练预算分配给多个独立分支，再对验证集选出的最佳检查点进行分支内和分支间平均，以突破单条训练轨迹的收益饱和。

Details

Motivation: 中训练能赋予预训练语言模型专门能力和推理能力，但同一轨迹上继续串行增加计算往往只带来少量收益，甚至损害部分能力，限制了可吸收的计算规模。

Result: 作者发现，从共享检查点按受控配方分叉的训练轨迹会到达可兼容但不同的参数区域，提供延长单条轨迹无法得到的多样性。Trajectory Soup把预算分配到多个独立分支，从验证集选取较强检查点，再结合轨迹内与轨迹间平均整合为单一模型。局部偏差—方差分析显示，分支间平均能消除轨迹内平均无法去除的残余误差；检查点筛选也会产生偏差，从而限制值得合并的检查点数量。不同规模、学习率计划、token预算和分支数实验中，在预算匹配下整体下游表现优于最强单轨迹平均，预算增加时仍持续改善；相同后训练流程后优势保留。

Insight: 单轨迹上的串行计算会饱和时，将预算用于产生兼容多样性的独立训练路径可能更有效。合并策略需要同时控制方差降低与验证筛选带来的偏差，而非无限增加检查点数量。

Abstract: Mid-training equips pretrained large language models with specialized and reasoning capabilities, but the returns of this stage are bounded since additional serial compute yields little further downstream improvement and can even degrade some capabilities, which places a practical ceiling on how much compute mid-training absorbs. We revisit how this compute should be allocated to a single run or multiple similar optimizations. We find that branches forked from a shared checkpoint under various controlled recipe reaches measurably different regions of parameter space, and establish a form of compatible diversity that extending one run cannot supply. Therefore, we introduce Trajectory Soup, which distributes a mid-training budget over several independent branches, and consolidates strongest checkpoints selected on validation through intra- and inter-trajectory averaging into a single model. A local bias and variance analysis separates the two averaging levels, showing that inter-trajectory averaging removes residual error beyond the reach of averaging within a trajectory, while checkpoint selection carries a bias that bounds how many checkpoints are worth merging. Across model scales, learning-rate schedules, token budgets, and trajectory counts, Trajectory Soup improves aggregate downstream performance over the strongest single-trajectory average under matched budgets and keeps improving as budgets expand, with the advantage preserved after an identical post-training pipeline. These results position trajectory allocation and merging as a practical way to extend the compute-scaling frontier of mid-training beyond serial saturation.

[248] Hidden Reasoning Must Leak, but Need Not Be Readable: Fundamental Opportunities and Limits for Chain-of-Thought Monitoring cs.LG | cs.AI | cs.CLPDF

Mohammadali Mohammadkhani, Madhava Krishna, Yash Sarrof, Michael Hahn

TL;DR: 论文研究思维链监控的理论边界：复杂隐式计算成功时通常必须泄露与输入近线性相关的信息，但这些信息未必能被现实监控器读懂。

Details

Motivation: 推理模型是否能在不暴露计算过程的思维链中暗中完成任务，是思维链监控能否用于安全监督的核心问题。作者考察这一点如何受任务难度、模型规模及监控者计算能力影响。

Result: 理论与实证结果显示，简单计算可以隐蔽完成；但超过一个随模型规模变化的难度阈值后，成功解题必然在思维链中泄露与隐藏任务输入近线性相关的信息，形成信息论意义上的痕迹。然而，泄露不等同于可读：在合理密码学假设下，即使单层Transformer也能在线加密其推理，使任何多项式时间监控器都无法提取隐藏计算信息。论文因此同时展示思维链监控能获得的信息和其可解释性限制；摘要未提供数值阈值。

Insight: 监控的根本边界包含两个层次：成功计算是否留下信息痕迹，以及受限计算能力的监控器能否解码该痕迹。信息论可见性并不保证实际可审计性。

Abstract: Can reasoning models trick chain of thought (CoT) monitors and perform hidden computation without revealing it in their thinking traces? We show that the answer depends on the underlying task difficulty and the model size. Simple computations can be performed covertly; however, beyond a threshold depending on model size, successfully solving the task necessarily leaks a near-linear amount of information about the covert task input into the CoT. Therefore, sufficiently complex hidden computation always leaves an information-theoretic footprint. However, concerningly, this leakage need not be readable: Under plausible cryptographic assumptions, even a one-layer Transformer can encrypt its reasoning online so that no polynomial-time monitor can extract information about the hidden computation. Overall, our theoretical and empirical results provide a holistic view of both the opportunities and the limitations of CoT monitoring.

[249] Port-Hamiltonian Latent Deliberation: Mitigating the Deliberation Drift Cliff in Test-Time Compute Scaling cs.LG | cs.AI | cs.CLPDF

Zeyu Jia

TL;DR: PH-LD用端口哈密顿潜在动力学协调表达能力、长时稳定性与计算效率，缓解循环潜在推理在计算步数增加后出现的性能崩塌。

Details

Motivation: 测试时计算扩展允许模型在连续潜表示中反复推理，但无约束递归模型虽在短迭代时有收益，在更深推理时可能发生Deliberation Drift Cliff，逻辑基准性能下降22%至62%。纯保守流稳定但限制推理表达，旋转流表达力强却漂移严重。

Result: 经过22轮理论和实证研究，作者提出Port-Hamiltonian Latent Deliberation及直接梯度纯张量Helmholtz-Hodge分解DG-HHD，以切空间投影张量参数化吸引流，并正交分解非零环流。纯保守势梯度流将漂移悬崖控制在3.40%，但峰值准确率只有32.73%；无约束旋转流峰值82.33%，漂移达36.87%。DG-HHD的Hodge机误差为1.65e-17、收缩误差5.55e-17，无需运行时自动微分，向量场和RK45 rollout分别加速1.84倍、2.09倍。15臂对称Pareto基准上峰值准确率58.67%，比保守HHD高25.94个百分点，K=32时仍有35.27%；迁移到小语言模型多跳因果推理后，准确率随计算从49.33%升至51.56%，分布外漂移悬崖为-0.66%。

Insight: 潜空间迭代推理需要兼顾吸引稳定性与非保守流提供的表达能力，单独追求其中一项会导致准确率瓶颈或长程漂移。结构化分解可显式分离两种动力学，并同时减少运行时求导开销。

Abstract: Test-time compute scaling has emerged as a cornerstone of advanced machine reasoning, yet performing iterative deliberation directly within continuous latent representation spaces reveals a catastrophic pathology: the Deliberation Drift Cliff. While unconstrained recurrent latent models achieve initial reasoning gains at short horizons (K <= 4), their reasoning collapses when extrapolated to deeper thinking steps (K >= 16), dropping by 22% to 62% across standard logical benchmarks. We resolve the trilemma among expressivity, Lyapunov stability, and computational efficiency in test-time latent reasoning through a 22-round empirical and theoretical investigation. We demonstrate that strictly conservative scalar potential gradient flows suppress long-range drift (cliff 3.40%) but bottleneck peak reasoning accuracy at 32.73%, whereas unconstrained rotational flows achieve high symbolic expressivity (82.33%) but suffer a severe 36.87% drift cliff. To resolve this geometric duality, we establish Port-Hamiltonian Latent Deliberation (PH-LD) and propose the Direct-Gradient Pure-Tensor Helmholtz-Hodge Decomposition (DG-HHD). DG-HHD parameterizes the attracting flow as a tangent projection tensor network while orthogonally decoupling non-zero circulation (Hodge machine error 1.65e-17, contraction error 5.55e-17), eliminating runtime autograd dependencies to achieve 1.84x vector field and 2.09x RK45 rollout speedups. In a 15-arm symmetrical Pareto benchmark, DG-HHD achieves 58.67% peak accuracy (+25.94% absolute gain over conservative HHD) and retains 35.27% at K=32. Transferred to small language model (SLM) multi-hop causal reasoning, DG-HHD delivers monotonic compute scaling (49.33% to 51.56%) and suppresses out-of-distribution drift (cliff -0.66%). All 30 Level 0 deterministic invariants are certified.

[250] Hierarchical Compression of Vision-Language Model Benchmarks cs.LG | cs.AI | cs.CL | cs.CVPDF

Hyunjong Ok, Seunggu Kang, Jaeho Lee

TL;DR: PRIMEBench以视觉感知为核心分层压缩VLM评测集，在大幅减少题目数量的同时尽量保留不同模型的排名。

Details

Motivation: 视觉语言模型能力评测覆盖面不断扩大、模型发布频繁，完整评测成本过高。语言模型基准压缩已有研究，但VLM评测中如何减少成本并保留模型排名仍未充分探索。

Result: PRIMEBench分四步压缩：清理无需看图即可回答及所有模型都答对的题目；按能力类别选代表性基准；用视觉感知方差（VAW）剪题；再剪减类别数量。VAW结合模型间表现方差与仅从多模态嵌入计算的视觉依赖分数，同时鼓励题目多样性。在不参与题目选择的模型上，发布的5%保留率下其平均排名保真度最高；整体套件删去超过97%的题目仍保留模型排序。分层设计允许使用者按计算预算在任一阶段停止。

Insight: VLM基准压缩应保留对视觉输入真正有依赖、且能区分模型的题目，并同时覆盖不同能力类别。固定保留率上的排名保真度和对模型换代的稳健性，决定压缩套件是否实用。

Abstract: Thorough evaluation of vision-language models (VLMs) has become prohibitively expensive, as benchmarks span an ever-broader spectrum of capabilities and new models arrive at a relentless pace. Benchmark compression methods that preserve model rankings at a fraction of the cost are well studied for language models, but for VLMs the question remains under-explored. We present PRIMEBench (Pruning Redundant Items for Multimodal Evaluation), a vision-aware hierarchical benchmark compression framework that substantially reduces evaluation cost while preserving model rankings. This hierarchical framework operates in four stages: data cleaning to remove items answerable without the image and all-correct items, category representative selection to pick one benchmark per capability category, item pruning with Vision-Aware Variance (VAW), and category-count pruning. VAW combines inter-model variance with a vision-dependence score computed from multimodal embeddings alone, while encouraging coverage of diverse items within each benchmark. On models held out from item selection, it has the highest mean fidelity at the released 5% retention. The hierarchical design lets practitioners stop at any stage to match their compute budget; the released suite removes over 97% of items while preserving model rankings. Beyond compression, our analyses show how VLM evaluation behaves as model panels grow and evolve, providing guidance for designing future benchmarks that are more efficient, robust to model turnover, and explicit about the limits of evaluation-side pruning.

[251] RLTL;DR: Self-improvement by Internalizing Self-generated Feedback cs.LG | cs.AI | cs.CL | stat.MLPDF

Michael Kirchhof, Eleonora Gualdoni, Andrew Szot, Khashayar Gatmiry, Aryo Lotfi

TL;DR: RLTL;DR让模型把失败后的验证器反馈压缩成自生成的一句关键提示，并逐步将任务到提示的映射内化，从而在无成功样例的难题上实现自我改进。

Details

Motivation: 可验证奖励强化学习通常反复尝试并强化成功轨迹；若任务极难、成功概率接近零，又无教师或示例解答，训练便几乎没有有效信号。

Result: 每次失败后，RLTL;DR把验证器输出交给策略，让其写下一条TL;DR式洞见；后续rollout以此前所有洞见为条件，直到找到解。训练还对这些上下文洞见反向传播，使模型内化任务到提示的映射。在过滤到Pass@128=0的工具调用和代码任务上，Qwen3.5-9B Thinking标准GRPO的Pass@1仅0%至1%；RLTL;DR训练时有洞见达到14%至31%，评测时无洞见上下文仍达12%至13%。进一步简化的SFTL;DR只用任务—洞见对做监督微调，不训练rollout；仅4,000对就几乎恢复RLTL;DR和完整rollout监督微调的全部表现。

Insight: 失败反馈本身可被模型压缩成可复用的任务策略提示，而不需要外部教师给出完整解答。将任务—洞见对应关系内化，能让模型在推理时不依赖额外提示继续保留改进效果。

Abstract: The common paradigm of reinforcement learning with verifiable rewards (RLVR) is to let agents make multiple attempts at a task, and optimize towards the successful ones. This becomes problematic in the realms of self-improvement, where tasks are so difficult that the agent has a low or even no chance of success, and where there are no teacher models or example solutions to distill from. In this paper, we introduce RLTL;DR. After each failed attempt, we show the policy the verifier outputs and let it write its own feedback, in the form of a single TL;DR insight. The next rollout is conditioned on all previous insights, and we sequentially sample rollouts until a solution is found. Moreover, we enable backpropagation on the in-context insights to internalize a direct task to insight mapping. On challenging tool-calling and coding datasets (filtered to Pass@128=0), standard GRPO training of a Qwen 3.5 9B Thinking policy stays flat at a Pass@1 of 0% to 1%. RLTL;DR breaks through this learning barrier, achieving a Pass@1 of 14-31% with insights in context during training and, crucially, 12-13% when no insight is in context at eval time. We identify that the key is the task to insight internalization. To study this further, we reduce our approach to SFTL;DR, training only on (task, insight) tuples, without showing or backpropagating on any rollouts. Training on only 4k of these tuples recovers almost the full performance of RLTL;DR and classical SFT on full rollouts. This demonstrates a promising compacted training paradigm of the form “on this sort of task, keep this sort of thing in mind”, which we hope to inspire future research on.

[252] Storage Is Not Strategy: State-Conditioned Support Control for LLM Unlearning cs.LG | cs.CLPDF

Tianhao Qian, Ziming Hong, Chongyang Gao, Kezhen Chen, Lixu Wang

TL;DR: 论文提出按实际遗忘干预效果而非存储定位信号挑选可编辑参数，并在优化过程中按需动态重排序，以降低大语言模型定向遗忘的副作用。

Details

Motivation: 许多局部遗忘方法先依据定位信号选出少量参数，再在整个优化中固定它们；但与目标关联最强的参数未必是最佳干预对象，优化推进后候选参数的价值也可能变化。

Result: 受控实验中，存储定位分数AUROC达0.981，但存储身份只在36个目标中的17个与更优干预一致，而LoRA在35/36个目标获胜。作者定义干预分数，按实际遗忘更新的预期效果并考虑附带损害来排序编辑组，形成静态干预价值基线Static-IV；再提出DIR-R，仅当校准探针支持比较时才动态重排候选。在Natural-TOFU上，20组方法—目标比较中19组有正的描述性优势，但若干接近零。在LACUNA定位精度基准上，DIR-R在六组NPO和SimNPO比较中终端效用均更高，增益分别为+0.431至+0.848、+0.503至+0.571。GradDiff下，相对Static-IV的主要四字段评估为6胜、6平、0负，配对增益均值/中位数为+0.165/+0.0025；不同字段下结果差异明显。

Insight: 参数与目标的相关性不能替代对候选干预真实效果和副作用的评估。遗忘流程应区分定位、初始干预选择和随检查点变化的支持集更新，并对不同目标函数的依赖保持谨慎。

Abstract: Many localized large language model (LLM) unlearning methods select a small parameter subset from a localization signal and keep it fixed during optimization. The parameters most associated with a target, however, need not be the best ones to update, and candidate interventions can change value as optimization proceeds. In a controlled experiment, a storage-localization score reaches an area under the receiver operating characteristic curve (AUROC) of 0.981, yet storage identity agrees with the better intervention on only 17/36 targets, while low-rank adaptation (LoRA) wins 35/36. We introduce Intervention Score, which ranks editable groups by the predicted effect of the actual unlearning update while accounting for collateral damage, and use it to form the static intervention-value baseline (Static-IV). We then introduce selective dynamic intervention re-ranking (DIR-R), which revisits that subset only when a calibrated probe justifies the comparison. On the Natural-TOFU dataset, our method has positive descriptive margins in 19/20 comparisons between methods and objectives, although several are near zero. On the LACUNA localization-precision benchmark, our mean terminal utility is higher in all six negative preference optimization (NPO) and SimNPO comparisons: NPO margins range from +0.431 to +0.848, and SimNPO margins range from +0.503 to +0.571. The gradient-difference (GradDiff) objective reveals substantial field dependence. Relative to Static-IV, the primary four-field GradDiff evaluation has six wins, six ties, and no losses, with mean and median paired gains of +0.165 and +0.0025. The evidence supports separating localization, initial intervention selection, and checkpoint-dependent support revision.

[253] Learning Beyond What You Sample: Off-Policy-Aware Cross-Model Trajectory Exchange for RLVR cs.LG | cs.AI | cs.CLPDF

Doohyuk Jang, Yoonsik Park, Gyouk Chu, Sihwan Park, Eunho Yang

TL;DR: GRAFT在RLVR全失败组缺少策略梯度信号时，从异构同伴模型交换成功和失败轨迹，并通过重要性校正控制跨模型分布偏差。

Details

Motivation: GRPO等RLVR依赖模型自身rollout中出现成功轨迹，但有限采样预算下可能一个组全失败，无法产生奖励驱动的更新信号。其他模型可能已在互补提示上找到成功轨迹，无需指定一个更强教师也可共享经验。

Result: GRAFT用有门控的同伴轨迹替换全失败组，同时传递同伴计算的优势；序列级兼容性加权和token级重要性比裁剪用于控制跨模型策略差异。在三组异构模型配对和五个数学推理基准上，相同单模型rollout预算下两种模型都持续优于GRPO，平均提高2.1分，模型级平均最高提升4.5分。保存的同伴轨迹保留了大部分收益：无需同时协同训练，仍比GRPO平均高1.8分。

Insight: 模型间能力互补可以补上单个策略采样不到的成功经验，但同伴轨迹属于异策略数据，不能直接等同自身rollout。门控、兼容性权重和重要性裁剪让经验复用同时保留分布校正。

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) methods such as GRPO rely on successful self-generated trajectories, but finite rollout budgets can produce all-fail groups with no reward-based policy-gradient signal. While additional rollouts improve the chance of success at higher cost, successful trajectories missing from one model’s rollouts may already have been discovered by another. Indeed, we observe that heterogeneous models often succeed on complementary prompts, creating opportunities for mutual learning without a designated stronger teacher. To exploit this complementarity, we propose GRAFT (Gated Replacement of Answer-Failed groups with peer Trajectories), an off-policy-aware framework that replaces all-fail groups with informative peer groups. GRAFT transfers both successful and unsuccessful peer responses with peer-computed advantages, while controlling cross-model mismatch through sequence-level compatibility weighting and token-level importance ratio clipping. Across three heterogeneous model pairs and five mathematical reasoning benchmarks, GRAFT consistently improves both models over GRPO with the same per-model rollout budget, gaining 2.1 points on average and up to 4.5 points in model-level average performance. Stored peer trajectories preserve most of the gains, improving over GRPO by 1.8 points on average without simultaneous co-training.

[254] Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoning cs.LG | cs.CLPDF

Md. Ismail Hossain, Humaira Kousar, Isidora Chara Tourni

TL;DR: OASIS保留在线策略自蒸馏目标，但用经标签验证的学生轨迹作为主要监督脚手架，并以学生自己的尝试提供教师上下文，从而只需最终答案标签。

Details

Motivation: 标准在线策略自蒸馏在学生自身采样轨迹上训练，却让教师读取特权参考解。学生轨迹未经验证时会产生模仿鸿沟，因为教师可利用学生无法获得的信息。研究发现，相比教师上下文是否正确，脚手架本身是否验证正确对下游准确率影响更大。

Result: 因子分析表明，模型规模增大虽会缩小未验证脚手架带来的模仿鸿沟，标准OPSD仍主要监督未验证轨迹；而已验证脚手架即便配上学生失败rollout作为教师上下文仍有效。OASIS保留OPSD目标，但主要监督按最终答案标签验证的在线轨迹，并用未经验证的模型尝试代替书面参考解作为教师上下文，因此只需最终答案标签。Qwen3-1.7B、4B、8B在AIME 2024、AIME 2025和HMMT 2025上，OASIS平均比基础模型提高3.2至3.8分；OPSD增益随规模从1.7B的3.05分降至8B的0.14分。8B时OASIS较OPSD高3.05分。

Insight: 自蒸馏的关键不只是教师看到什么，还在于学生轨迹是否真实正确、可作为学习脚手架。只用最终答案标签验证在线轨迹，可在无需长参考解的情况下维持模型规模增大后的蒸馏效果。

Abstract: On-policy self-distillation (OPSD) trains a student to match a privileged teacher distribution along its own sampled trajectory. Standard OPSD applies this supervision to unverified student rollouts while conditioning the teacher on privileged context, typically a reference solution. We separate these roles in a factorial analysis and find that scaffold correctness has a stronger effect on downstream accuracy than context correctness. Unverified scaffolds create an imitation gap because the teacher can use information unavailable to the student. This gap shrinks with model scale, yet OPSD continues to supervise mostly unverified trajectories. In contrast, verified scaffolds remain effective even when the teacher is conditioned on the student’s own unsuccessful rollout. Based on this finding, we introduce OASIS, which retains the OPSD objective but supervises mostly verified by label on-policy trajectories and replaces written solutions with unverified model-generated attempts as the teacher context. OASIS therefore requires only final-answer labels. Across Qwen3-1.7B, 4B, and 8B on AIME 2024, AIME 2025, and HMMT 2025, OASIS improves over the base model by 3.2–3.8 points on average, while OPSD’s gain falls from 3.05 points at 1.7B to 0.14 at 8B. At 8B, OASIS improves over OPSD by 3.05 points, showing that verified on-policy scaffolds preserve the effectiveness of self-distillation as models scale.

[255] Dr. OPD: Learning What to Follow for Optimal On-Policy Distillation of Large Language Models cs.LG | cs.AI | cs.CL | math.OCPDF

Zhenyu Wang, Tianze Wang, Linjun Zhang, Yifan Hu

TL;DR: Dr. OPD将在线策略蒸馏的token监督权重作为双层优化变量学习，让教师信号集中在最能改善学生任务奖励的位置。

Details

Motivation: 标准OPD把教师在每个token上的监督视为同等重要，但不同token纠错对最终任务表现的作用差别很大：有些能修正关键推理错误，另一些对答案影响很小。

Result: Dr. OPD把加权教师监督下的学生训练与最大化学生预期奖励的权重选择写成双层优化，并用高效交替求解器迭代更新token权重和策略：每轮先闭式更新权重，再对加权OPD目标执行一步梯度更新。正则条件下，理论分析表明其加权更新预期奖励高于普通OPD更新。在数学与代码任务、强教师到弱学生及同规模蒸馏设置中，Dr. OPD均超过受测基线；强到弱蒸馏时数学平均表现比普通OPD提高9.7分，并使较小学生超过较大教师。

Insight: 教师信号的价值取决于它能否提升最终任务奖励，而非每个token都应均匀跟随。学习监督权重可将蒸馏从逐token模仿转为面向学生预期表现的目标导向分配。

Abstract: On-policy distillation (OPD) trains a student on its own generated responses using dense, token-level supervision from a stronger teacher. Vanilla OPD treats all teacher signals equally, assuming that the teacher’s supervision is equally important for every token. However, teacher signals at different tokens may have very different effects on the student’s performance: some correct important reasoning errors, while others have little effect on the final answer. Motivated by this observation, we introduce Dr. OPD (OPD Done Right), which defines the optimal weighted OPD to maximize the student’s performance. We formulate Dr. OPD as a bilevel optimization problem in which the student learns from weighted teacher supervision, while the weights are selected to maximize the expected reward of the resulting student. To solve Dr. OPD, we develop an efficient iterative solver that updates the token weights and student policy alternatively. At each round, it updates weights in closed form and then takes one gradient step on the resulting weighted OPD objective. Under regularity conditions, we show that this weighted update achieves a higher expected reward than a vanilla OPD update. Empirically, across strong-to-weak and same-size distillation on math and code, Dr. OPD consistently outperforms all evaluated baselines. In particular, in the strong-to-weak distillation setting, Dr. OPD improves average math performance by $9.7$ points over vanilla OPD, and enables the smaller student to surpass its larger teacher.

[256] $S^3$: Spectral Null-Space Swap Makes Reasoning Models Efficient cs.LG | cs.AI | cs.CL | cs.CVPDF

Hongbo Ma, Sansheng Cao, Jiajun Fan, Bangji Yang, Ge Liu

TL;DR: S³把非思考模型的主奇异方向与思考模型的零空间分量组合，在保留思考后训练准确率的同时减少推理token开销。

Details

Motivation: 思维链训练能提升推理能力，但通常造成大量token开销。作者发现，推理能力的关键成分位于非思考模型主奇异方向定义的投影零空间中，而移除该子空间相关分量可提升效率且不损失思考模式后训练获得的准确率。

Result: Spectral Null-Space Swap（S³）无需训练，组合配对的非思考与思考检查点：保留非思考模型在自身主子空间内的分量，并从思考检查点取其子空间外的成分。评估覆盖2B至30B稠密与MoE架构、28种数学、多模态和音频推理环境。与完整思考模型相比，所有设置平均减少27.4%推理token，同时总体任务准确率提升1.0个百分点；例如HMMT25准确率提高8.3%，token速度提升33.0%。注意力熵分析显示保留分量产生更集中的注意力，简化优化分析解释零空间为何能降低注意力熵并提升推理效率。

Insight: 思维链增益不一定需要完整保留思考检查点的所有权重方向；与非思考模型主方向互补的零空间分量可能承载关键推理能力。权重组合可以在不再训练的条件下改善准确率与token效率。

Abstract: LLMs trained with Chain-of-thought excel in reasoning capability, but often come with excessive token cost. We find that the core of reasoning capacity lies in the Thinking model’s weight component within the null space of a projection defined by the corresponding Non-thinking model’s dominant singular directions, and removing the subspace component can largely improve reasoning efficiency without hurting the accuracy gained during thinking-mode post-training. Unlike existing efforts that mostly operate within the dominant subspace, we are the first to unveil the critical role of the null space and harness it for model optimization. Motivated by this finding, we propose Spectral Null-Space Swap ($S^3$), a training-free composition of paired Non-thinking and Thinking checkpoints. Our method keeps the Non-thinking model inside its own dominant subspace and takes the Thinking checkpoint outside it, improving reasoning efficiency while maintaining accuracy. We extensively evaluate $S^3$ on 2B-30B dense and mixture-of-experts (MoE) architectures spanning 28 evaluation environments across mathematical, multimodal, and audio reasoning domains. $S^3$ establishes new empirical Pareto Frontiers among training-free model composition strategies: across all settings, it reduces inference token overhead by an average of 27.4% compared to full Thinking models while simultaneously improving overall task accuracy by 1.0 percentage point (e.g., yielding +8.3% accuracy on HMMT25 alongside a 33.0% token speedup). We further use attention entropy for explanation and find that the retained component produces more concentrated attention, and we use a simplified analytical model about optimization to demonstrate why null-space can effectively reduce attention entropy, thereby improving the efficiency of reasoning.

[257] HeadGuard: Selective Head Protection for Low-Bit VLM KV-Cache Quantization cs.LG | cs.CVPDF

Nenad Banfic

TL;DR: HeadGuard在任意KV缓存量化器上增加固定高精度注意力头掩码，只保护少量敏感视觉KV头，以恢复低比特视觉语言模型准确率。

Details

Motivation: 低比特KV缓存可降低存储，却可能明显损害视觉语言模型准确率。作者希望不替换基础量化器，只识别少量对图像内容或输出高度敏感的物理KV头并保留高精度。

Result: HeadGuard离线按图像敏感度和输出敏感度选择约八分之一物理KV头；其图像key（可选value）保留BF16，其他图像条目沿用基础量化器。跨8个VLM、3种量化器及8个基准（6个判别、2个生成）评估，较弱量化器上的损失可恢复相当部分，Qwen与InternVL收益最大。2-bit时，最弱基线在8模型六任务判别均值从0.436提高至0.580；生成答案和caption对BF16输出的保真度也可提高。三种量化器及两种校准数据上平均准确率增益均持续存在；只保护key也可保留大部分恢复效果并降低估算存储成本。实验使用模拟量化。

Insight: 并非所有视觉注意力头都同等脆弱，少量关键头的高精度保留可与现有量化方案组合使用。只保护key能进一步降低成本，但当前评估基于模拟量化，实际系统收益仍需结合部署内核验证。

Abstract: Low-bit key-value (KV) cache quantization saves storage but can sharply degrade vision-language model (VLM) accuracy. We introduce HeadGuard, a composable head-protection method that augments a base KV-cache quantizer with a fixed high-precision mask. Image-sensitivity and output-sensitivity scores select physical KV heads offline, with approximately 1/8 protected in the main experiments; their image keys and optionally values remain in bfloat16 (BF16), while the base quantizes unprotected image entries. Across eight VLMs, three base quantizers, and eight benchmarks (six discriminative and two generative), HeadGuard recovers a substantial fraction of lost accuracy on weaker quantizers, with the strongest gains for Qwen and InternVL. At 2 bits, the six-task discriminative mean over eight models rises from 0.436 to 0.580 on the weakest base; protection can also improve generated answers and caption fidelity to BF16 outputs. Mean accuracy gains persist across all three quantizers with both tested calibration datasets. Keys-only protection retains substantial recovery at lower modeled storage cost. Evaluated through simulated quantization, HeadGuard offers a composable way to improve low-bit VLM accuracy without replacing the underlying quantizer.

[258] AdaKerNet: Neural Kernel Decoding for Task-Adaptive Prediction with Multimodal Large Models cs.LG | cs.AI | cs.CVPDF

Konstantinos D. Polyzos, Eleni Oikonomou, Tara Javidi

TL;DR: AdaKerNet为冻结多模态大模型表示学习任务自适应核解码器，无需访问或微调底层模型参数，适用于标注有限的预测任务。

Details

Motivation: 大基础模型便于迁移下游任务，但在监督稀缺时多模态模型仍难适配。直接微调要求访问模型参数，闭源模型通常不提供；训练常规神经解码器也可能在少标签场景下表现不足。

Result: AdaKerNet只使用冻结模型从文本、音频、图像及表格测量中得到的多模态表示。它联合学习Lipschitz受控的多模态特征、提供软结构先验的参考核，以及自适应形变该结构的轻量非线性预测器，从而拟合下游相关特征与几何关系。对BLIP-2、LLaVA-1.5、Qwen2.5-VL和Gemini Embedding 2的测试显示，在不同少标签预算下，AdaKerNet持续优于直接MLP、注意力、自编码器及核解码器基线；跨基线平均误差最多减少41%。消融分析也验证各组件互补。

Insight: 冻结多模态表示并不意味着只能使用固定线性或MLP读出；通过学习任务相关的核几何，可在不触碰基础模型权重的条件下适配少样本任务。

Abstract: Large foundation models have been introduced with the promise of efficient adaptation to downstream tasks. Yet, under limited supervision, MLLMs, an important class of large foundation models, remain challenging to adapt to various downstream tasks. Adaptation typically relies either on MLLM parameter fine-tuning or on training neural-based decoders. Both approaches struggle under limited supervision, while fine-tuning additionally requires access to model parameters, which is often unavailable for closed-source models. We introduce AdaKerNet, a novel learnable task-adaptive neural kernel decoder. AdaKerNet is fully agnostic to the parameters of the underlying MLLM and operates solely on its (frozen) rich representations obtained from the diverse available modalities. AdaKerNet relies on (i) a set of learnable, Lipschitz-controlled multimodal features derived from these MLLM representations; (ii) a reference kernel that provides a soft structural prior on those features; and (iii) a lightweight nonlinear neural predictor that adaptively deforms that structure. Learning the kernel representation and the neural predictor jointly within a unified optimization framework allows AdaKerNet to capture features and geometric relationships relevant to the downstream task. Numerical tests across four MLLMs: BLIP-2, LLaVA-1.5, Qwen2.5-VL, and Gemini Embedding 2, and multimodal inputs spanning text, audio, images, and tabular measurements demonstrate significant and consistent improvements over direct MLP, attention-, autoencoder- and kernel-based decoders, across a range of scarce-label budgets, with average error reduction of up to 41% across baselines. These results establish AdaKerNet as an effective approach for prediction from frozen multimodal representations in the scarce label regime. Additional structural ablations highlight the complementary contributions of AdaKerNet’s components.

[259] PE-OPSD: Internalizing Prompt Enhancement into Flow-matching Models via On-Policy Self-Distillation cs.LG | cs.CVPDF

Mingfeng Lin, Chengfei Cai, Lin Xu, Chengqian Ma, Yuxiang Wei

TL;DR: PE-OPSD将提示增强器产生的详细文本条件蒸馏进文生图流匹配模型，使模型部署时可直接理解简短原始提示。

Details

Motivation: 用户通常给出简短或欠明确的图像提示，而生成模型遵循指令往往需要更详细条件。常见系统在推理时额外运行提示增强器，增加延迟，并让提示扩写能力留在生成器之外。

Result: 在训练时，原始提示学生沿自身生成轨迹采样，增强提示教师则在学生访问的状态上提供向量场目标；这种在线监督把增强提示诱导的行为蒸馏给学生，无需增加文本—图像配对。部署时移除增强器和教师，学生直接根据原始提示生成。在多个模型家族、提示增强器和基准上，PE-OPSD取得受测基线中最强的总体提示忠实度，整体视觉吸引力也有正向提升，同时保留基础模型推理效率。

Insight: 提示扩写可以作为训练期特权信息，而不必成为每次推理都依赖的外部组件。沿学生自身生成状态蒸馏增强提示教师，既内化指令遵循行为，也避免额外文本增强延迟。

Abstract: Text-to-image users often provide concise and underspecified prompts, whereas generative models benefit from detailed textual conditions for reliable instruction following. Existing systems bridge this gap with Prompt Enhancers (PEs) that rewrite raw prompts at inference time, introducing additional latency and leaving prompt elaboration external to the generator. We instead view enhanced prompts as privileged training information and ask whether their benefits can be internalized. We propose Prompt-Enhanced On-Policy Self-Distillation (PE-OPSD) for text-to-image flow-matching models. During training, a raw-prompt student follows its own generation trajectory, while an enhanced-prompt teacher provides vector-field targets at the states visited by the student. This on-policy supervision distills the behavior induced by enhanced prompts into the raw-prompt student without requiring additional text–image pairs. At inference, both the PE and teacher are removed, and the student generates directly from raw prompts. Across multiple model families, PEs, and benchmarks, PE-OPSD achieves the strongest aggregate prompt fidelity among the evaluated baselines, yields positive aggregate visual appeal gains, and retains the base-model inference efficiency.

[260] NowcastDiT: Diffusion Transformers are Effective Precipitation Nowcasters cs.LG | cs.AI | cs.CVPDF

Haoran Xu, Xingzhuo Guo, Yuchen Zhang, Jincheng Zhong, Jianmin Wang

TL;DR: NowcastDiT展示标准扩散Transformer即可作为有效降水临近预报基础，并通过动力学感知噪声先验和时步奖励强化学习处理时空连续性与气象技能。

Details

Motivation: 降水临近预报需在强时空变化下给出准确短期预测。扩散模型适合复杂降水分布建模，但现有方案常采用越来越专门的架构，标准扩散Transformer的能力尚未充分评估。

Result: NowcastDiT基于标准DiT，通过适配其设计空间满足降水预报需求：动力学感知噪声先验改善时间连贯预测，端到端强化学习结合时步感知奖励优化气象技能。在SEVIR和MRMS基准上，模型在感知质量与气象技能两方面均达到当前先进表现。摘要未列具体数值。

Insight: 领域任务未必需要复杂专用架构；标准DiT配合与降水动力学和气象评估对应的训练目标即可成为可扩展基础。

Abstract: Precipitation nowcasting demands accurate short-term forecasts under strong spatiotemporal variability. Diffusion models are well suited to modeling complex precipitation distributions, yet existing approaches often introduce increasingly specialized designs, leaving the capability of a standard diffusion architecture underexplored. We show that a standard Diffusion Transformer already provides a simple and scalable foundation for precipitation nowcasting, with domain-specific requirements accommodated naturally within its design space. Based on this principle, we develop NowcastDiT and instantiate this flexibility through two complementary adaptations: a dynamics-aware noise prior for temporally coherent forecasts, and end-to-end reinforcement learning with timestep-aware rewards for meteorological skill. Experiments on SEVIR and MRMS benchmarks show that NowcastDiT achieves state-of-the-art performance in both perceptual quality and meteorological skill. These results suggest that standard DiT can serve as an effective foundation for precipitation nowcasting.

[261] Scaling Full Conformal Image Classifiers cs.LG | cs.CVPDF

Julio Silva-Rodríguez, Ender Konukoglu

TL;DR: T-FCP利用零样本视觉语言模型辅助剪除不可能标签，只对剩余候选执行完整共形预测，并以高效在线LDA适配降低高类别数图像分类成本。

Details

Motivation: 共形预测能提供具有分布无关覆盖保证的集合预测，适用于高风险图像分类；拆分式共形方法数据效率较低，而完整共形虽统计效率更高，却需在测试时按候选标签重拟合模型，计算成本难以扩展。

Result: Targeted Full Conformal Prediction（T-FCP）先用轻量归纳式共形预测器剪去不太可能的标签，再仅在剩余候选上执行完整共形预测，保留组合共形流程的形式保证并降低计算。作者还提出稳定在线LDA（SO-LDA），通过秩一逆协方差更新高效完成视觉语言模型适配。在包括ImageNet的多个基准上，T-FCP以适度测试时开销实现实用的完整共形图像分类，产生高效预测集合，其经验覆盖率比拆分式共形替代方案更稳定。

Insight: 完整共形推断的主要成本来自对庞大标签空间逐类计算，可先筛掉低可能标签再保留正式共形检验。筛选器不能破坏组合流程的覆盖保证，因此前置剪枝需纳入整体理论设计。

Abstract: Conformal prediction provides set-valued predictions with distribution-free coverage guarantees, making it attractive for high-stakes image classification. However, split conformal prediction is data-inefficient, while full conformal prediction (FCP), despite its stronger statistical efficiency, is computationally prohibitive at scale because it requires candidate-specific model refits at test time. We address this limitation by leveraging zero-shot vision-language models (VLMs) to guide scalable FCP in large label spaces. We introduce Targeted Full Conformal Prediction (T-FCP), which uses a lightweight inductive conformal predictor to prune unlikely labels and applies FCP only to the remaining candidates, reducing computation while retaining the formal guarantee of the combined conformal procedure. We further propose Stabilized Online LDA (SO-LDA), an efficient VLM adaptation solver based on rank-one inverse-covariance updates. Across multiple benchmarks, including ImageNet, T-FCP enables practical full-conformal image classification with modest test-time overhead, yielding efficient prediction sets and more stable empirical coverage than split conformal alternatives.

[262] Procedural Core: A Compact Recurrent Initialization for Vision Transformers cs.LG | cs.CVPDF

Zachary Shinnick, Christian Internò, Hemanth Saratchandran, Anton van den Hengel, Damien Teney

TL;DR: Procedural Core将程序化数据中学到的通用归纳结构压缩到一个小型循环Transformer，再扩展权重初始化不同宽度和深度的模型。

Details

Motivation: Transformer通常从随机权重开始，所有通用结构都需通过大规模优化学得。程序化合成数据能以较低成本获得通用归纳结构，但若每个目标模型都重做这段预训练，成本仍会重复。

Result: 作者用程序化数据训练最小循环Transformer，再将其紧凑权重扩展到任意宽度和深度的Transformer。初始化改善图像分类、DINO自监督视觉学习、自然语言和代码建模。图像分类中，把1M参数核心扩展为85M ViT-Base后，ImageNet top-1比随机初始化高2.2个百分点。分析显示循环结构是学习可迁移紧凑权重的关键。在ViT中，部分收益来自抑制高范数token：ImageNet-S零样本分割mAP从32.3升到42.9，VOC07定位CorLoc从9.9升到18.4，NYUv2深度RMSE从1.104降至0.998。无需领域或任务特定数据。

Insight: 通用归纳偏置可以先由小模型在程序化数据上学习，再作为可扩展的初始化复用，减少每个目标模型都从空白起步的成本。循环训练使小核心有能力将结构压缩进可迁移权重。

Abstract: Transformers are typically trained from random initialization, requiring all their capabilities to emerge from large-scale optimization. Recent work showed that a small amount of abstract procedurally generated data can help acquire generic inductive structure at low cost. However, this adds a pretraining stage that must be repeated for every target model. We propose Procedural Core, an initialization strategy that captures this generic structure into a compact set of weights that can be reused across models. We train a minimal recurrent transformer on procedural data, then expand its weights to initialize transformers of arbitrary width and depth. The resulting initialization improves performance on image classification, self-supervised visual learning (DINO), and modeling natural language (FineWeb-Edu) and code (CodeParrot). For image classification, expanding a 1M-parameter core to initialize an 85M-parameter ViT-Base improves ImageNet top-1 accuracy by 2.2 pp over standard random initialization. Our analysis identifies recurrence as essential for learning compact weights that transfer across models. In ViTs, we localize a key benefit in the suppression of high-norm tokens that produces substantial improvements in zero-shot segmentation (ImageNet-S mAP 32.3 to 42.9), object localization (VOC07 CorLoc 9.9 to 18.4), and depth estimation (NYUv2 RMSE 1.104 to 0.998). This demonstrates that transformers need not start from a blank slate, and can be initialized with generic capabilities at low cost with no domain- or task-specific data.

cs.NE [Back]

[263] Multi-Depth Temporal Fusion for Feedforward, Locally Trained Spiking Neural Networks cs.NE | cs.CV | cs.LGPDF

Aidin Attar, Eleonora Cicciarella, Michele Rossi

TL;DR: 多深度时间融合SNN保留早期脉冲证据，按时间一致性逐步加入中间和深层特征，从而提升局部在线学习下的图像和事件分类。

Details

Motivation: 作者关注多层卷积脉冲网络中哪些结构最适合局部、在线训练。深层网络若直接替换早期证据，可能丢失早期视觉时间信息；需要以稀疏脉冲方式融合多深度表示。

Result: 网络先将视觉输入编码为首次脉冲延迟，四层卷积主干用无监督STDP逐层训练；随后使用确定性多深度时间融合（MDTF），最终分类器以奖励调制STDP训练。MDTF保留早期时间证据，添加中间层稀疏残差事件，仅在深层特征与早期表示时间一致时纳入深层信息。在MNIST、Fashion-MNIST、CIFAR-10和N-MNIST上验证，Fashion-MNIST较传统STDP/R-STDP基线提高18.2个百分点，CIFAR-10提高29.2个百分点。活动预算分析表明，移除大量晚发或弱脉冲后仍保持高准确率，数据效率较高且减少事件处理需求。

Insight: 局部训练的脉冲网络可以用时间一致性来决定深层证据是否值得融合，而不是丢弃早期特征或一味累加所有层。该机制在高变化视觉任务上尤其能提升表现。

Abstract: We propose a new spiking neural network (SNN) design to process static images and event streams using time-to-first-spike (TTFS) latencies. Our key research question is which architectural choices best accommodate local and online learning in multi-layer convolutional SNNs. This question is addressed via an original framework combining residual-like connections with multi-depth feature aggregation and consensus. The full SNN pipeline features an early-vision front end, to convert raw visual data into sparse spike latencies, a four-layer convolutional backbone trained layerwise with unsupervised spike-timing-dependent plasticity (STDP), a deterministic Multi-Depth Temporal Fusion (MDTF) and a final classifier trained with reward-modulated spike-timing-dependent plasticity (R-STDP). Rather than replacing early features in deeper layers, the proposed MDTF preserves early temporal evidence, adding sparse residual events from intermediate layers, and incorporating deeper features only when they agree in time with earlier representations. The resulting architecture is experimentally validated across MNIST, Fashion-MNIST, CIFAR-10, and N-MNIST, delivering strong classification performance under a fully local learning regime. Selective multi-depth fusion significantly outperforms traditional STDP/R-STDP baselines on higher-variability visual tasks (achieving +18.2 pp on Fashion-MNIST and +29.2 pp on CIFAR-10). Furthermore, activity-budget analyses show that the network retains high accuracy even when removing a large fraction of late or weak spike events, confirming its high data efficiency and reduced event-processing requirements. The codebase is publicly available at github.com/aidinattar/multi-depth- temporal-fusion-snn.

cs.RO [Back]

[264] FineART: Fine-grained Annotated Robotic Trajectory Dataset and Vision-Language-Action Model for Bimanual Manipulation cs.RO | cs.AI | cs.CV | cs.LGPDF

Jade Choghari, Pepijn Kooijmans, Mansi Agarwal, Yusuf Umut Ciftci, Aseem Doriwala

TL;DR: FineART发布密集标注的双臂机器人操作数据集，并训练能预测自身下一子任务的视觉—语言—动作策略以支持长时程多步操作。

Details

Motivation: 真实机器人需完成长时程复杂双臂操作，而非孤立动作。既有数据集虽有大量单臂轨迹，但每段通常只有一条高层指令；少数双臂数据也只覆盖有限小时且子任务标注稀疏。

Result: FineART包含40,543个episode、1,718小时、533,913个子任务，覆盖151项任务。FineART-VLA策略可预测自身下一个子任务；以此方式中训练后，空间消歧任务成功率从32.0%升至100.0%，逐步的人类子任务指导使未见长时任务成功率从16.0%升至76.0%。迁移到新机器人只需少量微调，所需数据为基线的十分之一；还可零样本泛化到新硬件上的完全未见任务。作者开放数据、模型权重和训练代码。

Insight: 对长任务提供细粒度的子任务标注，使策略能在运行中规划下一个阶段，而不只模仿整段轨迹。把子任务预测纳入中训练，可减少换机器人后的数据需求并增强长时程泛化。

Abstract: Robots operating in real-world environments must execute complex, multi-step bimanual tasks over long horizons rather than single, isolated actions. Current manipulation datasets struggle to support this capability: although single-arm datasets reach hundreds of thousands of trajectories, they typically provide only one high-level instruction per episode while the rare bimanual effort that does label subtasks annotates only a fraction of its hours. We present FineART, a densely annotated bimanual manipulation dataset of 40,543 episodes, 1,718 hours, and 533,913 subtasks across 151 tasks. We also introduce FineART-VLA, a vision-language-action policy that predicts its own next subtask, and show that mid-training it this way yields substantial gains. Specifically, success on a spatial disambiguation task increases from 32.0% to 100.0%, and step-by-step human subtask guidance lifts success on an unseen long-horizon task from 16.0% to 76.0%. Furthermore, after minimal fine-tuning on a new robot, the policy requires one-tenth the data of baselines without mid-training and generalizes zero-shot to completely unseen tasks on the new hardware. We open-source the full dataset, model weights, and training code.

[265] Track-and-Complete: Learning Humanoid Skills from a Single Failed Human Video cs.RO | cs.CVPDF

Sarmad Idrees, Jongeun Choi

TL;DR: TRACC从一次失败的人类操作视频中提取失败前可用动作轨迹，再以任务完成奖励学习后续动作，使人形机器人无需成功演示也能学技能。

Details

Motivation: 从视频学习机器人技能通常要求成功的人类演示，这需要定制数据采集。失败视频虽未完成任务，但仍包含一段有效轨迹和预期目标，不应只被当作负面样本丢弃。

Result: Track-and-Complete（TRACC）先模仿失败前仍可用的运动前缀，将其作为先验；失败后再用任务完成奖励引导策略达到推断出的目标，不需要一条完整成功轨迹。方法在Oops!数据集的六个野外失败人类任务上评估，实验显示在缺少成功演示时可从失败尝试学习技能；摘要未给具体成功率。

Insight: 失败轨迹能提供局部可行行为和任务意图两类监督：前缀提供运动先验，最终目标奖励则引导策略弥补失败后的部分。将两者分开利用可扩展可用的人类示范来源。

Abstract: Learning humanoid skills from videos typically requires a successful human demonstration, which often demands custom data collection. Although failures have traditionally been treated only as negative examples in robot learning, they can still reveal a usable trajectory prefix before the task fails, as well as the intended outcome. To leverage this information from a failed-attempt video, we propose TRACC, a pipeline that imitates the useful portion of the motion trajectory and then completes the task based on the inferred task outcome. The usable motion prefix serves as prior knowledge until the failure occurs, after which the task-completion reward guides the policy to learn the intended task goal without requiring a successful task trajectory. We evaluate our method on six in-the-wild failed human tasks from the Oops! dataset. Our experimental results demonstrate the effectiveness of the proposed approach for learning from failed attempts when no successful demonstration is available. Thus, these findings establish failed human videos as a viable source of supervision for humanoid skill learning.

[266] V2X-WAM: A Cooperative World Action Model for End-to-End Autonomous Driving cs.RO | cs.CVPDF

Junwei You, Weizhe Tang, Can Wang, Yan Zhao, Jun Hua

TL;DR: V2X-WAM将车端与基础设施观察、轨迹生成及未来占用预测闭环结合，使自动驾驶规划显式考虑候选动作对未来交通世界的影响。

Details

Motivation: 路侧基础设施可补充车载感知，提供更广泛交通信息，但多数协同驾驶方法只用它增强当前场景表示，很少显式预测候选动作在交通动态变化中的未来后果。

Result: V2X-WAM构建可靠性感知的时空表示，并将基础设施信息压缩为紧凑量化消息以降低通信开销。多模态规划器基于协同表示生成候选轨迹，这些轨迹明确条件化未来占用与动态流预测；再把预测到的世界后果反馈给规划器修正轨迹，形成动作与未来演化闭环。在大规模真实协同驾驶数据集实验中，V2X-WAM相较代表性端到端协同方法持续提升规划准确性和安全性，同时改善未来世界预测并显著降低通信开销；消融研究验证了各设计作用。摘要未列具体指标。

Insight: 协同驾驶不应仅把基础设施观察融合进当前场景，还应将候选动作与未来交通状态联系起来，并据此迭代规划。量化压缩和可靠性感知表示使这种闭环可以兼顾通信效率。

Abstract: Vehicle-infrastructure cooperation can complement onboard sensing with broader and more informative observations of the traffic environment, providing valuable support for end-to-end autonomous driving. However, existing cooperative driving methods mainly exploit roadside information to enhance the representation of the current scene, while the future consequences of prospective driving actions are rarely modeled explicitly. This limits the ability of the planner to anticipate how its decisions may interact with the evolving traffic environment. To address this issue, we propose V2X-WAM, a cooperative world action model that tightly couples cooperative scene understanding, action generation, and future-world reasoning. V2X-WAM constructs a reliability-aware spatiotemporal representation from vehicle- and infrastructure-side observations, while compressing infrastructure information into a compact quantized message for efficient communication. Based on the resulting cooperative representation, a multimodal planner generates prospective trajectories, which explicitly condition future occupancy and dynamic-flow prediction. The predicted world consequences are then fed back to refine the planned trajectory, forming a closed interaction between action and future-world evolution. Experiments on a large-scale real-world cooperative driving dataset demonstrate that V2X-WAM consistently improves planning accuracy and safety over representative end-to-end cooperative driving methods, while achieving stronger future-world prediction and substantially lower communication overhead. Ablation studies further validate the effectiveness of the proposed design.

[267] RawVLA: Embodied Neural Image Signal Processor For Robotic Manipulation cs.RO | cs.CVPDF

Shuhong Liu, Heng Zhou, Lingfeng Qian, Yuhao Fang, Xianbao Hou

TL;DR: RawVLA让神经图像信号处理器直接适配RAW传感器观测，为冻结视觉—语言—动作策略提供与具身行为相关的图像处理。

Details

Motivation: VLA通常接收固定相机ISP输出的RGB图像，将成像流程排除在学习和评估之外。增益、传感器噪声、颜色响应、色调响应和位深等RAW到RGB处理因素可能实质影响动作预测和操作成功。

Result: 作者系统分析五类ISP因素，发现它们对动作预测及操作成功的影响差异显著。RawVLA是流式神经ISP，可为冻结VLA策略自适应渲染RAW观测，并将容量集中到与具身行为相关的成像因素。研究还构建RawVLA-Bench，在干净及恶劣采集条件下将图像处理显式纳入RAW域操作评估。实验显示，RawVLA在标准条件下保持原有性能，并在成像退化时显著提升鲁棒性；摘要未列具体分值。

Insight: 相机ISP不是策略之外的无关预处理，它会改变控制所依赖的视觉证据。把RAW处理纳入具身策略接口，可按任务需求调整成像，并评测系统在退化传感条件下的稳健性。

Abstract: Vision-language-action (VLA) models typically operate on RGB images produced by a fixed camera image signal processor (ISP), leaving the imaging pipeline outside the learning and evaluation loop. We systematically examine the consequences of this overlooked design choice across five fundamental ISP dimensions: gain, sensor noise, chromatic response, tonal response, and bit depth. Our analysis reveals that RAW-to-RGB processing materially shapes both action prediction and manipulation success, with different ISP dimensions exerting substantially different effects. Guided by these findings, we introduce RawVLA, a streaming neural ISP that adaptively renders RAW observations for frozen VLA policies while concentrating its capacity on the imaging factors relevant to embodied behavior. We further present RawVLA-Bench, a RAW-domain manipulation benchmark to expose image processing as an explicit evaluation variable across clean and adverse acquisition conditions. Experiments on RawVLA-Bench show that RawVLA preserves performance under standard conditions while substantially improving robustness under degraded imaging, establishing adaptive RAW processing as an effective interface between physical cameras and embodied policies.

Jiaming Wang, Duc Thang Nguyen, Jizhuo Chen, Volodymyr Shcherbyna, Diwen Liu

TL;DR: 论文将普通单目步行视频直接转换成社交导航策略状态空间中的闭环训练环境，以可通行地图和回放行人轨迹生成反事实机器人运动。

Details

Motivation: 训练稳健社交导航策略需要覆盖多样布局、地形及人类运动的模拟器，构建环境和手工规定行人行为代价高。作者希望从普通视频中直接提取训练所需信息，而无需重建或渲染完整逼真场景。

Result: 方法将静态场景表示为度量可通行性地图，可随反事实机器人运动刚性变换；从视频恢复的行人轨迹则按时间直接回放。由于社交导航主要取决于机器人可行走区域和附近行人移动，前向动力学可直接在策略状态空间定义并高效模拟，无需重建渲染像素观察。所得策略在独立Arena基准上成功率81.2%，强基线为75.0%；在真实机器人上无需微调，20次试验成功19次。

Insight: 社交导航训练不一定需要从视频重建 photorealistic 世界，若策略只依赖可通行区域和行人运动，可在决策状态空间直接构建闭环反事实环境。该抽象同时降低仿真成本，并在真实机器人测试中展示迁移能力。

Abstract: Training robust social-navigation policies requires simulators with diverse scene layouts, terrain, and human motion, but constructing such environments and specifying pedestrian behavior is costly. We propose an efficient pipeline that converts ordinary monocular walking videos directly into closed-loop social-navigation training environments in the policy’s state space. Our key observation is that local social navigation primarily depends on two types of information: where the robot can traverse and how nearby pedestrians move. We therefore represent the static scene as a metric traversability map, which can be rigidly transformed under counterfactual robot motion, while directly replaying the pedestrian trajectories recovered from the video over time. This abstraction allows us to define the forward dynamics directly in the policy’s state space and efficiently simulate counterfactual robot states without reconstructing or rendering photorealistic observations. The resulting policy achieves 81.2% success in the independent Arena benchmark, compared with 75.0% for the strongest baseline, and succeeds in 19/20 real-robot trials without policy fine-tuning.

[269] CogWAM: Aligning Semantic Cognition with World Action Modeling via Event-Driven Interfaces cs.RO | cs.CVPDF

Sen Wang, Liu Liu, Xinjiang Wang, Zequn Chen, Haoyi Jiang

TL;DR: CogWAM用持久语义状态记录已完成事件和当前子任务，并以进度条件化的世界与动作查询连接高层任务理解和机器人控制。

Details

Motivation: 机器人策略越来越多地结合语义推理和未来世界预测，但两者并存并不保证动作与任务进展同步。局部预测和动作若缺少任务级进度接口，模型难以保持跨动作块的整体目标。

Result: CogWAM只在观察到语义转变时更新持久Semantic State，保存已完成事件和当前子任务，使任务上下文跨多个动作块延续。进度条件化WORLD与ACTION查询分别提取与任务相关的信息用于未来世界预测和动作生成；训练时共享语义进度状态，推理时移除未来预测分支，直接根据观察和持久状态生成动作。语义训练策略进一步改进转移学习及闭环条件化。无需额外机器人动作预训练，CogWAM在RoboDojo得分/成功率为15.56/11.70%，并在BiCoord达到当前最佳；真实实验中完成闭环双臂操作，Semantic State重新生成次数比逐步更新少16.4次。

Insight: 持久的事件级状态可将高层任务进度带入后续局部动作，避免每一步重新推断完整任务。预测和行动共用该状态，而部署时去掉预测分支，有利于训练目标与实时控制协同。

Abstract: Robot policies increasingly incorporate semantic reasoning and future-world prediction, yet combining these capabilities does not guarantee that local predictions and actions remain aligned with task progress. We introduce CogWAM, a cognition-guided world-action model that establishes an explicit semantic interface between task reasoning and world-action learning through a persistent Semantic State, which stores completed task events and the active subtask. CogWAM updates this state only when observations indicate semantic transitions, allowing task-level context to persist across multiple action chunks. To bridge semantic context with physical prediction and control, CogWAM employs progress-conditioned WORLD and ACTION queries that selectively extract task-relevant information for future-world prediction and action generation. During training, the Semantic State provides shared task-progress context for both branches, while inference removes the future-prediction branch and directly generates actions from observations and the maintained state. We further introduce semantic training strategies to improve transition learning and closed-loop conditioning. Without additional robot-action pretraining, CogWAM achieves 15.56 / 11.70 % Score/SR on RoboDojo and state-of-the-art performance on BiCoord, while real-world experiments demonstrate closed-loop dual-arm manipulation with 16.4 fewer Semantic State regenerations than step-wise updating.

[270] PhysWAM: Physically Consistent World Action Model for Autonomous Driving cs.RO | cs.CVPDF

Dhruv Parikh, Fengcheng Yu, Quankai Gao, Jiawei Yang, Junjie Ye

TL;DR: PhysWAM在同一流匹配Transformer内联合生成多视角视频、度量深度和自车运动，并通过点云投影约束保证世界预测和动作之间的几何一致性。

Details

Motivation: 世界动作模型联合预测场景变化与智能体动作，但联合生成本身并未确保两者共享几何约束。自动驾驶需要保证深度、运动和观测场景物理相符。

Result: PhysWAM将多视角视频、度量深度和自车运动放入统一模型共同去噪。耦合点投影CPP把生成深度反投影成三维点，以生成的SE(3)自车运动变换，再与使用记录运动变换的LiDAR点云比较距离，从而联合监督深度和运动的物理一致性。推理时仅用无标签共识规则选择轨迹，不使用学习评分器或模拟器反馈。在NAVSIM v1/v2规划、零样本闭环迁移及未来视频/度量深度预测评估中，方法规划表现强且可迁移到未见驾驶环境，同时生成准确度量深度和时间连贯视频；CPP改善规划和深度预测。摘要未列具体数值。

Insight: 把深度点云与自车运动放入同一个几何关系中，可为世界生成和动作预测提供直接共享约束。无学习评分器的轨迹选择也表明物理一致建模可与简单推理时决策配合。

Abstract: World-action models (WAMs) jointly predict how a scene will evolve and how an agent should act, however joint generation alone does not necessarily impose a shared geometric constraint on these predictions. We present PhysWAM, a unified world-action model for autonomous driving that co-denoises multiview video, metric depth, and ego motion within a single flow-matching transformer. To ground world and action generation in measured scene geometry, we introduce Coupled Point Projection (CPP) that unprojects the generated depth into 3D points, transforms them using the generated $\mathrm{SE}(3)$ ego motion, and minimizes their distance to LiDAR points transformed using the recorded ego motion. This geometric constraint promotes physical consistency with the measured scene by jointly supervising generated depth and motion alongside their standard flow-matching objectives. At inference, trajectory selection relies only on a simple label-free consensus rule, with no learned scorer or simulator feedback. We evaluate PhysWAM across NAVSIM v1 and v2 planning, zero-shot closed-loop transfer, and future video and metric-depth prediction. Despite PhysWAM’s simple selection procedure, it achieves strong planning performance and transfers zero-shot to unseen driving environments. It also generates accurate metric depth and temporally coherent video, with CPP improving both planning and depth prediction. Together, these results demonstrate that the geometric relationship between scene depth and ego motion provides a direct way to couple world and action generation within a simple unified model.

[271] WorldLine: Action-Driven Visual Simulation for Robotic Manipulation cs.RO | cs.CVPDF

Shenghe Zheng, Wenbo Li, Jiyao Zhang, Bin Xia, Haoyang Huang

TL;DR: WorldLine从大量无动作标签机器人视频学习可迁移动力学，再用多种机器人动作轨迹做具身对齐，构成可用于策略评估和规划的动作驱动视觉模拟器。

Details

Motivation: 真实机器人学习受限于经验采集和候选行为评估成本。视频生成可提供可扩展视觉模拟基础，但模型常重视画面逼真而忽视动作遵循与机器人—物体动力学；现有动作条件模拟器又依赖稀缺且难跨控制空间共享的具身数据。

Result: WorldLine用超过10,000小时无动作标签机器人视频学习操作动力学，再用超过2,000小时、十多种具身平台动作轨迹进行动作落地。图像空间动作表示提供跨具身控制接口，多视角及失败样本增强训练结合关系正则提升交互敏感预测，机器人特定少步蒸馏实现高效因果rollout并保留关键运动。在留出及域外设置中，视觉质量和机器人运动一致性强；失败轨迹上机器人掩码IoU比最强基线高0.1626。在RoboTwin和AgiBot上预测轨迹成功的平均准确率74%，高于基线1个百分点。无需RoboTwin训练或适配时，其rollout比直接执行策略最多提升21.4个百分点任务成功率。

Insight: 可迁移动力学学习与异构动作接口可分阶段处理：大量无动作视频用于学习视觉动力学，多具身轨迹用于动作对齐。失败数据、关系约束和因果少步rollout则补足动作模拟的交互质量。

Abstract: Real-world robot learning is constrained by the cost of collecting experience and evaluating candidate behaviors. Video generation models offer a scalable foundation for visual simulators that predict action outcomes before physical execution. Yet they often favor visual plausibility over accurate action following and coherent robot–object dynamics, while action-conditioned simulators depend on scarce, embodiment-specific data that are difficult to share across incompatible control spaces. We introduce WorldLine, an action-driven visual simulator that decouples transferable dynamics learning from heterogeneous action grounding. WorldLine learns manipulation dynamics from more than 10,000 hours of action-free robot videos and grounds them using over 2,000 hours of action trajectories across more than ten embodiments. An image-space action representation provides a shared control interface across embodiments, while multi-view and failure-enriched training with relational regularization improves interaction-sensitive prediction. Robot-focused few-step distillation enables efficient causal rollout while preserving action-critical motion. Across held-out and out-of-domain settings, WorldLine maintains strong visual quality and robot-motion agreement; on failed trajectories, it improves robot-mask IoU by 0.1626 over the strongest baseline. It predicts trajectory success with 74% mean accuracy across RoboTwin and AgiBot, one percentage point above the strongest baseline. Without RoboTwin training or adaptation, its rollouts improve task success by up to 21.4 percentage points over direct policy execution. Together, these capabilities make WorldLine a scalable and efficient visual simulator for policy evaluation and embodied planning. More results are available at \href{https://zhengsh123.github.io/WorldLine/}{project page}.

[272] Counterfactual Video Generation Enables Scalable Humanoid Loco-Manipulation cs.RO | cs.CV | cs.GRPDF

Zihan Wang, Zhen Wu, Pieter Abbeel, Rocky Duan, Jitendra Malik

TL;DR: PRISM将少量真实人—物交互视频扩增为多样反事实生成视频，经接触锚定的实到仿流程重建并物理化，再训练可泛化的人形机器人移动操作策略。

Details

Motivation: 通过视觉模仿教人形机器人搬运不同物体很有前景，但收集多样、高质量交互视频困难，尤其难同时看到完整人体和无遮挡物体接触。

Result: PRISM先从少数真实示例视频用视频到视频生成产生数百个多样反事实人—物交互视频，再通过接触锚定的真实到仿真流程重建人体与物体运动，将不完美视频数据重定向为物理合理轨迹。类内多样性支持训练一个可泛化到类别内未见物体的策略。完整流程在真实机器人部署，不做真实世界微调；仅使用机载深度观察，即可对箱子、桶、收纳箱和球等新实例、尺寸及初始配置完成拾取、搬运和放置。摘要未报告任务成功率。

Insight: 少量真实示例可通过反事实视频生成扩展交互变化，再由接触约束和物理重定向过滤生成数据与机器人动力学之间的差异。按物体类别共享策略，能在新实例上迁移而不依赖真实机器人微调。

Abstract: Teaching humanoids loco-manipulation skills, such as carrying diverse objects, via visual imitation is a promising path toward generalist robots. However, collecting diverse, high-quality interaction videos, such as clips that clearly show a person’s full body and unoccluded interactions with objects, poses a practical barrier to scaling this approach. We propose PRISM, a real-to-sim-to-real framework that overcomes this limitation by amplifying a handful of real videos into a large, diverse training set. PRISM first generates hundreds of diverse “counterfactual” human-object interaction videos via video-to-video (V2V) generation from a few exemplar real videos. Our contact-anchored real-to-sim pipeline then reconstructs both human and object motions, retargeting this imperfect video data into physically plausible trajectories. The intra-class variability across these counterfactual videos lets us train a single policy that generalizes to unseen objects within each category. We demonstrate the full pipeline by deploying this policy on a real robot without any real-world fine-tuning. Using only onboard depth observations, our humanoid picks up, carries, and drops objects, including boxes, barrels, bins, and balls, across novel instances, sizes, and initial configurations.

q-bio.NC [Back]

[273] Better Behavioral Prediction, More Faithful Model Ablations? Evidence from Sequential Choice q-bio.NC | cs.CLPDF

Hanbo Xie

TL;DR: 在已知真实决策策略的序贯选择任务中，作者发现高预测准确率并不保证模型消融能忠实反映真实行为对输入信息的依赖。

Details

Motivation: 研究者常通过移除输入信息并观察预测性能变化，来推断这些信息对行为的重要性；这种解释成立的前提是模型依赖信息的方式能代表行为生成机制。作者在生成策略已知的合成任务中检验这一前提。

Result: 研究比较两种合成序贯老虎机任务中的 GRU、Transformer、微调 LLaMA 和认知模型，并改变奖励贡献。作者区分无奖励观测训练后的预测，以及对固定预测器替换供体奖励的响应。其一，在非平稳任务中，不看奖励训练的神经模型预测留出选择优于四种简单行为基线；其二，在匹配的供体替换实验中，准确预测器对干预的响应可能远弱于真实生成策略；其三，某些奖励权重下神经网络虽比合并强化学习模型预测更好，对选择概率变化的忠实度却更差，而且模型排序随任务改变。摘要未给具体数值。

Insight: 预测所需的信息与干预时应引起的真实行为变化不是同一性质。若要用输入消融解释行为机制，必须独立验证模型对消融的响应忠实度，不能仅凭预测准确率推断生成过程。

Abstract: Using predictive models to explain cognition requires more than accurate behavioral predictions. Input ablations offer an appealing route: remove information from a model and interpret the resulting performance change as evidence of its importance for behavior. Yet this inference assumes that the model’s dependence on information reflects the dependence of the process generating the behavior. We test it in two synthetic sequential bandit tasks with known generating policies, where past choices can remain informative when feedback is unavailable to a predictor. We compare GRUs and Transformers trained from scratch, a fine-tuned LLaMA model, and cognitive models across systematically varied reward contributions. Our analyses distinguish prediction after training without reward observations from the response of a fixed predictor to donor-reward replacement. Three findings emerge. First, in the restless task, neural models trained without rewards predict held-out choices better than four simple training-fitted behavioral baselines. Second, under matched donor replacement, accurate predictors can respond much less than the known generator. Third, at some reward weights, neural networks predict better than a pooled reinforcement-learning model but have less faithful changes in choice probabilities; the model ordering differs between the two tasks. These independent-test results separate information sufficient for prediction from response fidelity under a specified ablation in sequential choice. They motivate validating model-ablation responses independently of predictive performance before using them to infer how the observed behavior was generated.

Table of Contents

cs.CV [Back]

[1] Systematic Multi-Agent Vision-and-Language Navigation: Formulation, Benchmark, and Method cs.CV | cs.AI | cs.ROPDF

[2] CoVLM-Bench: A Real-World Benchmark for Cooperative Driving Question Answering and Planning cs.CVPDF

[3] CoDimRecon: Agentic Reconstruction of Sim-Ready 3D Scenes with Deformable Curves, Surfaces, and Volumes cs.CV | cs.GR | cs.ROPDF

[4] AerialDojo-200K: A Large-Scale Benchmark Suite for Open-World Aerial Object-Goal Search cs.CV | cs.AI | cs.ET | cs.MM | cs.ROPDF

[5] One Geometry, Different Outcomes: Readout-Dependent Effects of the Modality Gap in Vision-Language Models cs.CV | cs.AI | cs.LGPDF

[6] Xiaomi-OCR-0 Technical Report cs.CVPDF

[7] From Sharp Eyes to Expert Mind: Internalizing Expert Knowledge in MLLMs for Tampered Text Detection cs.CVPDF

[8] Exploring Learning Models for Topological Relationship Recognition from Image Data cs.CV | cs.LGPDF

[9] PreviewDiff: Multimodal Critic-Guided Search over Diffusion Latents cs.CV | cs.AI | cs.LGPDF

[10] Sparse-View Interpretable 3D Animal Behavior Representations for Neural Encoding and Decoding cs.CVPDF

[11] LeRF: Learning Reference Coordinate Frames for Perspective Taking Reasoning cs.CVPDF

[12] Mutually Adversarial Self-Training with Evolving Data for Unified Multimodal Models cs.CV | cs.AIPDF

[13] StructRL: Online Structured Reinforcement Learning for Long-Horizon Vision-Language-Action Tasks cs.CV | cs.LG | cs.ROPDF

[14] OTT3R: Multi-View 3D Reconstruction and Fast Dataset Generation at 1% Compute cs.CVPDF

[15] Compress to Remember: Learning Compact Memory via On-Policy Distillation for Long Video Generation cs.CV | cs.AI | cs.LGPDF

[16] LEGO-Anything: Coding Agents for 3D Scene Reconstruction cs.CVPDF

[17] DARE to Mitigate Hallucination: Dual-path Auto-Regressive-aware Editing cs.CVPDF

[18] RA-CFGCache: From Branch-Level Criteria to Guided-Risk Control under Classifier-Free Guidance cs.CVPDF

[19] Online Versatile Incremental Learning: Towards Class and Domain-Agnostic Adaptation at Any Time cs.CV | cs.AIPDF

[20] DynamicHOI: Coupled Dynamics for Physics-aware HOI Reconstruction cs.CVPDF

[21] Reimagine Video Dynamics cs.CVPDF

[22] Foresight at the Event Boundary: Evaluating Physical Prediction in Video World Models cs.CVPDF

[23] Benchmarking Vision-Language Models on Synapse Detection and Proofreading in Connectomics cs.CV | cs.AI | cs.LGPDF

[24] ThinkingGuard: Decoding Implicit Hazards via Step-by-Step Risk Attribution in Multimodal Large Language Models cs.CV | cs.AI | cs.LGPDF

[25] AffectReveal: Event-Grounded Emotion Recognition Beyond Visual Appearances cs.CVPDF

[26] How Medical VLMs Underutilize Their Vision Encoders: A Dermatology Perspective cs.CV | cs.AIPDF

[27] Beyond Legibility: Benchmarking Visual Text Rendering and In-Place Editing in Unified Video Generation cs.CVPDF

[28] CrossTimeEdit: A Decade-Spanning Cross-View Dataset and Reward-Guided Editing for Historical Street-View Generation cs.CVPDF

[29] Scaling Video Generation for Reasoning: At What Cost? cs.CV | cs.AI | cs.LGPDF

[30] Beyond Binary Preferences: Graded Preference Optimization for Limb-Motion Captioning cs.CVPDF

[31] VLM4Cluster: Benchmarking Deep Clustering In the Era of Vision-Language Pre-training cs.CVPDF

[32] Reprogramming Vision-Language Models via Structured Prompt Reparameterization cs.CVPDF

[33] FocusVTC: Efficient and High-Performance Visual Text Compression with Adaptive Resolution cs.CV | cs.AIPDF

[34] Drag as Evidence: Motion-Grounded Latent Recomposition for Drag-Based Editing cs.CVPDF

[35] When Semantics Matter: Reliability-Aware Semantic-Rhythm Control for Co-Speech Gesture Generation cs.CVPDF

[36] FastVR: Efficient Streaming Video Restoration with One-Step Diffusion cs.CVPDF

[37] DSPO: Diversity-aware Subjective Policy Optimization for Robust Emotional Reasoning cs.CVPDF

[38] Causal-EVC: Breaking Emotional Spurious Causality via Spatiotemporal Grounding and Counterfactual Intervention cs.CVPDF

[39] Dual-Mode Low-Rank Learner with Bridge-Prototype Ensemble for Vision-Language Class-Incremental Learning cs.CV | cs.AIPDF

[40] Decoding Affective Nuances: Enhancing MLLMs via Hierarchical Emotion Reasoning and Contrastive Discriminative Pruning cs.CVPDF

[41] Seeing What Should Be Heard: Diagnosing and Repairing Cross-Modal Shortcuts in Omni-Modal LLMs cs.CV | cs.CL | cs.LGPDF

[42] EGSD: Event-Grounded Self-Distillation for Streaming Video Understanding cs.CVPDF

[43] CurvSpec: Adaptive Multi-Curvature Learning for Partial Relevant Video Retrieval cs.CVPDF

[44] MeteoVerse: Unified Weather-Controllable Video World Model cs.CVPDF

[45] Learning via Self-Consistency for Diffusion-based Video Reasoning cs.CVPDF

[46] Motion Concept Unlearning in Video Diffusion Models cs.CVPDF

[47] On-Policy Visual Evidence Distillation cs.CV | cs.AI | cs.CLPDF

[48] GlassFormer: Learning Real-time Glass Segmentation using Radar-Depth Fusion cs.CV | cs.ROPDF

[49] RoXDrive: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving via Action-Faithful Rollouts cs.CVPDF

[50] S4VY: Segment Anything in Feed-Forward 4D Visual Geometry cs.CVPDF

[51] ProGuT: Label-Efficient Panoptic Segmentation for Forest Scenes cs.CVPDF

[52] Less Supervision, Better Generalization: Weakly Supervised Fake Region Localization in Diffusion-Edited Images cs.CVPDF

[53] DiffReID: Discriminative Diffusion Model for Object Re-Identification cs.CVPDF

[54] SafeVantage: Vantage-Aware Memory for Reliable Embodied Decisions cs.CVPDF

[55] SFE-VGGT: Source-Free VGGT Distillation for Event-Based Monocular Depth Estimation cs.CVPDF

[56] Representation Dynamics Reveal Semantic Saliency and Similarity for Visual Token Pruning in MLLMs cs.CVPDF

[57] Seg3DParts: Segmentation-Grounded Controllable Part-Level 3D Generation cs.CVPDF

[58] WeLike2Party! In-Context Motion Transfer for Multi-Human Image Animation cs.CV | cs.AIPDF

[59] DispFlow-GS: Displacement Flow Supervision with Motion Disentangling for Monocular Deformable 3D Gaussian Splatting cs.CVPDF

[60] Salt++: Context-Aligned Post-Training for Few-Step Streaming Multimodal Generation cs.CV | cs.AI | eess.IVPDF

[61] Parameterized Stripe Attention for Efficient Video Generation cs.CV | cs.AIPDF

[62] Beyond Readability: Evaluating Task Information Recoverability cs.CVPDF

[63] Visual Parallel Search: Learning to Search High-Resolution Images with Parallel Tile Inspection and Adaptive Zoom cs.CV | cs.AIPDF

[64] VesselBench-800K: A Large-scale Perception Benchmark for Multimodal Vessel Detection, Counting, and Density Estimation cs.CVPDF

[65] World2Motion: Turning Video World Models into 3D Human Motion Generators cs.CVPDF

[66] RBF-GNN: Rational Basis Functions for Pseudo-Coordinate based Graph Convolutions cs.CVPDF

[67] MotionInsight: Diagnosing Object Motion Deficiencies in Generated Videos cs.CV | cs.AIPDF

[68] GleanVID: Complementary Token Selection for Efficient Video Large Language Models cs.CV | cs.LGPDF

[69] Speed in the Blind Spot: An Interpretability Analysis of Dynamic Perception in VLMs for Autonomous Driving cs.CVPDF

[70] Spatial-OPSD: Self-Improving Spatial Reasoning via Label-Free Self-Distillation cs.CVPDF

[71] OmniRoute: Mapping Temporal Semantic Evidence to Audio-Visual Token Budgets for Efficient Omnimodal Large Language Models cs.CVPDF

[72] Real2Gym: Building Gyms from Videos, Bringing Skills to Robots cs.CVPDF

[73] LDM-is-AE: Latent Diffusion Model is an Auto-Encoder for End-to-End Image Generation cs.CVPDF

[74] Why MLLMs Struggle to Count: Overcoming Individuation and Aggregation Bottlenecks with ConvStack cs.CVPDF

[75] Waypoint-1.5: A Real-Time Video World Model for Consumer Hardware cs.CV | cs.HCPDF

[76] Task-Oriented Visual Feature Compression via Residual Vector Quantization for Device-Edge Multimodal Inference cs.CVPDF

[77] EviViT: Evidence-Adaptive Vision Transformers for Fine-Grained Perception cs.CVPDF

[78] HaPRL: Human-Anchored Process Reinforcement Learning for Visual Search Agent cs.CVPDF