cs.CL [Total: 21]
cs.CV [Total: 68]
cs.AI [Total: 5]
cs.LG [Total: 5]
cs.RO [Total: 4]
cs.SD [Total: 2]
cs.SE [Total: 1]
eess.IV [Total: 4]

cs.CL [Back]

[1] Lighthouse: On Building a Trustworthy Digital Assistant for Exploring History with Civil Rights Archives cs.CL | cs.AIPDF

John A. Aydin, Angelica Williams, Steven Dindl, Nathan Saunders, Kate Boyd

TL;DR: 论文提出Lighthouse，一个面向美国民权时期历史档案探索的可信数字助手/聊天机器人。系统采用神经符号方法，将LLM的神经学习与基于规则的符号求解结合，在保证答案可信和降低用户潜在伤害的同时扩大覆盖范围。它接入超过1.8万条原始资料，并用客观事实题和由专家聚合形成“金标准”的主观问答对进行评估。

Details

Motivation: 利用AI和多模态历史档案帮助用户有效研究历史，但生成式助手在历史档案场景中需要解决内容可信性和对用户潜在伤害的问题，因此需要构建可信的数字助手。

Result: 论文在可接受单一答案的事实客观题以及主观题上评估Lighthouse；主观题通过聚合独立专家答案构建“金标准”数据集。系统可访问超过18k条原始资料且易于扩展，但摘要未给出具体定量指标或SOTA比较。

Insight: 创新点在于面向可信历史档案助手的设计原则：强调内容可信性并建模/降低生成答案对用户的潜在伤害；方法上采用LLM与规则符号求解结合的神经符号框架，以在答案风险与覆盖泛化之间取得平衡。

Abstract: Artificial intelligence (AI), as a decision-support aid, provides new tools for studying history effectively using multimodal data, including text, images, and audio. We present, to our knowledge, the first digital assistant named Lighthouse–commonly called a chatbot–for exploring the Civil Rights period of the United States using data from South Carolina. From an AI perspective, the system’s novelty lies in its guiding design principles of (a) focusing on trustworthiness of content and (b) by modeling and reducing potential harm to users from generated answers. It implements these with a neuro-symbolic approach that combines neural learning through a large language model (LLM) with rule-based symbolic solving, balancing risk of answers with generality of coverage. We evaluate Lighthouse with both factual objective questions that admit a single acceptable answer as well as subjective questions. For the latter, a dataset of subjective question-answer pairs (QAs) is created by discovering a “gold standard” by aggregating answers from independent experts. Lighthouse has access to over 18k primary sources from a range of collections and can be easily extended.

[2] FusionEval: Benchmarking Model Fusion for Large Language Models cs.CL | cs.AIPDF

Shuo Cai, Yanggan Gu, Yuhang Liu, Su Lu, Yuanyi Wang

TL;DR: 论文提出 FusionEval，一个用于大语言模型模型融合的基准测试，覆盖长链推理和工具使用等能力。它构建受控专家池，整合有训练和无训练的融合方法，并在统一评测套件下比较能力保留与计算成本。实验表明无训练融合能保留推理但显著损害工具使用，而 MOPD 能保留工具使用并更均衡，但需要额外数据、推理和训练开销。

Details

Motivation: 现有模型融合评测对长链思维推理和工具使用覆盖不足，且平均性能可能掩盖单个专家能力的大幅损失。论文希望系统评估融合后专家能力是否保留以及所需计算成本。

Result: FusionEval 基于 Qwen3 和 Llama 构建数学、代码、科学、指令遵循和工具使用专家池，在八个公开基准上报告归一化平均/最差性能、运行时间和峰值 GPU 显存。实验发现无训练方法保留推理但显著降低工具使用；MOPD 保留工具使用并取得更均衡能力，但需额外数据准备、推理和训练。

Insight: 创新点在于受控专家池、统一融合方法与评测流程，以及同时衡量能力保留和计算成本。其可借鉴之处是使用归一化平均和最差性能揭示平均指标掩盖的能力损失，并系统比较有训练与无训练融合方法的权衡。

Abstract: Model fusion aims to integrate the capabilities of multi-expert models into a single model. Existing evaluations offer limited coverage of reasoning with long chains of thought and tool use, while strong average performance can mask substantial losses in individual expert capabilities. We introduce FusionEval, a benchmark that provides controlled expert pools, integrates fusion methods with and without additional training, and offers a unified evaluation suite. The pools contain domain experts independently trained from Qwen3 and Llama base models across math, code, science, instruction following, and tool use. The suite evaluates capability retention and computational cost on eight public benchmarks under comparable data access rules, reporting normalized average and worst performance, runtime, and peak GPU memory usage. Extensive experiments show that training-free methods preserve reasoning performance but substantially degrade tool use at both model scales. In contrast, MOPD preserves tool use and achieves a more balanced capability profile, but requires additional data preparation, inference, and training. Jointly retaining reasoning, instruction following, and tool use at low computational cost remains a challenge. We hope FusionEval supports future research on efficient model fusion that jointly preserves expert capabilities. Our code is available at https://github.com/Baicaihaochi/FusionEval_Benchmark.

[3] Consistency Is Not Validity: Self-Rewarded Moral Argumentation Makes Language Models Decisive but Pliable cs.CL | cs.AI | cs.LGPDF

Marius De Kuthy Meurers, Aaron Parisi, Bernd Bohnet

TL;DR: 论文提出用自我裁判的论证质量锦标赛对LLM进行强化学习训练，从经典小说中抽取并用逻辑求解器验证真实道德困境，在MoralChoice、ELEPHANT和MORABLES上评估。结果显示模型变得更果断且对选项顺序更不敏感，但谄媚和重新框定仍损害道德判断有效性；作者据此强调一致性不等于有效性。

Details

Motivation: 论文希望通奖励“好论证”来提升LLM的道德判断，基于“论证得好更可能判断得好”的假设；核心问题是可优化的一致性/稳定性是否真的代表有效的道德判断。

Result: 自评判模型对论证质量的评分与人类标注者相近；策略在三个外部基准上更果断且更抗选项顺序变化，但sycophancy和reframing仍是有效性挑战。两个消融显示合成困境和无masked thinking可匹配重排增益，但语域迁移更弱、谄媚更高。

Insight: 创新点在于从经典小说构造真实道德困境并用逻辑求解器验证冲突，以及用自我裁判的成对论证质量奖励进行RL训练。客观看，论文揭示了“可优化的一致性指标”未必等于“道德有效性”，对自我奖励/自判训练在道德推理中的可靠性与可塑性提出警示。

Abstract: LLMs are increasingly consulted for moral judgment. We aim to improve performance on such tasks by rewarding good arguments, on the premise that arguing well increases the likelihood of judging well. We train a reinforcement learning policy initialized from a state-of-the-art language model in a self-judged tournament with masked thinking: the generated arguments are judged pairwise on argument quality by the policy itself. As an empirical basis, we extract authentic dilemmas from classic novels, using a logic solver to verify that each constitutes a conflict. We show that our judge rates argument quality like human annotators, and the policy then adopts the language register that it rewards. Testing on the external benchmarks MoralChoice, ELEPHANT, and MORABLES, we find that the policy becomes more decisive and invariant to the order in which options are presented, but sycophancy and reframing remain a challenge to validity. Two ablation studies with synthetic dilemmas and unmasked thinking match the reordering gains, but show a weaker register shift and higher sycophancy. We conclude that consistency is not validity: consistency under reordering can be optimized without improving the moral judgment it is meant to signal.

[4] Self-Assessment Reinforcement Learning for Search-State Awareness in Retrieval-Augmented Large Language Models cs.CL | cs.AIPDF

Taewhoo Lee, Minju Song, Jungwoo Park, Chanwoong Yoon, Junha Jung

TL;DR: 论文提出自评估强化学习框架 SARL，让检索增强大语言模型在训练中学会感知自身搜索状态，即已获得哪些信息、还缺哪些信息，从而减少幻觉推理和过早终止搜索。SARL 通过双重奖励设计鼓励有效且高效的搜索轨迹，且测试时无需额外评估模块。

Details

Motivation: 现有搜索智能体缺乏搜索状态感知能力，难以判断信息是否充分，容易产生幻觉推理或过早结束搜索；论文希望在不增加测试时评估开销的情况下提升搜索行为。

Result: SARL 相比最强基线取得 24.3% 的相对提升，在多跳推理和深度研究基准上分别平均提升 35.9% 和 27.9%，并超过依赖辅助监督或多智能体协调的方法；作为独立强化学习框架且无推理时脚手架，还能在更长搜索视野下保持训练稳定。

Insight: 创新点在于用强化学习让模型内化搜索状态自评估能力，并通过双重奖励同时利用评估的纠错效果、惩罚对其的过度依赖，从而兼顾搜索有效性与效率。

Abstract: Recent advances in search agents have enabled large language models to tackle complex tasks by retrieving and synthesizing external information. However, existing search agents often lack search-state awareness, i.e., the ability to recognize what information has been obtained and what remains missing, leading to hallucinated reasoning and prematurely terminated search. To address this limitation, we propose Self-Assessment Reinforcement Learning (SARL), a reinforcement learning framework in which models learn to recognize their own search state during training and improve their search behavior accordingly, without requiring additional assessment at test time. SARL achieves this through a dual reward design that captures the corrective effect of each assessment while penalizing excessive reliance on it, jointly encouraging effective and efficient search trajectories. Under this setup, SARL achieves a 24.3% relative improvement over the strongest baseline, with average gains of 35.9% on multi-hop reasoning and 27.9% on deep research benchmarks against the strongest per-benchmark baselines. SARL further surpasses methods built on auxiliary supervision or multi-agent coordination, despite operating as a standalone RL framework with no inference-time scaffolding. Beyond final performance, SARL maintains training stability under extended search horizons, a regime where existing methods consistently degrade, enabling sustained gains as task complexity grows.

[5] Behind Aggregate Scores: What Systematically Varying Arena-Hard’s LLM Judge Reveals cs.CL | cs.AI | cs.CYPDF

Louise Yorke

TL;DR: 论文研究LLM judge（GPT-4.1在Arena-Hard v2.0上）在重复运行和改变指令（criteria、输出格式、任务框架）时，聚合分数稳定但个体判决（win/tie/loss）变化显著。发现默认设置重跑聚合分数仅变约1分，却有21.2%的prompt判决改变；某些指令变化导致判决改变比例增加28.2或6.7个百分点，而聚合分数可能大降或几乎不变。结论是稳定聚合分数不能保证判决可重复，建议报告prompt级一致性。

Details

Motivation: LLM judges被用于判断答案优劣，但需了解其判决的可重复性和对指令的敏感性；现有评估往往只关注聚合分数，可能掩盖个体判决的不稳定。

Result: 实验表明：默认设置重跑聚合分数变化约1分，但21.2%的prompt判决改变；某些指令变化下判决改变比例比不变重跑平均多28.2个百分点（聚合分数降4.09分）或6.7个百分点（聚合分数几乎不变）。

Insight: 创新点在于系统性地改变judge的指令和输出格式，揭示聚合分数掩盖个体判决不稳定性；客观分析：强调评估应报告prompt级一致性，对使用LLM judge作为偏好标签或评估个体响应的研究有重要警示意义。

Abstract: LLM judges are used to determine which of two answers is better. Trusting these verdicts requires knowing how repeatable they are and how much they depend on the judge’s instructions. We show that similar aggregate scores can conceal substantial changes in individual verdicts, both when the same setup is repeated and when the instructions change. Using 500 Arena-Hard v2.0 prompts with two models’ answers held fixed, we run GPT-4.1 as the judge under all eight combinations of criteria, output format and task framing, and repeat every setup unchanged at temperature zero on a pinned model version. Rerunning the default setup moves the aggregate score by about one point but changes the win/tie/loss outcome on 21.2% of prompts. Instruction changes introduce further sensitivity: in one setup, which asks for the verdict without reasoning and also drops several other instructions, outcomes change on 28.2 percentage points more prompts than the two setups’ unchanged reruns do on average, and the aggregate score falls by 4.09 points. In another, where the criteria prioritise accuracy, outcomes change on 6.7 percentage points more prompts by the same comparison, while the aggregate score barely moves. A stable aggregate score therefore does not establish that individual verdicts are repeatable. This distinction matters when those verdicts are used as preference labels or to assess individual responses. Evaluations that rely on individual verdicts should repeat each setup they use and report both aggregate scores and prompt-level agreement.

[6] Spectral Smoothing and Spiking Perturbations Reconfigure Capability Profiles and Selective Task Performance in Large Language Models cs.CL | cs.AIPDF

Qince Luan, Jianhui Xu, Yang Xu

TL;DR: 论文研究对大型语言模型前馈层中 up_proj 和 down_proj 矩阵的奇异值谱进行事后平滑与尖峰扰动，在五个开放 Llama 和 Qwen 指令模型上评估其对能力子任务、推理、真实性、幻觉、安全和人格探针的影响，并发现扰动引发结构化而非随机的性能变化，可作为探测能力组织与权衡的轻量干预。

Details

Motivation: 奇异值谱编码了 Transformer 前馈投影中的重要结构，但其对下游行为的因果作用仍缺乏探索；论文希望通过对奇异值谱进行可控扰动，理解谱结构如何影响模型能力分布与任务表现。

Result: 在五个开放 Llama/Qwen 指令模型及多类评测上，大正扰动更可靠地抑制语言理解和演绎推理表现，中等负扰动有时可改善事实知识或任务局部性能；敏感性随模型家族、评测模块和逐层谱不等式显著变化，未宣称达到 SOTA。

Insight: 创新点在于提出保留奇异向量和奇异值几何均值的事后谱平滑/尖峰干预，从而分离奇异值分散度的影响；并用事实知识、语言理解、演绎推理三维能力基解释性能变化，揭示了模型能力组织的结构化敏感性。

Abstract: Singular-value spectra encode important structure in transformer feed-forward projections, yet their causal role in downstream behavior remains underexplored. We study post-hoc spectral smoothing and spiking interventions on the up_proj and down_proj matrices. Positive perturbation stretches deviations from the geometric mean, making large singular values larger and small ones smaller (spiking), and vice versa. The intervention preserves singular vectors and the geometric mean of the singular values. Across five open Llama and Qwen instruction models, we evaluate the perturbed systems on capability-labeled subtasks, standard and chain-of-thought reasoning benchmarks, judge-scored truthfulness, hallucination, and safety tasks, and TRAIT personality probes. To interpret score shifts, we fit responses in a three-dimensional capability basis consisting of factual knowledge, language understanding, and deductive reasoning. The induced changes suggest structured rather than purely random sensitivity: large positive perturbations more reliably suppress language and deductive responses, moderate negative perturbations can sometimes improve factual or task-local performance on the evaluated grids, and sensitivity varies strongly by model family, benchmark module, and underlying layerwise spectral inequality. These results position spectral perturbation as a lightweight intervention for probing how capabilities are organized and traded off in large language models.

[7] From Clinical Judgment to Checkable Rules: Clinician-Led Guardrails for Mental-Health Chatbots cs.CL | cs.AI | cs.CYPDF

Chenhan Lyu, Kit Wislocki, Sarah Elizabeth Stoeckl, Maitreyi Sinha, Amir Rahmani

TL;DR: 论文提出一种由临床医生主导的可检查规则护栏：两名临床医生定义、最终确定并裁决四条安全检查规则，规则库经形式化一致性验证后作为LLM输出的防护层。每个判决可由命名规则和16个事实重算。作者将其与匹配模型、版本、量化、规则文本和解码方式的LLM评判者对比，分析分歧可定位性及测量成本，强调贡献不是更安全的判决，而是可检查、可追溯的护栏。

Details

Motivation: 心理健康场景中LLM安全失败评估往往难以检查和解释，常见提示工程、LLM作为评判者和宪法AI方法缺乏可审计性；论文希望把临床判断转化为形式化、可裁决、可重算的规则护栏。

Result: 在与模型、版本、量化、规则文本和解码匹配但缺少验证层事实计算的LLM评判者对比中，最大分歧为满足规则类：Llama-3.3-70B评判者在65条已含危机资源的回复中对55条（84.6%）仍要求危机资源，而事实提取器在55条中54条识别到资源存在，验证层按构造不会提出该要求；论文也同等强调评判者在唯一参考评分检查上误报危机警报更少。规范修订使过度拒答基准留出边界提示的阻断类结果从6.0%提升到16.0%，主要来自事实提取变化。

Insight: 创新点是将临床医生主导的规则库形式化验证并作为可检查护栏，使每个判决可追溯到命名规则和16个事实；客观看，其价值在于将同评判者的分歧局部化到规则和事实、将测量成本溯源到命名事实和不确定性默认，而非宣称更安全。

Abstract: Large language models (LLMs) deployed in mental-health contexts can be evaluated for safety failures, but the resulting judgments are often difficult to inspect or explain. Common approaches to improving safety rely on prompt engineering, LLM-as-a-Judge, and Constitutional AI. In this work, we describe a clinician-led alternative in which two clinicians define, finalize, and adjudicate the four safety checks of a rule base, which is formally verified for internal consistency and applied as a guardrail over LLM outputs. Here and in the title, clinician-led refers to this rule base, not to the fact extractor or the uncertainty default, which are ours. Every verdict of the resulting verification layer is recomputable from a named rule and sixteen facts. Against an LLM judge matched in model, revision, quantization, rule text, and decoding but without the layer’s computed 988/911 fact, the largest disagreement class, the satisfied-rule class, is structured: the Llama-3.3-70B judge, on its primary prompt, demands a crisis resource from 55 of the 65 replies (84.6%) that already contain one, while the same model, asked as a fact extractor, reports a recognized resource present on 54 of the 55. The verification layer never raises that demand, by construction. We report with equal prominence where this judge is stronger: fewer false crisis alarms on the one reference-scored check. We also report the measured cost (a specification revision raised block-class outcomes on held-out boundary prompts of an over-refusal benchmark from 6.0% to 16.0%, chiefly by changing fact extraction), the clinicians’ decision to keep our uncertainty default, and the analysis of the result. The contribution is not a safer verdict. It is a guardrail whose disagreements with the judge localize to named rules and facts and whose measured costs trace to named facts and the uncertainty default.

[8] ULME: Predictive Coupling for Mutually Reinforcing Embedding and Language Modeling cs.CL | cs.AIPDF

Yang Liu, Zhiqiang Yuan, Shu-Dong Huang, Jiancheng Lv, Liwei Yang

TL;DR: 论文提出 ULME，一个面向因果解码器式 LLM 的后训练框架，用「预测耦合」把文本嵌入与语言建模真正统一起来。具体做法是在 query 与 answer 之间插入一串可学习 slot，从两个指定 slot 分别读出语义嵌入和推理嵌入，并在共享主干上同时训练三个目标：对比损失把语义 slot 对齐到文档表示、语言建模损失监督答案生成、带 stop-gradient 的预测损失把推理 slot 对齐到答案的池化摘要，从而让生成反向塑造嵌入所编码的内容。

Details

Motivation: 嵌入与生成通常由两个目标不兼容的专用模型分别提供，在 RAG 和 agent 系统中并排部署；已有的统一模型（如 GritLM）虽共享参数，但只是在两种模式间切换注意力机制，或把两个目标并行优化、彼此从不互相监督，因此只是「共存」而非「相互塑造」。

Result: 在 Qwen2.5 与 Qwen3 两个模型家族、0.5B–8B 共八个规模上，单个 ULME checkpoint 相比最强开源基线在 MTEB 上提升 3.3–4.1 分，同时改善 BRIGHT 与 AdvBench-IR，并在 MMLU、GSM8K、MMLU-Pro、HumanEval 上与仅做生成微调的模型持平。

Insight: 摘要宣称的创新点在于以可学习 slot 链承载两种嵌入，并用 stop-gradient 预测损失建立生成对嵌入的单向监督，使三项损失共享同一因果主干；从客观角度看，其价值在于把「统一的本质是能力互相塑造而非参数共享」作为设计原则，并证明单一 checkpoint 能同时提升检索嵌入且不牺牲生成能力。

Abstract: Text embedding and text generation are two of the most useful capabilities of modern large language models, yet they are typically provided by separate specialized models (an embedder and a generator) trained with incompatible objectives and deployed side by side in retrieval-augmented generation and agent systems. Recent unified embedding-generation models such as GritLM share parameters between the two capabilities, but they either switch attention regimes between modes or optimize embedding and language modeling as parallel objectives that never supervise each other. We argue that these designs are only partially unified: the two capabilities coexist on the same weights, but do not shape each other. We propose ULME, a post-training framework that unifies embedding and language modeling on a causal decoder-only LLM through predictive coupling. ULME inserts a chain of learnable slots between the query and the answer, reads a semantic embedding and a reasoning embedding from two designated slots, and trains three objectives on a shared causal backbone: a contrastive loss anchoring the semantic slot to document representations, a language-modeling loss supervising answer generation, and a stop-gradient predictive loss that aligns the reasoning slot with a pooled summary of the answer, so that generation actively shapes what the embedding encodes. Across two model families (Qwen2.5, Qwen3) and eight scales spanning 0.5B-8B parameters, a single ULME checkpoint improves MTEB by 3.3-4.1 points over the strongest open-source baseline, tightens BRIGHT and AdvBench-IR simultaneously, and remains on par with a generation-only fine-tune on standard accuracy benchmarks (MMLU, GSM8K, MMLU-Pro, HumanEval), demonstrating that genuine unification yields mutually reinforcing rather than merely coexisting capabilities.

[9] Invisibilization as a Source of Probability Bias in Language Models cs.CL | cs.CYPDF

Christian Arnold, Ernesto Calvo, Guadalupe Gonzalez, Sebastián Vallejo Vera, Tiago Ventura

TL;DR: 论文研究LLM在涉及种族等受保护属性时因对齐而拒绝或回避评估信息，设计双瓮博弈测量这种“信息不可见化”造成的概率偏差。通过随机化报告者描述，让OpenAI和Anthropic模型决定是否信任不可验证报告，发现模型会按描述而非证据拒绝评估，并揭示审计盲区和缓解方式。

Details

Motivation: 对齐模型避免用种族、民族、性别等属性影响回答，可能在需要判断信息可信度时过度拒绝有价值信息；论文关注这种拒绝信息的成本及其隐藏偏差。

Result: 在6600次会话中，模型对Black描述报告者的拒绝评估率最高达46%，white描述为9%；但评估时Black略获更多信任。偏差随推理努力增加、按词语而非类别触发，警告指令反而加剧；采用基于多轮揭示真值的评分选项后，4/5配置末轮种族信任差距低于0.02。

Insight: 创新在于把“不可见化”操作化为拒绝/不评估率偏差，并指出平均信任或仅评估样本审计会漏检；发现词面触发和警告反效果。客观看，双瓮精确贝叶斯框架与重复轮次评分提供了可复现的偏差测量和缓解思路。

Abstract: Language models are aligned to avoid using protected attributes such as race, ethnicity, and gender to inform their responses. Therefore, they increasingly refuse or hedge when such attributes appear in a prompt. However, if a system must decide how much to learn from what people tell it, it is costly to refuse valuable information because of its source. In this article, we measure the cost of declining information in a two-urn game with an exact Bayesian answer. We present leading LLM models from two developers (OpenAI and Anthropic) with a prompt that requires them to decide how far to trust an unverifiable report from a participant whose description we randomize across ten configurations (6,600 sessions). We show that leading models often decline to assess the reporter, treating information as worthless at rates set by the description rather than by the evidence. At its widest, models decline a Black-described reporter 46% of the time and a white-described reporter 9%; yet, when they do assess the information, the Black-described reporter receives slightly more trust. The bias is invisible to audits that compare average trust or look only at assessed cases. It grows with reasoning effort, follows words rather than categories (models invoke race for “Black” but not for its Spanish translation negro, and never for “Indigenous”), and worsens under warning instructions a practitioner would try first. We also present a scoring option that reduces the refusal rate bias. When a model’s trust is scored against revealed truth over repeated rounds, it credits identical records equally. In four of five configurations from Anthropic and OpenAI, the racial gap in trust falls below 0.02 by the last rounds.

Zhaolu Kang, Yidi Wang, Yile Li, Siqi Zeng, Yingjie He

TL;DR: 论文提出 TRIAD，一个面向全模态大语言模型的多模态歧义消解基准。每个样本由文本-图像-音频三元组、问题和选项构成，设计为完整三元组可确定唯一答案，而移除任一模态后一致选项不再唯一。TRIAD 包含 86 个场景组、327 条手写查询和 18 类歧义，并评测 14 个全模态系统，发现模型在场景级消歧上仍显著落后人类。

Details

Motivation: 自然交流中的文本、图像和声音常单独不充分，需跨模态互补才能消除歧义；现有基准多测试已确定输入上的感知、识别或推理，难以评估全模态模型能否解决分布于文本、视觉和音频中的歧义。

Result: 在 TRIAD 上评测 14 个全模态系统，最佳模型在场景组准确率上仍落后人类超过 20 个百分点；移除答案选项后差距进一步扩大，模型组准确率降至个位数而人类仍稳健。留一法和转写/描述反事实分析还表明，当前系统在文本和图像存在时常低估音频线索。

Insight: 创新点在于将全模态评测从识别转向消歧，并以“完整三元组唯一、移除任一模态不唯一”的方式强制模型整合跨模态互补信息；同时提供歧义分类体系、基准和诊断协议，用于场景级消歧能力评估。

Abstract: Ambiguity is a central feature of natural communication: a sentence, image, or sound may remain underspecified until evidence from the other channels is considered. Yet current multimodal benchmarks mostly test perception, recognition, or reasoning over already determinate inputs, leaving unclear whether omni-modal large language models can resolve ambiguity distributed across text, vision, and audio. We introduce TRIAD, a benchmark for omni-modal ambiguity resolution. Each item is a text–image–audio triplet paired with a question and a set of answer options, and is constructed so that the full triplet determines a unique answer while removing any one modality makes the set of consistent options non-unique. TRIAD contains $327$ hand-written queries in $86$ scene groups and covers $18$ ambiguity categories across the three modalities. Evaluating $14$ omni-modal systems reveals that strong item-level performance does not translate into robust scene-level disambiguation: the best model still trails humans by over $20$ points in group accuracy. Removing answer options widens the gap further, driving model group accuracy to the single digits while humans remain robust. Leave-one-out and transcript/description counterfactuals further show that current systems often underuse audio cues when text and image are present. TRIAD reframes omni-modal evaluation around disambiguation rather than recognition, and provides a taxonomy, benchmark, and diagnostic protocol for measuring this capability.

[11] From Perpetrators’ Digital Footprints to Behavioral Communities: An Ontology-Driven Framework for Bengali Cyberbullying Analysis— FULL VERSION cs.CLPDF

Md. Nesarul Hoque, Md Hanif Seddiqui, Miskatul Anwara, Rudra Pratap Deb Nath

TL;DR: 论文提出面向孟加拉语网络欺凌的施害者中心本体驱动框架ODCBNA，整合需求驱动本体设计、知识图谱构建、行为建模、社区分析与影响建模。基于BenCoDiC语料识别出1063名至少发布3条欺凌评论或回复的施害者，构建64维行为与上下文特征向量及加权网络，并分析行为社区和影响传播。

Details

Motivation: 现有网络欺凌研究在内容检测、语义表示和网络分析之间较为割裂，缺乏可解释的施害者表示、多尺度结构建模，以及对行为驱动相似性和影响传播的解释，难以识别持续性施害者、刻画行为社区和评估潜在影响用户。

Result: 在BenCoDiC语料上构建含27,554条边的加权网络，Louvain识别出6个社区（模块度约0.495）和27个子社区；特征贡献分析显示Video、Impact和User Profile组对行为相似性与社区分离贡献最强，IM Greedy在6个社区均取得小幅正向增益，不同种子选择方法的扩散范围相当但种子重叠有限。

Insight: 创新点在于以施害者为中心，将本体设计、知识图谱、行为特征、社区发现和社区约束影响分析统一为可解释框架，并用特征贡献分析解释行为相似性与社区结构，为网络欺凌行为组织和潜在影响模式提供结构化分析路径。

Abstract: Cyberbullying involves behavioral and contextual patterns beyond isolated abusive messages. However, existing approaches remain fragmented across content detection, semantic representation, and network analysis, with limited interpretable perpetrator representations, multi-scale structural modeling, and explanations of behavior-driven similarity and influence propagation. These limitations hinder the identification of persistent perpetrators, characterization of behavioral communities, and assessment of potentially influential users. This paper proposes Ontology-Driven Cyberbullying Behavior and Network Analysis (ODCBNA), a perpetrator-centric framework integrating requirements-driven ontology design, knowledge graph construction, behavioral modeling, community analysis, and influence modeling. Using the Bengali Contextual and Diverse Cyberbullying (BenCoDiC) corpus as a use case, ODCBNA identifies $1{,}063$ users with at least three bullying comments or replies as perpetrators and represents each through a $64$-dimensional behavioral and contextual feature vector. Cosine similarity and $k$-nearest-neighbor modeling produce a weighted network with $27{,}554$ edges. Louvain analysis identifies six communities with a modularity of approximately $0.495$ and $27$ sub-communities. Feature-contribution analysis identifies the Video, Impact, and User Profile groups as the strongest contributors to behavioral similarity and community separation, while contextual dimensions support interpretation. Community-constrained influence analysis shows that IM Greedy achieves small positive gains in all six communities, although different seed-selection methods produce comparable diffusion spreads despite limited seed overlap. These findings provide an interpretable basis for understanding perpetrator behavior, community organization, and potential influence patterns in Bengali online environments.

[12] ReflectPO: Reflection-Guided Policy Optimization for Long-Horizon LLM Agents cs.CLPDF

Rumei Li, Xunliang Cai

TL;DR: 论文提出 ReflectPO，一种面向长程 LLM 智能体的强化学习框架，用于在工具使用和多步环境交互中实现自主错误恢复。该方法包含反思引导采样与树结构信用分配两个核心组件，通过轨迹分支差异、重复访问和父子奖励差异来构造恢复提示并分配动作级信用。实验表明其在两个长程工具使用基准上多数领域优于现有基线，平均整体性能最佳。

Details

Motivation: 长程任务包含顺序且相互依赖的决策，早期执行错误会沿决策链累积并影响后续交互历史与环境状态，因此需要提升智能体识别偏差并自主恢复的能力。

Result: 在两个长程工具使用基准上，ReflectPO 在大多数领域优于多个现有基线，并取得最佳的平均整体性能；摘要未给出具体数值。进一步分析显示其能提升智能体识别自身偏差、从错误状态恢复并继续推进任务完成的能力。

Insight: 论文宣称的创新在于反思引导采样与树结构信用分配：利用交互轨迹中的分支差异和重复访问构造错误恢复提示，并用父子奖励差异和恢复收益进行动作级信用分配。客观来看，将错误恢复信号融入采样与信用分配，有助于缓解长程决策中的误差累积问题，其树结构建模对多步智能体强化学习具有借鉴意义。

Abstract: With external tools, large language models (LLMs) can tackle complex tasks through sustained environmental interaction. LLM-based agents are therefore increasingly used to tackle complex long-horizon tasks in real-world environments. However, these tasks involve sequential, interdependent decisions and changes in environmental states. Early execution errors affect subsequent interaction histories and environmental states, and their effects accumulate along the decision chain. To address this challenge, we propose Reflection-Guided Policy Optimization (ReflectPO), a reinforcement learning framework for autonomous error recovery in long-horizon tasks. ReflectPO comprises two core components. The first is Reflection-Guided Sampling. We use branch differences and repeated visits in interaction trajectories to construct hints for error recovery. The second is Tree-Structured Credit Assignment. We organize sampled trajectories into trees and assign action-level credit using parent–child reward differences and recovery gains. Experiments over two long-horizon tool-use benchmarks show that ReflectPO outperforms multiple existing baselines in most domains. It also achieves the best overall performance on average. Further analyses show that ReflectPO effectively improves agents’ ability to recognize their own deviations, recover from erroneous states, and continue making progress toward task completion.

[13] Verbalizing Activation Differences with Natural Language Autoencoders cs.CLPDF

Andrea Blasi Núñez, Gianluca Barmina, Federico Torrielli, Thiusius Rajeeth Savarimuthu, Peter Schneider-Kamp

TL;DR: 论文研究自然语言自编码器（NLA）能否将大语言模型的内部激活及其差异转化为可读文本，并系统考察其组合结构、对微调模型的泛化、在模型差异分析中恢复微调目标以及专门化能力。实验覆盖三个模型家族的四个模型，结果表明NLA在多数问题上表现积极，是激活空间差异语言化的有前景方向。

Details

Motivation: 激活语言化器旨在把大语言模型的内部表示翻译成人类可读文本，NLA通过自然语言瓶颈重建激活，但现有方法在其训练设置之外的泛化能力尚不清楚。论文关注NLA能否理解组合激活、复用于微调模型，并用于模型差异分析。

Result: 实验显示：组合类比激活的语言化多能命名目标词而少提被减词；NLA可无需重训复用于轻量微调模型，仅最大模型重建质量明显下降；在Taboo模型生物与原始模型的激活差异上，多数能恢复隐藏秘密词，强化学习可进一步改善held-out表现；NLA可在微调模型不生成任何token时恢复微调目标；在涌现错位检测任务上未达SOTA，但优于黑盒基线。

Insight: 论文宣称的创新在于系统评估NLA对组合激活、微调泛化和模型差异分析的泛化能力，并证明其可语言化激活差异、恢复微调目标且可通过强化学习专门化。客观来看，其价值在于将NLA从单模型激活重建推进到模型差分与目标恢复，并揭示信息量最高的token位置可能位于用户提示之前。

Abstract: Activation verbalizers translate the internal representations of large language models into human-readable text. The recently proposed natural language autoencoders (NLAs) learn this by reconstructing activations through a natural language bottleneck. However, it remains unclear how well these methods generalize beyond the setting they were trained for. Here, we study this question for NLAs along four closely related questions: (i) Do NLA verbalizations exhibit compositional structure with respect to their input? (ii) Do NLAs generalize to finetuned versions of the model they were trained for? (iii) Can NLAs recover finetuning objectives in model diffing? (iv) Can NLAs be specialized for model diffing? Across four models from three families, our experiments provide evidence supporting a largely positive answer to all four questions. Specifically, verbalizations of composed analogy activations mostly name the target while rarely mentioning the subtracted word, and an NLA can be reused on lightweight finetunes of its original model without retraining, with reconstruction quality degrading noticeably only for the largest model. Verbalizing activation differences between Taboo model organisms and their original models recovers the concealed secret word for most organisms, and reinforcement learning on activation differences further improves this recovery on held-out organisms. We also find that NLAs can recover the finetuning objective without the finetuned model generating any token, since the most informative token positions precede the user prompt. NLAs do not yet match state-of-the-art model diffing methods on the emergent misalignment detection task, but outperform a blackbox baseline. Our findings suggest that NLAs can meaningfully verbalize differences in activation space, rendering NLAs a promising direction for future work.

[14] RetiJev: Fast and Source-Grounded Ophthalmic Decision Models cs.CL | cs.AIPDF

Bo Liu, Lianyu Wang, Meng Wang, Ching-Yu Cheng, Huazhu Fu

TL;DR: 论文提出面向眼科临床决策的RetiJev模型族，通过直接对候选答案打分而非自回归生成，实现快速、准确且隐私友好的决策。作者构建了来源可追溯的眼科临床决策/诊断与指南分级数据集及评测基准，并训练了0.8B-9B参数模型。

Details

Motivation: 自回归LLM在预定义行动决策中存在显著延迟和计算开销，而眼科临床决策支持需要快速、准确且隐私保护的推理，因此论文探索决策导向的替代方案。

Result: 在指南分级任务上，所有规模RetiJev均比27B LLM和商用Jev 1.13决策模型高10.4-23.0分；RetiJev-9B在临床决策上达到最高准确率，比两个基线高5.4-6.5分，在诊断上与之持平。单GPU回答一个病例仅需158ms，比LLM生成快3.5倍，比推理快470倍。

Insight: 论文宣称的创新点包括构建来源可追溯的眼科语料与评测基准，以及无需自回归解码、直接对候选答案打分的决策模型。客观来看，其核心创新是将临床决策建模为判别式打分/重排而非生成式推理，从而让小模型兼顾速度、准确性与可溯源数据利用。

Abstract: Ophthalmic clinical decision support requires fast, accurate, and privacy-preserving inference, yet autoregressive large language models (LLMs) incur substantial latency and computational overhead for decisions among predefined actions. We therefore investigate a decision-oriented alternative to generative reasoning. Specifically, we first introduce RetiJev-Data, a source-grounded ophthalmology corpus comprising 78,158 clinical-decision and diagnosis questions derived from articles in PubMed Central and a MeSH vocabulary covering 531 eye diseases, and 50,074 guideline-grading questions spanning 11 grading systems, along with a 24,662-question evaluation benchmark. Based on this, we develop RetiJev, a family of 0.8B-9B decision models that directly score candidate answers without autoregressive decoding. Across all model sizes, RetiJev outperforms both a 27B LLM and the commercial Jev 1.13 decision model on guideline grading by 10.4-23.0 points. RetiJev-9B further achieves the highest accuracy on clinical decisions, improving over both baselines by 5.4-6.5 points while matching them on diagnosis. It answers a case in 158 ms on one GPU, 3.5x faster than LLM generation and 470x faster than reasoning. Models are released at https://github.com/Awenbocc/RetiJev.

[15] Chunk Recomputation under Real Document Revisions: Measuring Update Cost and Retrieval Quality of Fixed, Recursive, and Content-Defined Chunking cs.CL | cs.IRPDF

Yuning Bian, Lele Xu, Yifan Yuan, Xinyu Yin, Chengyan Zhan

TL;DR: 论文研究 RAG 文档分块在真实文档修订下的更新成本问题：文档编辑导致分块边界移动，从而迫使未变文本重新嵌入。作者在 120 对中英文维基百科相邻修订上，对比固定大小、递归段落优先和内容定义分块三种无状态分块器，在严格字节级复用和冻结编码器条件下测量重计算成本，并评估检索质量。结果表明更新成本很高且随参数变化，而固定检索预算下不同分块器的证据召回率差异较小。

Details

Motivation: 现有 RAG 分块策略通常在静态文档快照上评估，但知识库文档会持续变化；分块边界移动会造成未修改文本的重复嵌入和更新开销，这一维度未被静态检索评估暴露。

Result: 在 116 篇共同成功样本上，固定大小分块需重计算新版本嵌入输入工作量的 57-68%，递归分块为 38-56%，内容定义分块为 43-54%；在固定 2048 token 检索预算下，同语言内平均证据召回率差异不足两个百分点。成本参数网格显示内容感知分块器排名随块长设置变化，而不重叠固定大小分块在所有长度匹配条件下重计算成本最高。

Insight: 论文的创新点在于用真实修订数据量化分块更新成本，并联合考察检索质量与重计算开销；其客观价值是指出边界漂移是未变文本被重计算的重要来源，主张将更新成本作为与检索质量并列的报告维度。

Abstract: Chunking strategies for retrieval-augmented generation are almost always evaluated on a static snapshot, yet the documents underlying a knowledge base continue to change. Because chunks are embedded independently, a boundary that moves after an edit forces unchanged text to be re-embedded. We measure this update cost on 120 pairs of adjacent Chinese and English Wikipedia revisions for three stateless chunkers (non-overlapping fixed-size, recursive paragraph-first, and content-defined chunking) under a strict byte-identical reuse rule and a frozen dense encoder. On the 116-article common-success pool, with frozen, length-calibrated parameters, fixed-size chunking recomputes 57-68% of its full new-version embedding-input workload, recursive chunking 38-56%, and content-defined chunking 43-54% (it failed on 4 of 62 Chinese articles), while at these parameters mean evidence recall under a fixed 2048-token retrieval budget differs by less than two percentage points within either language. A cost-only parameter grid shows that the ranking of the two content-aware chunkers changes with chunk-length settings, whereas non-overlapping fixed-size chunking has the highest recomputation cost in every length-matched condition. Controlled edits and a verbatim-presence diagnostic support boundary drift as one contributor to recomputation of text already present in the previous version. Update cost is therefore a substantial, parameter-sensitive dimension that static retrieval evaluations do not expose and that should be reported alongside retrieval quality.

[16] Prompt Sensitivity in Tool-Augmented Model Comparisons cs.CL | cs.AI | cs.LGPDF

Mateusz Modrzejewski, Tomasz Radzikowski

TL;DR: 论文研究工具增强语言模型后，模型在问题改写下比较是否更稳定。作者提出匹配协议，将工具调用流程拆分为答案读出、推理、是否调用、工具返回和未调用报告等步骤，并在六个开源模型、五个领域、五种问题表述下评估。结果显示工具在程序可直接回答时可提升准确率，但会加剧许多模型两两比较对表述的敏感性，且与调用频率、接口和指令有关。

Details

Motivation: 现有问答基准通常只用一种措辞提问，而改写问题可能改变两个模型的排名。论文关注工具增强是否能让模型比较更稳定，并指出仅看准确率提升无法回答该问题，因为工具同时改变了从问题到答案的多个环节。

Result: 在六个开源模型、五个领域和五种问题表述下，工具在由程序回答问题时提高准确率，但使许多成对比较的表述敏感性扩大；该扩大与模型调用频率相关，并受工具调用接口和指令影响。在分子领域，原生工具调用接口和第二种工具描述下未检测到敏感性扩大，而工具返回中间结果时敏感性仍存在；摘要未给出具体数值。

Insight: 创新点在于提出匹配协议分离工具增强流程中的各个步骤，从而分析工具对比较稳定性的影响，而非只看准确率。客观看，论文揭示了确定性工具报告并不保证模型比较稳定，并建议在准确率之外报告调用频率和多表述下的比较稳定性。

Abstract: Language models are compared on question-answering benchmarks that pose each question in one wording, and a reworded question can change which of two models scores higher. We ask whether giving the models tools makes such comparisons more stable. An accuracy gain cannot answer this, because tools change several steps between the question and the scored answer at once. We propose a matched protocol that separates those steps: the answer readout, reasoning, the decision to call, the tool’s answer, and the report supplied without a call. We apply it to six open models in five domains under five wordings. We find that tools raise accuracy where a program answers the question and widen the wording sensitivity of many pairwise comparisons. The widening follows how often a model calls, and call frequency follows the wording for some models and not others. It also depends on the interface and the instructions: a native tool-calling interface and a second tool description leave no detected widening in molecules, and sensitivity persists where tools return intermediate results. These findings show that deterministic reports do not guarantee stable comparisons. We recommend reporting call frequency and comparison stability under several wordings beside accuracy.

[17] CoWriter: Guiding Methodological and Synergetic Human-LLM Co-Authoring cs.CL | cs.AI | cs.HCPDF

Shmuel Amar, Benjamin Lieber, Ori Shapira, Debanjana Kar, Sebastian Maximilian Dennerlein

TL;DR: 论文提出 CoWriter，一种主动式智能体协作写作方案，将认知科学中专家人类（合）作者的方法论引入人类-LLM 协同写作全流程，主动引导写作并鼓励用户参与。系统面向长文本、基于来源材料的写作场景，旨在改善传统聊天+画布界面中人类角色被动、审阅疲劳和过度依赖 AI 的问题。

Details

Motivation: 现有 LLM 写作界面常让作者沦为被动监督者，导致审阅疲劳、过度依赖 AI，并在长文本写作中削弱人的主体性；已有系统多只处理孤立写作阶段，缺乏端到端的人机协同写作探索。

Result: 在探索性用户研究中，相比标准 LLM 基线，使用 CoWriter 的作者更能表达自身观点并保持更高文档所有权感，智能体成功交互次数达到 2.2 倍；外部读者评审也认为生成文本在故事性和融入个人独特经历等“人类触感”上更优。

Insight: 创新点在于将成熟的人类写作方法论转化为主动式智能体策略，贯穿整个写作工作流并主动促使用户参与，而非仅优化单一步骤；其可借鉴之处是结合认知科学的人写作实践来设计协同代理，以提升人机协作质量和用户主体性。

Abstract: While LLMs have transformed text generation, standard chat-and-canvas interfaces often reduce human authoring to quite passive supervision, causing review fatigue, over-reliance on AI, and the sidelining of the human role in long-form, source-grounded writing. Prior systems typically address isolated writing phases, leaving comprehensive end-to-end human-LLM collaboration largely unexplored. In this paper, we introduce CoWriter, a proactive agentic scheme that adapts expert human (co-)authoring methodologies from cognitive science to flexibly steer the entire writing workflow while actively encouraging the user to engage. In an exploratory user study, compared to a standard LLM baseline, authors using CoWriter better expressed their perspectives and maintained a higher sense of document ownership, supported quantitatively by $2.2\times$ more successful agent engagements. Furthermore, external reader judges rated the resulting texts higher in “human touch” qualities like storytelling and integrating unique personal experiences. Overall, our results demonstrate that proactively engaging users through established human practices is beneficial for effective LLM-supported co-authoring, providing useful methodologies, techniques and insights for future research on collaborative agents.

[18] Cola2: Co-Adaptive Hierarchical Language Modeling with Continuous Semantic Compression and Balanced Discrete Generation cs.CLPDF

Hongcan Guo, Yian Zhao, Feng Wang, Shen Nie, Guoqiang Wei

TL;DR: 论文提出 Cola2 框架，用于研究连续语义压缩、重建与生成之间的平衡，以及连续潜变量与离散文本生成器的协同适配。方法动态切分语义片段、无辅助损失地控制压缩率，并结合重建与生成监督学习紧凑可预测的潜表示，再用块因果掩码扩散解码器生成文本。实验验证了核心设计，并显示出有竞争力的、未饱和的扩展趋势。

Details

Motivation: 分层语言生成将连续语义建模与离散文本实现分离，但其效果取决于潜空间压缩什么信息，以及离散生成器如何利用这些潜变量。论文旨在解决潜表示学习、重建/生成平衡和连续-离散协同适配的问题。

Result: 在约 2100 亿训练 token 的五个下游任务上验证核心设计；使用 3460.3 亿源 token 时，未收敛 checkpoint 在八个任务上平均 60.10，并在 HellaSwag 和 StoryCloze 上超过 Llama-3-8B，而后者报告的预训练预算超过其 43 倍。

Insight: 创新点包括动态语义片段切分与无辅助损失的负反馈压缩率控制、重建与生成联合监督、带干净/噪声/丢弃潜变量条件的块因果掩码扩散解码器、用无分类器引导平衡重建与自主生成，以及用 stop-gradient 解耦 Flow Matching 先验学习与表示学习；推理时按固定片段布局重编码生成文本以保持潜历史一致。

Abstract: Hierarchical language generation separates continuous semantic modeling from discrete text realization, but its effectiveness depends on what the latent space compresses and how the discrete generator uses it. We propose Cola2, a framework that jointly studies continuous semantic compression, the balance between reconstruction and generation, and continuous-discrete co-adaptation. Cola2 dynamically segments text into semantic spans, controls the compression rate through auxiliary-loss-free negative feedback, and combines reconstruction and generation supervision to learn compact, predictive latent representations. A block-causal masked diffusion decoder is trained with clean, noisy, and dropped latent conditions, while classifier-free guidance adjusts the balance between semantic reconstruction and autonomous generation. Token-prediction gradients update the encoder through the latents, whereas Flow Matching prior learning is decoupled from representation learning by stop-gradient. During inference, generated text is re-encoded under a fixed span layout to keep the latent history consistent with the actual text context. Experiments on five downstream tasks with about 210 billion training tokens validate the core designs and show competitive, unsaturated scaling. With 346.03 billion source tokens, an unconverged checkpoint reaches a 60.10 average across eight tasks and surpasses Llama-3-8B on HellaSwag and StoryCloze despite its reported pretraining budget being over 43 times larger. These results demonstrate the potential of continuous-latent hierarchical language generation.

[19] How Depth Growth and Looping Shape In-Context Learning in Language Models cs.CL | cs.AI | cs.LGPDF

Ferdinand Kapl, Kaitlin Maile, Stefan Bauer, Johannes von Oswald

TL;DR: 论文研究循环结构和深度增长语言模型中参数复用对上下文学习（ICL）的影响，分析其如何影响ICL的获得、保持与机制。作者在最高3B参数的预训练语言模型和受控任务上发现，深度增长与循环模型更依赖上下文信息，更早获得ICL，并在权重中学习固定关联时更强地保持ICL；训练中深度增长会将早期归纳头传播到多层，且这些模型在少样本和后续token预测中更依赖归纳头。

Details

Motivation: 循环和深度增长模型通过跨深度复用计算提升推理能力，已有迹象表明其可能改善上下文学习，但参数复用为何以及如何影响ICL的获得、保持和内在机制尚不清楚。

Result: 在最高3B参数的预训练模型和受控任务上，深度增长/循环模型相比全深度稠密参考模型更依赖上下文、更早习得ICL并更强保持ICL；消融显示其少样本性能和后续token预测更依赖归纳排序注意力头。摘要未给出具体数值或公开benchmark，结论主要为定性和机制性证据。

Insight: 论文宣称的创新点在于系统分析深度增长与循环复用如何塑造ICL的习得、保持及注意力机制，并揭示二者共享一种偏向从上下文学习的归纳偏置；客观上可借鉴之处是将“软参数复用”与归纳头传播/依赖联系起来，为理解ICL机制和设计复用型模型提供证据。

Abstract: Looped and depth-grown language models reuse computations across depth and have attracted interest for their reasoning capabilities. Building on prior suggestive evidence of improved in-context-learning (ICL), we investigate how such reuse favors ICL, including how it shapes the acquisition, retention, and mechanisms of this ability. Our analysis spans pretrained language models up to 3B parameters and controlled tasks. The experiments provide evidence for a shared inductive bias toward learning from context: grown and looped models rely more on information in context and, compared with a full-depth dense reference, acquire ICL earlier in controlled tasks and retain it more strongly when learning fixed associations in the weights is also possible. Tracing attention heads through training reveals that depth growth propagates early induction heads across layers. Ablations show that grown and looped models rely more strongly on induction-ranked heads for both few-shot performance and later-token prediction. Together, these findings provide evidence for a shared inductive bias toward learning from context and suggest that (soft) parameter reuse shapes how this ability develops and which attention mechanisms support it.

[20] Towards User Simulators That Stay On Task And Still Sound Natural cs.CLPDF

Mustafa Omer Gul, Philippe Laban, Tanya Goyal, Jennifer Neville

TL;DR: 论文关注多轮长程交互中LLM助手性能退化的问题，提出用用户模拟器来评估和改进AI助手。现有用户模拟器难以同时做到忠实遵循复杂用户意图和表现得自然。作者提出一种强化学习方法，联合奖励任务意图遵守与行为自然性，并发布UserLM-2.0。

Details

Motivation: 为了在长期多轮任务中可靠评估和改进AI助手，需要既能忠实执行复杂用户意图、又具有真实对话行为的用户模拟器。现有方法存在权衡：提示式助手能遵循复杂意图但行为保真不足，专门模拟器更自然但难以保持复杂意图忠实。

Result: UserLM-2.0生成有效模拟对话的比例达到46.2%，即同时满足任务忠实性和自然性标准；此前最强用户模拟器仅为14.1%，该方法超过其3倍。

Insight: 论文宣称的创新点是联合优化任务意图遵守与行为自然性，并用轻量判别器与用户模拟器交替对抗训练来操作化自然度。客观来看，其可借鉴之处在于将“自然性”转化为可学习的判别信号，并与意图忠实性一起通过RL优化，从而缓解二者之间的权衡。

Abstract: Users increasingly rely on AI systems for extended, multi-turn tasks, yet evidence indicates that LLM-based system performance degrades over prolonged interaction. To evaluate and improve AI assistants in such settings, we need user simulators that faithfully follow complex user intents while exhibiting realistic conversational behavior. Existing approaches manifest a trade-off: prompted assistants can follow complex user intents but lack behavioral fidelity, while specialized user simulators better approximate conversational behavior but struggle to remain faithful to complex intents. In this paper, we propose a reinforcement learning (RL) recipe that jointly rewards task intent adherence and behavioral naturalness. A crucial contribution of the method is operationalizing naturalness with a lightweight discriminator trained in an alternating, adversarial loop with the user simulator. We release the resulting model, UserLM-2.0, which produces valid simulated conversations (i.e., meeting both task-faithfulness and naturalness criteria) 46.2% of the time, over three times the rate of the strongest prior user simulator (14.1%).

[21] Would an LLM Match into a Residency? Uncertainty, Reasoning and Image Use of Open Language Models on the Spanish MIR Examination cs.CL | cs.AI | cs.CY | cs.LGPDF

Miguel Fernandez-de-Retana, Unai Zulaika, Ruben Sanchez-Corcuera, Aritz Bilbao-Jayo, Aitor Almeida

TL;DR: 论文在西班牙MIR住院医师考试上系统评估了4个8–27B开放语言模型，不仅看答题准确率，还考察模型能否识别自己可能出错、医学微调是否改善置信度校准、正确答答案是否基于合理临床推理、以及模型是否真正使用所给图像。作者按考试规则对8个版本共1552道题评分，发现较大模型成绩可进入约前600–3000名并仍有41–43个专科可选，但弃答从不划算、医学微调降低过度自信却牺牲准确率且未改善置信度区分能力，并且相当比例正确答案的排除干扰项理由临床上不合理，移除或替换图像也不影响准确率。

Details

Motivation: 论文指出，仅以医学考试准确率作为临床使用证据不足；若卫生系统要在自有硬件上部署开放模型，还需知道模型是否知道自己可能出错、医学微调是否改善这种能力、正确答答案是否依赖可靠临床推理，以及模型是否真正利用图像信息。

Result: 在西班牙MIR考试1552题上，三个较大模型若全部作答可排名约前600–3000名，并有41–43个专科仍可选；但弃答从不划算，因为其最不自信答案的正确率仍高于25%的盈亏平衡点。医学微调降低了过度自信但牺牲准确率，且未改善置信度对正确/错误答案的区分；13–38%的正确答答案以临床上不合理的理由排除干扰项，且置信度未反映该问题；移除或替换所给上下文图像后准确率不变。

Insight: 论文的创新点在于把临床部署所需的能力拆解为不确定性、推理合理性和图像使用，并用考试规则评分、弃答收益分析和图像反事实扰动来检验开放模型。客观看，其价值主要是揭示了“考试高分”与“可临床部署”之间的差距，尤其是置信度校准、推理依据有效性和多模态输入实际利用方面的隐患。

Abstract: Accuracy on medical examinations is common evidence offered for the clinical use of language models, yet a health system that intends to run an open model on its own hardware needs to know considerably more: whether the model recognizes when it is likely to be wrong, whether medical fine-tuning improves this ability, whether its correct answers rest on sound clinical reasoning, or whether it actually uses the images it is given. We examine these questions for four open models of 8 to 27 billion parameters on the Spanish MIR residency examination, scoring 1,552 questions from eight editions under the exam’s own rules. Answering every question, the three larger models would rank among roughly the first 600 to 3,000 candidates, with 41 to 43 specialties still open to them. Abstention, however, never pays, because even their least confident answers are right more often than the 25% break-even. Medical fine-tuning reduced overconfidence at the cost of accuracy, without improving how well confidence separates right from wrong answers. Moreover, 13-38% of correct answers rejected a distractor for a clinically unsound reason that confidence did not reflect, and removing or swapping the provided contextual images left accuracy unchanged.

cs.CV [Back]

[22] V-JEPA 2 Meets AQA: Phase-Based Scoring and Reference-Based Feedback for Mountain Biking cs.CV | cs.AIPDF

Santhiya Rajan, Savio Cardoz

TL;DR: 论文研究弱标签条件下的山地自行车跳跃动作质量评估（AQA），使用95段现场视频和“好/差”二分类标签。方法用冻结的V-JEPA 2视频特征结合二维姿态与运动测量，按四个启发式阶段汇总，并用小型Transformer预测二分类分数；另设参考层与“好技术”片段比较，提出阶段、身体参数和方向提示。五折片段级交叉验证中完整评分器池化OOF AUROC为0.768、AP为0.812，但作者强调结果仅为探索性，外部探针和反馈可靠性均有限。

Details

Motivation: 论文关注山地自行车跳跃动作质量评估中只有粗粒度弱标签、缺少连续专家评分和阶段/方面级标注的问题，并尝试在有限监督下同时做二分类评分和候选纠正反馈。

Result: 五折片段级交叉验证中完整评分器池化OOF AUROC 0.768、AP 0.812；去除生物力学评分输入后AUROC升至0.809。17片段外部探针上早期检查点二分类准确率0.412、与专家分数Spearman相关-0.135；候选提示相对两位教练精确率0.118、召回率1.0，且方向判断分歧较大。

Insight: 论文宣称的创新点在于将冻结V-JEPA 2特征、启发式阶段汇总和基于参考片段的反馈生成结合到弱标签AQA中。客观看，其价值更偏向于明确该设定下评分与纠正反馈的经验边界和评估需求，而非证明可靠的车手独立评分或纠正反馈。

Abstract: We study mountain-bike action quality assessment (AQA) with weak labels and candidate feedback. The training corpus comprises 95 field-recorded jump clips with binary labels denoting good or poor technique. These are coarse categorical labels, not continuous expert quality scores; phase-boundary and aspect-level annotations are unavailable. Our pipeline summarizes four heuristic phases using frozen V-JEPA 2 video features and two-dimensional pose and motion measurements. A small transformer predicts a binary-label score. A separate reference layer compares phase-specific measurements with those of clips labelled as good technique to propose phase, body-parameter, and direction cues. We detail the measurements, network, and evaluation so that the limits of each output are explicit. In five-fold clip-level cross-validation, the full scorer obtained pooled out-of-fold AUROC 0.768 and average precision 0.812. These are exploratory figures because the reporting fold also selected the neural training epoch and riders were not separated between folds. Removing biomechanical scoring inputs raised AUROC to 0.809. A separate inner-selection run obtained AUROC 0.678, but used about 57 rather than 76 training clips per outer fold, so its difference from the test-selected run does not quantify selection bias. The unsupervised aspect head did not learn distinct factors. On a 17-clip external probe, an earlier checkpoint obtained binary accuracy 0.412 and Spearman association -0.135 with expert scores. Candidate cues had precision 0.118 and recall 1.0 against each of two coaches, with substantial disagreement about direction. These findings do not establish rider-independent scoring or reliable corrective feedback; they identify the empirical work needed to evaluate either use.

[23] EgoHand: Monocular 3D Hand Reconstruction from Egocentric Video cs.CV | cs.LGPDF

Chao-Yeh Chen, Tiffany Chen, Lin Duan, Tianyi Hu, Yu-Hsiang Huang

TL;DR: EgoHand 是一个面向第一视角视频的单目 3D 手部重建模型。它构建了约 1400 万图像的高质量训练语料，结合合成数据、真实场景视频和多相机动捕数据，并以 SOMA-Hand 为参数化表示，通过 Transformer 编码 2D 手部裁剪区域，引入相机线索和辅助 2D 分支提升几何一致性与图像对齐。

Details

Motivation: 第一视角视频中的单目 3D 手部重建面临场景多样、监督质量不足以及几何一致性差等问题，论文旨在构建高质量训练数据并提升模型在域外场景中的泛化能力。

Result: 在 HOT3D、ARCTIC 和 H2O 上做零样本评估，EgoHand 的关节位置、根节点平移和加速度误差低于对比基线，推理延迟约 9.69 ms；盲测用户偏好研究显示其重建手部姿态优于 Apple EgoDex 和 Ropedia Xperience-10M。

Insight: 创新点主要在于大规模高质量混合数据构建、SOMA-Hand 参数化表示与 Transformer 编码结合，以及相机线索和辅助 2D 分支带来的几何一致性与图像对齐提升；同时强调强零样本泛化、低延迟和开放商用许可。

Abstract: We introduce EgoHand, a monocular 3D hand reconstruction model that achieves state-of-the-art performance on egocentric video. Central to our approach is the curation of a high-quality training corpus of 14 million images, combining synthetic data, in-the-wild video, and multi-camera motion capture to balance scene diversity and supervision quality. The model adopts SOMA-Hand as its parametric representation and processes 2D hand crops with a transformer encoder, incorporating camera cues and an auxiliary 2D branch to improve geometric consistency and image alignment. Zero-shot evaluations on HOT3D, ARCTIC, and H2O demonstrate strong out-of-domain generalization, with lower joint-position, root-translation, and acceleration errors than competing baselines at an inference latency of approximately 9.69 ms. A blinded user preference study further shows that participants favor EgoHand’s reconstructed hand poses over those from Apple EgoDex and Ropedia Xperience-10M. We release the EgoHand model and source code under the OpenMDW license, allowing free commercial use of the model.

[24] R4DGS: Referring Segmentation in 4D Gaussian Splatting cs.CVPDF

Bangpu Chen, Yaxuan Li, Shirui Peng, Xiangtian Si, Liuxin Chu

TL;DR: 论文研究动态4D场景中的自然语言指代分割/grounding，提出Referring 4D Gaussian Splatting（R4DGS）任务，并构建带结构化查询标注的R4D-Bench-QA基准。作者提出ReferGaussian统一框架，将4DGS的低层动态高斯转化为持久、语言可寻址的实体，利用MLLM将复杂查询分解为可跟踪物体短语和结构化指代约束，实现无需场景特定语义优化的指代推理。

Details

Motivation: 现有4DGS方法主要用于动态重建与渲染，语义扩展往往依赖场景特定语义优化，且评测查询较简单，难以支持动态场景中多目标、推理密集、零目标或干扰表达的自然语言指代。

Result: 在R4D-Bench-QA和4D LangSplat协议上的实验表明，方法改进了时序目标选择，并在无需场景特定语义训练的情况下取得有竞争力的时空grounding表现；摘要未给出具体定量数值或明确SOTA声明。

Insight: 创新点在于定义R4DGS任务与结构化查询基准，提出EntityBank组织mask支持的高斯实体，并用MLLM分解复杂查询为可跟踪物体短语和结构化指代约束，实现training-free且无需场景特定语义优化的约束感知grounding。客观上看，其价值在于将4DGS表示、语言可寻址实体和复杂查询约束推理结合，提升动态指代场景的泛化与可解释性。

Abstract: Dynamic representations based on 4D Gaussian Splatting (4DGS) have shown strong capabilities in reconstruction and rendering, yet current methods remain limited in supporting complex natural-language referring in dynamic scenes. Existing 4D Gaussian methods are primarily designed for rendering-oriented dynamic modeling, while current semantic extensions often rely on scene-specific semantic optimization or are evaluated on relatively simple query settings. In this paper, we study Referring 4D Gaussian Splatting (R4DGS), a task for grounding temporally varying, multi-target or reasoning-intensive, and zero-target or distractor expressions in dynamic 4D scenes. To support this task, we introduce R4D-Bench-QA, a benchmark with structured query annotations. We further present ReferGaussian, a unified framework built upon 4DGS that converts low-level dynamic Gaussians into persistent, language-addressable entities for training-free referring inference. ReferGaussian organizes mask-supported Gaussian entities in an EntityBank, uses Multimodal Large Language Models (MLLMs) to decompose complex queries into trackable object phrases and structured referring constraints, and performs constraint-aware grounding without scene-specific semantic optimization. Experiments on R4D-Bench-QA and the 4D LangSplat protocol demonstrate improved temporal target selection and competitive spatiotemporal grounding without scene-specific semantic training. Project resources are available at https://chenbangpu.github.io/R4DGS/.

[25] MoRE: Motion-Regularized Encoding for Articulated 3D Generation cs.CVPDF

Guantian Zheng, Shan Jin

TL;DR: 论文提出 MoRE，一种运动正则化的关节物体 3D 生成方法，从单张图像同时恢复物体外观与关节机制。方法用轴分类和两个轴正交 Plücker 矩系数表示坐标对齐的关节线，并通过解码关节假设驱动 3D 锚点运动，与标注关节运动比较来正则化表示学习。推理时可对单组 rest-pose 高斯进行铰接，而无需为每个状态重新生成物体。

Details

Motivation: 从单图生成铰接物体不仅需要重建外观，还需恢复控制部件运动的关节机制；仅紧凑编码关节属性不足以保证重建结果保持正确运动，因此需要将关节属性学习与其诱导的运动一致性耦合起来。

Result: 在 PartNet-Mobility 和 ACD 上，MoRE 相比对比基线取得更强的部件包围框一致性、铰接状态几何与视觉相似度；关节预测相对 ArtiLatent 降低了轴、原点和范围误差，消融实验支持关节线编码及运动监督的有效性。

Insight: 创新点在于用坐标对齐关节线及 Plücker 矩系数消除沿轴原点选择的歧义，并通过可微期望将关节类型/轴方向重建与锚点运动监督耦合；客观看，其可借鉴之处是把属性表示学习与下游运动一致性联动，并用单组 rest-pose 高斯支持多状态铰接推理。

Abstract: Generating an articulated object from a single image requires recovering both its appearance and the mechanism governing its parts. A compact encoding of joint attributes does not by itself ensure that reconstruction preserves the resulting motion. We present MoRE, Motion-Regularized Encoding for articulated 3D generation. MoRE represents coordinate-aligned joint lines with axis classification and two axis-orthogonal Plücker moment coefficients, removing the arbitrary choice of an origin along the axis. During representation learning, decoded joint hypotheses are used to move corresponding three-dimensional anchors, and their motions are compared with those of annotated joints. A differentiable expectation over joint types and axis directions couples attribute reconstruction to the motion it induces. An image-conditioned flow learns the resulting functional latents, while a multi-state-adapted Gaussian model supplies appearance. At inference, recovered part assignments and joints articulate a single set of rest-pose Gaussians without regenerating the object for each state. Experiments on PartNet-Mobility and ACD show stronger part-box agreement, articulated-state geometry and visual similarity than the compared baselines, while joint prediction reduces axis, origin and range errors relative to ArtiLatent. Controlled ablations support the joint-line encoding and show that motion supervision reduces anchor-motion error. Code and pretrained checkpoints will be released in the future.

[26] Foundation4D: Towards a unified perception and reconstruction model for autonomous vehicles cs.CVPDF

NVIDIA, :, Sérgio Agostinho, Maximilian Apodaca, Emmanuel Attia

TL;DR: 论文提出 Foundation4D，一个面向多相机行车视频的统一前馈模型，可同时完成几何估计、3D 检测、语义占用和 Gaussian 重建。模型用共享 backbone 和分解式时空注意力融合多相机、多时刻观测，并在视频推理中用位姿图优化和跨窗口目标轨迹组装提升一致性。作者还构建了 PAI-Foundation4D 基准，包含 1,253 个驾驶片段的稠密深度和语义占用标注，并验证多任务联合训练对检测与几何估计均有增益。

Details

Motivation: 自动驾驶通常需要多个独立模型分别处理几何、检测、占用和重建，缺乏统一且高效的感知与重建框架。论文希望用统一前馈模型从多相机视频中同时输出多类任务结果，并解决跨相机/跨时间融合、轨迹一致性和标注基准缺失等问题。

Result: 在基准特定微调后，Foundation4D 在 Waymo 和 nuScenes 上取得最低深度 AbsRel，在 nuScenes 上取得最高检测 mAP，在 OpenOccupancy 上取得最高占用 mIoU。任务组合研究表明，检测与几何/占用联合训练相比仅检测训练提升 top-class mAP 1.2 点，几何与检测联合训练相比仅几何训练降低 camera-to-rig 平移误差 13.4%。

Insight: 创新点在于统一的多任务前馈架构、分解式时空注意力、视频级位姿图优化与目标轨迹组装，以及配套的数据标注流程和 PAI-Foundation4D 基准。客观来看，多任务协同增益和将预测几何/轨迹作为无激光雷达 NuRec 仿真重建先验，是其较有借鉴价值的部分。

Abstract: We introduce Foundation4D, a unified feed-forward model for geometry estimation, 3D detection, semantic occupancy, and Gaussian reconstruction from multi-camera driving video. A shared backbone uses factorized spatiotemporal attention to efficiently combine observations across cameras and time. Our video inference pipeline uses pose graph optimization to refine camera trajectories and assembles object tracks across input windows. We also develop a data annotation pipeline and release PAI-Foundation4D, a benchmark with dense depth and semantic occupancy labels for 1,253 driving clips. With benchmark-specific fine-tuning, Foundation4D achieves the lowest depth AbsRel on Waymo and nuScenes, the highest detection mAP on nuScenes, and the highest occupancy mIoU on OpenOccupancy among the compared methods. Our task-combination study shows benefits for both detection and geometry: co-training detection with geometry and occupancy improves top-class mAP by 1.2 points over detection-only training, while co-training geometry with detection reduces camera-to-rig translation error by $13.4%$ relative to geometry-only training. We further use the predicted geometry and trajectories as priors for lidarless NuRec, supporting reconstruction of driving scenes for simulation.

[27] Latent Space Jam: Layout Guided Video Synthesis cs.CVPDF

Asaf Joseph, Moshe Mandel, Ohad Fried

TL;DR: 论文提出一种无需额外训练的视频生成控制方法，利用预训练视频扩散模型和粗粒度布局来约束对象的位置与运动。方法分两阶段：先通过提示引导的交叉注意力生成粗略的时空视觉引导，再以该引导条件化同一模型生成高质量、布局一致的视频。该方法支持多样轨迹与对象交互，并保持较强的时间一致性。

Details

Motivation: 为视频生成加入布局和运动控制通常需要额外训练，带来计算资源、数据或架构改动成本。论文希望在不对预训练视频扩散模型进行额外训练的情况下实现可控视频合成。

Result: 大量实验表明，该方法在静态、动态和多对象场景设置下，于对象重叠、轨迹准确性和视觉质量方面一致优于先前工作。摘要未明确给出具体benchmark数值或SOTA表述。

Insight: 核心创新是免训练的两阶段布局引导视频生成框架：先利用交叉注意力产生粗视觉引导，再用其条件化原模型生成布局一致视频。客观来看，其可借鉴之处在于将布局控制转化为中间视觉引导，从而复用预训练扩散模型并支持复杂运动与交互。

Abstract: Adding control to video generation often requires training, which demands additional computational resources, data, or architectural changes. We propose a training-free approach that utilizes pretrained video diffusion models to control object layout and motion using coarse layouts. Our method operates in two passes: first, it steers spatiotemporal placement through prompt-guided cross-attention to produce a coarse visual guide; second, this guide conditions the same model to generate a high-quality, layoutconsistent video. The approach enables coordinated motion with strong temporal consistency, supporting diverse trajectories and object interactions without additional training. Extensive experiments show that our method consistently outperforms prior work in overlap and trajectory accuracy, as well as visual quality across static, dynamic, and multi-object scene settings.

[28] VisCurate: Evaluating Behavioral Equivalence of Visual Skills cs.CVPDF

Md Farhan Ishraq, Md Farhan Ishmam, Reaz Hassan Joarder, Md Hasanul Kabir

TL;DR: 论文提出VisCurate框架和基准，通过观察技能执行时的图像变化来整理视觉技能库，而非依赖名称或描述等文本信号。它定义了六种行为关系并关联安全的库编辑，在注入七类缺陷的300个基准实例上评估，并应用于真实未整理库。

Details

Motivation: 视觉技能库增长过程中会积累重复、错误标记和缺陷技能，而传统用于整理的文本信号（名称、描述、代码）无法反映技能对图像的实际视觉改变，因此需要基于执行行为来整理。

Result: 在困难负样本上，文本基线最多接受5/6的错误合并，而VisCurate验证器零错误合并并恢复51.2%真正可合并对；在两个真实库上发现92%共享技能冗余，且仅确认17个等价声明中的10个；最强自动整理器仅恢复约一半修复。

Insight: 创新点在于提出基于行为等价而非文本描述的技能整理范式，定义六种行为关系并关联安全编辑，构建可复现的缺陷注入基准，并实证表明行为信号比文本信号更可靠。

Abstract: Modern vision-based agents have shifted toward skill-based problem-solving, where a skill is an executable image-to-image operation, e.g., cropping, thresholding, or edge detection. As these skill libraries grow, they accumulate duplicated, mislabeled, and defective skills. The signals normally used to curate them (a skill’s name, description, or source code) say little about how the skill visually changes the image. We present VisCurate, a framework and benchmark that curates visual skill libraries from observed execution rather than from text. VisCurate runs each candidate skill pair on a shared probe battery and assigns one of six behavioral relations, each tied to a library edit that is safe to make. The benchmark injects seven defect types into a clean library at controlled rates, producing 300 reproducible instances with ground-truth relations and repairs. While text baselines accept up to five of six false merges on hard negatives, our verifier accepts none and recovers 51.2% of the pairs that are truly mergeable. On two real, uncurated libraries, VisCurate finds that 92% of the skills a package shares with its downstream copy are redundant, while execution confirms only 10 of 17 published equivalence claims. The curation problem also remains open, as the strongest of the nine automated curators recovers only about half of the repairs. We emphasize that visual skill libraries should be curated by behavior, not by description.

[29] SplatLab: Autonomous Scientific Discovery for 3DGS Research cs.CV | cs.AIPDF

Seemandhar Jain, Manmohan Chandraker

TL;DR: SplatLab 是一个面向 3D Gaussian Splatting（3DGS）的自主科研系统：给定研究提示、目标论文 URL 或无输入时，可复现已发表基线、提出扩展、在 Mip-NeRF 360 和 LLFF 上训练与消融、撰写论文，并将每个报告数值绑定到日志化训练运行。系统通过已验证复现骨架上的插件变异、Elo 锦标赛、失败证伪器和领域分类法等实现端到端自主研究。

Details

Motivation: 复现一篇 3DGS 论文通常需要数周专家工程，阻碍后续扩展；已有自主科研系统受限于固定模板、低可执行性和弱可追溯性，且尚未在活跃的计算机视觉子领域端到端验证。

Result: 在 PSNR 目标运行中，SplatLab 生成的论文结果超过其复现目标 +0.18 至 +3.78 dB，其中天体摄影星云渲染提升 +3.78 dB，且此前无对应 3DGS 方法；研究周期从数月研究生时间压缩到数天主要自主计算。

Insight: 创新点在于面向 3DGS 的端到端自主发现流程，以及可追溯科研机制：候选从验证复现骨架变异、八岛级联加权 Elo 锦标赛结合语言梯度反馈、失败证伪器扣除 Elo、所有数值绑定日志化训练且未支撑声明不可终止，以及七类 3DGS 分类法构建可迁移领域脚手架。

Abstract: Reproducing a published 3DGS paper takes weeks of expert engineering before any extension can be proposed. Prior autonomous-research systems read papers, draft hypotheses, and write code, but are limited by short fixed templates, low executability, and weak traceability, and none has been demonstrated end-to-end in a live computer-vision subfield with intense activity. SplatLab is an autonomous research system for 3D Gaussian Splatting that addresses these limitations. Given a research prompt, a target-paper URL, or no input at all, it reproduces a published baseline, proposes an extension, trains and ablates across Mip-NeRF 360 and LLFF, writes the manuscript, and binds every reported number to a logged training run. Five design choices distinguish it: (i) candidates mutate plugins from a verified-reproduction backbone matched to published baselines; (ii) a cascade-weighted Elo tournament across eight islands scores candidates on measured GPU PSNR with verbal-gradient feedback; (iii) a standing falsifier subtracts Elo on failure, integrating stress-test survival into selection; (iv) every reported number is bound to a logged training run, and the run cannot terminate while any claim is unbacked; (v) a seven-category 3DGS taxonomy wraps EVOLVE-BLOCK markers in domain-aligned scaffolds transferable across 3DGS repositories. On PSNR-targeted runs, SplatLab’s manuscripts beat their reproduced targets by +0.18 to +3.78 dB, including +3.78 dB on astrophotographic nebula rendering, for which no prior 3DGS method exists. The research cycle compresses from months of graduate-student time to days of mostly autonomous compute. Code, all manuscripts, experiment ledgers, and the cross-run paper store are available on the project page.

[30] AVAtt: A Free-Viewing Eye-Tracking Dataset and Benchmark for Visual Attention on Paintings cs.CV | cs.HC | eess.IVPDF

Mohamed Amine Kerkouri, Marouane Tliba, Aladine Chetouani, Alessandro Bruno

TL;DR: 论文提出 AVAtt，一个面向绘画自由观看的眼动追踪数据集与基准，包含跨越16个艺术运动、5个世纪的多幅绘画的扫描路径、注视点及丰富艺术元数据。论文在人类一致性上评测经典与深度显著性模型，并对44种扫描路径相似度度量进行排序，同时分析绘画中面孔、局部结构、构图规则和艺术风格对视觉注意的影响。

Details

Motivation: 现有显著性和扫描路径模型几乎都基于自然照片，而绘画在纹理、色彩和抽象程度上不同，且由艺术家构图引导视线，因此需要面向绘画的专用数据集与基准来推动艺术感知的视觉注意建模。

Result: 在 AVAtt 上对经典和深度显著性模型与人类一致性进行基准评测，发现面向艺术感知的模型仍有较大提升空间；同时排序了44种扫描路径相似度度量，并提出基于十种 CNN、ViT 和自监督编码器的无语言图像编码器指标族。控制观看偏差后的分析表明，面孔在半秒内被注视的频率高于预期，局部结构吸引注视，图像内容拟合的构图规则可预测注视位置，艺术风格影响观察者一致性。

Insight: 创新点在于构建了带精细眼动数据、时间显著性视频和艺术元数据的绘画自由观看数据集，并建立多维度基准；同时提出无语言图像编码器扫描路径相似度指标，并通过偏差控制揭示绘画特有的注意规律。

Abstract: Saliency and scanpath models are developed almost exclusively on natural photographs, yet paintings are composed by artists to guide the viewer’s eye and differ from photographs in texture, colour and abstraction. We introduce \avatt, a free-viewing eye-tracking dataset on \nImages{} paintings spanning sixteen art movements and five centuries, with \nScanpaths{} scanpaths and \nFixations{} fixations from \nParticipants{} participants (\spPerImageMin–\spPerImageMax{} observers per painting). For every painting we release individual scanpaths with fixation timing and eye–screen distance, fixation maps, saliency maps smoothed at one degree of visual angle, temporal saliency videos and rich art metadata. We benchmark classic and deep saliency models against human agreement, revealing substantial headroom for art-aware models, and rank 44 scanpath-similarity measures, including a new language-free family of image-encoder metrics built on ten CNN, ViT and self-supervised encoders. With controls that remove viewing biases, we show that faces are fixated \faceEnrich$\times$ more than expected and within half a second, that local structure draws the eye, that composition rules fitted from image content predict where people look, and that art style shapes how much viewers agree. Dataset, code and a Python package are available at: https://huggingface.co/datasets/Amine-CV/AVAtt.

[31] Unified Trajectory and 3D Pose Forecasting for Multi-Agent Sports Analysis cs.CVPDF

Marc Peral, Guillem Capellera, Luis Ferraz, Antonio Rubio, Stella Graßhof

TL;DR: 论文面向多智能体体育场景中未来人体运动预测，提出统一联合预测全局轨迹与3D身体姿态的框架。模型通过社会时间编码器在输入级融合轨迹和姿态，并用共享因子化注意力联合建模多智能体交互。方法还引入逐关节损失重加权和频域轨迹平滑，并修正重叠序列评估偏差，在WorldPose和3DPW上优于已有联合预测方法。

Details

Motivation: 未来人体运动中的全局轨迹与3D姿态紧密耦合，且团队运动中多玩家交互带来复杂社会动态，现有方法难以统一、公平地联合预测二者。论文旨在解决多智能体体育分析中轨迹与姿态联合建模及评估偏差问题。

Result: 论文在WorldPose和3DPW两个基准上相比已有联合预测方法取得提升，其中姿态精度提升最明显，表明所提联合建模方法可泛化到密集交互场景之外。

Insight: 创新点包括输入级融合轨迹与姿态、共享因子化注意力联合建模、强调动态关节的逐关节损失重加权、频域轨迹平滑，以及采用非重叠协议和无oracle多未来预测策略纠正评估偏差。客观来看，其价值在于将轨迹与姿态预测统一到多智能体社会时间建模框架，并关注联合预测评估的公平性。

Abstract: Forecasting future human motion requires jointly reasoning about an agent’s global trajectory and 3D body pose, which are tightly coupled, particularly in team sports where multiple interacting players introduce complex social dynamics. We exploit a socio-temporal encoder that fuses trajectory and pose at the input level and models them jointly through shared factorized attention. The model is further enhanced by a per-joint loss reweighting scheme that emphasizes dynamic joints and prevents degenerate solutions, and a frequency-domain trajectory smoothing term that improves trajectory prediction while indirectly benefiting pose forecasting. To ensure fair comparison with prior joint forecasting methods, we identify and correct evaluation biases arising from overlapping sequence construction, adopting instead a non-overlapping protocol and a deterministic, oracle-free strategy for multi-future prediction. Our approach improves over prior joint forecasting methods on both WorldPose and 3DPW, with the largest gains on pose accuracy, demonstrating that the proposed joint formulation generalizes beyond the densely interactive scenarios that motivate it.

[32] Visual Grounding First, Multimodal In-context Learning Follows cs.CV | cs.AIPDF

Minhyuk Seo, Minjae Lee, Chaeeun Lee, Wei Lin, Muhammad Jehanzeb Mirza

TL;DR: 论文指出多模态大语言模型的上下文学习瓶颈在于视觉定位，而非依赖额外训练数据。提出ProCoRe方法，通过对比强化学习增强视觉定位能力，无需ICL格式训练样本。在多个基准上提升分类准确率和captioning ROUGE-L，并引入新基准PICL测试模型对视觉演示的真正理解。

Details

Motivation: 现有多模态ICL方法依赖ICL专用数据集训练或任务向量提取，泛化差且导致遗忘；MLLM在演示中常无法关注任务相关视觉证据，过度依赖文本线索并产生幻觉输出。

Result: 在Qwen-VL-3-8B上，平均多模态ICL分类准确率提升5.6%，captioning ROUGE-L提升13.8%，超越依赖ICL数据的SOTA；并引入PICL个性化多模态ICL captioning基准。

Insight: 创新点在于识别视觉定位为多模态ICL关键瓶颈，提出对比强化学习优化视觉对齐（无需ICL训练数据），并设计PICL基准测试模型是否真正理解视觉演示而非依赖预训练语义先验。

Abstract: LLMs have shown strong in-context learning (ICL) capability, but extending it to Multimodal LLMs (MLLMs) remains challenging. Prior multimodal ICL methods often rely on ICL-specific datasets for additional training or task vector extraction, which can improve performance on the same ICL benchmark used for adaptation. However, they do not generalize well to other benchmarks and often induce forgetting in previously well-solved tasks. For this reason, we identify ‘visual grounding’ as the key bottleneck in multimodal ICL; MLLMs often fail to attend to task-relevant visual evidence in demonstrations, instead over-relying on textual cues and producing hallucinated outputs. To address this, we propose ProCoRe to enhance visual grounding through contrastive reinforcement learning on multimodal contrastive data for improving multimodal ICL. ProCoRe generates captions for contrastive image pairs, decomposes them into propositions, and optimizes contrastive rewards that encourage alignment with corresponding images while discouraging mismatched ones. Despite never seeing ICL-formatted training examples, ProCoRe improves average multimodal ICL classification accuracy by 5.6% and captioning ROUGE-L by 13.8% over the off-the-shelf Qwen-VL-3-8B model, outperforming ICL-data-dependent state of the arts across benchmarks. Finally, we introduce PICL, a personalized multimodal ICL captioning benchmark that tests whether MLLMs genuinely understand visual demonstrations rather than relying on pre-trained semantic priors.

Chenyao Yang, Peijin Wang, Yingchao Feng, Wenhui Diao

TL;DR: 论文提出了KinoUAV，一个用于安全稳定无人机视觉语言导航的运动动力学基准，包含数据集和KinoUAV-Agent。它通过分层运动动力学轨迹生成流程（考虑速度、加速度和jerk约束）和方向自由空间距离（DFD）预测来实现安全避障和稳定飞行，并设计了Cruise-Seek任务（先进入目标区域再搜索目标）。实验表明，KinoUAV-Agent在成功率、偏航率和碰撞率上优于现有方法。

Details

Motivation: 现有无人机导航方法未明确结合运动动力学建模和安全距离，导致偏航率高、碰撞频繁、导航成功率低，限制了安全性和运动稳定性。论文旨在解决这一问题。

Result: 在KinoUAV基准上，KinoUAV-Agent取得了最高的成功率，同时保持较低的绝对偏航率和碰撞率，优于现有方法。

Insight: 创新点包括：分层运动动力学轨迹生成流程（在速度、加速度和jerk约束下逐段生成轨迹，减少振荡）；引入方向自由空间距离（DFD）标签和预测，将感知与动作决策解耦以实现自主避障；以及Cruise-Seek任务设计（先进入目标区域再搜索目标），降低搜索难度并提高导航成功率。

Abstract: Unmanned aerial vehicles (UAVs) have recently emerged as important remote sensing platforms, and UAV navigation has shown great potential in remote sensing applications. However, existing methods do not explicitly incorporate kinodynamic modeling or safety distances into trajectory generation, resulting in high absolute yaw rates, frequent collisions, and low navigation success rates, which limit safety and motion stability. To address this challenge, we introduce KinoUAV, a kinodynamic benchmark comprising a dataset and KinoUAV-Agent. First, we develop a hierarchical kinodynamic trajectory generation pipeline that drives UAV through acceleration actions. It uses waypoint chains from global planning and generates trajectories segment by segment under velocity, acceleration, and jerk constraints, reducing oscillations and improving trajectory stability. Furthermore, we introduce directional free-space distance (DFD) labels into the constructed dataset to describe the available clearance for safe motion decisions. The proposed KinoUAV-Agent predicts DFDs and uses them to generate safe actions, decoupling perception from action decision-making. This design enables autonomous obstacle avoidance and improves navigation safety. The benchmark incorporates the proposed Cruise-Seek task, in which the UAV first enters the target region and then searches for the target, reducing the difficulty of target search and improving navigation success. Experiments on KinoUAV show that KinoUAV-Agent outperforms existing methods, achieving the highest success rate while maintaining lower absolute yaw rates and collision rates. These results demonstrate the effectiveness of kinodynamic modeling for safe and stable UAV navigation.

[34] GSHeadRelight: Fast Relightability for 3D Gaussian Head Synthesis cs.CVPDF

Henglei Lv, Bailin Deng, Jianzhu Guo, Xiaoqiang Liu, Pengfei Wan

TL;DR: 论文提出 GSHeadRelight，面向3D高斯人头合成中的快速可重光照问题，利用生成式3D高斯和统一的神经辐射传输表示来建模入射光响应，实现肖像重光照与新视角合成。方法仅使用野外图像训练，在重光照效果和渲染速度上均优于已有3D感知肖像重光照方法。

Details

Motivation: 肖像重光照与新视角合成在VR/AR和摄影等应用中很重要，但现有方法依赖昂贵且有限的光照棚数据，或基于生成式NeRF导致3D一致性和实时性较差，因此需要一种无需昂贵监督、高质量且快速的3D感知肖像重光照方案。

Result: 论文宣称仅使用野外图像即可取得最先进的重光照结果，并且相比已有3D感知肖像重光照研究渲染速度提升约12倍。

Insight: 创新点在于将生成式3D高斯与统一的神经辐射传输表示结合，使表示对入射光呈线性响应，从而在避免光照棚监督的同时提升重光照质量和实时渲染性能。

Abstract: Relighting and novel view synthesis of human portraits are essential in applications such as portrait photography, virtual reality (VR), and augmented reality (AR). Despite recent progress, 3D-aware portrait relighting remains challenging due to the demands for photorealistic rendering, real-time performance, and generalization to unseen subjects. Existing works either rely on supervision from limited and expensive light stage captured data or produce suboptimal results. Moreover, many works are based on generative NeRFs, which suffer from poor 3D consistency and low real-time performance. We resort to recent progress on generative 3D Gaussians and design a lighting model based on a unified neural radiance transfer representation, which responds linearly to incident light. Using only in-the-wild images, our method achieves state-of-the-art relighting results and a significantly faster rendering speed ($\times12$) compared to previous 3D-aware portrait relighting research. Project page: http://www.geometrylearning.com/GSHeadRelight/.

[35] FedBST: Bilateral Self-Training for Unsupervised Federated Classification cs.CVPDF

Aijing Yu, Kuangpu Guo, Jian Liang, Xiao-Yu Zhang

TL;DR: 论文提出 FedBST，一种用于无监督联邦分类的双边自训练框架。它同时利用高置信度样本和模糊样本进行伪标签学习，并在本地优化中引入不确定性感知置信度惩罚（UACP），在推理阶段采用视觉-文本原型集成与校准，以缓解严重非独立同分布下的性能退化。

Details

Motivation: 现有基于 CLIP 等视觉语言模型的无监督联邦学习方法多只使用高置信度预测生成伪标签，忽略了模糊样本信息，导致数据利用不足和决策边界偏置，尤其在严重 non-IID 场景下性能下降。

Result: 实验表明，在标准联邦聚合协议下，FedBST 相比现有方法取得一致的性能提升；摘要未给出具体数据集、数值或 SOTA 细节。

Insight: 创新点在于双边自训练同时挖掘高置信度与模糊样本，并通过 UACP 惩罚过度自信预测以降低错误伪标签风险；推理时融合不确定性感知的本地视觉原型与文本原型，实现轻量化的本地分布适配。

Abstract: Unsupervised federated learning with Vision-Language Models (e.g., CLIP) offers a promising solution for label-free classification, yet its performance often degrades under severe non-IID shifts. A pioneering method, FedCoPL, typically relies on self-training strategies that exclusively prioritize high-confidence predictions for pseudo-labeling. However, this selective approach neglects the information of ambiguous samples (often referred to as ``waist” samples), leading to the underutilization of data and biased decision boundaries. To address this limitation, we propose FedBST, a bilateral self-training framework that effectively exploits both high-confidence and ambiguous data. Specifically, for local optimization, we introduce uncertainty-aware confidence penalization (UACP). UACP targets ambiguous samples by penalizing overconfident predictions to implicitly promote higher uncertainty, thereby preventing erroneous pseudo-labeling. At inference, we employ a visual-text ensemble strategy. This approach aggregates uncertainty-aware local visual prototypes and ensembles them with textual prototypes, achieving lightweight adaptation to local distributions via prototype-based calibration. Experiments demonstrate that FedBST achieves consistent improvements over existing methods with standard federated aggregation protocols.

[36] CODI: Improving Self-Supervised Vision Transformers with Cross Distillation cs.CVPDF

Siran Dai, Qianqian Xu, Peisong Wen, Yang Liu, Xiaochun Cao

TL;DR: 论文针对自监督ViT图像级表示强但patch级表示对密集预测不可靠的问题，分析发现[CLS] token的全局语义集中于少量高注意力patch，而密集预测错误多出现在低注意力区域；提出跨蒸馏CODI，在标准自蒸馏框架中将少量教师class-token分布混入每个教师patch-token目标，为被mask的patch提供与注意力无关的全局语义校正。

Details

Motivation: 自监督ViT的patch级表示不可靠，且图像级自蒸馏梯度受[CLS]-to-patch注意力质量加权，导致低注意力区域难以获得全局语义监督；论文希望在不改变架构的情况下缓解该不匹配。

Result: 摘要称在密集预测和图像级benchmark上，在对齐预训练设置下CODI同时提升两类性能；但未给出具体数据集、数值或SOTA对比。

Insight: 创新点在于通过局部梯度分析揭示[CLS]注意力读出对自蒸馏梯度的影响，并提出把教师class-token分布混合进patch-token目标的轻量交叉蒸馏，使每个masked patch获得attention-independent语义校正；客观看，其创新主要是诊断驱动的目标函数改进，兼容现有自蒸馏架构。

Abstract: Self-supervised Vision Transformers learn strong image-level representations, but their patch-level representations can be unreliable for dense prediction. We study this mismatch and find that the class token concentrates global semantics on a small number of representative patches, while dense prediction errors often occur on low-attention regions. We provide a local gradient analysis showing that, along the [CLS] attention readout, image-level self-distillation gradients are weighted by [CLS]-to-patch attention mass. To provide masked patches with global semantics without changing the architecture, we propose Cross Distillation (CODI), which mixes a small amount of the teacher class-token distribution into each teacher patch-token target. CODI gives every masked patch an attention-independent semantic correction while preserving the standard self-distillation architecture. Experiments on dense and image-level benchmarks show that CODI improves both dense and image-level performance under aligned pretraining settings. Code is available at https://github.com/EldercatSAM/CODI.

Zixu Wang, Huaquan Zhang, Shoukai Xu, Jinwu Hu, Zhiquan Wen

TL;DR: 论文提出FETA，一种面向视觉-语言导航（VLN）的反馈高效测试时自适应方法，旨在以更少人工干预提升模型在未见环境中的适应性能。方法通过分层样本选择定位真正有信息量的轨迹和关键动作，并仅对关键步查询单次二元反馈，再结合差异感知熵优化更新策略。

Details

Motivation: 现有VLN智能体在分布偏移下性能显著下降；基于无监督熵最小化的TTA易沿轨迹传播错误并引发级联失败，而现有引入人类反馈的方法在轨迹级均匀监督所有步，未定位真正有信息的动作，导致反馈利用低效和优化粒度不匹配。

Result: 在多个基准上的实验表明，FETA以更少人工干预达到SOTA性能，并在噪声反馈和持续自适应设置下保持鲁棒。

Insight: 创新点在于反馈高效的分层样本选择：先筛选策略熵波动大的轨迹，再用熵-梯度显著性准则选择关键动作并仅查询单次二元反馈；同时采用差异感知熵目标，仅在查询步更新策略，并按预测-反馈差异自适应缩放权重。客观上看，其可借鉴之处是将测试时自适应从轨迹级粗粒度监督转为关键动作级精准反馈，从而兼顾反馈效率与优化可靠性。

Abstract: Vision-and-Language Navigation (VLN) requires agents to follow instructions and navigate to target locations. However, current VLN agents often suffer severe performance degradation in unseen environments due to distribution shifts. Test-Time Adaptation (TTA) offers a promising paradigm to bridge this gap, yet existing methods that rely on unsupervised entropy minimization can propagate errors through the trajectory, causing cascading failures in navigation. Recent methods mitigate this by incorporating human feedback, but they apply trajectory-level supervision uniformly across all steps without localizing the actions is truly informative, leading to inefficient feedback utilization and mismatched optimization granularity. To address these limitations, we propose FETA, a Feedback-Efficient Test-Time Adaptation method that improves both feedback efficiency and optimization reliability. To reduce unnecessary human interventions, we introduce a hierarchical sample selection strategy to localize informative samples. Specifically, we first filter trajectories whose normalized policy entropy exhibits large fluctuations. Within each selected trajectory, we further select the critical action via an entropy-gradient saliency criterion and query a single binary feedback on its correctness. A discrepancy-aware entropy optimization objective then updates the policy only at the queried step, with adaptive scaling that assigns larger relative weights to samples with greater prediction-feedback discrepancy. Extensive experiments across multiple benchmarks demonstrate that FETA achieves state-of-the-art performance with fewer human interventions, while remaining robust under noisy feedback and continual adaptation settings.

[38] One-shot federated unsupervised domain adaptation with Smoothed Knowledge Distillation and teacher refinement cs.CV | cs.AIPDF

Ali Abedi, Q. M. Jonathan Wu, Ning Zhang, Farhad Pourpanah

TL;DR: 论文提出SKD-ETR，一种一次性联邦无监督域适应框架，通过平滑知识蒸馏SKD和基于EMA的教师细化ETR，在单轮通信下利用源模型集成的平滑软伪标签训练目标域学生模型，并将目标域知识回传细化源教师模型。方法无需复杂聚合，旨在提升伪标签质量、学生泛化性和噪声鲁棒性。

Details

Motivation: 现有联邦无监督域适应方法通常需要多轮通信和复杂聚合策略，且易受噪声伪标签和源域知识不一致影响。论文希望在一次通信轮次内高效完成域适应并提升鲁棒性。

Result: 在OfficeHome、Office-Caltech和DomainNet上的实验表明，SKD-ETR取得有竞争力的性能，同时具有通信和计算效率，并在噪声监督下表现鲁棒。

Insight: 创新点在于将单轮蒸馏、源模型集成的平滑软伪标签与EMA教师细化结合，实现学生与教师间的双向知识细化，从而避免复杂聚合和额外通信开销。

Abstract: Federated unsupervised domain adaptation (FUDA) addresses the challenge of adapting models to an unlabeled target domain using decentralized source domains while preserving data privacy. However, existing FUDA methods typically require multiple communication rounds, rely on complex aggregation strategies, and often struggle with noisy pseudo labels and inconsistent source knowledge. To address these challenges, we propose SKD-ETR, a novel one-shot FUDA framework that combines Smoothed Knowledge Distillation (SKD) and Exponential moving average-based Teacher Refinement (ETR). SKD trains a student model on the target domain using smoothed soft pseudo labels generated by the ensemble of source models. This reduces overconfidence, mitigates noise, and improves robustness. ETR further refines each source model by interpolating its parameters toward the student via exponential moving average (EMA), thereby transferring target-domain knowledge back to the teachers. This bidirectional refinement enhances pseudo-label quality and student generalization without additional communication overhead. SKD-ETR eliminates complex aggregation by initializing the student model randomly and performing a single-round distillation process. Extensive experiments on OfficeHome, Office-Caltech, and DomainNet demonstrate that SKD-ETR achieves competitive performance while being communication- and computation-efficient, and robust under noisy supervision. The code for this paper is available at https://github.com/A-Abedi/SKD_ETR.

[39] Class-Grounded Batik Captioning under Scarce Text Annotation: Label Retention and Source-Specific Error Propagation cs.CVPDF

Novanto Yudistira, Muhammad Dzaka Aufa Fadhillah, Irawati Nurmala Sari

TL;DR: 论文研究在专家文本标注稀缺条件下，利用类别标签监督蜡染（batik）文化遗产图像描述，构建了由 LoRA 适配的 DINOv2 分类器与 4-bit NF4 量化并 LoRA 适配的 Qwen2.5-VL-7B 组成的级联系统。通过 3×2 评估考察三种描述器条件和两种推理提示对模型适配与预测类别注入的影响，并分析类别信息保留及上游分类错误向生成描述传播的现象。

Details

Motivation: 针对文化遗产图像专家文本标注稀缺的问题，论文希望利用紧凑的类别标签作为监督信号来辅助图像描述生成，并重点研究类别条件注入能否保留正确标签信息，以及分类器错误如何传递到最终描述中。

Result: 在 150 张描述评估图像上，三种注入预测类别的条件均达到 0.9067 的真实类别标签提及率；分类器单独测试的 pooled accuracy 为 96.10%，公共源 300 张图像上为 98.33%，本地 33 张图像上为 75.76%，本地子集 supported-class macro-F1 和 balanced accuracy 分别为 0.8493 和 0.8302。论文还发现基于参考的描述指标与图文兼容性分数会给出不同的模型排名，定性样例同时显示正确类别身份被保留和分类错误被传播到流畅描述中。

Insight: 论文的创新点在于将类别标签作为稀缺文本标注下的紧凑监督，并系统评估类别条件级联描述中的标签保留与源特定错误传播；其可借鉴之处是显式分离分类器与描述器条件、比较不同提示和评估指标，从而揭示上游分类错误会以流畅文本形式进入下游描述，且单一指标可能不足以评价此类系统。

Abstract: Class labels offer compact supervision for describing cultural-heritage images when expert text annotations are scarce. We investigate a batik-captioning cascade that integrates a LoRA-adapted DINOv2 classifier with Qwen2.5-VL-7B, adapted using LoRA and loaded in 4-bit NF4. A $3\times2$ evaluation combines three captioner conditions with two inference prompts to examine adaptation and predicted-class injection. Across 150 caption-evaluation images, all three injected conditions achieve a ground-truth class-label mention rate of 0.9067. The classifier’s separate test yields 96.10% pooled accuracy, with 98.33% on 300 public-source images and 75.76% on 33 local images. Supported-class macro-F1 and balanced accuracy on the local subset are 0.8493 and 0.8302, respectively. The classifier and captioner use separate partitions, and their training–evaluation overlap remains unquantified. Differences in label semantics and training support also affect interpretation of the source-specific accuracy gap. Reference-based caption metrics and image–text compatibility scores produce different model rankings. Qualitative examples show both retention of correctly predicted identities and propagation of classification errors into fluent descriptions. These findings characterize how class conditioning preserves label information and transfers upstream errors into batik captions under scarce text annotation.

[40] AgentDreamer: Agentic 3D Motion Reasoning for Text-Controllable Physics Simulation cs.CVPDF

Mijeong Kim, Suho Park, Junha Park, Bohyung Han

TL;DR: 论文提出 AgentDreamer，一个由智能体引导的优化框架：先将文本指令转化为结构化的 3D 语义运动先验和稠密轨迹，再优化可微分 MPM 物理模拟器使仿真运动在满足场景动力学约束的同时贴合该先验。论文还提出基于掩码文本恢复的信息论指标 Motion-IG，用于衡量仿真运动对文本-运动对齐的贡献。实验表明该方法相较视频引导方式能提升文本对齐并生成更符合请求物理行为的仿真。

Details

Motivation: 解决文本指令控制物理仿真的问题，即如何把语言中的运动语义转化为物理可行、时序合理的 3D 运动，并避免仅依赖视频引导带来的对齐不足。

Result: 实验显示 AgentDreamer 相比基于视频的引导方法提升了文本对齐效果，生成的仿真更贴近所请求的物理行为；同时提出 Motion-IG 作为文本-运动对齐的评估指标，但摘要未给出具体 benchmark 或定量数值。

Insight: 创新点在于用智能体将语言解析为结构化语义运动先验和稠密 3D 轨迹，并与可微分 MPM 仿真优化结合，实现语义规划与物理落地的解耦；同时提出基于掩码文本恢复的 Motion-IG，从信息增益角度评估运动对文本对齐的贡献。

Abstract: We propose AgentDreamer, an agent-guided optimization framework that aligns text instructions with simulated motion through explicit 3D motion planning. Given a text instruction and scene geometry, an off-the-shelf agent translates the instruction into a structured semantic motion prior, specifying what should move, in which direction, and in what temporal order. The agent represents this prior using parametric functions over a series of temporal chunks, yielding dense 3D trajectories. We then optimize the parameters of a differentiable MPM simulator such that its rollout best matches the prior while remaining constrained by the scene dynamics. This allows the agent to provide semantic guidance from language, while the simulator grounds it in physically feasible motion. We further introduce Motion-IG, an information-theoretic metric for text-motion alignment based on masked text recovery. By comparing the likelihood of recovering masked motion-related content with and without the simulated motion, Motion-IG measures the information contributed by the motion beyond that available from the remaining text and scene context. Experiments show that AgentDreamer improves text alignment over video-based guidance, producing simulations that more closely follow the requested physical behavior.

[41] Zero-Shot Domain Generalization via Domain Prompt Learning cs.CVPDF

Shiming Chen, Shuhuang Chen, Salman Khan, Fahad Shahbaz Khan

TL;DR: 论文提出DoP（Domain Prompt Learning），面向无源域数据的零样本域泛化问题，基于CLIP等大型视觉语言模型，在测试时通过文本和视觉两种模态的提示学习获取目标域知识。DoP先用文本域学习模拟多种分布偏移并学习风格令牌，再用视觉域学习结合测试图像自监督细化目标域相关风格令牌，从而识别未见目标域中的未见类别。

Details

Motivation: 现有域泛化方法通常依赖源域数据学习域不变特征或提示，但大规模单源或多源数据的收集与标注往往不可行，限制了实际应用。论文希望在无法访问源域数据的条件下，对未见目标域和未见类别实现域泛化。

Result: 在PACS、VLCS、OfficeHome、DomainNet等常用域泛化基准以及ImageNet上，DoP持续优于当前最先进的域泛化方法。

Insight: 创新点在于将域泛化转化为基于视觉语言模型的零样本测试时域提示学习，不依赖源域数据；通过文本空间的多样风格令牌学习与多样性/一致性约束模拟分布偏移，并利用测试视觉数据进行自监督视觉域学习以细化目标域风格令牌，形成文本-视觉双模态域提示适应机制。

Abstract: Domain generalization (DG) aims to improve the generalization of the model to any unseen domains. Targeting this goal, existing DG methods mainly focus on learning domain-invariant features or prompts from the given source domains. However, these methods rely on source-domain data, and collecting and annotating large-scale single- or multi-source datasets is often infeasible, limiting the practical applicability of DG. To tackle this challenge, we propose {\underline{Do}}main \underline{P}rompt Learning (DoP), a method built on a large vision-language model (e.g., CLIP) for zero-shot domain generalization, which recognizes unseen classes in unseen target domains without access to source-domain data. DoP essentially learns the target domain knowledge via prompt both in text and visual modalities to overcome the domain shift in text-visual modalities under the test-time adaptation formulation. Specifically, DoP first takes text domain learning (TDL) to simulate various distribution shifts by learning diverse style tokens implicitly in the text space with inter-style diversity loss and intra-style consistency loss. Subsequently, DoP employs visual domain learning (VDL) to further refine the target-domain-relevant style token generated by TDL by making use of the test visual data with self-supervised learning. Extensive experiments on popular DG benchmark datasets (e.g., PACS, VLCS, OfficeHome, and DomainNet) and ImageNet show that our DoP consistently outperforms the state-of-the-art DG approaches. Codes are available at: https://github.com/shiming-chen/DoP.

[42] PhyEvolve: Multi-Agent State-Evolution Planning for Physics-Aware Video Generation cs.CVPDF

Yang-Hao Zhou, Lixing Tan, Zhiqi Li, Guang Yang, Xiao Yang

TL;DR: 论文提出PhyEvolve，一个多智能体框架，通过结构化状态演化规划增强用户提示，使现有文本到视频生成模型能够理解并生成具有连续物理状态演化的视频。该框架包含Director、Physical Evolver和Composer三个智能体，无需重新训练或修改模型，仅通过提示替换即可工作，在四个主要基准上取得SOTA。

Details

Motivation: 现有文本到视频生成模型视觉保真度虽高，但难以理解用户提示中隐含的物理演化，导致生成视频缺乏连续的物理状态变化。论文旨在通过自然语言提示增强来提升生成视频的物理一致性和状态连续性。

Result: 在四个主要基准上的大量实验表明，PhyEvolve达到SOTA性能，平均超越最前沿的智能体基线7.9个百分点。

Insight: 创新点在于将物理感知视频生成转化为多智能体结构化状态演化规划：Director解耦物理事件与观察变化，Physical Evolver构建因果链接的状态转移，Composer利用这些转移增强原始提示。客观来看，其创新在于不修改生成模型本身，而是通过显式因果推理和提示增强来引导现有生成器，为物理一致性视频生成提供了一种轻量且通用的解决方案。

Abstract: Recent advances in text-to-video generation have substantially improved visual fidelity, yet existing models still struggle to understand the physical evolution implicit in user prompts, making it difficult to generate videos with continuous physical state evolution. Here, we propose PhyEvolve, a multi-agent framework that enhances natural-language prompts through structured state-evolution planning. PhyEvolve consists of three agents: a Director, a Physical Evolver, and a Composer. Given a user prompt and an optional first frame, the Director grounds the scene and disentangles physical events from observation changes. The Physical Evolver uses scene evidence to identify physical mechanisms then constructs causally linked state transitions that unfold throughout the video, whether visible or not. The Composer utilizes these transitions to enhance the original prompts. Without retraining or model modification, PhyEvolve works directly with existing generators through prompt replacement. Extensive experiments on four major benchmarks show that PhyEvolve achieves state-of-the-art performance, outperforming the cutting-edge prior agentic baseline by 7.9 percentage points on average. These results demonstrate that explicit causal state-evolution planning improves physical consistency and state continuity in generated videos through natural-language prompt enhancement.

[43] Learning from Next-Frame Prediction: Towards Unified and Scalable Visual Representation Learning cs.CV | cs.AIPDF

Yifeng Yao, Jinghan Li, Zhihao Zhang, Yajiao Xiong, Yang Jin

TL;DR: 论文提出 VANE，一个基于嵌入空间下一帧预测的自监督视觉预训练框架。它用因果编码器总结过去观测，并由状态预测器推断整帧表示，同时用冻结语义 tokenizer 提供固定嵌入空间，并用轻量生成解码器重建未来帧 patch 以保留细节。通过为静态图像构造局部到全局裁剪序列，VANE 用同一目标统一图像与视频建模。

Details

Motivation: 现有掩码视频预训练侧重从上下文恢复缺失内容，而非从过去预测未来；自回归方法逐 token 预测，但表示质量仍不如联合嵌入方法。论文希望重新审视未来预测的粒度和空间，缩小这一差距。

Result: 实验表明 VANE 随模型和数据规模扩展而提升，在统一编码器中于多数 anticipation、recognition 和 dense perception 基准达到 SOTA；在 EPIC-KITCHENS-100 动作预测上，mean-class recall@5 超过 V-JEPA 2.1 7.1 个点。

Insight: 创新点在于将下一帧预测放在固定语义嵌入空间中进行，以时序演化作为主要学习信号；辅以生成解码器保留细粒度空间细节，并用局部到全局裁剪序列统一图像和视频的预训练目标。

Abstract: Video provides a natural source of supervision: what happens next serves as a learning signal for visual world modeling. Yet masked video pretraining focuses on recovering missing content from surrounding context rather than predicting the future from the past. Autoregressive methods instead predict forward one token at a time, but have yet to match joint-embedding methods in representation quality. This gap motivates us to revisit the granularity and space of future prediction. In this work, we introduce VANE, a self-supervised visual pretraining framework built on next-frame prediction in embedding space. A causal encoder summarizes past observations for a state predictor that infers the representation of the entire next frame, making temporal evolution the primary learning signal. This prediction is carried out in a fixed embedding space provided by a frozen semantic tokenizer, emphasizing scene content and structure. To retain fine-grained spatial detail alongside semantic content, a lightweight auxiliary generative decoder reconstructs future-frame patches conditioned on the predicted representations. To unify visual modeling across images and videos, we construct a local-to-global crop sequence for still images, allowing the same encoder to predict newly revealed regions from earlier crops under the same objective. Experiments show that VANE improves with model and data scaling, achieving SOTA on most anticipation, recognition, and dense perception benchmarks among unified encoders. Especially on EPIC-KITCHENS-100 action anticipation, it surpasses V-JEPA 2.1 by 7.1 points in mean-class recall@5. These results demonstrate that next-frame prediction in embedding space is effective and scalable for visual world modeling.

[44] SmolVLA-MoE: Sparse Mixture-of-Experts for Efficient Multi-Task Robotic Manipulation cs.CVPDF

Shiyang Zhang, Miao Yang, Zhengda Li, Yang Shao, Ruifang Zhai

TL;DR: 论文提出SmolVLA-MoE，将稀疏混合专家（MoE）引入轻量级视觉-语言-动作模型的动作专家FFN层，通过多个并行专家FFN和token级路由器让每个token自适应选择专家，并加入带可调权重的共享专家以平衡预训练能力继承与任务特化。方法在推理时仅激活单个专家FFN，约5亿激活参数、约8亿总存储参数，在LIBERO四个任务套件上取得84.25%平均成功率。

Details

Motivation: 现有轻量级VLA模型通常使用单一稠密动作专家处理所有任务，在有限计算预算下限制了多任务机器人操作能力，论文希望在不显著增加推理开销的情况下提升多任务性能。

Result: 在LIBERO基准的四个任务套件上，SmolVLA-MoE平均成功率为84.25%，高于SmolVLA的82.75%和OpenVLA-7B的76.5%，同时保持与SmolVLA相当的推理效率。

Insight: 创新点在于将稀疏MoE细粒度地引入动作专家的FFN层，并用token级路由实现按需专家选择；同时通过带可调权重的共享专家兼顾预训练能力继承与任务特化。客观来看，其价值在于以较少激活参数获得更强多任务能力，为轻量VLA模型提供了一种高效的参数扩展与专家化方案。

Abstract: Vision-Language-Action (VLA) models have shown promising results in multi-task robotic manipulation by leveraging visual and language understanding for action generation. However, existing lightweight VLA models employ a single dense action expert to handle all tasks, which limits their multi-task capability under constrained computational budgets. To address this issue, we propose SmolVLA-MoE, a novel framework that introduces sparse Mixture-of-Experts (MoE) into the FFN layers of the action expert in VLA models. SmolVLA-MoE replaces dense FFN layers with multiple parallel expert FFNs and a token-level router, enabling each token to adaptively select the most suitable expert for action generation. We further introduce a shared expert with a tunable weight parameter to balance the inheritance of pre-trained capabilities and task-specific expert specialization. Through sparse activation, SmolVLA-MoE activates only a single expert FFN during inference, with a total activated parameter count of approximately 500M including the frozen VLM backbone, while benefiting from approximately 800M total stored parameters. We evaluate our method on four task suites of the LIBERO benchmark. SmolVLA-MoE achieves an average success rate of 84.25%, outperforming SmolVLA (82.75%) and OpenVLA-7B (76.5%), while maintaining comparable inference efficiency. Our code is publicly available at https://github.com/dudu435/smolvla-moe.

[45] Where Did the Light Come From? Diagnosing Cast-Shadow Reasoning in VLMs and Borrowing the Missing Shadow Prior from Image Generation Models cs.CVPDF

Ryugo Morita, Shunsuke Kitada, Stanislav Frolov, Brian Bernhard Moser, Ko Watanabe

TL;DR: 论文提出 ShadowBench——一个基于真实照片构建的阴影推理基准，包含阴影方向读取、缺失阴影方向推断、阴影物理一致性判断三类任务，并构造了正确阴影、去阴影、人工标注错误方向三种场景状态。作者评测了 19 个开源视觉语言模型（2B~78B）及前沿闭源模型，发现 VLM 能较好读出可见阴影方向，但推断缺失阴影方向能力很弱，判断物理一致性时甚至退化到随机水平。最后提出将 VLM 通过图像生成模型（IGM）进行路由，借助 IGM 已有的阴影先验显著提升其推断与判别能力。

Details

Motivation: 阴影是光照、几何与场景结构共同作用的可观测结果，真正理解阴影的模型应能读取阴影方向、推断缺失阴影应落何处，并判断阴影与场景是否物理一致。现有 VLM 是否具备这种因果性光照理解尚缺乏系统诊断，作者希望定位其能力短板并寻找缺失的关键要素。

Result: 在 ShadowBench 上，19 个开源 VLM 读取可见阴影方向显著高于随机且随规模提升，但推断缺失方向表现很弱；在消除 yes-bias 的无偏二选一测试中，一致性判断降至随机水平，仅 Gemini-3.1-pro 略高于随机且远低于人类，CoT 无帮助。将 VLM 经由 IGM 路由后，其方向推断明显超过直接尝试，并能识别此前误接受的错误方向阴影。

Insight: 论文的核心洞见在于：VLM 缺失的不是推理能力而是光照/阴影先验知识，而这类知识已隐含在图像生成模型中，可通过路由或蒸馏方式借用。其创新点在于以三态场景构建可诊断的阴影推理基准，并提出用生成/重建目标弥补判别式 VLM 的物理常识缺口，为光照感知 VLM 指明方向。

Abstract: Shadows are observable consequences of latent physical causes, arising from the interaction between illumination, object geometry, and scene structure. A model that understands shadows should be able not only to read a shadow’s direction, but also to infer where a shadow should fall when it is absent, and to judge whether a shadow is physically consistent with the scene. We introduce ShadowBench, built from real photographs, which poses these three questions to vision-language models (VLMs). From each scene, we build a correct-shadow, a shadow-removed, and a human-annotated wrong-direction state. Evaluating 19 open VLMs (2B to 78B) and frontier proprietary models, we find that VLMs read a visible shadow’s direction well above chance and improve with scale, yet infer the absent direction only weakly and fall to chance at judging physical consistency even under a bias-free two-alternative test that removes the yes-bias of single-image naturalness questions. Only Gemini-3.1-pro exceeds chance on validity, and even then only modestly and far below humans. Chain-of-Thought (CoT) does not help. The missing ingredient is not reasoning but knowledge that image generation models (IGMs) already hold. Routing a VLM through an IGM lifts its inference well above its direct attempt and helps it flag the wrong-direction shadows it otherwise accepts. Cast-shadow direction is a single facet of illumination understanding, but the sharp gap it reveals between VLMs and IGMs suggests image-generation or reconstruction objectives as a route toward illumination-aware VLMs.

[46] FilmFlow: Agentic System for 3D Virtual Film Cinematography cs.CVPDF

Yiming Zhang, Jiarui Guo, Haojia Wei, Yifei Liu, Yifan Yang

TL;DR: 论文提出 FilmFlow，一个面向动态 3D 环境的智能体框架，可将自然语言指令或参考视频转换为可执行的相机轨迹。系统由 Director 智能体统筹，通过并发原子操作组合平移、旋转和构图等高层运动，并借助 Interpreter、Scout、Critic 三个专门智能体完成意图解析、初始视角选择和迭代视觉反馈优化。论文还构建了 CinemaBench 基准，用于评估文本指令和视频参考引导的相机生成。

Details

Motivation: 复杂相机运动对视觉叙事至关重要，但从电影摄影意图到可控、可执行的相机轨迹仍很困难，尤其需要兼顾空间推理、时序协调和视觉构图。

Result: 在 CinemaBench 上，该基准包含 150 个 Blender 场景、750 个文本引导任务和 60 个参考视频，FilmFlow 在多种基线中取得最强基准性能和最高用户偏好，展现出有效的 3D 电影摄影能力。

Insight: 创新点在于用智能体分工将摄影意图解析、视角选择、轨迹规划和渲染反馈闭环结合，并以并发原子操作统一控制平移、旋转与构图；同时提出覆盖文本与视频参考的相机生成基准，客观上有助于推动可执行 3D 摄影研究。

Abstract: Complex camera motion is central to visual storytelling, yet translating cinematographic intent into controllable, executable camera trajectories remains challenging. We introduce FilmFlow, an agentic framework that converts natural-language instructions or reference videos into explicit camera trajectories for dynamic 3D environments. To generate these trajectories, a Director agent constructs executable camera plans by composing high-level motions from concurrent atomic operations that jointly control translation, rotation, and framing. The Director is supported by three specialized agents: an Interpreter that extracts scene-grounded cinematographic intent, a Scout that selects initial viewpoints, and a Critic that evaluates rendered results and provides visual feedback for iterative plan refinement. We also introduce CinemaBench, comprising 150 Blender scenes from self-created and licensed sources, with five camera instructions per scene for 750 text-directed tasks and 60 reference videos associated with a subset of scenes. CinemaBench supports evaluation of both text-directed camera generation and video reference-directed camera generation. Across a diverse set of evaluated baselines, FilmFlow achieves the strongest benchmark performance and the highest user preference, demonstrating effective spatial reasoning, temporal coordination, and visual composition for executable 3D cinematography.

[47] DeXplicit: Making Physical State Explicit in Dexterous Manipulation World Models cs.CVPDF

Adam Lee, Mutian Xu, Yukang Cao, Shulin Tian, Ziwei Liu

TL;DR: 论文提出DeXplicit，一种灵巧操作世界模型。它不直接让视频生成模型隐式学习物理动力学，而是先在度量3D空间中显式预测物体为中心的运动与形变，再把该3D预测作为视频生成的强骨架，将外观沿预测的3D运动传输并投影到未来视角，从而渲染出视觉结果。这种显式3D优先的分解还支持高效因果生成，每个块只需两步去噪。

Details

Motivation: 现有方法要么把动作条件动力学、相机运动和视觉合成纠缠在视频生成模型里，要么用点轨迹表示动力学但缺乏显式物体结构，难以得到结构化、几何上可解释的未来世界状态。

Result: 在HOT3D、OakInk2和PhysTwin上，相对PointWorld将moved-point error降低27-40%，在HOT3D上共享渲染器时object fidelity IoU+提升47%；两步因果生成相比其14B双向配置在视频合成上实现57倍加速。

Insight: 核心创新是“显式3D优先”的分解：先用轻量3D动力学网络预测物体为中心的运动与形变，再据此传输外观并渲染，避免从像素中隐式学动力学；同时该结构天然支持高效因果式、少步去噪的生成。

Abstract: Before acting on an object, a robot needs to anticipate how its hand motion will change the physical world and what the resulting scene will look like. Existing approaches typically entangle action-conditioned dynamics, camera motion, and visual synthesis within a video generation model, or represent dynamics as point trajectories without explicit object structure. We introduce DeXplicit, a dexterous manipulation world model that instead predicts how the world evolves explicitly in metric 3D before rendering its visual consequences. Given an initial scene and a future hand trajectory, a lightweight 3D dynamics network predicts object-centric motion and deformation, providing structured and geometrically grounded future world states. These predictions then serve as a strong scaffold for video generation: appearance is transported through the predicted 3D motion and projected into future views, allowing the renderer to focus on completing uncertain and newly visible content rather than implicitly learning world dynamics from pixels. This explicit-3D-first decomposition also enables efficient causal generation, requiring only two denoising steps per block while conditioning on generated history. Across HOT3D, OakInk2, and PhysTwin, DeXplicit reduces moved-point error by 27-40% relative to PointWorld and improves object fidelity IoU+ by 47% on HOT3D under a shared renderer. Our design with two-step causal generation achieves a 57x speedup from our 14B bi-directional configuration in video synthesis.

[48] DRILL: Training World Models to Improve Policies, Not Predict Pixels cs.CVPDF

Zhaolu Kang, Tailong Luo, Lei Wei, Guangyuan Dong, Sihan Liu

TL;DR: 论文指出现有用于VLA策略强化学习后训练的视频世界模型都以单步像素预测为训练目标，而这类模型在此场景中的真正作用是从想象rollout中产生能改进策略的学习信号，二者存在明显错配。作者提出DRILL双层优化框架，通过在内层对想象rollout做GRPO更新后、以外层策略回报为目标来更新世界模型，并给出两种实现：DRILL-VWI（按策略访问分布、GRPO组相对优势和模型置信度重加权预测误差的闭式替代）与DRILL-IMG（用截断Hessian-向量积穿透内层更新的完整元梯度方法）。

Details

Motivation: 世界模型被用来减少VLA策略RL后训练中的环境交互，但通常仍被训练成一步像素预测；作者证明像素MSE更低反而可能带来更差的策略梯度，因此目标是解决世界模型训练目标与下游策略效用之间的错配问题。

Result: 在5个操作仿真器和2个VLA骨干网络上，DRILL-IMG的平均成功率分别比WMPO高13.1个百分点、比RLVR-World高7.9个百分点，并且与RLVR预训练互补；同时DRILL-VWI在假设下可恢复DRILL-IMG元梯度的局部一步分量，且一个面向策略效用的世界模型即使下一帧像素MSE高17%仍能带来更好的下游策略。

Insight: 核心创新是把世界模型的训练目标从像素级预测转为它所产生的策略梯度/下游回报，利用双层优化与元梯度直接优化策略效用；并设计了无需额外仿真rollout的闭式重加权替代方法（策略访问分布×GRPO优势×置信度），兼具理论联系与实际效率。

Abstract: World models are increasingly used to reduce environment interaction when post-training vision-language-action (VLA) policies with reinforcement learning. Yet they are typically trained to predict pixels one step ahead, even though their role here is not merely to render visually faithful videos but to produce learning signals from imagined rollouts that improve the policy. We show that this mismatch is substantial: lower pixel MSE can produce worse policy gradients, and a world model trained for policy utility can yield better downstream policies despite 17% higher pixel MSE on the next frame. We propose the Downstream-Return Imagination Learning Loop (DRILL), a bilevel framework that updates the world model to maximize the policy’s return after an inner GRPO update on imagined rollouts. DRILL has two instantiations. DRILL-VWI is a closed-form surrogate, requiring no additional simulator rollouts, that reweights prediction errors by policy visitation, the group-relative GRPO advantage, and model confidence. DRILL-IMG is a full meta-gradient method that differentiates through the inner update using truncated Hessian-vector products. We show that DRILL-VWI recovers the local one-step component of the DRILL-IMG meta-gradient under score-compatibility and small-residual assumptions. Across five manipulation simulators and two VLA backbones, DRILL-IMG raises mean success rate by 13.1 points over WMPO and 7.9 points over RLVR-World, and complements RLVR pretraining rather than competing with it. More broadly, we believe video world models for VLA reinforcement learning can be productively trained by the policy gradients they produce, not only by pixel-level prediction.

[49] Loop to Look Better: Learning Adaptive Depth for Question-Guided Perception in Vision-Language Models cs.CVPDF

Wencheng Ye, Anning Hu, Hongye Fang, Tianyi Wang, Junchi Yan

TL;DR: 论文提出 LoopLook，一种轻量级编码器侧循环视觉感知框架，使预训练视觉语言模型能够根据问题自适应地进行多轮视觉特征细化。它在中后层视觉编码窗口中循环注入问题表示，并通过学习停止机制和 token 级 Mixture-of-Looks 自适应决定循环深度并融合多轮表示。该方法将循环机制从推理阶段引入感知阶段，把一次性视觉编码转变为任务引导的自适应反复细化过程。

Details

Motivation: 现有视觉语言模型通常用单次、任务无关的视觉编码器获取视觉 token，导致视觉证据获取成为瓶颈：模型即使推理更久，也未必能细化回答所需的关键视觉信息。论文希望让视觉感知像推理一样具备迭代性和任务导向性。

Result: 在四个骨干模型和八个基准上，LoopLook 相比基线一致提升，MMStar 最高提升 8.6%，HallusionBench 最高提升 8.1%。分析显示其能按任务难度自适应分配深度，在降低输入分辨率时更好保留任务相关视觉信息，且首 token 时间仅增加 5.1% 至 6.5%。

Insight: 创新点在于编码器侧循环感知、每轮注入问题表示、学习式停止机制自适应深度，以及 token 级 MoL 融合不同循环步的视觉表示。客观来看，其可借鉴之处是将循环计算从语言推理迁移到视觉证据获取，并以轻量插件方式增强预训练 VLM 的任务导向感知能力。

Abstract: Visual perception should be iterative and task-directed, with representations refined in response to the question and the need for further processing. While Vision-Language Models (VLMs) increasingly support extended reasoning, their visual tokens are still typically acquired in a single, task-agnostic encoder pass. This creates an evidence-acquisition bottleneck: models may reason longer without refining the visual evidence needed to answer the question. We propose LoopLook, a lightweight encoder-side recurrent perception framework that equips pretrained VLMs with adaptive visual depth. LoopLook recurrently revisits a middle-to-late window of the vision encoder, injecting question representations at each loop to guide visual refinement. A learned stopping mechanism adaptively determines the recurrent depth, while a token-wise Mixture-of-Looks (MoL) combines representations from different loops, allowing each visual token to selectively integrate information acquired across recurrent steps. Across four backbones and eight benchmarks, LoopLook consistently improves performance over the baseline, yielding gains of up to 8.6% on MMStar and 8.1% on HallusionBench. Our analysis shows that LoopLook progressively refines visual attention and adaptively allocates depth according to task difficulty. It also better preserves task-relevant visual information under reduced input resolution, while increasing time to first token (TTFT) by only 5.1-6.5%. By bringing recurrence from reasoning back into perception, LoopLook transforms visual perception from a one-shot glimpse into an adaptive process of recurrent refinement.

[50] RECAP: Looking Once Is Not Enough for Vision-Language Reasoning cs.CVPDF

Zhaolu Kang, Tailong Luo, Chenxin Li, Yingjie He, Zhenyu Yu

TL;DR: 论文研究视觉语言模型长链推理中的时间接地失败，发现推理过程中对图像证据的“视觉回访”峰值与答案正确性相关且具有因果作用；提出GRPO兼容训练方法RECAP，将回访轨迹转化为信用分配信号，把最终奖励传播到视觉关键推理步骤，无需额外奖励、rollout或推理时改动；在多个VLM骨干上提升图像接地推理表现。

Details

Motivation: 长链思维链推理能提升VLM，但生成过程中模型可能逐渐偏离图像证据，而强化学习通常只提供最终答案奖励，难以判断哪些视觉推理步骤应获得信用，存在时间接地与信用分配问题。

Result: 在Qwen2.5-VL-7B、InternVL3-8B和LLaVA-OV-7B上，RECAP稳定优于GRPO和VPPO，相对SFT基座在HallusionBench上取得+6.1至+6.5的提升，并保持推理和纯文本性能；在LoRA微调、中文零样本、视觉扰动及人工评估中也表现稳健。

Insight: 论文的创新在于发现视觉回访峰值可预测正确性并具因果影响，且该信号不能由原始注意力强度、不确定性、logit margin或隐状态范数解释；RECAP进一步用回访条件优势估计、动态折扣和门控价值预测实现无需额外监督或推理开销的过程级信用分配。

Abstract: Long chain-of-thought reasoning improves vision-language models (VLMs), but it also exposes a temporal grounding failure: as generation unfolds, models may drift from image evidence while reinforcement learning still provides only a final-answer reward. We study this gap through visual revisit, a temporally localized reactivation of image evidence during reasoning. Across three VLM backbones, revisit peaks are predictive of correctness and causally linked to performance: masking them degrades accuracy more than masking non-peaks, while injecting revisit-like peaks into failed rollouts partially restores correct answers. Control analyses show that this signal is not explained by raw attention strength, uncertainty, logit margin, or hidden-state norm. We propose RECAP, a GRPO-compatible training method that turns detached revisit traces into a credit-assignment signal. RECAP uses revisit-conditioned advantage estimation, dynamic discounting, and gated value prediction to propagate final rewards through visually important reasoning steps, without extra rewards, additional rollouts, or inference-time changes. Across Qwen2.5-VL-7B, InternVL3-8B, and LLaVA-OV-7B, RECAP consistently improves over GRPO and VPPO, yielding $+6.1$–$+6.5$ HallusionBench gains over the SFT base while preserving reasoning and text-only performance. Results on larger LoRA-tuned models, Chinese zero-shot benchmarks, visual perturbations, and human evaluation further demonstrate robust improvements in image-grounded reasoning.

[51] CAVC: Convergence-Adaptive Audio-Visual Cache for Modality-Asymmetric Diffusion Generation cs.CV | cs.SDPDF

Lifan Jiang, Yibo Zhao, Senlin cheng, Bolei Xu, Xiaobo Li

TL;DR: 论文提出CAVC，一种无需训练的模态不对称缓存框架，用于加速基于扩散的音视频生成。该方法观察到音频和视频在缓存后的可恢复性存在不对称：音频侧漂移可通过少量终端联合去噪步骤恢复，而视频侧退化则会导致持续细节丢失。因此CAVC采用三明治结构计算分配：中间阶段缓存音频去噪器输出以绕过音频去噪和跨模态注意力，同时保留视频分支的完整时空细化，末尾恢复几个联合步骤以减少跨模态漂移并提高同步性。

Details

Motivation: 现有音视频扩散生成方法在每个时间步都执行联合去噪和跨模态注意力，计算冗余。论文发现模态不对称的可恢复性：音频侧缓存引起的漂移可由少量终端步骤恢复，而视频侧缓存会导致不可逆的细节丢失。因此需要一种选择性缓存策略，仅加速可恢复的音频计算，同时保留视频的完整细化。

Result: 在多个公开基准上，CAVC实现了1.10–1.35倍的墙钟加速，改善了几种视觉动态和时序统计指标，同时保持了有竞争力的音频保真度和音视频对齐性能。

Insight: 论文宣称的创新点在于利用模态不对称可恢复性设计无需训练的缓存框架，并采用三明治结构（中间缓存音频、首尾联合去噪）。从客观角度看，其创新之处在于首次显式利用音视频模态在扩散采样中的恢复性差异进行选择性加速，且方法通用、无需额外训练，为多模态扩散模型的高效推理提供了新思路。

Abstract: Diffusion-based audio-visual generation methods typically perform joint denoising of audio and video latents throughout the entire sampling trajectory, repeatedly executing modality-specific denoising branches and cross-modal attention at every timestep. However, we observe that such uniform joint computation is not always necessary. The key is a modality-asymmetric recoverability: audio-side drift introduced by mid-stage caching can be substantially reduced by only a few terminal joint denoising steps, whereas video-side degradation caused by caching leads to persistent loss of motion and temporal details. This asymmetry suggests that inference acceleration should selectively cache recoverable audio-side computation while preserving full video refinement. Motivated by this observation, we propose \textbf{Convergence-Adaptive Audio-Visual Cache (CAVC)}, a training-free modality-asymmetric caching framework. CAVC adopts a sandwich-structured computation allocation: after detecting that audio denoiser outputs enter a cache-tolerant regime, it caches and reuses the audio denoiser output to bypass audio denoising and cross-modal attention in the middle stage, while the video branch continues independent spatio-temporal refinement; a few joint steps are then restored at the end to reduce accumulated cross-modal drift and improve synchronization. Experiments across multiple public benchmarks demonstrate that CAVC achieves 1.10–1.35x wall-clock acceleration, improves several visual dynamics and temporal statistics, and maintains competitive audio fidelity and audio-visual alignment.

[52] Direct Forcing: A Single-Stage Training Recipe for Few-Step Autoregressive Text-to-Video Generation cs.CV | cs.AIPDF

Haojun Xu, Jie Huang, Xin Lu, Mingchen Zhong, Zihao Fan

TL;DR: 论文提出Direct Forcing，一种面向少步自回归文本到视频生成的单阶段训练框架。它把参考视频同时用作因果延续的配对监督和自生成rollout的目标分布，通过Causal Continuation Alignment与Rollout Distribution Matching联合训练，并用Dynamic Gradient Balancing平衡两类梯度。方法从双向Wan2.1-1.3B出发，在250次更新内得到四步自回归生成器，无需单独初始化阶段或在线扩散分数监督。

Details

Motivation: 现有少步自回归视频生成训练通常将因果适配与分布匹配分成多个阶段，甚至依赖ODE初始化检查点和在线扩散分数监督，流程复杂且目标分布与参考视频定义不完全一致。论文希望用单阶段训练同时学习因果延续能力和目标视频分布。

Result: 从Wan2.1-1.3B开始，Direct Forcing在250次更新内训练出四步自回归生成器，无需单独初始化阶段或在线扩散分数监督，取得VBench Total 84.63和Quality 86.01。消融实验验证了动态梯度平衡的有效性，并发现更深特征更利于延续对齐、中间特征更利于rollout分布匹配。

Insight: 创新点在于将参考视频的双重角色统一到一个训练阶段：既作为历史延续的配对示例，又作为自生成rollout的分布目标；结合CCA、基于MMD的Rollout Distribution Matching和动态梯度平衡。客观来看，其价值主要在于简化少步自回归视频生成训练配方，并给出不同表征层对两类训练信号的偏好分析。

Abstract: Few-step autoregressive video generation enables low-latency streaming, yet prevailing training pipelines separate causal adaptation from distribution matching. Recent representation-based text-to-video methods retain this separation by starting from ODE-initialized checkpoints, even when the target distribution is defined directly by reference videos. We introduce Direct Forcing, a single-stage framework that uses reference videos both as paired examples of how a history continues and as a target population for self-generated rollouts. Causal Continuation Alignment learns paired spatiotemporal correspondence under reference histories, while Rollout Distribution Matching uses maximum mean discrepancy to align the representation distributions of self-generated and reference videos. Together, these complementary signals allow a pretrained video model to learn causal continuation and the target video distribution in a single training stage. Dynamic Gradient Balancing adapts the relative gradient contributions of continuation alignment and distribution matching during training. Starting from bidirectional Wan2.1-1.3B, Direct Forcing learns a four-step autoregressive generator in 250 updates without a separate initialization stage or online diffusion-score supervision, achieving 84.63 VBench Total and 86.01 Quality. Ablation studies further demonstrate the benefits of dynamic gradient balancing and reveal distinct representation preferences: deeper features benefit continuation alignment, whereas intermediate features yield better performance for rollout distribution matching.

[53] Verus-Insight: Towards Explainable Artifact Detection in AI-Generated Videos cs.CVPDF

Zhuoran Zhang, Yang Shi, Qixun Wang, Tengfei Liu, Yuqi Tang

TL;DR: 论文面向AI生成视频的可解释伪影检测，将伪影检测建模为检查规划、证据验证和证据聚合三个互补能力组成的过程，并提出Verus-Insight模型及Verus-Bench基准。方法通过人工锚定的全流程监督和渐进式能力训练，提升细粒度伪影识别与视频级来源判断。

Details

Motivation: 现有基于伪影的AI生成视频检测器可能漏检细微伪影，或虽给出正确来源判断但未忠实识别支撑证据，因此需要更可靠且可解释的伪影检测方法。

Result: 在多个基准上的实验表明，Verus-Insight在细粒度伪影分析和AI生成视频检测上均表现较强，尤其在恢复人类可感知伪影方面有显著提升；分析显示三类能力互补且可更可靠地将检查目标转化为有证据支撑的伪影判断，同时不增加虚假伪影声明。

Insight: 创新点在于将伪影检测结构化为检查规划、证据验证与证据聚合三阶段，并引入人工锚定全流程监督、渐进式能力训练及带细粒度人工标注的Verus-Bench，从而兼顾检测性能与证据可解释性。

Abstract: Recent advances in video generation have made synthetic videos increasingly realistic, placing greater demands on reliable AI-generated video detection. Existing artifact-based detectors typically identify generation artifacts as evidence for source prediction, yet they may either fail to recognize subtle artifacts or reach the correct source decision without faithfully identifying the evidence underlying it. In this work, we systematically investigate how to design and train effective artifact detectors and formulate artifact detection as a structured process comprising three complementary capabilities: Inspection Planning, which determines what should be examined; Evidence Verification, which verifies whether the inspected content exhibits genuine generation artifacts; and Evidence Aggregation, which integrates the verified evidence into the final source decision. Building on this formulation, we introduce Verus-Insight, trained with human-anchored full-process supervision and progressive capability training to strengthen both fine-grained artifact detection and video-level detection. We further introduce Verus-Bench, a benchmark with fine-grained human annotations of perceptible artifacts for evaluating artifact detection. Extensive experiments across diverse benchmarks demonstrate that Verus-Insight achieves strong performance in both fine-grained artifact analysis and AI-generated video detection, with substantial gains in recovering human-perceived artifacts. Further analyses reveal that the three capabilities are complementary yet distinct, and that Verus-Insight more reliably converts relevant inspection targets into supported artifact evidence without increasing spurious artifact claims.

[54] Query-Time Calibration of CLIP Features for Open-Vocabulary 3D Segmentation cs.CV | cs.AIPDF

Dominik Engel, Yousef Rajeh, Leon Sick, Ivan Viola

TL;DR: 论文提出一种查询时校准方法，用于提升基于CLIP特征提升到3D高斯重建后的开放词汇分割性能。方法不重新训练或重新提取特征，而是通过减去场景共享方向、用通用词表白化文本原型，并结合软聚类与带相关性 margin 的 CSLS 进行多类和单查询预测。

Details

Motivation: lifted CLIP 特征在单个场景中常出现强方向集中，原始余弦相似度未考虑场景特定特征分布，导致开放词汇3D分割比较不充分。论文旨在不改变已提升特征场的前提下，通过更合理的场景条件化比较方式提升分割效果。

Result: 在7种特征提升方法和4个数据集上，该方法在多类分割与单查询分割中均一致提升 mIoU，且无需重训练、源图像重编码或特征重提升。

Insight: 创新点在于将问题从特征提升转向查询时校准与场景条件化比较：通过场景中心化改善类间相对特征分离，并用参考词表白化和 CSLS margin 增强文本-视觉匹配。其可即插即用于现有 lifted 表示，具有较好的通用性和实用性。

Abstract: Lifting vision-language features such as CLIP into 3D Gaussian reconstructions enables open-vocabulary segmentation through natural-language queries. Although CLIP represents a broad range of semantic concepts, the lifted features of individual scenes can exhibit strong directional concentration, while raw cosine similarity does not account for the scene-specific feature distribution. We show that substantial segmentation gains can be obtained by changing how a fixed lifted feature field is compared with text. Our method partially subtracts a shared scene direction from visual features and text prototypes and whitens text prototypes using a fixed reference vocabulary of generic terms. It then uses soft clustering for multiclass prediction and CSLS with a relevance margin against the reference vocabulary for single-query prediction. Beyond reducing directional concentration, scene centering improves class-relative feature separation in every evaluated scene. Across seven lifting methods and four datasets, the resulting query-time calibration consistently improves mean intersection over union in both multiclass and single-query segmentation. These gains require no retraining, source-image re-encoding, or feature re-lifting, demonstrating the value of scene-conditioned comparison for making better use of existing lifted representations.

[55] Watch and Learn: Teaching Visual In-Context Learning by Removing Textual Shortcuts cs.CVPDF

Nimrod Shabtay, Asaf Yehudai, Avihu Dekel, Eli Schwartz, Raja Giryes

TL;DR: 论文发现视觉语言模型在上下文学习中依赖文本捷径而忽视视觉示例，提出无文本捷径的任务无关训练集MIME和DTAD训练策略，迫使模型从视觉演示中推断任务规则，从而提升视觉上下文学习能力并保持单图性能。

Details

Motivation: VLM相比文本模型在上下文学习上表现较弱，初步研究表明其依赖文本捷径、未充分利用视觉演示，作者假设VLM未充分训练视觉上下文学习。

Result: 在三个VLM上，MIME微调使vICL准确率提升8.1至18.0点，保持单图性能并改善多图和纯文本ICL；TrueMICL上Qwen3-VL-8B+MIME匹配Claude Opus 5等前沿模型，DTAD可超越；加入LLaVA 1.5 13B指令微调使vICL准确率翻倍以上且不损单图能力。

Insight: 创新点在于构建无文本捷径的vICL训练集MIME，并提出DTAD在训练中随机隐藏演示标签以迫使模型关注演示图像；客观看，论文强调vICL数据应成为VLM训练标准组件，并通过去除捷径和注意力正则提升视觉示例利用。

Abstract: Vision-language models (VLMs) struggle with in context learning compared to their textual counterparts. Our preliminary study shows that VLMs rely on textual shortcuts and underutilize the visual demonstrations. We therefore hypothesize that VLMs are insufficiently trained to perform visual in-context learning (vICL). To this end, we construct MIME, a task-agnostic vICL training set without textual shortcuts. We ensure the instruction does not state the rule linking the demonstration images to their answers, so the model must infer it from the visual demonstrations. Fine-tuning on MIME improves vICL accuracy by 8.1 to 18.0 points across three VLMs, preserves single-image performance, and improves multi-image and text only in-context learning. In contrast, stating the rule in the training data sharply reduces this gain. We further propose DTAD (Demonstration Text Attention Dropout), which randomly withholds demonstration labels during training so that the VLM must infer the rule from the demonstration images. While achieving comparable vICL accuracy to fine-tuning on MIME, DTAD yields more consistent attention to the demonstration images. On TrueMICL, Qwen3-VL-8B trained on MIME matches frontier models such as Claude Opus 5, and surpasses them with DTAD. Finally, adding MIME to the instruction-tuning mix of LLaVA 1.5 13B more than doubles its vICL accuracy without hurting single-image abilities, whereas a same-size multi-image set improves it only slightly. These results suggest that vICL data should be a standard component of VLM training.

[56] Making MLLM Latents Matter: Factor-Aware Predictive Learning for Cinematic Video Generation cs.CVPDF

Songlin Yang, Zhe Wang, Xiaotong Zhao, Xi Wang, Ran Zhang

TL;DR: 论文提出因子感知预测学习FAPL，用于电影级视频生成。该方法先让视频扩散Transformer有效利用冻结多模态大语言模型的潜在表示，再通过成对视频的因子跨度替换实现角色、场景、动作和相机等因子的可控编辑，最后冻结DiT并优化MLLM与连接器，使模型能从稀疏文本或文本加完整视频等有限输入中进行时间推理。实验在稀疏到视频、因子感知编辑和多镜头生成三个任务上均优于基线。

Details

Motivation: 电影视频生成需要从稀疏输入进行时间推理，并实现对角色、场景、动作和相机等因子的感知控制；现有多模态大语言模型虽有先验，但其表示并不天然可被视频生成模型有效利用，甚至可能被DiT绕过。

Result: 摘要未给出具体定量指标，但称在稀疏到视频、因子感知编辑和多镜头生成三个电影视频生成任务上 consistently 优于基线，并验证了FAPL对MLLM特征使用、时间控制和因子协调的贡献。

Insight: 创新点在于将“有效使用MLLM潜在表示”设为共享前提，并用三阶段互补学习信号分别强化生成器接地、因子跨度替换控制和有限输入下的时间预测；客观看，其价值在于通过训练机制强制DiT利用MLLM表示，并将因子控制与时间推理解耦训练。

Abstract: Cinematic video generation requires both temporal inference from limited inputs (i.e., sparse rather than frame-by-frame guidance) and factor-aware control of character, scene, action, and camera. Multimodal large language models (MLLMs) offer useful priors for both capabilities, but their representations are not automatically actionable for video generation: a Video Diffusion Transformer (DiT) may bypass them through native conditioning pathways, and using them does not ensure either factor-aware control or generation-compatible temporal prediction. We propose Factor-Aware Predictive Learning (FAPL), which establishes effective MLLM latent use as a shared prerequisite and then targets the two capabilities with complementary learning signals. First, latent-only generator grounding trains the DiT with a frozen MLLM and connector, strengthening the DiT’s use of MLLM latents for generation. Second, for factor-aware control, paired videos specify a target-factor change and non-target preservation, and factor-span replacement trains the DiT to realize this relationship in its output. Third, for temporal inference, text-only and text-plus-complete-video observations share a video target, and the trained DiT is then frozen while the MLLM and connector are optimized through it. This final stage trains limited-input representations for generation while retaining complete-observation reconstruction as a complementary training mode. Extensive experiments demonstrate our consistently better performance on three cinematic video generation tasks (i.e., sparse-to-video, factor-aware editing, and multi-shot generation) over baselines and validate FAPL’s contributions to MLLM feature use, temporal control, and factor coordination.

[57] Diagnostic Loss: Asymmetric Supervision for Radiology Report Generation cs.CV | cs.LGPDF

Ignacy Kolton, Wenxuan Li, Zongwei Zhou, Jarosław B. Ćwikła, Arkadiusz Sitek

TL;DR: 论文针对放射学报告生成中现有医学视觉语言模型偏向文本流畅性、忽视诊断关键信息的问题，提出Diagnostic Loss（DLoss）。该方法在解码器词元概率上直接施加可微、与架构无关的非对称监督，鼓励参考报告支持的诊断获得足够概率，同时抑制不支持诊断的虚假概率。作者在5万例腹部CT扫描和多种3D MedVLM架构上验证了该方法可提升诊断忠实度。

Details

Motivation: 现有MedVLMs通常采用词元级最大似然目标训练，优先保证文本一致性而非诊断关键术语，容易漏报关键诊断或引入错误诊断。论文将这一问题归结为传统语言建模目标与放射报告非对称诊断要求之间的目标错配。

Result: 论文在5万例腹部CT扫描、多种3D MedVLM架构以及分布内和分布外报告生成基准上训练和评估DLoss。结果表明DLoss提升了诊断忠实度，同时在常规语言生成指标上保持有竞争力的性能；摘要未明确宣称达到SOTA。

Insight: 创新点在于提出可微、架构无关的序列级非对称监督损失：分别聚合支持诊断与不支持诊断相关词表的概率质量，并通过跨词元位置的非对称池化，要求支持诊断至少在某一位置具有足够概率，同时在所有位置抑制不支持诊断的虚假概率峰值。该方法可直接用于现有3D MedVLMs，无需修改视觉骨干、语言解码器或推理流程，也不增加推理时计算。

Abstract: Automated radiology report generation requires models to do more than produce fluent clinical narratives: they must explicitly state supported diagnoses while avoiding unsupported ones. However, existing medical vision-language models (MedVLMs) are typically trained with token-level maximum likelihood objectives, which prioritize text agreement over diagnostically critical terms. As a result, models can miss key diagnoses and introduce wrong ones. We identify this as an objective misalignment between conventional language modeling and the asymmetric diagnostic requirements of radiology reporting. To encode these requirements, we introduce Diagnostic Loss (DLoss), a differentiable, architecture-agnostic objective that operates directly on decoder token probabilities. DLoss separately aggregates probability mass over vocabularies associated with diagnoses supported by the reference report and with unsupported alternatives. Using asymmetric pooling across token positions in the report, it encourages supported diagnoses to receive sufficient probability in at least one position while suppressing spurious probability spikes for unsupported diagnoses across all positions during training. The loss can be applied to existing 3D MedVLMs without modifying their vision backbones, language decoders, or inference procedures, and introduces no additional inference-time computation. We train and evaluate DLoss on 50,000 abdominal CT scans across several 3D MedVLM architectures on both in-distribution and out-of-distribution report-generation benchmarks. Across these settings, DLoss improves diagnostic faithfulness while maintaining competitive performance on conventional language-generation metrics, showing the value of asymmetric sequence-level supervision for radiology report generation. The code for DLoss is available at https://github.com/gmum/DLoss.

[58] ResGVC: Residual Coding for High-fidelity Generative Video Compression cs.CVPDF

Ding Ding, Daowen Li, Ying Chen, Kai Li

TL;DR: 论文提出ResGVC，一种面向高保真生成视频压缩的残差编码框架。该方法先通过学习的视频编解码器获得重建，再将重建与原图之间的失真分解为多个去噪步骤对应的残差段并传输，从而在扩散去噪过程中显式引导生成朝向原始视频。残差段由以扩散Transformer为熵模型的残差编解码器进一步压缩，并在像素空间直接去噪以规避潜空间信息瓶颈。

Details

Motivation: 现有生成视频压缩方法多依赖解码端扩散模型和生成先验进行去噪，且常在潜空间操作，缺乏对原始信号的显式约束，并可能受潜表示瓶颈导致不可逆信息损失。

Result: 实验表明，相比此前最先进方法，ResGVC在LPIPS和PSNR指标上分别实现63.7%和83.8%的BD-rate降低，同时具有更优的视觉质量。

Insight: 创新点在于将编解码失真分解为逐去噪步的残差段进行传输，以显式源约束引导扩散去噪；使用扩散Transformer作为残差熵模型增强分布建模；并在像素空间直接去噪以避免潜空间瓶颈。客观上看，该工作将残差编码与生成式压缩结合，兼顾保真度与生成先验利用，是较有借鉴价值的框架设计。

Abstract: Generative video compression (GVC) methods usually employ decoder-side diffusion models to denoise reconstructed signals based on generative priors. Besides, most of these methods perform denoising in latent space rather than directly in pixel domain. In this paper, we propose ResGVC, a residual coding framework for GVC that guides denoising toward original pictures by transmitting residual signals. ResGVC first encodes the original pictures within a learned video codec, and then decomposes the distortion between the codec reconstruction and the original pictures into a sequence of residual segments, where each residual segment is acquired at each denoising step. Residual segments can further be compressed by a novel residual codec with a diffusion transformer as an entropy model. Unlike prior methods, ResGVC introduces explicit source constraints throughout denoising rather than relying on decoder-side generation alone. The diffusion-based entropy model also enables more effective distribution modeling capability based on the powerful generative priors. Moreover, denoising directly in pixel space avoids the irreversible information loss caused by latent representation bottlenecks. Experimental results show that, compared with the previous state-of-the-art method, ResGVC achieves 63.7% and 83.8% BD-rate reductions in terms of LPIPS and PSNR, respectively, while also delivering superior visual quality.

Xiaoting Xie, Feng Qiao, Xianwei Zhang, Hailong Chen, Kaiwen Zheng

TL;DR: 论文提出I²-KAN，一个可解释的多路径多模态情感分析框架，将模态内信息和成对模态间交互信息分别保留在独立的结构路径中，每条路径通过任务解耦线性门控去噪瓶颈（TDL-GDB）精炼，再经结构化Kolmogorov-Arnold网络融合以保留显式路径关联并支持多级解释。实验表明该方法在多个基准上性能强，并揭示模态的独立预测效用与交互条件预测效用可能不一致。

Details

Motivation: 现有融合方法将模态特定表示和跨模态交互混入联合表示，难以区分预测效用来自模态本身还是来自与其他模态的交互；且模态的独立预测效用可能无法忠实反映其交互条件下的预测效用。

Result: 在CMU-MOSI、CMU-MOSEI和CH-SIMS v2三个基准上取得强整体性能；消融和可解释性分析一致表明，低独立预测效用的模态仍可通过成对交互保持信息量。

Insight: 创新点在于用独立结构路径显式分离模态内与成对模态间信息，结合TDL-GDB精炼和结构化KAN融合实现多级解释；客观而言，其价值在于从结构和解释层面区分独立效用与交互条件效用，并揭示多模态效用的交互依赖性。

Abstract: Multimodal sentiment analysis (MSA) aims to infer sentiment by integrating linguistic, acoustic, and visual information. However, many fusion methods integrate modality-specific representations and cross-modal interactions into joint representations, making it difficult to determine whether predictive utility arises from a modality itself or from its interactions with others. More fundamentally, a modality’s standalone predictive utility may not faithfully reflect its interaction-conditioned predictive utility. To address this issue, we propose I$^2$-KAN, an interpretable multi-pathway framework that explicitly maintains intra-modal and pairwise inter-modal information in separate structural pathways. Each pathway is refined by a Task-Decoupled Linear-Gated Denoising Bottleneck (TDL-GDB) to learn compact and task-relevant representations, followed by structured Kolmogorov-Arnold Network fusion that retains explicit pathway associations and supports multi-level interpretation. Extensive experiments on CMU-MOSI, CMU-MOSEI, and CH-SIMS v2 demonstrate strong overall performance. Ablation and interpretability analyses consistently show across the three benchmarks that low standalone predictive utility does not necessarily imply low interaction-conditioned predictive utility: a modality that appears weak in isolation can remain informative through pairwise interactions. These findings highlight the interaction-dependent nature of multimodal utility and suggest that modality utility should be characterized from both standalone and interaction-conditioned perspectives.

[60] Where Do Long Captions Fail? Position-Aware Diagnosis and Reinforcement Learning for Detailed Image Captioning cs.CVPDF

Yuanze Hu, Zhichao Yang, Junwei Jing, Xin Yang, Wenxuan Liu

TL;DR: 论文关注详细图像描述中“事实出现在什么位置”而非仅关注事实数量，提出位置感知的评分表示，将密集人工参考分解为原子视觉声明，并按四分位分析生成描述中支持、幻觉和泛化内容。诊断发现长描述中部易出现无依据的具体化，尾部易退化为模糊非证据语言。基于此提出 PAGER，用 GRPO 训练并设计位置相关奖励，在详细描述基准上改善位置级 grounding 和成对偏好，同时保持有竞争力的全局描述质量。

Details

Motivation: 现有详细图像描述多被当作“数量问题”，但自回归 VLM 按序生成，后段可能在更弱或更噪的视觉 grounding 下产生内容，因此需要研究长描述中事实出现的位置及其失败模式。

Result: 在 detailed-caption benchmarks 上，PAGER 提升了位置级 grounding 和成对描述偏好，并保持有竞争力的全局描述质量；摘要未明确声称达到 SOTA。

Insight: 创新点在于用同一套有序原子声明表示同时完成位置感知诊断和位置特定奖励设计，并针对中部无依据具体化、尾部空洞化及长度控制分别设计 GRPO 信号，形成诊断到修复的闭环。

Abstract: Detailed image captioning is usually framed as an amount problem: a model should mention more visual facts while avoiding hallucination. However, autoregressive VLMs generate captions as ordered sequences, and later generations may be produced under weaker or noisier visual grounding. We therefore argue that long-form captioning should be studied not only by what facts are mentioned, but also by where they appear. We introduce a position-aware rubric representation that decomposes dense human references into atomic visual claims, verifies generated captions against those claims, and induces a quartile-wise composition of supported, hallucinated, and generic content. This lens reveals two qualitatively different failures in long captions: models tend to produce unsupported specificity in the middle, while the final portion often escapes into vague, non-evidential language rather than simply accumulating more hallucination. A same-reference human control does not exhibit these patterns; additional human audits examine the visual validity of the induced labels. Motivated by this diagnosis, we propose PAGER (Position-Aware Grounded Evidence Reinforcement), a GRPO training objective with separate signals for supported fine-grained middle evidence, evidence-bearing tail continuation, claim coverage, and content-aware length control. The result is a closed loop: the same ordered claim representation identifies where long captions fail and supplies the position-specific reward needed to repair those failures. Experiments on detailed-caption benchmarks show that PAGER improves position-wise grounding and pairwise caption preference while preserving competitive global caption quality.

[61] Through a PRISM, Clearly: Step-Sequence Reasoning for Mistake Recognition in Procedural Videos cs.CV | cs.CLPDF

Olga Loginova, Oleksandr Bezrukov, Frank Keller

TL;DR: 论文提出PRISM，用于程序性视频中的步骤错误识别：在给定步骤边界下，将视频片段用语言描述锚定，并用在正确执行上训练的轻量隐马尔可夫模型把描述映射到程序状态，再通过参考对齐与语义检查识别步骤内容和顺序错误。论文还将不同数据集的标注统一为插入、删除、替换、换位四类编辑错误，并在三个benchmark上比较gold与VLM生成描述及多种表示形式。

Details

Motivation: 程序性错误识别需要检测多步骤活动相对正确流程的偏差，而现有数据集的标注和评估方式异构，且需要可解释的显式步骤序列推理接口。

Result: 在三个benchmark上，训练得到的局部解码器相比最近候选匹配取得一致提升，序列解码带来依赖数据集的额外收益；语义角色标注在六组匹配的grounded比较中均提升Any-F1，但增益随数据集和配置选择而变化，并未 uniformly 提升。

Insight: 创新点在于把语言grounding作为视频片段与程序状态之间的可解释接口，结合轻量HMM序列建模与四类编辑错误统一评估；客观上也清晰刻画了语义表示和解码策略各自对错误识别的贡献。

Abstract: Procedural mistake recognition requires identifying deviations from valid multi-step activities. We study post-completion recognition from videos with benchmark-provided step boundaries. We introduce PRISM (Procedural mistake Recognition via Interpretable Semantic Modeling), which grounds video segments in language and maps their descriptions to procedural states using a lightweight hidden Markov model trained on correct executions. Reference alignment and semantic checks identify errors in step content and order. To support evaluation across heterogeneous benchmarks, we map dataset-specific annotations to four edit-based error types: insertion, deletion, substitution, and transposition. Across three benchmarks, we compare gold and VLM-generated descriptions in raw, semantic-role, and action-object forms. Decoder controls show consistent gains from the trained local decoder over nearest-candidate matching, with additional, dataset-dependent benefits from sequence decoding. Semantic role labeling improves Any-F1 in all six matched grounded comparisons, although the gains vary across datasets and do not translate into uniform improvements under separate configuration selection. These results establish language grounding as an effective interface for explicit procedural mistake reasoning and characterize the contributions of representation and decoding.

[62] WildDepth-v2: A Synchronized RGB–LiDAR–Radar Dataset and Benchmark for Metric Depth, Motion, and Tracking of Wildlife cs.CVPDF

Muhammad Aamir, Sangyun Shin, Naoya Muramatsu, Matthew Wijers, Amir Patel

TL;DR: 论文提出 WildDepth-v2，将同步 RGB-LiDAR 野生动物数据集扩展到包含成像雷达，并构建用于度量深度、运动与跟踪的 benchmark。数据集包含 37 个 RGB-LiDAR-4D 雷达序列和隐藏测试集，评估 RGB-only 距离估计、RGB+稀疏 LiDAR 深度补全及基于 LiDAR/雷达/单目深度的度量 3D 跟踪。

Details

Motivation: 野生动物距离与运动测量对种群丰度估计、形态测量和运动生态学很重要，但单目视频缺乏度量尺度，现有野生动物数据集也缺乏度量真值。论文旨在提供同步多模态数据和标准化 benchmark，以支持野生动物度量深度、运动和跟踪研究。

Result: 在 48,723 个动物帧上，object-level 雷达校准使单目深度 AbsRel 达 0.120，优于 scene-level LiDAR 校准的 0.198，且推理无需 LiDAR；animal-level LiDAR 校准更准，AbsRel 为 0.052。431 条轨迹上，每轨迹一次雷达缩放将 3D 位置误差从 2.20 m 降至 0.58 m，检测函数偏差从 -13% 降至 -2%，与每轨迹一次 LiDAR 校准相当；雷达 Doppler 迁移到 LiDAR 生成的运动掩码密度为单独雷达的 3.7 倍、精度 72%，并降低移动动物 LiDAR 累积误差 37%，径向速度估计 r=0.60 但约低估 50%。

Insight: 创新点包括引入据称首个成像雷达野生动物记录，构建带校验的隐藏测试集并按动物/背景像素分别评估，以及利用 object-level 雷达校准在推理时替代 LiDAR 进行单目度量深度校准。客观看，多模态同步度量真值、跨模态校准与 Doppler-LiDAR 融合运动掩码为野生动物感知和生态监测提供了可复用 benchmark 与融合思路。

Abstract: Measuring animal distance and motion in metric units underpins abundance estimation, morphometrics, and movement ecology, yet monocular video lacks metric scale and wildlife datasets lack metric ground truth. We present WildDepth 2, extending the synchronized RGB-LiDAR WildDepth dataset in three directions. (i) Imaging radar: 37 RGB-LiDAR-4D-radar sequences (19,716 frames) with per-point Doppler velocity and a measured error budget, to our knowledge the first imaging-radar wildlife recordings. (ii) Benchmarking: a frozen, checksummed hidden test set for RGB-only distance estimation and RGB+sparse-LiDAR depth completion, evaluated separately on animal and background pixels. (iii) Motion and tracking: metric 3D trajectories obtained by lifting 2D tracks using LiDAR, radar, or monocular depth. Across 48,723 animal frames, object-level radar calibration improves monocular depth over scene-level LiDAR calibration without requiring LiDAR at inference (AbsRel 0.120 vs. 0.198), although animal-level LiDAR remains more accurate (0.052). Across 431 tracks, once-per-track radar scaling reduces 3D position error four-fold (2.20 to 0.58 m) and distance-sampling detection-function bias from -13% to -2%, matching once-per-track LiDAR calibration. Transferring radar Doppler onto LiDAR produces motion masks 3.7 times denser than radar alone at 72% precision, reduces moving-animal LiDAR accumulation error by 37%, and estimates radial speed (r = 0.60), albeit with approximately 50% underestimation.

[63] GM-GReFEL: Geometry-Guided Spatiotemporal State-Space Learning for Robust Dynamic Facial Expression Recognition cs.CV | cs.LGPDF

Yudhistira Arditya Pratama, Yi-Zeng Hsieh

TL;DR: 论文提出GM-GReFEL，用Modulated GroupMamba时空状态空间网络替代时空ViT，以线性复杂度处理视频片段，并加入GReFEL可靠性模块，根据预测熵融合分类器输出与可训练几何锚点投票，从而提升动态面部表情识别在噪声标签下的鲁棒性。模型通过静态到动态权重迁移和Triple Loss训练，在DFEW、MAFW和FERV39k上取得有竞争力结果。

Details

Motivation: 无约束视频中的动态面部表情识别面临两个耦合问题：自注意力计算成本随片段长度二次增长，众包标签存在噪声。论文旨在以线性成本的时空状态空间建模，并结合面部几何先验提升鲁棒性。

Result: 在DFEW五折交叉验证中最佳折UAR/WAR为66.57%/76.46%，比最强对比方法低不超过0.3个百分点；MAFW为45.66%/62.27%，超过所有对比方法；FERV39k五划分均值42.30%/52.66%具有竞争力。DFEW五折均值64.07%/74.54%与近期大预训练模型相当，MAFW均值38.90%/54.37%低于最佳；DFEW配对t检验显示时间建模带来+1.61 UAR（单侧p=0.031），延迟接近ViT-B/16（47.4 vs. 46.8 ms/clip）。

Insight: 论文宣称的创新点包括线性复杂度的GroupMamba时空状态空间主干、通道亲和调制门控、基于归一化熵的几何锚点可靠性融合，以及静态到动态权重迁移和Triple Loss。客观看，其价值在于用线性复杂度状态空间模型降低长视频建模成本，并利用不确定性动态引入几何先验以增强噪声标签鲁棒性。

Abstract: Recognizing facial expressions in unconstrained video suffers from two coupled problems: self-attention grows quadratically in cost with clip length, and crowdsourced labels are noisy. We present GM-GReFEL, a spatiotemporal state-space network with a geometry-aware decision stage. Its Modulated GroupMamba backbone replaces a spatiotemporal Vision Transformer. Six channel groups are scanned separately, four over the frame plane and two over time, so the cost grows linearly with sequence length ($\mathcal{O}(N)$); after each layer, a Channel Affinity Modulation gate mixes the groups again. On top of the backbone, a GReFEL reliability module turns the normalized entropy of each prediction into a mixing weight between the classifier output and a vote from trainable geometric anchors, so that facial geometry counts for more when the network is unsure. The model is initialized by static-to-dynamic weight transfer and trained with a Triple Loss. In five-fold cross-validation, the best fold scores 66.57%/76.46% UAR/WAR on DFEW, 0.3 points or less below the strongest compared method, and 45.66%/62.27% on MAFW, ahead of all compared methods; on FERV39k the mean over five splits, 42.30%/52.66%, is competitive. The five-fold means are 64.07%/74.54% on DFEW, on par with recent large pre-trained models, and 38.90%/54.37% on MAFW, below the best of them. A paired $t$-test on DFEW indicates a modest benefit of temporal modeling (+1.61 UAR points, one-sided $p=0.031$). Attention stays on plausible facial regions, and latency is close to a ViT-B/16 baseline (47.4 vs.\ 46.8 ms per clip). Code: https://github.com/ardiytama/GroupMamba-DFER.

[64] RankAlign: Vision-Language Representation Alignment via Rank Transformation cs.CVPDF

Enzhe Zhao, Marco Fiorucci, Lamberto Ballan

TL;DR: 论文提出 RankAlign，一种基于秩变换的类级无配对视觉-语言表示对齐方法。它在置换匹配前将各模态内部的相似度核转换为序数关系表示，使匹配对相似度值的严格单调重参数化不敏感，并保持原有对应空间与匹配求解器不变。方法给出了精确恢复与稳定性理论保证，并在多种冻结视觉/语言编码器和数据集上验证了更优、更稳定的类级对应恢复，包括标签损坏和域偏移场景。

Details

Motivation: 视觉和语言模型可能收敛到兼容的隐空间几何，但在无跨模态监督下恢复未配对的类级对应仍困难；现有关系匹配直接比较模态内相似度核，而同一语义结构在不同模态中可能有不同的数值参数化。

Result: 在多种冻结视觉与语言编码器及多个数据集上，RankAlign 提升了类级对应恢复的效果与稳定性，并在标签损坏和域偏移下表现更稳健；论文还提供了精确恢复和部分序数一致下的稳定性保证。

Insight: 创新点在于匹配前对模态内相似度核做秩归一化，将其转为对单调重参数化不变的序数关系表示，同时不改变对应空间和匹配求解器，具有即插即用性；理论上刻画了序数一致性与对应恢复稳定性的关系。

Abstract: While the Platonic Representation Hypothesis posits that vision and language models converge toward compatible latent geometries, recovering class-level correspondences between unpaired modalities without cross-modal supervision remains challenging. Existing relational matching methods compare within-modality similarity kernels directly, although the same semantic structure may be represented through different numerical parameterizations across modalities. We propose RankAlign, a rank-based normalization strategy for class-level unpaired vision-language matching. RankAlign transforms each within-modality similarity kernel into an ordinal relational representation before permutation matching, while leaving the correspondence space and matching solver unchanged. The resulting representation is invariant to strictly increasing reparameterizations of similarity values while preserving the ordinal structure used for matching. We provide exact-recovery and stability guarantees that characterize correspondence recovery under both exact and partial ordinal agreement. Experiments across diverse frozen vision and language encoders and multiple datasets demonstrate improved and more stable class-level correspondence recovery, including under label corruption and domain shift.

[65] Beyond Watching and Searching: Benchmarking Agents for Video-centered Deep Research Reports cs.CV | cs.CLPDF

Jiahao Meng, Qize Yang, Yuxuan Wang, Yunfei Chu, Wensi Dai

TL;DR: 论文提出“以视频为中心的深度研究报告生成”任务：给定视频和用户请求，智能体需识别研究问题、检索外部多模态证据，并生成可交互且链接到相关视频时刻的研究报告。作者构建了OmniDR-Bench基准和基于技能的工作流，并提出Omni Atomic Rubric Trees（Omni-ARTs）进行细粒度评估。基准包含50个中英文任务、19个真实世界子类别和6414条评估标准。

Details

Motivation: 现有深度研究智能体基准主要面向开放网页证据，缺乏对以源视频为中心的研究报告生成能力的评估。该任务具有实际价值，可帮助用户定位视频关键时刻、验证论断、复现演示流程并基于视频内容做出决策。

Result: 在18种智能体配置上评估，最佳系统仅得63.3/100，说明当前智能体仍有明显提升空间。论文指出当前系统在内容覆盖广度、跨模态因果与逻辑分析、引用支持准确性方面仍存在不足。

Insight: 论文的创新点在于定义视频中心深度研究报告生成任务，提出技能化工作流、OmniDR-Bench基准以及Omni-ARTs原子化评分树，并通过ARC流程构建评估标准。客观来看，其创新在于将深度研究智能体从开放网页扩展到视频驱动的多模态证据整合与交互式报告生成，并引入细粒度、可审计的评估体系。

Abstract: Deep research agents synthesize open-web evidence into long reports, but current benchmarks do not evaluate research centered on a source video. We introduce Video-Centered Deep Research Report Generation. Given a video and user request, an agent identifies research questions, retrieves external multimodal evidence, and produces an interactive report linked to relevant video moments. This task has direct practical value because it helps users locate relevant moments, verify claims, reproduce demonstrated procedures, and make informed decisions from video content. We present a skill-based workflow and OmniDR-Bench for this task. For evaluation, we propose Omni Atomic Rubric Trees (Omni-ARTs). Each tree contains three dimensions, 23 shared directions, and task-specific atomic binary criteria. The directions test audiovisual grounding, multimodal evidence use, and interactive presentation. Audited Rubric Construction (ARC) builds these trees through independent evidence collection, automated checks, behavioral calibration, and iterative human review. OmniDR-Bench contains 50 Chinese and English tasks across 19 realworld subcategories. It includes 6,414 criteria. Across 18 agent configurations, the best system scores 63.3 out of 100. Current agents still need broader content coverage, stronger causal and logical analysis across modalities, and more accurate citation support. We further analyze models, harnesses, skill configurations, video categories, and research trajectories. The code and data are released at https://github.com/marinero4972/OmniDR-Bench.

[66] HiSTEVE 1.0: Harnessing Human-Interface VLA Agents for Open-World Games cs.CVPDF

Zecong Tang, Zian Guan, Yuncheng Guo, Zhanqiu Zhang, Yiwen Guo

TL;DR: 论文提出 HiSTEVE，一种面向 Minecraft 等开放世界游戏的分层 VLA 智能体：上层由能力感知的智能体框架、结构化记忆和 VLM 规划器负责任务编排，下层用单个 VLA 通过纯视觉感知和原生键鼠操作执行子任务。方法还引入策略耦合的任务完成检测器，并通过领域知识注入、模仿学习和环境反馈强化学习提升可靠性，最终在原子任务和长时程任务上取得优异表现。

Details

Motivation: 现有开放世界游戏智能体要么具备原生视觉运动控制但缺乏长时程自主性，要么依赖环境 API 或预定义技能进行高层规划，难以同时实现长时程自主与可靠的原始界面控制。论文旨在弥合这两类能力之间的 gap，让智能体像人类玩家一样在开放世界游戏中自主完成长时程任务。

Result: HiSTEVE 在 MineStudio Bench 上取得 85.0% 的原子任务成绩，在 OpenHA 上平均成功率达到 37.4%，均达到 SOTA 水平；同时能够从零开始完成制作铁镐等长时程任务。

Insight: 创新点在于将能力感知的分层编排与单一 VLA 原生控制结合，用结构化记忆和 VLM 规划器约束子任务选择，并用策略耦合的完成检测器可靠判定子任务结束；控制层通过分模式 RMSNorm 分支和独立鼠标移动 token 嵌入缓解 GUI 与非 GUI 交互干扰，并通过知识注入、模仿学习与强化学习提升可靠性和恢复能力。

Abstract: Building virtual embodied agents that operate in open-ended games like human players is a long-standing goal requiring both long-horizon autonomy and reliable native-interface control. As a procedurally generated open world, Minecraft provides a natural testbed for studying the integration of these capabilities. Existing agents typically provide either native visuomotor control without long-horizon autonomy or high-level planning executed through environment APIs or predefined skills. To bridge this gap, we propose HiSTEVE, a hierarchical VLA agent with a capability-aware agent harness. At the orchestration level, we equip the harness with structured memory for tracking progress and a VLM planner for selecting subtasks within the VLA’s capabilities. To reliably determine subtask completion, we further introduce a policy-coupled task completion state detector that conditions on VLA policy context to signal candidate completion. At the control level, a single VLA executes all subtasks with vision-only perception and native keyboard-and-mouse actions. To mitigate interference between heterogeneous GUI and non-GUI interactions, we introduce regime-specific RMSNorm branches and separate mouse-movement token embeddings. We also train the VLA through world knowledge injection, imitation learning, and reinforcement learning with environment feedback to improve reliability and recovery. HiSTEVE achieves state-of-the-art atomic-task performance, scoring 85.0% on MineStudio Bench and a 37.4% average success rate on OpenHA, while also completing long-horizon tasks such as crafting an iron pickaxe from scratch like human players.

[67] A Visual Grounding Benchmark for Retinal OCT with Multimodal Large Language Models cs.CVPDF

Ronald Fecso, José Morano, Botond Fazekas, Paul Widmann-Sedlnitzky, Dorota Himel

TL;DR: 论文提出首个视网膜OCT生物标志物视觉定位基准RetinaGroundBench，包含815张B-scan、 ophthalmologists标注的4,537个边界框，覆盖8种疾病的13种生物标志物；同时提出AggIoU指标评估碎片化预测的并集覆盖情况，并评测27个开源多模态大模型。为缩小定位差距，作者构建8B的RetGrounder，通过OCT特定微调融合定位与文本信息。

Details

Motivation: 多模态大模型虽能回答医学图像临床问题，但可信使用要求其先定位所报告的病灶，避免依赖虚假捷径；而MLLM在视网膜OCT临床发现上的视觉定位能力尚缺乏系统评测。

Result: 在RetinaGroundBench上，27个开源MLLM（0.8-38B）的视觉定位仍远未解决：最佳通用模型macro F1仅23.7%，召回率持续偏低，尽管生物标志物存在性问题的平衡准确率可达73%；8B的RetGrounder将定位F1提升至33.9%，相对提升43%。

Insight: 创新点在于构建首个OCT生物标志物视觉定位基准，并提出适应MLLM碎片化预测的AggIoU评价指标；同时用OCT特定策略融合定位与文本信息进行微调，为医学MLLM的可信视觉定位提供了评测与改进参考。

Abstract: Multimodal large language models (MLLMs) can answer clinical questions about medical images, but trustworthy use requires visual grounding: models should localize the pathology they report rather than rely on spurious shortcuts. Whether MLLMs can visually ground clinical findings in retinal optical coherence tomography (OCT), the most widely used imaging modality in ophthalmology, remains largely unexplored. To address this gap, we introduce RetinaGroundBench, the first benchmark for biomarker visual grounding in retinal OCT. It comprises 815 B-scans exhaustively annotated by ophthalmologists with 4,537 bounding boxes covering 13 biomarkers across 8 diseases. Because MLLMs often fragment a single lesion into multiple partial predictions, we propose AggIoU, a metric that evaluates whether the union of overlapping predicted boxes covers the annotated region. Across 27 open-source MLLMs (0.8-38B parameters), we find that visual grounding remains largely unsolved: the best general-purpose model achieves a macro F1 of just 23.7%, with consistently low recall, despite reaching up to 73% balanced accuracy on biomarker presence questions. To narrow this gap, we developed RetGrounder, an MLLM fine-tuned using an OCT-specific strategy integrating grounding and textual information. Despite its relatively small size (8B), RetGrounder improves grounding F1 to 33.9%, a 43% relative improvement. Our benchmark provides a reference point for measuring progress in the field of OCT diagnosis with MLLMs.

[68] Moving Regions, Not Moving Objects: Segmenting What Actually Moves under Camera Motion cs.CVPDF

Wenhua Jin, Tiancheng Chen, Sheng Tang, Jiahao Li, Yupeng Liu

TL;DR: 论文提出 MoRSe，将运动区域分割重新定义为区域级而非实例级任务：联合输入两帧 RGB 图像，直接预测真正发生运动的区域掩码，并区分真实场景运动与相机运动造成的表观运动。方法无需光流、点轨迹或额外掩码生成器，同时发布 MoRSe-5K 基准，用于评估静态、动态-静态和动态场景下的运动区域定位能力。

Details

Motivation: 现有运动目标分割通常在序列或实例层面把整个目标标记为运动，即使只有局部区域在动；基于实例级提议的瞬时方法也可能在目标未运动时错误输出掩码。论文认为运动不是实例的固有属性，必须通过至少两帧比较确定，因此需要区域级、双帧联合的运动分割。

Result: MoRSe 在 MoRSe-5K 基准上优于专门的运动目标分割方法和经运动阈值处理的稠密 4D 重建方法，并在无运动序列上产生最少的误报。

Insight: 创新点在于从“运动目标”转向“运动区域”，用双帧联合建模和单 RGB 编码器直接预测运动区域，避免多阶段基础模型堆叠及光流、点轨迹等额外输入；并构建覆盖相机与场景运动条件的 MoRSe-5K 基准来系统评估真实运动与表观运动的区分。

Abstract: Moving object segmentation is conventionally formulated at the sequence and object level, marking an entire instance as moving once it moves anywhere in a video. Recent instantaneous formulations distinguish moving from stationary intervals, but still generate masks at the object level through frozen segmentation foundation models. Because the mask is derived from an instance-level proposal, motion is typically attributed to the entire instance rather than localized to the regions that actually move. Moreover, such instance-level predictions can produce masks even when no part of the instance is actually moving, revealing a more fundamental issue: motion is not an intrinsic property of an instance; it can only be determined by comparing at least two frames. Building on this premise, we introduce MoRSe, which jointly interprets two frames to predict a moving-region mask, distinguishing actual scene motion from apparent motion induced by the camera. A single RGB encoder replaces multi-stage foundation stacks, so no optical flow, point tracks, or separate mask generator is needed at inference. We release MoRSe-5K, a benchmark of 42 sequences with Static, Dynamic-Static, and Dynamic settings designed to evaluate the model’s ability to distinguish and localize moving and stationary regions under different camera and scene motion conditions. MoRSe outperforms both specialized moving object segmentation methods and dense 4D reconstruction methods with motion-thresholded predictions, while producing the fewest false positives on sequences where nothing moves.

[69] Lightweight Frame Compensation for Real-Time Polyp Segmentation under Frame Skipping cs.CVPDF

Weihao Gao

TL;DR: 论文提出轻量级帧补偿模块LFC，用于帧跳过场景下的实时息肉分割。LFC仅77K参数、即插即用，置于冻结分割骨干前，通过三帧窗口和时空注意力融合连续帧生成补偿帧，并用不同随机种子模型生成的伪标签训练。实验表明其能恢复大部分因跳帧损失的精度，部分设置达到逐帧推理精度，同时保持2.05X–2.55X吞吐提升。

Details

Motivation: 实时结肠镜息肉分割受严格延迟预算限制，帧跳过虽能提升吞吐，但会引入运动模糊和病灶错位，降低时间一致性。论文旨在以轻量、可部署的方式补偿跳帧带来的精度退化。

Result: 在三个息肉视频基准上，LFC恢复了直接跳帧造成的大部分精度损失，部分设置达到逐帧推理精度，并保持2.05X–2.55X吞吐增益；同时分析了CNN、金字塔Transformer和全局注意力Transformer骨干对跳帧的敏感性谱，LFC对高容量Transformer骨干补偿增益最大。

Insight: 创新点在于将轻量时空注意力补偿模块前置到冻结分割骨干之前，实现即插即用且无需专家标注的伪标签训练；并通过跨架构时间稳定性分析揭示局部/全局聚合方式对跳帧敏感性的影响，为实时分割部署提供可借鉴思路。

Abstract: Real-time polyp segmentation in computer-assisted colonoscopy is hampered by the difficulty of deploying high-accuracy models under strict latency budgets. Frame skipping, which performs inference every K frames and reuses or interpolates the rest, improves throughput but introduces motion blur and lesion misalignment that degrade temporal consistency. To close this gap, we propose the Lightweight Frame Compensator (LFC), a 77K-parameter plug-and-play module inserted before a frozen segmentation backbone. Operating on a three-frame window, LFC fuses consecutive frames via spatio-temporal attention into a single compensated frame, and is trained only on pseudo-labels generated by a differently seeded model, without any expert annotation. Experiments on three polyp video benchmarks show that LFC recovers most of the accuracy degraded by direct frame skipping and, in several settings, reaches the accuracy of frame-by-frame inference, while retaining a 2.05X–2.55X throughput gain. We further evaluate the temporal stability of three backbones spanning CNN, pyramid Transformer, and global-attention Transformer architectures, revealing a spectrum of sensitivity to frame skipping that arises from how locally or globally each architecture aggregates misaligned input. The compensation gain of LFC follows this spectrum: it is largest for high-capacity Transformer backbones, which achieve the best static accuracy but suffer the most under skip-frame inference. LFC thus offers a lightweight, pluggable, and deployment-friendly route to real-time polyp segmentation.

[70] Reinforcing intra-operative tumour localisation with weak supervision from pre-operative appearances cs.CVPDF

Sidhart Sami, Ali Sharjeel, Sri Atluri, Miao Zhao, Caroline Roney

TL;DR: 论文提出一种跨模态弱监督框架：先在术前MR图像上训练肿瘤存在分类器，再利用其跨域弱信号引导强化学习智能体在术中CT上定位肝细胞癌肿瘤；同时比较直接迁移、少样本适应和合成迁移策略。合成迁移先将CT翻译为合成MR再定位，无需配对MR-CT训练图像。

Details

Motivation: 术中CT上肿瘤可见性差，导致标注数据难以获取；术前勾画经配准迁移又受术中形变和缺乏配对术前-术中数据限制，因此希望利用术前MR外观辅助术中CT肿瘤定位。

Result: 在外部HCC队列上，合成迁移优于直接迁移、少样本适应以及仅CT基线，展示了跨模态无配对训练下的定位效果。

Insight: 创新点在于用术前MR训练的肿瘤存在分类器产生跨域弱监督信号，并以其引导强化学习定位；合成迁移将CT转为合成MR以缩小域差距，实现无需配对MR-CT图像的术中肿瘤定位。

Abstract: Limited tumour visibility on intra-operative images complicates surgical navigation, including liver hepatocellular carcinoma (HCC) targeting on computed tomography (CT) imaging. This makes labelled datasets difficult to curate for deep learning, and transferring pre-operative delineations through registration remains challenging due to intra-operative deformation and lack of paired pre- and intra-operative data. We propose to transfer tumour appearances learned from pre-operative magnetic resonance (MR) images to guide localisation on intra-operative CT through weak supervision. A classifier is trained to classify tumour presence on pre-operative images. This is applied to windows on intra-operative images, and despite the domain gap, it assigns higher tumour-presence probabilities on average to tumour-containing windows. These weak cross-domain signals guide a reinforcement learning agent to localise tumours on intra-operative images. We compare direct transfer, few-shot adaptation and synthetic transfer (trained using unpaired MR and CT), which translates CT into synthetic MR before localisation. On an external HCC cohort, synthetic transfer outperformed other tested methods and CT-only baselines. Our framework enables cross-modality tumour localisation without paired MR-CT training images.

[71] ReCoG: Relational Concept Graph for Training-Free Personalization cs.CVPDF

Changseon Yu, Hyeokjun Kweon

TL;DR: 论文提出ReCoG，一种用于免训练个性化的关系概念图方法。它将个性化概念表示为节点，并用有向边显式编码概念对之间的判别性线索；推理时先检索候选短名单，再基于关系边进行成对比较以消除歧义，选择持续胜出的概念。论文还提出FINGER基准，用于测试易混淆概念集上的个性化识别。

Details

Motivation: 现有免训练个性化方法多将概念孤立处理，检索到的描述性证据可能合理但缺乏判别性，尤其在高度混淆的细粒度类别中。论文认为个性化识别本质上是关系性的，正确识别取决于目标概念与其最近替代概念之间的具体区别。

Result: 在FINGER基准上，ReCoG显著优于先前方法，且在越易混淆的设置中提升越大；在captioning和VQA等已有个性化基准上也取得一致最佳表现。

Insight: 创新点在于将概念间成对判别线索显式建模为关系图，并在推理阶段利用关系边进行候选比较来消除歧义；同时提出面向易混淆概念集的FINGER评测，强调关系证据对免训练个性化识别的价值。

Abstract: Training-free personalization adapts a frozen vision-language model (VLM) to recognize user-defined concepts from only a few exemplars by retrieving concept entries at inference time. While recent retriever-based methods enrich each concept with descriptive evidence, they largely treat concepts in isolation, so the retrieved information can be plausible yet non-discriminative, especially when the personalized concept set contains highly confusable, fine-grained categories. We argue that personalization is inherently relational: correct recognition depends on the specific distinctions between a concept and its nearest alternatives. To this end, we propose ReCoG, a Relational Concept Graph for retrieval-augmented personalization. ReCoG represents concepts as nodes and augments them with directed edges that explicitly capture discriminative cues between concept pairs, elicited by a frozen VLM during database construction. At inference, ReCoG retrieves a shortlist and resolves ambiguity via explicit pairwise comparisons conditioned on the corresponding relational edges, selecting the concept that consistently wins against competing candidates. We also introduce FINGER benchmark to test personalized recognition on confusable concept sets. Across FINGER, ReCoG substantially outperforms prior methods, with the largest gains in the most confusable regimes. ReCoG also achieves consistently best performance on established personalization benchmarks including captioning and VQA, confirming that relational evidence is broadly beneficial beyond fine-grained settings. Project Page: https://csy-98.github.io/ReCoG/

[72] Retrieval-Augmented Online Textual Memory for Episodic Memory Video Question Answering cs.CVPDF

Raffaele Cali, Giuseppe Lando, Rosario Forte, Antonino Furnari

TL;DR: 论文围绕第一人称长视频的情景记忆视频问答，在将视频转为带时间定位字幕的在线文本记忆框架下，系统研究检索单元、查询形式、上下文预算和重排序策略对问答的影响。最佳配置Parent先匹配细粒度字幕子块，再将完整父字幕提供给推理器，在QAEgo4D-Closed上取得56.4%准确率，并匹配强全记忆与检索基线，同时只暴露少量文本记忆。

Details

Motivation: 长第一人称视频需压缩为可高效检索且保留答题证据的记忆表示；论文关注在线文本记忆框架中检索对情景记忆视频问答的作用，并试图厘清检索策略如何影响最终答案。

Result: 最佳配置Parent在QAEgo4D-Closed多选题上达到56.4%准确率，匹配强全记忆和检索基线，但仅使用小部分文本记忆；分析发现增大检索预算虽提升时间Hit@K和Recall@K，却可能因视觉上合理但误导的干扰项和不确定字幕证据而降低准确率。

Insight: 创新点在于固定底层生成记忆，将检索作为主要变量系统比较检索单元、查询、预算与重排序；客观启示是检索增强的第一人称记忆不应只优化证据召回，还应关注检索上下文的可靠性、具体性和答案判别价值，推动超越原始字幕检索的证据感知结构化情景记忆。

Abstract: Long egocentric videos pose a central challenge for multimodal language models: the visual stream must be compressed into a memory representation that can be searched efficiently while still preserving the evidence needed for question answering. Building on an online textual-memory framework in which first-person videos are converted into temporally grounded captions, this work systematically studies the role of retrieval in episodic-memory video question answering. We evaluate how different retrieval units, query formulations, context budgets, and reranking strategies affect multiple-choice answering on QAEgo4D-Closed, isolating retrieval as the main variable while keeping the underlying generated memory fixed. Our best configuration, Parent, matches fine-grained caption subchunks, then provides their full parent captions to the reasoner. It achieves 56.4 % accuracy, matching strong full-memory and retrieval baselines while exposing only a small fraction of the textual memory at question time. More importantly, our analysis shows that better temporal coverage does not necessarily translate into better answers: increasing the retrieval budget improves temporal Hit@K and Recall@K, but can reduce accuracy by introducing visually plausible distractors and uncertain caption evidence. These findings suggest that retrieval-augmented egocentric memory should not be optimized only for evidence recall, but also for the reliability, specificity, and answer-discriminative value of the retrieved context. The study motivates more structured episodic memories that move beyond raw caption retrieval toward evidence-aware representations for long-form egocentric reasoning.

[73] Data-Driven Vision-Language Models for Remote Sensing: A Survey cs.CV | cs.LGPDF

Yue Zhou, Shujun Zhao, Xue Yang, Ruigang Li, Tianwen Zhang

TL;DR: 本文是一篇面向遥感领域的视觉语言模型综述，系统梳理了以数据为中心的遥感视觉语言数据集发展现状，并给出从数据构建、模型微调到性能评估的全流程实践指南。论文呼吁学术界为遥感各子领域构建更丰富、更高质量的视觉语言数据集，最终汇聚成任务覆盖广、地理分布多样的综合遥感视觉语言语料库。

Details

Motivation: 在大模型时代，遥感研究因计算资源约束，正从针对特定任务优化网络架构转向利用遥感垂直领域数据微调通用视觉语言模型，创新路径由模型中心转向数据中心。然而，社区仍缺乏系统性、以数据为中心的综述来指导高质量遥感视觉语言数据集的构建及基于其的微调与评估。

Result: 本文为综述性工作，未报告具体定量实验结果或在特定benchmark上的性能水平；其主要产出是对遥感视觉语言数据集发展现状的系统梳理，以及覆盖数据构建到微调评估全流程的实践指南。

Insight: 论文宣称的创新点在于从数据为中心视角系统综述遥感视觉语言模型，并提供面向大模型时代的全流程实践指南。客观来看，其价值在于推动遥感领域从模型架构创新转向数据生态建设，强调高质量、多样化、广地理覆盖的领域语料库对通用人工智能发展的重要基础作用。

Abstract: During the deep learning era, innovations in remote sensing (RS) vision models primarily focused on optimizing network architectures for specific tasks and conducting end-to-end training. However, with the advent of the Vision-Language Models (VLMs) era, constrained by substantial computational resource demands, the mainstream research paradigm has shifted towards adapting general-purpose VLMs using RS vertical domain data for fine-tuning, rather than modifying their core architectures. This transition marks a change in the innovation pathway from model-centric to data-centric. Despite the increasing importance of data, the academic community still lacks a systematic, data-centric survey to guide researchers in constructing high-quality RS vision-language datasets for various application scenarios and in conducting effective model fine-tuning and performance evaluation based on these datasets. To address this gap, this paper presents a systematic review of the development status of RS datasets tailored for VLMs. It further provides a comprehensive practical guide for RS researchers in the large model era, covering the entire pipeline from data construction to fine-tuning and evaluation. This paper aims to encourage the academic community to build more diverse and higher-quality vision-language datasets for specialized RS sub-fields. Ultimately, the goal is to converge these efforts into a comprehensive RS vision-language corpus that offers extensive task coverage and a wide geographical distribution, thereby laying a solid data foundation for advancing towards general artificial intelligence in RS.

[74] MVLoMa: Multi-View Local Feature Matching with Any Geometry Foundation Model cs.CVPDF

Xinyang Lu, Shengzhe Wei, Sheng Tang, Weiqi Zhang, Juntong Fang

TL;DR: MVLoMa 是一种面向多视图的稀疏星型图局部特征匹配器，将 LoMa 从图像对扩展到中心图像与多个邻居的联合匹配。它先用冻结的多视图几何基础模型获取跨视图几何上下文并注入局部描述子，再通过自注意力和双向中心-邻居交叉注意力一次前向匹配所有边。训练时利用深度有效且相互可见的稠密warp构造正样本，并惩罚违反对极几何的高置信匹配。

Details

Motivation: 传统局部特征匹配通常按图像对独立处理，而SfM和视觉定位常需将一张参考图与多个检索或共视邻居匹配。逐边应用成对匹配器会重复参考侧计算，且无法利用其余观测提供的互补多视图几何证据。

Result: 在HyperSim、ScanNet++、Mip-NeRF 360和Tanks and Tuples的25视图星型图上，MVLoMa在三个阈值下取得最佳宏平均相对位姿AUC（58.65/85.41/90.99）；匹配器比顺序评估24条中心-邻居边快2.20倍。方法支持DA3、π^3、VGGT和VGGT-Ω等冻结几何编码器，其中DA3-Giant在编码器消融指标上最佳。

Insight: 创新点在于将成对局部特征匹配扩展到星型多视图联合匹配，把冻结多视图几何基础模型的跨视图上下文注入局部描述子，并单次前向完成中心到所有邻居的匹配；监督中结合可见性过滤、几何负样本和对极几何约束。客观看，核心价值是复用基础几何模型并消除参考端重复计算。

Abstract: Local feature matching is typically formulated for image pairs, although structure-from-motion and visual localization commonly match one reference image against many retrieved or covisible neighbours. Applying a pairwise matcher independently to these centre-neighbour edges duplicates reference-side computation and prevents the matcher from exploiting complementary multi-view geometric evidence from the remaining observations. To address these limitations, we introduce MVLoMa, a sparse star-graph matcher that extends LoMa from image pairs to multiple views. MVLoMa first uses a frozen multi-view geometry model to jointly describe all images in a star, then injects the resulting cross-view geometric context into local descriptors. It then matches the centre image to all neighbours in a single forward pass through per-view self-attention and bidirectional centre-neighbour cross-attention. For supervision, we derive positive correspondences only from depth-valid, mutually visible dense warps and explicitly penalize high-confidence matches that violate epipolar geometry. On 25-view star graphs from HyperSim, ScanNet++, Mip-NeRF 360, and Tanks and Temples, MVLoMa obtains the best macro-average relative-pose AUC among the compared methods at all three thresholds (58.65/85.41/90.99). Its matcher is 2.20 times faster than sequential evaluation of the 24 centre-neighbour edges. The same design supports four tested frozen geometry encoders – DA3, $π^3$, VGGT, and VGGT-$Ω$ – with DA3-Giant performing best on all reported encoder-ablation metrics. Controlled ablations further isolate the effects of geometry-aware descriptors, visibility filtering, geometric negative supervision, and joint multi-view inference.

[75] WeVisOne-VL: Unified Visual Perception from Objects to Pixels cs.CVPDF

Tianming Liang, Junyi Pan, Yuxi Li, Canmiao Fu, Zhipeng Huang

TL;DR: 论文提出WeVisOne-VL，一个统一的视觉语言模型，通过结构化下一个token预测整合对象级和像素级感知，支持目标检测、指代定位、计数、OCR、分割和抠图等多种任务，实现从点到alpha matte的任意粒度定位。模型通过特殊 token将轻量级像素解码器集成到VLM中生成实例级像素输出，并采用三阶段课程SFT和混合任务RL训练。在超过30个基准上取得领先性能。

Details

Motivation: 现有视觉语言模型对细粒度视觉感知不可靠，难以满足真实世界部署中对可靠感知系统的需求。论文旨在统一对象级和像素级感知，提升VLM的细粒度定位能力。

Result: 在超过30个基准测试中，WeVisOne-VL在对象级和像素级任务上均达到领先性能，同时对对象缺失查询具有稳健的拒绝能力。摘要未提供具体数值，但宣称领先。

Insight: 创新点包括：通过 token将轻量级像素解码器集成到VLM中实现实例级像素输出；支持从点到alpha matte的任意粒度定位；采用三阶段课程SFT和混合任务RL训练策略。客观分析，其统一架构和训练策略对结合像素级任务与VLM具有借鉴意义。

Abstract: Existing vision-language models (VLMs) remain unreliable for fine-grained visual perception. In this report, we present WeVisOne-VL (WeChat Vision All-in-One), a powerful VLM that unifies object-level and pixel-level perception through structured next-token prediction. It supports multiple visual tasks including object detection, referring localization, counting, OCR, segmentation, and matting, enabling localization at any user-specific granularity, from points and boxes to masks and alpha mattes. Architecturally, we integrate a lightweight pixel decoder into the VLM via a special token to generate instance-level pixel outputs. We train the model with a three-stage curriculum SFT followed by mixed-task RL. Across over 30 benchmarks, WeVisOne-VL demonstrates leading performance on both object-level and pixel-level tasks, while maintaining robust rejection for object-absent queries. By releasing WeVisOne-VL to the open-source community, we hope to advance VLMs from broad visual understanding toward truly reliable perception systems for real-world deployment.

[76] A Task-Centric Survey of Long-Document Visual Question Answering: Taxonomy, Evolution, and Future Directions cs.CV | cs.CL | cs.IRPDF

Yufei Tong, Guanjie Cheng, Tianle Wu, Leilei Zhang, Hang Zheng

TL;DR: 本文是一篇关于长文档视觉问答（Long-Document VQA）的任务中心综述，将任务分解为文档解析、证据组织与检索、生成器组织与学习、评估四个组件，并构建细粒度分类体系，梳理各组件动机、技术回应及跨组件联系。文章重点关注显式检索，同时覆盖直接阅读，并通过时间分析总结研究重点从多模态内容理解向充分高保真证据获取与持续证据管理的扩展。

Details

Motivation: 现有综述多强调模型架构或技术路线，较少从任务瓶颈出发解释技术选择和研究优先级如何形成。本文旨在以任务为中心重新组织长文档视觉问答研究，揭示各环节瓶颈对技术方案的影响。

Result: 作为综述论文，本文未提出新模型或报告benchmark定量结果，而是提供任务分解、细粒度分类体系、时间演化分析和未来方向展望。

Insight: 创新点在于以任务为中心的组件化综述框架，系统追踪文档解析、证据检索、生成器学习和评估之间的动机与技术联系，并关注显式检索与直接阅读的关系。其可借鉴之处是围绕证据充分性、可验证与可纠正理解、预算感知协调及持续文档理解提出未来研究议程。

Abstract: Visual document understanding is essential for acquiring knowledge from visually rich documents and is attracting growing research interest. Existing surveys emphasize architectures or technical approaches, with limited attention to how task bottlenecks shape technical choices and research priorities. We present a task-centric survey that decomposes long-document visual question answering (VQA) into four components: document parsing, evidence organization and retrieval, generator organization and learning, and evaluation. For each, we develop a fine-grained taxonomy and trace motivations, technical responses, and cross-component connections. We focus on explicit retrieval while covering direct reading. Our temporal analysis traces expanding priorities from multimodal content understanding to sufficient, high-fidelity evidence acquisition and sustained evidence management. We outline future directions in evidence sufficiency, verifiable and correctable understanding, and budget-aware coordination, with an outlook on continual document understanding.

[77] AnyGameRenderer: Entity-Consistent Generative Rendering Across Games and an Entity-Level Benchmark cs.CVPDF

Ruicong Liu, Liangyang Ouyang, Fengbo Lan, Zhixiang Wang, Kaipeng Zhang

TL;DR: 论文提出 AnyGameRenderer，一种视频到视频的生成式渲染器，可将 AI 编码智能体生成的“程序员美术”游戏画面转换为高质量角色、动作和特效画面，同时保持布局、相机以及每个实体的位置与出现/消失时序一致。方法通过 LoRA 微调 33B 的 MiniMax-H3 ref2va 视频生成模型，并使用智能体游戏、Unreal Engine 合成数据和真实游戏录制构建约 1 万对训练片段。论文还提出实体级 benchmark 评估实体对应关系。

Details

Motivation: AI 智能体已能快速生成可玩的小游戏，但画面通常只是灰盒、彩色几何体和语义色块，视觉质量差；现有生成式渲染难以在跨游戏类型时保持布局、相机和实体时序一致。

Result: 在包含 59 个留出案例、753 个实体的实体级 benchmark 上，相比基座模型，AnyGameRenderer 将时间匹配率从 50.9% 提升到 62.9%，漏检数量变化从 34.7% 降到 21.3%，并减少实体幻觉和未渲染情况。

Insight: 论文宣称的创新点在于实体一致的跨游戏生成式渲染、仅用普通视频作为游戏状态输入且无需 G-buffer/类别标签/低模代理，以及提出实体级评测 benchmark。客观看，其贡献还包括基于大视频生成模型的 LoRA 适配和多源配对数据构建，将“实体对应保持”作为生成渲染的核心约束与评价指标。

Abstract: AI coding agents can now build small playable games in minutes, but the results usually look like programmer art: gray blockouts, colored geometric primitives, and flat semantic blobs. We present AnyGameRenderer, a video-to-video generative renderer that turns such footage into high-quality visuals with characters, motion detail, and effects. Throughout, it keeps the layout, camera, and every entity’s position and timing intact. Our AnyGameRenderer generalizes across a wide range of genres that agents build, such as tower defense, auto-battlers, top-down survivors, turn-based battles, kart racing, and crowd runners. AnyGameRenderer takes plain video as its game-state input, with no complex representations such as engine G-buffers, per-object class labels, or low-poly proxies. Gray blockouts, colored primitives, semantic maps, and wireframes all work as input. It also preserves entity correspondence: group counts stay correct, and entities appear and disappear on the right frame. We implement AnyGameRenderer by LoRA fine-tuning MiniMax-H3 ref2va, a 33B reference-video-conditioned video generation model. Training uses $\sim$10k paired clips from three sources: (i) multi-stream rendering data from 38 agent-made games exported from GEAR, an AI game-creation tool; (ii) two-stream rendering data from Unreal Engine, with programmatically controlled crowds, count changes, and entity scales; and (iii) real gameplay recordings, with pairs reconstructed via 3D-box detection. To evaluate the entity correspondence, we propose an entity-level benchmark of 59 held-out cases with 753 entities, scored against ground-truth boxes. Compared with the base model, AnyGameRenderer better preserves entity correspondence: temporal match rises from 50.9% to 62.9%, missed count changes fall from 34.7% to 21.3%, and fewer entities are hallucinated or left unrendered.

[78] Ask About the Right Moment: Training-Free Object Identity Reasoning for Video-Language Models cs.CVPDF

Di Yang, Xiaochang Li, Muyang Liu, Yufei Wang, Yanhai Xiong

TL;DR: 论文提出免训练方法“Track, Then Bind”，解决视频语言模型跨帧物体身份推理问题。方法先用可提示视频跟踪器跟踪候选对象并编号，再用冻结VLM回答静态问题将目标绑定到轨迹，最后从跟踪状态文本推导答案。在VET-Bench、Perception Test和TOC-Bench上显著优于现有模型和直接问答方式。

Details

Motivation: 视频语言模型在单帧中能识别物体，但无法在时间上保持物体身份，导致在需要追踪物体（如贝壳游戏）的任务上表现接近随机。论文旨在无需训练即可实现跨帧物体身份推理。

Result: 在VET-Bench两个任务上解决（对比微调模型92%/90%，VLM投票30-32%）；Perception Test最难杯子游戏达55.4%（最佳已发表41.5%）；TOC-Bench达48.6%（直接回答37.1%）。使用Qwen3-VL-32B，单一配置覆盖所有基准。

Insight: 创新点在于将语义身份绑定（在证据可见时固定身份）与时间身份维护（由跟踪器携带）分离，实现免训练；使用跟踪器绘制轨迹编号辅助VLM绑定；从跟踪状态文本推导答案避免模型重新解释视频。客观分析：方法简单有效，利用现成跟踪器和VLM，无需训练，具有通用性。

Abstract: Many questions about a video concern one physical object. Where does the ball end up after the cups are shuffled? Which card is the queen? How often was the mug covered? Video-language models (Video-LMs) recognise such objects in single frames but do not keep them apart over time. On real shell-game videos, open models from 8B to 235B parameters, their thinking variants, a reinforcement-learned tool agent and three training-free tool agents all stay within ten points of chance. We separate semantic identity binding from temporal identity maintenance. An object’s identity should be fixed at a moment when the evidence for it is visible, and carried afterwards by a tracker. \emph{Track, Then Bind} implements this principle without training. A promptable video tracker follows every candidate object and draws it with a track number. A frozen vision-language model answers a static question about the identifying evidence, which binds the target to a track. The answer is then derived from the tracked state in text, without asking the model to reinterpret the original video during read-out. A single configuration covers all benchmarks. With Qwen3-VL-32B it solves both VET-Bench tasks, where a fine-tuned trajectory model reaches 92% and 90% and the same VLM voting over 16 samples stays at 30–32%. It answers 55.4% of the hardest Perception Test cup games, where the best published model answers 41.5%, and, using track-derived timelines, 48.6% of TOC-Bench questions, against 37.1% for the same model answering directly. Drawing the same tracks on the video and asking the question directly falls 12–19 points short of the agent on real cup games.

[79] MoG-VLM: Efficient Video Anomaly Detection through Motion-Guided Vision-Language Models cs.CV | cs.AI | cs.LGPDF

Khaled Waleed Dawoud, Muhammad Zaigham Zaheer, Khan Muhammad, Muhammad Haris Khan

TL;DR: 论文提出MoG-VLM，一个面向视频异常检测的运动引导视觉语言模型框架。该方法利用运动作为轻量控制信号，通过时序梯度幅值（TGM）进行内容感知采样，并采用分层双流设计，仅在异常可能性升高的帧附近触发密集多帧VLM推理，最后融合瞬时语义线索与时间动作动态。目标是在零样本、免训练设置下减少VLM帧级推理开销，同时保持或提升异常检测性能。

Details

Motivation: 现有VLM用于视频异常检测时密集逐帧推理难以实时部署，而均匀采样忽略视频内容动态，既可能在静态片段浪费计算，也可能漏掉局部异常。论文希望用轻量运动信号自适应决定何时需要VLM推理，从而更高效地分配计算资源。

Result: 在XD-Violence、UCF-Crime、MSAD和ShanghaiTech上分别达到91.91%、84.30%、87.88%和78.96%的AUC-ROC，并在三个benchmark的零样本、免训练设置下取得新SOTA。XD-Violence上较此前最佳提升0.57%，同时实现1.31倍推理加速。

Insight: 摘要宣称的创新点在于将运动仅作为轻量控制信号、提出TGM内容感知采样，以及anchor触发的上下文窗口扩展与早停机制。客观来看，其核心价值是将VLM容量按内容动态稀疏分配，证明“更少帧但更智能地处理”比均匀稠密推理更有效，同时保持零样本免训练实用性。

Abstract: Vision-Language Models (VLMs) have shown strong potential for video anomaly detection, but dense frame-level inference makes them impractical for real-time deployment. Existing acceleration commonly relies on uniform sampling, which ignores content dynamics and can either waste computation on static segments or miss localized anomalies. We propose MoG-VLM a motion-adaptive anomaly detection framework that leverages motion purely as a lightweight control signal to decide when VLM inference is necessary. To this end, we introduce Temporal Gradient Magnitude (TGM) which performs content-aware sampling to concentrate computation on informative regions while avoiding redundant processing. We further introduce a hierarchical dual-stream design in which dense multi-frame VLM reasoning is activated only around frames with elevated anomaly likelihood, via anchor-triggered context-window expansion with early termination. A fusion module then reconciles these instantaneous semantic cues with temporal action dynamics. Across four benchmarks, our method achieves 91.91%, 84.30%, 87.88%, and 78.96% AUC-ROC on XD-Violence, UCF-Crime, MSAD, and ShanghaiTech, respectively, establishing new state of the art on three benchmarks under the zero-shot, training-free setting. On XD-Violence, we improve over the previous best result by +0.57% while delivering a 1.31x inference speedup. These results demonstrate that content-aware allocation of VLM capacity, processing fewer frames more intelligently rather than all frames uniformly, is a more effective axis of progress than uniformly densifying VLM inference across frames. Code is available on GitHub: https://github.com/kdawoud91/MoG-VLM.

[80] AgentVQA: A Unified Benchmark for Agentic Visual Understanding cs.CVPDF

Advait Gupta, Sachin Konan, Tom Yin, Gabriel Sarch, Zhuang Liu

TL;DR: 论文提出 AgentVQA，一个用于评估视觉语言模型在智能体场景中视觉理解能力的统一离线基准，覆盖 Web Agents、Robotics、Egocentric Videos、Games 和 Spatial Planning 五个领域。作者从 23 个源数据集中人工筛选并改编 1000 个具有挑战性的真实智能体样例，构建可扩展的离线 VQA 评测，并验证其模型排名与对应在线评测具有一致性。

Details

Motivation: 现有在线智能体评测成本高、难以规模化，而离线 VQA 基准又有限且碎片化，因此需要统一、可扩展且能反映真实交互场景的离线基准来评估 VLM 的智能体视觉理解能力。

Result: AgentVQA 覆盖 5 个智能体领域、23 个源数据集和 1000 个样例；在 22 个被评估模型中最佳准确率仅为 58.5%，且基准上的模型排名能够跟踪对应在线评测结果。

Insight: 论文的创新点在于构建跨五类智能体领域的统一离线 VQA 基准，并从真实交互数据中筛选改编样例且验证与在线评测排名一致；跨域错误分析进一步指出，主要瓶颈不是任务误解或下游逻辑推理，而是对当前及演化视觉状态的 grounding 与维持，包括 UI 元素定位、运动理解、时序状态跟踪和 3D 空间关系推断。

Abstract: Vision-language models (VLMs) can perform a broad range of tasks across diverse settings, yet their performance in agentic contexts remains poorly understood. Although online evaluations in simulators and real-world environments are essential for measuring interactive agent performance, they are costly and challenging to scale. Complementary offline VQA benchmarks can enable faster and more reliable comparisons across agentic domains, yet existing benchmarks remain limited and fragmented. To address this gap, we introduce AgentVQA, an offline benchmark for evaluating VLMs on realistic agentic scenarios across five agent domains: Web Agents, Robotics, Egocentric Videos, Games, and Spatial Planning. We build AgentVQA by manually selecting 1000 challenging, real-world agent examples from 23 source datasets and adapting them for reliable offline evaluation. The resulting benchmark preserves realistic scenarios from interactive environments across diverse domains, and its model rankings track those from corresponding online evaluations. Across the 22 evaluated models, the best achieves only 58.5% accuracy. Our cross-domain analysis shows that errors are driven primarily by failures to ground and maintain accurate representations of the current and evolving visual state, including localizing relevant UI elements, interpreting motion, tracking state changes over time, and inferring 3D spatial relationships, rather than by task misinterpretation or downstream logical reasoning.

[81] LACToS: Local Anomaly Detection via Conditional Typicality of Superpixels cs.CV | cs.LGPDF

Jihoon Hong, Alireza Kheirandish, Rohan Sanda, Sara Fridovich-Keil

TL;DR: 论文提出一种局部异常检测方法LACToS，用于识别图像中空间局部的异常或分布外区域。该方法基于统计典型性建模，并通过将像素分组为超像素、使用高效代理估计典型性来提升可计算性。作者在医学影像、人脸和X射线安检三个场景中验证了方法的鲁棒性。

Details

Motivation: 现有局部异常检测方法多针对工业缺陷检测设计，在其他应用场景中性能可能明显下降，尤其是当正常样本分布呈现多模态时。论文旨在处理更复杂的inlier分布，提升局部OOD检测的泛化性和鲁棒性。

Result: 论文在医学影像、人脸和X射线安检三个不同应用上评估LACToS，结果显示其相较已有方法表现更稳健；摘要未给出具体数值或明确SOTA结论。

Insight: 创新点在于从统计典型性角度进行局部OOD检测，并引入超像素分组和计算高效代理来使条件典型性估计可行。客观来看，其价值主要在于面向多模态正常分布的场景泛化能力，以及将像素级异常检测转化为超像素级典型性建模的思路。

Abstract: Local anomaly detection is the task of identifying spatially confined anomalous or out-of-distribution (OOD) regions within an otherwise in-distribution (ID) image. Most existing methods are designed for a specific context, with a strong focus on industrial defect detection, where they achieve excellent performance on well-established benchmarks. However, we observe that these methods can exhibit significant performance drops in other contexts, especially when the inlier distribution is multimodal. To account for this inlier complexity, we propose LACToS, a principled local OOD detection method based on statistical typicality and made tractable through two innovations: (i) grouping pixels into superpixels and (ii) using a computationally efficient proxy to estimate typicality. We evaluate LACToS across three diverse applications spanning medical imaging, human faces, and X-ray security screening. LACToS performs more robustly than prior methods across these benchmarks. The code is released here: https://github.com/voilalab/LACToS.

[82] CellTrack-Bench: Microscopic Cell Tracking Benchmark for Vision-Language Models cs.CVPDF

Ilan Theodoro, Jordão Bragantini, Loïc A. Royer

TL;DR: 论文提出了CellTrack-Bench，一个用于评估视觉语言模型（VLMs）在显微镜细胞追踪任务上性能的基准。该基准从15个标准细胞追踪数据集构建，涵盖四种成像模态，包含1,537个问题（对应9,041个细胞链接决策），并筛选出需要更多推理的困难子集。作者评估了不同环境框架下的9个前沿闭源模型和13个开放权重模型，并详细分析了失败模式。

Details

Motivation: 大型语言模型在科学领域（如细胞追踪）的通用准确性相比专用算法仍存疑，且细胞追踪需要时空意识，但缺乏针对VLMs的标准化评估基准。

Result: 论文构建了CellTrack-Bench基准并系统评估了多种VLMs，但摘要未报告具体定量性能（如准确率或SOTA水平），主要贡献是基准和失败模式分析。

Insight: 创新点在于构建了首个针对VLMs的细胞追踪基准，涵盖多模态、大规模问题，并通过识别不同算法共同失败来筛选困难样本。其困难子集设计和系统评估框架为VLMs在科学领域的应用提供了可借鉴的评测方法。

Abstract: Large language models have shown outstanding performance on text-first tasks such as coding, math, and writing. Recent efforts are shifting models’ training to include scientific problems that often live in highly specialized domains beyond text, such as protein structure and biological images. However, in these scenarios, their general-purpose accuracy compared with specialized algorithms remains in question. One such domain is cell tracking, which requires spatiotemporal awareness of cell dynamics. To evaluate this, we created a new benchmark, CellTrack-Bench, which assesses Vision-Language Models’ (VLMs) cell-tracking-related performance against specialized trackers. The full benchmark draws from 15 standard cell-tracking datasets across four imaging modalities and comprises 1,537 problems corresponding to 9,041 cell-linking decisions. We selected a hard subset requiring greater reasoning effort by identifying failures shared across distinct algorithmic trackers. With this proposed dataset, we: (i) evaluated different environment harnesses for VLMs; (ii) benchmarked 9 frontier closed-source models and 13 open-weight models; and (iii) provided a detailed analysis of their failure modes. Code, the stored model outputs, and a pipeline that regenerates the benchmark from the source datasets are available at https://github.com/royerlab/celltrack-bench.

[83] F2G-Pose: Geometry-Aware Foundation Feature Lifting for Direct RGB-D Category-Level Object Pose Estimation cs.CVPDF

Jinyu Zhang, Haitao Lin, Jiashu Hou, Xiangyang Xue, Yanwei Fu

TL;DR: 论文提出 F2G-Pose，一种用于类别级 RGB-D 物体位姿估计的实时直接单次前向框架。它从单帧 RGB-D 观测中同时预测物体位姿、度量尺寸和相机坐标系下的补全形状，通过将稠密视觉基础特征提升到部分点云并结合几何编码与专家混合 Transformer 实现。

Details

Motivation: 类别级 RGB-D 位姿估计受类内差异、对称性和部分观测影响而高度歧义；现有中间结构方法易产生误差传播，扩散模型方法推理延迟高且后处理复杂，难以兼顾可靠性与效率。

Result: 方法在 SOPE 和 ROPE 上达到 SOTA 精度，仅位姿推理可达 28 FPS；在 HouseCat6D 上展示较强相机坐标系几何，在 HANDAL 上展示真实物体迁移能力，并进行了真实机器人操作实验。

Insight: 创新点在于将冻结的视觉基础模型稠密特征提升到点云，结合 DGCNN 局部几何 token 化与专家混合 Transformer 建模物体级结构；同时联合尺寸预测和相机系形状补全，以稠密几何监督增强度量一致性，并用单次前向直接预测避免扩散式候选提取。

Abstract: This paper studies category-level RGB-D object pose estimation, which recovers an object’s 3D rotation and translation without instance-specific CAD models or reference views. This task is highly ambiguous due to intra-class variations, symmetries, and partial observations. Current solutions struggle to balance reliability and efficiency: methods based on intermediate structures are prone to error propagation, while diffusion models suffer from high inference latency and complex post-hoc candidate extraction. To address this, we present F2G-Pose, a real-time direct, single-pass framework that predicts category-level pose, metric size, and camera-frame completed shape from a single RGB-D observation in one feed-forward pass. F2G-Pose lifts dense visual foundation features to partial point clouds, encodes local geometry with DGCNN tokenization, and integrates object-level structure with a Mixture-of-Experts Transformer. Joint size prediction and camera-frame shape completion encourage metric consistency through dense geometric supervision. With its task-specific network trained only on pose-annotated synthetic SOPE data and its frozen RADIO backbone externally pretrained, F2G-Pose achieves state-of-the-art accuracy on SOPE and ROPE while running at 28 FPS for pose-only inference, and further demonstrates strong camera-frame geometry on HouseCat6D and real-object transfer on HANDAL. Analyses of latency, robustness, shape completion, and GenPose++ candidate behavior further demonstrate the efficiency and reliability of F2G-Pose, with real-world manipulation experiments demonstrating deployment potential under segmented RGB-D inputs. The project page, code, and pretrained models are made available at https://jrryzh.github.io/F2G-Pose/.

[84] HiP3R: Hierarchical Pose Graphs for Long-Horizon 3D Reconstruction cs.CVPDF

Zunnan Xu, Shangchen Zhou, Yihang Luo, Chen Change Loy

TL;DR: 论文提出 HiP3R，用层次化位姿图解决长视频 3D 重建中分块前馈推理导致的局部重建坐标系不一致和重复相机估计问题。方法先优化块级 Sim(3) 图确定全局布局与尺度，再固定该布局逐帧估计共享 SE(3) 相机，将重叠窗口预测融合为统一轨迹。实验表明其能提升相机轨迹和 3D 重建效果，并适用于 chunk-based 与 streaming 后端。

Details

Motivation: 长视频包含数千帧，前馈模型受计算和内存限制只能处理短窗口或重叠分块，导致局部重建坐标不一致、共享相机重复估计，需要传播相邻重叠和远距离回环约束以实现全局一致。

Result: 在多个 benchmark 上验证了 HiP3R 对相机轨迹和 3D 重建的改进效果；摘要未给出具体定量指标，但强调无需额外训练或微调即可用于分块式和流式后端。

Insight: 创新点在于用层次化位姿图平衡块级 Sim(3) 图的全局效率与尺度鲁棒性，以及帧级 SE(3) 优化的局部灵活性：先固定全局布局再逐帧细化共享相机，从而融合重叠窗口预测并保持度量尺度。

Abstract: Feed-forward models accurately estimate camera motion and 3D geometry from short video windows. However, computational and memory costs prevent direct inference on sequences containing thousands of frames. Long videos must therefore be processed as overlapping chunks. This division produces independent local reconstructions with inconsistent coordinate frames and duplicate estimates of shared cameras. Achieving global consistency requires propagating constraints from adjacent overlaps and distant revisits across the sequence. This requirement naturally motivates pose graph optimization. However, the choice of graph level creates a trade-off. Chunk-level graphs efficiently resolve scale drift and close large loops. Yet, their rigid transformations cannot correct internal errors or conflicting frame predictions. Conversely, frame-level optimization offers local flexibility, but scaling it globally is computationally expensive and can break metric scale. HiP3R resolves this trade-off using a hierarchical pose graph. HiP3R first optimizes a robust chunk-level $\mathrm{Sim}(3)$ graph to establish the global layout and scale. It then holds this layout fixed and estimates one shared $\mathrm{SE}(3)$ camera per frame, turning overlapping window predictions into a single trajectory. Experiments across multiple benchmarks demonstrate the effectiveness of HiP3R in improving camera trajectories and 3D reconstruction. Our method applies to both chunk-based and streaming backends without additional training or fine-tuning. The codes are available at https://kkakkkka.github.io/HiP3R.

[85] WorldForge V2: Eliciting Camera-Trajectory Alignment in Video Diffusion Models cs.CVPDF

Chenxi Song, Yanming Yang, Tong Zhao, Ruibo Li, Joey Tianyi Zhou

TL;DR: 论文提出WorldForge V2，在预训练视频扩散模型中实现精确相机轨迹控制，同时保留场景外观与物体运动。方法结合基于warped references的训练无关迭代引导、运动相关latent通道选择、双路径纠错和v-cache加速，并引入Trajectory LoRA与两阶段训练，以极少数据实现可靠相机控制。

Details

Motivation: 现有视频扩散模型在空间任务中受限于相机控制不精确、时空不一致以及相机运动与场景动态纠缠；任务特定微调和后处理warping又会引入伪影、泛化差或计算成本高。

Result: 在多个数据集上的定量与定性评估表明，该方法在相机控制视频生成中实现了SOTA的生成质量和轨迹精度。

Insight: 创新点在于将训练无关引导与轻量Trajectory LoRA结合，通过光流分析选择性引导运动相关latent通道，并用双路径比较纠正warped reference伪影；v-cache降低细化成本，两阶段训练使单个训练场景即可泛化到未见场景。客观看，其价值在于较好分离运动控制与外观、降低训练数据需求并兼顾生成质量。

Abstract: Video diffusion models have learned rich 3D/4D priors, making them promising for scene generation and video re-rendering. However, their use in spatial tasks is limited by imprecise camera control, spatial and temporal inconsistencies, and entangled camera motion with scene dynamics. Existing approaches, such as task-specific fine-tuning and post-process warping, often introduce visual artifacts, generalize poorly, or incur high computational costs. We propose WorldForge V2 to achieve accurate camera control while preserving the generative capabilities of pretrained video models. We first develop a training-free guidance framework using warped references to iteratively refine generation along a target camera path. Optical-flow analysis identifies motion-related latent channels for selective guidance, helping separate motion control from appearance. A dual-path strategy further compares guided and unguided predictions to correct artifacts from imperfect references. Together, these components guide viewpoint changes while preserving scene appearance and object motion. To reduce refinement costs, v-cache approximates the correction using the preceding denoising prediction, avoiding an additional model evaluation. We further extend motion localization from latent channels to network blocks and introduce Trajectory LoRA, which concentrates adaptation on blocks most responsive to trajectory guidance. A two-stage training strategy teaches these adapters to denoise the states encountered during guided generation, using limited supervision effectively. This enables reliable camera control with very little training data, with a single training scene already supporting generalization to unseen scenes. Quantitative and qualitative evaluations across multiple datasets show that our method achieves state-of-the-art generation quality and trajectory accuracy in camera-controlled video generation.

[86] UniVOE: A Removal-Driven Framework for Interaction-Aware Universal Video Object Editing cs.CVPDF

Shuzhou Yang, Lanqing Guo, Lingen Li, Weiqi Li, Zhao Zhong

TL;DR: 论文提出 UniVOE，一个以视频物体移除为驱动的通用视频物体编辑框架，同时训练物体移除与注入模型。该方法先利用视频修复先验训练能处理交互效应的移除器，再用真实视频生成大规模物体存在/缺失配对数据，反向训练注入模型，并通过奖励后训练对齐人类偏好。

Details

Motivation: 视频物体移除与注入互为逆操作，但注入需要大规模高质量配对监督，真实配对难获取且合成数据真实性和多样性不足；因此论文希望利用较易训练的移除任务来反哺注入任务。

Result: 实验对比强开源与商业系统，UniVOE 在物体移除和注入两个任务上均达到 SOTA，尤其在具有挑战性的交互效应场景中表现突出。

Insight: 创新点在于移除驱动的数据生成与双向训练范式、显式建模物体交互效应的移除器，以及引入奖励模型进行后训练以提升注入结果的感知真实性和人类偏好一致性。

Abstract: Video object removal and injection are reciprocal editing operations between object-present and object-absent video states, yet they differ substantially in training difficulty. Removal can be effectively bootstrapped from video inpainting and further adapted to erase object-related interaction effects. Injection, however, requires large-scale paired supervision to model object appearance, dynamics, and scene-dependent effects, while perfectly aligned real-world pairs are difficult to collect and synthetic data lacks sufficient realism and diversity. To address this challenge, we present UniVOE, a removal-driven framework for training both video object removal and injection models. We first train an effect-aware remover by transferring video inpainting priors and finetuning on object-present/object-absent pairs. The trained remover converts real object-present videos into object-absent counterparts, creating large-scale paired supervision for injection. An injection model is trained on these pairs in the reverse direction to recover objects together with their dynamics and interaction effects. Moreover, since pixel-level reconstruction alone cannot adequately capture perceptual realism, we introduce a reward-based post-training method that aligns injection results with human preferences using a proprietary reward model. Experiments against strong open-source and commercial systems demonstrate that UniVOE achieves state-of-the-art performance on both object removal and injection, particularly in challenging interaction-effect cases.

[87] Gate3DGen: A Pre-processing Model that Selectively Purifies Real-World Images for 3D Generation cs.CVPDF

Xiaolin Li, Zihao Huang, Fanpeng Meng, Chen Yang, Guanjun Wu

TL;DR: 论文提出 Gate3DGen，一种用于单图 3D 生成的选择性图像预处理模型。它通过退化感知路由器判断输入是否需要净化，仅对退化图像进行一步式净化，并保持干净输入不变，从而在不重新训练现有 3D 生成器的情况下提升真实场景图像质量及下游 3D 几何生成效果。

Details

Motivation: 现有单图 3D 生成在高质量基准上表现较好，但面对光照不均、模糊、遮挡等真实世界图像时性能显著下降；同时，预处理需要决定何时干预，并保留对 3D 生成重要的视觉线索。

Result: 在合成基准上，该方法平均仅需 2.3 秒即可提升图像保真度和下游 3D 几何质量，相对退化输入将 TRELLIS.2 的网格距离降低 61.5%；在真实照片上的 VLM 评估和人类偏好也验证了其身份保持与退化校正能力。

Insight: 创新点在于将预处理设计为可选择门控模块，而非对所有输入统一处理；通过退化感知路由、面向 3D 生成的一步净化器及保持物体身份/轮廓/纹理线索的空间损失，并结合合成数据引擎提供成对监督，实现无需重训练生成器的即插即用式增强。

Abstract: Single-image 3D generation has achieved high fidelity on curated benchmarks, yet degrades sharply on real-world images with uneven illumination, blur, and occlusions. Purifying such inputs requires deciding when to intervene and preserving the visual cues needed for 3D generation, while leaving clean inputs unchanged. We introduce Gate3DGen, a selective pre-processing model that gates image inputs to existing 3D generators without retraining them. A degradation-aware router bypasses inputs classified as clean and extracts features to guide purification of the remaining inputs. Conditioned on these features, a 3D-oriented one-step purifier purifies degraded inputs with spatial losses that encourage preservation of object identity, contours, and textural cues. To provide paired supervision, we construct a synthetic data engine that renders clean views of 3D assets and synthesises composite degradations. On the synthetic benchmark, the purifier improves image fidelity and downstream 3D geometry in only 2.3s on average, reducing TRELLIS.2 mesh distance by 61.5% relative to degraded inputs. VLM evaluation and human preferences on real photographs further support identity preservation and degradation correction. More demos and code are available at https://supra-3d.github.io/gate3dgen.

Shengqiong Wu, Meng Luo, Bobo Li, Ziwei Liu, Shuicheng Yan

TL;DR: 该综述以“世界建模”为统一视角，系统梳理视频基础模型与视频世界模型的研究进展，将视频视为潜在时序世界的有损、部分且有序的投影，并围绕阅读、写作、共享和交互四类互补操作组织相关文献。文章覆盖任务、表示、模型、数据集与基准，并总结该领域从片段级识别与感知保真转向持久状态、可控演化、物理与因果有效性以及基于交互和下游决策评估的趋势。

Details

Motivation: 视频理解与生成模型发展迅速，但二者目标不同，且都涉及视频观察与不断演化世界之间的关系。论文旨在用世界建模的统一框架弥合理解与生成，并区分序列层面的时序连贯与底层世界状态的一致性。

Result: 作为综述，论文没有提出新方法的定量实验结果；其主要结果是建立四操作分类框架，并梳理研究重点从片段识别和感知保真转向持久状态、可控演化、物理与因果有效性及交互式评估，同时提炼开放挑战。

Insight: 创新点在于以“世界建模”统一视频理解与生成，将视频视为潜在时序世界的部分有损投影，并提出阅读、写作、共享、交互四个互补操作。客观来看，其可借鉴之处是强调世界状态一致性而非仅序列连贯，并把评估落到交互与下游决策上。

Abstract: Video foundation models have advanced rapidly in understanding and generating visual sequences, while video world models seek to capture the underlying dynamics of the world they depict. Despite their different objectives, both concern the relationship between video observations and an evolving world. This survey adopts world modeling as a unifying lens for examining these developments. Viewing an observed video as a partial, lossy, and temporally ordered projection of a latent temporal world, we organize the literature around four complementary operations: reading infers world properties from observed video; writing synthesizes video from specified or learned world conditions; sharing integrates understanding and generation through common modeling mechanisms; and interacting predicts and updates world states under actions and evolving observations. This perspective distinguishes temporal coherence in observed or generated sequences from consistency in the underlying world state. Across tasks, representations, models, datasets, and benchmarks, we trace a shift in emphasis from clip-level recognition and perceptual fidelity toward persistent state, controllable evolution, physical and causal validity, and evaluation grounded in interaction and downstream decisions, and we distill the open challenges that follow.

[89] Sensorimotor Alignment: A Platonic Proxy for Embodied Policy Success cs.CV | cs.LG | cs.ROPDF

Yunfei Xie, Howard H. Qian, Shiyi Lan, Kaiyu Hang, Chen Wei

TL;DR: 论文关注机器人基础模型评估成本高的问题，提出一种免训练的离线指标 Sensorimotor Alignment（S^2），用于衡量具身模型感知表征与物理运动几何结构的对齐程度。作者发现该指标与多种具身模型家族的下游机器人任务成功率正相关，可作为闭环评估前的低成本模型筛选代理，并能在昂贵微调前识别有潜力的 VLM 骨干。论文还将 Platonic Representation Hypothesis 扩展到具身模型，为 VLA 和 WAM 中的现象提供表征层面的解释。

Details

Motivation: 机器人基础模型通常依赖昂贵的闭环 rollout 评估，亟需低成本的离线代理指标。论文受 Platonic Representation Hypothesis 启发，探究运动轨迹是否可视为一种模态，以及具身模型能力提升时其感知表征是否应趋近物理运动结构。

Result: 摘要未给出具体数值或 benchmark，但声称 S^2 在多种具身模型家族中与下游机器人任务成功正相关，可作为闭环评估前的低成本离线筛选指标，并能在 VLA 微调前识别有前景的 VLM 骨干。

Insight: 创新点在于提出免训练、离线的传感器运动对齐度量，将 PRH 从视觉-语言模态扩展到具身运动模态，并用表征层面对齐解释 VLA/WAM 的成功现象；客观看，其价值在于为具身模型提供低成本预筛和可解释性代理指标。

Abstract: Robotic foundation models are typically evaluated through costly closed-loop rollouts, which motivates cheap offline proxies for model capability. The Platonic Representation Hypothesis (PRH) suggests that as representations improve, different modalities such as vision and language converge toward a shared statistical model of reality. We ask whether motion trajectories form another such modality: as embodied models become more capable, should their sensory representations converge toward the structure of physical motion? To examine this, we propose Sensorimotor Alignment ($S^2$), a training-free offline metric that measures how well a model’s sensory representation aligns with the geometry of physical motion. We find that $S^2$ correlates positively with downstream robot task success across embodied model families, making it a low-cost offline proxy for prioritizing models before closed-loop evaluation. $S^2$ can also identify promising vision-language model (VLM) backbones even before costly fine-tuning into vision-language-action models (VLAs). Beyond this correlation, $S^2$ offers representation-level explanations for phenomena observed in VLAs and world action models (WAMs). With $S^2$, we extend PRH to embodied models and make sensorimotor alignment empirically measurable, offering insight into why an embodied model succeeds beyond its closed-loop success rate.

cs.AI [Back]

[90] Group Relative Policy Optimisation Improves Multilingual Speech Recognition in Low-Resource Languages cs.AI | cs.CLPDF

Tim Wenjie Hu, Evelyn Nafula Ouma, Benjamin Akera, John A. Quinn

TL;DR: 论文将组相对策略优化（GRPO）用于基于Whisper-large-v3的非LLM编码器-解码器多语言语音识别模型，在51种非洲语言上研究低资源ASR。方法旨在缓解教师强制训练带来的暴露偏差，使模型在推理时更能从自身预测错误中恢复，并最终提出公开可用的SunflowerASR模型。

Details

Motivation: 自回归编码器-解码器ASR模型通常用教师强制训练，存在暴露偏差，推理时难以纠正自身预测错误；已有GRPO强化学习主要验证于基于LLM的大模型和主要语言，本文关注其在较小非LLM模型和低资源语言上的效果。

Result: SunflowerASR在AfriVox-v2上取得已发表最佳宏平均WER 0.227，在SimbaBench上取得0.372；当训练数据稀缺或输入音频含噪、退化时，方法收益最明显。

Insight: 创新点在于把GRPO从LLM式ASR扩展到较小的非LLM编码器-解码器模型，并以Whisper-large-v3跨51种非洲语言验证；客观看，其价值在于用强化学习缓解暴露偏差，并证明该方法在低资源和噪声退化场景下更有效。

Abstract: Autoregressive encoder-decoder speech recognition models trained with teacher forcing can suffer from exposure bias, limiting their ability to recover from their own prediction errors at inference time. Recent work has shown that reinforcement learning with Group Relative Policy Optimisation (GRPO) improves speech recognition in major languages with LLM-based models. We extend this line of work to smaller, non-LLM encoder-decoder ASR models, using Whisper-large-v3 as a case study across 51 African languages. The benefits of this approach are most pronounced when training data is scarce or when input audio is noisy or degraded. Our resulting model, SunflowerASR, achieves the best published macro-averaged WER on the AfriVox-v2 (0.227) and SimbaBench (0.372) benchmarks, and is publicly available.

[91] Disentangling Robustness and Sycophancy in Multilingual LLMs cs.AI | cs.CLPDF

Rhitabrat Pokharel, Ameeta Agrawal

TL;DR: 论文研究多语言LLM中模型同意用户错误前提的现象，区分是因语言能力不足还是谄媚行为。提出SC-matrix交叉评估模型能力和谄媚性，发现相同谄媚率可能掩盖不同失败构成，高语言能力不总能防止谄媚，推理缓解策略优于指令策略，表明多语言谄媚是复合现象需分别诊断。

Details

Motivation: 现有多语言LLM研究仅记录模型是否同意错误前提，未区分是谄媚还是语言能力不足，本文旨在解耦这两种情况以准确诊断问题。

Result: 通过SC-matrix分析发现：(1)相同总体谄媚率可能掩盖不同失败构成；(2)高语言能力并不总能防止谄媚；(3)基于推理的缓解措施优于基于指令的策略。

Insight: 创新点在于提出SC-matrix将能力与谄媚解耦，提供细粒度分析框架，揭示多语言谄媚是不同失败的复合体，并强调推理缓解策略的有效性。

Abstract: When a multilingual LLM agrees with a user’s false premise, is it being sycophantic, or does it simply not know better in that language? Current research does not distinguish these cases: they record only whether the model agreed, not why. We introduce the SC-matrix, that crosses a model’s competence (whether it answers correctly under no pressure) with its sycophancy (whether it maintains that answer under pressure). Our results show that (1) identical aggregate sycophancy rates can mask different failure compositions, (2) high competence in a language does not always protect against sycophancy, and (3) reasoning-oriented mitigations outperform directive-based strategies. We find that multilingual sycophancy is not a single phenomenon but a composite of different failures that require distinct diagnosis.

[92] SCOPE: Compiling Natural-Language Policies into Neurosymbolic Predicate Programs cs.AI | cs.CLPDF

Arnon Mazza, Elad Levi

TL;DR: 论文提出 SCOPE，将自然语言策略编译为可执行的神经符号谓词程序：由小模型、确定性抽取器和组合逻辑代码共同执行，并围绕策略本身联合优化程序与训练数据。该方法面向 LLM 智能体在多轮对话和工具调用轨迹上的复合策略执行，在真实策略任务基准上取得最高平均准确率，且程序可迁移到廉价模型上部署，兼顾稳定性与可解释性。

Details

Motivation: 现有护栏模型对复合策略效果下降，LLM-as-a-judge 成本高且预测不稳定；已有神经符号方法虽可分解策略，但符号层表达力不足且只针对固定数据优化，而非直接优化策略。论文希望实现可靠、可解释、低成本的生产级策略执行。

Result: 在真实世界策略任务基准上，SCOPE 相对提示优化、任务分解和工作流生成方法，在不同任务和模型上取得最高平均准确率；用最弱执行器优化的程序可无需再优化迁移到其他模型，在廉价模型上达到接近前沿推理模型的准确率，并产生更稳定的预测。

Insight: 创新点在于把自然语言策略编译成由神经谓词、确定性抽取器和代码逻辑组成的可执行程序，并联合优化程序与数据，通过针对程序偏离策略之处生成对抗反例来提升对齐；客观看，其价值在于将策略逻辑显式化、让每个判定可由谓词输出解释，并实现跨模型迁移的低成本部署。

Abstract: Policy enforcement for LLM agents has shifted from single-turn content moderation to compound, application-specific policies evaluated over multi-turn transcripts and tool-call trajectories. Fine-tuned guardrail models are efficient but degrade on compound policies, and while LLM-as-a-judge approaches achieve higher accuracy, they incur high inference cost and tend to produce unstable predictions. Neurosymbolic methods offer a natural alternative, decomposing the policy into narrow neural decisions composed under explicit logic. However, existing symbolic layers lack the required expressiveness and are optimized against fixed data rather than the policy itself. To address this, we introduce SCOPE, which compiles a policy into an executable program of neural predicates executed by small models, deterministic extractors, and code that composes their outputs. The program is optimized jointly with its training data, using adversarial generation of counterexamples that target where the program deviates from the policy. We evaluate SCOPE on a benchmark of real-world policy tasks against prompt optimization, task decomposition, and workflow generation methods, achieving the highest average accuracy across tasks and models. Programs optimized with the weakest executor transfer to other models without further optimization. This allows them to be deployed in production on inexpensive models, where they reach accuracy comparable to frontier reasoning models. Unlike these monolithic models, SCOPE produces substantially more stable predictions, and each verdict is interpretable through the predicate outputs that produced it. Additionally, we validate the key components of our method through an ablation study. SCOPE thus enables reliable, interpretable, and efficient policy enforcement at production scale.

[93] The Value of Memory in Language Agents cs.AI | cs.CLPDF

Zehao Lin, Erenkai Zhu, Helan Hu, Xixuan Hao, Chunyu Li

TL;DR: 论文提出一个决策论框架，将语言智能体记忆的价值刻画为从原始历史到持久状态、实际使用以及昂贵恢复与修订过程中所避免的任务损失，并分析信息保留、检索使用、反馈和需求不确定性。作者还用成本感知模拟、五个最小反例以及 LoCoMo 等诊断，揭示记忆在保存、使用和测量层面的失败，并给出操作风险界限与可测试原则。

Details

Motivation: 记忆对语言智能体的价值不只在存储本身，而在于哪些区分能被保留、受限智能体能否有效使用、需求变化时能否恢复或修订；论文希望统一刻画这些因素并进行可操作的风险评估。

Result: 在 LoCoMo、双项目更新任务和纵向案例上诊断出记忆保存、使用与测量失败，例如固定检索证据时保存的账本会改变后续答案，固定智能体输出时评估者修订会改变分数；摘要未报告 SOTA 或具体数值结果。

Insight: 论文以“避免任务损失”为单位量化原始/保留记忆价值，提出风险-成本前沿、价值剖面，并指出对答案零互信息的指针也能降低精确恢复成本；客观看，将记忆价值拆成保存-使用-恢复/修订链条并强调测量依赖性，具有方法论借鉴意义。

Abstract: Memory enables language agents to carry useful distinctions from past experience into future decisions. Its value depends on which distinctions survive, whether a bounded agent can use them, and what can be recovered or revised as demands evolve. We develop a decision-theoretic framework that follows this value from raw history through persistent state, operational use, and costly recovery and revision. Original and retained value are expressed in units of avoided task loss; risk-cost frontiers describe attainable recovery and preservation across uses. Unknown demand yields value profiles, while interactive feedback changes the distribution of future work. A cost-aware simulation condition and five minimal counterexamples expose the roles of information timing, retrieval access, component interaction, feedback and demand uncertainty. In particular, a pointer with zero mutual information about the answer can still reduce the cost of exact recovery. Diagnostics on LoCoMo, a two-project update task and longitudinal cases exhibit failures of preservation, use and measurement. At fixed retrieved evidence, a saved ledger changes subsequent answers; at fixed agent outputs, an evaluator revision changes scores. The framework connects these distinctions to operational risk bounds and testable principles for allocating memory, preserving recovery options and revising evidence dependencies.

[94] Direct Pass@$k$ Optimization for Thought Pruning: Balancing Quality and Diversity in Test-Time Reasoning cs.AI | cs.CLPDF

Mahdi Salmani, Adel Javanmard, Devansh Gupta, Meisam Razaviyayn

TL;DR: 论文提出一种面向思维剪枝的直接 Pass@k 优化方法：在仅生成短前缀后选择候选解池继续生成，以最大化池中至少一个部分解最终得到正确答案的概率。该方法将质量与多样性统一到同一优化目标中，从而在测试时推理中平衡探索与利用。

Details

Motivation: 测试时扩展需要生成大量候选解并运行到完成，计算成本高；现有基于前缀的剪枝方法分为质量导向和多样性导向两类，但很少显式建模并优化二者之间的权衡。

Result: 在八个开放权重推理模型上，该方法优于现有质量导向和多样性导向的剪枝策略；当保留中等数量的候选解继续生成时，提升尤为明显。

Insight: 核心创新是直接优化所选候选池的 Pass@k，同时奖励高质量部分解并惩罚冗余，使质量与多样性的平衡由优化准则自然实现，而非依赖分离的启发式设计。

Abstract: Test-time scaling has emerged as a key inference strategy for reasoning models, where a model generates a large number of candidate solutions in the hope of finding at least one correct solution. Running all candidate solutions to completion can be computationally demanding, so recent work prunes the candidate pool after generating only a short prefix, committing further computation only to the surviving candidates. Existing pruning strategies divide broadly into two categories: Quality-based methods assign a score to each partial solution and discard those predicted to ultimately lead to an incorrect final solution, while diversity-based methods select partial solutions that follow different reasoning trajectories and are therefore likely to cover different final solutions. The tension between these two objectives is an instance of exploration versus exploitation, yet little to no prior work on pruning exposes this trade-off directly, let alone optimizes it. In this paper, we introduce a pruning procedure that addresses this trade-off directly by optimizing pass@$k$ over the selected solution pool. In particular, it chooses the pool to maximize the probability that at least one of its partial solutions, when continued to completion, yields a correct answer. Because this objective simultaneously rewards high-quality partial solutions and penalizes redundancy among them, quality and diversity are naturally balanced by the criterion itself rather than through separate ad-hoc design choices. On eight open-weight reasoning models, our method outperforms existing quality-based and diversity-based pruning strategies. The gains are particularly pronounced when the pruning procedure keeps a moderate number of partial solutions for continued generation, highlighting the value of directly balancing quality and diversity during pruning.

cs.LG [Back]

[95] ChemBART: Pre-training a Reaction Language Model for Comprehensive Synthesis Planning cs.LG | cs.CLPDF

Kenan Li, Yijian Zhang, Jin Wang, Haipeng Gan, Zeying Sun

TL;DR: 论文提出 ChemBART，一个基于 BART 的反应语言模型，采用以反应为中心的预训练策略，在完整反应表达式（Reactant > Reagent > Product）上进行掩码填充预训练。该模型可统一处理前体/试剂生成、温度-产率回归和分子性质分类等化学任务，并结合 MCTS 与强化学习进行多步合成路线设计。最显著的是，所提出的多步合成路线和反应条件经过湿实验验证，发现更短合成路径且产率较文献基准提升超过 20%。

Details

Motivation: 现有计算机辅助合成规划中的 LLM 方法多局限于单分子表示，只能处理前体预测等孤立单任务场景，难以覆盖综合合成规划所需的多样化学任务与反应条件信息。

Result: 论文将 ChemBART 与 MCTS 结合用于多步合成路线设计，并在湿实验中验证所提出的路线和反应条件；结果显示可获得更短合成路径，且产率相较已有文献基准提升超过 20%。

Insight: 创新点在于以完整反应表达式为中心进行 BART 掩码填充预训练，将前体/试剂生成、产率回归、性质分类等多类化学任务统一到同一模型框架，并进一步融入 MCTS/强化学习实现多步合成规划，打通计算预测与湿实验合成验证的闭环。

Abstract: Recent advances in large language models (LLMs) have demonstrated transformative potential across diverse fields. However, in computer-aided synthesis planning (CASP), current LLM methodologies are often restricted to single-molecule representations, limiting their application to isolated, single-task scenarios such as precursor prediction. Addressing this gap, we present ChemBART, a BART-based language model that adopts a reaction-centric pre-training strategy. By leveraging a mask-filling pre-training task on holistic reaction expressions (Reactant > Reagent > Product), ChemBART effectively solves a variety of chemical problems, including precursor/reagent generation, temperature-yield regression, and molecular property classification. Building on these capabilities, we demonstrate the integration of ChemBART with Monte Carlo tree search (MCTS) for multi-step synthesis route design, where the model serves to optimize the policy and value functions within a reinforcement learning framework. Beyond single-molecule pre-trained LLMs that are constrained to specific applications, ChemBART unifies diverse chemical tasks to facilitate synthesis planning with comprehensive implementation information. Most significantly, the proposed multi-step synthesis routes and reaction conditions were validated in the wet lab, confirming the discovery of shorter synthetic pathways with a >20% yield improvement over established literature benchmarks. Our work highlights the potential of reaction-focused pre-training and showcases the broad utility of ChemBART in closing the loop between in silico prediction and wet-lab synthesis.

[96] One Teacher, Many Views: Reducing Reference Sensitivity in On-Policy Self-Distillation cs.LG | cs.CLPDF

Khouloud Saadi, Di Wang

TL;DR: 论文研究on-policy self-distillation（OPSD）中参考解答敏感性导致的目标不稳定性问题，提出Monte Carlo Privileged Distillation（MCPD），将特权上下文建模为部分参考视图的分布并对教师边际分布进行蒸馏，在数学和科学推理基准上取得比全参考OPSD更一致且更好的监督效果。

Details

Motivation: 现有OPSD使用单一完整参考解答作为特权教师的条件，但参考只是推理过程的一种实现，会编码顺序、措辞和解题路径，导致教师目标过度依赖学生推理时不可用的信息，替换参考会显著改变教师分布，即参考诱导的目标不稳定性。

Result: 在数学和科学推理基准上，MCPD产生更参考一致的监督，并一致优于全参考OPSD。

Insight: 创新点在于将特权上下文视为部分参考视图的分布，通过对多个视图下的教师下一词分布取概率平均来蒸馏边际教师分布，并证明该平均等价于最小化期望教师-学生KL散度的精确目标。

Abstract: On-policy self-distillation (OPSD) improves reasoning by training a model on its own trajectories with dense token-level supervision from a privileged copy conditioned on a reference solution. Existing methods typically use a single teacher distribution conditioned on the complete reference. We identify a limitation of this formulation: a reference is only one realization of the reasoning process. Although it provides evidence about the answer, it also encodes an ordering, phrasing, and solution path. Full-reference conditioning can therefore produce correct targets that depend excessively on information unavailable to the student at inference. We show that replacing one valid reference with another, while holding the problem and student state fixed, can substantially alter the teacher distribution, a phenomenon we call reference-induced target instability. To address this instability, we introduce Monte Carlo Privileged Distillation (MCPD), which models privileged context as a distribution over partial reference views and distills the marginal teacher distribution. MCPD evaluates the teacher under multiple views and averages the resulting next-token distributions. We show that this probability-space average is the exact target induced by minimizing the expected teacher-to-student KL divergence over privileged views. Across mathematical and scientific reasoning benchmarks, we show that MCPD produces more reference-consistent supervision and consistently improves over full-reference OPSD.

[97] Inherited, Not Invented: Tracing the Provenance of Wait in Reasoning Models cs.LG | cs.CLPDF

Zhizhuo Kou, Sizai Hou, Long Chen, Jian Yang, Junyu Luo

TL;DR: 论文追踪推理模型中“wait”标记的来源，发现它主要由训练数据继承而来，而非强化学习从零发明。作者通过频率、位置、形式、用法等无标签指标和注入错误测试分析多个开放模型的训练历史，表明RL可在“wait”近乎缺失时仍大幅提升MATH-500准确率。论文还警示仅凭标记频率判断RL诱导推理不可靠，LLM-as-a-judge修订指标可能反转人工标注趋势。

Details

Motivation: 已有观点认为RL教会推理模型写“wait”再重新检查步骤，但准确率提升并不依赖该词，易混淆RL真正学到的能力与从训练数据继承的模式。论文旨在厘清“wait”是RL发明还是训练数据继承，并反思相关评测指标。

Result: 在缺少“wait”的基座上，零RL使MATH-500准确率显著提升而“wait”仍近乎缺失；mid-training和推理轨迹微调可在多数模型中植入该标记。基座已有“wait”时，OLMo-3和MiMo的RL-Zero配方会放大它，而同基座的KL-free GRPO不会；OLMo-3配方还改变其形式与敏感性，SFT后RL几乎不改变它，作者GRPO配方会抹去微调植入的标记。

Insight: 创新点在于从训练历史追踪“wait”的来源，用无标签的频率、位置、形式和用法指标及注入错误测试区分“继承”与“发明”。客观看，论文重要贡献是揭示标记频率不能代表RL诱导推理，并提醒LLM-as-a-judge修订指标可能不可靠。

Abstract: Reinforcement learning (RL) is credited with teaching reasoning models to write {wait} before reconsidering a step, yet large RL accuracy gains do not require the word. This view can confuse what RL discovers with what a model inherits from its training data. We trace the word through open models’ training histories, reading it with label-free indices of frequency, position, form and usage and an injected-error test, all computed from sampled outputs. We find that zero-RL almost never coins the marker: on bases that lack the word, RL raises MATH-500 accuracy substantially while {wait} stays near absent. It arrives with training data: mid-training in OLMo-3, and fine-tuning on reasoning traces installs it in most models whose base lacks it. Where the base already uses it, both released RL-Zero recipes (OLMo-3, MiMo) multiply it, whereas KL-free GRPO from the same base does not; OLMo-3’s recipe wears down its form and makes it a more sensitive but more liberal alarm; RL after supervised fine-tuning barely changes it, and our GRPO recipe erases a marker seeded by fine-tuning. Finally, we show that marker frequency alone does not indicate RL-induced reasoning and caution against LLM-as-a-judge revision metrics, as automated judges can reverse human-labelled trends.

[98] Inside-JEPA: From Prediction to Relational Grounding in Latent World Models cs.LG | cs.CV | cs.ROPDF

Hossein Resani, Javen Qinfeng Shi

TL;DR: 论文提出 Inside-JEPA，在 JEPA 的潜空间世界模型状态中显式加入面向有序实体对的关系通道，并利用逆动力学损失在无需关系标签的情况下将这些通道与动作导致的关系变化对齐。实验表明，仅靠潜预测无法使关系通道获得有效接地，而逆动力学监督能显著提升关系 AUROC，并支持基于关系目标的一步动作排序。

Details

Motivation: JEPA 能在潜空间中进行高效预测和规划，但其潜状态不透明，缺乏对实体间关系的显式表示与可解释接地，论文旨在让潜世界模型从预测走向关系接地。

Result: 在 Something-Something V2 上，逆动力学监督使关系 AUROC 在十个有标注框设置中提升 0.065-0.103；在 RH20T 机器人数据留出任务上提升 0.081；使用现成检测跟踪在五种关系上提升 0.050-0.061。给定动作时，预测关系变化接近观测未来关系质量（AUROC 0.599 vs 0.621）；一步关系目标下正确动作进入前五的比例为 43%，优于仅预测潜变量的 24% 和频率先验的 17%。

Insight: 创新点在于将有序实体对的关系通道嵌入 JEPA 潜状态，并证明逆动力学监督可在无关系标签时接地这些通道。客观来看，论文揭示了单纯潜预测不足以形成关系表示，而动作恢复信号是促成关系接地的关键监督。

Abstract: Joint-embedding predictive architectures (JEPAs) learn world models in latent space, enabling efficient planning but leaving their states opaque. We introduce Inside-JEPA, which places relational channels for ordered entity pairs inside the JEPA latent state and shows that an inverse-dynamics loss can ground these channels without relation labels when its head reads them, either alone or as part of the full state. On Something-Something V2, latent prediction alone performs no better than a random relation head, while inverse-dynamics supervision that recovers actions from channel changes improves relational AUROC by .065-.103 across ten settings with annotated boxes, by .081 on RH20T robot episodes with held-out tasks, and by .050-.061 across five relations using off-the-shelf detection and tracking. An inverse head on entity latents decodes actions more accurately but barely improves relational grounding; shuffled labels, two label-free objectives, and action labels on only 5% of the pairs also fail to ground the channels. Given an action, predicted relational changes approach the quality of observed future relations (AUROC .599 vs .621). For one-step relational goals, a supervised readout of the 16 channels ranks the correct action among the top five of 65 in 43% of cases, compared with 24% for prediction-only latents and 17% for a frequency prior.

[99] Multi-Token Prediction and Alternative Tokenization Improve Learning Latent Multi-Hop Graph Search at Scale cs.LG | cs.CLPDF

Arvid Frydenlund, FuTe Wong, Katherine Lambert, Frank Rudzicz

TL;DR: 论文研究语言模型能否在未见图上根据起点和终点生成最短路径，指出此前随图规模增大而失败的两个原因：监督信号和分词方式。作者提出多Token预测（MTP）与替代分词方法，并证明二者作用正交，从而突破此前规模限制，在更大图上稳定学习多跳图搜索。

Details

Motivation: 现有语言模型在大规模图上学习最短路径/多跳图搜索时会遇到规模扩展瓶颈，论文旨在分析失败原因并解决该任务随图规模增大而无法学习的问题。

Result: 摘要未给出具体定量指标或benchmark数值，但声称结合MTP和替代分词后突破先前观察到的scaling wall，能够在远超以往限制的规模上一致且稳定地学习该任务。

Insight: 创新点在于从监督信号和tokenization两个角度解释失败，理论结合实证证明MTP有效且其收益与图拓扑无关，并提出替代分词并论证两个因素正交。客观上，该工作对LLM算法推理任务中的训练目标设计和输入表示/分词方式具有借鉴意义。

Abstract: Given an unseen graph, a start node, $s$, and an end node, $t$, can a language model generate the shortest path from $s$ to $t$? Prior work has found that models fail to learn this task as the graphs scale. We show that this failure has two causes: the supervision signal and the tokenization. First, we theoretically establish and empirically demonstrate the effectiveness of multi-token prediction (MTP), and show that the benefits of MTP are topologically agnostic. Second, we introduce alternative tokenizations and show that the two causes are orthogonal. Using these methods, we break through the scaling wall observed by prior work, consistently and stably learning to solve the task at scales far beyond previous limits.

cs.RO [Back]

[100] TerraTransfer++: Demonstration-Free End-to-End Planner cs.RO | cs.AI | cs.CV | cs.LGPDF

Zikang Xiong, Weixin Li, Zhouchonghao Wu, Akshay Rangesh, Saarth Bonde

TL;DR: 论文提出TerraTransfer++，一种无需人类演示的端到端自动驾驶训练框架：先在向量化模拟器中用自博弈预训练策略，再蒸馏出流匹配轨迹教师，并通过轻量适配器在冻结视觉骨干上训练闭环策略和规划器两个学生。方法只需图像与场景状态配对数据，不依赖专家驾驶轨迹。

Details

Motivation: 现有端到端自动驾驶训练成本高：模仿学习依赖车队规模日志且长尾安全场景覆盖稀疏，闭环强化学习受样本效率、渲染成本和视觉骨干前向开销限制。论文希望利用向量化模拟器自博弈的高吞吐获得碰撞、近失和恢复等自然多样状态，降低对专家演示的依赖。

Result: 闭环策略在3D Gaussian splatting闭环基准HUGSim上超过所有已发表模仿式端到端方法；规划器在NAVSIM navhard达59.4 EPDMS，超过DriveZero-Scale的57.1，在navtest达95.5 PDMS，且无需人类演示。

Insight: 创新点在于用自博弈生成长尾状态分布并蒸馏为流匹配轨迹教师，再以动作KL散度、批次关系低秩结构损失和轨迹匹配损失将冻结视觉骨干的轻量适配器对齐到教师；所有监督目标回溯自博弈策略而非日志轨迹，从而无需专家演示。客观看，其核心是将自博弈的高吞吐优势转化为端到端学生训练信号，并通过教师蒸馏实现无演示对齐。

Abstract: End-to-end autonomous driving has achieved state-of-the-art performance on benchmarks and real-world deployments. Its standard training recipe, however, is expensive across all existing paradigms: imitation learning requires fleet-scale driving logs whose behavior distribution only thinly covers the safety-critical tail; closed-loop reinforcement learning in a photorealistic simulator, while offering diverse behaviors, is bottlenecked by sample-efficiency, per-step cost of rendering, and a forward pass through a large vision backbone. Self-play in vectorized simulators changes the economics: at millions of rollout steps per second, one can obtain a state distribution with natural diversity in collisions, near-misses, and recoveries that no driving log contains, and a highly reactive policy emerges from a basic driving-quality reward. Our approach exploits this asymmetry. We pretrain a single policy by self-play, distill its rollouts into a flow-matching trajectory teacher that also plans, and train lightweight adapters on a frozen pretrained vision backbone to reproduce the latent space of either teacher through the action KL divergence, a batch-relational low-rank structural loss, and, for the planner, a trajectory-matching loss. Every target traces back to the self-play policy, so alignment never supervises against a logged trajectory: a paired dataset of (image, scene-state) frames suffices, with no need for the curated expert demonstrations that imitation pretraining is built on. From this one self-play teacher we obtain two end-to-end students: a closed-loop policy that outperforms every published imitation-based end-to-end method on the photorealistic 3D Gaussian splatting closed-loop benchmark HUGSim, and a planner that reaches 59.4 EPDMS on NAVSIM navhard, above every published method including DriveZero-Scale (57.1), and 95.5 PDMS on navtest, without human demonstrations.

[101] Self-Distilling Future Video Representations for World Action Models cs.RO | cs.CVPDF

Huiwon Jang, Suhyeok Jang, Hyeonmo Kang, Seungku Kim, Jinwoo Shin

TL;DR: 论文提出 FRED 框架，用于提升世界动作模型（WAMs）在单步去噪中从噪声预测未来视频表示的质量，从而让动作解码器更高效、更准确地生成机器人策略。FRED 通过自蒸馏，以真实未来观测提取的表示为目标，并引入历史观测作为时间上下文来增强未来表示预测。

Details

Motivation: 现有 WAMs 为避免视频生成的高成本迭代去噪，常让动作解码器依赖单步去噪得到的中间未来视频表示，因此动作预测性能直接受限于这些表示的质量。论文旨在提升单步从噪声预测未来表示的质量，并增强长时程与泛化场景下的机器人策略学习。

Result: 在多种真实机器人操作场景中，FRED 一致优于对比 WAMs，并在长时程任务和零样本环境泛化的域外任务上取得更大提升；在仅有人类视频、未出现在机器人训练数据中的任务上达到 50.0% 成功率，而最佳对比 WAM 为 19.4%。

Insight: 创新点在于用自蒸馏方式训练 WAMs，从噪声单步预测未来视频表示，并以真实未来观测的表示作为监督目标；同时将历史观测作为时间上下文条件输入视频模型。客观看，该设计直接针对高效 WAMs 中“单步未来表示质量不足”的关键瓶颈，并通过时间上下文和自蒸馏提升表示学习。

Abstract: World Action Models (WAMs) have emerged as a promising approach to learning generalist robot policies using rich spatiotemporal priors from pre-trained video diffusion models. A recent line of WAMs improves inference efficiency by conditioning a separate action decoder on intermediate future video representations extracted from a single denoising step, avoiding costly iterative denoising required for video generation. This design makes action prediction depend directly on the quality of these future representations. In this paper, we introduce Future REpresentation learning via self-Distillation with temporal context (FRED), a framework that trains WAMs to improve the quality of future video representations from noise in a single denoising step. Specifically, FRED uses self-distillation to predict future representations from noise, using representations extracted from ground-truth future observations as targets. We further condition the video model on historical observations, providing temporal context for predicting future representations better. Across diverse real-world manipulation scenarios, FRED consistently outperforms competing WAMs. It achieves particularly large gains on long-horizon and out-of-domain tasks under zero-shot environment generalization, and substantially improves human-to-robot skill transfer, reaching 50.0% task success on tasks absent from robot training data and demonstrated only in human videos, compared with 19.4% for the best competing WAM.

[102] Revisiting Distribution Matching for State-to-Vision Policy Distillation in Laparoscopic Robotic Manipulation cs.RO | cs.CVPDF

Mingyu Sheng, Jianan Fan, Dongnan Liu, Guoyan Zheng, Ron Kikinis

TL;DR: 论文面向腹腔镜机器人操作中的state-to-vision策略蒸馏问题，提出以具备特权状态观测的state-based策略作为教师，训练仅依赖RGB图像和机器人本体感知信息的vision-based策略。该方法旨在避免手术环境中难以可靠获得精确状态的问题，并减少对大量专家演示的依赖。实验在仿真环境的多类手术操作任务上验证了所提蒸馏框架相较通用RL训练视觉策略更稳定、一致。

Details

Motivation: 传统基于RL的端到端手术机器人控制依赖精确特权状态，但在手术环境中难以可靠获取；纯视觉RL虽可避免显式状态估计，却训练困难；模仿学习又需要大量难以获得的专家演示。因此论文关注如何更有效地获得视觉操作策略。

Result: 论文在仿真环境中的多种手术操作任务上进行了实验。结果表明，在腹腔镜机器人操作场景下，用通用RL方法训练视觉策略具有挑战性，而提出的state-to-vision策略蒸馏在跨任务上表现出更一致和稳定的性能；摘要未给出具体定量指标或SOTA对比。

Insight: 摘要宣称的创新点在于将特权状态策略到视觉策略的蒸馏框架用于腹腔镜机器人操作，以减少专家演示需求并绕开精确状态估计。客观来看，其可借鉴之处在于复用state-based教师策略为vision-based策略提供监督，从而结合特权状态训练优势与视觉输入的部署便利性。

Abstract: End-to-end control policies trained via Reinforcement Learning (RL) have emerged as an alternative to conventional surgical robotic control approaches, with recent studies exploring state-based policies trained on privileged state observations. However, such approaches rely on precise physical states that are difficult to obtain reliably in surgical environments. Vision-based RL provides an alternative by learning manipulation policies directly from RGB observations, avoiding explicit state estimation and manually engineered perception pipelines while enabling policies to acquire task-relevant visual representations. Nevertheless, obtaining vision-based policies via pure RL methods for surgical manipulation remains challenging. Thus, imitation learning (IL) becomes another solution, yet it requires a large number of expert demonstrations, which are difficult to obtain in minimally invasive surgery. In this work, we propose a framework for state-to-vision policy distillation in laparoscopic robotic manipulation. Specifically, a state-based policy with privileged observations is employed as the teacher, and a vision-based policy learns from RGB observations and robot proprioceptive information, which requires less expert demonstrations than IL and introduces superior performance than RL methods. We conduct extensive experiments on various surgical manipulation tasks in the simulated environment. Experimental results show that, in the context of laparoscopic robotic manipulation, training vision-based policies with general-purpose RL methods is challenging, whereas our proposed state-to-vision policy distillation leads to more consistent and stable performance across tasks.

[103] Memory before Goal: Role of Pre-History Memory for Humanoid Loco-Manipulation cs.RO | cs.CV | cs.LGPDF

Chenyuan Chen, Ran Ding, Lechen Zhang, Zewen He, Yoshihiko Nakamura

TL;DR: 论文面向动态场景中的人形机器人全身移动操作任务，指出仅依赖当前观测的VLA策略难以处理非马尔可夫性，提出WholeBodyMem对象引导的感知记忆模块，通过SAM3跟踪任务物体并基于物体事件因果选择关键帧，将外观、几何和时间信息编码为对象token供动作专家交叉注意力读取。作者还构建了HuMeGo人形记忆基准，包含6个仿真和6个真实Unitree G1任务。

Details

Motivation: 动态场景中物体会变化、被遮挡或随机器人转身离开视野，关键线索在动作前出现而当前观测已丢失，导致仅看当前帧的全身VLA策略无法完成依赖历史信息的移动操作。

Result: 在HuMeGo上，WholeBodyMem优于无记忆和均匀记忆基线，将基础策略成功率从仿真29.1%提升到72.6%，真实机器人从36.7%提升到88.3%，并在SIMPLE任务上从70.2%提升到86.9%。

Insight: 创新点在于以任务物体事件为中心、因果选择少量关键历史帧而非均匀增加记忆帧，并用融合外观、几何和时间的对象token增强VLA动作专家；同时提出了面向人形移动操作的记忆基准。客观看，其可借鉴之处是“保留正确帧而非更多帧”的对象级稀疏记忆设计和事件驱动的时间选择机制。

Abstract: Humanoid loco-manipulation in dynamic scenes is non-Markovian: cues vanish as objects change or are hidden, or leave the view as the robot turns and walks, yet whole-body vision-language-action (VLA) policies see only the present. We present WholeBodyMem, an object-guided perceptual memory that keeps the pre-history of object events seen before the robot acts on its goal. SAM3 tracks the task objects, and their events causally select key frames within an adaptive budget, which the action expert reads by cross-attention together with object tokens encoding appearance, geometry and time. We also contribute HuMeGo, a humanoid memory benchmark with six simulated and six real-world Unitree G1 tasks. Keeping the right frames rather than more frames, WholeBodyMem surpasses memoryless and uniform-memory baselines, raising its base policy’s success from 29.1% to 72.6% in simulation and from 36.7% to 88.3% on the real robot, while also improving the standard SIMPLE tasks (70.2% to 86.9%).

cs.SD [Back]

[104] MixJudge: Learning a Language-Queryable Critic for Vocal Mixing cs.SD | cs.CLPDF

Kuan-Hsun Ho, Yuan-Chiao Cheng, Yen-Tung Yeh, Yi-Hsuan Yang

TL;DR: 论文提出了MixJudge，一个用于声乐混音的语言可查询音频评论家。通过构建包含11个混音问题及多个严重级别的课程进行训练，MixJudge能够检索混音问题、区分相反问题、排序严重程度，并将其判断能力应用于下游的菜单选择任务，倾向于选择更接近专业混音的版本。

Details

Motivation: 混音不仅需要处理还需要评估结果，但现有音频-语言模型评估未单独测试判断具体混音问题的能力，且此类专业监督稀缺。因此需要训练一个能显式判断混音问题的模型。

Result: 实验表明MixJudge在检索混音问题、区分相反问题和排序严重程度上表现良好；在菜单选择任务中，其判断常偏好声学上更接近留出专业版本的候选，说明显式判断能支持比较和下游决策。

Insight: 创新点在于构建了多问题多严重级别的混音课程，并训练语言可查询的评论家，将混音评估转化为可查询任务。客观分析，其利用课程学习缓解监督稀缺，并验证判断能力可迁移到下游决策，为音频-语言模型在专业领域的应用提供了新思路。

Abstract: Mixing requires not only applying processing but also assessing the resulting mix. Audio-language models can provide conversa- tional mixing advice, but existing evaluations do not separately test whether they can judge specific mixing issues. Since professional supervision for such judgments is scarce, we construct a curriculum of 11 vocal-mix issues at multiple severity levels and use it to train MixJudge, a language-queryable audio critic. MixJudge shows its ability to retrieve mixing issues, distinguish opposite issues, and order the severity of an issue. Moreover, its judgments transfer to menu selection: given a set of processing candidates, MixJudge often favors versions acoustically closer to a held-out professional version, demonstrating how explicit mixing judgment can support comparison and downstream mixing decisions.

[105] Predicting Music Valence and Arousal via Textual and Multimodal In-Context Learning in Large Language Models cs.SD | cs.AI | cs.CLPDF

Chi-En Dai, Chih-Pin Tan, Yi-Hsuan Yang

TL;DR: 论文研究用大语言模型的上下文学习（ICL）提升音乐连续维度情感预测，即效价（valence）和唤醒度（arousal）。方法不直接预测情感分数，而是以小规模任务专用基模型的初始预测为基础，利用辅助数据集的上下文示例进行细化；评估了基于歌词和生成音频描述文本的ICL变体，以及融合原始音频的多模态变体。实验在MER60作为严格留出测试集上显示，尤其效价预测显著提升。

Details

Motivation: 音乐唤醒度较易从音频预测，但效价因主观且依赖上下文而难以建模；论文希望探索无需训练的ICL能否作为推理阶段策略，借助LLM的上下文示例改进连续音乐情感预测。

Result: 在MER60测试集上，文本ICL用80个示例将效价R^2从0.110提升到0.427；多模态变体仅用10个示例达到0.346。结果表明ICL对效价提升尤其明显，且多模态在少样本下也有效。

Insight: 创新点在于将LLM的ICL用于细化小模型的情感预测而非直接回归分数，并系统比较文本（歌词、生成音频描述）与多模态（原始音频）上下文示例的效果。客观看，其可借鉴之处是把连续情感预测转化为推理时示例驱动的预测修正，并利用辅助数据集和训练-free策略提升主观效价建模。

Abstract: While arousal can be reliably predicted from musical audio, emotional valence remains difficult to model due to its subjective and context-dependent nature. In this work, we investigate the potential of in-context learning (ICL) with large language models to advance continuous valence and arousal prediction. Rather than predicting emotion scores directly, our framework leverages contextual demonstrations from an auxiliary dataset to refine the initial predictions produced by a small, task-specific base model. We systematically evaluate a text-based ICL variant using lyrics and generated audio captions, alongside a multimodal variant that incorporates raw audio signals. Using the entire MER60 dataset strictly as a held-out test set, our experiments demonstrate substantial performance gains, particularly for valence. Text-based ICL improves the valence $R^2$ from 0.110 to 0.427 with 80 demonstration shots, while the multimodal variant achieves 0.346 with only 10 shots. These findings suggest that training-free ICL offers an effective inference-time strategy for current language models to predict continuous music emotion.

cs.SE [Back]

[106] Tracing Hallucinations in LLM Reasoning for UML Class Diagram Generation: An Empirical Taxonomy and Mitigation Study cs.SE | cs.CLPDF

Mojdeh Rahmanian, Foutse Khomh, Ashkan Sami, Yanchao Yu

TL;DR: 论文针对大语言模型从自然语言需求生成UML类图时的幻觉问题，提出一个覆盖提示忠实性、代码忠实性和推理质量三个维度、共11个子类型的经验分类体系，并基于DeepSeek-R1、DeepSeek-V4-Pro的推理轨迹进行人工与双LLM标注分析。研究发现幻觉类型分布具有模型依赖性，并通过在生成提示中加入约束开展缓解实验，验证定位推理过程幻觉有助于针对性提示优化。

Details

Motivation: 现有评估多只看最终UML类图，缺少对具备推理能力LLM中间推理轨迹的检查，难以揭示幻觉在需求理解、推理和代码生成过程中的来源。论文希望建立面向推理过程的幻觉分类并探索可操作的缓解方式。

Result: 在60条人工标注和120条双LLM标注的DeepSeek-R1轨迹上，代码忠实性是R1最常见维度，而DeepSeek-V4-Pro以提示忠实性为主；推理轨迹长度与幻觉频率无显著单调关联，需求复杂度与幻觉频率负相关。在13个Golden UML ModelSet案例、三个Qwen3模型上，加入提示忠实性和推理质量约束使汇总平均F1提升4.0个百分点，事后代码忠实性审查无一致额外收益，关系建模始终得分最低。

Insight: 创新点在于将UML类图生成幻觉从最终输出前移到推理轨迹层面，按需求到推理、推理到PlantUML代码、推理内部质量三维度细分为11类，并证明轨迹级定位可指导提示约束式缓解。客观看，其价值主要是诊断性分类和实证关联分析，而非全新生成模型；模型族泛化性仍需验证。

Abstract: Large Language Models (LLMs) can generate UML class diagrams from natural language requirements, but hallucinations may compromise the correctness and completeness of the resulting diagrams. Existing evaluations mostly assess the final diagram, without examining the intermediate traces of reasoning-capable LLMs, which can reveal where these hallucinations arise. We develop an empirical taxonomy of eleven hallucination sub-types organised into three dimensions: Prompt Faithfulness (deviations between requirements and reasoning), Code Faithfulness (discrepancies between reasoning and generated PlantUML code), and Reasoning Quality (problems within the reasoning). The taxonomy was derived from 60 manually annotated DeepSeek-R1 traces, applied to 120 further DeepSeek-R1 traces through a dual-LLM annotation pipeline, and examined on 20 DeepSeek-V4-Pro traces. Code Faithfulness was the most frequent dimension for DeepSeek-R1 in both datasets, whereas Prompt Faithfulness dominated for DeepSeek-V4-Pro, suggesting model-dependent failure patterns. In the manually annotated traces, reasoning-trace length showed no significant monotonic association with hallucination frequency, whereas requirement complexity was negatively associated with it, particularly for Prompt Faithfulness errors. In a mitigation experiment on 13 Golden UML ModelSet cases with three Qwen3-based models, adding Prompt Faithfulness and Reasoning Quality constraints to the generation prompt raised the pooled mean F1 score by 4.0 percentage points, whereas a post-hoc Code Faithfulness review provided no consistent further benefit. Relationship modelling remained the lowest-scoring element throughout. These results suggest that localising hallucinations within the generation process can inform targeted, prompt-based mitigation. Whether these findings generalise beyond the DeepSeek and Qwen3 model families remains to be tested.

eess.IV [Back]

[107] H-2DGS: Height-Field Gaussian Splatting for Sharp and Deformable Neurosurgical Surface Reconstruction eess.IV | cs.CV | cs.GRPDF

Zhang Zichuan, Bart Brouwers, Rembrandt Bakker, Maud Landers, Geert-Jan Rutten

TL;DR: 论文提出 H-2DGS，一种用于神经外科皮层表面重建的高度场 2D 高斯泼溅方法。其采用三阶段流程：先进行鲁棒 3DGS 重建，再将深度融合到共享 TSDF 表面，最后用曲面导向的高斯基元表示该表面；同时通过分离高度、颜色、不透明度基以及邻居过渡目标协调重叠块。方法还支持术后 MRI 深度形变和跟踪颜色监督的交替优化。在五个真实神经外科病例中实现低多视角深度不一致、7.97MB 紧凑表示和 29–42ms/帧交互渲染，重建表面清晰可形变，并能与 MRI 解剖和纤维束成像整合。

Details

Motivation: 神经导航通常以 2D MRI 切片呈现，难以与手术显微镜下观察到的皮层表面对应，因此需要统一的 3D 可视化。但有限手术视角下传统 3DGS 容易形成厚且几何模糊的不透明度带，遮挡内部结构，论文旨在重建具有清晰边界的皮层表面。

Result: 在五个真实神经外科病例上，H-2DGS 取得较低的多视角深度不一致，模型表示仅 7.97MB，并达到每帧 29–42ms 的交互渲染速度。重建的清晰可形变表面可与纤维束成像、MRI 解剖以及术后切除腔清楚整合。

Insight: 创新点在于将高度场/曲面导向的 2D 高斯基元与 TSDF 表面融合结合，把 3DGS 重建转化为更显式、几何一致的表面表示，并通过分离基和邻居过渡目标提升局部几何与外观一致性。客观来看，该工作针对有限视角手术场景的几何模糊问题，兼顾了重建质量、紧凑性和交互渲染，具有较强实际应用价值。

Abstract: Neurosurgical navigation commonly presents magnetic resonance imaging (MRI) as 2D slices, making it difficult to relate the anatomy in those slices to the cortical surface observed through the surgical microscope. We seek a common 3D visualization that combines a photorealistic reconstruction of the exposed cortex during tumor surgery with whole-brain MRI context and diffusion tractography pathways. This requires the reconstructed cortex to form a clear boundary around the brain anatomy rather than a diffuse volume that obscures internal structures. Conventional 3D Gaussian Splatting (3DGS) can reproduce observed views but, under limited surgical view coverage, may distribute opacity through a thick and geometrically ambiguous band. We introduce height-field 2D Gaussian Splatting (H-2DGS), a three-stage pipeline that concentrates a robust 3DGS reconstruction, fuses its depth into a shared truncated signed distance field (TSDF) surface, and represents that surface with curved, surface-oriented Gaussian primitives. Separate height, color, and opacity bases provide local geometry and appearance, while a neighbor-transition objective coordinates overlapping patches. The explicit surface further supports postoperative alignment: MRI-derived depth deforms the reconstruction, and tracked-color supervision with alternating optimization preserves its appearance. Across five real neurosurgical cases, H-2DGS gives low multi-view depth inconsistency, a compact 7.97MB representation, and interactive rendering at 29–42ms per frame. The resulting sharp, deformable surface integrates clearly with tractography and MRI anatomy, including a postoperative resection cavity.

[108] Vision Transformers for Estimating Irradiance Using Data Scarce Sky Images eess.IV | cs.AI | cs.CV | cs.LGPDF

David Hamlyn, Sunny Chaudhary, Tasmiat Rahman

TL;DR: 论文面向天空图像数据稀缺场景下的漫射水平辐照度（DHI）估计，提出一种融合物理信息与无监督云分割的双框架方法。方法先利用地外辐照度和循环时间编码等大气参数表征天空条件，再通过先预测合成全球水平辐照度（GHI）并作为特征来精化DHI估计，同时采用双并行架构处理原始与叠加增强的鱼眼天空图像。在Chilbolton天文台数据上，仅用单月训练并在全年不相交测试集评估，取得接近完整数据训练SOTA ViT的效果。

Details

Motivation: 准确估计DHI对光伏系统优化和能源预测很重要，但缺乏地面仪器的地区难以实现；现有ViT方法依赖大规模长期高质量天空图像，数据稀缺严重限制实际部署。

Result: 在Chilbolton天文台数据上，以单月数据训练并在时间不相交的全年测试集评估，顺序框架和双并行框架的RMSE分别与完整数据集训练的SOTA ViT相差约2-3 W/m²和1-6 W/m²。

Insight: 创新点在于将物理信息大气参数与时间编码、GHI辅助的序贯估计、原始与物理信息无监督云分割叠加图像的双并行处理相结合，以较低数据依赖提升DHI估计的可扩展性和成本效益。

Abstract: Accurate estimation of diffuse horizontal irradiance (DHI) is critical for optimising photovoltaic system performance and energy forecasting yet remains challenging in regions lacking comprehensive ground-based instrumentation. Recent advancements using Vision Transformers (ViTs) trained on extensive sky image datasets have shown promise in replacing costly irradiance measurement equipment, but the scarcity of long-term, high-quality sky imagery significantly restricts practical implementation. Addressing this critical gap, this study proposes a novel dual-framework approach designed for data-scarce scenarios. First, calculated atmospheric parameters, including extraterrestrial irradiance and cyclic time encodings, are integrated to represent sky conditions without utilising any instrumentation. Next, a sequential pipeline initially predicts synthetic global horizontal irradiance (GHI) and uses it as a feature, to refine DHI estimation. Finally, a dual-parallel architecture simultaneously processes raw and overlay-enhanced fisheye sky images. Overlays are generated through unsupervised, physics-informed cloud segmentation to highlight dynamic sky features. Empirical validation is performed using data from the Chilbolton Observatory, chosen for its temperate climate and frequent cloud variability. To simulate data-scarce conditions, models are trained on a single month (e.g., January) and evaluated across a temporally disjoint, full-year test set. Under this setup, the sequential and dual-parallel frameworks achieve RMSE values within 2-3 W/m^2 and 1-6 W/m^2, respectively, of a state-of-the-art ViT trained on the complete dataset. By combining physics-informed modelling with unsupervised segmentation, the proposed method provides a scalable and cost-effective solution for DHI estimation, advancing solar resource assessment in data-constrained environments.

[109] GateFormer: Forecast-Space Decomposition for Intra-Hour Solar Irradiance Prediction from Sky Images eess.IV | cs.AI | cs.CV | cs.LGPDF

David Hamlyn, Sunny Chaudhary, Jonathon Hare, Tasmiat Rahman

TL;DR: 论文提出 GateFormer，一个以实测 GHI 为条件的双解码器 ViT，用于基于天空图像的日内小时级（intra-hour）全局水平辐照度（GHI）预测。方法的核心是“预测空间分解”：将实测 GHI 嵌入视觉 token 序列，与天空图像 patch 特征通过自注意力交互；共享表征分出 Trajectory Head（预测未来 GHI 序列）和 Delta Head（预测最终时域相对最新观测 GHI 的变化量），两者在预测空间自适应加权，并用幅度与方向监督对残差做校正。

Details

Motivation: 亚小时级调度、平衡与储能控制需要能预判太阳能发电快速波动的预测；准确的天空图像辐照度预测既要融合云层视觉动态与近期辐照度状态，又要适应不同的短期辐照度变化模式，而传统做法难以兼顾。

Result: 在 NREL-TSI 上 +30 分钟 RMSE 为 125.01 W/m²，配对时间戳评估下比 SOTA Transformer 基线提升 9.6%；在 NREL-TSI、Chilbolton、SIRTA 三个基准上取得最低最终时域 RMSE，相对 smart persistence 的预测技巧为 22.97%–28.86%；消融显示自适应加权优于单解码器与固定平均，监督校正进一步降低 RMSE 1.68 W/m²、MAE 4.86 W/m²，正爬坡工况下 RMSE 降低 26.27 W/m²。

Insight: 创新点在于把“预测组合”从输出后处理搬到预测空间内部：用双头分别建模轨迹与增量，并以可学习的自适应权重融合，再以幅度/方向监督显式修正残差；同时把实测 GHI 作为 token 注入 ViT，实现图像与辐照度状态的多模态早期融合。客观来看，其可借鉴之处是把不同预测范式（绝对轨迹 vs 相对变化）的互补性显式建模并端到端学习融合权重，而非固定平均。

Abstract: Sub-hourly dispatch, balancing, and storage control increasingly depend on forecasts that anticipate rapid variability in solar generation. Accurate sky-image-based irradiance forecasting requires a combination of visual cloud dynamics with recent irradiance state while adapting to different short-term irradiance-change regimes. To address this, we introduce GateFormer, a GHI-conditioned dual-decoder ViT for intra-hour global horizontal irradiance (GHI) forecasting through forecast-space decomposition. GateFormer embeds observed GHI into the visual token stream, allowing interaction with sky-image patch features through self-attention. From this shared representation, a Trajectory Head predicts future GHI sequence, while a Delta Head predicts final-horizon change from the latest observed GHI. The two final-horizon forecasts are adaptively weighted in prediction space, with magnitude- and direction-supervision targeting the remaining error. On the NREL-TSI benchmark, GateFormer attains 125.01 W/m^2 RMSE at +30 min, improving over a state-of-the-art transformer baseline by 9.6% under paired-timestamp evaluation. Across NREL-TSI, Chilbolton, and SIRTA, GateFormer achieves the lowest final-horizon RMSE among the evaluated baselines, with forecast skill of 22.97-28.86% relative to smart persistence. Component ablations show adaptive weighting outperforms both individual decoders and fixed averaging, distinguishing GateFormer from conventional forecast combination, while the supervised correction further reduces RMSE by 1.68 W/m^2 and MAE by 4.86 W/m^2. Ramp-regime diagnostics show larger gains during operationally important positive ramps, reducing RMSE by 26.27 W/m^2. These results identify forecast-space decomposition as an alternative forecasting formulation, with potential relevance to other multimodal short-horizon forecasting tasks and deployment using only sky images and measured GHI.

[110] Self-Supervised Multibeam Sonar Restoration via Coherent Forward Modeling eess.IV | cs.CV | eess.SPPDF

Zixin Wang, James Hopgood, Mike Davies

TL;DR: 论文提出一种物理一致的自监督框架，用于多波束成像声纳的联合去斑与去模糊。方法基于随机相干成像模型构造条件均值前向算子，并利用配准多帧声纳序列在无干净真值条件下训练，同时引入鲁棒等变成像稳定病态反演，并提出无真值的前向预测误差评估指标。

Details

Motivation: 多波束声纳图像常受相干斑点和系统模糊双重退化，而干净反射率目标难以获取；现有去斑方法多依赖简化噪声模型、合成干净目标或盲点自监督，且通常不联合建模去斑与去模糊。

Result: 在两个真实多波束声纳数据集上，与代表性基线相比，方法在前向预测一致性、均匀区域斑点抑制和可见结构边界保持之间取得较好平衡。

Insight: 创新点在于用相干成像物理模型推导条件均值前向算子，并证明理想帧间条件下多帧配对损失可等价于前向预测损失；结合随机相干模拟的鲁棒等变成像与无真值评估指标，提升了自监督反演的可解释性和稳定性。

Abstract: Multibeam imaging sonar produces intensity images degraded by coherent speckle and system-induced blur, while clean reflectivity targets are rarely available. Existing despeckling methods often rely on simplified intensity-domain noise models, synthetic clean targets, or blind-spot self-supervision, and typically do not jointly model deblurring and speckle suppression. We propose a physics-consistent self-supervised framework for joint despeckling and deblurring of multibeam imaging sonar. A stochastic coherent image-formation model yields a conditional-mean forward operator that relates latent scene reflectivity to expected intensity through the squared magnitude of the system point spread function. We then use registered multi-frame sonar sequences to train without clean ground truth images, showing that, under an idealized inter-frame conditional-mean condition, the expected paired loss equals the corresponding forward-prediction loss up to a network-independent constant. To stabilize the ill-conditioned inverse problem, we further adapt robust equivariant imaging using stochastic coherent simulation and introduce a paired forward prediction error metric for ground-truth-free evaluation. Experiments on two real multibeam sonar datasets demonstrate a favourable balance between forward-prediction agreement, homogeneous-region speckle suppression, and preservation of visible structural boundaries compared with representative baselines.

Table of Contents

cs.CL [Back]

[1] Lighthouse: On Building a Trustworthy Digital Assistant for Exploring History with Civil Rights Archives cs.CL | cs.AIPDF

[2] FusionEval: Benchmarking Model Fusion for Large Language Models cs.CL | cs.AIPDF

[3] Consistency Is Not Validity: Self-Rewarded Moral Argumentation Makes Language Models Decisive but Pliable cs.CL | cs.AI | cs.LGPDF

[4] Self-Assessment Reinforcement Learning for Search-State Awareness in Retrieval-Augmented Large Language Models cs.CL | cs.AIPDF

[5] Behind Aggregate Scores: What Systematically Varying Arena-Hard’s LLM Judge Reveals cs.CL | cs.AI | cs.CYPDF

[6] Spectral Smoothing and Spiking Perturbations Reconfigure Capability Profiles and Selective Task Performance in Large Language Models cs.CL | cs.AIPDF

[7] From Clinical Judgment to Checkable Rules: Clinician-Led Guardrails for Mental-Health Chatbots cs.CL | cs.AI | cs.CYPDF

[8] ULME: Predictive Coupling for Mutually Reinforcing Embedding and Language Modeling cs.CL | cs.AIPDF

[9] Invisibilization as a Source of Probability Bias in Language Models cs.CL | cs.CYPDF

[10] TRIAD: Benchmarking Omni-Modal Ambiguity in Multimodal Large Language Models cs.CLPDF

[11] From Perpetrators’ Digital Footprints to Behavioral Communities: An Ontology-Driven Framework for Bengali Cyberbullying Analysis— FULL VERSION cs.CLPDF

[12] ReflectPO: Reflection-Guided Policy Optimization for Long-Horizon LLM Agents cs.CLPDF

[13] Verbalizing Activation Differences with Natural Language Autoencoders cs.CLPDF

[14] RetiJev: Fast and Source-Grounded Ophthalmic Decision Models cs.CL | cs.AIPDF

[15] Chunk Recomputation under Real Document Revisions: Measuring Update Cost and Retrieval Quality of Fixed, Recursive, and Content-Defined Chunking cs.CL | cs.IRPDF

[16] Prompt Sensitivity in Tool-Augmented Model Comparisons cs.CL | cs.AI | cs.LGPDF

[17] CoWriter: Guiding Methodological and Synergetic Human-LLM Co-Authoring cs.CL | cs.AI | cs.HCPDF

[18] Cola2: Co-Adaptive Hierarchical Language Modeling with Continuous Semantic Compression and Balanced Discrete Generation cs.CLPDF

[19] How Depth Growth and Looping Shape In-Context Learning in Language Models cs.CL | cs.AI | cs.LGPDF

[20] Towards User Simulators That Stay On Task And Still Sound Natural cs.CLPDF

[21] Would an LLM Match into a Residency? Uncertainty, Reasoning and Image Use of Open Language Models on the Spanish MIR Examination cs.CL | cs.AI | cs.CY | cs.LGPDF

cs.CV [Back]

[22] V-JEPA 2 Meets AQA: Phase-Based Scoring and Reference-Based Feedback for Mountain Biking cs.CV | cs.AIPDF

[23] EgoHand: Monocular 3D Hand Reconstruction from Egocentric Video cs.CV | cs.LGPDF

[24] R4DGS: Referring Segmentation in 4D Gaussian Splatting cs.CVPDF

[25] MoRE: Motion-Regularized Encoding for Articulated 3D Generation cs.CVPDF

[26] Foundation4D: Towards a unified perception and reconstruction model for autonomous vehicles cs.CVPDF

[27] Latent Space Jam: Layout Guided Video Synthesis cs.CVPDF

[28] VisCurate: Evaluating Behavioral Equivalence of Visual Skills cs.CVPDF

[29] SplatLab: Autonomous Scientific Discovery for 3DGS Research cs.CV | cs.AIPDF

[30] AVAtt: A Free-Viewing Eye-Tracking Dataset and Benchmark for Visual Attention on Paintings cs.CV | cs.HC | eess.IVPDF

[31] Unified Trajectory and 3D Pose Forecasting for Multi-Agent Sports Analysis cs.CVPDF

[32] Visual Grounding First, Multimodal In-context Learning Follows cs.CV | cs.AIPDF

[33] KinoUAV: A Benchmark for Safe and Stable UAV Vision-Language Navigation with Kinodynamic Modeling cs.CVPDF

[34] GSHeadRelight: Fast Relightability for 3D Gaussian Head Synthesis cs.CVPDF

[35] FedBST: Bilateral Self-Training for Unsupervised Federated Classification cs.CVPDF

[36] CODI: Improving Self-Supervised Vision Transformers with Cross Distillation cs.CVPDF

[37] Asking Less, Adapting Better: Feedback-Efficient Test-Time Adaptation for Vision-and-Language Navigation cs.CV | cs.AIPDF

[38] One-shot federated unsupervised domain adaptation with Smoothed Knowledge Distillation and teacher refinement cs.CV | cs.AIPDF

[39] Class-Grounded Batik Captioning under Scarce Text Annotation: Label Retention and Source-Specific Error Propagation cs.CVPDF

[40] AgentDreamer: Agentic 3D Motion Reasoning for Text-Controllable Physics Simulation cs.CVPDF

[41] Zero-Shot Domain Generalization via Domain Prompt Learning cs.CVPDF

[42] PhyEvolve: Multi-Agent State-Evolution Planning for Physics-Aware Video Generation cs.CVPDF

[43] Learning from Next-Frame Prediction: Towards Unified and Scalable Visual Representation Learning cs.CV | cs.AIPDF

[44] SmolVLA-MoE: Sparse Mixture-of-Experts for Efficient Multi-Task Robotic Manipulation cs.CVPDF

[45] Where Did the Light Come From? Diagnosing Cast-Shadow Reasoning in VLMs and Borrowing the Missing Shadow Prior from Image Generation Models cs.CVPDF

[46] FilmFlow: Agentic System for 3D Virtual Film Cinematography cs.CVPDF

[47] DeXplicit: Making Physical State Explicit in Dexterous Manipulation World Models cs.CVPDF

[48] DRILL: Training World Models to Improve Policies, Not Predict Pixels cs.CVPDF

[49] Loop to Look Better: Learning Adaptive Depth for Question-Guided Perception in Vision-Language Models cs.CVPDF

[50] RECAP: Looking Once Is Not Enough for Vision-Language Reasoning cs.CVPDF

[51] CAVC: Convergence-Adaptive Audio-Visual Cache for Modality-Asymmetric Diffusion Generation cs.CV | cs.SDPDF

[52] Direct Forcing: A Single-Stage Training Recipe for Few-Step Autoregressive Text-to-Video Generation cs.CV | cs.AIPDF

[53] Verus-Insight: Towards Explainable Artifact Detection in AI-Generated Videos cs.CVPDF

[54] Query-Time Calibration of CLIP Features for Open-Vocabulary 3D Segmentation cs.CV | cs.AIPDF

[55] Watch and Learn: Teaching Visual In-Context Learning by Removing Textual Shortcuts cs.CVPDF

[56] Making MLLM Latents Matter: Factor-Aware Predictive Learning for Cinematic Video Generation cs.CVPDF

[57] Diagnostic Loss: Asymmetric Supervision for Radiology Report Generation cs.CV | cs.LGPDF

[58] ResGVC: Residual Coding for High-fidelity Generative Video Compression cs.CVPDF

[59] I$^2$-KAN: Interpretable Intra- and Inter-Modal Multi-Pathway Representation Learning for Multimodal Sentiment Analysis cs.CVPDF

[60] Where Do Long Captions Fail? Position-Aware Diagnosis and Reinforcement Learning for Detailed Image Captioning cs.CVPDF

[61] Through a PRISM, Clearly: Step-Sequence Reasoning for Mistake Recognition in Procedural Videos cs.CV | cs.CLPDF

[62] WildDepth-v2: A Synchronized RGB–LiDAR–Radar Dataset and Benchmark for Metric Depth, Motion, and Tracking of Wildlife cs.CVPDF

[63] GM-GReFEL: Geometry-Guided Spatiotemporal State-Space Learning for Robust Dynamic Facial Expression Recognition cs.CV | cs.LGPDF

[64] RankAlign: Vision-Language Representation Alignment via Rank Transformation cs.CVPDF

[65] Beyond Watching and Searching: Benchmarking Agents for Video-centered Deep Research Reports cs.CV | cs.CLPDF

[66] HiSTEVE 1.0: Harnessing Human-Interface VLA Agents for Open-World Games cs.CVPDF

[67] A Visual Grounding Benchmark for Retinal OCT with Multimodal Large Language Models cs.CVPDF

[68] Moving Regions, Not Moving Objects: Segmenting What Actually Moves under Camera Motion cs.CVPDF

[69] Lightweight Frame Compensation for Real-Time Polyp Segmentation under Frame Skipping cs.CVPDF

[70] Reinforcing intra-operative tumour localisation with weak supervision from pre-operative appearances cs.CVPDF

[71] ReCoG: Relational Concept Graph for Training-Free Personalization cs.CVPDF

[72] Retrieval-Augmented Online Textual Memory for Episodic Memory Video Question Answering cs.CVPDF

[73] Data-Driven Vision-Language Models for Remote Sensing: A Survey cs.CV | cs.LGPDF

[74] MVLoMa: Multi-View Local Feature Matching with Any Geometry Foundation Model cs.CVPDF

[75] WeVisOne-VL: Unified Visual Perception from Objects to Pixels cs.CVPDF

[76] A Task-Centric Survey of Long-Document Visual Question Answering: Taxonomy, Evolution, and Future Directions cs.CV | cs.CL | cs.IRPDF

[77] AnyGameRenderer: Entity-Consistent Generative Rendering Across Games and an Entity-Level Benchmark cs.CVPDF