cs.CL [Total: 16]
cs.CV [Total: 82]
cs.AI [Total: 8]
cs.HC [Total: 1]
cs.LG [Total: 1]
cs.RO [Total: 2]

cs.CL [Back]

[1] CCDWIKI: Decoupling Case Formulation from Response Generation in Longitudinal CBT Dialogue cs.CLPDF

Mingwen Zhang, Ruiyang Song, Qinying Wang, Yuchen Wang, Changsheng Ma

TL;DR: 论文提出CCDWiki，将纵向认知行为治疗对话中的个案概念化与回应生成解耦：它作为即插即用顾问，把每个临床假设编码为可变条目，并绑定支持或矛盾证据，跨会话动态强化、保留或撤回假设，再将概念化结果提炼为简洁指导，供未修改的治疗语言模型生成回应。

Details

Motivation: 现有对话系统多通过检索或静态画像跟踪会话历史，缺乏对假设证据状态（暂定或已确立）的表示能力，容易把暂定解释过早固化为事实，从而偏置后续治疗干预。

Result: 实验表明，该方法在概念化质量和干预质量上均有显著提升，且无需重新训练骨干语言模型；摘要未给出具体benchmark或定量数值。

Insight: 创新点在于将纵向CBT治疗形式化为基于证据的假设裁决过程，并用可变证据绑定条目显式维护假设的认知状态，从而把个案概念化从回应生成中解耦，作为可插拔模块增强现有治疗模型。

Abstract: Cognitive Behavioral Therapy (CBT) demands more than empathic validation, requiring instead a structured formulation of the interplay among situations, automatic thoughts, emotions, and behaviors. Therapists formalize this interplay through a Cognitive Conceptualization Diagram, where maintaining longitudinal coherence necessitates continuous evidential adjudication to separate tentative hypotheses from established findings. Current dialogue systems track conversational history via retrieval or static profiles but lack the representational capacity to encode epistemic status. Consequently, provisional interpretations are often prematurely reified as established facts, inadvertently biasing downstream interventions. Addressing this limitation, CCDWiki is introduced as a plug-and-play advisor conceptualizing longitudinal therapy as evidence-grounded hypothesis adjudication. The approach encodes each clinical hypothesis as a mutable entry bound to corroborating and contradicting evidence. Across sessions, accumulating data strengthens supported entries, preserves provisional ones, and retracts contradicted assumptions. The resulting formulation distills into concise guidance for unmodified therapeutic language models. Experiments demonstrate substantial improvements in both conceptualization and intervention quality without requiring backbone retraining.

[2] Contextual Pictograms for Repairing Multilingual ASR Errors cs.CL | cs.HCPDF

Mariana Hernandez, Hector Luna, Carlos Bruno, Luis Ricardo Santos, Elena Simoes

TL;DR: 论文提出一种用于修复多语言ASR转写错误的交互界面，将备选转写、音频回放与针对不确定概念的上下文相关图形符号结合，初期支持法语和西班牙语的功能性话语。作者计划通过平衡实验评估图形反馈能否帮助听者恢复说话者意图，并分析语义不匹配和额外交互时间。

Details

Motivation: 语音转写错误在竞争词不熟悉或视觉相似时难以被用户察觉和修正，论文关注如何帮助听者识别并修复多语言ASR错误。

Result: 摘要未报告实验结果；论文计划开展平衡实验，验证图形反馈是否优于仅提供文本备选，并量化语义不匹配与额外交互时间。

Insight: 创新点在于将备选转写、回放和上下文校验的图形符号结合，用于不确定概念的ASR错误修复，并显式考虑错误图像可能强化误解的风险及多语言场景。

Abstract: Speech transcription errors can be difficult to recognize when competing word hypotheses are unfamiliar or visually similar. We describe an interface that combines alternative transcripts, replay and contextually checked pictograms for selected uncertain concepts. The system will initially support French and Spanish functional utterances. A counterbalanced study will assess whether pictorial feedback helps listeners recover the speaker’s intended action beyond text alternatives alone. We will also quantify semantic mismatch and additional interaction time, since an inappropriate image can reinforce the wrong interpretation rather than repair it.

[3] Phonological Knowledge of Chinese Characters in Language Models cs.CLPDF

Xiaozhe Ji, Renfen Hu, Hai Hu

TL;DR: 论文提出 PinyinBench 基准，系统评估语言模型对汉字语音知识（同音字、声母、韵母、声调、多音字）的掌握与表征方式，并通过行为评估、分层探针和激活引导研究 17 个纯文本与多模态模型。研究发现模型对声母、韵母掌握较强，但声调信息更难访问和操纵，且音频训练提升可解码性却不必然提升因果操控效果。

Details

Motivation: 关注语言模型究竟掌握多少汉字语音知识，以及这些知识如何被表征和使用，尤其区分声母、韵母等音段信息与声调信息。

Result: 在 PinyinBench 上，前沿模型 DS-v4-flash 在两字无共同语音特征的同音判断中达到 90%+，但两字仅声调不同时准确率降至 72%；探针显示声调从 embedding 高度可解码，音段信息随深度提升明显，激活引导对音段有效而对声调效果较弱。

Insight: 创新在于构建受控中文语音基准并联合行为评估、分层探针与激活引导，揭示音段与声调具有不同的表征和功能轮廓；客观看，其价值在于强调声调是语言模型语音知识中的难点，且可解码性不等于可因果操控性。

Abstract: How much phonological knowledge do language models (LMs) possess about Chinese characters, and how is this knowledge represented and used? We introduce PinyinBench, a controlled benchmark covering homophones, phonological features (initials, finals, and tones), and polyphonic characters. We investigate 17 text-only and multimodal LMs through three complementary approaches: behavioral evaluation, layer-wise probing, and activation steering. Behaviorally, models demonstrate substantial knowledge of initials and finals, with a frontier model (DS-v4-flash) scoring 90%+ in homophone judgment, when two characters share no phonological features. However, its accuracy drops sharply to 72% when two characters differ only in lexical tone. Probing reveals that tone is highly decodable from embeddings and improves modestly across layers, whereas segmental information starts less decodable and improves substantially with depth. Activation steering shows that segmental representations can systematically influence generation, while tone exhibits much weaker effects. Furthermore, audio-bearing training improves phonological decodability, including tone, without corresponding improvements in steering effectiveness. Together, these findings reveal distinct representational and functional profiles for segmental and tonal information, with tonal information much harder to access and manipulate, demonstrating the necessity for combined evaluation, probing, and causal intervention for understanding linguistic knowledge in LMs.

[4] Learning When Human Decisions Matter: Executor-Relative Consultation Value for LLM Agents cs.CL | cs.AIPDF

Shuli Jiang, Khushbu Pahwa, Yuting Zhang, Yi Zhang, Wei Xia

TL;DR: 论文研究LLM智能体在有限人类咨询预算下应针对哪些决策向人类求助的问题。作者提出咨询价值蒸馏（CVD），用蒙特卡洛执行者 rollout 离线估计与具体执行者相关的人类咨询价值，并蒸馏为轻量预测器用于预算分配，部署时无需再做 rollout 价值估计。

Details

Motivation: LLM能力提升但人类决策容量有限，而人类仍能提供私有上下文、权威判断等智能体缺失的信息；核心难点是咨询价值依赖下游执行者，同一个人工输入对不同智能体的收益不同。

Result: 在五个批量和三个序列设置、两个前沿执行者上，CVD通过联合与延续感知的价值学习改进分配；在适用场景下总体优于直接提示和Value of Information，并在显著减少rollout监督时保持稳健，低搜索成本下近似分配几乎无损。

Insight: 创新点在于将人类咨询价值建模为执行者相关价值，并用离线蒙特卡洛rollout蒸馏轻量预测器，避免部署时昂贵价值估计；联合与延续感知学习也提升预算分配效果。

Abstract: Large language models increasingly automate complex tasks, but human decision capacity does not scale with their capabilities. In many applications, humans remain essential for providing private context, authority, or judgments unavailable to the agent. We formulate human decision capacity allocation: given multiple consultation opportunities and a limited budget, at which decisions should an LLM agent consult a human? The central challenge is estimating consultation value, which is executor-dependent because the same human input may benefit different downstream agents differently. We introduce Consultation Value Distillation (CVD), which estimates consultation values offline using Monte Carlo executor rollouts and distills these values into lightweight predictors for budgeted allocation, requiring no rollout-based value estimation at deployment. Across five batch and three sequential settings with two frontier executors, CVD improves allocation through joint and continuation-aware value learning, while independent consultation values remain competitive in several settings. CVD also broadly outperforms direct prompting and Value of Information where applicable, remains robust with substantially less rollout supervision, and enables nearly lossless approximate allocation at low search cost.

Yuliang Liu, Xinyu Wang, Xiang Bai

TL;DR: 论文研究未翻译文本中，模型将未见语言引入共享语义空间时，起关键作用的是熟悉文字还是亲属语言。作者提出 GlyphMind：一种无分词器的视觉 Transformer，将文本作为像素读取，并在 27 种书写系统间对齐翻译。实验表明，熟悉文字而非语言亲缘关系决定未见语言能否进入共享语义空间。

Details

Motivation: 许多语言只有书面文本而几乎没有翻译，模型能否利用未翻译文本学习并接入多语言语义空间尚不明确；已有因素中“亲属语言”和“熟悉文字”常同时出现，难以区分。

Result: 在从未见过的未翻译老挝语上适配后，GlyphMind 检索到英语翻译的频率是最强简单基线的 7 倍；将老挝语改写为不熟悉文字后无法进入，即使其近亲泰语在预训练中；Santali 用熟悉的天城文可进入，用其本族文字则不行。

Insight: 创新点在于用无分词器视觉 Transformer 直接读文字像素，并在多书写系统间对齐翻译，从而分离“语言亲缘”与“文字熟悉度”。客观看，其核心发现是：文字/脚本熟悉度是未见语言进入共享语义空间的关键，转写为熟悉文字可为无翻译语言打开多语言模型接入路径。

Abstract: Hundreds of languages have written text but almost no translations, and whether a model learning one from untranslated text builds on a related language or familiar letters is unclear: they usually coincide. Here we show that familiar letters, not a related language, let an unseen language into a shared space of meaning. GlyphMind, a tokenizer-free vision transformer that reads text as pixels and lets any sentence be shown in any script, was pretrained to align translations across 27 writing systems. Adapted on untranslated Lao, a language and script it had never seen, it retrieved English translations seven times as often as the strongest simple baseline. Rewritten in unfamiliar letters, Lao did not enter although its close relative Thai was in pretraining; Santali, which had none, entered in Devanagari, familiar from Hindi, but not in its own script. Transliteration into familiar scripts may thus open multilingual models to languages without translations.

[6] Holistic Parallel Supervision for Expressive Speech to Speech Translation cs.CLPDF

Hao Zhang, Ismail Rasim Ulgen, Bismarck Bamfo Odoom, Rui Liu, Zihan Zhang

TL;DR: 论文关注表达性语音到语音翻译（S2ST）中源语音与目标语音在内容、韵律、时序和说话人身份上联合对齐监督缺失的问题。作者提出结合专业配音与多语言声音转换构建整体并行监督数据，并提出端到端 DirectS2ST 直接从源语音预测完整目标语音表示。实验表明该监督能提升现有表达性 S2ST 系统的表达实现，DirectS2ST 增益最大。

Details

Motivation: 现有数据集很少在内容、韵律、时序和说话人身份四个维度同时对齐源语音与目标语音，导致系统多依赖间接条件或迁移机制；论文旨在获得更直接、整体的并行监督。

Result: 在英德和英西翻译任务上，整体并行监督使所有评估系统的表达性实现均有提升；DirectS2ST 在内容、韵律、时序、说话人相似度和人工评价表达性上取得最大增益，但摘要未给出具体数值或SOTA表述。

Insight: 摘要宣称的创新点包括首个系统研究面向表达性S2ST的整体并行监督，以及专业配音加多语言声音转换的数据构建策略和直接预测完整目标语音表示的DirectS2ST。客观看，其价值在于把四维对齐的数据监督与端到端建模结合，验证整体并行监督对表达性S2ST的有效性。

Abstract: Expressiveness is central to speech-to-speech translation (S2ST), yet existing datasets rarely align source and target speech jointly in content, prosody, timing, and speaker identity. Consequently, current systems often rely on indirect conditioning or transfer mechanisms. To obtain this supervision, we introduce a training-data construction strategy that combines professional dubbing, which provides human-performed correspondence in content, prosody, and timing, with multilingual voice conversion to reduce speaker mismatch. We evaluate this supervision using existing expressive S2ST systems and DirectS2ST, an end-to-end method developed in this work that predicts the complete target-speech representation directly from source speech. Experiments on English-German and English-Spanish show that holistic parallel supervision improves expressive realization across all evaluated systems. DirectS2ST exhibits the largest gains in content, prosody, timing, speaker similarity, and human-rated expressiveness. To our knowledge, this work provides the first systematic investigation of holistic parallel supervision for expressive S2ST.

[7] Adapting Qwen3 to Tajik from 0.6B to 8B: Lower Text Loss Did Not Mean Better Understanding cs.CLPDF

Tohir Saidzoda, Madina Abdusalomzoda, Nasimjon Rajabov, Akhrorkhon Ikromov, Komron Giyosiev

TL;DR: 论文系统研究将Qwen3（0.6B到8B）适配到塔吉克语，发现仅用LoRA在新闻和书籍上训练虽能降低塔吉克语文本loss，但并未提升阅读理解能力。只有8B模型在约3.75亿token原始塔吉克语文本上继续预训练，并经过监督微调后，主模型e3在Belebele上比Qwen3-8B高11.4分。论文还透明报告了训练记录丢失、数据未记录、4-bit发布不含transformer层等问题。

Details

Motivation: 关注小语种适配中文本loss下降是否真正代表语言理解能力提升，并评估不同规模Qwen3在塔吉克语上的LoRA、继续预训练和监督微调效果。

Result: LoRA适配器使各规模模型文本loss降低0.60到1.45 nats/token，但在Belebele上1.7B和8B准确率不变、4B下降3.7分、0.6B上升4.2分但接近随机；8B继续预训练后Belebele提升9.9分，e3最终比Qwen3-8B高11.4分，英语MMLU变化-1.2分；0.6B重配方后文本loss接近8B，但选择题接近随机且英语MMLU下降7.6分。

Insight: 核心创新在于实证揭示文本loss与阅读理解能力可能脱钩，并强调模型规模、继续预训练和监督微调阶段对小语种适配的重要性；同时系统记录失败实验和工程问题，提升研究透明度与可复现性警示。

Abstract: Small LoRA adapters trained on Tajik news and books lowered Qwen3’s Tajik text loss at every size we tried, from 0.6B to 8B, by 0.60 to 1.45 nats per token, but none of them improved Tajik reading comprehension where it can be measured. On Belebele the 1.7B and 8B adapters left accuracy unchanged and the 4B adapter lowered it by 3.7 points; at 0.6B the score rose by 4.2 points but sits near chance and coincides with a change in which answer letter the model prefers, so we do not read it as understanding. At 8B, continued pretraining on about 375 million tokens of raw Tajik text was followed by a Belebele gain of 9.9 points. After a further supervised fine-tuning stage, our main model, e3, is 11.4 points above Qwen3-8B on Belebele; its English MMLU changed by -1.2 points, and a loss larger than 3.3 points is excluded. The same heavy recipe applied to Qwen3-0.6B-Base brought its text loss close to that of the 8B model after books-only pretraining, while its multiple-choice answers stayed near chance and its English MMLU fell by 7.6 points. We report 15 trained models and their 5 starting models from June to October 2025, scored in 2026 on one harness with paired intervals, and we report what went wrong: lost training records, a thinking-mode dataset with almost no reasoning in it, later branches whose training data was never documented, and a 4-bit release that contains no transformer layers.

Xanh Ho, André Greiner-Petter, Florian Boudin, Yun-Ang Wu, Sunisth Kumar

TL;DR: 论文介绍了 NTCIR-19 的 SciClaimEval 试点任务，旨在验证科学论文中的声明是否可由论文内表格和图表支持，并构建了包含 180 篇机器学习、自然语言处理和医学论文真实声明及对比证据版本的数据集。任务包括判断证据是否支持声明和从两个证据版本中选择支持版本两个子任务，并汇总了参赛系统与结果。

Details

Motivation: 科学声明验证多关注文本证据，而表格和图表等跨模态证据的验证尚缺基准；该任务希望评估模型能否在论文内部跨模态核验声明，推动真实科研场景下的证据核验。

Result: 共有22支队伍注册、14支提交177个运行；多数基于多模态大模型通过提示和多阶段推理。子任务1最佳系统达到98.1%成对准确率，但前六名均依赖对比证据对，单独判断每个声明-证据对的系统低24.3分；开源模型在精心设计流程下可媲美闭源模型，而实际单声明核验仍是开放问题。

Insight: 创新点在于构建科学论文中声明与表格/图表配对的对比式跨模态验证数据集，并设置支持性判断与证据选择两个子任务。客观看，其价值还包括揭示对比设置下的高成绩可能高估实际单证据核验能力，强调需发展不依赖成对对比的实用验证方法。

Abstract: This paper presents an overview of \emph{SciClaimEval}, a pilot task at NTCIR-19 on verifying scientific claims against the tables and figures of the papers in which they appear. Our dataset pairs authentic claims from 180 papers in machine learning, natural language processing, and medicine with contrastive versions of their tables and figures, one supporting the claim and one minimally modified so that it does not. The task comprises two subtasks: deciding whether a piece of evidence supports a claim, and selecting which of two evidence versions does. Twenty-two teams registered, the highest number among NTCIR-19 tasks, and 14 submitted 177 runs. Most systems built on multimodal large language models, relying on prompting and multi-stage inference rather than task-specific fine-tuning. On the first subtask, the best systems reached 98.1% pair accuracy, but all six top-ranked systems relied on the contrastive pairs; the best system that judged each claim-evidence pair on its own scored 24.3 points lower. Open-weight models, embedded in carefully designed pipelines, matched proprietary ones. Yet verifying a single claim against its evidence, the setting that matters in practice, remains an open problem.

[9] The Harness Is the Model: Agent Scaffolding Changes a Fixed LLM’s Compliance with Banking Conduct Rules, Unevenly Across Models cs.CL | cs.AI | cs.CYPDF

Mehmet Murat Huyut

TL;DR: 论文研究在固定底层LLM不变的情况下，改变智能体运行框架（scaffolding/harness，即决定模型看到什么、能调用哪些工具、谁来检查其行为的系统外壳）是否会改变模型对银行行为准则的合规性。作者用4个开源模型×5种harness×156个预注册案例（共3120个两轮对话episode，覆盖11类模拟欧盟监管规则）进行确定性打分，发现harness对合规率的影响在模型间极不均衡：总体仅87.3%～92.1%，但单一模型内最佳与最差harness可差11.5个百分点，足以使其排名从第一跌到垫底。

Details

Motivation: 银行验证的是单个语言模型，实际部署的却是模型加harness组成的系统，而监管合规评估往往只针对模型本身。论文要回答：当模型固定时，harness是否会显著改变其对银行行为规则的合规表现，从而质疑“验证模型即验证系统”的惯例。

Result: 总体合规率仅随harness在87.3%～92.1%间波动，但模型内差异巨大（最大11.5个百分点，可致排名反转），另一模型则无显著差异，模型×harness交互作用只是提示性而非结论性；同模型合规审查器将违规率从9.6%降到6.4%且未增加过度拒答（与预注册预测相反）；涉及日期计算的规则在所有harness下均失败；探索性分析显示26%的违规伴随客户消息否认已执行的操作，仅看对话记录会被误判为正确拒答。

Insight: 创新点在于将评估单元从“模型”转向“模型-harness对”，并用执行工具轨迹而非对话文本做确定性合规打分，揭示了对话记录与实际行动日志之间的偏差；客观来看，其预注册设计、多harness对照和违规/过度拒答分离计数的实验框架，对LLM智能体在受监管场景下的评测方法论具有较强借鉴价值。

Abstract: Banks validate language models, but they deploy systems: a model inside a harness that decides what it sees, which tools it can call and who checks its actions. We test whether that harness changes compliance with banking conduct rules when the model is held fixed. Four open-weight models each ran five harnesses (a pipeline that makes one structured decision, a ReAct-style tool loop, a loop in which the policy is reachable only through search, a loop with a long benign customer memory, and a loop with a same-model compliance reviewer) on 156 pre-registered cases in a simulated retail bank whose 11 rule families paraphrase EU consumer-credit, payments, investor-protection, data-protection and anti-money-laundering provisions. Every one of the 3120 two-turn episodes was scored deterministically from the executed tool trace, with violations and over-refusals counted separately. Pooled across models the harness moved the correct-case rate only between 87.3% and 92.1%, but within a model it mattered a great deal: for one model the spread between its best and worst harness was 11.5 percentage points, enough to move it from first to last place among the four models, while for another the harness made no detectable difference; the model x harness interaction itself was suggestive rather than conclusive. The same-model reviewer cut violations from 9.6% to 6.4% without raising over-refusals, contrary to our pre-registered prediction. Rules that require date arithmetic failed in every harness. In an exploratory analysis with two LLM judges and a blind human check, 26% of violations were accompanied by a customer message that denied the action the agent had just executed, which a transcript-only review would record as a correct refusal. For model-risk management the unit of validation should be the model-harness pair, and conduct testing should read the action log, not the conversation.

[10] Loop Before You Leap: What Recurrent Depth Can and Cannot Buy in Data-Limited Pretraining cs.CL | cs.LGPDF

Daichi Hotta, Yohei Kobashi, Takeshi Kojima, Yusuke Iwasawa, Yutaka Matsuo

TL;DR: 论文在BabyLM 100M词数据受限预训练中检验循环深度能否缩小多模态模型与纯文本基线的差距。纯文本下，12层循环4次在匹配推理计算时比标准48层Transformer高5.4个BLiMP百分点；但多模态训练仍使BLiMP下降约3.8点，循环未明显缓解。语言数据及其呈现方式变化似乎是主要掉点来源，视觉前缀平均提升1.85点但三随机种子下不确定，模型虽使用视觉信息，其对语法泛化的贡献仍不明确。

Details

Motivation: 多模态模型在BabyLM Challenge中常低于纯文本基线；作者希望验证额外循环计算能否在不增加参数和训练数据的情况下缩小这一差距。

Result: 纯文本预注册比较中，12层循环4次比48层标准Transformer高5.4个BLiMP百分点；多模态训练在两种架构下均使BLiMP降低约3.8点。无图像对照比纯文本基线低5.7点，加入视觉前缀平均提升1.85点，但三种子下不显著。

Insight: 创新点在于统一实现下控制参数与数据预算、匹配推理计算来评估循环深度，并区分多模态语言数据/呈现变化与视觉信息本身的影响。客观看，其价值主要是揭示循环深度收益未必迁移到多模态数据受限场景，且单向量视觉接口下视觉对语法泛化的作用有限。

Abstract: Multimodal models have often scored below text-only baselines in the BabyLM Challenge (Klerings et al., 2024; Takmaz et al., 2025). We test whether additional recurrent computation helps close this gap. Recurrence increases computation by reusing layers without adding parameters or training data. Using a single implementation, we compare standard and looped Transformers under a 100M-word budget, with three seeds per main condition. Each image is represented by one feature vector from a frozen DINOv2 encoder, used as a prefix token. Caption words count toward the budget, so multimodal training also changes the language data. In a pre-registered text-only comparison, a 12-layer stack repeated four times exceeds a standard 48-layer Transformer by 5.4 BLiMP percentage points at matched inference compute. However, multimodal training lowers BLiMP accuracy by about 3.8 points in both architectures, with no clear mitigation from recurrence. An exploratory loop-side no-image control, trained with the multimodal language data and batching schedule, scores 5.7 points below the text-only baseline. Adding the visual prefix raises accuracy by 1.85 points on average, but this contrast is uncertain with three seeds. These results suggest that changes in language data and its presentation are a major source of the drop. Diagnostic tests show that the models use visual information; its contribution to grammatical generalization remains uncertain under this single-vector interface.

[11] Mass Producing Training Data with Surprising Side Effects cs.CL | cs.AI | cs.LGPDF

Yaowen Ye, Jacob Steinhardt, Sarah Schwettmann, Cassidy Laidlaw

TL;DR: 论文提出自动发现“具有惊人副作用的训练数据集”（DSSEs），即表面像普通后训练数据、但会诱导语言模型产生特定域外行为的数据。作者用LLM评判三个目标，并通过进化、强化学习和潜意识学习搜索高分数据，经重评分过滤后构建了覆盖8种行为、共6575个DSSE的数据集，许多副作用可跨模型家族和规模泛化。

Details

Motivation: 训练数据可能在训练域外对语言模型行为产生意外副作用，但现有相关例子仅数十个且多为手工构造或偶然发现，难以系统研究这种泛化现象。论文希望自动化、大规模地发现此类数据，为研究训练副作用和部署前预测奠定基础。

Result: 论文自动产出6575个DSSE，覆盖8种行为，许多数据诱导的副作用可跨模型家族和规模泛化；自动指标与ML研究者的判断一致，且前沿模型在预测副作用方面优于人类。

Insight: 创新点在于将训练数据意外副作用从孤立案例扩展为可规模化生成与评估的数据集：定义“诱导目标行为、像普通后训练数据、对人类审计者令人惊讶”三目标，用LLM评判结合进化、RL和潜意识学习搜索，并构建测试平台用于研究泛化和部署前副作用预测。客观看，其可借鉴之处是“普通数据+惊人副作用”的构造范式和自动化发现评估流程。

Abstract: Training data can have unexpected side effects on language model behavior outside the training domain: insecure code data can induce broad misalignment, and outdated bird names can induce a 19th-century persona. Understanding such generalization is hard because only dozens of isolated examples exist, each constructed by hand or discovered by accident. We propose automatically discovering such datasets with surprising side effects (DSSEs). We define three objectives for DSSEs: they should (1) elicit a target behavior, (2) read like ordinary post-training data, and (3) be surprising to human auditors reading the data. We operationalize each with LLM judges, and search for high-scoring datasets via evolution, reinforcement learning, and subliminal learning approaches. After re-scoring and filtering with held-out metrics, our pipeline produces a collection of 6,575 DSSEs across 8 behaviors, many of which generalize across model families and sizes. We find our automated metrics track ML researchers’ judgments, and frontier models outperform humans at predicting side effects. These datasets form a testbed for studying generalization and lay the groundwork for predicting training side effects before model deployment. Explore them at https://dsse.transluce.org.

[12] Twists and Turns: A Graph-Based Benchmark and Framework for Long-Term Memory in Exploration Tasks cs.CLPDF

Chongyuan Dai, Yaling Shen, Zirui Song, Junfeng Liao, Jinpeng Hu

TL;DR: 论文提出面向长期探索任务的长期记忆评测基准LExMem与图记忆框架LExGraph。作者将科研、软件开发等探索过程建模为图结构，其中分支方案通过修订与重新考虑相互连接，并构建224段对话和2018个问答实例。LExGraph将对话增量组织为互联记忆节点，并结合混合检索与邻居扩展进行证据检索。

Details

Motivation: 现有长期记忆研究主要关注事实回忆与更新，忽视了探索任务中方案试验、中间结果反馈、早期尝试修订或重启之间的复杂关联，因此需要新的评测与建模方法。

Result: 实验表明LExMem对现有基于检索和记忆的系统构成显著挑战，而LExGraph取得更优性能，并能有效泛化到外部基准。

Insight: 创新点在于用图结构刻画长期探索过程，并引入演化机制提升结构复杂度以构建基准；同时以图组织记忆节点并通过混合检索和邻居扩展提升长期记忆检索效果。

Abstract: Long-term problem-solving processes, such as scientific research and software development, proceed along an evolving trial-and-error trajectory toward a central goal. We term these \textit{exploration tasks}, which require models to remember not only facts but also how solutions are tested against intermediate outcomes and early attempts are continually revised or revived. However, existing long-term memory research focuses on factual recall and updates, overlooking the interconnected nature of exploration. We observe that exploration processes naturally form graphs, with branching approaches linked by revisions and reconsiderations. Motivated by this, we propose \textsc{LExMem}, a graph-guided benchmark designed to evaluate \underline{\textsc{Mem}}ory in \underline{\textsc{L}}ong-term \underline{\textsc{Ex}}ploration tasks. We model exploration processes as graphs and introduce an evolutionary mechanism to increase their structural complexity, yielding 224 dialogues and 2,018 question-answer instances derived from graph structures. Building on the same insight, we propose \textsc{LExGraph}, a graph-based memory framework that incrementally organizes dialogues into interconnected memory nodes and retrieves evidence through hybrid retrieval and neighbor expansion. Extensive experiments demonstrate that \textsc{LExMem} poses a significant challenge to existing retrieval- and memory-based systems, whereas \textsc{LExGraph} achieves superior performance and generalizes effectively to external benchmarks. Codes and datasets are available at https://github.com/mbzuai-nlp/LExMem.

[13] When Do Summaries Become Instructions? Unsupported Policy in LLM Agents’ Self-Written Handoff Summaries cs.CLPDF

Wenhui Chu

TL;DR: 论文研究长时运行的 LLM agent 在压缩上下文时生成的“交接摘要”是否会写入原始记录并不支持的权限或要求，这些句子是否会改变后继实例的行为，以及它们从何而来。作者构建带真实权限账本的合成工具使用任务，并用同一快照重放、将可疑句子替换为中性填充来做因果测试。结果显示，不支持的限制出现在 1.2% 和 2.4% 的可读摘要中，不支持的授权出现在 0.1% 和 1.3% 中；预注册重放对限制的影响不显著，错误批准被中性化后未授权动作有所下降，而修改系统提示以说明要求来源能显著减少不支持的限制。

Details

Motivation: 长任务 agent 依赖摘要替代大部分历史，后继实例会把摘要当作事实与规则依据；若摘要写出记录不支持的权限或要求，可能导致越权操作或过度保守，论文旨在审计这一风险并定位其来源。

Result: 两个开放权重模型的不支持限制占可读摘要的 1.2% 和 2.4%，不支持授权占 0.1% 和 1.3%；预注册重放对限制不显著（p=0.58），事后重放效应脆弱（p=0.027）；消除错误批准使未授权动作从 120 次留出运行中的 24 次降至 7 次（事后；后续 48 次中 17→11）；将系统提示改为说明要求来源后，不支持限制从开发集 5.0% 降至 1.8%，留出模板从 1.8% 降至 0.3%（p<10^-9）。

Insight: 创新在于把摘要保真问题从“信息是否准确”扩展到“规范性政策是否被虚构”，提出带真实权限账本的同快照重放与中性填充审计框架，并发现错误限制多源自 agent 把先前“该动作需批准”的推理固化为规则；轻量级系统提示改写即可显著缓解，具有较强可复现性与实践借鉴价值。

Abstract: Long-running LLM agents compact their context, replacing most history with a summary. We ask whether handoff summaries state permissions or requirements the record does not support, whether such sentences change what the next instance (the successor) does, and where they come from. Synthetic tool-use tasks carry a ground-truth permission ledger; same-snapshot replay reruns the successor with those whole sentences replaced by neutral filler. Two AI readers, reading only rule-flagged held-out summaries, confirmed unsupported restrictions in 1.2% and 2.4% of all readable summaries from two open-weight models, and unsupported authorizations in 0.1% and 1.3%. For restrictions, a pre-specified replay test was not significant (p = 0.58); a follow-up, planned after it, found a fragile effect (p = 0.027). Neutralizing false approvals cut unauthorized actions from 24 to 7 of 120 held-out runs (post hoc; later set: 17 to 11 of 48). Most development restrictions followed the agent’s earlier reasoning that an action needs approval (post hoc). In pre-specified tests, replacing the system prompt’s “Some actions need approval.” with one saying where requirements come from cut unsupported restrictions from 5.0% to 1.8% of development summaries and, on held-out templates, from 1.8% to 0.3% (both p < 10^-9), while requests for approvals the ledger requires barely changed. Code and data: https://github.com/Chuw1/handoff-audit

[14] Reasoning over Non-Parametric Memory in Multilingual LLMs: A Cognitive-Based Analysis of Relational Knowledge cs.CL | cs.AIPDF

Zohreh Abbasi Liasi, Batoul Hammoud, Anas Madkoor, Ehsaneddin Asgari

TL;DR: 论文构建了 Rel-Bloom，一个基于 30 个 Wikidata 关系生成的多语言关系推理诊断集，覆盖 Remember、Understand、Apply、Analyze、Evaluate 五个 Bloom 认知层级，以及 Factual、Counterfactual、Neutralized 三种事实性条件，共 450 道选择题。作者在 43 个模型、6 种语言和两种多语言设置下进行了 226,800 次预测，系统分析 LLM 对提示中直接给出的关系证据进行推理的能力。

Details

Motivation: RAG 假设语言模型能够依据上下文中的关系进行推理，而不是退回到预训练时记忆的关联；论文希望隔离检索环节，直接考察多语言 LLM 对上下文中给定关系证据的条件推理能力，并诊断不同认知层级和事实性条件下的表现。

Result: 开源模型系列从 Apply 到 Analyze 普遍出现大幅准确率下降，Large Granite 桶达到 47.2 个百分点，而 Gemini 模型未出现该骤降；Gemini 3.1 Pro 在其他四层平均 99.5%，但 Evaluate 仅 60.7%。反事实上下文通常难于事实上下文，题目和关系难度排序在不同语言间较一致。

Insight: 创新点在于用 Bloom 认知层级和事实性条件系统化构造多语言关系推理诊断，并在大规模模型与语言上分析非参数记忆推理。客观启示是更强模型能力并不自动保证可靠的证据条件推理，RAG 评估需区分记忆召回、上下文关系推理与高阶评估，并关注跨语言共享的难度结构。

Abstract: Retrieval-augmented generation (RAG) assumes that language models can reason over relations stated in their context rather than fall back on associations memorized during pretraining. We study this evidence-conditioned relational reasoning in isolation, with the facts supplied directly in the prompt rather than retrieved. \textsc{Rel-Bloom} is a controlled multilingual diagnostic of 450 multiple-choice questions built from 30 Wikidata relations; the same source relations are instantiated at five Bloom-inspired levels (Remember, Understand, Apply, Analyze, Evaluate) and under three veracity conditions (Factual, Counterfactual, Neutralized). Across 43 models, six languages, and two multilingual settings (226,800 predictions), three findings stand out. First, accuracy drops sharply from Apply to Analyze in every evaluated open-weight model series, reaching 47.2 percentage points for the Large Granite bucket, but not for either Gemini model. Second, Evaluate items remain a bottleneck even for models near ceiling on the other task types: Gemini~3.1 Pro averages 99.5% on the other four levels but 60.7% on Evaluate. Third, item- and relation-level difficulty rankings are consistently preserved across languages, indicating a difficulty structure partly shared across languages within this benchmark. Across knowledge conditions, Counterfactual contexts are generally harder than Factual ones, while removing entity identities can improve higher-level performance in some model regimes. Greater model capability therefore does not by itself ensure reliable reasoning over supplied relational evidence. Relevant resources are available at: https://huggingface.co/spaces/QCRI/non-parametric-memory-relational-reasoning.

[15] Ranking Agreement Does Not Reveal Paragraph-Edit Agreement in Two Released Paper Acceptance Predictors cs.CL | cs.LGPDF

Shaoliang Yang, Jun Wang

TL;DR: 论文检验快速论文接收分类器能否用于为更慢的评估器筛选段落修订。作者在198篇ICLR论文上比较原始稿和保持事实、主张强度与归因不变的段落改写，发现分类器与两个已发布接受预测器在原稿排名上有一致性，但在“哪些改写会提高分数”上几乎无一致性；改用全32位精度复核后结论仍成立。

Details

Motivation: 作者关注能否用快速接受分类器先筛选修订，再交给更慢、更昂贵的评估器，并检验现有论文上的排名一致性是否足以支持这种用法。

Result: 在198篇ICLR论文上，分类器与一个预测器的原稿排名接近，与另一个中等一致，但对改写是否提分无可检测一致性；全32位重评分后编辑一致性仍近零，每篇从5个改写中选择时平均原始分仅比随机选择高不到2/100。

Insight: 论文的核心创新是指出“排名一致性”不等于“段落编辑一致性”，并强调修订选择器必须针对实际目标评估器及其排序的修订进行验证；实验还通过自动筛选改写和精度复核增强了可信度，但结论目前限于表现类改写和同家族的两个开放预测器，尚未验证迁移到昂贵评估器。

Abstract: A fast acceptance classifier could select revisions for a slower evaluator. We ask whether agreement on existing papers supports that use. We compare a classifier of word and document features with two released text-only acceptance predictors fine-tuned from the same language model with three billion parameters. For 198 papers from the International Conference on Learning Representations, we compare model rankings of the original papers and of paragraph rewrites screened automatically for unchanged facts, claim strength, and attribution. The classifier ranks original papers closely with one predictor and moderately with the other, but shows no detectable agreement with either on which rewrites raise scores. The tests specified in advance used reduced numerical precision. Rounding error dominated the small score responses. We therefore rescored the released weights in full 32-bit precision as a post hoc check; edit agreement remained near zero. At this precision, when each predictor chooses among five rewrites per paper, its average raw score is under two points out of 100 higher than with random choice. The evidence covers presentation rewrites and two open predictors from one family; transfer to costly evaluators remains untested. Revision selectors should therefore be checked against the intended evaluator on the revisions they rank.

[16] Before You Continue: Evaluating Time-Sensitive Safety Awareness in Conversational Assistants cs.CLPDF

Peidi Dong, Yunze Xiao, Yu-Lin Tsai, Yichen Zheng, Heyuan Huang

TL;DR: 论文提出TASC基准，用于评估LLM对话助手能否在用户未明确重复时间约束时识别并应用时间相关限制。基准包含3,418个输入、128个模板及33个场景，并测试13个模型。结果显示助手在违规建议中仅30.5%考虑时间约束，且62.1%的失败发生在模型已正确计算经过时间却未应用约束的情况下。

Details

Motivation: LLM助手越来越多参与具有时间依赖约束的日常任务，但用户后续请求可能不再提及这些约束，助手需要主动识别并调整建议；论文旨在评估这种时间敏感的安全意识能力。

Result: 在13个模型上，助手仅在30.5%的违规行为案例中考虑时间约束，且很少产生不必要警告；更关键的是，62.1%的案例中模型虽正确计算经过时间却未应用约束，显式提供时间信息也仅能解决约一半失败。

Insight: 论文创新在于构建TASC基准，将时间计算能力与应用时间约束能力区分评估，并揭示时间推理与对话建议应用之间的差距；客观看，其价值在于把时间敏感安全问题转化为可量化基准，而非提出新模型。

Abstract: Large language model (LLM) assistants increasingly help users with everyday activities that involve time-dependent constraints. However, users may not mention these constraints in later requests, requiring assistants to recognize and adjust their advice accordingly. To evaluate such capability, we introduce TASC (Temporal Action Safety in Conversation), a benchmark that evaluates whether LLMs apply time-dependent constraints when responding to conversational requests. TASC consists of 3,418 inputs derived from 128 templates, along with a separate set of 33 scenarios designed to compare models’ ability to calculate elapsed time with their ability to apply that information. Across 13 models, assistants account for time-dependent constraints in only 30.5% of cases where the proposed action violates a rule, while rarely warning users unnecessarily. More importantly, models fail to apply these constraints in 62.1% of cases despite correctly calculating the elapsed time. Explicitly providing the timing resolves only about half of these failures, revealing a gap between temporal reasoning and its application in conversational advice.

cs.CV [Back]

[17] VAR-Q: Tuning-free KV Cache Quantization for Visual Autoregressive Image and Video Generation cs.CV | cs.PFPDF

Boxun Xu, Jiaji Lu, Yuxuan Yin, Zihu Wang, Ziyue Liu

TL;DR: 论文针对视觉自回归生成中KV缓存内存过大的问题，分析了视觉自回归KV缓存的离群值结构，提出了无需调优的量化框架VAR-Q，通过沿特征和块轴分组量化，显著降低KV内存并支持更大推理批次。

Details

Motivation: 视觉自回归生成中，KV缓存内存超过权重，限制了大规模部署。现有语言模型的KV量化方法不适用于视觉自回归的块级解码特性，需要专门设计。

Result: 在九个模型上实现低比特量化，KV内存减少高达78.5%，质量下降可忽略，并支持6倍大的推理批次。

Insight: 创新点在于发现视觉自回归KV缓存的离群值沿特征和块轴组织，并利用生成循环的块边界自然分组，实现无需离群检测和校准的量化，结合位打包内核降低运行时内存。

Abstract: Visual autoregressive generation spans two dominant paradigms: next-scale prediction for images and video, and next-frame prediction for video. Both decode block-by-block rather than token-by-token, yielding KV caches whose statistics diverge from those of language models. At scale, KV memory exceeds the weights and limits large-scale deployment caused by a large amount of visual tokens. Our analysis of these caches reveals three findings. First, a small set of massive-value channels carries the functional load: ablating these outlier channels collapses generation, whereas ablating a matched set of inlier channels leaves quality intact. Second, the outliers organize along two axes, feature and block, with magnitudes persistent within each cell and shifting across cells. Third, this structure arises from two independent mechanisms: projection bias and RoPE-induced scattering. Guided by these findings, we propose VAR-Q, a tuning-free KV-cache quantization framework that groups along the same two axes. The generation loop provides block boundaries for free, removing the need for outlier detection or calibration. Paired with bit-packing kernels for runtime memory reduction, VAR-Q delivers low-bit quantization across nine models spanning both paradigms, cutting KV memory by up to 78.5% with negligible quality degradation and enabling 6x larger inference batches.

[18] Visual Forensic Detection of Fabricated Bangla News Photocards using Deep Learning cs.CVPDF

Anindya Kundu, Kabya Mithun Saha, Md. Muhaiminul Islam, Fayek Ahmed, Aminul Kader Bulbul

TL;DR: 论文将孟加拉语虚假新闻图卡检测重新定义为视觉图像伪造检测，构建了包含969张真实和1077张伪造图卡的数据集，并用16种变换扩增至32736个样本。实验比较了ANN、紧凑CNN和ImageNet预训练YOLOv8n-cls，其中预训练模型在同折评估下达到97.36%准确率。

Details

Motivation: 孟加拉国大量本地化虚假信息以模仿知名媒体品牌的单图新闻摘要形式传播，误导信息存在于像素中而非可读文本，因此传统文本事实核查工具难以检测。论文旨在用视觉取证方法识别伪造新闻图卡。

Result: 紧凑四块BN CNN仅约49万参数，在无泄漏分组五折交叉验证下从头训练达87.45%±0.53%，分类头热启动达88.75%±1.55%；ImageNet预训练YOLOv8n-cls达97.36%±0.42%，显示紧凑模型之上仍有较大提升空间。

Insight: 创新点在于构建孟加拉语新闻图卡伪造数据集、将任务转为视觉伪造检测，并采用防止增强泄漏的分组交叉验证协议；客观看，方法本身偏基准与诊断分析，紧凑CNN存在欠拟合，预训练骨干的显著优势说明该任务仍需更强视觉表征。

Abstract: In Bangladesh, much localized misinformation spreads as photocards: single-image news summaries that imitate the branding of well-known news organizations. Because the misleading message lives in the pixels rather than in machine-readable text, text-based fact-checking tools cannot see it. We recast photocard fake-news detection as visual image forgery detection. We build a dataset of 969 authentic and 1,077 fabricated Bangla photocards scraped from public Facebook pages: authentic cards from the verified pages of seventeen national television broadcasters, fabricated ones from pseudo-broadcaster pages, anonymous profiles, and public groups. Sixteen photometric and morphological transformations per image grow the corpus to 32,736 samples. A fully connected ANN reaches only about 52% accuracy, its loss stuck at ln(2) ~ 0.693, showing that flattening the image destroys the spatial cues forgery detection needs. A compact, fully batch-normalized four-block CNN with 489,954 parameters, about 1,644x smaller than the ANN, is evaluated with leakage-free stratified group five-fold cross-validation, which keeps every augmented version of a card in one fold. Under this protocol the CNN reaches 87.45% +/- 0.53% pooled accuracy from scratch and 88.75% +/- 1.55% when its classifier head is warm-started from the ANN. An ImageNet-pretrained YOLOv8n-cls backbone (1.44M parameters) fine-tuned on the same folds reaches 97.36% +/- 0.42%, marking the headroom above the compact model. We report per-class precision, recall, F1, and pooled confusion matrices, diagnose and fix a one-class collapse caused by an unnormalized classifier head, and give a manual error analysis and bias-variance diagnosis showing that the compact CNNs under-fit.

[19] mmHand: Toward Pixel-Level-Accuracy Hand Localization Using a Single Commodity mmWave Device cs.CVPDF

Xiaoyu Zhang, Zhengxiong Li, Chenhan Xu, Luchuan Song, Huining Li

TL;DR: 论文提出mmHand，一种基于单个商用毫米波设备的无设备、隐私保护的动态手部定位系统。该系统通过mmImage工具将原始毫米波数据转换为二维图像表示，设计质量评估与像素空间标注框架，并提出跨模态空间特征增强模型，在毫米波雷达U-V像素坐标系中实现像素级手部定位。

Details

Motivation: 手部定位应用广泛，但现有视觉方法存在隐私问题、穿戴式方法影响舒适性，受数据模态限制。论文希望利用单个商用毫米波设备实现无设备、隐私保护且高精度的动态手部定位。

Result: 在12名受试者、三种场景下进行实验，并通过四项指标验证了系统的有效性；摘要未给出具体数值或与SOTA模型的定量对比。

Insight: 创新点包括：提出mmImage生成工具及毫米波数据的二维图像格式表示；设计面向毫米波数据的质量评估方法和像素空间标注框架；提出跨模态空间特征增强模型以提升空间特征提取和像素级定位能力。客观来看，其可借鉴之处在于用图像化表示和跨模态特征增强桥接毫米波感知与视觉式定位范式。

Abstract: The hand localization problem has been a longstanding focus due to its many applications. The task involves modeling the hand as a singular point and determining its position within a defined coordinate system. However, due to data modality limitations, existing hand localization technologies face several challenges. For example, vision-based localization raises privacy concerns, while wearable-based methods compromise user comfort. In this paper, we introduce mmHand, a new device-free, privacy-preserving dynamic hand localization system with pixel-level accuracy, using a single commodity mmWave device. We first propose a mmImage generation tool to fully extract spatial information from raw mmWave data and introduce a novel 2D image-format representation of mmWave data. Next, we design a framework that provides a new quality evaluation method and pixel space labeling for the mmWave data. Finally, we present a cross-modality spatial feature-enhanced model with high spatial feature extraction capabilities, which can accurately localize hand positions at the pixel level in the mmWave radar U-V pixel coordinate system. We evaluate the system with experiments on 12 subjects in three scenarios, and the results across four metrics demonstrate the effectiveness of our hand localization system.

Xiaofei Sun, Peng Wang, Huayao Liu, Xiaoxiao Xu, Jun Song

TL;DR: 论文提出 Omni-Encoder，用同一个完全共享参数的 SigLIP-2 Vision Transformer 统一处理图像、视频、音频和音视频输入。方法包括模态原生输入映射、将 mel 频谱转换为与视觉 patch 对齐 token 的 1D 卷积音频 tokenizer、基于绝对时间戳的多模态 RoPE，以及交替模态训练与梯度累积。实验在 16 个跨四模态 benchmark 上取得有竞争力的结果，并显示联合训练优于各模态单独训练。

Details

Motivation: 现有图像、视频、音频编码器通常独立训练，跨模态对齐主要依赖将各模态事后绑定到冻结的锚点空间，难以实现端到端联合优化。论文关注共享参数的多模态编码器是否以及如何有效工作。

Result: 在覆盖图像、视频、音频和音视频的 16 个 benchmark 上，Omni-Encoder 用一套共享参数取得有竞争力的结果；摘要称联合训练相对各模态单独训练均有提升，但未明确宣称达到 SOTA。

Insight: 核心创新是用单一共享 SigLIP-2 ViT 端到端联合建模四类模态，并通过音频 tokenizer 对齐视觉 patch、时间戳参数化多模态 RoPE 统一时空坐标、交替模态梯度累积实现稳定联合优化。可借鉴点在于避免按模态加 adapter 或锚点绑定，转而从输入、位置编码到训练调度系统性地支持参数共享。

Abstract: Contrastive pretraining has advanced image-text understanding, yet image, video, and audio encoders are still trained independently, with cross-modal alignment realized mainly through post-hoc binding to a frozen anchor space. We present Omni-Encoder, a general multimodal encoder that processes image, video, audio, and audio-visual inputs within a single SigLIP-2 Vision Transformer with fully shared parameters. Unlike anchor-centric binding or per-modality adapters, all modalities are optimized jointly end-to-end, and the framework is organized from input to training. A modality-native input stage first maps each signal into the shared embedding space, including a 1D-convolutional audio tokenizer that converts mel-spectrograms into tokens aligned with visual patches. A multimodal rotary position embedding parameterized by absolute timestamps then places visual frames and audio segments in a unified spatiotemporal coordinate system. Finally, alternating-modality training interleaves image, video, audio, and temporally aligned audio-visual batches and accumulates their gradients into one update, which we find central to stable joint optimization. Omni-Encoder achieves competitive results on 16 benchmarks across all four modalities with one shared parameter set, and joint training improves each modality over its separately trained counterpart. Code and models will be publicly released.

[21] PEEK: One-Step-Look-Ahead Exposure Bias Correction for Diffusion Sampling cs.CV | cs.AIPDF

Yunwei Bai, Ying Kiat Tan, Tsuhan Chen

TL;DR: 论文提出PEEK，一种无需训练的确定性扩散采样器插件，用于缓解采样过程中去噪网络输入与真实数据不匹配造成的暴露偏差。PEEK在每个可用去噪步先做一次临时一步更新，在“前瞻”状态评估去噪器，并与再下一步预测插值生成下一噪声状态；同时通过前缀缓存和基线噪声调度重基准缓存高效搜索插值调度。

Details

Motivation: 扩散图像生成中，去噪网络与数值采样器并不完全匹配；采样时去噪器基于偏离真实数据的输入预测下一步，形成训练-测试差距（暴露偏差），导致生成图像模糊、细节不足或形状异常。

Result: 在8个像素空间和潜空间扩散模型上，覆盖无条件、标签引导和文本引导设置及多种输出分辨率，PEEK在NFE匹配基线基础上稳定提升，FID最高改进49.3%，额外计算成本很小，每模型一次性校准最多约2.2分钟。

Insight: 创新点在于将扩散采样中的暴露偏差显式归因于去噪器与采样器不匹配，并提出训练免费的一步前瞻校正插件：用更高信号噪声状态下的预测插值修正下一状态。客观上看，其可借鉴之处是无须重训即可兼容多类扩散模型，并通过插值调度搜索与缓存机制降低额外开销。

Abstract: Diffusion image generators are successful due to accurate denoising networks and strong numerical samplers. Yet the two components are not fully matched. During sampling, the denoiser predicts for the next step based on flawed input that differ much from the real data. This train-test gap, or exposure bias, caps image generation quality (i.e., blurry, lacking in details or awkward in shape). To mitigate the problem, we propose PEEK, a novel training-free deterministic sampler plugin. At each eligible denoising step, PEEK first performs a provisional one-step update, evaluates the denoiser once at the look-ahead state, and interpolates it with the one-step-later prediction to form the next noisy state. By taking the additional step, we obtain a generally more-image-like prediction from the higher-signal noisy state, mitigating the exposure bias. To optimize the interpolation positions, PEEK searches for an interpolation schedule search made efficient by the prefix and baseline noise-schedule-rebasing caches. On 8 pixel-space and latent-space diffusion models with unconditional, label-guided, and text-guided settings across various output resolutions, PEEK consistently improves over NFE-matched baselines, with FID improvements up to 49.3% at minimal extra computation cost (e.g., the one-time-per-model calibration takes at most 2.2 minutes across our experiments.). Project website is at https://dachengruoque.github.io/PEEK-Video/.

[22] Augmentation with Reinforcement Transformation for Low-textured Scientific Sketch Image Retrieval cs.CVPDF

Zhiqiang Wang, Sheri L. Barton, Jennifer H. Johnson, Yi He, Xingquan Zhu

TL;DR: 论文针对低纹理科学素描图像检索任务，提出强化变换增强方法ART，通过多臂老虎机自适应选择旋转、翻转、缩放、拉伸等图像变换，构建奖励引导的视图库，用于预训练模型的自监督微调，从而学习更有效的素描表示并提升检索准确率。

Details

Motivation: 低纹理科学素描缺乏现代深度模型常用的表面纹理和视觉变化，学习与检索困难；同时素描对图像变换敏感，随机增强可能引入失真并损害表示学习，因此需要自适应选择有益变换。

Result: 在Mote Marine Laboratory提供的Florida manatee素描上，ART在transformed-scar retrieval上取得71.40%平均Top-1准确率，高于随机增强选择的68.83%；在location-aware sketch retrieval上达到55.84%，显著高于冻结CLIP的18.18%。

Insight: 创新点在于将数据增强选择建模为多臂老虎机问题，用奖励持续评估不同变换的有效性，并在自监督微调前自适应构建视图库；选择过程与后续表示损失解耦，避免随机增强带来的结构失真，为低纹理图像检索提供了可借鉴的自适应增强思路。

Abstract: Low-textured sketch images are widely used in scientific applications, including wildlife conservation, archaeological illustration, and forensic composites. Learning from and retrieving such images is challenging because they lack detailed surface textures and visual variations commonly exploited by modern deep learning models. This paper proposes Augmentation with Reinforcement Transformation (ART) to fine-tune pretrained models to learn effective representations for accurate sketch image retrieval. Our key observation is that sketches are particularly sensitive to image transformations, such as rotation, flipping, zooming, and stretching. While some transformations preserve their structural integrity, others may introduce distortions that hinder representation learning. Instead of randomly choosing transformations to construct fine-tuning views, ART formulates augmentation selection as a multi-armed bandit (MAB) problem, where transformations are treated as arms and their effectiveness is continuously evaluated through rewards. This enables ART to adaptively construct reward-guided view banks before self-supervised fine-tuning; the subsequent representation loss does not feed back to ART. On Florida manatee sketches provided by Mote Marine Laboratory, ART achieves 71.40% average Top-1 accuracy for transformed-scar retrieval, compared with 68.83% for random augmentation selection, and 55.84% for location-aware sketch retrieval, compared with 18.18% for frozen CLIP.

[23] SurgTransfer: Multi-Step Latent Action Modeling for Action Transfer in Surgical World Model cs.CV | cs.AI | cs.LGPDF

Saurabh Koju, Love Panta, Deepak Thapa, Rudra P. K. Poudel, Prajwal Ghimire

TL;DR: 论文提出SurgTransfer，一个面向手术世界模型的多步潜在动作建模框架，旨在从无标注手术视频中学习上下文不变的潜在动作，并实现跨不同手术上下文的手术动作迁移。方法不依赖真值运动学动作或特定任务专家标注，并在SurgToolLoc-2022和PitVis-2023数据集上结合定量指标与外科专家定性评估进行验证。

Details

Motivation: 手术世界模型有潜力用于术中指导、无风险训练和专家手术操作迁移，但现有方法多依赖真值运动学动作或专家标注，适用范围受限；同时，直接套用自然领域的潜在动作学习范式难以捕捉手术流程的分层性和目标导向结构。

Result: 论文在公开数据集SurgToolLoc-2022和PitVis-2023上进行了大量实验，并通过定量指标和外科专家定性评估验证方法有效性；摘要未报告具体数值结果或明确SOTA结论。

Insight: 创新点在于提出多步潜在动作学习框架，从无标注视频中学习可跨手术上下文迁移的上下文不变潜在动作，并强调建模手术过程的分层与目标导向结构；客观上看，其贡献主要是将自然视频中的潜在动作迁移范式扩展到手术领域，并针对手术流程特性进行多步建模改进。

Abstract: Surgical world models have the potential to provide intra-operative guidance, accelerate risk-free training and facilitate transfer of expert surgical maneuvers. However, the use of world models for transferring surgical expertise has been largely unexplored. Moreover, recent approaches in this domain have mainly relied on ground truth kinematics actions or expert annotations for specific tasks, limiting their applicability beyond the narrow range of tasks with ground-truth annotations. In contrast, existing works in the natural domain have shown promising results in learning latent actions solely from unlabeled videos and transferring the learned actions across contexts. We observe that this paradigm is short-sighted for the surgical domain, failing to capture the hierarchical and goal-directed structure of surgical procedures. Thus, we propose a novel multi-step action learning framework to learn context-invariant latent actions that are easily transferable across various surgical contexts. We perform extensive experiments on the publicly available datasets, SurgToolLoc-2022 and PitVis-2023, verifying our approach with both quantitative metrics and qualitative evaluation by surgical experts. We have published our code and the implementation details at https://github.com/bhattarailab/Surgical-Vision-World-Model.

[24] From Tokens to Trajectories: Parameter-Efficient Continuous Adaptation of Unified Multimodal Models cs.CVPDF

Gayatri Deshmukh, Onkar Susladkar, Adheesh Juvekar, Halil Kilicoglu, Ismini Lourentzou

TL;DR: 论文提出FlowShift，一种参数高效的后训练框架，将预训练的离散统一多模态模型转换为连续整流流生成器，以绕过离散token化导致的量化瓶颈。方法用连续潜变量投影替换离散视觉接口，在注意力层引入LoRA，并通过双目标速度向量蒸馏（DOVD）对齐高保真连续教师与真实连续轨迹，从而在保持语义推理和多模态理解的同时提升细粒度视觉生成质量。

Details

Motivation: 离散统一多模态模型虽具备强语义推理能力，但离散token化带来严格量化瓶颈，导致细粒度视觉细节丢失和生成图像伪影严重；论文旨在不牺牲预训练语义先验的情况下提升连续视觉生成保真度。

Result: 在领先离散UMM上应用FlowShift后，COCO-30K FID和美学分数显著提升，同时保留超过99.5%的多模态理解能力；消融实验表明DOVD是生成质量提升的关键。

Insight: 创新点在于将离散UMM后训练为连续整流流生成器而非丢弃语义先验，结合连续潜变量投影、LoRA和双目标速度向量蒸馏实现参数高效且语义不漂移的连续适配；可借鉴的是用教师与真实轨迹双目标蒸馏弥合离散到连续的生成差距。

Abstract: While discrete tokenization in Unified Multimodal Models (UMMs) establishes strong semantic reasoning, it introduces a strict quantization bottleneck that degrades fine-grained visual details, often resulting in structurally accurate but artifact-heavy images. We introduce FlowShift, a parameter-efficient post-training framework that bypasses this bottleneck by converting pre-trained discrete UMMs into continuous rectified-flow generators. Instead of discarding the backbone’s semantic priors, FlowShift replaces discrete visual interfaces with continuous latent projections and applies LoRA to the attention layers. To efficiently boost visual quality without semantic drift, we propose Dual-Objective Velocity Vector Distillation (DOVD), which aligns the model with a high-fidelity continuous teacher and ground-truth continuous trajectories. This approach drastically improves perceptual quality while preserving prompt adherence and reasoning. When applied to leading discrete UMMs, FlowShift significantly improves COCO-30K FID and aesthetic scores, while retaining over 99.5% of multimodal understanding capabilities. Ablations confirm DOVD drives generation quality. Ultimately, FlowShift achieves high-fidelity continuous generation without sacrificing pre-trained semantic grounding.

[25] From Activation to Intervention: Interpreting Sparse Visual Features through Reconstruction cs.CVPDF

Jianyang Gu, Jacob Beattie, Mridul Khurana, Julia Chae, Xi Xiao

TL;DR: 论文提出Reconstructive SAE Intervention（RecSAEI）框架，将稀疏自编码器（SAE）视觉特征的解释从依赖强激活图像的被动观察，转变为沿SAE解码器方向干预表示并重建视觉输出的主动方法。该方法旨在消歧同一位置频繁激活的多语义特征，并在多个数据集和视觉编码器上验证有效性，还将[CLS] SAE干预归因到稀疏的patch SAE解码器方向。

Details

Motivation: 传统SAE特征解释依赖最强激活的图像或像素区域，但当纹理、颜色等不同语义特征频繁在同一位置激活时，激活证据存在歧义，难以准确说明特征语义。

Result: 摘要未给出具体定量指标或benchmark对比，仅声称方法在多个数据集和多种视觉编码器上成功消歧特征含义，并能稳定目标响应、揭示基本视觉模式与高级语义之间的组合关系。

Insight: 创新点在于用主动干预和视觉重建为SAE特征语义提供直接证据，而非被动观察激活；并将[CLS]级SAE干预归因到稀疏patch级SAE解码器方向，从而暴露低层视觉模式与高层语义的组合关系。

Abstract: Sparse autoencoders (SAEs) decompose visual representations into individual features, offering a promising basis for explaining model behavior. Interpreting these features typically relies on images or pixel regions that most strongly activate them. However, this evidence becomes ambiguous when multiple features with different semantics frequently activate in the same locations, such as texture and color in the same patterns. We propose Reconstructive SAE Intervention (RecSAEI), a framework that shifts feature explanation from passive observation to active visual reconstruction. By intervening on representations along SAE decoder directions and reconstructing the visual output, we provide direct evidence of a feature’s semantic meaning. We demonstrate that our method successfully disambiguates feature meanings across multiple datasets and vision encoders. Furthermore, our framework grounds [CLS] SAE interventions in a sparse set of patch SAE decoder directions. Beyond stable target response, this attribution also exposes the compositional relationship between elementary visual patterns and high-level semantics. Our results advocate for intervention reconstruction as an essential complementary signal for explaining neural networks.

[26] How Finely Can Vision-Language Models See Biology? Bio-VQA: A Benchmark for Taxonomic, Morphological, Spatial and Individual Reasoning cs.CV | cs.AIPDF

S M Rayeed, Qian Ma, Charles V. Stewart, Yao Ma

TL;DR: 论文提出 Bio-VQA，一个面向生物视觉理解的大规模视觉问答基准，覆盖层级分类推理、细粒度属性定位、定量空间推理和个体重识别四个方面。基准包含来自 11 个公开数据集的 21.7 万张图像和 186 万个问题，并将对错评分扩展为分级评分。作者评估了 18 个 3.75B 至 38.4B 参数的开源权重模型，发现通用视觉能力难以迁移到生物领域。

Details

Motivation: 生物多样性监测中，单张图像需要回答物种分类、数量、性状和个体是否见过等问题，但现有基准多局限于少数类群的物种识别，缺乏对视觉语言模型生物视觉能力的系统测试。

Result: 在 2 至 10 个个体的计数任务上，最佳模型仅 41.7% 的准确率，不到其在日常物体上的一半；分类准确率从门级的 0.72 降至学名的 0.12，模型规模扩展主要改善粗粒度层级。与领域专家模型相比，差距从纲级 0.731 对 BioCLIP-2 的 0.874 扩大到学名级 0.269 对 0.544；性状尺寸测量中 18 个模型有 15 个不优于固定预测基线，对抗条件下重识别中 18 个模型有 17 个低于随机水平。

Insight: 论文的创新在于构建覆盖四个生物视觉理解维度的大规模 VQA 基准，引入更细粒度的分级评分，并通过广泛模型评估揭示通用视觉能力向生物领域迁移的显著不足及规模扩展的局限，为生物领域 VLM 评测提供了系统化框架。

Abstract: In biodiversity monitoring, a single image raises several questions: which taxon, how many, which trait, and whether the individual has been seen before. Vision-language models (VLMs) could answer all of these from the image, but their ability is largely untested in biology, as existing benchmarks focus mostly on species identification within a few taxonomic groups. We present Bio-VQA, the largest biological visual question answering (VQA) benchmark and the only one to span four aspects of biological visual understanding: hierarchical taxonomic reasoning, fine-grained attribute grounding, quantitative spatial reasoning, and individual re-identification. Bio-VQA contains 1.86M questions over 217K images from 11 public datasets, and extends right-or-wrong scoring to graded scoring, so a near-miss counts for more than a distant one. Evaluating 18 open-weight models from 3.75B to 38.4B parameters, we find that general visual capabilities do not transfer to biological organisms: the best model counts animals exactly on 41.7% of images with 2 to 10 individuals, less than half its accuracy on everyday objects at the same counts. Taxonomic accuracy falls to one sixth, from 0.72 at phylum to 0.12 at scientific name, and scaling helps mostly at coarse ranks. The gap to a domain specialist widens accordingly, from 0.731 against BioCLIP-2’s 0.874 at class (1.2x) to 0.269 against 0.544 at scientific name (2x). On trait size measurement, 15 of the 18 models do no better than an image-blind baseline that predicts one fixed number for every specimen, and on re-identification, 17 of the 18 models fall below chance under adversarial conditions. These failures persist across a tenfold range of model size. We hope Bio-VQA becomes a standard test of how well VLM capabilities transfer to the biological domain.

[27] Discarded or Unreachable? Diagnosing and Repairing Spatial Relation Failure in Vision–Language Similarity cs.CV | cs.AIPDF

Mahdi Mostajabdaveh, Zirui Zhou

TL;DR: 论文发现对比式视觉-语言模型在左右空间关系上几乎只能达到随机水平，并将该症状归因于两种不同故障：一是关系信息仍保留在池化嵌入中，但原生图像-文本相似度无法访问（access failure）；二是池化过程直接丢弃了关系，而稠密特征仍保留它（pooling failure）。作者提出一种零参数、查询条件化的“空间矩（spatial moment）”读出方式，从patch网格重新计算几何信息，并把它作为向量与原生嵌入并列存储，从而同时修复这两类故障。

Details

Motivation: 对比式视觉语言模型对左右等空间关系识别接近随机，但论文指出同一症状背后有“不可达”和“被丢弃”两种成因，对应的修复方式应当相反，因此需要先诊断再修复，而不是笼统地改进相似度。

Result: 在两张物体的干净照片上，十个编码器的横向极性可从翻转奇分量线性解码，准确率0.77-0.99，但在百万条caption搜索下，十个中有九个在原生打分下仍为随机；在C4轨道上，池化保留排列轴0.998-1.000却丢失极性，而patch网格保留了极性。方法在118,287张图像索引上把左右成对准确率从0.51-0.53提升到0.81-0.96，21个类别匹配候选中的条件R@1从0.05提升到0.37-0.75，每批查询耗时不足1毫秒，且对不含关系的查询完全复现原生排序。

Insight: 创新点在于把空间关系失败细分为“access failure”与“pooling failure”，并证明二者都位于稠密特征下游，因此可用同一个读出器统一覆盖；提出的零参数、查询条件化空间矩读出直接从patch网格重算几何并旁路存储，在不破坏原生排序的前提下修复空间关系，兼具诊断价值和轻量可部署性。

Abstract: Contrastive vision-language models score left/right relations near chance, but the symptom has two causes that call for opposite repairs: the relation may survive in the pooled embedding yet be unreachable by native image-text similarity, or pooling may discard it while the dense features retain it. On clean two object photographs, across ten encoders, the lateral polarity is linearly decodable from the flip odd component of the pooled vector at 0.77-0.99, yet a search over a million captions leaves nine of the ten at chance under native scoring: an access failure. On controlled C4 orbits, where the relation is the only thing that varies, pooling keeps the arrangement axis at 0.998-1.000 and loses its polarity, which the patch grid retains: a pooling failure. Both sit downstream of the dense features, so one readout covers both. Our zero parameter, query conditioned spatial moment recomputes the geometry from the patch grid and stores it as a vector beside the native embedding. On a 118,287 image index it raises left/right pairwise accuracy against reversed geometry negatives from 0.51-0.53 to 0.81-0.96 and conditional R@1 among 21 category matched candidates from 0.05 to 0.37-0.75, at under a millisecond per batched query, while reproducing the native ranking exactly for queries that name no relation.

[28] Whose Values Are Generated? Measuring Visual Value Leakage in Multilingual Text-to-Image Generation cs.CVPDF

Zucan Lyu, Binwei Yao, Rheeya Uppaal, Jing Yao, Xiaoyuan Yi

TL;DR: 论文提出多语言基准 VisionValueBench，将 Hofstede 的六个文化价值维度操作化，用于测量文本到图像（T2I）系统在生成中性社会场景时隐含的文化价值取向。作者发现，即使提示在价值上中立，生成图像仍会系统性偏向特定价值取向，并将此现象称为“视觉价值泄露”。评估覆盖 960 个社会场景和六种 T2I 配置，显示全球 T2I 系统传达的文化价值具有明显同质性。

Details

Motivation: 现有 T2I 文化表征评估主要关注地标、服饰、活动等可识别文化元素，忽视了生成图像所传达的隐性文化价值。论文旨在填补这一空白，衡量多语言 T2I 系统是否以及如何在不同语言和地理线索下产生文化价值偏向。

Result: 论文在 960 个社会场景和六种 T2I 配置上评估发现，生成图像在六个价值维度上一致偏向相同取向，例如平等高于等级、个人自主高于群体义务、享乐高于克制。提示语言对价值取向影响有限，地理线索可使部分维度向相关文化价值偏移，但整体主流取向基本保留，表明 T2I 系统的文化价值输出高度同质。

Insight: 论文的创新点在于将 Hofstede 文化价值维度引入多语言 T2I 评测，提出“视觉价值泄露”概念，并从显性文化元素转向隐性价值取向进行系统测量。客观来看，其可借鉴之处是构建了可操作的价值维度基准，并揭示语言与地理线索对生成文化价值的影响有限，为后续文化对齐和多样性评估提供了新视角。

Abstract: Text-to-image (T2I) systems serve users across languages and cultural contexts. However, evaluations of cultural representation focus mainly on recognizable elements such as landmarks, clothing, and activities, overlooking the cultural values that generated images convey. To bridge this gap, we introduce VisionValueBench, a multilingual benchmark that operationalizes six of Hofstede’s cultural value dimensions (e.g., individualism versus collectivism) to investigate how T2I systems frame neutral social situations through a particular cultural orientation. We find that value-neutral prompts, which admit diverse cultural interpretations, nonetheless yield images that favor particular value orientations, a phenomenon we call visual value leakage. We evaluate prompts varying in language and geographic cues across 960 social scenarios and six T2I configurations. Generated images consistently favor the same value orientations across all six dimensions, for example, equality over hierarchy (low power distance), individual autonomy over group obligation (individualism), and enjoyment over restraint (indulgence). Prompt language has limited influence on these orientations, whereas geographic cues shift some dimensions toward the associated cultural values while largely preserving the prevailing orientations. These findings reveal substantial homogeneity in the cultural values conveyed by T2I systems worldwide.

[29] Multimodal Fusion Methods for Whole-Slide Images and Clinical Data in Inflammatory Bowel Disease cs.CV | cs.AI | cs.LGPDF

Ankana Banerjee, Mark van der Giezen, Ole Gunnar Aasprong, Emiel A. M. Janssen, Øyvind Steinsbø

TL;DR: 论文提出一个模块化多模态融合框架，将全切片图像（WSI）的切片级嵌入与每次就诊的临床数据嵌入输入15种可替换融合方法，在炎症性肠病（IBD）前瞻性队列中预测与STRIDE-II对齐的缓解结局，并系统比较不同融合策略在多个临床终点和基线/随访输入下的表现。

Details

Motivation: 多模态AI融合组织学与临床数据在计算病理中很活跃，但不同融合方法很少在相同条件下、跨不同临床终点进行公平比较，论文旨在填补这一空白。

Result: 在IBD前瞻队列的6个预测场景中，最佳融合方法达到macro-F1 0.57–0.75，并在4个场景中优于单模态模型；结果随结局和输入而异，且图像依赖程度与缓解类型相关，组织学缓解最高、内镜缓解最低。

Insight: 创新点在于构建可互换融合方法的统一评估框架，并在相同输出维度和预测头下比较15种融合策略；同时提出“图像依赖”分析，揭示准确率相近的方法对WSI模态的依赖程度可能显著不同，为多模态融合的可解释评估提供新视角。

Abstract: Multimodal AI fusion of histology and clinical data is an active research area in computational pathology, but fusion methods are rarely compared under identical conditions and across different clinical endpoints. We present a modular framework in which slide-level embeddings, obtained by training an attention-based aggregator over patch embeddings from a frozen foundation model, and per-visit clinical embeddings are fed into 15 interchangeable fusion methods with a shared output dimension and prediction head (except decision-level methods). Using a prospective cohort of newly diagnosed inflammatory bowel disease patients, we predicted STRIDE-II-aligned remission outcomes from baseline or follow-up visits. The best fusion method reached macro-F1 of 0.57-0.75, outperforming both single-modality models in four of six scenarios, and differed by outcome and input. Image reliance, the drop in balanced accuracy when the slide embedding is removed, tended to follow the remission type - highest for histological and lowest for endoscopic remission; methods with similar accuracy relied on the slide to very different degrees.

[30] GaussianRenderFormer: Transformer-based Rendering of 3D Gaussian Splats cs.CV | cs.GR | cs.LGPDF

Shahaf Valencia Levy, Sagie Benaim

TL;DR: 论文提出 GaussianRenderFormer，用 Transformer 直接从一组 3D Gaussian splats 和相机参数生成图像，而不依赖光栅化器。方法将每个 Gaussian 编码为 view-independent token，并用 view-dependent decoder 将相机光线小 patch 转为像素。作者在 Objaverse 数据上训练并评估，结果显示该方法能泛化到未见物体，但相比传统光栅化仍有明显 PSNR 差距。

Details

Motivation: 论文关注能否让神经网络自主学习渲染 3D Gaussian splats，从而移除渲染器中的显式 rasterizer。其出发点是探索纯 Transformer 直接渲染 3DGS 的可行性，并分析其性能瓶颈。

Result: 在 26,820 个 Objaverse 物体上训练（每个裁剪到 20,000 个 Gaussian），在 300 个未见物体上达到 38.08 dB PSNR（以完整未裁剪 splats 为参考）；对同样裁剪输入进行光栅化可达 44.02 dB。诊断显示差距并非泛化问题，而更可能来自 view decoder 瓶颈：显式提供每个 Gaussian 的投影 footprint 可提升 1.4 dB，且不能排除欠训练因素。

Insight: 创新点在于用 Transformer 替代传统光栅化流程，将每个 Gaussian 作为 view-independent token，并通过 view-dependent decoder 让光线 patch attend 到这些 token 来直接输出像素。客观来看，其价值在于提出纯神经渲染 3DGS 的清晰架构，并通过诊断实验定位 view decoder 瓶颈，为后续改进提供方向。

Abstract: Can a neural network learn to render 3D Gaussian splats on its own, with no rasterizer in the renderer? We present GaussianRenderFormer, a transformer that takes a set of Gaussians and a camera and outputs the image directly. Each Gaussian becomes one token in a view-independent encoder, and a view-dependent decoder turns small patches of camera rays into pixels by attending to those tokens. Trained on 26,820 Objaverse objects pruned to 20,000 Gaussians each, GaussianRenderFormer renders 300 unseen objects at 38.08 dB PSNR against the full, unpruned splats, while rasterizing the same pruned input reaches 44.02 dB. The gap is not one of generalization, and our diagnostics are consistent with a bottleneck in the view decoder: telling it the projected footprint of each Gaussian, which it could in principle infer, adds 1.4 dB. Under-training is not ruled out. Code, trained models and videos: https://svlwoof.github.io/gaussianrenderformer

[31] CAPTOR: text-encoder-free grounding by counting caption co-occurrence over the visual words of a frozen encoder cs.CVPDF

Elad Yoshai, Natan T. Shaked

TL;DR: CAPTOR 提出一种无需文本编码器的文本查询视觉定位方法：将冻结图像编码器的 patch token 聚类为视觉词，利用有 caption 图像中各视觉词的 patch 占比与词项共现统计，估计 term-by-patch 打分矩阵。该矩阵分解为 term-by-word 表与与查询无关的图像因子，使图像只需编码一次，新词只需增加一行表项。方法无需训练，在 COCO、Pascal VOC、Cityscapes 上优于无训练基线，并在病理切片上超过使用病理文本编码器的最强基线。

Details

Motivation: 许多图像编码器（如仅用病理切片训练的模型）缺乏与语言对齐的文本编码器，难以计算文本词项与图像 patch 的匹配矩阵。论文旨在不训练文本编码器的情况下，为任意冻结图像编码器提供文本查询定位接口。

Result: 在 COCO、Pascal VOC 和 Cityscapes 上均领先所有无训练基线；COCO 上按单物体所属单元格计，mIoU 达 0.565，对比 0.502。其表现与同 backbone 训练文本对齐的方法相当，但无需梯度、文本编码器和数倍 caption。在留出乳腺癌切片上，比使用病理文本编码器的最强基线高 0.160 mIoU；经 CAPTOR 读取，仅切片训练的 UNI 在公共网格上超该类基线 0.132。

Insight: 创新点在于把冻结图像编码器的 patch token 聚类成视觉词，并通过 caption 中共现统计构造词项签名和 term-by-word 表，再通过矩阵分解实现查询无关的图像编码。其客观可取之处是无需文本编码器和训练，即可为未接触语言的图像编码器提供文本接口，且新词扩展成本低。

Abstract: Grounding a text query in an image needs a matrix scoring every term against every patch. Vision-language models compute it with an aligned text encoder, which many image encoders, such as histopathology-related models trained on slides alone, lack. The Caption-Anchored Patch-Term Occurrence Representation (CAPTOR) estimates the matrix without a text encoder. It clusters a frozen image encoder’s patch tokens into visual words and describes each captioned image by its share of patches in each word. A term signature is the standardized difference of those shares between images whose captions mention the term and the rest. The matrix factorizes into a term-by-word table and a query-independent image factor, so an image is encoded once and a new term adds one table row. Against every training-free baseline, CAPTOR leads on Common Objects in Context (COCO), Pascal Visual Object Classes (VOC) and Cityscapes. On COCO, scored on the cells one object owns, it reaches 0.565 mean intersection over union (mIoU) against 0.502. On all three, it is also level with a text alignment trained on the same backbone, which needs gradients, a text encoder and several times the captions. On held-out breast-cancer slides, it exceeds the strongest baseline scoring patches with a pathology text encoder by 0.160 mIoU. Read through CAPTOR, UNI, an encoder trained on slides alone, exceeds such baselines by 0.132 on a common grid. Counting can therefore give a text interface to any image encoder trained without language, wherever captioned images exist.

[32] Spatio-Temporal Multi-Order Interaction for Video Recognition cs.CV | cs.AI | cs.LGPDF

Kevin D. Nguetche, Lilian Hollard, Eric Granger, Marco Pedersoli

TL;DR: 论文提出 VideoMogaNet，用于视频动作识别。方法基于 MogaNet 的多阶空间表示，引入仅深度卷积与隐式门控聚合的并行多阶时序建模，并设计空间-时间双交互模块，在降低时序分支计算开销的同时提升时空表示能力。

Details

Motivation: 现有视频时序方法常依赖显式跨通道聚合，通道数上呈二次复杂度，计算开销大；同时 CNN 偏局部阶交互、Transformer 偏全局阶交互，时空表示存在偏差。

Result: 在 UCF101、HMDB51、Kinetics-400、Diving48 多个视频基准上，VideoMogaNet 达到与 SOTA 动作识别方法相当或更好的性能，同时显著降低时序分支计算成本。

Insight: 创新点在于用仅深度卷积和隐式门控实现通道数线性的并行多阶时序处理，并提出不增加计算成本的空间-时间双交互，从而兼顾多阶时空表示与效率。

Abstract: Many temporal approaches for video recognition rely on explicit cross-channel aggregation projections, which have quadratic complexity in the number of channels, introducing significant computational overhead. In addition, the spatio-temporal representation used for video recognition is biased either toward local order interactions (for CNNs) or toward global order interactions (for transformers). Starting from the multi-order spatial representation of MogaNet, our method introduces two complementary contributions to tackle these problems. First, for the temporal modelling, we introduce parallel multi-order processing with depthwise-only convolutions and implicit gating-based aggregation, which is linear in the number of channels, inducing a considerable computational reduction. Second, we propose a dual interaction between spatial and temporal dimensions, which allows a better spatio-temporal representation without increasing the computational cost. By combining these two contributions, our proposed VideoMogaNet achieves performance on par with or better than state-of-the-art action recognition approaches across multiple video benchmarks (UCF101, HMDB51, Kinetics-400, Diving48), while considerably reducing the cost of the temporal branch.

[33] NitroFusion2: Bringing Adversarial Loss to the Text Stream of Multimodal Diffusion Transformers cs.CVPDF

Dar-Yen Chen, Yi-Zhe Song

TL;DR: 论文提出NitroFusion2，将对抗扩散蒸馏的判别表示从图像特征扩展到多模态扩散Transformer（MMDiT）的文本流。作者引入文本流对抗蒸馏（TSAD），利用中间文本token状态作为新的对抗表示，并提出方面线索判别（ACD），通过让判别器关注排版、物体或风格等不同方面来引导对抗训练强调的属性。该方法将少步蒸馏从4-8步推进到1-2步。

Details

Motivation: 传统对抗扩散蒸馏主要依赖图像特征进行判别，但MMDiT的文本流通过联合注意力与图像产生依赖关系，因此需要重新审视判别表示的选择，以提升少步扩散生成的质量和可控性。

Result: NitroFusion2在可比质量下将少步蒸馏从常规4-8步降至1-2步；单步生成可产生清晰的渲染文本和丰富的细粒度细节，使一步生成更具实用性；两步生成在评估基准上保持竞争力。

Insight: 摘要宣称的创新点是将MMDiT文本流中间状态作为对抗蒸馏表示，并用ACD按样本的不同方面条件化判别器以控制训练强调的属性。客观来看，其创新在于利用多模态Transformer中文本流与图像条件的耦合关系，把文本流变成可引导的对抗监督信号，而不仅依赖图像特征。

Abstract: Adversarial diffusion distillation has conventionally relied on image features for discrimination. We revisit this choice for multimodal diffusion transformers (MMDiTs), whose text stream becomes image-dependent through joint attention. We introduce Text-Stream Adversarial Distillation (TSAD), which exploits intermediate text-token states as a new adversarial representation. Because the text stream has already read the image while remaining tied to the text condition, changing that condition changes what information it extracts from the image. Building on this property, we introduce Aspect-Cued Discrimination (ACD), which conditions the discriminator on different aspects of the sample, such as typography, objects, or style, to steer which properties are emphasized during adversarial training. Building on the expert-discriminator design of NitroFusion, we integrate TSAD and ACD in NitroFusion2. NitroFusion2 brings few-step distillation from the conventional 4-8-step regime down to just 1-2 steps at comparable quality. At one step, it produces sharp rendered text and rich fine-grained details, making one-step generation practical rather than merely a stress test for distillation. At two steps, it remains competitive across the evaluated benchmarks. These results establish the MMDiT text stream as a powerful and steerable representation for adversarial diffusion distillation.

[34] GeoDIVE: Geospatial Descent Informed by Value Estimation cs.CVPDF

Hamza Iqbal, Dan Cher, Eric Xing, Pranavi Kolouju, Abby Stylianou

TL;DR: 论文提出 GeoDIVE，将多分辨率卫星图像库建模为森林，并用最佳优先遍历决定优先编码哪些瓦片；在每个访问节点上，轻量模型基于查询和子树表示预测后代中最优查询相似度的乐观估计，从而避免对全区域进行穷举编码与搜索。该方法面向图像查询和开放词汇文本查询的大规模多尺度地理空间目标搜索，并发布了 Git-10M 的对象级图文查询基准。

Details

Motivation: 农业、基础设施规划、环境监测和灾害响应等任务需要在大范围区域内寻找特定对象、结构和空间模式，但这些目标分散且尺度差异巨大，朴素嵌入、维护和搜索整个空间代价很高。论文旨在降低多尺度卫星图像搜索的编码与搜索成本。

Result: 在图像查询和开放词汇文本查询评估中，GeoDIVE 在图像查询下以约 10% 的编码成本达到与穷举搜索相当的效果，并在两种模态的各个预算设置下均领先所有测试基线。作者还发布了基于 Microsoft 建筑轮廓、OpenStreetMap 多边形和 VLM 生成描述构建的 Git-10M 对象级图像与文本查询，用于后续多尺度地理空间搜索评测。

Insight: 创新点在于将多分辨率卫星图像画廊视为森林，用最佳优先遍历和价值估计模型预测子树中可能隐藏的最佳查询相似度，以乐观估计指导按需编码与搜索。客观来看，其可借鉴之处是将层次化空间索引与学习式价值估计结合，把昂贵编码预算集中到更有希望的区域，并配套提出对象级多尺度地理空间检索基准。

Abstract: Finding specific objects, structures, and spatial patterns supports critical work in agriculture, infrastructure planning, environmental monitoring, and disaster response. These features, however, are often dispersed across very large regions and span dramatically different spatial scales, which defines a search space that is costly to embed, maintain, and search naively. We introduce GeoDIVE, a model that treats a multiresolution satellite image gallery as a forest and uses best-first traversal to determine which tiles to encode. At each visited node, a lightweight model then predicts an optimistic estimate of the best query similarity hidden among its descendants using the query and subtree representations. In evaluations with image and open-vocabulary text queries, GeoDIVE matches exhaustive search at 10% of the encoding cost for image queries and leads all tested baselines at every budget in both modalities. To support future evaluation of multiscale geospatial search, we also release object-level image and text queries for Git-10M, constructed from Microsoft building footprints and OpenStreetMap polygons with VLM-generated captions. Source code and models will be made publicly available.

[35] C-WFDM: A ControlNet-Guided Wavelet-Fusion Diffusion Model for Multimodal Brain MRI Completion cs.CVPDF

Muhammad Nabi Yasinzai, Remika Mito, Mangor Pedersen

TL;DR: 论文提出 C-WFDM，一种基于 ControlNet 扩展的小波融合扩散模型，用于多模态脑 MRI 的受试者级缺失模态补全。模型在预训练 WFDM 主干上保留目标模态和元数据条件，并用同一受试者的已有 MRI 对比度通过目标特定 ControlNet 适配器引导 T2w、FLAIR 和 T1Gd 的合成。方法面向真实汇总数据集中模态可获得性不均的问题，并在 BrainScape 补全实验中与 ALDM、cWDM、Pix2Pix 比较。

Details

Motivation: 多模态脑 MRI 数据集中不同模态的可用性不均，T1w 较常见而 T2w、FLAIR、T1Gd 相对缺失；现有缺失模态基准多从完整四模态病例中人为屏蔽对比度，可能不能反映汇总数据集中的真实缺失模式，因此需要更贴近实际的受试者特定缺失模态补全方法。

Result: 在 BrainScape 补全实验中，C-WFDM 与 ALDM、cWDM、Pix2Pix 使用 SSIM、PSNR、MAE 比较，在 T2w 和 FLAIR 补全上取得所评估模型中的最佳结果，SSIM 分别为 0.9232 和 0.8904。

Insight: 创新点在于将 WFDM 扩展为 ControlNet 引导的小波融合扩散模型，使用同一受试者的 MRI 对比度作为目标特定条件，并仅在需要时使用生成的中间对比度；客观上看，其价值在于结合真实模态可用性模式、受试者内多模态条件和小波融合扩散生成能力来提升缺失模态补全。

Abstract: Multimodal brain MRI provides complementary anatomical, tissue, and pathological information to support downstream artificial intelligence (AI) applications. However, modality availability is uneven across public and pooled neuroimaging datasets: T1-weighted (T1w) MRI is widely represented, T2-weighted (T2w) and fluid-attenuated inversion recovery (FLAIR) MRI are less consistently available, and gadolinium-enhanced T1-weighted (T1Gd) MRI is relatively scarce. Pooled datasets therefore contain participants with different combinations of acquired MRI contrasts, reflecting variation in the study designs, acquisition protocols, and clinical or research objectives of the contributing studies. Many missing-modality benchmarks simulate incomplete MRI by withholding contrasts from complete four-modality cases, which may not represent the observed availability patterns in pooled datasets. To address this gap, we introduce C-WFDM, a ControlNet extension of our Wavelet-Fusion Diffusion Model (WFDM), for subject-specific missing-modality completion. C-WFDM is built on an earlier BrainScape snapshot comprising 157 public MRI datasets and 45,880 preprocessed scans. The pretrained WFDM backbone retains target-modality and metadata conditioning, while target-specific ControlNet adapters use MRI contrasts from the same participant to guide synthesis. C-WFDM synthesizes T2w from T1w, FLAIR from T1w and T2w, and T1Gd from T1w, T2w, and FLAIR. Acquired conditioning contrasts are used when available, and generated intermediate contrasts are used only when required. We compare C-WFDM with ALDM, cWDM, and Pix2Pix using paired structural similarity index (SSIM), peak signal-to-noise ratio (PSNR), and mean absolute error (MAE). In the BrainScape completion experiment, C-WFDM achieves the best T2w and FLAIR results among the evaluated models, with SSIM values of 0.9232 and 0.8904, respectively.

[36] Learning Hierarchical Patch Splitting Policies for Faster Vision Transformers cs.CV | cs.AI | cs.LGPDF

Aditya Gupta, Jean Sebastien Dandurand, Kai Qiu, Rohan Choudhury, Laszlo A. Jeni

TL;DR: 论文提出SPLIT，一种学习式分层patch拆分与图像token化策略，根据模型认为对任务有意义的信息动态分配token分辨率，从而缩短ViT输入token序列。方法在分类任务上以更少token匹配全分辨率ViT基线，并在COCO、ADE20K等密集预测任务上保持竞争力。

Details

Motivation: ViT受输入token序列长度瓶颈限制；标准patchification使用固定patch大小并忽略区域内容，而现有自适应patch方法依赖局部熵等噪声像素启发式，未基于模型对任务相关信息的理解来分配计算。

Result: SPLIT在分类任务上匹配全分辨率ViT基线精度且使用少30%的token；在COCO和ADE20K密集预测上取得有竞争力的精度，表明语义对齐的patchification可泛化到分类之外。

Insight: 创新点在于用可学习的split policy替代像素级启发式，依据模型语义和任务相关性分配token分辨率。客观上，将自适应token化与模型内部表征对齐，兼顾效率与密集预测泛化，是可借鉴的方向。

Abstract: Vision Transformers (ViTs) are bottlenecked by the length of their input token sequence. Standard patchification fixes this length with a single patch size applied across the image, regardless of regional content. Prior work on adaptive patch sizing reduces token count by assigning coarser patches to visually simple regions, but relies on noisy pixel-level heuristics such as local entropy to make patchification decisions. We argue that this compute allocation should be grounded in the model’s interpretation of task relevant information, not in raw pixel statistics. We introduce Split Policy Learned Image Tokenization which learns to allocate token resolution based on what the model finds meaningful, producing shorter token sequences that are better matched to task-relevant content. SPLIT matches the accuracy of full-resolution ViT classification baselines while using 30% fewer tokens. The same gains hold on dense prediction: SPLIT achieves competitive accuracy on COCO and ADE20K, demonstrating that semantically aligned patchification generalises well beyond classification.

[37] Navigational Safety of Marine Surface Vehicles in Restricted Visibility: From Intelligent Vision to Vision-Centered Multi-Sensor Fusion cs.CVPDF

Ryan Wen Liu, Fengchuan Song, Tianxiang Wang, Guobei Peng, Youngrong Kim

TL;DR: 该论文是一篇综述，面向受限能见度下海上水面载具的航行安全，梳理从智能视觉到以视觉为中心的多传感器融合的研究进展。文章介绍退化图像成因，回顾船载视频稳像、视觉感知增强、目标检测与跟踪、可航行区域分割等视觉方法，并总结红外-可见光、雷达-视觉、LiDAR-视觉、AIS-视觉等融合方案、开源数据集与安全关键应用，最后讨论开放挑战和未来方向。

Details

Motivation: 船载相机性价比高但视觉感知易受平台不规则运动、恶劣天气和远距离成像等受限条件影响，单一视觉难以保障受限能见度下MSV的态势感知与航行安全，因此需要系统梳理智能视觉及以视觉为中心的多传感器融合方法。

Result: 论文为综述性文章，未提出新方法或给出定量实验结果；主要贡献是归纳相关方法、开源数据集和安全关键应用，并指出开放挑战与未来研究方向，同时提供可定期更新的海事数据集链接。

Insight: 创新点在于以“受限能见度下航行安全”为主线，建立从智能视觉关键任务到以视觉为中心的多传感器融合的系统性分类与综述框架，并整合开源数据集、应用场景和挑战，为智能船舶导航与智能水上交通系统研究提供参考。

Abstract: Marine surface vehicles (MSVs) have attracted considerable attention for their commercial, scientific, and military applications. Owing to their favorable performance-to-cost ratio, shipboard cameras are widely used to detect and recognize potentially hazardous situations across diverse operational scenarios. However, visual perception is inevitably degraded by restricted conditions, including irregular motion of the imaging platform, adverse weather, and long imaging distances. To improve situational awareness and navigation capabilities, complementary sensors, such as the automatic identification system (AIS), radar, and light detection and ranging (LiDAR), etc., have become essential shipboard equipment. Multi-sensor fusion can provide richer perceptual information and help ensure navigational safety of MSVs in restricted visibility. This article reviews recent progress from intelligent vision to vision-centered multi-sensor fusion for newcomers and researchers in intelligent ship navigation and intelligent waterborne transportation system (IWTS). After introducing the generation of quality-degraded images in restricted visibility, we will review intelligent vision methods for four key issues, i.e., shipboard video stabilization, visual perception enhancement, object detection and tracking, and navigable area segmentation. We then examine vision-centered multi-sensor fusion methods, including infrared-visible image fusion, radar-vision fusion, LiDAR-vision fusion, AIS-vision fusion, and other multi-sensor fusion scenarios. Open-source datasets and safety-critical applications are also summarized and discussed. Finally, we highlight open challenges and future research directions. The collected maritime datasets are available at https://github.com/MAPGroup/Datasets and will be regularly updated.

[38] DualCast: Dual-Image Conditioned Diffusion Transformers for Digital Human E-Commerce Video Generation cs.CVPDF

Lifan Jiang, Yibo Zhao, Senlin Cheng, Bolei Xu, Xiaobo Li

TL;DR: 论文提出DualCast，一种面向数字人电商视频生成的双图条件扩散Transformer框架。该方法显式区分产品图像与人物图像的不对称角色，通过双时空槽注入、掩码引导区域加权损失和图像条件dropout，缓解产品失真、身份漂移及单参考坍缩问题，并构建了1万条电商视频数据集用于训练。

Details

Motivation: 现有支持多参考图的视频生成模型未明确区分“展示什么产品”和“由谁展示”的角色差异，导致生成视频偏向单一参考，出现产品变形、人物身份漂移或单参考坍缩。论文旨在通过角色解耦的双图条件建模提升数字人电商视频中产品外观、人物身份和整体风格的一致性。

Result: 摘要称实验结果表明DualCast性能较强，并构建了10K电商视频数据集用于训练；但未给出具体定量指标、对比benchmark或SOTA结论。

Insight: 创新点在于显式建模产品与人物条件的非对称角色，将双图潜在特征注入DiT序列前后边界以实现序列级条件分离，并结合产品/人脸/重叠区域加权监督和条件dropout统一支持双图、仅产品、仅人物生成。客观看，其可借鉴之处是角色解耦的条件注入机制与关键区域监督设计。

Abstract: Digital human e-commerce video generation requires preserving product appearance, human identity, and overall video style during dynamic demonstrations. Although recent models support multiple reference images, their general conditioning schemes do not explicitly distinguish the asymmetric roles of “what is presented” and “who presents it.” Consequently, generated videos may favor a single reference, resulting in product distortion, identity drift, or single-reference collapse. To address this issue, we propose DualCast, a role-addressed dual-image conditioning framework for video generation. By explicitly modeling the distinct roles of product and human conditions, DualCast reduces role ambiguity and feature competition between references. First, a dual-spatiotemporal slot injection mechanism VAE-encodes the product and human images separately and places their latent features at the front and rear boundaries of the DiT sequence, enabling sequence-level condition separation and directed injection. Second, a mask-guided region-weighted loss strengthens supervision over product, face, and product-face overlap regions, improving generation fidelity in critical areas. Third, image condition dropout enables a single model to support dual-image, product-only, and human-only generation. In addition, we construct a curated dataset of 10K e-commerce videos for model training. Experimental results demonstrate the strong performance of DualCast.

[39] Generative AI for urban planning: synthesizing land-use maps with satellite imagery via DualControlNet cs.CV | cs.AIPDF

Qingqi Song, Yuzhou Chen, Lingqian Hu, Shenhao Wang

TL;DR: 论文提出 DualControlNet，一种面向城市规划的生成式土地利用模型，可联合生成高分辨率土地利用图与卫星影像，并支持文本提示和多维条件控制。模型在 Orlando 和 Chicago 的 4 万余个配对图块上训练，从土地利用、空间一致性和图像质量三方面评估，展示了可控城市场景生成能力。

Details

Motivation: 现有土地利用模型多为基于数值数据的预测式转变模拟，难以从简单文本提示合成新的土地利用图；城市生成式AI又存在单模态、可控性弱、仅用通用图像质量评估等问题。论文旨在解决多模态联合生成、可控生成和领域化验证的挑战。

Result: 相比单模态基线，DualControlNet 将土地利用多样性一致性提升 62.8%、建筑覆盖率一致性提升 16.3%，跨模态空间一致性达到基线两倍以上，视觉质量达到三倍；场景实验表明其可通过多维控制提高密度与多样性，并响应“高密度开发”等文本提示。

Insight: 创新点在于将土地利用图与卫星影像联合生成，融合多模态、可控性和可验证性，并用土地利用与空间一致性等领域指标评估。可借鉴之处是面向专业领域的生成模型不应只追求通用图像质量，而应结合领域约束、跨模态一致性和文本/多维控制来提升实用性。

Abstract: Various land-use models characterize land-use patterns and facilitate scenario planning. However, existing models are largely predictive, forecasting transitions based on numeric data without synthesizing new land-use maps from simple text prompts. Meanwhile, generative AI in urban systems remains unimodal, weakly controllable, and evaluated mostly by generic image quality rather than domain-specific metrics. To address these gaps, we introduce DualControlNet, a generative land-use model that jointly synthesizes high-resolution land-use maps and satellite imagery, addressing multimodality, controllability, and verification challenges. Trained on over 40,000 paired tiles from Orlando and Chicago, DualControlNet is evaluated using land-use, spatial consistency, and image quality metrics. Relative to unimodal baselines, DualControlNet improves land-use diversity consistency by 62.8% and building coverage ratio consistency by 16.3%, more than doubles cross-modal spatial consistency, and triples visual quality. Scenario-based experiments demonstrate that DualControlNet supports sustainable development through multidimensional controls, increasing density and diversity, and responding to text prompts like “high-density development.” Overall, this work advances land-use modeling via a multimodal, controllable, and verifiable generative framework, lowering technical barriers and enabling intuitive exploration of urban futures.

[40] Feature Puzzles: Single-Image Supervision for Multi-View 3D-Aware Segmentation cs.CVPDF

Jiahao Ma, Qiang Zhang, Miaomiao Liu, David Ahmedt-Aristizabal, Chuong Nguyen

TL;DR: 论文提出Feature Puzzles框架，仅使用单图像语义分割监督学习多视角一致分割。通过图像裁剪和几何变换构造多视角一致伪标签，并利用预训练3D基础模型的多视角一致特征和轻量即插即用网络进行分割。该方法支持多种提示和开放词汇3D分割，在NVOS上取得最佳mIoU，在SPIn-NeRF和ScanNet++上具有竞争力，并展示零样本开放词汇迁移。

Details

Motivation: 多视角一致感知通常需要昂贵的视频或标定多视角数据，而大规模单目数据更廉价且多样。论文旨在仅用单图像语义分割监督来学习多视角一致分割，降低数据成本。

Result: 在NVOS上取得最佳mIoU，在SPIn-NeRF和ScanNet++上与使用逐场景优化或域内多视角训练的基线相比保持竞争力，并展示零样本迁移到开放词汇3D分割。

Insight: 创新点在于通过图像裁剪和几何变换从单图像构造多视角一致伪标签，并利用预训练3D基础模型的特征和轻量网络实现多视角一致分割，支持多种提示和开放词汇扩展。客观分析：将单图像监督与3D基础模型特征结合，方法简单有效，具有良好扩展性。

Abstract: Humans can easily recognize the same objects across different viewpoints of a 3D scene, whereas teaching models such multi-view-consistent perception typically requires videos or calibrated multi-view captures, which are considerably more expensive and less diverse than large-scale monocular data.~In this paper, we introduce Feature Puzzles, a novel framework for learning multi-view-consistent segmentation using only single-image semantic segmentation supervision. Specifically, we propose a simple yet effective pipeline that constructs multi-view-consistent segmentation pseudo-labels through image cropping and geometric transformations. We further introduce a lightweight, plug-and-play network that leverages the multi-view-consistent image features learned by existing pretrained 3D foundation models and the generated pseudo-labels to perform multi-view-consistent segmentation. Importantly, our flexible framework supports various types of prompts provided from one or multiple views and can be readily extended to open-vocabulary 3D segmentation. Under this single-image supervision, our feed-forward model achieves the best mIoU on NVOS, remains competitive on SPIn-NeRF and ScanNet++ against baselines using per-scene optimization or in-domain multi-view training, and demonstrates zero-shot transfer to open-vocabulary 3D segmentation. Visual comparisons are available on our project page: https://featurepuzzles.github.io/

[41] Chronos: Sequential Identity-Preserving Video Generation by Temporal Anchoring cs.CV | cs.MMPDF

Mengqi Huang, Wenchuan Wang, Zhendong Mao, Yijing Tu, Zhike Liu

TL;DR: 论文提出Chronos框架，用于身份保持视频生成，将单段人脸身份保持与多段连续动作生成统一到同一时序建模范式中。方法包含语义提示锚定、视觉帧锚定和基于度量引导的候选路由，以在文本与视觉条件下兼顾身份一致性和时序可控性。

Details

Motivation: 现有方法主要依赖参考图像引导和身份感知条件来提升静态人脸对齐，但过度关注静态身份对齐会造成身份保真度与时序可控性之间的冲突，在大姿态变化和多段连续动作下容易出现身份漂移或动作抑制。

Result: 论文在ACM MM 2026身份保持视频生成挑战赛的Track 1和Track 2中均获得第一名，表明方法在人脸身份保持和连续动作生成场景下均有效。

Insight: 创新点在于用统一的时序锚定思路连接单段与多段视频生成，并通过语义提示细化、视觉关键帧锚定和多生成器候选多维评估路由来提升身份一致性与生成质量。

Abstract: Identity-preserving video generation aims to synthesize high-quality and visually realistic videos from textual and visual conditions while faithfully maintaining the subject’s identity across diverse motions. Prior methods have extensively introduced reference-guided control signals and identity-aware conditioning to improve facial alignment from reference images. However, improving static identity alignment alone creates a fundamental tension between identity fidelity and temporal controllability, which frequently causes identity drift or motion suppression under large pose variations and multi-segment sequential actions. To address this, we propose Chronos, a framework that unifies single-segment facial identity preservation and multi-segment sequential action generation under a common temporal formulation. Specifically, Chronos integrates Semantic Prompt Anchoring to adaptively refine the user prompt, Visual Frame Anchoring to transform conditions into visually grounded keyframe anchors for visual guidance, and Metric-Guided Candidate Routing to select the highest-quality videos from heterogeneous generators via multi-dimensional evaluation. Extensive evaluations demonstrate the superiority of our approach, which achieved first place in both Track 1 and Track 2 of the ACM MM 2026 Identity-Preserving Video Generation Challenge, demonstrating its effectiveness across both facial identity preservation and sequential action generation settings.

[42] DecoupAttn: Decoupled Sparse Attention for Training-Free Acceleration of Video Diffusion cs.CVPDF

Yanjin Li, Bingbing Qiu, Qiang Huang, Rui Qian, Yanfeng Li

TL;DR: 论文提出DecoupAttn，一种无需训练的解耦稀疏注意力方法，用于加速少步视频扩散变换器。它通过语义聚类补偿所有键，并利用固定位置的少量精确注意力块纠正误差，同时引入率失真目标分配预算和重分配缓存更新机制。

Details

Motivation: 现有稀疏注意力在加速视频扩散模型时存在识别关键块的开销大、稀疏预算分配次优以及跨步复用掩码导致误差累积等问题。论文观察到被跳过块的误差在补偿每个键后会相互抵消，从而设计解耦方案。

Result: 摘要未明确报告定量实验结果或基准对比，仅说明方法无需训练，离线编译一次计划，推理使用标准块稀疏核和融合补偿核。

Insight: 创新点包括：提出解耦稀疏注意力范式，将语义聚类补偿与固定位置精确纠正分离；引入率失真目标进行预算分配；设计重分配缓存动态更新漂移的键；整体无需训练且兼容标准稀疏核实现。

Abstract: Sparse attention accelerates few-step video diffusion transformers by computing a few key blocks exactly and compensating for the rest. However, existing sparse attention still faces key challenges, including identification overhead from finding critical blocks at every denoising step, suboptimal budget allocation that sets sparsity budgets uniformly or by local metrics, and cross-step error accumulation as reused masks or clusters go stale. Observing that the errors left on skipped blocks cancel each other once every key is compensated, we propose DecoupAttn, a new paradigm of decoupled sparse attention: semantic clusters compensate every key, and exact attention corrects their error on a few blocks fixed once offline by position. To support this paradigm, we introduce Distortion Budget, a rate-distortion objective for budget allocation that places exact computation across timesteps, layers, and heads where it removes the most distortion that compensation leaves in the final latent. Built on this budget, we further propose Reassign Cache, in which each step reassigns the drifting keys to cached centroids and refreshes compensation while the budgeted blocks stay fixed. Moreover, DecoupAttn is training-free: a few calibration clips compile one plan offline, and inference runs standard block-sparse kernels with one fused compensation kernel. Our code will be released after submission.

[43] Modality-Aware Expert Pruning for MoE-Based Multimodal Large Language Models cs.CV | cs.CLPDF

Donggeun Kim

TL;DR: 论文提出MAEP，一种面向基于MoE的多模态大语言模型的免训练专家剪枝框架。它通过单次前向传播按模态分解专家重要性，利用Delta-H输出衡量移除专家带来的MoE输出变化，并将低注意力视觉token作为稳定的视觉侧剪枝信号、文本侧重要性作为保留信号，跨层全局排序剪枝专家。

Details

Motivation: MoE-MLLM中专家参数占模型内存90%以上，但现有专家剪枝方法统一聚合所有token的重要性，忽略了同一专家在视觉与语言处理中可能承担不同角色。

Result: 在三个MoE-MLLM的多模态与文本基准上，MAEP在多数多模态基准上优于模态无关基线，文本任务保持竞争力；可减少约27GB BF16专家权重（约占总内存47%），使约58GB模型可放入单张40GB GPU。

Insight: 创新点在于模态感知的专家重要性分解、重分配感知的Delta-H指标，以及数据集稳定的低注意力视觉token剪枝信号；客观看，其免训练、单次前向和跨层全局排序也提升了实用性与部署价值。

Abstract: Mixture-of-Experts (MoE) enables multimodal large language models (MLLMs) to scale capacity efficiently, but expert parameters dominate memory even though only a few experts are activated per token: they account for over 90% of model size in modern MoE-MLLMs (e.g., 54 GB out of about 58 GB in BF16 for InternVL3.5-30B-A3B and Qwen3-VL-30B-A3B). Expert pruning has emerged as an effective compression strategy, but existing methods aggregate importance uniformly across tokens, ignoring that the same expert may serve different roles for vision versus language processing. To address this, we propose MAEP (Modality-Aware Expert Pruning), a training-free framework that decomposes expert importance by modality in a single forward pass. MAEP introduces (1) Delta-H Output ($ΔH$), a redistribution-aware metric measuring the MoE output change when an expert is removed, and (2) low-attention vision tokens as a vision-side pruning signal that is dataset-stable: its expert-importance ranking remains stable across different datasets. Text-side importance serves as a preservation signal while vision-side importance serves as a pruning signal, and experts are ranked globally across layers. Experiments on three MoE-based MLLMs across multimodal and text benchmarks show that MAEP outperforms modality-agnostic baselines on the majority of multimodal benchmarks while remaining competitive on text-only tasks, reducing the BF16 expert weight footprint by about 27 GB (about 47% of total memory) so that the full weight footprint of an approximately 58 GB model fits within a single 40 GB GPU. Code is available at https://github.com/nota-github/MAEP.

[44] CLEAR: Color-Texture Latent Alignment for Single Image Reflection Removal cs.CV | cs.GRPDF

Tu Vo, Sheir A. Zaheer, Chan Y. Park

TL;DR: 论文面向单图像反光去除（SIRR），提出CLEAR框架，利用预训练潜在扩散模型作为自然图像先验，并通过物理引导的分离器一次性输出透射层、反光层和残差层的粗潜在表示。针对输入跳连会带回反光边缘、潜在压缩导致颜色漂移和细节损失的问题，设计混合特征对齐模块（HFAM）对高频特征进行颜色与纹理对齐。方法为确定性、单时间步、非迭代的潜在框架，在六个真实基准上取得综合保真度与感知指标最佳平均表现。

Details

Motivation: 现有潜在复原方法常将退化输入的高频特征通过跳连送回解码器，这对模糊、噪声、雾霾等有效，但在SIRR中会同时增强反光层的结构化边缘。潜在压缩还会引入颜色漂移和细节损失，因此需要一种既能保留透射层纹理又能抑制反光色细节的框架。

Result: 在六个真实世界SIRR基准上，CLEAR取得保真度与感知指标的最佳平均值，并在所有数据集上取得最低LPIPS，且在最具挑战性的基准上提升最大。

Insight: 创新点在于使用确定性单步潜在扩散先验而非迭代生成，并用物理引导分离器围绕透射-反光-残差形成模型组织预测；HFAM将输入跳连视为受污染的层混合特征，借助干净透射对齐的颜色查询和真值特征监督修正高频内容，从而保留透射一致纹理并衰减反光细节。

Abstract: Single Image Reflection Removal (SIRR) seeks to recover the transmitted scene from a photograph captured through glass. Recent latent restoration methods use pretrained generative priors, but many recover detail by feeding high-frequency features from the degraded input back into the decoder. This works for degradations such as blur, noise, or haze, where input detail mostly belongs to the scene. It is less suitable for SIRR, where reflections add a second structured layer: directly reusing the input’s skip connections can sharpen the image while also bringing reflection edges back. Latent compression also introduces color drift and fine-detail loss. We propose CLEAR (Color-Texture LatEnt Alignment for Single Image Reflection Removal), a deterministic, single-timestep, non-iterative latent framework that uses a pretrained latent diffusion backbone as a natural-image prior. A physics-guided separator organizes the prediction around the transmission, reflection, and residue formation model and outputs coarse layer latents in one forward pass. The Hybrid Feature Alignment Module (HFAM) then treats input-derived skips as contaminated layer-mixture features. It rectifies their high-frequency content with a clean-transmission-aligned color query, learned from the input features themselves under ground-truth feature supervision, keeping transmission-consistent texture while attenuating reflection-colored detail. CLEAR achieves the best average across fidelity and perceptual metrics on six real-world benchmarks and the lowest LPIPS on every dataset, with the largest gains on the hardest benchmarks.

[45] PathoS2VD: From 2D Histopathology to Pseudo Z-Stack Representations cs.CVPDF

Jiamu Wang, Keunho Byeon, Ji-Uk Kim, Jin Tae Kwak, Heounjeong Go

TL;DR: 论文提出 PathoS2VD 生成框架，从有限病理 z-stack 数据中学习 z 轴形态先验，并将其迁移到常规单平面 2D 病理图像。方法基于图像到视频的潜扩散模型，从单张 2D 图像合成伪 z-stack，以捕捉相邻焦平面的合理形态变化，而非重建真实组织体积；同时引入保持先验的混合微调策略以适配新域。实验表明合成伪 z-stack 具有较好的跨平面形态连续性，并能提升下游前列腺癌分类性能。

Details

Motivation: 传统病理诊断主要依赖 2D 薄切片，但组织本身具有 3D 形态结构，单平面切片会丢失 z 轴上的重要形态线索，限制人工判读和计算分析。论文希望以低成本方式为常规 2D 病理图像补充 z 轴形态信息，避免依赖专用体积成像。

Result: 摘要称合成伪 z-stack 表现出真实的跨平面形态连续性，并在下游前列腺癌分类任务上相比传统 2D 表示取得一致提升；未给出具体数值或明确 SOTA 声明。

Insight: 创新点在于将 2D 病理到伪 z-stack 的生成建模为图像到视频潜扩散任务，学习可迁移的 z 轴形态先验，并通过保持先验的混合微调适配大规模临床数据。客观来看，其价值在于无需原生 z-stack 采集即可增强常规病理图像，但生成结果是合理的相邻焦平面形态变化，而非真实组织体积重建。

Abstract: For decades, diagnostic pathology has relied on thin two-dimensional (2D) tissue sections, despite biological tissues exhibiting inherently three-dimensional (3D) morphological organization. Because a single section captures only one focal plane, important morphological cues along the z-axis are inevitably discarded, limiting tissue representation for both human interpretation and computational analysis. In this work, we propose a generative framework that learns z-axis morphological priors from pathology z-stack data, offering a cost-effective alternative to dedicated volumetric imaging, and transfers this knowledge to conventional single-plane histopathology. Specifically, an image-to-video (I2V) latent diffusion model (LDM) is trained to synthesize a pseudo z-stack from a single 2D image, capturing plausible morphological variations across adjacent focal planes rather than reconstructing physical tissue volumes. To enable deployment on large-scale clinical datasets without native z-stack acquisition, we further introduce a prior-preserving mixture fine-tuning strategy that adapts the model to new domains while retaining the learned z-axis morphological prior. Extensive experiments demonstrate that the synthesized pseudo z-stacks exhibit realistic cross-plane morphological continuity and consistently improve downstream prostate cancer classification compared with conventional 2D representations. Our results suggest that learning transferable z-axis priors from limited z-stack data provides a practical way to enrich routine pathology images with complementary morphological information. Code: https://github.com/QuIIL/PathoS2VD.

[46] Clips2Story: Training-Free Video Storyboarding and Editing Using Multimodal Retrieval-Embedded Generation cs.CV | cs.AI | cs.LG | cs.MMPDF

Junnuo Wang, Weihan Xu, Hao-Wen Dong

TL;DR: 论文提出Clips2Story，一个无需训练的基于LLM的视频故事板生成流程。给定视频片段池和关键词，先提取多模态描述符检索相关片段，再由LLM选择排列成带叙述的可编辑故事板，最后组装片段并检索辅助视觉生成视频。在合成数据集上验证了检索和故事板生成的有效性，人类评估优于基线。

Details

Motivation: 现有视频编辑与摘要方法通常假设源视频具有时间顺序并保留原始叙事结构，但实际编辑常面对碎片化、无顺序的片段池，需要根据用户关键词重新组织成连贯故事。

Result: 在从100个长视频构建的合成数据集上，客观评估和定性例子显示能有效检索关键词相关片段并生成连贯故事板；人类评估在叙事连贯性、关键词对齐和整体真实感上优于强基线。

Insight: 创新点在于无需训练，结合多模态检索与LLM进行故事板编排，处理无序片段池；利用LLM选择排列片段并生成连接叙述。客观分析：将检索增强生成思路用于视频编辑，解耦片段检索与叙事生成，并引入辅助视觉检索提升叙述丰富度。

Abstract: Video editors often work with large collections of fragmented footage and must select and organize relevant clips into a coherent story. However, many existing video editing and summarization methods assume a temporally ordered source video and aim to preserve its original narrative and temporal structure. In this paper, we introduce Clips2Story, a training-free, LLM-based pipeline that enables users to create keyword-specified short videos from a pool of video clips. Given a pool of clips and a user-specified keyword, we first extract text-based multimodal descriptors for each clip and retrieve the clips most relevant to the keyword. We then provide the textual descriptors of these candidate clips to an LLM, which selects and arranges them into an editable storyboard with connective narrations. Finally, we assemble the selected clips and retrieve accompanying visuals for the narrations to compose the output video. To evaluate the proposed pipeline, we construct a synthetic evaluation dataset from 100 long-form videos across five genres by segmenting each video into short clips using a scene-change detection model and shuffling the resulting clips to remove their temporal order. Through objective evaluation and qualitative examples, we demonstrate that Clips2Story can effectively retrieve keyword-relevant clips and compose coherent storyboards based on user-specified keywords. Human evaluation further shows that Clips2Story outperforms a strong baseline in terms of narrative coherence, keyword alignment, and overall realism.

Xuanjing Zhang, Qi Guo, Yanan Qiao, Yong Han, Junge Bo

TL;DR: 论文提出FedIA，一种面向联邦提示学习（FPL）的实例感知跨模态表示框架。它利用全局文本提示作为语义锚点为每个候选类选择相关图像块，并据此生成实例级局部文本表示，再通过自适应全局-局部融合为每个图像-类对校准共享与个性化知识的贡献。该方法将FPL中的个性化从客户端级推进到实例级，同时保留跨客户端共享语义。

Details

Motivation: 现有FPL方法通常采用全局共享提示加客户端特定局部提示，但局部提示对整个客户端的所有样本相同，且全局与局部提示以输入无关的固定方式组合。这忽略了同一客户端内样本的视觉差异，也无法针对每个输入动态平衡共享知识与个性化知识。

Result: 在域偏移、病理性标签偏斜和Dirichlet标签偏斜三类设置下，FedIA均优于FPL基线，相对最新基线取得最大准确率提升1.44%-4.37%。消融实验进一步验证了实例感知表示生成和自适应全局-局部融合的互补贡献。

Insight: 创新点在于将个性化粒度从客户端级细化到实例级：用全局文本提示作为客户端不变语义锚点筛选类相关图像块，并以选出的视觉证据条件化生成局部文本表示；同时设计自适应融合模块按图像-类对校准全局与局部表示贡献。客观看，其可借鉴之处是将跨模态对齐用于抑制干扰区域，并实现输入相关的共享-个性化知识融合。

Abstract: Federated Prompt Learning (FPL) enables parameter-efficient adaptation of pre-trained Vision-Language Models across decentralized clients without exchanging raw data. To cope with cross-client heterogeneity, existing FPL methods typically pair a globally shared prompt with a client-specific local prompt. However, this design personalizes only at the client level: the same local prompt is applied to all samples of a client, and the global and local prompts are combined in an input-agnostic, fixed manner. Consequently, it overlooks the substantial visual variations among samples within the same client and cannot adjust the balance between shared and personalized knowledge to each input. To address these limitations, we propose Federated Prompt Learning with Instance-Aware Cross-Modal Representations (FedIA), an FPL framework that learns instance-aware cross-modal representations. Specifically, FedIA first uses global text prompts as client-invariant semantic anchors to select image patches most relevant to each candidate class, suppressing distracting regions. The selected visual evidence then conditions the generation of instance-aware local text representations. Finally, an adaptive global-local fusion module calibrates the contributions of local and global text representations for each image-class pair. In this way, FedIA shifts personalization in FPL from the client level to the instance level while preserving the semantic knowledge shared across clients. Extensive experiments under domain shift, pathological label skew, and Dirichlet-based label skew show that FedIA outperforms FPL baselines, with maximum accuracy gains of 1.44%-4.37% over the latest baseline across the three settings. Ablation studies and analyses further verify the complementary contributions of instance-aware representation generation and adaptive global-local fusion.

[48] MotoVerse: A Benchmark for Predictive Perception of Motorized Two-Wheelers in Dense Unstructured Traffic cs.CVPDF

Varun A. Paturkar, Shankar Gangisetty, C. V. Jawahar

TL;DR: 论文提出 MotoVerse，一个面向机动两轮车（MTW）自车视角的预测感知基准，包含密集非结构化交通中的 949 段机动片段及 3.1 万余名周围交通参与者的轨迹标注，支持自车骑手动作预判（ERMA）和交通智能体轨迹预测（TATP）两项任务。作者还提供 MTW 特定挑战分类，并对传统预测模型、通用/驾驶专用 VLM 及 VLA 模型进行评测，揭示视频模型与零样本 VLM 的差异及自车运动对预测难度的影响。

Details

Motivation: 机动两轮车在全球南方个人出行中占主导，但在自动驾驶与 ADAS 的预测感知研究中长期被忽视。论文旨在填补缺少 MTW 自车视角、密集非结构化交通下预测感知基准的问题。

Result: 构建了含 949 段片段、31,000+ 交通智能体（含 16,000+ MTW）轨迹标注的基准，并评测多类模型。实验发现：视频骨干网络在机动前 1/5 即可达到接近完整观测精度，零样本 VLM 需观测大部分机动过程；零样本通用/驾驶专用 VLM 两项任务上接近简单基线，LoRA 适配显著提升动作预判但仍落后传统模型的轨迹预测；左转和动态自车运动下所有模型误差最高。

Insight: 创新点在于从 MTW 自车视角构建密集非结构化交通预测感知基准，提供 ERMA 与 TATP 双任务、MTW 特定挑战分类及挑战感知评测。其价值还在于系统揭示视频模型、零样本 VLM、LoRA 适配和自车运动对预测性能的影响，为智能两轮车预测 AI 提供可借鉴的评测框架。

Abstract: Motorized Two-Wheelers (MTWs) dominate personal transport across much of the Global South, yet remain largely overlooked in predictive perception research for autonomous driving and ADAS. We introduce MotoVerse, a benchmark for predictive perception from an MTW’s ego-centric perspective. MotoVerse comprises 949 maneuver clips collected in dense, unstructured traffic and trajectory annotations for over 31,000 surrounding road agents, including over 16,000 MTWs, enabling two complementary tasks: Ego-Rider Maneuver Anticipation (ERMA) and Traffic Agent Trajectory Prediction (TATP). Each clip is further annotated with an MTW-specific challenge taxonomy comprising dynamic ego-motion, weak traffic structure and contextual occlusion, enabling challenge-aware evaluation. We benchmark established conventional prediction models, general-purpose Vision-Language Models (VLMs), driving-specific VLMs and vision-language-action models. Our experiments reveal three key insights: (i) video backbones reach most of their full-observation accuracy from the first fifth of a maneuver, whereas zero-shot VLMs need most of the maneuver to be observed; (ii) zero-shot VLMs, whether general-purpose or driving-specific, largely remain close to simple baselines on both tasks; a preliminary LoRA study shows that adaptation lifts anticipation sharply but leaves trajectory prediction behind conventional models; and (iii) the rider’s own motion drives forecasting difficulty: for every model, including a static baseline, errors are highest during left turns and rise under dynamic ego-motion. We believe MotoVerse will serve as a challenging benchmark for safer and more robust predictive AI for intelligent two-wheelers. The dataset and evaluation code are publicly available at https://github.com/VarunIIITH/MotoVerse.

[49] ChanSFormer: A Channel Agnostic Vision Transformer for Multi-Channel Cell Painting Images cs.CVPDF

Jingwei Zhang, Srinivasan Sivanandan, Dimitris Samaras

TL;DR: 论文提出 ChanSFormer，一种面向多通道高内容成像的通道无关视觉 Transformer。它用解耦的空间-通道注意力和独立通道 CLS token 取代可学习通道嵌入，从而在支持新通道的同时保留通道身份与跨通道信息流；并在 CHAMMI、JUMP-CP 和 So2Sat 上验证分类、跨数据集迁移和吞吐量优势。

Details

Motivation: 多通道成像中通道高度异质且实验配置不断变化，实际骨干网络需要具备通道无关性。现有通道自适应 ViT 依赖全局自注意力和可学习通道嵌入，难以处理新通道：丢弃嵌入会损失通道身份，独立处理各通道又会消除关键跨通道交互。

Result: 在 JUMP-CP 分类任务上，ChanSFormer 比 SOTA 方法准确率高 3.93%-4.27%；在自监督跨数据集迁移设置下，同分类任务准确率最高提升 2.72%；解耦注意力降低了通道×空间序列的二次复杂度，吞吐量提升 55%-313%。

Insight: 创新点在于用解耦空间-通道注意力和独立通道 CLS token 替代刚性通道嵌入，既保持通道身份又保留跨通道信息流，并天然支持新通道。独立通道 token 还提升了自监督表示学习效果，并支持基于表征的通道采样；客观看，其价值在于以更高效注意力结构统一通道无关性和跨通道建模。

Abstract: High-content multi-channel imaging, from Cell Painting assays to remote sensing, is central to modern scientific pipelines. Because these channels are highly heterogeneous and experimental configurations frequently evolve, a practical vision backbone must be fundamentally channel-agnostic. Current channel-adaptive vision transformers rely on global self-attention with learnable channel embeddings, which can be computed only for previously known channels. Attempts to overcome this limitation of new channels either discard the embeddings, losing channel identity, or process each channel independently, thereby eliminating critical cross-channel interactions. To address this, we introduce ChanSFormer, an efficient Vision Transformer that replaces rigid embeddings with disentangled spatial-channel attention and individual channel CLS tokens. This architecture natively preserves both distinct channel identities and cross-channel information flow. Furthermore, the individual channel class token design unlocks more informative feature representations for self-supervised learning and enables representation-based channel sampling while remaining channel-agnostic. We evaluate ChanSFormer on two biology multi-channel datasets, CHAMMI, JUMP-CP and a satellite dataset, So2Sat. Experimental results show that ChanSFormer outperforms state-of-the-art methods by 3.93%-4.27% on JUMP-CP in classification accuracy. It also demonstrates exceptional cross-dataset transferability under a self-supervised setting, outperforming previous methods by up to 2.72% in accuracy in the same classification tasks. Furthermore, the disentangled attention reduces the quadratic complexity of channel-by-spatial sequence length of the global attention baseline, thus improving throughput by 55%-313%. Code is available at https://github.com/cvlab-stonybrook/ChanSFormer.

[50] USI-Brain: Enhancing Underwater Spatial Intelligence via Spatial Data-Driven Vision-Language Modeling cs.CVPDF

Zhiliang Zhu, Zhiqiang Zhao, Xie’an Jiang, Yingxin Lai, Kangye Zheng

TL;DR: 论文面向水下场景的空间推理能力，构建了一个空间数据引擎，通过图像增强、2D与几何辅助的问答生成以及专家过滤，将视觉外观与物体级几何关联起来，产出覆盖五类能力组的427万条QA对；基于该语料训练了视觉-语言模型USI-Brain，并提出水下先验引导校正器（UPR）对视觉特征进行门控式修正；同时发布包含6,525条QA对的评测基准USI-Bench。

Details

Motivation: 水下空间推理高度依赖物体边界与几何线索，而散射、颜色失真和低能见度会遮蔽这些线索；现有水下数据虽包含图像、标注和深度图，但为视觉-语言模型提供的监督是碎片化的，缺乏连贯的空间-语言监督信号。

Result: 在USI-Bench的15项任务指标上，USI-Brain与USI-Brain+UPR分别比Qwen3-VL-8B-Instruct平均高出5.76分和7.63分（百分制带符号差值）；消融显示，在关闭UPR时，几何辅助监督使深度与顺序任务分别提升4.8和6.1分；在相同训练数据下，UPR使场景描述任务提升18.0分。在NautData、UWBench和MarineEval上，32项指标中有28项取得最佳或并列最佳成绩，且评测仅使用原始图像。

Insight: 论文宣称的创新在于：以空间数据引擎将几何深度信息转化为语言监督信号，构建大规模水下空间QA语料；提出轻量且即插即用的水下先验引导校正器（UPR）；并建立USI-Bench评测体系。从客观角度看，其可取之处是把几何先验显式注入视觉-语言训练，并通过门控机制在不依赖深度图推理的前提下增强水下空间感知。

Abstract: Underwater spatial reasoning depends on object boundaries and geometric cues that are obscured by scattering, color distortion, and low visibility. Existing underwater resources capture these cues through images, annotations, and depth maps, but provide fragmented supervision for vision-language models. We present a spatial data engine that connects visual appearance with object-level geometry through enhancement, 2D and geometry-assisted question generation, and expert filtering. It produces 4.27M QA pairs across five capability groups. We use this corpus to train USI-Brain and introduce an Underwater Prior-guided Rectifier (UPR) that applies gated corrections to visual features. USI-Bench evaluates the resulting capabilities with 6,525 held-out QA pairs. On its 15 task metrics, USI-Brain and USI-Brain + UPR improve over Qwen3-VL-8B-Instruct by 5.76 and 7.63 points on average, respectively, using signed differences on 100-point scales. Ablations show that geometry-assisted supervision improves depth and ordering by 4.8 and 6.1 points with UPR disabled, while UPR adds 18.0 points in scene description under matched training data. Both variants use raw images at evaluation. Across NautData, UWBench, and MarineEval, USI-Brain achieves the best or joint-best score on 28 of 32 metrics. Model weights and USI-Bench will be released at https://github.com/SKY-Zhu-91/USI-Brain.

[51] RI-Prune: Relation and Information Aware Visual Token Pruning for Multi-Image Vision-Language Models cs.CVPDF

Dongjin Lee, Eunju Lee, Jinhee Jang, Yoonji Lee, Youngbin Kim

TL;DR: 论文针对多图像视觉语言模型（LVLM）中视觉token预算过大的问题，指出现有基于注意力的剪枝方法在多图像场景下性能下降且存在位置偏差。通过因果探针实验揭示注意力信号是弱代理后，提出一种无需训练且不依赖注意力的预解码器选择器RI-Prune，它利用投影器输出的查询相关性、跨图像对应（基于紧凑的每图像原型）和图像内代表性三个信号进行z-normalized组合打分，并执行平衡的每图像Top-K选择。在多个多图像基准和三个LVLM骨干上，RI-Prune在token减少与任务性能的权衡上优于先前方法，尤其在激进token预算下。

Details

Motivation: 现有视觉token剪枝方法主要面向单图像场景，依赖注意力显著性信号估计token重要性。然而在多图像场景中，这种注意力中心的设计迁移性差，导致性能显著下降、强位置偏差，且随着输入图像数量增加，判别性注意力信号减弱。论文通过因果探针实验验证注意力选择的重要性有限，从而需要一种更适合多图像推理的剪枝方法。

Result: 在多个多图像基准和三个LVLM骨干上，RI-Prune在token减少与任务性能的权衡上优于先前剪枝方法，尤其在激进的token预算设置下表现更佳，实现了更强的效率-性能平衡。

Insight: 论文的创新点在于：1）通过因果探针实验（Leave-One-In和Leave-One-Deletion）实证了注意力信号在多图像推理中的弱代理性；2）提出训练无关、不依赖注意力的预解码器剪枝框架RI-Prune，综合查询相关性、跨图像对应和图像内代表性三个信号；3）采用平衡的每图像Top-K选择以保留跨图像信息。客观分析认为，其核心创新是重新定义了多图像场景下的token重要性度量，摒弃注意力并引入跨图像对应与图像内代表性，且无需额外训练，具有实用性和可迁移性。

Abstract: Large vision-language models (LVLMs) have recently been extended to handle multiple input images, but this extension dramatically increases the visual token budget, making visual-token pruning not merely an efficiency option but a practical necessity. Existing visual-token pruning methods, primarily designed for the single-image setting, rely on attention-derived saliency signals to estimate per-token importance. However, this attention-centric design transfers poorly to the multi-image regime: it causes substantial performance degradation on multi-image benchmarks, suffers from strong positional bias, and loses discriminative attention signals as the number of input images increases. To test whether attention-selected tokens are truly important for multi-image reasoning, we conduct two complementary causal probes Leave-One-In and Leave-One-Deletion, which measure token importance by restoring individual tokens to a zeroed visual sequence or removing them from the full sequence. Attention-based pruners provide only marginal gains over random selection, showing that attention is a weak proxy for visual-token importance in multi-image reasoning. Motivated by this gap, we propose Relation and Information Aware Pruning (RI-Prune), a training-free, attention-free pre-decoder selector for multi-image visual token pruning. RI-Prune scores each visual token using a z-normalized combination of three signals computed from projector outputs: query relevance, cross-image correspondence with compact per-image prototypes, and intra image representativeness. By integrating these signals, RI-Prune performs balanced per-image Top-K selection to preserve information across input images. Across multiple multi-image benchmarks and three LVLM backbones, RI-Prune achieves a stronger trade-off between token reduction and task performance than prior pruning methods, particularly under aggressive token budgets.

[52] TouchéWorld tech report : Towards Online Unified Camera–Hand Exploration in Generative Worlds cs.CV | cs.AIPDF

Jiaao Yu, Yanjian Zhang, Haoyang Jiang, Guillaume Wisniewski, Nadi Tomeh

TL;DR: TouchéWorld 是一个将手势驱动交互融入相机可控视频世界模型的框架。给定初始图像和场景描述，它生成同时受时间对齐的相机与手部控制条件约束的视频，并支持基于生成视觉反馈的分块级在线控制更新。论文还构建了联合相机-手部训练数据流程，并提出了 TouchéBench 评测基准。

Details

Motivation: 现有视频世界模型主要支持相机导航探索，缺少连续手部运动这一细粒度交互通道；论文旨在统一相机探索与手部驱动交互，让用户能在生成世界中与遇到的人或物体互动。

Result: 摘要未给出具体定量结果或 SOTA 对比，主要贡献是提出框架、联合数据构建方法、在线生成机制以及 TouchéBench，用于评估手部运动跟随、相机-手协调、在线交互和物体响应。

Insight: 创新点在于将空间手部骨架条件与数值手部运动特征结合到预训练相机控制世界模型中，并通过参数高效适配、因果适配和少步蒸馏支持在线分块生成；同时设计了从异构第一人称视频构建联合控制数据的流程和相应评测基准。

Abstract: Recent video world models allow users to explore generated environments through camera navigation. Continuous hand motion provides a fine-grained action channel for interacting with the people and objects encountered during exploration. We present TouchéWorld, a framework that integrates gesture-driven interaction into a camera-controlled video world model. Given an initial image and a scene-description prompt, the model generates video conditioned on temporally aligned camera and hand controls. We construct joint camera–hand training data from heterogeneous egocentric videos through annotation completion, consistency-based filtering, and temporal alignment. Building on a pretrained camera-controlled world model, we combine spatial hand-skeleton conditioning with numerical hand-motion features through parameter-efficient adaptation. Causal adaptation and few-step distillation support chunk-level online generation, allowing users to update their controls based on generated visual feedback. We further introduce TouchéBench to evaluate hand-motion following, camera–hand coordination, online interaction, and object response. Together, these components establish a framework for studying continuous exploration and hand-driven interaction within generative worlds.

[53] SleuthVLM: Do We Really Need Audio-in-Video for Omni Video Understanding? cs.CVPDF

Peiran Wu, Fangfei Gou, Zhaoxi Mu, Changhao Zhang, Jian Liu

TL;DR: 论文提出 SleuthVLM，一个基于 VLM 的智能体式视频理解框架：它不直接接收视频中的音频，而是先把语音转写成文本，再让 VLM 自主检索和分析长视频，无需预定义工作流。作者在多个 omni 视频理解基准上验证该方案，发现音频直接输入并非必需，智能体能力才是关键。

Details

Motivation: 现有 omni 模型通常联合处理采样帧和音频，并依赖扩大模型规模与 omni 训练数据来提升性能；论文质疑视频理解是否真的需要模型直接“听”音频，探索用转写文本加智能体推理替代直接音频输入。

Result: 在四个 omni 基准、视频时长最高 90 分钟的评测中，SleuthVLM 取得最佳平均准确率 75.0%，比 Gemini 3.8 Flash 高 2.6 个百分点；在 Video-MME-v2 排行榜上以 77.8% 登顶；基于开源 27B VLM 时，超过所有开源 omni 模型。

Insight: 创新点在于用语音转文本解耦音频感知与视频理解，并构建无需预定义流程的智能体式 VLM 自主搜寻视频信息。客观来看，论文挑战了“omni 理解必须依赖音频模态对齐”的假设，指出瓶颈更多在智能体能力而非听觉输入，为 omni 视频理解提供了简单有效的新起点。

Abstract: Omni models answer questions about videos by jointly processing sampled frames and audio in one turn, with progress driven largely by scaling model size and omni training data. We ask whether a model needs to hear the audio directly at all, and present SleuthVLM, an agent harness built on a VLM that never hears the audio directly: speech will be transcribed into text, and the VLM will sleuth the video on its own without any predefined workflow. On four omni benchmarks with videos up to 90 minutes, SleuthVLM achieves the best average accuracy: \textbf{75.0%}, 2.6 points above Gemini 3.8 Flash, and it also tops the Video-MME-v2 leaderboard with \textbf{77.8%}; with an open-source 27B VLM, it beats all open-source omni models. We draw three findings: SleuthVLM beats a model that glances; hearing is not the bottleneck, agentic ability is; omni understanding needs better agentic ability, not just more omni data for modality alignment. So we provide a simple but effective starting point for doing omni video understanding. Omni understanding needs better agents, not just bigger models that hear.

[54] GeoBiaset: A Counterfactual Benchmark for Demographic Bias in World-Level Geolocalization cs.CVPDF

Niccolò Niccoli, Silvia Dani, Marco Mistretta, Andrew D. Bagdanov, Marco Bertini

TL;DR: 论文提出GeoBiaset，首个测量世界级图像地理定位模型受图像前景人物表观种族影响的基准，包含9,541张反事实编辑图像（固定真实坐标）。同时提出POPI指标量化模型对人口统计扰动的韧性。评估SOTA地理定位和多模态LLM发现，当插入人口统计不匹配群体时大陆级性能下降16-47%，且模型在有地标时关注地标，无地标时转向面部特征。

Details

Motivation: 现有工作仅发现地理定位模型存在数据集不平衡和区域性能差异，但未研究哪些视觉线索（如人物种族）驱动这些差异。论文旨在通过反事实基准揭示人口统计偏差对地理定位的影响机制。

Result: 在GeoBiaset上评估SOTA地理定位和多模态LLM，大陆级性能下降16-47%；控制掩码实验表明遮挡和背景漂移不能完全解释性能下降；地标与无地标分析及梯度加权热图显示模型在有地标时优先关注地标，无地标时转向面部特征。

Insight: 创新点包括：首个反事实基准系统测量种族偏差对地理定位的影响；提出POPI指标量化模型对人口统计扰动的韧性；揭示模型注意力机制在有无地标时的转变。客观分析：通过可控反事实编辑隔离种族因素，提供可量化偏差度量，对公平性研究有借鉴意义。

Abstract: World-level image geolocalization is an increasingly important task, yet the biases of current models remain poorly understood. Prior work has identified coarse dataset imbalances noting, for instance, that some cities are absent from training sets and that models tend to perform better in wealthier regions. However, no one has yet examined which visual cues drive these disparities. To do so, we introduce GeoBiaset, the first benchmark designed to measure how geolocalization models are influenced by the apparent race of a person inserted into the image foreground. The dataset consists of 9,541 images, pairing existing geotagged background images with counterfactual edits spanning seven races, holding the ground-truth coordinates fixed. Controlled masking experiments show that occlusion and background drift do not fully explain geolocalization degradation. Alongside the benchmark, we introduce the Person-Origin Perturbation Indicators (POPI), novel metrics that quantify model resilience to these demographic perturbations by measuring whether predictions are pulled toward the region associated with the inserted group. Our evaluation of state-of-the-art geolocalization and multimodal large language models reveals a 16-47% drop in continent-level performance when a subject from a demographically mismatched group is inserted. A landmark versus no-landmark analysis and gradient-weighted heatmaps show that models preferentially attend to geographic landmarks when present, but shift attention to facial features in their absence.

[55] STARE: Scene Text-Aware multimodal Retrieval for Everyday objects cs.CVPDF

Kento Tokura, Ryosuke Korekata, Daichi Yashima, Kei Katsumata, Kanon Amemiya

TL;DR: 论文面向服务机器人根据自由形式指令检索和搬运日常物体的任务，提出场景文本感知多模态检索方法STARE。该方法同时利用视觉特征、空间关系与环境中的场景文本，引入跨语言视觉提示增强场景文本理解，并建模指令中命名实体与目标物体间的复杂关系。作者在GoGetIt和TextCaps-test两个新基准上评估，并在零样本迁移物理实验中验证。

Details

Motivation: 现有服务机器人需理解包含视觉特征、空间关系及场景文本的自由形式指令，仅靠单一模态难以准确定位目标物体，因此需要统一的多模态检索与场景文本理解能力。

Result: 在GoGetIt和TextCaps-test两个新基准上，STARE在所有标准多模态检索指标上优于全部基线方法；零样本迁移物理实验中总体成功率达到80%。

Insight: 创新点在于将场景文本作为检索线索显式引入日常物体检索，并通过跨语言视觉提示增强场景文本解释，同时建模指令中命名实体与对应物体的复杂关系。

Abstract: In this study, we focus on service robots that retrieve and transport everyday objects based on free-form instructions that may refer to visual features and scene text. This is challenging because it requires a multimodal understanding of the visual features and spatial relationships of objects, as well as scene text embedded in the environment. To address this, we propose Scene Text-Aware multimodal Retrieval for Everyday objects (STARE), which leverages visual and textual cues in response to free-form instructions. STARE introduces Crosslingual Visual Prompt to enhance scene text interpretation, and models the complex relationships between named entities in the instructions and their corresponding objects. We evaluate STARE using two novel benchmarks, GoGetIt and TextCaps-test, both of which consist of real-world indoor/outdoor images and instructions involving object manipulation and navigation. The results show that STARE outperforms all baseline methods on standard multimodal retrieval metrics. Furthermore, in physical experiments conducted under a zero-shot transfer setting, STARE achieves an overall success rate of 80%. Our project page and code are provided in the Supplementary Materials.

[56] Drift-adaptive Representation Preservation for Domain-Incremental Learning cs.CV | cs.AIPDF

Hyunmin Kong, Gyutae Oh, Jitae Shin

TL;DR: 论文针对域增量学习（DIL）中适配新域会导致旧域遗忘的问题，提出用表示漂移（过去概念的当前表示与固定锚点的偏差）作为遗忘风险的可测量代理，并按漂移大小自适应分配表示保持力度：高漂移的域-类对获得更强的巩固损失约束，同时在原型构建中被降权。为在不存储原始历史数据的前提下获得漂移信号，方法复用JEPA架构，仅保存骨干特征空间中的紧凑上下文摘要并重放其context-to-target关系来测量漂移。

Details

Motivation: 预训练视觉模型虽具强通用表示，但在输入分布顺序变化的DIL场景中适配新域会显著损害旧域性能；且作者观察到遗忘在域和类之间并不均匀，无法仅由训练顺序解释，因此需要按域-类概念逐个跟踪并度量遗忘风险。

Result: 在DomainNet、Office-Home、CORe50和CDDB-Hard四个benchmark上均取得了较好的精度-遗忘权衡，训练内存显著低于精度相当的方法，且存储开销为对比方法中最小。

Insight: 创新点在于以可测量的表示漂移替代主观假定的重要性来分配表示保持资源，并将漂移信号同时用于巩固损失加权与原型降权；同时借助JEPA的上下文-目标关系重放机制，在不保存原始数据的情况下实现漂移度量，兼顾了隐私友好与低存储。

Abstract: Pretrained vision models provide strong general-purpose representations, but in Domain-Incremental Learning (DIL), where the input distribution changes sequentially, adapting to new domains often degrades performance on previously learned ones. We observe that forgetting is not uniform across domains and classes and is not explained by training order alone. This motivates us to track each past domain-class concept individually. We use representation drift, defined as the deviation of a past concept’s current representation from a fixed anchor, as a measurable surrogate for forgetting risk. The measured drift then serves two complementary roles. A consolidation loss assigns stronger preservation to high-drift pairs, while prototype construction down-weights highly drifted domains to reduce their influence on the class reference. Preservation is thus allocated by measured drift rather than assumed importance. To obtain this signal during training without storing past raw data, we repurpose a Joint-Embedding Predictive Architecture (JEPA). For each domain-class pair, we store only a compact context summary in the backbone feature space and rerun it through the learned context-to-target relation to measure its drift from the fixed anchor. Across DomainNet, Office-Home, CORe50, and CDDB-Hard, our method sustains a strong accuracy-forgetting balance on every benchmark, using substantially less training memory than accuracy-competitive methods and the smallest storage among compared methods.

Jiali Chen, Ruitao Pu, Yingke Chen, Dezhong Peng

TL;DR: 论文针对跨模态检索中模态不完整与标签噪声共存的问题，提出语义先验引导框架SPIRN。该方法以固定语义先验作为一致参考，联合进行鲁棒表示学习、噪声标签处理与缺失模态补全，并在多个基准上验证有效性。

Details

Motivation: 现有不完整跨模态检索方法多依赖类别标签作为语义基准，难以应对真实场景中标签噪声与模态缺失同时出现的问题，因此需要一种能同时处理双重挑战的鲁棒框架。

Result: 论文在四个广泛使用的基准数据集上进行了大量实验，覆盖多种缺失模态和噪声标签场景，结果表明SPIRN优于11种先进方法。

Insight: 论文宣称首次在跨模态检索社区联合处理模态不完整与标签噪声；其创新在于用固定语义先验统一指导表示学习、噪声判别与标签校正，并通过质量感知反向学习、先验引导全局标签校准和置信度驱动的选择性插补提升鲁棒性。

Abstract: Cross-modal retrieval (CMR) aims to bridge the semantic gaps between modalities by learning shared representations. Unfortunately, due to the dynamism and inconsistencies in real-world data acquisition, multimodal datasets frequently encounter severe modality incompleteness. Existing incomplete CMR methods adhere to a label-driven paradigm, relying heavily on categorical annotations as the semantic benchmark. However, label noise is difficult to avoid and often coexists with modality incompleteness. To address this challenge, we propose a novel semantic prior-guided framework for incomplete cross-modal retrieval with noisy labels, termed SPIRN. Specifically, SPIRN uses fixed semantic priors as a consistent reference for robust representation learning, noisy-label handling, and missing-modality completion. Based on this reference, Semantic Priors Guidance (SPG) establishes a stable category-level semantic space, Quality-aware Robust Learning (QRL) dynamically evaluates sample quality to distinguish clean samples from noisy ones and handles noise via reverse learning, Prior-guided Global Label Calibration (PGLC) rectifies corrupted annotations for refined supervision, and Confidence-driven Selective Imputation (CSI) recovers missing modalities from a cross-modal memory bank while filtering unreliable imputations with an increasing threshold. To the best of our knowledge, this is the first framework to jointly tackle the dual challenges of modality incompleteness and label noise in the CMR community. Extensive experiments on four widely used benchmark datasets demonstrate that SPIRN outperforms 11 state-of-the-art methods in various scenarios involving both missing modalities and noisy labels.

[58] ReactWorld: Structuring Interaction Responses for Embodied Video Generation cs.CV | cs.ROPDF

Hanwei Liu, Huijia Zhang, Hanzhang Wang, Guangyu Liu, Hanbei Zhan

TL;DR: 论文提出ReactWorld，一种由结构化交互预期引导的具身视频生成模型。它在生成骨干特征空间中用交互响应场预测器预测参与者角色、未来运动响应强度和局部运动变化，并通过响应反馈适配器将对预测/置零局部动态输入的对象特征差异转为对象特征增量，同时把预测位移转为时间注意力的位置偏置，以指导对象特征更新和跨帧信息检索。

Details

Motivation: 具身视频世界模型需根据机器人动作预测场景变化，但生成的对象响应常与任务意图和接触动力学不一致；且关键响应往往集中在局部区域和特定时间区间，因此需要类似思维链的结构化交互预期来约束生成。

Result: 在RBench上平均得分77.51，比PhysisForcing高3.29分，并在全部九个类别上均有提升；在EZS-Bench上Domain得分为86.98，平均得分为81.87。

Insight: 创新点在于将结构化交互响应预期引入具身视频生成，并用响应反馈适配器把局部动力学差异转化为对象特征增量和时间注意力位置偏置，从而以可解释的交互先验和生成反馈共同指导视频演化。

Abstract: Embodied video world models predict scene changes caused by robot actions, yet generated object responses may remain inconsistent with task intent and contact dynamics. Critical responses in embodied scenes often concentrate in local regions and specific time intervals. Inspired by chain-of-thought reasoning, we introduce ReactWorld, an embodied video generation model guided by structured interaction anticipation. In the generation backbone’s feature space, an interaction response field predictor anticipates how the interaction will unfold, explicitly describing participant roles, future motion response intensity, and local motion changes. A response feedback adapter forms increments to object features from the difference between the object branch’s outputs with predicted and zeroed local dynamics inputs. It also converts predicted displacements into positional biases for temporal attention. These two mechanisms guide object feature updates and information retrieval across frames, respectively. Through interaction anticipation and generative feedback, ReactWorld achieves an average score of 77.51 on RBench, outperforming PhysisForcing by 3.29 points with improvements in all nine categories. On EZS-Bench, it achieves a Domain score of 86.98 and an average score of 81.87.

[59] RDT-3D: Resolution-Decoupled Training for High-Fidelity 3D Geometry Generation cs.CVPDF

Jiale Zhang, Yang Liu, Chengyang Li, Jin Yuan, Qianxi Jia

TL;DR: 论文针对高保真3D几何生成中训练与推理分辨率不一致的问题，提出分辨率解耦训练方法RDT：在训练时以512分辨率的RoPE坐标引导配合1536分辨率几何监督，使模型在推理时能从粗坐标稳健外推细节。论文还构建了Dual3D-300K大规模配对数据集，包含30万水密3D点云并在512和1536分辨率间显式对齐，用于缓解高分辨率数据瓶颈并支持高分辨率3D生成评测。

Details

Motivation: 现有由粗到细的3D生成框架虽能兼顾全局结构与局部细节，但训练时坐标引导与几何监督分辨率对齐，推理时却从粗空间坐标解码高分辨率几何，导致严重分布偏移并损害细节质量。论文旨在解决这一训练-推理分辨率差异问题。

Result: 摘要称大量实验表明RDT能显著改善尖锐几何区域的高保真生成效果，Dual3D-300K可作为高分辨率3D生成的可扩展基准；但未给出具体定量指标、对比基准或SOTA结论。

Insight: 创新点在于揭示并显式建模粗到细3D生成中的训练-推理分辨率错配，通过低分辨率RoPE引导与高分辨率几何监督的解耦训练实现细节外推；同时构建跨512/1536分辨率对齐的大规模水密点云数据集，为高分辨率3D生成提供数据与评测支撑。

Abstract: High-fidelity 3D geometry generation requires maintaining global structural coherence alongside fine-grained surface details. While recent coarse-to-fine frameworks leverage Rotary Position Embeddings (RoPE) to condition local queries, we reveal a critical training-inference resolution discrepancy: coordinate guidance and geometric supervision are aligned during training, whereas inference decodes high-resolution geometry from coarse spatial coordinates. This mismatch introduces a severe distribution shift that degrades fine details. To overcome this, we propose Resolution-Decoupled Training (RDT), which anchors RoPE guidance at 512 resolution while supervising geometry at 1536, enabling robust detail extrapolation. Additionally, to eliminate high-resolution data bottlenecks, we present Dual3D-300K, a large-scale paired dataset comprising 300K watertight 3D point clouds explicitly aligned across 512 and 1536 resolutions. Extensive experiments demonstrate that RDT substantially improves sharp geometric regions, while Dual3D-300K offers a scalable benchmark for high-resolution 3D generation.

[60] PTQ4SAM3: Semantics-Preserving Post-Training Quantization for Segment Anything Model 3 cs.CVPDF

Xi Xiao, Yunbei Zhang, Lin Zhao, Janet Wang, Xinru Jiang

TL;DR: 论文针对 SAM 3 在部署时内存和计算开销大的问题，提出语义保持量化 SPQ（Semantics-Preserving Quantization），用于缓解训练后量化导致的提示对齐漂移。方法通过概念敏感参数校准加权、残差特征保持和视频记忆均值误差校正，在离线阶段完成且不增加推理算子。

Details

Motivation: SAM 3 支持文本提示分割和视频目标跟踪，但存储与计算成本高；PTQ 虽能降低比特数，却可能导致量化模型选错目标或跨帧丢失目标，即 prompt-alignment drift。

Result: 在 W4A8 图像编码器设置下，SPQ 将 RefCOCO Pr@0.5 从 PTQ4SAM 的 42.1 提升到 72.5，并保持 54.8 COCO mask AP；在 LVIS、RefCOCO 变体和 SA-V 上取得一致增益，CLIP 研究也显示图文检索性能提升。

Insight: 创新点在于将量化误差与提示对齐/概念语义稳定性关联，利用固定文本概念进行无任务标注校准，并通过残差特征保持和视频记忆均值误差校正实现离线语义保持量化，且不引入额外推理算子。

Abstract: Segment Anything Model 3 (SAM 3) can segment objects from text prompts and track them across video, but its memory and computation costs make deployment difficult. Post-training quantization (PTQ) reduces these costs by using fewer bits. However, a quantized model can select the wrong object or lose track of its target across frames. We call this prompt-alignment drift. Controlled tests show that perturbing low-variance feature directions strongly affects target selection. We propose Semantics-Preserving Quantization (SPQ), which gives more calibration weight to concept-sensitive parameters, preserves residual features, and corrects the mean error in video memory. Fixed text concepts guide calibration without task annotations. All three components operate offline and add no inference operators. With a W4A8 image encoder, SPQ improves RefCOCO Pr@0.5 from 42.1 to 72.5 over PTQ4SAM while retaining 54.8 COCO mask AP. Experiments on LVIS, RefCOCO variants, and SA-V show consistent gains. A CLIP study also shows improved image-text retrieval.

[61] Grounding in the Dark: Text-Only Training for Zero-Shot Video Temporal Localization cs.CV | cs.AIPDF

Divyanshu Mishra, Sabine Sternig, Rakshith Shetty, Erhan Gundogdu

TL;DR: 论文探索仅用文本描述训练LLM、推理时再输入视频的零样本视频时序定位设置，完全不使用成对视频-文本数据训练。作者构建13.4万条文本时序语料，基于Qwen3-8B和CLIP/LLM2CLIP文本嵌入做LoRA适配，并提出免训练的最近实体替换（NER）来缓解文本到视频的模态鸿沟。

Details

Motivation: 精确时序标注和大规模视频训练成本高，论文希望验证无视觉-语言预训练的纯文本LLM能否仅凭文本视频描述学会时序定位，并在推理时接收视频，从而减少对成对视频标注的依赖。

Result: LLM2CLIP文本编码器在长描述上优于CLIP，Temporal-TextVid上mtIoU为61.54对53.86；但直接用帧嵌入推理仅得Charades-STA 4.99、QV-Highlights 11.12 mtIoU。NER将结果提升到14.45和20.72 mtIoU，分别比最强模态鸿沟基线高5.00和1.71点。

Insight: 创新点在于纯文本训练实现零样本视频时序定位，并用免训练的最近实体替换将视频帧映射为文本实体嵌入均值以桥接模态鸿沟。客观看，该工作揭示了文本到视频迁移的可行性但仍有明显模态差距，更可靠的定位可能依赖文本与视觉表示更好对齐的编码器。

Abstract: Video-LLMs are typically trained on paired video-text data for temporal tasks, yet collecting precise temporal annotations and training on large-scale video datasets are both costly. We investigate whether a text-only LLM, with no vision-language pretraining, can be trained on textual video descriptions alone to perform temporal grounding and receive video only at inference time. This setting removes paired video data from training altogether and is a step toward training multimodal models without paired annotations. We generate a 134K-example corpus of textual sequences with up to 100 time steps, adapt Qwen3-8B with LoRA on CLIP or LLM2CLIP text embeddings, and evaluate on Charades-STA and QV-Highlights without training on videos from either benchmark. LLM2CLIP, whose text encoder supports longer inputs, represents detailed descriptions better than CLIP (61.54 versus 53.86 mtIoU on Temporal-TextVid) and transfers them better to video. Still, replacing text with frame embeddings at inference yields only 4.99 mtIoU on Charades-STA and 11.12 on QV-Highlights, revealing a large modality gap. To bridge it, we propose Nearest Entity Replacement (NER), a training-free mapping that represents each video frame by the normalized mean of its nearest text-entity embeddings, so that the model receives the kind of input it was trained on. NER reaches 14.45 mtIoU on Charades-STA and 20.72 on QV-Highlights, outperforming the strongest of four families of modality-gap baselines by 5.00 and 1.71 points. NER thus enables partial transfer from text to video, while more reliable grounding will likely require encoders with better-aligned text and visual representations.

[62] DSRS: Teaching MLLMs to Perceive, Reason, and Segment in Complex Driving Scenes cs.CVPDF

Sicen Guo, Zhengfei Song, Kuanhsun Chen, Xu Huang, Ping Zhong

TL;DR: 论文提出面向复杂驾驶场景的推理分割任务DSRS，要求多模态大模型理解隐含指令、推断空间关系、识别潜在风险并遵循交通规则，最终生成准确分割掩码；同时构建基准数据集并提出DSRS-Net，通过引入几何特征增强推理分割能力。

Details

Motivation: 现有推理分割框架依赖MLLM的通用世界知识，但缺乏驾驶场景所需的场景特定感知，难以处理复杂交通规则、风险与空间关系。

Result: 摘要未给出具体数值；实验表明DSRS-Net在DSRS任务及多个公开数据集上的语言驱动分割任务中表现优越。

Insight: 创新点在于定义驾驶场景推理分割任务并构建基准，同时设计DSRS-Net在不破坏原有多模态融合机制的前提下融合几何特征，缓解RGB-only预训练导致的模态表示不匹配。

Abstract: Existing reasoning segmentation frameworks typically employ multimodal large language models (MLLMs) with broad world knowledge to extract semantics in a human-like manner and generate corresponding segmentation masks. Nevertheless, these frameworks lack the necessary scene-specific awareness. This paper introduces driving scene reasoning segmentation (DSRS), a scene-specific task that requires models to interpret implicit instructions, infer spatial relations, detect potential risks, handle challenging conditions, and comprehend underlying traffic rules, ultimately generating accurate segmentation masks. Meanwhile, we establish a benchmark dataset to facilitate future research on the DSRS task. To pioneer this task, we propose DSRS-Net, a novel architecture with advanced reasoning segmentation capability for automated driving. Specifically, DSRS-Net seamlessly infuses geometric features for deeper reasoning segmentation, without compromising the original multimodal fusion mechanism. This strategy effectively mitigates the modality representation mismatch in MLLMs caused by RGB-only pretraining, while enhancing the integration of geometric, contextual, and linguistic features. Extensive experiments demonstrate the superior performance of DSRS-Net in both driving scene reasoning segmentation and other language-driven segmentation tasks across various public datasets. The source code and dataset will be released upon publication.

Xinghao Li, Chenchen Tan, Youyang Qu, Shujie Cui, Xingliang Yuan

TL;DR: 论文研究多模态大型推理模型中的跨模态关联遗忘问题，提出将遗忘目标定义为特定的“视觉到知识”关联，而非单纯样本、概念或知识内容。作者提出参数高效的 Visual-Keyed Counterfactual Rebinding（VKCR）框架，通过定位关联条件参数子空间并进行反事实重绑定，抑制指定视觉上下文下的目标行为，同时保留允许上下文中的知识访问。论文还构建 CLEAR-R 和 MLLMU-R 评估目标泄漏、幻觉重关联、推理保持和模态特定效用。

Details

Motivation: 多模态大推理模型在推理中可能通过跨模态关联访问敏感信息；现有遗忘方法主要针对样本、概念或知识内容，未将知识与访问它的跨模态关联分离，且推理模型中目标信息可能从最终答案消失但仍留在中间推理中，或被错误替代知识替换。

Result: 实验表明，VKCR 能显著降低图像条件下的目标泄漏和幻觉重关联，并在异构任务形式下有效，同时保持非敏感视觉推理能力和允许的纯文本知识访问。

Insight: 创新点在于提出跨模态关联遗忘任务，将遗忘目标从知识内容转向指定视觉到知识关联；VKCR 通过关联条件参数子空间定位和反事实重绑定实现参数高效编辑，并引入 CLEAR-R、MLLMU-R 以更细粒度评估泄漏、幻觉重关联、推理保持和模态效用。

Abstract: Multimodal large reasoning models (MLRMs) integrate visual inputs with textual and parametric knowledge, allowing sensitive information to be accessed through cross-modal associations during reasoning. This raises privacy concerns when a visual input should no longer provide access to such information. Machine unlearning can remove this influence without retraining the model from scratch. However, multimodal unlearning methods mainly target samples, concepts, or knowledge content, without separating the knowledge from the cross-modal association through which it is accessed. In reasoning models, target information may disappear from the final answer while remaining in intermediate reasoning, or be replaced by incorrect substitute knowledge. We therefore study cross-modal association unlearning, where the forget target is a designated visual-to-knowledge association: the model should suppress target-bearing behavior under the designated visual context while preserving access to the same knowledge in permitted contexts. We propose Visual-Keyed Counterfactual Rebinding (VKCR), a parameter-efficient framework for cross-modal association editing. VKCR first localizes the edit to a compact association-conditioned parameter subspace. It then reduces support for the designated target at task outputs and counterfactually rebinds the affected reasoning behavior by discouraging target-bearing or unsupported continuations while favoring valid non-sensitive alternatives. We further introduce CLEAR-R and MLLMU-R to evaluate target leakage, hallucinated reassociation, reasoning preservation, and modality-specific utility. Experiments show that VKCR substantially reduces image-conditioned target leakage and hallucinated reassociation, including across heterogeneous task formulations, while preserving non-sensitive visual reasoning and permitted text-only knowledge.

[64] Rohbau3D-Semantics: Dense Semantic and Instance Labels for Point Clouds of Indoor Shell Construction Scenes cs.CVPDF

Lukas Rauch, Thomas Braml

TL;DR: 论文发布 Rohbau3D-Semantics 数据集，在既有 Rohbau3D 毛坯施工场景激光扫描基础上增加密集语义与实例级标注。数据包含来自14个测量站点的504个点云场景，标注17类建筑构件及背景，并提供全景/立方体图等多模态表示生成脚本、基准划分和语义分割基线实验。

Details

Motivation: 现有室内数据集多为合成数据或由重建网格采样得到，缺少真实激光扫描特性与施工毛坯场景的高质量密集标注基准；该工作旨在连接合成室内数据集、网格派生基准和小规模施工专用数据之间的空白。

Result: 数据集包含504个点云场景、14个测量站点、17个构件类别加1个背景类，并提供实例标注、基准划分和语义分割基线实验；摘要未给出具体精度或SOTA结果。

Insight: 创新点在于面向真实毛坯施工现场激光点云构建高质量密集语义/实例标注，强调真实扫描的非均匀采样、测量伪影、反射和细粒度几何细节，并提供严格标注质控流程及多模态表示生成脚本，支持3D场景理解、施工分析和多模态场景分析。

Abstract: We present the Rohbau3D-Semantics dataset, which adds dense semantic and instance-level labels to laser scans of the previously released Rohbau3D domain data from shell construction sites. The dataset comprises 504 point cloud scenes from 14 survey sites, each captured with a terrestrial laser scanner and stored in a scanner-centered local coordinate system. Unlike point clouds sampled from reconstructed meshes or synthetic building models, the data preserve the characteristic properties of real laser scanning, including non-uniform sampling, measurement artifacts, reflections, and fine geometric detail. Combined with manual labels, expert oversight during pointwise annotation, and full-scene review, this yields a high-quality benchmark for semantic scene understanding in complex built environments. The dataset includes 17 class labels of building components and one background class, along with instance annotations for object-level analysis. In addition to the 3D point clouds, the release also provides scripts for deriving geometrically aligned panoramic and cube-map representations of color, LiDAR reflection intensity, depth, surface normals, semantic labels, and instance labels from the 3D arrays. We describe the acquisition pipeline, annotation protocol, quality-control process, and panorama-generation procedure, and provide benchmark splits and baseline experiments for semantic segmentation. By combining precise terrestrial laser scans, dense annotations, and a wide variety of shell-construction scenes, Rohbau3D-Semantics bridges the gap among synthetic indoor datasets, mesh-derived benchmarks, and smaller construction-specific resources. The dataset is intended to support research on 3D scene understanding, construction-site analytics, and multimodal scene analysis in the built environment. Access and download the full dataset via https://github.com/RauchLukas/rohbau3d

[65] Annotator-Free Object Detection via Foundation Model Collaboration cs.CVPDF

Yasiru Ranasinghe, Vishal M. Patel

TL;DR: 论文提出一种无需目标数据集人工标注的目标检测框架，利用预训练基础模型（分割与视觉语言模型）协作生成分割提案和语义标签，并通过自精炼训练循环迭代提升轻量开放集检测器；还引入基于地标的聚类机制摊销LVLM标注以降低成本。方法可适应未见类别和域，无需人工标注且兼容现有检测架构。

Details

Motivation: 目标检测受高标注成本和监督模型闭集假设限制，难以扩展到新类别；论文旨在消除目标数据集人工标注并降低对昂贵LVLM查询的依赖。

Result: 在COCO、LVIS、ODinW-35和Roboflow-100上评估，零样本设置下COCO达到60.8 AP、LVIS达到47.4 AP，较SOTA开放词汇和长尾检测方法有显著提升。

Insight: 创新点在于基础模型协作的模块化无标注检测流程、自精炼训练循环，以及基于地标的聚类机制以摊销标签并提升一致性；客观看，将分割提案、多模态语义推理与开放集检测器迭代结合，形成可扩展的无人工标注适配方案。

Abstract: Object detection research remains constrained by high annotation costs and the closed-set assumptions of supervised models, limiting scalability to novel categories. We present a target-dataset annotation-free object detection framework that eliminates the need for human labeling on the target dataset by leveraging the complementary strengths of pretrained foundation models, including segmentation and vision-language systems, combined with a self-refining training loop. To further reduce reliance on expensive LVLM queries, we introduce a landmark-based clustering mechanism that amortizes labeling across image clusters, improving label consistency while lowering computational cost. Our method autonomously generates segmentation proposals, assigns semantic labels through multimodal reasoning, and iteratively improves both localization and classification using a lightweight open-set detector. This modular pipeline enables adaptation to completely unseen object classes and domains without any human-provided target-dataset annotations, while maintaining compatibility with existing detection architectures. Evaluations on COCO, LVIS, ODinW-35, and Roboflow-100 demonstrate substantial performance gains over state-of-the-art open-vocabulary and long-tail detection methods, achieving 60.8 AP on COCO and 47.4 AP on LVIS under zero-shot settings. Beyond strong benchmark results, our approach provides a scalable foundation for real-world detection systems in unknown environments, where annotation is infeasible.

[66] NextGAN: Towards Simple and Scalable GAN Training for Image and Video Generation cs.CV | cs.AIPDF

Xinhao Lou, Dongye Xie, Peng Sun, Chenyi Zhuang, Kaicheng Yu

TL;DR: 论文提出 NEXTGAN，一种简单且可扩展的对抗式少步生成训练方法。它通过让标量 critic 的输入梯度回归到冻结表征空间中参考特征与生成特征之间的位移来训练 critic，并保留标准势函数式 GAN 生成器更新。该方法可适配预训练图像和视频生成器，在 ImageNet 256×256 和文本到视频生成上取得较强的少步生成结果。

Details

Motivation: 传统对抗损失只监督 critic 的数值，而没有直接规定用于更新生成器的梯度。论文希望设计更直接、简单且可扩展的对抗训练目标，使预训练图像和视频生成器支持一步或少步采样，同时避免辅助损失、在线教师分数和轨迹依赖。

Result: 在 ImageNet 256×256 上，NEXTGAN 使用 pixel-space JiT-B 实现一步 FID 2.87，使用 latent-space DiT-XL/1 实现一步 FID 1.54，其对抗形式进一步达到 1.36。在文本到视频生成中，单个 Wan2.1-1.3B 模型支持一、二、四步采样，并在 1.3B 对比方法中于一步和两步取得最高 VBench 总分；两步得分 83.13，分别超过 rCM 和 AnyFlow 1.25 和 0.47 分。

Insight: 创新点在于将 critic 训练转化为对其输入梯度的回归，使梯度匹配冻结表征空间中独立配对的参考特征与生成特征位移，并在固定生成器时等价于带梯度能量惩罚的对抗目标。客观来看，该方法把监督信号从 critic 值扩展到更直接的梯度/位移信息，且无需辅助损失、在线教师分数或轨迹，具备较好的简单性和对大模型、图像与视频任务的扩展性。

Abstract: Adversarial training enables one- and few-step image and video generation, but conventional adversarial losses supervise critic values without directly specifying the gradients used to update the generator. We present NEXTGAN, which trains a scalar critic by regressing its input gradient onto displacements between independently paired reference and generated features at interpolated locations in frozen representation spaces. For a fixed generator, this regression is equivalent to an adversarial objective with a gradient-energy penalty, while the generator retains the standard potential-based GAN update. The same objective adapts pre-trained image and video generators with up to 14B parameters, without auxiliary losses, online teacher scores, or trajectories. On ImageNet $256 \times 256$, NEXTGAN achieves one-step FIDs of 2.87 with pixel-space JiT-B and 1.54 with latent-space DiT-XL/1; its adversarial form further reaches 1.36. For text-to-video generation, a single trained Wan2.1-1.3B model supports one-, two-, and four-step sampling, achieving the highest VBench total scores among the compared 1.3B methods at one and two steps. At two steps, NEXTGAN scores 83.13, exceeding rCM and AnyFlow by 1.25 and 0.47 points, respectively. Code is available at https://github.com/LINs-lab/NextGAN.git.

[67] MonoLaDN: Language-Guided DeNoising for Monocular 3D Object Detection cs.CV | cs.AIPDF

Kiet Dang Vu, Cao Le Minh Khoa, Nguyen Viet Tri Pham, An Nguyen Gia, Tam Le Thuc Ho

TL;DR: 论文提出MonoLaDN，一种语言引导去噪框架：训练时利用冻结VLM从图像和真值框生成关于遮挡、深度、尺寸和角度的结构化几何推理，将其编码为查询并加噪，训练单目3D检测器重构干净推理，从而把VLM几何先验内化到纯视觉检测器中；推理时丢弃VLM推理路径，不增加参数和延迟。

Details

Motivation: 单目3D检测需从单视图推断深度，本质病态；现有Transformer检测器仅从图像特征和3D框标注学习几何，缺乏VLM从大规模预训练中获得的高层语义与几何先验，而直接部署VLM做检测又推理成本过高。

Result: 在KITTI 3D和nuScenes上，MonoLaDN一致优于当前最先进检测器，并对新域具有较好的泛化能力；摘要未给出具体数值指标。

Insight: 创新点在于将VLM的结构化几何推理作为训练期监督，通过“加噪—重构”去噪目标迫使检测器的深度引导特征内化VLM先验，同时保持推理阶段纯视觉、零额外参数和延迟。客观来看，该工作把重型VLM知识以训练期蒸馏/去噪方式注入轻量检测器，兼顾性能与部署效率，具有较强实用价值。

Abstract: Monocular 3D object detection offers a low-cost route to 3D perception, but it remains fundamentally ill-posed, since depth must be inferred from a single view. Recent transformer-based detectors learn geometry purely from image features and 3D box annotations, lacking the high-level semantic priors that vision-language models (VLMs) acquire from large-scale pretraining. VLMs offer such priors, but deploying one directly as a detector is impractical due to its heavy inference cost. We propose MonoLaDN, a language-guided denoising framework that transfers the geometric reasoning of a VLM into a monocular 3D detector while keeping the deployed model purely vision-based. During training, a frozen VLM produces structured reasoning about each object’s occlusion, depth, dimension, and angle from the image and its ground-truth boxes. We encode this reasoning into queries, perturb them with noise, and train the detector to reconstruct the clean reasoning. Reconstructing the corrupted reasoning forces the detector’s depth-guided features to internalize the VLM’s geometric priors, so this knowledge remains available even after the reasoning path is discarded at inference, adding no latency or parameters to the deployed model. Experiments on KITTI 3D and nuScenes show that MonoLaDN consistently outperforms state-of-the-art detectors and generalizes well to new domains.

[68] Beyond Scene Shortcuts: Action-Centric Video-Text Alignment for Fine-Grained Action Understanding cs.CVPDF

Zhiliang Zhu, Jiazheng Huang, Dawei Zhao

TL;DR: 论文关注细粒度视频动作理解中模型依赖静态场景和显著物体而非真实运动来生成动作描述的问题，并将其定义为环境 grounded 动作幻觉（EGAH）。作者提出以动作为中心的视频-文本对齐框架 STARR，通过动作-环境解耦和基于 GRPO 的注意力一致性奖励来打破场景-动作捷径，并构建 FineActionR&R 数据。基于 Qwen2.5-VL-7B 的 FineActionVLM-R1 在 MotionBench 和 HAU-MCQ-Bench 上取得提升。

Details

Motivation: 细粒度动作理解要求视觉语言模型在时空证据中理解智能体做什么、如何做以及为何做，但更丰富的监督可能适得其反，使模型基于静态场景和显著物体生成详细动作描述而非关注运动，即 Environment-Grounded Action Hallucination（EGAH）。论文旨在缓解该失败模式，增强动作聚焦的 grounding。

Result: 基于 Qwen2.5-VL-7B 构建的 FineActionVLM-R1 在 MotionBench 上平均准确率较基座模型提升 3.01 个百分点，并在 HAU-MCQ-Bench 上达到 91.50%。定性比较显示其细粒度动作描述更忠实，环境诱发的幻觉更少。

Insight: 论文提出 EGAH 这一场景捷径导致的动作幻觉问题，并通过 Action-Environment Dissociation 打破场景-动作捷径，在 GRPO 中引入 Attention Consistency Reward 强化动作聚焦 grounding，同时构建融合时序局部描述与层次/时序因果推理的 FineActionR&R 数据。客观看，其创新在于以动作中心对齐和强化学习注意力约束抑制环境捷径，并结合针对性数据构造提升细粒度动作理解。

Abstract: Fine-grained action understanding requires vision-language models to ground what an agent does, how it acts, and why it acts in spatio-temporal evidence. However, richer supervision can backfire: fine-tuned models may generate detailed action descriptions grounded in static scenes and salient objects rather than motion, a failure we call Environment-Grounded Action Hallucination (EGAH). To mitigate EGAH, we propose Staged Attention Rectification and Reinforcement (STARR), an action-centric video-text alignment framework that breaks scene-action shortcuts through Action-Environment Dissociation and reinforces action-focused grounding with an Attention Consistency Reward within Group Relative Policy Optimization. We further construct FineActionR&R, combining temporally localized captions with hierarchical and temporal-causal reasoning data. Built on Qwen2.5-VL-7B, FineActionVLM-R1 improves average MotionBench accuracy by 3.01 percentage points over its base model and achieves 91.50% on HAU-MCQ-Bench. Qualitative comparisons further indicate more faithful fine-grained action descriptions with fewer environment-induced hallucinations.

[69] Prism: Bidirectional Self-Teaching for Unified Multimodal Models cs.CV | cs.AIPDF

Peng Zhang, Qian Chen, Xianyin Zhang, Yimin Tian, Ruite Fei

TL;DR: 论文提出 Prism，一种面向统一多模态模型（UMM）的双向自教学后训练方法。它将模型自生成的交错轨迹解耦为文本链和图像链，评估两条链的可靠性，并据此构造双向训练目标，从而同时提升视觉理解与生成能力。

Details

Motivation: 现有 UMM 后训练方法要么把理解能力当作固定教师来提升生成，要么依赖真实标注轨迹同时监督理解和生成，存在监督方向单一或依赖外部标注的问题。论文希望利用模型自生成的交错轨迹实现理解与生成的双向协同提升。

Result: 在 SenseNova-U1 上，生成侧 GenEval 从 85.8 提升到 89.2（+3.4），DPG 从 86.1 提升到 87.3（+1.2）；理解侧 MME 总分从 2282.6 提升到 2311.6，MMVP 从 58.0 提升到 62.0。在 BAGEL 上也观察到一致提升。

Insight: 创新点在于像棱镜分光一样，将自生成交错轨迹解耦为文本链和图像链，并根据可靠性状态构造双向自训练目标，且目标构造不依赖外部验证器或外部标注的纠正轨迹。客观来看，其价值在于把可靠性引导的自训练机制用于统一多模态模型，实现理解与生成的协同改进。

Abstract: Unified Multimodal Models (UMMs) aim to support visual understanding and generation within a single model. Existing post-training methods either use understanding as a fixed teacher to improve generation, or rely on ground-truth trajectories to supervise both understanding and generation. Prism instead derives bidirectional supervision from self-generated interleaved rollouts, constructing training targets according to the reliability of their text and image chains. We propose Prism, a bidirectional self-teaching post-training method for UMMs. Like a prism decomposing mixed light, Prism decouples a coupled interleaved trajectory into a text chain and an image chain, assesses the reliability of each chain, and constructs a corresponding training target from their reliability states. Experiments on both SenseNova-U1 and BAGEL show that Prism improves both understanding and generation. On the generation side, GenEval improves from 85.8 to 89.2 (+3.4), and DPG improves from 86.1 to 87.3 (+1.2) on SenseNova-U1. On the understanding side, the total MME score improves from 2282.6 to 2311.6, and MMVP improves from 58.0 to 62.0. Consistent improvements are observed on BAGEL. These results highlight that Prism enables coordinated improvements in generation and understanding by constructing reliability-guided self-training targets from self-generated trajectories, without relying on an external verifier or externally annotated corrected trajectories during target construction.

[70] Traffic-VL: Reinforced Vision-Language Models for Generalizable End-to-End Traffic Signal Control cs.CV | cs.AIPDF

Xingchen Zou, Maonan Wang, Kemou Jiang, Jie Li, Yiqi Chen

TL;DR: 论文提出SynTra-V和Traffic-VL，用于弥合交通信号控制中的视觉域差距与感知到决策差距。SynTra-V将物理交通仿真转换为逼真多视角监控图像并进行语义审核；Traffic-VL是一个80亿参数的视觉语言模型，可直接将多视角图像和自然语言相位描述映射为信号决策，无需中间结构化状态，并结合离线强化学习与闭环在线策略优化。

Details

Motivation: 现有交通信号控制依赖专用感知流程生成的结构化状态，视觉方法又常使用理想化仿真图像并先转换为中间表示，难以直接利用路侧视觉观测并实现端到端控制。

Result: 实验表明Traffic-VL与结构化状态的Traffic-R1相当，并优于基于图像的控制器和通用视觉语言模型（包括Gemini 3 Pro）；在未见拓扑、退化感知条件、真实监控视频及十路口三周部署中展现出较强的视觉泛化和受限在线控制效果。

Insight: 创新点在于构建带语义审核的逼真多视角交通仿真数据SynTra-V，并让VLM直接从图像和语言相位描述输出控制决策；同时采用离线强化学习适配视觉编码器、在线阶段冻结编码器进行闭环逐步轨迹策略优化，提升视觉泛化与部署可行性。

Abstract: Existing traffic signal control methods typically rely on structured traffic states produced by dedicated perception pipelines. Emerging visual approaches often use idealized simulator images and convert them into intermediate representations before control, limiting their ability to directly exploit roadside visual observations. To bridge both the visual-domain and perception-to-decision gaps, we introduce SynTra-V and Traffic-VL. SynTra-V converts physics-based traffic simulations into photorealistic multi-view surveillance imagery across diverse road topologies and sensing conditions, and automatic semantic auditing checks that control-critical geometry, vehicle movements, and lane semantics match the simulator state. Traffic-VL is an eight-billion-parameter vision-language model that directly maps synchronized multi-view images and natural-language phase descriptions to signal decisions without intermediate structured states. Building on Traffic-R1’s two-stage reinforced reasoning framework, Traffic-VL adapts its vision encoder via offline reinforcement learning on SynTra-V and freezes it during online visual stepwise trajectory policy optimization in closed-loop simulation. Experiments show that Traffic-VL matches structured-state Traffic-R1 while outperforming image-based controllers and general-purpose vision-language models, including Gemini 3 Pro. Evaluations on unseen topologies, degraded sensing conditions, real surveillance footage, and a three-week deployment across ten intersections further demonstrate robust visual generalization and effective constrained online control.

[71] FlashRelight: Portrait Video Relighting with Dynamic Lighting cs.CVPDF

Jiahui Sheng, Le Li, Limin Lin, Jiahao Li, Wansen Feng

TL;DR: 论文提出 FlashRelight，一个用于动态光照下人像视频重光照的两阶段框架。该方法先通过去光照抑制源视频中的原始光照，得到光照中性的人像视频，再合成目标动态光照。框架支持环境贴图序列、动态光照文本和光照参考视频三种控制方式，并用静态 OLAT 数据与生成的动态光照人像视频进行低成本混合训练。

Details

Motivation: 人像视频重光照需保持真实感与时序稳定，而在动态光照下，光照方向、强度和颜色随时间变化，使任务更具挑战性。论文旨在实现灵活、可控且时序一致的动态人像视频重光照。

Result: 摘要称大量实验表明 FlashRelight 能实现时序连贯、真实感强的动态重光照，并支持跨视频光照迁移；但未提及具体 benchmark、定量指标或 SOTA 对比。

Insight: 主要创新在于将源光照抑制与目标动态光照合成解耦的两阶段设计，并结合环境贴图、文本和参考视频三类光照控制模态；同时采用静态 OLAT 与生成动态光照视频的混合训练策略，降低动态光照数据获取成本。

Abstract: Portrait video relighting aims to modify the illumination of a portrait video while preserving photorealism and temporal stability. The task becomes particularly challenging under dynamic lighting, where illumination direction, intensity, and color vary over time. To tackle this problem, we introduce FlashRelight, a novel two-stage framework for dynamic portrait video relighting. FlashRelight first suppresses the source illumination through delighting, and then synthesizes the target dynamic lighting on an illumination-neutral portrait video. FlashRelight supports three complementary control modalities: envmap sequences for explicit frame-level lighting control, dynamic lighting text for semantic illumination editing, and lighting reference videos for visual lighting transfer. Training follows a low-cost hybrid strategy that combines static one-light-at-a-time (OLAT) data with generated dynamic-lighting portrait videos, covering both controlled illumination variation and realistic portrait motion. Extensive experiments show that FlashRelight achieves temporally coherent and photorealistic dynamic relighting while enabling flexible cross-video lighting transfer. Code is available at https://github.com/vivoCameraResearch/FlashRelight

[72] NanoKV: High-Ratio KV Cache Compression for Autoregressive Video Generation cs.CVPDF

Jiacheng Lu, Haoyi Zhu, Sipei Yi, Enze Xie, Cheng Zhuo

TL;DR: 论文提出 NanoKV，一种在冻结主干的条件下用于自回归视频生成的高倍率 KV 缓存压缩框架。它利用 token 之间和注意力头之间的冗余，把共享内容存入聚类中心，并将头与通道联合的残差编码为低位系数；针对 Key 和 Value 在注意力中的不同作用采用不同的拟合目标，其中 Key 解码器以保留下游模块输出为优化目标，Value 则以量化路径下的忠实重建为目标。还给出可选变体 NanoKV-Fast，它复用已存储系数做混合维度注意力，从而在不改主干的情况下降低计算量。

Details

Motivation: 自回归视频模型需依赖历史 KV 缓存来维持长序列生成的场景一致性，导致 KV 存储与历史注意力计算成为内存和算力瓶颈；过度压缩会损害生成质量，而重建全维 KV 又几乎不减少注意力计算，因此需要兼顾高压缩率与生成质量、并能真正降低计算的方法。

Result: 在三个自回归视频主干上一致改善质量与存储之间的权衡：在 HY-WorldPlay 上实现 26.09× 有效 KV 压缩，约为 KVTC 的两倍，同时 PSNR 提升 1.0 dB；NanoKV-Fast 保持 20.53× 压缩并相对 KVTC 获得约 1.15× 端到端加速。

Insight: 创新点在于生成感知的压缩设计：按 Key 和 Value 在注意力中的角色分别设定拟合目标，其中 Key 以保留下游块输出为导向，Value 以量化路径下重建为导向，并结合跨 token 与跨头冗余的聚类中心加低比特系数表示；此外还利用同一压缩表示支持混合维度注意力，在不改主干的前提下实际减少计算。

Abstract: Autoregressive video models rely on historical key-value (KV) caches to preserve scene consistency over extended rollouts, making KV storage and repeated attention over history a growing memory and computational bottleneck. Aggressive compression can degrade generation quality, while reconstructing full-dimensional KV leaves attention computation largely unchanged. We introduce NanoKV, a frozen-backbone framework for generation-aware high-ratio KV compression. NanoKV exploits redundancy across tokens and attention heads by storing shared content in cluster centers and encoding joint head-channel residuals into low-bit coefficients. Crucially, NanoKV uses different fitting objectives for Keys and Values according to their distinct roles in attention: Key decoders are optimized to preserve downstream block outputs, whereas Value encoders and decoders are fitted for faithful reconstruction through the deployed quantization path. Building on the same compressed representation, the optional NanoKV-Fast variant reuses the stored coefficients for mixed-dimensional attention, reducing computation without modifying the video backbone. Across three autoregressive video backbones, NanoKV consistently improves the quality-storage trade-off. On HY-WorldPlay, NanoKV achieves $26.09\times$ effective KV compression, nearly twice that of KVTC, while improving PSNR by 1.0 dB. NanoKV-Fast retains $20.53\times$ effective compression and achieves an approximately $1.15\times$ end-to-end speedup over KVTC.

[73] Entropogram-guided Visual Realignment Decoding: Capturing the Rhythm of Multimodal Reasoning cs.CV | cs.MMPDF

Jiaqi Deng, Zhe Fu, Yakun Chen, Dichang Zhang, Zonghan Wu

TL;DR: 论文从推理生成动态视角研究多模态大语言模型（MLLM）中的幻觉问题，发现高置信度熵模式和局部熵突变分别对应过早收敛和推理不稳定，据此提出熵感知自省框架，在解码时跟踪熵动态并在可疑状态进行视觉重对齐，从而提升推理准确性和幻觉鲁棒性。

Details

Motivation: MLLM在长链跨模态推理中仍存在幻觉，现有方法多从静态不确定性角度处理，缺乏对解码过程中模态信息整合动态的理解。

Result: 在六个多模态推理基准、九种评估设置下，方法在推理准确性和幻觉鲁棒性上均取得稳健提升。

Insight: 将熵视为反映模态信息整合过程的结构化信号，并识别出两种与幻觉相关的熵模式；据此设计动态自省与视觉重对齐机制，为缓解MLLM幻觉提供了新的生成动态视角。

Abstract: Multimodal large language models (MLLMs) have demonstrated remarkable progress on cross-modal reasoning tasks, yet they continue to suffer from hallucinations in long-chain reasoning processes. In this work, we revisit hallucination in MLLMs from the perspective of reasoning generation dynamics. We argue that entropy is not merely a static measure of uncertainty but a structured signal that reflects how modality information is integrated throughout decoding. Through systematic analysis, we identify two characteristic entropy patterns associated with hallucination behaviors: (1) a generally high-confidence regime corresponds to premature convergence to inappropriate multimodal interactions; and (2) local abrupt entropy transitions are associated with reasoning instability and hallucination onset. Building on these observations, we propose an entropy-aware self-introspective framework for MLLM reasoning. Our method continuously tracks entropy dynamics during decoding and performs targeted visual re-alignment when suspicious hallucination states are detected, encouraging the model to re-examine salient visual evidence at critical reasoning stages. Extensive experiments across six multimodal reasoning benchmarks spanning nine distinct evaluation settings demonstrate that our approach delivers robust improvements in reasoning accuracy and hallucination robustness, while providing new insights into the generation dynamics underlying MLLM failures. The code is available at: https://github.com/jden4524/SelfRetrospectiveDecoding.

Runmin Zhang, Linshan Li, Si-Yuan Cao, Zhu Yu, Lingyu Zhu

TL;DR: 论文提出UGD-FM，一种基于跨模态条件流匹配的无对齐图像引导去噪框架，处理严重目标噪声和大幅度二维跨模态错位共存的挑战。通过渐进流匹配过程使图像恢复和对齐引导聚合相互增强，并引入阶段聚焦训练策略和自适应稀疏引导传播，实现高效少步推理。实验表明仅需1-2步推理即达到SOTA去噪性能，且兼容现代单图像恢复骨干。

Details

Motivation: 现有引导去噪方法假设输入空间对齐或仅处理小基线校正对，无法应对严重噪声与大幅度跨模态错位共存的场景。本文旨在解决这一更挑战的无对齐引导去噪问题。

Result: 实验表明UGD-FM仅需1或2个推理步骤即达到SOTA去噪性能，并与现代单图像恢复骨干兼容。

Insight: 论文创新点包括：将引导去噪建模为渐进流匹配过程，使恢复与对齐相互增强；阶段聚焦且推理一致的训练策略，聚焦任务关键阶段并减轻训练-推理不匹配；自适应稀疏引导传播，通过感知和几何辅助损失正则化实现高效大范围匹配。客观分析认为，跨模态条件流匹配用于无对齐引导去噪具有新颖性，渐进式相互增强和少步推理策略有实际价值。

Abstract: This work presents UGD-FM, a novel unaligned image guided denoising framework based on cross-modal conditional flow matching. Unlike previous approaches that assume spatially aligned inputs or only handle small-baseline rectified pairs, we address a more challenging setting where severe target noise and large-range two-dimensional cross-modal misalignment coexist. Specifically, we formulate guided denoising as a progressive flow matching process, allowing image restoration and aligned guidance aggregation to mutually reinforce each other. To support efficient few-step inference, we introduce a stage-focused and inference-consistent training strategy that focuses on task-critical denoising stages, and mitigates the training-inference mismatch. We further propose adaptive sparse guidance propagation for efficient and reliable large-range matching, regularized by perceptual and geometric auxiliary losses. Experiments demonstrate that UGD-FM achieves state-of-the-art denoising performance with only one or two inference steps, and is compatible with modern single-image restoration backbones.

[75] DreamRealm: Exploring and Materializing Hallucination Worlds for Trustworthy Vision-Language Models cs.CVPDF

Benlei Cui, Chunyang Chai, Yiliang Zhang, Xiaoqian Xia, Chengwen Yao

TL;DR: 论文提出了DreamRealm，一个面向多模态大模型（MLLM）的后训练框架，用于缓解视觉幻觉。该框架先从目标模型中挖掘已观察到以及潜在的幻觉回答，再通过最小限度的图像编辑，让原本错误的回答在编辑后的图像上成为正确答案；随后训练模型将每个回答匹配到其对应的支持图像上。具体而言，数据构造流程RealmForgePipe用于挖掘和实现幻觉，训练目标RealmMatch则在图像-回答组合上构建似然得分矩阵，并结合监督学习与双向对比损失；基于该流程，论文构建了Realized Hallucination Dataset（RealmSet）。

Details

Motivation: 现有方法通过检索、生成或编辑图像构造视觉对比，但这些对比并非围绕模型自身的幻觉构建，因此监督信号难以对准模型真实的感知混淆；另一类方法只在固定图像下修正回答或做偏好学习，忽视了输入图像一侧的监督设计。论文希望像人类纠正误解那样，把幻觉“具象化”为可判别的图像证据。

Result: 以Qwen3.5-9B为基座，DreamRealm在六个视觉感知与幻觉基准上均有提升，其中HallusionBench准确率提升7.26个百分点，MMVP成对准确率提升11.34个百分点；数据构造与训练方法的消融实验进一步验证了有效性。

Insight: 创新点在于把模型自身的（含潜在）幻觉转化为经过最小编辑后“变为正确”的图像-回答对，并提出“幻觉归属（hallucination assignment）”范式，用似然得分矩阵配合双向对比损失强化正确匹配、抑制错误匹配，从图像输入端提供了新的监督信号设计思路。

Abstract: Multimodal large language models (MLLMs) remain prone to visual hallucinations when correct responses depend on subtle visual details. One line of post-training methods constructs visual contrasts through image retrieval, generation, or editing to improve fine-grained perception. However, these contrasts are not built around the model’s own hallucinations, making it difficult to ensure that supervision targets its actual perceptual confusions. Another line suppresses hallucinated outputs through response correction or preference learning under fixed image conditions, but leaves the design of supervision signals on the image-input side underexplored. Inspired by how humans correct misconceptions, we introduce DreamRealm, a post-training framework that explores diverse latent hallucinated responses, minimally edits original images to make them valid, and trains the model to assign each response to its supporting image. Our data construction pipeline, RealmForgePipe, mines observed and latent hallucinations from a target model and materializes them through image editing. Using hallucinations mined from Qwen3.5-9B, we construct the Realized Hallucination Dataset (RealmSet). Our training objective, RealmMatch, constructs a likelihood score matrix across image–response combinations and combines supervised learning with bidirectional contrastive losses to reinforce correct matches and suppress mismatches. Compared with the original Qwen3.5-9B model, DreamRealm improves performance across six visual perception and hallucination benchmarks, including gains of 7.26 percentage points in HallusionBench accuracy and 11.34 percentage points in MMVP pair accuracy. Together with ablation studies on data construction and training methods, these results demonstrate that hallucination assignment is a promising paradigm for improving visual perception and mitigating hallucinations.

[76] CoDecide: Content-Adaptive Video Decoding for High-Throughput Video Analytics cs.CVPDF

Shristy Shah, Thomas B. Moeslund, Kamal Nasrollahi

TL;DR: 论文提出 CoDecide，一种面向视频档案回溯分析的内容自适应解码系统：它利用压缩码流中的运动信息，并蒸馏冻结 DINOv2 教师提供的无标签语义新颖性信号，训练轻量压缩域学生模型来判断哪些帧/区域包含新内容，从而只解码必要帧并重建必要区域。对跳过帧，系统利用运动矢量传播已解码帧的检测结果。

Details

Motivation: 视频档案分析常受解码瓶颈限制；当检测器只运行在部分选定帧上时，解码会取代检测成为主要开销，因此需要只解码出现新内容的帧，而不是对每帧都解码。

Result: 在 VIRAT、MEVA 和 ShanghaiTech 上采用 leave-one-dataset-out 协议评估，CoDecide 在相同解码预算下比已有压缩域选择器恢复更多运动目标检测，甚至优于直接在目标相机上训练的竞争者，并且相比逐帧解码检测流水线快 15 到 24 倍。

Insight: 核心创新在于用冻结 DINOv2 教师提供的无标签语义新颖性信号监督轻量压缩域学生，将“有运动”与“语义上新出现”区分开，并用运动矢量把已解码帧检测传播到跳过帧；该设计可泛化到新相机部署而无需逐相机重训。

Abstract: Analytics on video archives are bottlenecked by decoding. In a dense pipeline that runs a detector on every frame, decoding is a small fraction of the runtime, but once the detector is restricted to selected frames, decoding takes over as the main cost. We introduce CoDecide, a retrospective analytics system that only decodes the frames where new content appears, using signals from the compressed bitstream to find them. Motion tells us something is moving, but not whether it is new to the scene or has been there the whole time. To make that distinction, we supervise a lightweight compressed-domain student on a label-free novelty signal from a frozen DINOv2 teacher, which fires on semantically new regions rather than on any motion. Consequently, the student model generalizes to new camera deployments without requiring per-camera retraining. At inference, it decides which frames to decode and which regions to reconstruct. Detections from the decoded frames are propagated to the skipped frames using motion vectors. We evaluate CoDecide with a leave-one-dataset-out protocol on VIRAT, MEVA, and ShanghaiTech. Against a dense per-frame detector, CoDecide recovers more moving-object detections than prior compressed-domain selectors at an equivalent decoding budget, outperforming even a competitor trained directly on the target camera. CoDecide runs 15 to 24x faster than a pipeline that decodes and detects every frame. Code is available at https://github.com/shristy51/CoDecide.

[77] EmoReward: Self-Evolving Reward Calibration for Emotional Video Generation RL cs.CV | cs.AIPDF

Mingzhe Zheng, Yue Wu, Jing Tan, Chang Zou, Dengyang Jiang

TL;DR: 论文提出 EmoReward，一种面向情感视频生成强化学习的自演化奖励校准机制。它从情感提示中规划具体 grounding cues，用多种异构信号原语对生成视频打分，并通过 Reflective Reward Gating 在样本级校准不同奖励的融合权重，再用 RewardMemory 根据上一轮训练中各类情感场景的 rollout 结果估计各原语可靠性，从而在下一轮训练中调整多奖励权重且不更新奖励模型。

Details

Motivation: GRPO 等 RL 后训练在代码、数学等可验证领域有效，但视频质量尤其是情感表达的评价高度主观、没有唯一正确答案。现有固定视觉质量、运动质量和文本对齐奖励难以捕捉情感意图，易引发 reward hacking；RLHF 又难扩展且受个体审美差异影响，难以校准叙事语境下面部、身体动作和色彩线索。

Result: 在 HunyuanVideo 1.5 上，EmoReward 在多种 RL 算法下均取得提升，并在 SAGE-GRPO 下与其他 agentic reward 方法进行比较；消融实验以及在 SD 3.5 和 FLUX 上的迁移实验进一步支持其自适应奖励校准对视觉生成的有效性。

Insight: 创新点在于将多奖励权重分配转化为可自演化的可靠性校准问题：利用实际训练轨迹中的奖励证据估计不同情感原语的可信度，并通过实例级门控处理阶段、人脸置信度、冲突和细微性等因素。客观来看，其可借鉴之处是用 RewardMemory 跨训练轮校准融合权重而无需更新奖励模型，以及面向情感视频生成的细粒度证据规划与动态多奖励融合。

Abstract: Group Relative Policy Optimization (GRPO) has advanced generative model post-training in verifiable domains such as code and mathematics, but evaluating video quality, especially emotional expression, often depends on subjective aesthetics and has no uniquely correct target. Existing approaches typically rely on fixed rewards for visual quality, motion quality, and text alignment that poorly capture emotional intent and can invite reward hacking, while RLHF is difficult to scale and subject to individual aesthetic differences that complicate the calibration of facial, body motion, and chromatic cues across narrative contexts. We introduce EmoReward, a new self-evolving training mechanism that distills evidence reliability from reward traces collected during an actual training run to adaptively guide multi-reward weight allocation in subsequent training by determining which emotional evidence to trust and how much to weight it. EmoReward first plans specific grounding cues from the emotional prompt to guide evaluation, then uses heterogeneous signal primitives to score the generated video during the act stage. Reflective Reward Gating calibrates their contributions through instance-level gates for phase, face confidence, conflict, and subtlety, then renormalizes weights over the active set to produce the sample’s fused reward. RewardMemory then estimates each primitive’s reliability from rollout scores and outcomes within each emotional regime in the finished training run, then fixes bounded multipliers that calibrate fusion weights in the next training run without updating reward models. Experiments on HunyuanVideo 1.5 show improvements across multiple RL algorithms and compare EmoReward with other agentic reward methods under SAGE-GRPO. Ablations and transfer experiments on SD 3.5 and FLUX further support adaptive reward calibration for visual generation.

[78] Spatially and Compositionally Controlled Multimodal Glioblastoma MRI Synthesis with Latent Diffusion cs.CVPDF

Steven Glass, Max Van Puyvelde, Marija Pizurica, Mia Meyer, Olivier Gevaert

TL;DR: 论文提出一个两阶段潜在扩散模型框架，用于生成空间位置和子区域组成显式可控的2D胶质母细胞瘤MRI图像。第一阶段用分割LDM生成符合指定位置、范围、轴位和组成的肿瘤分割图，第二阶段用模态条件LDM结合ControlNet生成T1、T1Gd、T2和FLAIR图像。实验表明生成分割精度较高，ControlNet可提升多模态图像生成质量，专家评分接近真实图像。

Details

Motivation: 胶质母细胞瘤在不同MRI模态下形态和组成高度异质，临床数据集常存在病理类型不平衡问题，论文旨在实现可显式控制肿瘤空间分布与组成的多模态MRI合成，以缓解数据不平衡并支持可控数据增强。

Result: 在留出测试集上，生成分割的平均质心误差为0.842像素，子区域比例误差为0.795个百分点；ControlNet在四种模态上均改善FID，将汇总FID从42.59降至36.12；专家阅读研究平均质量评分为真实图像6.60、ControlNet生成图像6.83、完全合成图像6.77。

Insight: 创新点在于将胶质母细胞瘤MRI合成分解为可控分割图生成与模态条件图像生成两阶段，并引入位置、空间范围、轴位和子区域组成等显式控制条件，同时利用ControlNet增强生成图像与目标分割的一致性。客观来看，其可解释的空间与组成控制及专家评估设计对医学图像可控合成有借鉴意义。

Abstract: Glioblastoma exhibits substantial heterogeneity in tumour morphology and composition across MRI modalities, often making available clinical datasets highly unbalanced in terms of patient pathologies. We propose a two-stage latent diffusion model (LDM) framework for the generation of explicitly controllable 2D glioblastoma MRI scans. First, a segmentation LDM generates tumour maps conditioned on location, spatial extent, axial position, and subregion composition. Second, a modality-conditioned LDM with ControlNet generates T1, T1Gd, T2, and FLAIR images adhering to the requested segmentation. On a held-out test set, generated segmentations achieved a mean centroid error of 0.842 pixels and a subregion proportion error of 0.795 percentage points. ControlNet improved Fréchet inception distance (FID) across all four modalities, reducing pooled FID from 42.59 to 36.12. An expert reader study yielded mean quality ratings of 6.60 for real images, 6.83 for ControlNet-generated images, and 6.77 for fully synthetic images.

[79] BudgetSplat: Budget-Controllable Generalizable Gaussian Splatting cs.CVPDF

Yuxi Hu, Jun Zhang, Wei Mao, Yuchen Rao, Friedrich Fraundorfer

TL;DR: 论文提出 BudgetSplat，一个可控制预算的前馈 3D Gaussian Splatting 框架。它根据用户指定的保留比例自适应生成最终高斯表示，并通过重要性估计、空间覆盖和跨视角冗余联合建模来压缩冗余高斯基元。单个训练模型可在不同输入视角数和多个测试时预算下工作，无需按预算重训练或逐场景压缩。

Details

Motivation: 现有像素对齐的前馈高斯泼溅方法容易生成大量冗余高斯基元，导致表示大小和渲染成本随图像分辨率与输入视角数快速增加，缺少可按用户预算控制最终表示规模的能力。

Result: 在室内重建和跨数据集泛化实验上，方法在不同输入视角数和保留比例下表现出一致的质量—大小权衡，在保持渲染质量的同时显著降低最终表示存储和渲染成本；摘要未给出具体数值或明确 SOTA 声明。

Insight: 创新点在于将高斯压缩建模为重要性估计、空间覆盖与跨视角冗余的联合问题，并提出 Intra-Cross Mask Supervision、预算条件特征调制和 Coverage-Aware Budget Selection，使前馈模型具备预算可控且无需逐预算重训练的能力。

Abstract: Feed-forward 3D Gaussian Splatting enables efficient generalizable novel view synthesis, but existing pixel-aligned methods often generate highly redundant Gaussian primitives, causing the final representation size and rendering cost to grow rapidly with image resolution and the number of input views. We propose BudgetSplat, a budget-controllable feed-forward framework that adapts the final Gaussian representation to a user-specified preservation ratio. Rather than treating compaction as simple importance ranking, we formulate it as a joint problem of importance estimation, spatial coverage, and cross-view redundancy. A single trained model supports varying numbers of context views and multiple test-time budgets without per-budget retraining or per-scene compression. To realize this formulation, Intra-Cross Mask Supervision provides structure-, coverage-, and redundancy-aware guidance for learning Gaussian importance, while budget-conditioned feature modulation adapts Gaussian opacity and attributes to the requested preservation ratio. Coverage-Aware Budget Selection further combines patch-wise pre-selection, view-wise opacity ranking, and 3D redundancy suppression to construct compact yet spatially distributed Gaussian representations. Experiments on indoor reconstruction and cross-dataset generalization demonstrate consistent quality–size trade-offs across different input view counts and preservation ratios, substantially reducing final representation storage and rendering cost while preserving rendering quality.

[80] SpikePose: Lightweight Video Human Pose Estimation cs.CV | cs.NEPDF

Yunhao Huang, Kaushik Bhargav Sivangi, Tianyi Yu, Fani Deligianni

TL;DR: 论文提出SpikePose，一种轻量级视频人体姿态估计器，将帧重置单步脉冲编码器与运动对齐热图记忆（MAM）结合。编码器提供逐帧空间证据，MAM估计关节位移、对齐历史热图并与当前预测自适应融合，从而在脉冲骨干外完成紧凑的时序推理。方法在MPII和NTU60 Cross-Subject固定子集上评估空间精度、运动误差和估算算术能耗。

Details

Motivation: 视频人体姿态估计需兼顾定位精度、运动质量和能耗，而脉冲神经网络在大型非事件驱动视频任务上与经典ANN仍有差距。论文希望在保持轻量和低能耗的同时，引入有效时序建模来提升脉冲姿态估计的实用性。

Result: 在MPII和NTU60 Cross-Subject固定子集上，与架构匹配的ANN+MAM相比，SpikePose估算算术能耗降低91.4%，PCK_HB(0.5)仅下降1.12点。加入MAM使速度误差和加速度误差分别降低28.13%和31.46%，定位略优，仅增加0.003M参数和每帧0.0023解析GFLOPs。

Insight: 创新点在于将时序推理限制在脉冲骨干外的轻量MAM模块中，通过运动对齐热图记忆实现历史信息自适应融合，并以帧重置单步脉冲编码器提供空间证据。客观看，其可借鉴之处是用外置轻量时序模块桥接SNN与ANN在大规模非事件视频任务上的性能差距，同时以很小参数和计算代价显著改善运动质量并大幅降低估算能耗。

Abstract: Efficient video human pose estimation must balance localisation accuracy, motion quality, and energy cost. We present SpikePose, a lightweight video pose estimator that pairs a frame-reset, single-step spiking encoder with motion-aligned heatmap memory (MAM). The encoder supplies per-frame spatial evidence, while MAM estimates joint displacement, aligns historical heatmaps, and adaptively fuses them with current predictions. By confining temporal reasoning to a compact, lightweight module outside the spiking backbone, SpikePose helps bridge the gap between spiking and classical Artificial Neural Networks (ANN) on large-scale, non-event-driven video tasks. We evaluate spatial accuracy, motion errors, and estimated arithmetic energy on MPII and a fixed NTU60 Cross-Subject subset. Against the architecture-matched ANN with MAM, SpikePose cuts estimated arithmetic energy by 91.4% at a cost of just 1.12 points in head-bone-normalised PCK at 0.5 threshold (PCK_HB(0.5)). Within SpikePose, adding MAM reduces velocity and acceleration errors by 28.13% and 31.46% while slightly improving localisation, at a cost of only 0.003M parameters and 0.0023 analytical GFLOPs per frame. Code will be released on GitHub upon acceptance.

[81] Learning to Track Cells from Sparse Annotations in 3D Time-Lapse Microscopy cs.CVPDF

Yusuke Uchida

TL;DR: 论文面向三维延时显微图像中仅有稀疏细胞标注的细胞追踪与谱系重建问题，提出一套整合稀疏标注检测、基于图像特征的候选连接校准以及谱系图事件联合选择的流水线。方法先用2.5D与3D检测模型融合并抑制未标注细胞的错误背景监督，再利用稠密光流、学习到的细胞匹配模型及分裂前后特征校准普通连接和分裂概率，最后在由评价指标导出的代理目标下联合选择连接、分裂与节点纳入，并进行间隙填充和时间坐标细化。

Details

Motivation: 从3D延时显微图像重建细胞谱系需要区分细胞运动与分裂并推断时序一致的连接，但仅有部分细胞被标注时，将未标注区域视作背景或错误对应会带来有问题的训练监督。论文旨在解决稀疏标注条件下的可靠检测与谱系连接推断问题。

Result: 在来自2个斑马鱼胚胎的199个视频上进行5折交叉验证，流水线总体得分为0.9762；在候选与校准值固定时，联合选择比允许替换现有分裂并重优化普通连接的顺序方法高0.0105，提升主要来自分裂事件恢复。

Insight: 创新点在于融合2.5D与3D检测以抑制未标注细胞的错误背景监督，并结合稠密光流运动特征、学习式细胞匹配特征和分裂前后特征校准连接/分裂概率；进一步以评价指标导出的代理目标联合选择谱系图事件。客观看，其可借鉴之处是将检测、连接校准和谱系事件选择统一到稀疏标注场景下，并用联合优化替代分步后处理。

Abstract: Reconstructing cell lineages from 3D time-lapse microscopy requires distinguishing cell motion from division and inferring temporally consistent connections. When only a subset of cells is annotated, however, training procedures that treat unannotated regions as background or incorrect correspondences can be problematic. We propose a cell tracking pipeline that integrates detection from sparse annotations, calibration of candidate connections using image features, and joint selection of lineage graph events. For detection, we combine the outputs of 2.5D and 3D models trained to suppress erroneous background supervision at unannotated cells. We then calibrate ordinary link and division probabilities using motion features from dense flow, correspondence features from a learned cell-matching model, and features from models that identify cells immediately before division, dividing parents, and daughters immediately after division. Ordinary links, divisions represented as daughter pairs, and node inclusion are selected jointly under a surrogate objective derived from the evaluation metric, followed by gap filling and temporal coordinate refinement. The pipeline achieves an overall score of 0.9762 in 5-fold cross-validation on 199 videos from 2 zebrafish embryos. With candidates and calibrated values held fixed, joint selection outperforms a sequential method that allows replacement of existing divisions and reoptimization of ordinary links by 0.0105. The improvement arises primarily from division recovery.

[82] Trick-or-Treat: Adversarial Evaluation of Multimodal Medical Reasoning with Synthetic Images cs.CV | cs.AIPDF

Oishi Banerjee, Eujin Lee, Sung Eun Kim, Alexandra N. Willauer, Christine Y. Zhou

TL;DR: 论文提出 Trick-or-Treat，一个利用合成临床照片与真实胸片配对的对抗性多模态医学推理基准，用于暴露前沿模型在跨模态医学诊断中的幻觉行为。该基准为多种疾病构建匹配和不匹配图像对，并通过三个前沿模型的实验揭示模型面对不匹配信息时会忽略冲突并幻觉出与胸片更一致的临床发现。

Details

Motivation: 现有医学数据集很少配对不同影像模态（如胸片与外部临床照片），导致多模态医学推理评估困难，也难以系统检验模型的跨模态幻觉。论文希望构建更具挑战性的对抗评估框架，暴露前沿模型在医学推理中的错误模式。

Result: 在三个前沿模型上，不匹配图像对中模型幻觉出临床显著照片发现的比例为45%–57.5%，而相同照片单独展示时为25%–37.5%。面对不匹配信息时，模型倾向于忽略冲突并幻觉出更匹配胸片的发现以支持统一诊断。

Insight: 创新点在于用合成临床照片可控地改变临床发现，同时保持人物特征和拍摄环境一致，从而减少混杂，并构建共享胸片但照片匹配/不匹配的对抗性医学多模态基准。客观来看，其价值在于将合成图像用于受控对抗评估，系统揭示跨模态冲突下的幻觉与诊断合理化倾向。

Abstract: We used medical image synthesis to create a challenging multimodal evaluation framework that exposes hallucination behaviors in frontier models. Evaluating multimodal reasoning has historically been difficult because existing medical datasets rarely pair different image modalities, such as radiographs and external clinical photographs. To address this gap, we introduce Trick-or-Treat, an adversarial multimodal medical benchmark that pairs real chest radiographs with synthetic clinical photographs to challenge models’ cross-modal reasoning. For each of a wide range of conditions, we created two pairs of images, where the pairs share a single chest radiograph but contain different external photographs. In matched pairs, the chest radiograph and photograph both contain findings suggesting the same disease (e.g., varicella pneumonia suggested by lung opacities and crusted skin lesions seen in chickenpox). In mismatched pairs, the photograph depicts an alternative clinical finding that would not lead to the same unifying diagnosis (e.g. replacing crusted skin lesions with ordinary acne). Synthetic photograph generation allows clinical findings to be varied while the person’s features and photographic setting stay consistent, reducing confounding. In an investigation of three frontier models, we found that models hallucinated clinically significant photograph findings in 45-57.5% of mismatched pairs, up from 25-37.5% for the same photographs shown alone. When faced with mismatched pairs, models tended to ignore mismatches and instead hallucinated findings that better matched the radiograph (e.g. misreading acne as crusted skin lesions to justify diagnosing chickenpox). Trick-or-Treat both demonstrates concerning error pattern in frontier models’ medical reasoning and provides a proof of concept for the use of synthetic images in adversarial multimodal evaluation.

[83] Dynamical State-Space Diffusion for Spatiotemporal Prediction cs.CVPDF

Woojun Lee, Hankyeol Kim, Pilsung Kang

TL;DR: 论文提出Dynamical State-Space Diffusion（DSSD），将扩散过程中的噪声破坏从逐帧独立噪声改为具有时序相关性的结构化噪声。DSSD利用稳定对角状态空间模型的阻尼振荡模态构建可学习时间核，并用平稳协方差的Cholesky因子作为因果、可逆且方差保持的时间算子，使denoiser预测时间相关噪声。实验在科学场和机器人视频数据集上验证其优于共享denoiser的扩散基线。

Details

Motivation: 传统时序扩散预测通常对未来帧独立加噪，导致时序依赖完全由denoiser学习。论文希望让corruption过程本身捕获多时间尺度和振荡相关性，从而减轻denoiser负担并提升时空预测建模能力。

Result: 在科学场和机器人视频数据集上，DSSD优于与其共享denoiser的扩散基线；摘要未给出具体定量数值或SOTA声明，但强调学习到的时间模态会随数据集自适应演化。

Insight: 创新点在于将状态空间模型的阻尼振荡模态引入扩散加噪过程，构造可学习的时间相关噪声算子；并通过平稳协方差Cholesky因子/Kalman innovations形式保证因果性、精确可逆、逐帧噪声方差保持及干净端恒等，同时不改变denoiser架构。

Abstract: Diffusion models for temporal prediction typically corrupt each future frame with independent noise, leaving temporal dependence entirely to the denoiser. We propose Dynamical State-Space Diffusion (DSSD), which learns the temporal structure of the corruption process itself. DSSD builds a temporal kernel from the damped oscillatory modes of a stable diagonal state-space model, whose learned decay rates and frequencies let the corruption capture multiple timescales and oscillatory correlations, while the denoiser architecture remains unchanged. Running this model on a finite clip, however, requires the state before the first frame, which is unobserved; starting from a zero state leaves the early frames with less noise than the diffusion schedule prescribes. We avoid this by using the Cholesky factor of the model’s stationary covariance as the temporal operator. We prove that this operator is causal, exactly invertible, keeps each frame’s noise variance equal to that of standard diffusion, and reduces to the identity at the clean end of the process, and that it is exactly the Kalman innovations form of the state-space model. The denoiser is trained to predict the temporally correlated noise, and sampling removes and reapplies the temporal mixing at each step. On scientific-field and robot-video datasets, DSSD outperforms diffusion baselines that share its denoiser, and the evolution of its learned modes during training shows that the temporal operator adapts to each dataset.

[84] Sekai: Incremental Event Memory for Generalized Reasoning Video Segmentation cs.CVPDF

Zhengtao Liu, Jinrong Xu, Shiu-Hong Kao, Huixin Sun, Yicong Li

TL;DR: 论文关注在线推理视频分割中目标集合随查询相关事件动态变化的问题，提出广义推理视频分割新任务GRVS，并构建因果标注的Isekai基准。作者提出Sekai在线框架，通过事件记忆推理器维护查询相关事件与目标状态，并用记忆引导解码器在当前帧定位有效目标。

Details

Motivation: 现有在线推理视频分割通常假设视频中目标集合固定，忽略了流式推理中新事件发生会导致满足查询的目标集合随时间演变甚至从空集变为一个或多个目标。论文旨在解决目标有效性持续维护与当前帧有效目标定位的问题。

Result: 在自建Isekai基准上，Sekai取得50.33 J&F，较最强基线提升3.2个点，显著优于现有在线推理视频分割方法。

Insight: 创新点在于将目标集合建模为随事件累积而演变的状态，并用结构化事件记忆显式维护查询相关事件和目标有效性，再通过记忆引导解码器实现当前帧定位，为动态目标集合的在线视频推理分割提供了新任务、基准和方法框架。

Abstract: Online reasoning video segmentation (Online RVS) requires a model to segment query-referred objects as a video streams in, using only the frames observed so far. Existing formulations largely assume a fixed target set throughout the video, overlooking a fundamental property of streaming reasoning: as new events occur, the objects that satisfy a query may change over time. To address this, we propose a new task, Generalized Reasoning Video Segmentation (GRVS), where the target set depends on query-relevant events accumulated over the observed video history. As new evidence accumulates, the target set can evolve from empty to one or multiple objects, requiring a model to continuously maintain target validity while localizing the valid objects currently in view. To evaluate GRVS, we build Isekai, a diagnostic benchmark of 60 videos and 197 queries across 11 temporal query families, with 12,569 scoring points whose target sets are causally annotated over time. We further propose Sekai, an online framework with two complementary components: an Event Memory Reasoner (EMR) that maintains query-relevant events and evolving target states in structured memory, and a Memory-Grounded Decoder (MGD) that identifies valid targets from this memory and localizes them in the current frame. Experiments show that Sekai significantly outperforms existing online RVS methods, achieving 50.33 J&F and improving over the strongest baseline by 3.2 points. Code, example annotations, and evaluation scripts are available at https://github.com/zenastangbi/Sekai.

[85] Turns Matter: Advancing Streaming Video RL via Turn-Level Credit cs.CV | cs.CL | cs.LGPDF

Shuai Dong, Kanghui Tian, Linghan Chen, Wenqi Shao, Hongrui Wu

TL;DR: 论文关注流式视频理解中模型需在多轮交互中反复决定“回复”还是“继续观察”的问题，提出 TurnWise，一种基于 turn 级信用分配的 GRPO 框架。该方法将学习信号从轨迹级优势细化到每个流式决策，并通过 turn 均值策略优化平衡回复与等待决策的权重，从而提升流式策略学习效果。

Details

Motivation: 现有将 GRPO 用于流式视频理解的方法主要关注奖励设计，但仍沿用单一轨迹级优势，导致不同 turn 共享信用。这可能在早期成功后强化冗余回复，或在早期失败后抑制后续有用追问，因此需要让学习信号与每个 turn 的流式决策对齐。

Result: 摘要称大量实验表明该方法优于先前用于流式视频理解的强化学习方法，但未给出具体 benchmark、数值或是否达到 SOTA 的定量结果。

Insight: 创新点在于 turn 级信用分配：利用 silence-completed prefix scoring 从每个决策的剩余回报构造组相对优势，分离其与早期奖励的学习信号；同时采用 turn 均值策略优化，在每个 turn 内平均 token loss 再跨 turn 平均，使回复与等待决策不受输出长度影响而等权。该方法无需 learned critic 或额外 rollouts，客观上是将信用分配和损失聚合粒度从轨迹级推进到 turn 级。

Abstract: Streaming video understanding requires models to repeatedly decide whether to respond or continue observing. Recently, Group Relative Policy Optimization (GRPO) has been applied to this setting, with existing approaches emphasizing reward design while retaining a single trajectory-level advantage across turns. Such shared credit can reinforce redundant replies after earlier successes and discourage useful follow-ups after earlier failures. We propose TurnWise, a turn-level GRPO framework that aligns learning with individual streaming decisions. TurnWise addresses the mismatch between trajectory-level feedback and turn-level interaction, promoting informative responses and reducing redundancy. Specifically, turn-level credit assignment uses silence-completed prefix scoring to construct group-relative advantages from each decision’s remaining return, separating its learning signal from earlier rewards. Turn-mean policy optimization averages token losses within each turn and then across turns, giving response and waiting decisions equal weight regardless of output length. These complementary designs improve streaming policy learning without a learned critic or additional rollouts. Extensive experiments show our method outperforms prior RL approaches for streaming video understanding.

[86] Coupled Scene Graph Anticipation: When With-Constraint Recall Hides Structural Inconsistency cs.CV | cs.LGPDF

Zeno Nongmaithem, Anushttha Srivastava, Anshit Sinha, Apoorv Sisodia, Suman Chowdhury

TL;DR: 论文针对场景图预测（SGA）中连续时间模型独立处理每个主体-对象对导致结构不一致的问题，提出CoupledSGA，通过关系交互图和结构一致性损失来耦合关系。同时发现现有With-Constraint评估协议无法捕捉结构一致性改进，并分析了损失函数的梯度行为。在Action Genome上实验显示降低矛盾率但牺牲召回率。

Details

Motivation: 现有SGA模型将每个主体-对象对独立演化，无法建模关系间交互，导致互斥预测（如同时“坐”和“站”）。目的是强制结构一致性。

Result: 在Action Genome上，激活结构一致性损失后，R@20从42.21%降至35.18%，但No-Constraint矛盾率从45.92%降至4.07%。匹配预算下CoupledSGA的R@20为40.48%，低于SceneSayerODE的44.78%。作者声明这不是SOTA，而是测试平台，强调了评估协议的局限性。

Insight: 创新点包括：1) 引入关系交互图耦合ODE中的关系；2) 提出结构一致性损失；3) 揭示With-Constraint评估的局限性并证明其无法反映结构一致性改进；4) 分析损失梯度行为。客观来看，论文指出了评估指标的问题，并提出了更合理的报告方式（No-Constraint结果+矛盾率）。

Abstract: Scene Graph Anticipation (SGA) predicts object relationships in future video frames. Continuous-time SGA models evolve each subject-object pair with an independent Neural ODE, so relationships sharing an entity cannot influence one another during integration, and nothing prevents mutually exclusive predictions, such as “sitting on” and “standing on”, for the same pair. We introduce CoupledSGA, which links such relationships through a Relationship Interaction Graph with graph attention inside the ODE vector field, and adds a Structural Consistency Loss ($\mathcal{L}{SC}$) based on a matrix of mutually exclusive predicates. Developing CoupledSGA revealed a limitation of the evaluation protocol. With-Constraint evaluation keeps one predicate per group for each subject-object pair, so its R@K and mR@K cannot capture the within-group violations targeted by $\mathcal{L}{SC}$: we prove they are unchanged whenever each group’s winning predicate and its score are held fixed. We further show that the gradient of $\mathcal{L}{SC}$ depends on predicate scores rather than ground-truth labels: for two-predicate rules it amplifies the ordering the model already holds, and when all scores in a group are equal it suppresses predicates in more exclusion rules more strongly, explaining why the penalty can lower recall while improving consistency. On Action Genome, activating $\mathcal{L}{SC}$ after two warm-up epochs reduces R@20 from 42.21% to 35.18% while lowering the No-Constraint contradiction rate from 45.92% to 4.07% (matched-budget R@20: 40.48% CoupledSGA vs. 44.78% SceneSayerODE). CoupledSGA is a testbed, not a state-of-the-art claim: With-Constraint recall captures the cost of enforcing structural consistency but not the improvement. We recommend reporting No-Constraint results alongside a contradiction rate for methods that constrain predicate co-activation.

[87] EvoWorld: Towards Out-of-Sight World Evolution in Generative Video Models cs.CV | cs.AIPDF

Yequan Bie, Dengyang Jiang, Zhixuan Chen, Xiaozhuan Liang, Haoxuan Che

TL;DR: 论文关注生成视频模型作为视觉世界模拟器时缺乏因果世界演化推理的问题，提出“视野外因果演化”新任务：模型需渲染事件、推断其对世界实体的后果，并在后续甚至视野外呈现因果一致的结果。作者提出EvoWorld框架，由视觉语言模型充当“世界大脑”维护世界状态并推理状态转移，配合分段自回归视频渲染器实现视觉生成，同时构建EvoWorld-Data和EvoWorld-Bench用于训练与评测。

Details

Motivation: 现有扩散视频模型擅长生成视觉合理的像素，但缺乏显式世界状态维护和事件条件后果推理，容易产生行为静态或因果不一致的输出；论文旨在让视频模型从“渲染世界”走向“推理世界如何演化”。

Result: 实验表明EvoWorld在事件条件因果演化方面显著优于近期模型，并在EvoWorld-Bench上保持较好的视觉质量与一致性；摘要未给出具体定量数值，主要作为该新问题的参考基线。

Insight: 创新点在于提出视野外因果演化任务及配套数据与评测基准，并采用“VLM世界大脑+分段自回归渲染器”的协作框架，将世界状态推理与视觉渲染解耦，为推理引导的自主世界演化提供参考。

Abstract: Generative video models are increasingly used as visual world simulators, yet their primary strength lies in rendering plausible pixels rather than reasoning about how a world should evolve. Coherent world evolution requires more than visual fidelity: when an event occurs, a model must infer its consequences for the entities in the world and then render the resulting state. For example, after a wild beast appears, the model should not only depict the wild beast, but also infer whether a nearby person should flee, take cover, or fight back based on the realized scene. Existing diffusion-based video models lack an explicit mechanism for maintaining world state and reasoning over such event-conditioned consequences, often producing visually plausible but behaviorally static or causally inconsistent outputs. To make this reasoning gap concrete and measurable, we propose a new problem, Out-of-Sight Causal Evolution, which requires a model to render an event, reason about its consequences for the evolving world state, and reveal a causally plausible outcome as the world continues to evolve, even out of sight. As a baseline reference solution, we introduce EvoWorld, a brain-renderer collaboration framework in which a vision-language model serves as the world brain: it observes generated segments, maintains an evolving account of the world, and reasons about plausible state transitions, while a segment-autoregressive video renderer realizes them visually. We further introduce EvoWorld-Data and EvoWorld-Bench for training and evaluating event realization, causal reactions, and revisit consistency. Experiments show that EvoWorld substantially improves event-conditioned causal evolution over recent models while preserving visual quality and consistency, establishing a reference point for reasoning-guided, autonomous world evolution beyond the current field of view.

[88] Route First, See Later: Enabling Vision Agents to Reason Scientifically by Separating Routing Policy from Action Spaces cs.CV | cs.AIPDF

Shubhabrata Mukherjee, Jeffrey Donatelli, Daniela Ushizima

TL;DR: 论文提出ZETA架构，将科学视觉智能体中的“路由策略”（理解用户意图并选择工具）与“动作执行”（调用具体视觉工具）显式解耦：先解析科学意图，再调用描述、检测、计数、分割等专用工具，并把测量视为基于显式几何的确定性操作而非生成式输出。该架构使路由决策与下游执行产物可分别观测和评估，从而区分“意图选择错误（错路由）”与“视觉执行失败”。作者在51条科学意图查询的零样本基准上做了首个控制式路由研究。

Details

Motivation: 现有多工具智能体架构把意图理解与视觉执行混在一起，导致失败时无法判断是推理出错（错路由）还是感知失败（工具执行无效）；这种缺乏可审计性使智能体难以用于高风险的科学研究自动化场景。

Result: 在51条科学意图查询的零样本基准上，显式路由语义能显著提升路由准确率，尤其对语义相邻的动作边界；效果因模型而异。对Qwen 3.8，分割路由准确率从6.2%提升至75.0%，检测路由从60.0%提升至100%。

Insight: 创新点在于把路由策略显式化为可指定的一层，并在固定合法动作接口的前提下首次进行控制式研究，从而将路由正确性与执行正确性拆分为两个独立可评估的失败模式；同时把科学测量建模为基于显式几何的确定性操作，并用执行轨迹让路由决策与下游产物分别可查，为可靠科学视觉智能体提供可审计的基础设施。

Abstract: Scientific image-analysis workflows require agents to select among diverse capabilities like description, detection, counting, and segmentation. Current multi-tool architectures conflate intent interpretation with visual execution, masking whether a failure arises from flawed reasoning (misrouting) or failed perception (invalid tool execution). This lack of auditability renders agents unreliable for high-stakes scientific automation. We introduce ZETA, an extensible architecture that operationalizes the decoupling of routing policy from action execution. ZETA resolves scientific intent before invoking specialized visual tools, treating measurement as a deterministic operation grounded in explicit geometry rather than generative output. This architecture exposes the “black box” of agent decision-making, allowing routing correctness and downstream execution to be evaluated as independent failure modes. To our knowledge, we provide the first controlled study in scientific vision-agent routing that isolates explicit routing-policy specification while holding the legal action interface fixed. Evaluation on a zero-shot benchmark of 51 scientific-intent queries shows that explicit routing semantics can substantially improve routing accuracy, particularly at semantically adjacent action boundaries, although improvements vary across models. For Qwen 3.8, explicit routing-policy specification improves segmentation routing accuracy from 6.2% to 75.0% and detection routing accuracy from 60.0% to 100%. ZETA’s execution traces make routing decisions and downstream artifacts separately observable, providing a basis for distinguishing intent-selection errors from visual execution failures. ZETA establishes the necessary infrastructure to isolate execution errors from interpretive ones, a critical requirement for deploying reliable scientific vision agents.

[89] Benchmarking Fine-Grained Visual Reasoning for Remote Sensing in Urban Management cs.CVPDF

Jiawei Zhang, Rizhao Cai, Dayan Guan

TL;DR: 论文面向城市管理中的遥感细粒度视觉推理，构建了基于无人机图像的细粒度基准 UMBench，覆盖交通监测、城市建设、环境质量和灾害预警等任务。评估发现现有多模态大模型常依赖场景级先验而忽略细粒度视觉证据，因此提出一种无需训练的“证据优先”思维链方法，引导模型检查细粒度视觉证据。实验表明该方法能提升开源和商业 MLLM 在 UMBench 上的表现。

Details

Motivation: 现有 MLLM 评估对城市管理遥感中的细粒度视觉推理关注不足，模型容易依赖场景级先验而忽视关键视觉证据，难以满足实际城市管理任务需求。

Result: 论文构建了 UMBench 基准并用于评测当前 MLLM；实验显示所提 evidence-first chain-of-thought 方法可提升开源和商业 MLLM 在 UMBench 上的性能，但摘要未给出具体数值或 SOTA 结论。

Insight: 创新点在于构建面向城市管理多任务的无人机遥感细粒度视觉推理基准，并提出无需训练的证据优先思维链方法。客观看，其价值在于揭示 MLLM 依赖场景先验的问题，并提供可迁移的推理引导思路。

Abstract: Fine-grained visual reasoning is essential for remote sensing in urban management, yet remains underexplored in the evaluation of multimodal large language models (MLLMs). We introduce UMBench, a fine-grained urban remote sensing benchmark built from unmanned aerial vehicle (UAV) images for multiple urban management tasks such as traffic monitoring, urban construction, environmental quality, and disaster warning. Evaluations on UMBench reveal that current MLLMs often rely on scene-level priors and overlook fine-grained visual evidence. To address this issue, we further propose a novel evidence-first chain-of-thought method that directs models to inspect fine-grained visual evidence in a training-free manner. Experiments show that our method can improve the performance of both open-source and commercial MLLMs on UMBench.

[90] VA-TDM: A Unified View for Improving Few-Step Diffusion Models by Velocity Adversarial Distillation cs.CVPDF

Yihong Luo, Tianyu Pang, Tianyang Hu

TL;DR: 论文提出速度对抗蒸馏（VAD）框架，从速度场角度统一了分数蒸馏、Fisher匹配和矩匹配等方法，并进一步构建VA-TDM，通过对学生采样轨迹的部分去噪过程进行监督，结合过渡感知的在线速度学习和双形式VAD监督，提升少步扩散/流模型的生成质量。

Details

Motivation: 少步扩散蒸馏虽能高效生成，但仅用少量采样步数时质量难以保持；现有基于分数蒸馏的分布匹配方法依赖准确估计中间步的学生分数，而少步采样中间步估计困难且可能与更新分布不一致，因此需要更灵活可靠的监督方式。

Result: 实验表明VA-TDM能有效学习少步生成器，在多种骨干的文本到图像和文本到视频任务上取得四步生成SOTA，并可扩展到33B MiniMax-H3；VA-TDM-H3在人类偏好上明显优于少步Fast-H3，并更接近原始多步H3。

Insight: 创新点在于用学习到的critic比较学生与教师速度场并将分歧转化为学习反馈，统一了多种蒸馏/匹配目标；VA-TDM进一步沿学生轨迹提供部分去噪监督，并结合过渡感知在线速度学习与双形式VAD监督，为少步扩散蒸馏提供了统一且可扩展的新视角。

Abstract: Few-step distillation offers efficient generation with diffusion and flow models, but preserving quality with only a few sampling steps remains challenging. Distribution-matching methods via score distillation provide a strong foundation, yet they rely on accurately tracking the student’s changing distribution. At intermediate generation steps in few-step sampling, student-score estimation can be difficult, and commonly used estimators may describe a different distribution from the one used to update the student. Flow matching allows a more flexible approach to supervision through velocity fields constructed from the student’s sampling process. Building on this perspective, we introduce Velocity Adversarial Distillation (VAD), which uses a learned critic to compare student and teacher fields and turn their disagreement into feedback for learning. VAD unifies score distillation, Fisher-based matching, and moment matching, also explaining existing heuristic MMD updates and delivering new training objectives. We further develop VA-TDM, which follows trajectory distribution matching: providing supervision for partial denoising along the student trajectory. Beyond this, VA-TDM combines transition-aware online velocity learning and dual-form VAD supervision across intermediate student transitions. Extensive results show that VA-TDM is powerful for learning few-step generators, achieving state-of-the-art four-step performance in both text-to-image and text-to-video generation across various backbones and scaling effectively to 33B MiniMax-H3. In particular, our VA-TDM-H3 notably outperforms the few-step Fast-H3 and is closer to the original many-step H3 according to human preference.

[91] Open-HumanScale: An Open Recipe for Scaling Human and Robot Experience for Embodied Foundation Models cs.CVPDF

Juncheng Ma, Jiaxin Li, Jiankai Tu, Bo Liang, Guodong Lv

TL;DR: 论文提出首个全开放配方 Open-HumanScale，构建 5 万小时带手部运动与自然语言标注的自我中心人类视频语料 Open-HumanScale-50K，系统研究纯人类视频预训练并发现预训练时长与下游机器人成功率的经验对数线性缩放规律。基于此训练 HumanWAM-Vanilla，并进一步提出联合人-机器人预训练的视觉目标条件世界-动作模型 HumanWAM-Goal，在仿真与真实机器人上取得强性能，并开源数据、流程、模型与评测套件。

Details

Motivation: 论文关注如何将大规模自我中心人类视频转化为可泛化的机器人操作策略，指出原始数据规模并不足够，仍需严格的数据策展与训练配方。其出发点是填补开放、可复现的规模化人类经验预训练方案的空白。

Result: HumanWAM-Vanilla 经机器人后训练在 RoboTwin 2.0 上达到 Clean2Clean 92.08%、Clean2Random 68.88%，在 RoboDojo 上平均成功率 14.17%、平均进度分 19.00。HumanWAM-Goal 通过人-机器人混合数据缩放，将 RoboTwin 2.0 后训练性能提升至 Clean2Clean 92.70%、Clean2Random 84.60% 的 SOTA 水平，并在真实机器人上展现无需参数更新的涌现具身上下文学习能力。

Insight: 论文宣称的创新点包括首个全开放的 5 万小时自我中心人类视频预训练配方与语料、经验对数线性缩放规律、纯人类预训练的世界-动作模型，以及联合人-机器人预训练的视觉目标条件模型。客观来看，其核心价值在于将大规模数据策展、标注、预训练流程与闭环仿真评估一体化开源，并通过混合数据缩放显著提升随机化场景泛化和真实机器人自适应能力。

Abstract: Egocentric human video offers a scalable source of manipulation priors for robot learning, yet translating raw data scale into generalizable robot policies demands rigorous data curation and training recipes. We introduce Open-HumanScale, the first fully open recipe for scaling egocentric pretraining to 50,000 hours, anchored by Open-HumanScale-50K—a curated egocentric video corpus densely annotated with hand motion and natural language. Leveraging this corpus, we systematically investigate pure human pretraining via closed-loop simulation evaluations and uncover an empirical log-linear scaling law linking pretraining data hours to downstream robot success rates. Grounded in these findings, we develop HumanWAM-Vanilla, a world–action model pretrained exclusively on 50k hours of egocentric human video. Upon robot post-training, HumanWAM-Vanilla reaches 92.08% success on RoboTwin 2.0 Clean2Clean and 68.88% on Clean2Random, alongside a 14.17% mean success rate and a 19.00 mean progress score on RoboDojo. To push the capability frontier further, we explore joint human–robot pretraining with HumanWAM-Goal, a visual-goal-conditioned world–action model. Hybrid data scaling unlocks broad zero-shot generalization across both simulation and physical benchmarks, propelling post-trained RoboTwin 2.0 performance to new state-of-the-art levels: 92.70% on Clean2Clean and a substantial leap to 84.60% on Clean2Random. On physical robots, HumanWAM-Goal further exhibits emergent embodied in-context learning, dynamically adapting its manipulation behavior to unseen visual goals without parameter updates. To support reproducible scaling research in embodied AI, we open-source the Open-HumanScale-50K corpus, data curation and pretraining pipelines, alongside the pretrained and post-trained HumanWAM checkpoints, inference code, and simulation evaluation suites.

[92] Context-Aware Autoregressive Image Generation for Emerging Reasoning Properties cs.CVPDF

Jixuan Ying, Haoyu Liu, Timing Yang, Tingyu Zhu, Predrag Neskovic

TL;DR: 论文研究自回归图像生成中的生成顺序问题，指出现有固定或启发式的顺序（如下一patch、随机顺序、按尺度一步生成）会忽略图像结构依赖。作者提出上下文感知的自回归生成框架，让模型根据全局上下文和当前局部信息动态决定下一个token的位置，从而自适应图像结构与语义依赖。实验表明该方法能提升结构鲁棒性，并展现出更强的指令理解与遵循能力。

Details

Motivation: 现有自回归图像生成通常采用预定义或启发式生成顺序，容易忽略图像中的重要结构依赖，限制结构鲁棒性和推理能力。论文希望让生成顺序由上下文动态决定，以更好适配图像结构和语义关系。

Result: 摘要称大量实验表明上下文感知生成能显著提升不同图像自回归范式（包括masked autoregressive和scale-wise autoregressive）的结构鲁棒性；同时观察到新兴推理特性，模型能更好理解复杂语义指令并更忠实地遵循生成指导。摘要未给出具体benchmark名称和定量数值。

Insight: 创新点在于将“生成顺序”作为自回归图像建模的关键未充分探索因素，并提出基于全局上下文与局部信息动态选择下一token位置的上下文感知解码框架。其可借鉴之处是跨不同AR范式提升结构鲁棒性，并可能解锁更强的推理感知视觉生成能力。

Abstract: Autoregressive image generation has recently emerged as a competitive visual generation paradigm, yet its generation order is often predefined or heuristic, such as next-patch prediction, random-order prediction, or one-step generation for specific scales. We observe that these strategies can cause models to overlook important structural dependencies in images, thereby limiting both structural robustness and reasoning capability. In this work, we propose a context-aware autoregressive generation framework, where the model dynamically determines the next token location based on global context and current local information. Rather than following a fixed or random order, our method allows the generation process to adapt to image structure and semantic dependencies. Extensive experiments show that context-aware generation significantly improves structural robustness across different image AR paradigms, including masked autoregressive and scale-wise autoregressive models. Beyond quantitative improvements, we further observe emerging reasoning properties: the model can better understand complex semantic instructions and generate images that more faithfully follow the given guidance. These findings suggest that generation order is a critical yet underexplored factor in image autoregressive modeling, and that context-aware decoding can unlock stronger reasoning-aware visual generation capabilities. Code is available at https://github.com/Hpesojyjx/Saccades-AR.

[93] Where to Connect? Boosting MLLMs via Dynamic Gated Pathways across ALL ViT and LLM Layers cs.CVPDF

Yingying Yan, Jiaqi Tang, Wei Wei, Qianzhou Wang, Lei Zhang

TL;DR: 论文研究多模态大语言模型（MLLMs）中视觉信息应接入LLM何处的问题，提出动态门控路径DGP：在每一层ViT与每一层LLM之间建立全连接路由，并让每个LLM层根据自身语义状态动态选择需要的视觉粒度。方法基于LLaVA-1.5-7B/13B验证，在多个多模态基准上取得稳定提升。

Details

Motivation: 主流MLLMs通常将固定单层视觉表示注入LLM，并把所有LLM层视为单一消费者，忽略了不同LLM层对视觉粒度的差异化需求；现有跨层融合方法多依赖编码器侧压缩或预定义稀疏/层次连接，难以建模ViT与LLM层次间的自适应交互。

Result: 基于LLaVA-1.5-7B和13B，DGP相对LLaVA-1.5-7B基线持续提升，例如SQA^I +4.2、MM-Vet +3.3、VizWiz和LLaVA^W +3.1，并在多数基准上超过先前多层视觉融合方法。

Insight: 创新点在于用全连接路由打通所有ViT层与所有LLM层，并通过每个LLM层基于自身语义状态的动态门控来选择视觉层，实现自适应跨层视觉信息接入；逐层路由分析还表明不同LLM层会按需选择不同视觉粒度，兼具有效性与可解释性。

Abstract: A central yet under-explored question in Multimodal Large Language Models (MLLMs) is where to connect—how visual information from a hierarchical vision encoder should be wired into the layer-wise semantics of a large language model. Mainstream MLLMs inject a fixed single-layer visual representation into the LLM and treat all LLM layers as a single consumer, ignoring that different layers demand different visual granularities. Recent multi-layer fusion methods either compress hierarchical features on the encoder side or rely on predefined sparse or hierarchical layer-to-layer connections, and therefore stop short of modeling adaptive cross-layer interactions between the ViT and LLM hierarchies. We address this gap with DGP (Dynamic Gated Pathways), a routing mechanism that establishes all-to-all connections between every ViT layer and every LLM layer, with each LLM layer dynamically gating its preference over visual layers conditioned on its own semantic state. Built on top of LLaVA-1.5-7B and 13B, DGP delivers consistent gains over the LLaVA-1.5-7B baseline (e.g., +4.2 on SQA$^{I}$, +3.3 on MM-Vet, +3.1 on VizWiz and LLaVA$^{W}$) and surpasses prior multi-layer visual fusion methods on most benchmarks. Layer-wise routing analyses further show that different LLM layers do select different visual granularities on demand, supporting both the effectiveness and interpretability of the proposed dynamic gated pathways. The code is available: https://github.com/YingyingYan1/DGP.

[94] Visual Token Routing: Learning to keep tokens that matter cs.CV | cs.LGPDF

Pranjal Aggarwal, Sean Welleck

TL;DR: 论文提出 Visual Token Routing (VTR)，面向 computer-use agents 的视觉 token 冗余问题，联合训练视觉语言模型与轻量级时空路由器，对当前和历史截图中的每个图像区域判断其视觉 token 是否进入语言模型，从而学习稀疏视觉表示。VTR 在保持动作预测所需信息的同时显著减少视觉 token 数量。

Details

Motivation: Computer-use agents 在长程任务中计算成本高，关键瓶颈是每张截图被表示为数千个视觉 token，并在长交互历史中累积成长上下文；作者观察到多数视觉 token 对下一步动作语义无关或在相邻截图间时间冗余。

Result: 仅训练 2000 步，VTR 在 OSWorld、CUA-World 和视频游戏动作预测上匹配全分辨率模型性能，同时减少 2.5-5.8 倍视觉 token，推理成本最多降低 3.3 倍，prefill 最多加速 2.9 倍；在 OSWorld 上比降低图像分辨率和其它 token 丢弃基线最多高 12.8 分。

Insight: 创新点在于联合训练 VLM 与轻量级时空路由器，动态决定每个图像区域 token 是否进入语言模型，并用自适应损失在 token 稀疏性与性能退化之间权衡；客观看，其可借鉴之处是将视觉 token 剪枝从静态启发式提升为可学习、时序感知的路由机制，并直接服务长程 CUA 场景。

Abstract: Computer-use agents (CUAs) have recently achieved superhuman performance on several long-horizon benchmarks. However, their computational cost limits their use in long-horizon tasks and real-time domains such as video games. A key bottleneck is representing each screenshot as thousands of visual tokens, which accumulate into long contexts as agents reason over extended interaction histories. We observe that most of these visual tokens are either semantically irrelevant to the agent’s next action or temporally redundant across screenshots. Building on this observation, we introduce Visual Token Routing (VTR), a joint training framework for learning sparse visual representations that preserve the information needed for action prediction. In VTR, we train the vision-language model together with a lightweight spatio-temporal router that examines each image region in the current and previous screenshots and decides whether its visual token enters the language model. We optimize the router using an adaptive loss to keep the number of visual tokens as small as possible while minimizing performance degradation. With just 2,000 training steps, VTR matches the performance of the full-resolution model on OSWorld, CUA-World, and video game action prediction, while using 2.5-5.8x fewer visual tokens. These savings reduce inference cost by up to 3.3x and speed up prefill by up to 2.9x. VTR also outperforms lowering the image resolution and other baselines for dropping tokens, by up to 12.8 points on OSWorld. Overall, VTR enables computer-use agents to process only the visual tokens they need, allowing them to be used in long-horizon tasks and real-time domains at a fraction of the computational cost. Code and models are available at https://pranjal2041.github.io/vtr

[95] PixelSpline: Continuous Space-Time Video Super-Resolution via Endpoint-Anchored Shared Pixel Trajectories cs.CVPDF

Wanqi Yuan, Nianyi Li

TL;DR: 论文提出 PixelSpline 连续时空视频超分框架，利用长上下文编码获得更可靠的端点表示，并在请求的高分辨率网格上连续提升外观特征。核心是以端点为锚，将每个像素的运动建模为两个端点共享的单一连续轨迹，并用该轨迹完成中间帧颜色扭曲和 HR 特征对齐后的残差重建。

Details

Motivation: 现有连续时空视频超分方法通常仅从两个端点帧推断运动，并独立建模两个运动方向，容易产生对应关系模糊和轨迹不一致问题。

Result: 实验覆盖空间和时间上采样设置，表明所提框架在质量与效率之间取得了较好的权衡；摘要未给出具体 benchmark 数值或明确 SOTA 结论。

Insight: 创新点在于引入长上下文端点表示、连续 HR 特征提升，以及端点锚定的共享像素轨迹来统一双向运动建模。客观来看，将两端运动约束为同一连续轨迹有助于提升轨迹一致性和重建效率。

Abstract: Existing continuous space-time video super-resolution methods often infer motion from only two endpoint frames and model the two motion directions independently, making them vulnerable to ambiguous correspondence and inconsistent trajectories. We introduce a continuous STVSR framework that combines long-context encoding, continuous HR feature lifting, and an endpoint-anchored PixelSpline. Long-range observations provide more reliable endpoint representations, while continuous upsampling constructs detailed appearance features directly on the requested HR grid. PixelSpline explicitly models each pixel’s motion as a single continuous trajectory shared by both endpoints. Using this trajectory, our method warps endpoint colors into a coarse intermediate frame and aligns the HR features for residual reconstruction. Experiments across spatial and temporal upsampling settings demonstrate that the proposed framework offers a favorable quality–efficiency trade-off.

[96] Procedural Adversarial Learning cs.CVPDF

Jack Nugent, Jia Deng

TL;DR: 论文提出程序化对抗学习（PAL）框架，通过极小极大训练与对抗评估，在程序化生成器的控制参数空间中搜索困难样本。作者提出Simple-PAT，用零阶优化按需生成对抗训练样本，并构建PAL-Dojo，覆盖表面法线估计、图像分割和人体姿态估计。实验表明该方法能显著降低跨模型黑盒对抗攻击下的最坏情况误差。

Details

Motivation: 神经网络在多数样本上平均误差低，但在看似简单的场景中仍会失败；这些失败样本未必存在于固定数据集中，因此需要主动搜索困难实例以提升鲁棒性。

Result: 在跨模型黑盒对手、总查询量5000万的设定下，Simple-PAT相比标准训练将最坏情况误差几何平均降低3.15倍，相比最强鲁棒训练基线降低2.63倍。

Insight: 创新点在于将可解释程序化生成器的低维控制参数作为对抗搜索空间，并用零阶优化按需生成训练样本；同时提供PAL-Dojo基准生成器集合，连接程序化数据生成与对抗鲁棒训练。

Abstract: For many computer vision tasks, neural networks can be trained to produce a low average error, yet they still fail on seemingly simple scenes. Reducing these failures requires first finding them, but they may not be in a fixed dataset. We study procedural adversarial learning (PAL), a framework of minimax training and adversarial evaluation which searches the input space of a procedural generator for difficult instances. The procedural generator is an interpretable program mapping a low-dimensional vector of control parameters (e.g. layout, lighting, pose) to a rendered image and its label. We introduce the Simple-PAT algorithm, which uses zeroth-order optimization in the procedural generator’s parameter space to propose adversarial training samples that are generated on demand. We also create PAL-Dojo, a collection of efficient procedural generators for surface normal estimation, image segmentation, and human pose estimation. Against a cross-model black-box adversary with 50M aggregate queries, Simple-PAT reduces worst-case error by a geometric mean of 3.15x compared to standard training and 2.63x compared to the strongest robust training baseline. Code is available at https://github.com/princeton-vl/procedural-adversarial-learning.

[97] Motion-guided Contrastive Regularization For Latent World Models cs.CVPDF

Yiwen Chen, Qiqi Hou, Jisoo Jeong, Yinhao Zhu, Huaizu Jiang

TL;DR: 论文研究JEPA式潜在世界模型中的表征退化问题，指出仅靠分布正则无法保证保留控制所需信息，并提出运动引导对比正则。该方法利用同一回合候选观测与光流相似度构造软对比目标，使潜在表征保留回合内状态变化与相关转移动态。

Details

Motivation: 解决静态坍缩问题：编码器保留回合特定静态背景内容，却抑制与操作控制相关的状态变化，导致随机背景干扰下规划性能显著下降。

Result: 在OGBench-Cube随机背景下，规划成功率从LeWM的52%提升到78%；在LIBERO-Object上从14%提升到54%，并在另外四个视觉控制基准上保持相当性能。

Insight: 创新点在于识别静态坍缩失败模式，并用运动引导对比正则和光流软目标降低静态背景身份对动作结果区分的作用，同时保留回合内状态区分与相关转移相似性。

Abstract: JEPA-style world models learn action-conditioned latent dynamics without reconstructing observations, but distributional regularization alone does not ensure that their representations preserve information needed for control. We identify static collapse, a failure mode in which the encoder preserves episode-specific static content while suppressing manipulation-relevant state changes. Because static content is temporally predictable yet diverse across episodes, this failure can persist despite regularizers such as SIGReg. A controlled study on OGBench-Cube shows that introducing randomized backgrounds substantially degrades planning performance while leaving the underlying manipulation task unchanged. To address this problem, we propose motion-guided contrastive regularization. Our approach compares predicted latent transitions with transitions to candidate observations from the same episode, reducing the usefulness of static background identity for distinguishing action outcomes. Optical-flow similarities provide soft contrastive targets, allowing candidates with similar motion to receive similar supervision. Combined with latent prediction and SIGReg, our objective encourages representations to preserve within-episode state distinctions and similarities between related transitions. Our method improves planning success over LeWorldModel (LeWM) from 52% to 78% on OGBench-Cube with randomized backgrounds and from 14% to 54% on LIBERO-Object, while maintaining comparable performance on four other visual-control benchmarks.

[98] From Pixels to Actions - Towards Native Vision-Language-Action Policies cs.CVPDF

Weichen Fan, Haiwen Diao, Penghao Wu, Siyuan Hu, Ziqi Huang

TL;DR: 论文提出 NEO-VLA，一种无视觉编码器的视觉-语言-动作框架：图像经轻量 patch 卷积层 token 化后，与语言在主干网络中联合演化，并让基于 flow matching 的动作专家从不同深度的指令条件视觉表示中提取信息。研究发现，原生无编码器架构在受控比较中优于基于编码器的架构，且动作关键信号常早于最终视觉表示达到峰值。该工作主张具身控制应访问计算层级中的多层视觉表示，而非固定编码器输出。

Details

Motivation: 传统 VLA 模型通常先用视觉编码器压缩感知，再进行动作推理，但操作任务往往依赖局部几何、边界、深度和细微状态变化等易被抽象丢失的视觉信号。论文由此质疑动作策略是否应继承固定视觉端点，还是直接访问不断演化的视觉计算过程。

Result: 在受控比较中，无编码器的原生架构优于基于编码器的对应方法；全量微调可缩小但无法消除差距，单纯扩大规模也不能弥补。逐层探测显示动作关键信号常在最终视觉表示之前达到峰值；在多种操作场景和分布偏移下，NEO-VLA 取得更强且更鲁棒的控制效果。

Insight: 创新点在于移除独立视觉编码器，用轻量 patch 卷积 token 化图像，并让视觉与语言在主干中联合演化，使 flow-matching 动作专家能够利用多深度、指令条件的视觉表示。客观来看，其核心价值是重新设计 VLA 中视觉信息到动作的接口，强调动作策略应直接访问层级化视觉计算而非固定编码输出。

Abstract: Vision-language-action models are typically built on a simple premise: perception is first compressed by visual encoders, and action reasoning begins only afterwards. Yet manipulation often hinges on precisely the visual signals most vulnerable to this abstraction, e.g., local geometry, boundaries, depth, and subtle state changes. This raises a fundamental question: should action policies inherit a fixed visual endpoint, or directly access evolving visual computation? We introduce NEO-VLA, a structurally encoder-free vision-language-action framework. Images are tokenized by lightweight patch convolutional layers and evolve jointly with language throughout the backbone, allowing a flow-matching action expert to draw from instruction-conditioned visual representations at different depths. This simple change reveals a consistent pattern. Native architectures outperform encoder-based counterparts under controlled comparisons; full fine-tuning reduces but does not erase the gap, and additional scale alone cannot recover it. More strikingly, layer-wise probing shows that the action-critical signals often peak well before the final visual representation. Across diverse manipulation settings and distribution shifts, NEO-VLA consequently achieves stronger and more robust control. Together, these results suggest that embodied control benefits from access to visual representations across the computational hierarchy, rather than a fixed encoder output.

cs.AI [Back]

[99] Challenge-Aware Knowledge Recruitment with Meta-Agent Supervision for Medical Multi-Agent Reasoning cs.AI | cs.CL | cs.LGPDF

Riccardo Riccio, Haitz Sáez de Ocáriz Borde, Pietro Liò

TL;DR: 论文针对大语言模型在困难医学推理任务上性能下降、多智能体辩论常不如强单模型提示的问题，提出挑战感知自适应招募方法。该方法引入外部CHALLENGER检查推理轨迹，识别事实或专业知识差距，并触发针对性知识检索或讨论中专家招募。在MEDAGENTSBENCH困难子集等实验中，该方法提升了多个数据集的准确性，并揭示了检索时机和协调干预渠道的重要性。

Details

Motivation: 现有系统仅从问题本身选择专家和检索知识，而非从推理过程中出现的差距出发，导致多智能体辩论在困难医学推理任务上效果不佳。论文旨在解决如何根据推理动态需求自适应招募外部资源的问题。

Result: 在MEDAGENTSBENCH困难子集（862个问题，9个生物医学数据集）上，挑战感知中间检索优于前期检索，在多专家骨干上提升8/9数据集的准确性，比仅Challenger增加6个百分点；Challenger触发的专家招募将GPT-4.1-mini MedQA准确性从46%提高到54%，泛化数据集方向为正；但简单结合检索和专家招募会降低性能。

Insight: 创新点在于提出挑战感知的自适应招募机制，利用外部CHALLENGER动态识别推理中的知识或专家缺口，并协调检索与专家招募的时机和信号。客观分析认为，其核心贡献是强调了“何时”以及“为何”招募外部资源在多智能体医学推理中的关键设计作用，而非单纯扩大检索或专家池。

Abstract: Large language models perform strongly on standard medical QA but degrade on harder reasoning tasks, where multi-agent debate often fails to beat strong single-model prompting. We argue that a key limitation is that existing systems select experts and retrieve knowledge from the question alone, rather than from gaps that emerge during reasoning. We propose challenge-aware adaptive recruitment: an external CHALLENGER inspects the evolving reasoning trace and identifies specific factual or expertise gaps. These signals trigger targeted knowledge retrieval or mid-discussion specialist recruitment. On the MEDAGENTSBENCH hard subset (862 questions across nine biomedical datasets), upfront retrieval provides little benefit, while challenge-aware in-between retrieval performs better; on a multi-specialist backbone it improves accuracy on 8 of 9 datasets and adds 6 percentage points over Challenger Only. Ablations indicate benefits from both retrieval timing and gap-specific query signals, and a live web-search experiment shows that the timing advantage persists across retrieval backends. For specialist recruitment, self-triggered adaptation is unreliable, whereas Challenger-triggered recruitment raises GPT-4.1-mini MedQA accuracy from 46% to 54%, with positive direction on all tested generalization datasets. Naively combining retrieval and specialist recruitment degrades performance, highlighting the need to coordinate adaptive intervention channels. These results show that when and why external resources are recruited is a key design choice in multi-agent medical reasoning.

[100] SlackBench: Benchmarking Agents on Collaborative Projects Grounded in Real Code Repositories cs.AI | cs.CLPDF

Mingxuan Li, Huanzhi Mao, Ruitao Zheng, Jarod Stanbury James, Yunze Li

TL;DR: 论文提出 SlackBench，一个面向真实协作项目的 AI Agent 评测基准。它基于隐私保护重建的真实学术研究项目，构建了包含多方对话、决策演进以及对应 GitHub 代码和 Google Docs 变更的可复现环境，并要求 Agent 联合推理工作场所沟通信息与不断变化的代码仓库状态。基准包含 140 个查询，覆盖异构来源检索、不可回答/错误前提问题拒答、目标导向摘要和长对话情境推断等任务。

Details

Motivation: 现有基准多依赖合成数据或孤立任务，难以反映真实知识工作中信息分散在对话、代码和演进决策里的复杂多主体协作场景。论文希望评测 Agent 在需要定位相关频道与私信、恢复长讨论上下文、检查仓库状态并整合冲突证据的真实科研与工程流程中的能力。

Result: 论文在 33 个前沿模型和 Agent harness 上评测，最佳系统得分约 80%，大多数 Agent 低于 66%，表明该基准仍留有较大提升空间。

Insight: 主要创新在于将真实协作沟通与持续演进的代码仓库状态结合，构建可复现、隐私保护且贴近真实科研项目的 Agent 评测环境；任务设计强调跨异构来源检索、证据综合、拒答和长对话情境推断，而非静态语料问答。

Abstract: AI Agents are becoming digital coworkers that can browse the web, write code, run experiments, and assist humans over extended horizons. Yet existing benchmarks fall short of real knowledge work: many rely on synthetic data or isolated tasks, missing the complex, multi-party dynamics of collaborative projects, where critical information is scattered across conversations, code, and evolving decisions. Even simple workplace questions may require locating relevant channels and direct messages, recovering context from long discussions, inspecting repository state, reconciling stale or conflicting evidence, and synthesizing across sources. We introduce SlackBench, a benchmark for evaluating agents on realistic research and engineering workflows that require joint reasoning over workplace communication and evolving repository state. Grounded in privacy-preserving reenactments of real academic research projects, SlackBench provides reproducible environments with realistic multi-party dialogue, evolving decisions, and corresponding code changes grounded in GitHub repositories and Google Docs. It contains 140 queries spanning heterogeneous-source agentic search, abstention on unanswerable or false-premise questions, goal-directed summarization, and circumstance inference over long conversations. These tasks are not reducible to static corpus question answering: agents must determine which conversations matter, traverse threads, inspect repository state, and synthesize evidence across communication and code artifacts. Across 33 frontier models and agent harnesses, the best system scores around 80% and most agents score below 66%, highlighting substantial headroom for improvement. SlackBench thus provides an extensible framework for evaluating agents in collaborative work environments. The SlackBench dataset is available at https://huggingface.co/datasets/HuanzhiMao/SlackBench .

[101] Iterated creation, grading, and revision of data science projects by language models cs.AI | cs.CLPDF

Rohan Alexander

TL;DR: 论文系统评估了智能体AI模型能否端到端完成数据科学项目，以及通过“写作者生成—评分者评估—写作者修订”的迭代反馈能带来多大提升。实验覆盖5家开发者的7个模型，每个模型轮流充当写作者和评分者，共49种组合、每种运行3次，共147次初始提交。结果显示项目质量平均分从59%提升到76%，但提升幅度高度依赖模型角色分配，并存在自评分偏高和评分噪声问题。

Details

Motivation: 关注当前智能体AI模型能否独立完成包含数据下载、清洗、分析和论文撰写的完整数据科学项目，并探究外部反馈、修订机会和额外预算对最终项目质量的影响。

Result: 147次初始提交中107次可运行并渲染，平均分从59%升至最终轮76%；按写作者提升3到46个百分点，按评分者提升13到32个百分点。模型自评通常比他人平均高约9个百分点，轮次间约一半差异处于评分者自身噪声内；反馈总体最有效，但仅提供修订机会或增加时间/工具调用预算也能使部分模型显著提升。

Insight: 创新在于构建多模型、多角色、多轮迭代的端到端数据科学项目评测框架，并系统分析写作者-评分者组合、自评偏差、评分噪声以及反馈/修订/预算的分离影响。客观来看，其价值主要是揭示了智能体迭代闭环中的角色不对称性：弱模型通常不能提升强模型，且自评分和评分噪声会影响自动评估可靠性。

Abstract: We evaluate the extent to which agentic AI models can complete an end-to-end data science project, and how this improves with feedback. A ‘writer’ model with specific tools creates a complete data science project including downloading, cleaning, and analyzing a dataset, and writing a short paper about it. The project is executed and a ‘grader’ model evaluates it. The writer can then revise its submission, and this repeats for a total of three rounds. We consider seven models from five developers: Anthropic, OpenAI, Google, DeepSeek, and Z.AI. Each model is both a writer and a grader, so there are 49 writer-grader pairs and we run each pair three times. Of the 147 initial submissions, 107 run and render, and the average score increases from 59 per cent to 76 per cent after the final round of feedback. By writer the extent of the increase ranges from 3 percentage points for Anthropic’s Opus 5.5 to 46 percentage points for Anthropic’s Haiku 4.5, and by grader it ranges from 13 percentage points for Google’s Gemini 3.8 Flash to 32 percentage points for Anthropic’s Haiku 4.5. When a model grades its own writing the score tends to be about 9 percentage points higher than the average of what the other models give it. About half of the difference that we see from round to round is within the grader’s own noise. Feedback generally results in the highest marks, but even just having the chance to revise, or having the additional budget, in terms of time or tool calls, without any feedback, results in considerable increases for some of the models but not for others. We find that how much such a loop helps depends on which model plays which role, but weaker models generally do not improve stronger models.

[102] BrainBind: Bridging Synchronized EEG and fMRI through Long-Form Naturalistic Video Representations cs.AI | cs.CVPDF

Shixiao Wang, Rining Wu, Jian K. Liu

TL;DR: 论文提出 BrainBind，将同步采集的 EEG 和 fMRI 映射到冻结的长视频嵌入空间，并把该空间作为共享的视觉-语义参照而非字面神经编码，用于研究自然长视频观看下的跨模态神经检索。作者在 CineBrain 数据上构建了覆盖两个视频锚点和四种系统配置的五种子 Recall@K 的 2×4 矩阵，并利用 episode-disjoint 评估分析重复刺激身份和叙事上下文对检索结果的影响。

Details

Motivation: 现实视觉是动态且场景丰富的，EEG 和 fMRI 在时间与空间尺度上互补，但现有神经解码系统多单独对齐一种模态，且常使用静态图像或短片段。论文关注同一长视频自然观看中同步记录的 EEG/fMRI，旨在解决共享刺激、重复叙事上下文和大量语义相似候选下的联合建模与检索问题。

Result: 在 CineBrain 上，五种子实验显示不同代理空间和系统配置具有不同检索剖面：使用 Omni 时 fMRI 对完整候选列表的组织最强，MRR=23.82、nDCG@10=25.31；Joint-Dual 取得最高的平均 label-aware R@1=13.52%，并呈现不同的全排名剖面。episode-disjoint 评估进一步量化了重复刺激身份和叙事上下文对随机划分结果的贡献。

Insight: 创新点在于将同步 EEG 和 fMRI 对齐到冻结的长视频嵌入空间，并把该空间作为共同视觉-语义参照，而不是直接学习字面神经编码；同时通过双视频锚点×四种系统配置的检索矩阵和 episode-disjoint 评估，分离代理选择、系统配置、重复刺激身份与叙事上下文对跨模态检索的影响。

Abstract: Real-world vision is dynamic and scene-rich, while EEG and fMRI resolve it at complementary temporal and spatial scales. Most neural decoding systems, however, align one modality in isolation and mostly against static images or short clips. We study the synchronized case: EEG and fMRI are recorded during the same long-form naturalistic viewing, giving a shared stimulus but also recurring narrative context and many semantically similar candidates. We present BrainBind, a framework that maps synchronized EEG and fMRI into frozen long-form video embedding spaces and treats those spaces as common visual-semantic references, not literal neural codes. On CineBrain, a five-seed Recall@K 2 x 4 matrix across two video anchors and four system configurations shows distinct retrieval profiles. With Omni, fMRI organizes the full candidate list most strongly (MRR = 23.82, nDCG@10 = 25.31), while Joint-Dual gives the highest mean label-aware R@1 (13.52%) with a distinct rank-wide profile. The complementary episode-disjoint evaluation quantifies the contribution of repeated stimulus identity and narrative context to the random split. Across these experiments, proxy choice and system configuration produce distinct top-rank and rank-wide retrieval profiles within the present synchronized dataset. The code is available at https://github.com/Forainest789/Brainbind.

[103] Routes of Reasoning: Reading and Reshaping Test-Time Compute in MoE Models cs.AI | cs.CL | cs.LGPDF

Shengjie Wang, Mohsen Fayyaz, Nanyun Peng

TL;DR: 论文研究MoE推理模型的过度思考问题，提出将推理轨迹按句切分并标注为规划、执行、复述、事实、反思、结论六类行为，发现MoE路由器logits的线性探针能较好预测这些行为且跨数据集泛化。基于此，在推理时选择性编辑专家路由分数，以行为级方式抑制过度思考并优化测试时计算。

Details

Motivation: 推理模型常在找到正确答案后继续验证和重复推导，造成计算浪费甚至性能下降；现有token预算或steering主要控制推理时长，难以针对具体推理行为进行调控。论文希望利用MoE路由器实现免训练、行为级的行为控制。

Result: 在四个模型和四个基准上，该方法使生成token数中位数减少30.4%，最高减少63.3%；准确率下降不超过0.5个百分点且经常提升。路由器logits探针性能接近残差流探针且特征约少一半，行为信号在六个数据集、五个领域上具有泛化性。

Insight: 创新点在于将MoE路由器作为可解释的推理行为接口，通过句子级六类行为标注和router logits探针读取推理状态，并在推理时编辑专家路由分数抑制过度思考。其价值在于免训练、轻量且兼顾可解释性与测试时计算优化，把MoE路由从效率机制拓展为行为级调控机制。

Abstract: Reasoning models allocate test-time compute across distinct behaviors such as planning, deriving, and verifying. However, models frequently “overthink,” continuing to verify and re-derive long after finding a correct answer. This wastes expensive compute and can even degrade performance. Existing interventions, such as token budgets or steering, primarily regulate reasoning duration rather than targeting specific behaviors. In this work, we demonstrate that Mixture-of-Experts (MoE) routers provide a natural, training-free interface for behavioral control. We first split reasoning traces into sentence-level segments and label each with one of six reasoning behaviors (planning, execution, restatement, fact, reflection, and conclusion), so that every token carries the behavior of its segment. Linear probes on router logits predict this label nearly as well as probes on the residual stream, the model’s full hidden state and the standard target for probing, while using about half as many features. These signals generalize: experts associated with a reasoning behavior on one dataset predict the same behavior on the others, across six datasets spanning five domains (math, code, science, knowledge, and commonsense). Building on this, we selectively edit expert router scores during inference to curb overthinking. Across four models and four benchmarks, this intervention reduces the number of generated tokens by a median of 30.4% (and up to 63.3%). Crucially, it encourages the model to stop once the problem is solved, rather than skipping essential reasoning steps. Accuracy never drops by more than 0.5 percentage points, and frequently improves. Ultimately, MoE routing offers both an interpretable lens into reasoning states and a lightweight mechanism for optimizing test-time compute.

[104] A Systematic Benchmark for Evaluating Multimodal Personalized Reasoning of Vision-Language Models cs.AI | cs.CVPDF

Dohyun Kim, Junyong Lhim, Kwanyong Park

TL;DR: 论文提出 MPRBench，一个用于评估多模态大语言模型（MLLMs）多模态个性化推理能力的综合基准，覆盖 12 个子任务和真实场景挑战，并支持代表性错误案例的可解释分析。实验表明当前个性化 MLLMs 在个性化推理上仍存在明显困难。

Details

Motivation: 现实中人们常需综合异构个人信息进行决策，但 MLLMs 的多模态个性化推理能力尚未被充分研究，缺乏系统、全面的评估基准。

Result: 论文通过广泛实验发现，当前个性化 MLLMs 仍难以完成个性化推理，主要受个性化概念识别不准确和对无关个人信息敏感等问题影响；摘要未给出具体数值或 SOTA 结论。

Insight: 主要创新在于构建覆盖多任务与真实挑战的 MPRBench，并提供可解释的错误案例分析；其价值在于系统揭示多模态个性化推理的关键失败模式，为后续模型改进提供评测基础。

Abstract: People often make decisions by holistically reasoning over heterogeneous forms of personal information. In this paper, we study the relatively underexplored capability of multimodal personalized reasoning in multimodal large language models (MLLMs). To this end, we introduce \texttt{MPRBench}, a novel and comprehensive benchmark specifically designed to evaluate personalized reasoning across a wide range of tasks and real-world challenges. \texttt{MPRBench} consists of 12 sub-tasks and additionally supports interpretable analysis of representative error cases. Through extensive experiments, we find that current personalized MLLMs still struggle with personalized reasoning due to several key challenges, including inaccurate recognition of personalized concepts and sensitivity to irrelevant personal information. We hope that MPRBench will stimulate further research on personalized reasoning in MLLMs. Code and data are available at https://github.com/kkdh15/MPRBench

[105] LegalEvolver: Agentic Legal Judgment Prediction via Rubric-Guided Self-Evolution cs.AI | cs.CLPDF

Xinyu Yang, Jiejun Tan, Chenlong Deng, Chenghao Zhang, Tongyu Wen

TL;DR: 论文提出 LegalEvolver，一个面向法律判决预测（LJP）的查询优化框架，用于生成更具法律信息量的检索查询。方法先用启发式优化查询作为种子提升轨迹采样效率，再将 rollout 经验总结为评分细则（rubrics），通过多轮迭代的 rubric 优化与 rubric 引导强化学习实现自我进化，从而提升法律检索与判决预测的端到端表现。

Details

Motivation: 法律判决预测需要多步推理和外部法律证据检索，但现有法律搜索智能体常直接根据初始请求构造查询，忽视潜在法律争点和法律上重要的细节，导致检索查询质量成为性能瓶颈。

Result: 在多个基准上的端到端实验表明查询优化对 LJP 有效；与直接 SFT 后接 RL 的对比验证了 rubric 引导的有效性；进化 rubric 进一步优于固定 rubric，说明自我进化带来收益。

Insight: 创新点在于面向 LJP 的法律信息查询优化、用启发式优化查询作为种子提高轨迹生成效率，以及将 rollout 经验总结为可迭代优化的 rubrics，并用 rubric 引导强化学习实现多轮自我进化。

Abstract: As LLM-based agents develop, they have been increasingly applied in the legal domain. Legal Judgment Prediction~(LJP) requires models to conduct multi-step reasoning and external legal evidence retrieval. Existing legal search agents’ performance is often constrained by the quality of retrieval queries. They often formulate queries directly from the initial user request, overlooking latent legal issues and legally salient details. To address this issue, we propose \emph{LegalEvolver}, a query optimization framework specifically designed to generate legally informative queries for LJP-oriented retrieval. Sampling high-quality optimized queries and their corresponding search trajectories is challenging, as relying solely on unguided high-temperature rollouts is both inefficient and constrained by the model’s initial capabilities. We therefore heuristically optimize queries as seeds, improving the efficiency of trajectory generation. Furthermore, we summarize rollout experience into rubrics and use them to guide multi-round iterative self-evolution. Iteratively performing rubric optimization and rubric-guided RL enables the model to surpass its initial performance ceiling. End-to-end performance on extensive benchmarks demonstrates the effectiveness of query optimization on LJP. Comparisons with direct SFT followed by RL demonstrate the effectiveness of rubric guidance. Evolving rubrics further outperform fixed rubrics, confirming the benefits of self-evolution.

[106] Interactive World Simulation with Cosmos for Physical AI cs.AI | cs.CV | cs.LG | cs.ROPDF

NVIDIA, :, Anis Ahmad, Jon Allen, Malik Aqeel Anwar

TL;DR: 论文提出一个基于 Cosmos 世界基础模型的共享框架，用于 Physical AI 中的实时交互式世界仿真。该框架根据动作和场景控制流式生成视觉观测，使人类用户和具身智能体能够与动态虚拟环境交互，并同时支持动作视觉后果预测和基于外部模拟器物理状态生成观测。作者在相机导航、机器人操作、可控视频迁移和多视角自动驾驶仿真等场景中展示了该框架，并开源了代码与模型检查点。

Details

Motivation: 论文关注 Physical AI 训练与评估对可交互、实时、可控虚拟环境的需求，试图解决如何将世界基础模型适配为响应动作与场景控制的交互式仿真框架，并统一支持用户交互和具身智能体交互。

Result: 摘要未给出定量结果或具体 benchmark 指标，仅定性地展示了该框架在相机导航、机器人操作、可控视频迁移和多视角自动驾驶仿真中的应用示例。

Insight: 论文宣称的创新点是一个面向 Physical AI 的共享实时交互式世界仿真框架，能够流式响应动作与场景控制，并兼容动作驱动预测和外部模拟器物理状态驱动两种模式。客观来看，其价值主要在于将 Cosmos 世界基础模型系统化适配到多类交互仿真任务，并开源代码与模型检查点，为后续 Physical AI 环境构建提供统一基础。

Abstract: We present a shared framework that adapts Cosmos world foundation models for real-time interactive world simulation in Physical AI. By streaming visual observations in response to actions and scene controls, the framework enables users and embodied agents to interact with evolving virtual environments. It supports both predicting the visual consequences of actions and generating observations from physical state maintained by an external simulator. We demonstrate this approach across camera navigation and robot manipulation, controllable video transfer, and multi-view autonomous vehicle (AV) simulation. These examples provide a shared foundation for developing interactive virtual environments for Physical AI training and evaluation. Our code and model checkpoints are available at https://github.com/nvidia/cosmos and https://huggingface.co/collections/nvidia/cosmos3-examples. The project website is available at https://research.nvidia.com/labs/cosmos-lab/cosmos-simulation.

cs.HC [Back]

[107] Robots in the City: Persona-Based Perceptions of Urban Robots cs.HC | cs.CL | cs.ROPDF

Keonhee Jang, Kanghoon Lee, Seokwoo Yun, Namwoo Kim, Yoonjin Yoon

TL;DR: 论文研究了城市公共空间中机器人（配送机器人、人形机器人）的存在与类型如何影响街道的安全感感知。作者生成18张首尔街景图像（变化因素包括时段、道路等级和机器人条件），让两个多模态大模型（Qwen3.8-27B、Gemma4-26B）各自以500个合成人物画像（persona）为条件进行两两图像的“更安全”选择，并让25名真人完成相同任务进行对照。

Details

Motivation: 行人越来越多地在城市公共空间中遇到机器人，却往往不与之交互；然而机器人如何改变街道的安全感仍不清楚，论文旨在考察机器人的存在与类型在不同城市场景下对感知安全的影响。

Result: 两个模型均偏好无机器人的街道，且在配送机器人与人形机器人之间更偏好配送机器人；机器人在夜间被选择更多、老年persona选择更少，道路等级影响较小。模型与人类在“夜间更接受机器人”上一致，但在机器人类型上差异明显：模型远少选人形机器人，Gemma对人形机器人的夜间效应约为人类的三倍。定性上，模型理由中配送机器人被视为物理障碍，人形机器人则被视为威胁或保护来源。

Insight: 创新点在于用人物画像条件化的多模态大模型作为多样化人类感知的模拟代理，并对比真人判断、结合模型给出的理由进行语义分析，揭示机器人对安全感的影响取决于出现时段；客观来看，其价值还在于指出persona条件化MLLM只能捕捉人类判断的一部分，为用LLM模拟人类感知的局限性提供了实证证据。

Abstract: Pedestrians increasingly encounter robots in urban public spaces without interacting with them, but how robots change the perceived safety of a street remains unclear. We examine how robot presence and type affect perceived safety across urban contexts. We generate 18 images of Seoul streets that vary in time of day, road class, and robot condition (no robot, a delivery robot, or a humanoid). Two multimodal large language models (MLLMs), Qwen3.8-27B and Gemma4-26B, each conditioned on 500 synthetic personas, choose the safer image in each pair, and 25 human respondents complete the same task. Both models prefer streets without robots and favor the delivery robot over the humanoid. Robots are chosen more often at night and less often by older personas, whereas road class has smaller effects. In the model rationales, the delivery robot appears as a physical obstacle, whereas the humanoid appears as either a threat or a source of protection. Like humans, both models chose robots more often at night, but chose the humanoid over the delivery robot far less often, and Gemma showed a night effect for the humanoid about three times that of humans. These results show that the effect of a robot on perceived safety depends on when it appears, and that persona-conditioned MLLMs capture only part of human judgments.

cs.LG [Back]

[108] What Does Multi-Turn Reinforcement Learning Really Improve in LLM Agents? cs.LG | cs.CLPDF

Kejian Zhu, Zhuoran Jin, Shangqing Tu, Tianyi Men, Yupu Hao

TL;DR: 论文系统分析了多轮智能体强化学习在LLM Agent中究竟带来了什么提升，发现其收益高度依赖交互预算，并通过Pass@(K,T)评估揭示轨迹级RL优势需要足够轮次预算，且主要提升错误恢复与长程鲁棒性而非单步决策质量。为缓解预算依赖带来的部署风险，论文讨论过程监督信号并提出在训练中变化交互预算的BudgetMix RL，实验显示其在多种执行设置下表现稳健。

Details

Motivation: 多轮智能体RL在长程任务上表现突出，但其性能收益对交互约束高度敏感，论文旨在厘清多轮RL真正改善了哪些能力，并缓解预算依赖导致的部署风险。

Result: 基于采样广度K和交互深度T的Pass@(K,T)评估表明，轨迹级RL优势依赖充足轮次预算，预算收紧时收益显著下降；智能体RL并不均匀提升单步决策质量，而主要增强错误恢复和长程韧性。BudgetMix RL在多样化执行设置下取得稳健性能。

Insight: 创新点在于从交互预算维度重新审视多轮智能体RL的收益来源，区分单步决策与长程恢复能力，并提出通过训练时变化交互预算的BudgetMix RL来提升部署鲁棒性，同时初步讨论过程监督信号的利弊。

Abstract: While multi-turn agentic reinforcement learning (RL) excels on long-horizon tasks, its performance gains remain highly sensitive to interaction constraints. Evaluating across sampling breadth $K$ and interaction depth $T$ via $Pass@(K,T)$, we show that trajectory-level RL advantages depend on sufficient turn budgets, diminishing significantly when budgets tighten. Besides, agentic RL does not uniformly elevate single-step decision quality; rather, its strength concentrates in error recovery and long-horizon resilience. To mitigate the deployment risks caused by this budget dependence, we provide a preliminary discussion on the benefits and limitations of process supervision signals and introduce BudgetMix RL as an alternative that varies interaction budgets during training. Experiments demonstrate that BudgetMix RL achieves robust performance across diverse execution settings. Project Page: https://rethinking-agentic-rl.github.io/

cs.RO [Back]

Changyeon Lee, Kelin Yu, Zheng Wei, Shuna Ni, Ruohan Gao

TL;DR: 论文提出 OdorSpaces，一个基于物理仿真的视觉-嗅觉具身导航平台与基准，通过将 FDS 的 3D 气味传输与扩散模拟集成到 Habitat-Sim，使智能体能在逼真室内环境中结合视觉和嗅觉观察进行寻源导航。基准包含不可见气体泄漏定位与早期火灾定位两个场景，并给出一个融合视觉与嗅觉的多模态强化学习初始策略。

Details

Motivation: 嗅觉提供重要的空气传播化学线索，但在具身 AI 中研究不足，已有机器嗅觉工作多聚焦语义气味感知，缺少视觉-嗅觉联合导航的仿真平台与标准评测。

Result: 实验表明所提出的多模态强化学习策略优于多种基线，为后续视觉-嗅觉具身导航研究提供了强参考；摘要未给出具体定量指标或明确 SOTA 声明。

Insight: 创新点在于构建物理级气味传输/扩散仿真与逼真视觉环境结合的导航基准，并定义气体泄漏和早期火灾两类寻源任务；客观看，其价值主要在推动嗅觉具身导航的仿真标准化与多模态策略评测。

Abstract: Olfaction provides critical information about airborne chemical cues, yet it remains largely underexplored in embodied AI, with limited prior machine olfaction work focusing primarily on semantic odor perception. We introduce OdorSpaces, a physics-based simulation platform and benchmark for visual-olfactory embodied navigation. OdorSpaces augments Habitat-Sim with 3D odor transport and dispersion simulated by Fire Dynamics Simulator (FDS), enabling agents to navigate photorealistic indoor environments using both visual and olfactory observations. The benchmark includes two representative source-seeking scenarios: invisible gas-leak localization and early-stage fire localization. We develop an initial multimodal reinforcement learning policy that integrates visual observations and olfactory measurements over time. Experiments show that the policy outperforms a diverse set of baselines, establishing a strong reference for future research on visual-olfactory embodied navigation.

[110] Perceptive Humanoid Motion from Simulation cs.RO | cs.AI | cs.CV | cs.LGPDF

Siyuan Hu, Abulikemu Abuduweili, Yue Meng, Shipeng Liu, Quentin Leboutet

TL;DR: 论文提出 HALO 人形机器人自主系统，通过程序化生成的千个模拟家庭合成视觉-语言-运动数据，训练机器人掌握坐下、导航到指定物体、侧身过半开门、蹲爬过桌底和推箱子等技能，并能组合技能完成长程任务。系统未在真实家庭中训练，却零样本迁移到物理世界，在六个未建图真实环境中完成 360 次试验，成功率达 91%。

Details

Motivation: 人形机器人通用自主性受数据瓶颈限制：将像素和自由语言映射到低层关节命令需要大规模配对监督数据，而此类数据不存在；人类遥操作或穿戴设备采集数据耗时至少 1:1，难以扩展。

Result: 在六个未建图的真实环境中进行 360 次试验，HALO 零样本迁移达到 91% 成功率，且训练中从未见过真实家庭。

Insight: 创新点在于用程序化生成的大规模模拟家庭和合成视觉-语言-运动数据，以计算扩展替代人工采集；同时学习可复用技能库并编排解决复合长程任务，实现仿真到现实的零样本迁移。

Abstract: Humanoid robots have the potential to automate a wide range of physical tasks. Progress toward generalist humanoid autonomy, however, remains constrained by a data bottleneck: mapping pixels and free-form language to low-level joint commands requires paired supervision, and no corpus of such data exists at sufficiently large scale. Some approaches rely on human teleoperation and wearable capture devices, which require at least an hour of human time to produce an hour of data. We present HALO, a humanoid autonomy system powered by a pipeline that synthesizes vision-language-motion data across a thousand procedurally generated homes. The corpus scales with compute rather than human labor. HALO acquires a repertoire of physical skills, including sitting on chairs, navigating to named objects, stepping sideways through half-open doors, crouching and crawling beneath tables, and pushing boxes. It orchestrates these skills to solve composite, long-horizon tasks. Without ever seeing a real home during training, HALO transfers zero-shot to the physical world, achieving a 91% success rate across 360 trials in six unmapped real-world environments. See https://apple-aiml-research.github.io/ml-halo/ for code and videos.

Table of Contents

cs.CL [Back]

[1] CCDWIKI: Decoupling Case Formulation from Response Generation in Longitudinal CBT Dialogue cs.CLPDF

[2] Contextual Pictograms for Repairing Multilingual ASR Errors cs.CL | cs.HCPDF

[3] Phonological Knowledge of Chinese Characters in Language Models cs.CLPDF

[4] Learning When Human Decisions Matter: Executor-Relative Consultation Value for LLM Agents cs.CL | cs.AIPDF

[5] Familiar letters, not related languages, let unseen languages into a shared space of meaning cs.CLPDF

[6] Holistic Parallel Supervision for Expressive Speech to Speech Translation cs.CLPDF

[7] Adapting Qwen3 to Tajik from 0.6B to 8B: Lower Text Loss Did Not Mean Better Understanding cs.CLPDF

[8] Overview of the NTCIR-19 SciClaimEval Task: Cross-modal Claim Verification in Scientific Papers cs.CLPDF

[9] The Harness Is the Model: Agent Scaffolding Changes a Fixed LLM’s Compliance with Banking Conduct Rules, Unevenly Across Models cs.CL | cs.AI | cs.CYPDF

[10] Loop Before You Leap: What Recurrent Depth Can and Cannot Buy in Data-Limited Pretraining cs.CL | cs.LGPDF

[11] Mass Producing Training Data with Surprising Side Effects cs.CL | cs.AI | cs.LGPDF

[12] Twists and Turns: A Graph-Based Benchmark and Framework for Long-Term Memory in Exploration Tasks cs.CLPDF

[13] When Do Summaries Become Instructions? Unsupported Policy in LLM Agents’ Self-Written Handoff Summaries cs.CLPDF

[14] Reasoning over Non-Parametric Memory in Multilingual LLMs: A Cognitive-Based Analysis of Relational Knowledge cs.CL | cs.AIPDF

[15] Ranking Agreement Does Not Reveal Paragraph-Edit Agreement in Two Released Paper Acceptance Predictors cs.CL | cs.LGPDF

[16] Before You Continue: Evaluating Time-Sensitive Safety Awareness in Conversational Assistants cs.CLPDF

cs.CV [Back]

[17] VAR-Q: Tuning-free KV Cache Quantization for Visual Autoregressive Image and Video Generation cs.CV | cs.PFPDF

[18] Visual Forensic Detection of Fabricated Bangla News Photocards using Deep Learning cs.CVPDF

[19] mmHand: Toward Pixel-Level-Accuracy Hand Localization Using a Single Commodity mmWave Device cs.CVPDF

[20] When Does Sharing Work? Scaling a Shared SigLIP2 Across Image, Video, Audio, and Audio-Visual Representations cs.CV | cs.AIPDF

[21] PEEK: One-Step-Look-Ahead Exposure Bias Correction for Diffusion Sampling cs.CV | cs.AIPDF

[22] Augmentation with Reinforcement Transformation for Low-textured Scientific Sketch Image Retrieval cs.CVPDF

[23] SurgTransfer: Multi-Step Latent Action Modeling for Action Transfer in Surgical World Model cs.CV | cs.AI | cs.LGPDF

[24] From Tokens to Trajectories: Parameter-Efficient Continuous Adaptation of Unified Multimodal Models cs.CVPDF

[25] From Activation to Intervention: Interpreting Sparse Visual Features through Reconstruction cs.CVPDF

[26] How Finely Can Vision-Language Models See Biology? Bio-VQA: A Benchmark for Taxonomic, Morphological, Spatial and Individual Reasoning cs.CV | cs.AIPDF

[27] Discarded or Unreachable? Diagnosing and Repairing Spatial Relation Failure in Vision–Language Similarity cs.CV | cs.AIPDF

[28] Whose Values Are Generated? Measuring Visual Value Leakage in Multilingual Text-to-Image Generation cs.CVPDF

[29] Multimodal Fusion Methods for Whole-Slide Images and Clinical Data in Inflammatory Bowel Disease cs.CV | cs.AI | cs.LGPDF

[30] GaussianRenderFormer: Transformer-based Rendering of 3D Gaussian Splats cs.CV | cs.GR | cs.LGPDF

[31] CAPTOR: text-encoder-free grounding by counting caption co-occurrence over the visual words of a frozen encoder cs.CVPDF

[32] Spatio-Temporal Multi-Order Interaction for Video Recognition cs.CV | cs.AI | cs.LGPDF

[33] NitroFusion2: Bringing Adversarial Loss to the Text Stream of Multimodal Diffusion Transformers cs.CVPDF

[34] GeoDIVE: Geospatial Descent Informed by Value Estimation cs.CVPDF

[35] C-WFDM: A ControlNet-Guided Wavelet-Fusion Diffusion Model for Multimodal Brain MRI Completion cs.CVPDF

[36] Learning Hierarchical Patch Splitting Policies for Faster Vision Transformers cs.CV | cs.AI | cs.LGPDF

[37] Navigational Safety of Marine Surface Vehicles in Restricted Visibility: From Intelligent Vision to Vision-Centered Multi-Sensor Fusion cs.CVPDF

[38] DualCast: Dual-Image Conditioned Diffusion Transformers for Digital Human E-Commerce Video Generation cs.CVPDF

[39] Generative AI for urban planning: synthesizing land-use maps with satellite imagery via DualControlNet cs.CV | cs.AIPDF

[40] Feature Puzzles: Single-Image Supervision for Multi-View 3D-Aware Segmentation cs.CVPDF

[41] Chronos: Sequential Identity-Preserving Video Generation by Temporal Anchoring cs.CV | cs.MMPDF

[42] DecoupAttn: Decoupled Sparse Attention for Training-Free Acceleration of Video Diffusion cs.CVPDF

[43] Modality-Aware Expert Pruning for MoE-Based Multimodal Large Language Models cs.CV | cs.CLPDF

[44] CLEAR: Color-Texture Latent Alignment for Single Image Reflection Removal cs.CV | cs.GRPDF

[45] PathoS2VD: From 2D Histopathology to Pseudo Z-Stack Representations cs.CVPDF

[46] Clips2Story: Training-Free Video Storyboarding and Editing Using Multimodal Retrieval-Embedded Generation cs.CV | cs.AI | cs.LG | cs.MMPDF

[47] FedIA: Instance-Aware Cross-Modal Representations for Federated Prompt Learning cs.CV | cs.LGPDF

[48] MotoVerse: A Benchmark for Predictive Perception of Motorized Two-Wheelers in Dense Unstructured Traffic cs.CVPDF

[49] ChanSFormer: A Channel Agnostic Vision Transformer for Multi-Channel Cell Painting Images cs.CVPDF

[50] USI-Brain: Enhancing Underwater Spatial Intelligence via Spatial Data-Driven Vision-Language Modeling cs.CVPDF

[51] RI-Prune: Relation and Information Aware Visual Token Pruning for Multi-Image Vision-Language Models cs.CVPDF

[52] TouchéWorld tech report : Towards Online Unified Camera–Hand Exploration in Generative Worlds cs.CV | cs.AIPDF

[53] SleuthVLM: Do We Really Need Audio-in-Video for Omni Video Understanding? cs.CVPDF

[54] GeoBiaset: A Counterfactual Benchmark for Demographic Bias in World-Level Geolocalization cs.CVPDF

[55] STARE: Scene Text-Aware multimodal Retrieval for Everyday objects cs.CVPDF

[56] Drift-adaptive Representation Preservation for Domain-Incremental Learning cs.CV | cs.AIPDF

[57] Semantic Prior-Guided Incomplete Cross-Modal Retrieval with Noisy Labels cs.CVPDF

[58] ReactWorld: Structuring Interaction Responses for Embodied Video Generation cs.CV | cs.ROPDF

[59] RDT-3D: Resolution-Decoupled Training for High-Fidelity 3D Geometry Generation cs.CVPDF

[60] PTQ4SAM3: Semantics-Preserving Post-Training Quantization for Segment Anything Model 3 cs.CVPDF

[61] Grounding in the Dark: Text-Only Training for Zero-Shot Video Temporal Localization cs.CV | cs.AIPDF

[62] DSRS: Teaching MLLMs to Perceive, Reason, and Segment in Complex Driving Scenes cs.CVPDF

[63] Reasoning-Aware Cross-Modal Association Unlearning in Multimodal Large Reasoning Models cs.CV | cs.AIPDF

[64] Rohbau3D-Semantics: Dense Semantic and Instance Labels for Point Clouds of Indoor Shell Construction Scenes cs.CVPDF

[65] Annotator-Free Object Detection via Foundation Model Collaboration cs.CVPDF

[66] NextGAN: Towards Simple and Scalable GAN Training for Image and Video Generation cs.CV | cs.AIPDF

[67] MonoLaDN: Language-Guided DeNoising for Monocular 3D Object Detection cs.CV | cs.AIPDF

[68] Beyond Scene Shortcuts: Action-Centric Video-Text Alignment for Fine-Grained Action Understanding cs.CVPDF

[69] Prism: Bidirectional Self-Teaching for Unified Multimodal Models cs.CV | cs.AIPDF

[70] Traffic-VL: Reinforced Vision-Language Models for Generalizable End-to-End Traffic Signal Control cs.CV | cs.AIPDF

[71] FlashRelight: Portrait Video Relighting with Dynamic Lighting cs.CVPDF

[72] NanoKV: High-Ratio KV Cache Compression for Autoregressive Video Generation cs.CVPDF

[73] Entropogram-guided Visual Realignment Decoding: Capturing the Rhythm of Multimodal Reasoning cs.CV | cs.MMPDF

[74] Unaligned Image Guided Denoising via Cross-modal Conditional Flow Matching cs.CVPDF

[75] DreamRealm: Exploring and Materializing Hallucination Worlds for Trustworthy Vision-Language Models cs.CVPDF

[76] CoDecide: Content-Adaptive Video Decoding for High-Throughput Video Analytics cs.CVPDF

[77] EmoReward: Self-Evolving Reward Calibration for Emotional Video Generation RL cs.CV | cs.AIPDF