cs.CL [Total: 30]
cs.CV [Total: 67]
cs.CY [Total: 1]
cs.AI [Total: 8]
cs.NI [Total: 1]
cs.LG [Total: 6]
cs.HC [Total: 2]
cs.RO [Total: 6]
cs.CR [Total: 1]
cs.IR [Total: 1]

cs.CL [Back]

[1] Assert, don’t describe: Linguistic features that shift LLM reasoning about animal welfare cs.CL | cs.AIPDF

Jasmine Brazilek, Harper Dunn

TL;DR: 本文研究了动物福利文本中十种语言特征对Llama-3.2-1B模型在动物福利推理中立场偏好的影响。通过在保留的动物福利基准上进行词汇匹配的立场对比探测，发现其中八种特征能产生统计显著的立场偏移。

Details

Motivation: 动机在于探究动物福利倡导者的大量文本（这些文本常被用于训练语言模型）中，不同的语言特征如何影响LLM在动物福利问题上的推理立场，从而为相关文本创作提供指导。

Result: 在动物福利基准测试中，七种特征（如断言确定性、明确道德词汇等）显著增强了模型支持动物福利的推理，而两种特征（如模糊语言和具体感官描述）则削弱了该立场，第一人称视角无显著影响。

Insight: 创新点在于系统性地量化了特定领域（动物福利）文本的语言特征对LLM微调后推理立场的影响。核心发现是：在可能用于LLM训练的文本中，明确断言立场（而非中立描述）能更有效地塑造模型的推理倾向。

Abstract: Animal-welfare advocates produce a lot of writing, and increasingly that writing trains the language models that millions of people then ask about animal welfare. Using vocabulary-matched stance-contrast probes on a held-out animal-welfare benchmark, we measure how each of ten linguistic features changes Llama-3.2-1B’s preference for pro-animal-welfare reasoning when used as fine-tuning data. Eight of the ten features produce statistically significant shifts. Seven move the model toward stronger pro-animal-welfare reasoning: assertive certainty, explicit moral vocabulary, emotion words, evaluative claims, narrative structure, depicted harm severity, and immediate temporal framing. Two move it the other way: hedged language and concrete sensory description both dilute the pro-animal-welfare stance. First-person perspective has no statistically significant effect. The practical recommendation for anyone writing animal-welfare text that may end up in LLM training corpora: assert a position rather than describe a scene neutrally. The features that shift the model are the ones that make the writer’s position explicit; the features that dilute it hold animal-welfare content but withhold stance.

[2] Investigating LLM’s Problem Solving Capability – a Study on Statics Questions cs.CL | cs.AIPDF

Tanner Culleton, Hung-Fu Chang

TL;DR: 本研究通过模型蒸馏方法，评估了大型语言模型（LLMs）在解决静力学问题上的能力。研究构建了纯文本、带图表以及修改数值的三个数据集，实验发现LLMs在纯文本问题上表现良好，但在引入图表并需要多步推理时准确性下降。分析表明，性能下降主要源于多步推理困难以及难以在连续解题阶段中一致地应用提取的视觉信息，而非图像识别限制。

Details

Motivation: 现有研究多依赖公开问题数据集，缺乏针对特定工程教育主题（如机械工程静力学）的系统性分析。本研究旨在弥补这一空白，通过模型蒸馏而非直接提问的方式，更细致地评估LLMs在解决特定类型工程问题上的实际能力。

Result: 实验结果表明，LLMs在纯文本静力学问题上表现良好，但在引入图表（需要多步推理）的数据集上准确性显著下降。该结果基于通过蒸馏ChatGPT构建的特定静力学问题数据集得出，揭示了LLMs在复杂工程问题解决上的当前局限。

Insight: 创新点在于采用模型蒸馏过程来生成特定领域的评估数据集，并系统分析了图表引入对LLM解题能力的影响。客观来看，研究强调了多步推理和跨步骤信息一致性（而非单纯的视觉理解）是当前LLMs处理复杂工程问题的关键瓶颈，为未来模型能力评估和提升提供了重要方向。

Abstract: Large Language Models (LLMs) have rapidly influenced many aspects of society, particularly education, due to their demonstrated ability to complete assignments and examinations across a wide range of subjects. Although prior studies have examined the educational impact of LLMs, much of the existing work relies on public or open problem datasets and lacks topic-specific analysis. In engineering education, especially within mechanical engineering, systematic investigations of LLM performance on specific problem types remain limited. Instead of using traditional methods that directly ask textbook questions to an LLM tool, our study adopts a model distillation process to evaluate LLM capabilities in solving statics problems. By distilling ChatGPT, we extracted 25 text-only statics questions and further constructed two additional datasets by adding diagrams and modifying their numerical values. Experimental results show that while LLMs perform well on text-only statics problems, their accuracy decreases when diagrams are introduced and the problems require multi-step reasoning. Further analysis suggests that this performance drop is not primarily caused by limitations in image recognition, but rather by difficulties in multi-step reasoning and in consistently applying extracted visual information across successive solution stages.

[3] Helpfulness Hurts: Domain-Dependent Degradation of Mid-Trained Compassion Values Under Post-Training cs.CL | cs.AI | cs.CYPDF

Jasmine Brazilek, Juliana Seawell

TL;DR: 本文研究了在Llama 3.1 8B模型上，使用不同领域的后训练数据（有益性 vs. 代码）进行监督微调（SFT）或强化学习（GRPO）后，对其中期训练所灌输的动物同情价值观的保留情况的影响。研究发现，有益性训练会显著损害动物同情价值观和一般道德推理能力，而代码训练则能更好地保留这些价值观，且这种影响在跨语言场景下表现出不同模式。

Details

Motivation: 标准的后训练流程（如SFT和RL）旨在使语言模型变得更有帮助，但这些过程可能会无意中损害预训练或中期训练阶段灌输的价值观。本文旨在探究后训练数据的领域（有益性领域与代码领域）是否会对中期训练灌输的动物同情价值观的保留产生不同的影响。

Result: 在Animal Harm Benchmark (AHB 2.2)上，有益性训练显著降低了动物同情表现（SFT: 35.7% vs. 65.2%；GRPO: 18.7% vs. 32.0%）。在英文MORU基准测试中，有益性训练使一般道德推理能力下降了25.5个百分点（46.4% vs. 71.9%）。然而，在多语言MORU基准测试中，这种领域效应消失了。动物同情效应则能跨语言一致地迁移。

Insight: 论文的创新点在于揭示了后训练数据的领域对模型价值观保留的特定影响，并发现通过中期训练灌输的价值观比领域特定后训练带来的推理能力改进编码得更深、更具跨语言一致性。这为在构建蕴含价值观的模型时，选择代码领域而非有益性领域进行后训练以更好地保留核心价值观提供了实证依据。

Abstract: Standard post-training pipelines apply supervised fine-tuning (SFT) and reinforcement learning (RL) to make language models helpful, but these processes may inadvertently degrade values instilled during pre-training. We investigate whether the domain of post-training data differentially affects the retention of animal compassion values in a Llama 3.1 8B model mid-trained on compassion-oriented synthetic data, using both SFT (helpfulness via Dolly-15k vs. coding via Magicoder-110K) and GRPO (helpfulness via RLHFlow vs. coding via Magicoder), evaluated on the Animal Harm Benchmark (AHB 2.2) and MORU benchmark (Moral Reasoning Under Uncertainty). Helpfulness training significantly degrades animal compassion relative to coding training on AHB (SFT: 35.7% vs. 65.2%; GRPO: 18.7% vs. 32.0%), replicating across two independent helpfulness datasets and two training paradigms. On English MORU items, helpfulness training degrades general moral reasoning by 25.5 percentage points (46.4% vs. 71.9%), a striking gap that rivals the compassion effect in magnitude. However, this effect does not transfer cross-lingually: on the multilingual MORU benchmark, the domain effect disappears (SFT: 52.3% vs. 51.2%). In contrast, the animal compassion effect transfers consistently across languages, with Magicoder’s AHB percentage-point gain over the base model 4.5 times larger on non-English items than English items. This divergence suggests that values instilled through mid-training are encoded more deeply and cross-lingually than reasoning improvements from domain-specific post-training. These results suggest that, for labs building on value-laden mid-training, coding-domain post-training may better preserve mid-trained values than helpfulness post-training without harming general reasoning capabilities.

[4] Low Resource Multimodal Translation of Nepali Spoken Words into Emotion-Conditioned Sign Language Avatars cs.CL | cs.AIPDF

Jatin Bhusal, Salma Tamang

TL;DR: 这篇论文提出了NEST-V1框架，这是一个低资源多模态翻译系统，能够将尼泊尔语口语单词转换为带有情感条件的尼泊尔手语虚拟化身。该研究作为一个概念验证，聚焦于四个常见单词和三种情感状态，验证了其核心技术的可行性。

Details

Motivation: 动机在于解决低资源语言（如尼泊尔语）中，整合情感表达的手语通信系统研究不足的问题，旨在为听力障碍社区开发实时、富有情感表现力的手语交流工具。

Result: 在包含50名说话者的600个标注音频样本数据集上，其轻量级架构的共享声学编码器实现了81.1%的自动语音识别准确率和79.21%的情感识别准确率。与独立模型架构相比，系统参数效率提升了37%，总参数量仅为22.1M，适合边缘部署。

Insight: 创新点在于提出了一个参数高效的、共享编码器的多模态框架，将语音识别和情感分类任务结合，为低资源环境下的情感感知手语翻译建立了技术基础，并提供了一个可扩展的框架，便于未来扩展到更大词汇量和更丰富的情感表达。

Abstract: Sign language communication systems, that integrate emotional expression remain underexplored, particularly for low-resource languages. This pilot study presents NEST-V1 (Nepali Emotion and Speech Transformer - Version 1), a proof-of-concept multimodal framework that demonstrates the feasibility of generating emotion-conditioned Nepali Sign Language avatars from spoken input. As a preliminary investigation, we focus on four common Nepali words (“thank you”, “hello”, “house”, “me”) across three emotional states (happy, neutral, sad) to validate our core technical approach. Our lightweight architecture employs a shared acoustic encoder for simultaneous Automatic Speech Recognition and emotion classification, achieving 81.1% ASR accuracy and 79.21% emotion recognition accuracy on a dataset of 600 labeled audio samples from 50 speakers. The system demonstrates 37% parameter efficiency compared to separate model architectures while maintaining a lightweight footprint with only 22.1M parameters suitable for edge deployment. This pilot work establishes the technical foundation for emotion-aware sign language translation in low-resource settings and provides a scalable framework for future expansion to larger vocabularies and more diverse emotional expressions. Our preliminary results indicate the viability of real-time, emotionally expressive sign language communication systems for the hearing-impaired community, with clear pathways for enhancement in subsequent development phases.

[5] Context Recycling for Long-Horizon LLM Inference cs.CL | cs.AI | cs.LGPDF

Derek Thomas

TL;DR: 本文提出了一种名为ContextForge的系统，用于解决大语言模型在长对话中因上下文窗口限制和令牌使用效率低下而导致的性能下降问题。该系统通过结合结构化查询生成、外部记忆检索和受控合成，实现了跨轮次的任务相关信息维护与计算复用，从而在不依赖完整上下文重放的情况下减少令牌开销并保持回答质量。

Details

Motivation: 大语言模型在短上下文推理中表现出色，但在长对话任务中，由于上下文窗口限制和令牌使用效率低下，其性能会下降，因此需要一种方法来高效地管理和复用长对话中的相关信息。

Result: 在一个包含15轮对话、测试多轮推理、回指和结构化医疗查询领域转移的基准上，与使用相同底层模型的基线智能体相比，ContextForge在保持可比响应准确性的同时，提高了对话一致性并减少了令牌消耗。

Insight: 论文的创新点在于提出了“上下文回收”的概念，通过结构化查询、外部记忆和受控合成来选择性保留和复用长对话中的关键信息，这是一种无需扩大上下文窗口或重新训练模型即可扩展LLM长任务能力的实用方法。

Abstract: Large language models (LLMs) exhibit strong capabilities in short-context reasoning but degrade in performance over long conversational horizons due to context window limitations and inefficient token usage. We introduce ContextForge, a system for context recycling that maintains task-relevant information across turns by combining structured query generation, external memory retrieval, and controlled synthesis. The system enables efficient reuse of prior computation without relying on full context replay, reducing token overhead while preserving answer quality. We evaluate ContextForge using a 15-turn conversational benchmark that tests multi-turn reasoning, back-references, and domain shifts across structured healthcare queries. Compared to a baseline agent using identical underlying models, ContextForge demonstrates improved consistency and reduced token consumption, while maintaining comparable response accuracy. These results suggest that context recycling provides a practical approach for extending LLM capabilities in long-horizon tasks without requiring larger context windows or model retraining. Code and evaluation artifacts are available at https://github.com/Betanu701/ContextForge.

[6] Where Larger Models Excel: The Primacy of Constraint-Guided Reasoning cs.CLPDF

Guan-Yi Lin, Hen-Hsen Huang

TL;DR: 本文研究了大型语言模型在推理任务上优于小型模型的根本原因，发现性能差距主要源于约束引导推理能力。通过开发AdvCluster自动化框架，系统分析了数学、物理、化学和编程等领域的推理差异，构建了大型模型优势的分类体系。

Details

Motivation: 旨在探究大型语言模型在推理基准测试中持续优于小型模型的根本原因，揭示性能差距背后的具体推理能力差异。

Result: 在多个领域基准测试中，大型模型稳定领先：Qwen3-32B比Qwen3-8B平均提升6.43%，GPT-OSS-120B比GPT-OSS-20B平均提升7.38%。通过AdvCluster框架识别出大型模型具有稳定优势的问题类型。

Insight: 提出约束引导推理是大型模型的核心优势，包括识别显式和隐式约束、组织结构化推理、排除不可行路径和验证中间步骤的能力。AdvCluster框架为模型能力分析提供了可量化的自动化方法论。

Abstract: Larger language models consistently outperform smaller ones on reasoning benchmarks, yet the reasoning differences underlying this gap remain underexplored. Across benchmarks in mathematics, physics, chemistry, and programming, we observe stable performance gaps: averaged over datasets, Qwen3-32B outperforms Qwen3-8B by 6.43%, while GPT-OSS-120B exceeds GPT-OSS-20B by 7.38%. To study the reasoning differences behind these gains, we develop AdvCluster, an automated framework that identifies questions where the larger model shows a stable advantage, extracts fine-grained advantage descriptions from paired reasoning traces produced by larger and smaller models, and organizes them through semantic clustering with quantitative evaluation and selection guided by a reviewer model. Our analysis yields a systematic taxonomy of larger model reasoning advantages, spanning both common advantages that recur across domains and specialized advantages associated with particular domains. Across these patterns, a recurring theme is Constraint-Guided Reasoning: larger models are better at identifying explicit and implicit constraints, organizing them into structured reasoning, and using them to rule out infeasible paths and verify intermediate steps.

[7] From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Models cs.CL | cs.AI | cs.CV | cs.LG | cs.MMPDF

Haoxiang Sun, Tao Wang, Li Yuan, Jian Zhao, Jiancheng Lv

TL;DR: 本文首次系统性地综述了多模态大语言模型（MLLMs）中统一视觉-语言感知范式的演变。作者将MLLM感知形式化为一种内在的、类似人类先天感知的统一能力，提出了一个五阶段分类法来追溯其范式演进，并概述了当前挑战和未来研究方向。

Details

Motivation: 现有综述往往将视觉和语言感知分开讨论，缺乏从真正统一视角审视MLLMs中跨模态感知演变的系统性研究。本文旨在填补这一空白，为理解MLLMs的感知能力提供整合框架。

Result: 作为一篇综述论文，未报告具体定量结果，但系统梳理了MLLM感知范式的演进历程、各阶段的代表性方法和里程碑（如OpenAI的O系列和DeepSeek的R系列模型），并指出了该领域的发展趋势。

Insight: 创新点在于首次将MLLM感知形式化为一种内在的、统一的视觉-语言能力，并提出了一个五阶段的分类法来系统刻画其范式演变。这为理解和推动面向通用人工智能（AGI）的统一多模态智能研究提供了基础框架和路线图。

Abstract: Multimodal Large Language Models (MLLMs) have recently made remarkable progress in unifying vision-language understanding and reasoning, especially following the introduction of models such as OpenAI’s O-series and DeepSeek’s R-series, which have driven a paradigm shift toward perception-centric intelligence. However, there remains a lack of systematic surveys that examine perception from a truly unified vision-language perspective – one that treats vision and language as an inseparable modality. Existing reviews are often fragmented, focusing separately on either vision or language, and thus rarely capture the cross-modal evolution of perception as an integrated capability. To bridge this gap, we present the first systematic survey of unified vision-language perception in MLLMs. Specifically, we (1) formalize MLLM perception as an intrinsic, unified vision-language capability analogous to human innate perception, (2) introduce a five-stage taxonomy tracing the paradigm evolution of MLLM perception and survey representative methods and milestones at each phase, and (3) identify open challenges and outline promising research directions toward truly general, unified multimodal intelligence. We hope our study will provide both a foundational understanding and an actionable roadmap to foster further innovation on the path toward artificial general intelligence (AGI).

[8] Soft Token Alignment for Cross-Lingual Reasoning cs.CLPDF

Jiayi He, Jungsoo Park, Wei Xu, Alan Ritter

TL;DR: 本文提出了一种名为SOLAR的辅助目标函数，用于在监督微调过程中对齐跨语言的软标记表示，以英语为枢纽，旨在解决多语言大语言模型在不同语言间语义等价提示下产生不一致推理和答案的问题。

Details

Motivation: 多语言大语言模型在生成过程中，随着模型对离散输出标记的确定，会变得越来越语言特定，导致语义等价的推理路径在不同语言间出现分歧，这促使研究者寻找一种与任何单一词汇项或脚本关联较少的跨语言对齐信号。

Result: 在四个多语言推理基准测试中，SOLAR相比基础模型将准确率提升了高达+17.7个百分点，相比标准监督微调提升了+3.8个百分点，且在低资源语言上提升最大。

Insight: 论文的创新点在于提出通过软标记（概率加权的词汇嵌入混合）的连续表示来对齐跨语言语义信息，这有助于在多语言推理过程中保留共享的语义结构，并显著增强了最终层的跨语言相似性和减少了语言簇的可分离性。

Abstract: Multilingual large language models often produce inconsistent reasoning and answers for semantically equivalent prompts in different languages. Prior work suggests that intermediate representations can be relatively language-agnostic, but generation becomes increasingly language-specific as models commit to discrete output tokens. This is problematic because language-specific lexical choices can cause semantically equivalent reasoning paths to diverge across languages. These divergences motivate searching for a cross-lingual alignment signal that is less tied to any single vocabulary item or script. We propose SOLAR, an auxiliary objective for supervised fine-tuning that aligns soft-token representations across languages, using English as a pivot. Soft tokens are probability-weighted mixtures over the vocabulary embeddings, yielding continuous representations that can aggregate information from semantically related tokens across languages. We then align each non-English soft-token summary to its English counterpart in the shared embedding space. Across four multilingual reasoning benchmarks, SOLAR improves accuracy by up to +17.7 points over the base model and +3.8 over standard supervised fine-tuning, with the largest gains on low-resource languages. SOLAR also strengthens final-layer cross-lingual similarity and substantially reduces language-cluster separability, suggesting that aligning soft-token representations helps preserve shared semantic structure during multilingual reasoning.

[9] Utilizing Cognitive Signals Generated during Human Reading to Enhance Keyphrase Extraction from Microblogs cs.CL | cs.DL | cs.HC | cs.IRPDF

Xinyi Yan, Yingyi Zhang, Chengzhi Zhang

TL;DR: 该论文研究如何利用人类阅读时产生的认知信号（脑电图EEG和眼动追踪）来增强微博等短文本的关键词提取任务。通过将8个EEG特征和17个眼动特征融入基于注意力机制的AKE模型，实验表明认知信号能持续提升性能，其中EEG特征贡献最大，而多模态信号组合显示出部分互补性但也可能存在冗余。

Details

Motivation: 解决微博等短文本平台内容嘈杂、分散导致自动关键词提取困难的问题，现有研究仅用眼动信号存在生理、采集和解码限制，因此探索EEG信号是否能作为补充来提升AKE效果。

Result: 在ZuCo认知语言处理语料库上评估，认知信号（尤其是EEG特征）能一致提升AKE性能，EEG与眼动特征组合的结果介于两者之间，表明部分互补但可能存在冗余或噪声。

Insight: 创新点在于首次将EEG信号引入微博关键词提取，并通过将认知信号注入注意力层（soft-attention输入和self-attention查询向量）来减少模型结构对信号的扭曲；客观来看，多模态认知信号融合为短文本NLP任务提供了新的认知证据来源。

Abstract: Microblogging platforms generate massive amounts of short, noisy, and dispersed user content, making automatic keyphrase extraction (AKE) an important but challenging task. Prior studies have used eye-tracking signals to improve microblog-based AKE because such signals reflect readers’ attention to salient words. However, eye tracking alone is limited by physiological, acquisition, and feature-decoding constraints. To address this issue, we investigate whether electroencephalogram (EEG) signals can complement eye-tracking signals for AKE. Using the ZuCo cognitive language processing corpus, we select 8 EEG features and 17 eye-tracking features and incorporate them into microblog-based AKE models. To reduce possible distortion of cognitive signals by model structures, we inject these features into the input of the soft-attention layer and the query vectors of the self-attention layer. We then evaluate different combinations of cognitive signals across AKE models. The results show that cognitive signals produced during reading consistently improve AKE performance, regardless of feature combinations and model architectures. EEG features bring the largest gains, while combining EEG and eye-tracking features yields performance between the two individual signal types, suggesting partial complementarity but also possible redundancy or noise. These findings indicate that EEG signals provide useful cognitive evidence for microblog-based AKE and that multimodal cognitive signals deserve further investigation.

[10] Speaking Numbers to LLMs: Multi-Wavelet Number Embeddings for Time Series Forecasting cs.CL | cs.AIPDF

Defu Cao, Zijie Lei, Muyan Weng, Jiao Sun, Yan Liu

TL;DR: 本文提出TempoWave，一种即插即用的时间小波数字接口，用于改进大语言模型在时间序列预测中的数值嵌入。该方法通过多小波多尺度系数将每个标量观测值映射为数字级嵌入，从而在保持数值格式、数字身份和对归一化操作鲁棒性的同时，向LLM暴露细粒度局部波动和宏观全局结构。在五个上下文丰富的预测基准测试中，TempoWave一致优于标准数值标记化和替代嵌入接口，达到了新的最先进水平。

Details

Motivation: 大语言模型在上下文感知的时间序列预测中具有吸引力，但其面向语言的离散标记化和嵌入接口与连续数值不匹配，常常损害数值顺序和预测可靠性。

Result: 在五个上下文丰富的预测基准测试中，TempoWave一致改进了基于LLM的预测器，超越了标准数值标记化和替代嵌入接口，达到了新的最先进水平。

Insight: 创新点在于提出了一个基于多小波、多尺度系数的数字级嵌入接口，将数值以与Transformer兼容的形式进行编码，从而更好地将LLM的上下文推理与精确预测耦合。这揭示了数值接口是一个关键瓶颈，而原则性的多分辨率嵌入是有效的解决方案。

Abstract: Large language models (LLMs) are attractive for context-aware time series forecasting because they can integrate heterogeneous textual signals, yet their discrete, language-oriented tokenization and embedding interfaces are misaligned with continuous numerical values, often harming numerical ordering and forecasting reliability. We propose TempoWave, a plug-and-play temporal wavelet digit interface that maps each scalar observation into digit-wise embeddings constructed from multi-wavelet, multi-scale coefficients. By directly overriding standard token representations, TempoWave seamlessly exposes both fine-grained local fluctuations and macro global structures in a transformer-compatible form, ensuring that precise numerical formatting, distinct digit identity, and robustness to common normalization operations are maintained throughout the LLM pipeline. Experiments across five context-enriched forecasting benchmarks demonstrate that TempoWave consistently improves LLM-based forecasters over standard numeric tokenization and alternative embedding interfaces, achieving a new state-of-the-art. These results highlight the numeric interface as a key bottleneck and suggest that principled multi-resolution embeddings can better couple LLMs’ contextual reasoning with precise forecasting. Our code is available at https://github.com/DC-research/TempoWAVE and our model can be accessed at https://huggingface.co/Melady/TempoWAVE.

[11] Comparing BERT Sentence-Pair Classification and Few-Shot LLM Prompting for Detecting Threat and Solution Framing in German Climate News cs.CLPDF

Raven Adam, David Maier, Marie Kogler

TL;DR: 本文系统比较了两种自动化方法，用于检测德语气候新闻中句子层面的威胁框架和解决方案框架：一种是基于开放权重大语言模型（Llama 4 Maverick）的少样本提示方法，另一种是基于德语BERT模型（deepset/gbert-large）的句子对微调分类方法。两种方法均采用独立的二元分类器，并在一个包含440篇奥地利报纸文章、经过人工标注的语料库上进行评估。

Details

Motivation: 新闻媒体在塑造公众对气候变化的认知中起着核心作用，报道是强调威胁还是解决方案对受众参与和政策支持有可测量的影响。自动化检测这些句子层面的框架模式，可以使研究人员分析难以手动编码的大规模语料库。

Result: 在手动标注的德语气候新闻语料库上，微调的BERT分类器在威胁和解决方案任务上的F1分数均达到0.83，而基于LLM的分类器F1分数为0.78。消融研究证实，与单句输入相比，提供前一句作为上下文能显著提升BERT的分类性能。

Insight: 论文的创新点在于对微调编码器模型与提示生成模型在计算社会科学文本分类任务上进行了系统比较。客观来看，其方法上的亮点包括：在BERT模型中引入句子对分类以利用上下文信息，以及在LLM方法中结合思维链推理和带置信度评分的结构化输出。这为特定领域（如德语新闻）的小样本分类任务提供了实用的技术路线对比。

Abstract: News media play a central role in shaping public perceptions of climate change, and whether coverage emphasizes threats or solutions has measurable effects on audience engagement and policy support. Automated detection of these framing patterns at the sentence level would allow researchers to analyze large corpora that are infeasible to code manually. We present a systematic comparison of two approaches for classifying sentences from German-language climate news articles as threat-oriented, solution-oriented, both, or neither. The first approach uses few-shot prompting with an open-weights large language model (Llama 4 Maverick), employing chain-of-thought reasoning and structured output with confidence scoring. The second approach fine-tunes a German BERT model (deepset/gbert-large) for sentence-pair classification, where the preceding sentence provides contextual information for the target sentence. Both approaches implement two independent binary classifiers, one for threat framing and one for solution framing. We evaluate both methods on a corpus of 440 Austrian newspaper articles that were manually coded following a detailed coding scheme developed with domain experts. The fine-tuned BERT classifiers achieve an F1 score of 0.83 for both the threat and solution tasks, while the LLM-based classifiers reach an F1 of 0.78. An ablation study confirms that providing the preceding sentence as context improves BERT classification performance substantially compared to single-sentence input. These results contribute to the growing body of work comparing fine-tuned encoder models with prompted generative models for text classification in computational social science.

[12] The Inattentional Gap: Task-Conditioned Language and Vision Models Omit the Safety-Critical Signals They Can Otherwise Report cs.CL | cs.AI | cs.CVPDF

Kwan Soo Shin

TL;DR: 该论文揭示了任务条件化语言和视觉模型存在一种‘疏忽性差距’现象：当模型被限定于特定任务时，会抑制其报告共现的安全关键信号，而这些信号在无约束条件下本可被报告。这种现象在放射学和驾驶文本场景以及胸片视觉任务中普遍存在，且不随模型规模减小，导致基准测试安全性与实际安全性脱节。

Details

Motivation: 论文旨在解决AI安全评估中的一个关键问题：现有评估仅关注模型检测指定危险的能力，但实际事故往往源于未被明确指定的危险，这暴露了基准测试安全性与真实世界安全性之间的脱钩风险。

Result: 在放射学文本、驾驶文本和胸片视觉任务中，所有测试模型均表现出信号抑制现象；该现象不随模型规模减小，在推理模型中持续存在，且受模型架构影响大于规模影响；相同模型在无约束条件下报告这些信号的比率显著更高。

Insight: 创新点在于提出了‘疏忽性差距’这一概念，揭示了任务条件化导致的系统性盲区，挑战了仅依赖指定危险评估安全性的范式；客观分析认为，这强调了在安全关键系统中需设计更全面的评估方法，以捕捉模型在开放环境中的真实行为。

Abstract: AI safety is evaluated by how reliably a model detects the hazards it is told to find, yet accidents often arise from the hazard no one specified. We show that conditioning a language or vision model on a narrow task suppresses its reporting of co-present, safety-critical signals it can otherwise report, a machine analogue of human inattentional blindness arising from a different mechanism. Across radiology and driving text scenarios and chest-radiograph vision tasks, suppression appeared in every model tested, did not diminish with scale, persisted in a reasoning model, and varied more by model family than by size, while the same models reported these signals at substantially higher rates when unconstrained. We name this dissociation the Inattentional Gap and argue that it decouples measured benchmark safety from real-world safety: a system can score near-perfectly on the hazards an evaluation specifies while remaining blind to those that cause harm.

[13] \textsc{DiARC}: Distinguishing Positive and Negative Samples Helps Improving ARC-like Reasoning Ability of Large Language Models cs.CL | cs.AIPDF

Yuxuan Yang, Feiyang Li, Yile Wang

TL;DR: 本文提出了一种名为DiARC的方法，通过区分正负样本来提升大型语言模型在ARC类抽象推理任务上的性能。该方法借鉴偏好对齐思想，构建了三种负样本生成策略（输出级视觉变换、DSL级规则反转和任务特定规则编辑），以提供信息丰富的近似错误选项。实验表明，DiARC在多个ARC类基准测试上均能持续提升基线模型的性能。

Details

Motivation: 当前基于开源大模型的ARC类推理方法效果不佳，而依赖闭源模型则成本过高；现有工作主要关注数据增强和正样本监督，但作者认为解决ARC类问题还需要通过区分负样本来提升模型的推理能力。

Result: 在多个ARC类基准测试上的实验结果表明，DiARC方法能持续提升基线模型的性能。

Insight: 创新点在于将偏好对齐思想引入ARC类推理，并系统性地构建了三种负样本生成策略，使模型能从近似错误中学习，这为提升模型在少样本模式归纳和推理任务上的能力提供了新思路。

Abstract: The Abstraction and Reasoning Corpus (ARC;~\citealp{chollet2019measure}) contains tasks that require summarizing patterns from limited grid samples and predicting output grids. Recently, many large language model based approaches have attempted to transform it into a text-based reasoning task. However, methods based on open-source models have generally yielded unsatisfactory results, while those relying on closed-source models are too costly. Current efforts mainly focus on data augmentation, constructing ARC-like data for more comprehensive supervised fine-tuning. In this work, we argue that solving ARC-like problems requires not only \textit{positive} sample supervision but also the ability to improve model reasoning by distinguishing \textit{negative} samples. To this end, we draw on the idea of preference alignment and propose \textsc{DiARC}, a method that constructs preference pairs to enable the model to distinguish between them. Specifically, we propose three ways to construct negative samples, including output-level visual transformations, DSL-level rule inversion, and task-specific rule editing. The resulting negative samples provide informative near-miss alternatives while keeping the observed demonstrations unchanged. Experimental results across multiple ARC-like benchmarks show that \textsc{DiARC} consistently improves performance over baseline models. The code is released at https://github.com/szu-tera/DiARC.

[14] Zero-shot Tweet-Level Stance Detection Enhanced by External Knowledge and Reflective Chain-of-Thought Reasoning cs.CLPDF

Yiju Huang, Wenxian Wang, Lijun Zhou, Rui Tang, Xiao Lan

TL;DR: 该论文提出了一种名为KIRP的零样本推文级立场检测框架，旨在解决短文本上下文稀疏性以及隐含目标与文本内容关联性不足的挑战。该框架通过整合外部知识图谱进行数据增强，并利用反思性思维链推理来提取和验证隐含目标，同时采用立场感知对比学习和三层迭代原型网络来更好地区分’中立’与’不相关’标签。

Details

Motivation: 解决零样本推文立场检测中的两个主要问题：短文本的上下文稀疏性，以及隐含目标与文本内容之间关联性的建立。现有方法过度依赖外部知识而忽略了文本内部实体的语义线索，且难以区分’中立’和’不相关’的立场标签。

Result: 在SemEval-2016、WT-WT和自建的KIRP-D数据集上达到了最先进的性能。具体而言，在SemEval-2016上三分类F1得分为84.05%，在WT-WT和KIRP-D上四分类F1得分分别为84.99%和79.18%。

Insight: 创新点在于将外部知识图谱与文本内部实体重组相结合进行数据增强，并引入反思性思维链推理来显式处理隐含目标；通过立场感知对比学习和迭代原型网络实现了对’中立’与’不相关’标签的细粒度区分，同时构建了首个日语推文级立场检测数据集。

Abstract: Zero-shot tweet-level stance detection confronts two primary challenges: (1) mitigating the context sparsity inherent in short texts, and (2) establishing the relevance between implicit targets and textual content. While existing methods primarily focus on incorporating external knowledge, they neglect the intrinsic semantic cues embedded within key intra-textual entities. Furthermore, current models exhibit limited capability in determining the relevance of unseen targets to the given text, thereby struggling to differentiate between “neutral” and “irrelevant” stance labels. To address these issues, we first construct a four-class, multi-topic Japanese tweet dataset. To our knowledge, this is the first Japanese tweet-level dataset for stance detection. We then propose KIRP, a zero-shot stance detection framework. It integrates external knowledge with entity reorganization for data augmentation and employs prompt chaining for reasoning. Specifically, the framework incorporates knowledge graphs to supplement and reorganize key textual entities, while reflective Chain-of-Thought (CoT) reasoning extracts and validates implicit targets. To better distinguish “neutral” from “irrelevant” labels, we adopt stance-aware contrastive learning to capture discriminative features and design a three-layer iterative prototype network for fine-grained classification. Experimental results on SemEval-2016, WT-WT, and KIRP-D show that KIRP achieves state-of-the-art performance. KIRP obtains F1 scores of 84.05% (three-class) on SemEval-2016, and 84.99% and 79.18% (four-class) on WT-WT and KIRP-D, respectively.

Qinkai Zhang, Yanyan Zhao, Xin Lu, Yulin Hu, Pengtao Han

TL;DR: 本文介绍了SocialPersona基准，用于评估多模态大语言模型从用户社交媒体时间线中推断其偏好并用于个性化对话的能力。该基准基于171位普通用户的纵向多模态数据构建，包含文本、图像、时间戳及2,597个人工验证的偏好标签，支持用户画像构建和个性化回复生成两个任务。

Details

Motivation: 现有个性化语言模型评估主要关注模型能否回忆用户在对话中明确陈述的偏好，但更全面的个性化需要模型从用户自然留下的多模态痕迹中推断其真实偏好，这是一个更具挑战性的能力。

Result: 实验表明，专有和开源MLLMs能够识别广泛的兴趣领域，但在细粒度和近期兴趣上的性能下降，并且当需要利用推断出的画像进行个性化对话时，性能进一步退化。文本和图像提供了互补的偏好信号。

Insight: 创新点在于提出了首个基于真实用户纵向多模态社交媒体数据来评估MLLMs推断和利用‘显露偏好’能力的基准。该基准强调了跨模态、长时程用户建模的挑战，并为开发能推断并基于用户真实偏好行动的智能助手提供了衡量和推进工具。

Abstract: Personalized language-model assistants are often evaluated through a memory lens: can a model recall preferences users have explicitly stated in dialogue? More comprehensive personalization demands a harder capability – inferring what users care about from the multimodal traces they naturally leave behind. We introduce SocialPersona, a benchmark for evaluating whether multimodal large language models (MLLMs) can recover revealed preferences from longitudinal social-media timelines and use them in dialogue. Built from longitudinal timelines of 171 everyday, non-promotional social-media users, SocialPersona contains text, images, timestamps, and 2,597 human-verified preference tags across seven interest domains, separating stable interests from recent interests. It supports two tasks: constructing structured user profiles from multimodal context and generating responses aligned with inferred profiles. Experiments with proprietary and open-weight MLLMs show that models can identify broad interest domains, yet their performance drops on fine-grained and recent interests and degrades further when inferred profiles must be used to personalize dialogue. Together with evidence that text and images provide complementary preference signals, these results indicate that robust cross-modal, long-horizon user modeling remains a key challenge, and that SocialPersona can help measure and advance progress toward assistants that infer and act on revealed preferences.

[16] Beyond Logical Forms: LLM-Extracted Patterns for Fallacy Classification cs.CL | cs.AIPDF

Eleni Papadopulos, Firoj Alam, Giovanni Da San Martino

TL;DR: 本文提出了一种利用大型语言模型从谬误示例及其解释中归纳提取抽象逻辑结构和上下文语言线索的模式，以提升谬误分类性能的框架。研究通过零样本和少样本实验验证了该方法的有效性，并展示了其跨数据集的泛化能力。

Details

Motivation: 在信息爆炸时代，逻辑谬误常以微妙形式出现，导致自动化分类困难，因此研究结合抽象逻辑结构和上下文语言线索是否有助于谬误分类。

Result: 在不同LLM和零样本、单样本配置下，该方法相比零样本基线有显著统计改进，并优于竞争方法；跨数据集实验验证了其泛化能力，确立了数据驱动模式提取作为生成逻辑表示的有效方法。

Insight: 创新点在于利用LLM从谬误数据中归纳提取模式，将逻辑结构与语言线索结合，为逻辑表示学习提供了数据驱动的可泛化方法。

Abstract: In today’s fast-paced information era, logical fallacies, defined as defective patterns of reasoning, inevitably contribute to the growth of information disorder. However, often fallacies appear in nuanced forms that complicate automated classification. In this study, we investigate whether merging abstract logical structures with context-level linguistic cues proves beneficial for fallacy classification, developing a framework that inductively extracts such patterns from fallacious examples and their explanations using Large Language Models (LLMs). We evaluate the impact of these patterns across different LLMs and experimental zero- and one-shot configurations, showing statistically significant improvements over zero-shot baselines and outperforming competing approaches. Cross-dataset experiments validate generalization, establishing data-driven pattern extraction as an effective method for generating logical representations.

[17] OPID: On-Policy Skill Distillation for Agentic Reinforcement Learning cs.CLPDF

Shuo Yang, Jinyang Wu, Zhengxi Lu, Yuhao Shen, Fan Zhang

TL;DR: OPID是一种用于智能体强化学习的策略内技能蒸馏框架，它直接从已完成的策略轨迹中提取分层技能监督，包括捕获全局工作流的回合级技能和捕获关键时间步局部决策知识的步骤级技能，并通过关键优先路由机制将所选技能注入交互历史，结合结果优势进行策略优化。

Details

Motivation: 解决基于结果的强化学习在语言智能体中轨迹级稀疏奖励提供的中间决策指导不足，以及现有策略内自蒸馏方法依赖外部技能记忆或检索特权上下文导致维护成本高且在多轮交互中与当前策略诱导的状态分布不匹配的问题。

Result: 在ALFWorld、WebShop和基于搜索的QA基准测试中，OPID通常比仅基于结果的RL和现有技能蒸馏基线提高了智能体性能、样本效率和鲁棒性。

Insight: 创新点在于直接从策略轨迹中提取分层技能作为密集的、分布匹配的后见监督，并通过关键优先路由机制动态选择技能，将技能蒸馏优势与结果优势结合，保持了RL作为主要训练目标的同时引入了有效的中间监督。

Abstract: Outcome-based reinforcement learning provides a stable optimization backbone for language agents, but its sparse trajectory-level rewards provide little guidance on which intermediate decisions should be reinforced or suppressed. On-policy self-distillation offers dense token-level supervision, yet existing skill-conditioned variants often rely on external skill memories or retrieved privileged context, which are costly to maintain and can be mismatched with the state distribution induced by the current policy in multi-turn interaction. We propose \textbf{OPID} (\textbf{O}n-\textbf{P}olicy Sk\textbf{i}ll \textbf{D}istillation), a framework that extracts skill supervision directly from completed on-policy trajectories. OPID represents trajectory hindsight as hierarchical skills: episode-level skills capture global workflows or failure-avoidance rules, while step-level skills capture local decision knowledge at critical timesteps. A critical-first routing mechanism uses step-level skills when critical decisions are identified and falls back to episode-level skills as default guidance otherwise. The selected skill is injected into the interaction history, allowing the old policy to re-score the same sampled response under both original and skill-augmented contexts. The resulting log-probability shift yields a token-level self-distillation advantage, which is combined with the outcome advantage for policy optimization. OPID thus preserves RL as the primary training objective while introducing dense, distribution-matched hindsight supervision. Experiments on ALFWorld, WebShop and Search-based QA demonstrate that OPID generally improves agent performance, sample efficiency, and robustness over outcome-only RL and existing skill-distillation baselines. Our code is available at https://github.com/jinyangwu/OPID/tree/main.

[18] Information-Aware KV Cache Compression for Long Reasoning cs.CL | cs.AIPDF

Jushi Kai, Zhuiri Xiao, Alexandra Birch, Zhouhan Lin

TL;DR: 本文提出了一种名为InfoKV的KV缓存压缩框架，旨在解决大语言模型长推理任务中KV缓存过大的问题。该方法从信息论角度出发，结合预测不确定性和注意力分数来评估token的重要性，从而更有效地压缩缓存。

Details

Motivation: 现有KV缓存压缩方法主要依赖注意力权重来估计token重要性，但注意力仅捕捉上下文相关性，忽略了与预测不确定性和token信息量相关的信息论信号。

Result: 在Llama-3.1、Llama-3.2和DeepSeek-R1模型上的长上下文推理基准测试表明，InfoKV在长预填充和解码场景中均持续优于现有的基于注意力的KV压缩方法。

Insight: 创新点在于引入了“前向影响”这一前瞻性指标来衡量压缩token对未来上下文的影响，并提出了一个结合预测不确定性、层级表示演化和注意力分数的熵感知压缩框架，从信息论角度提升了token重要性评估的全面性。

Abstract: Reasoning capability has advanced rapidly in large language models (LLMs), leading to an increasing size of key-value (KV) cache in both prefilling and decoding stages. Existing KV cache compression methods mainly rely on attention weights to estimate token importance. While attention effectively captures contextual relevance, it overlooks complementary information-theoretic signals related to predictive uncertainty and token informativeness. In this paper, we revisit token importance from a forward-looking perspective and introduce \textit{Forward Influence}, a metric that measures how compressed tokens affect future contexts. Our analysis reveals that tokens selected by attention scores mainly influence nearby contexts, whereas tokens associated with high predictive uncertainty exhibit substantially stronger influence on distant future contexts. Based on the observation, we propose \textbf{InfoKV}, an entropy-aware KV cache compression framework that incorporates information-theoretic signals. It combines token-level predictive uncertainty with layer-wise representation evolution and integrates the resulting entropy scores with attention scores during reasoning. Experiments on long-context reasoning benchmarks with Llama-3.1, Llama-3.2, and DeepSeek-R1 demonstrate that InfoKV consistently outperforms existing attention-based KV compression methods in both long prefilling and decoding scenarios.

[19] GAVEL: Grounded Caption Error Verification and Localization cs.CLPDF

Zixian Gao, Atsushi Hashimoto, Kuniaki Saito

TL;DR: 本文提出了GAVEL任务，旨在联合解决图像-文本对的错误验证、解释和视觉证据定位问题，以应对视觉语言模型（VLMs）常产生的幻觉或不一致输出。作者还构建了相应的数据集和基准，并训练了一个监督基线模型进行评估。

Details

Motivation: 针对视觉语言模型（VLMs）经常产生文本与图像未正确对齐的幻觉或不一致输出，现有方法不仅需要检测这种错位，还需解释差异并定位其视觉证据。

Result: 实验表明，即使是强大的闭源模型在GAVEL任务上也表现不佳，而基于人工标注训练集训练的监督基线模型在定位和解释指标上均取得了持续改进。

Insight: 创新点在于将错误验证、解释和定位整合为一个统一的GAVEL任务，并提供了配套的数据集和基准，为模型的可解释性和对齐能力提供了可学习的监督信号。

Abstract: Vision-language models (VLMs) often produce hallucinated or inconsistent outputs, where text and images are not properly aligned. Addressing this issue requires not only detecting misalignment but also explaining the discrepancy and localizing its visual evidence. We introduce GAVEL (Grounded Caption Error Verification and Localization), a task that jointly addresses verification, explanation, and localization for image-text pairs. To support systematic evaluation, we also present a corresponding dataset and benchmark. We further train a supervised baseline on the human-annotated training split to assess whether GAVEL provides learnable supervision for these abilities. Experiments show that even strong closed-source models struggle on GAVEL, while the supervised baseline yields consistent improvements across grounding and explanation metrics.

[20] ReaORE: Reasoning-Guided Progressive Open Relation Extraction Empowered by Large Reasoning Models cs.CL | cs.AIPDF

Xin Lin, Liang Zhang, Guoqi Ma, Hongyao Tu, Jinsong Su

TL;DR: 本文提出了一种名为ReaORE的推理引导渐进式开放关系抽取框架，旨在解决开放关系抽取中模型对未见关系类型泛化能力不足的问题。该框架通过从粗到细的关系推理，先进行关系过滤以生成初始关系集，再进行关系预测以通过细粒度比较推理区分易混淆关系。

Details

Motivation: 当前开放关系抽取方法存在局限性：基于聚类的方法无法生成关系标签且泛化能力差，而直接使用大语言模型生成标签的方法又缺乏足够的判别能力来区分易混淆关系。ReaORE旨在克服这些缺点，实现更可靠的泛化。

Result: 在两个广泛使用的开放关系抽取数据集上进行的广泛实验表明，ReaORE的性能优于现有的基线方法。

Insight: 核心创新在于引入了从粗到细的渐进式推理机制，将关系抽取分解为关系过滤和关系预测两个阶段，结合了多方面的理解、基于嵌入的相似性以及细粒度的比较推理，以增强对未见关系的泛化能力和对易混淆关系的判别能力。

Abstract: Open Relation Extraction (OpenRE) requires a model to extract unseen relations between head and tail entities from unstructured text for real-world applications. The core challenge of OpenRE lies in achieving reliable generalization to unseen relation types. Current OpenRE approaches either employ clustering techniques, which cannot generate relation labels and suffer from poor generalization, or rely on direct relation label generation via Large Language Models (LLMs), which lack sufficient discriminative capacity to distinguish easily confused relations. To address these limitations, we propose Reasoning-guided progressive OpenRE (ReaORE), a framework for performing relation extraction through coarse-to-fine relation reasoning. Specifically, ReaORE consists of two key stages: (i) relation filtering, which reasons over multiple aspects to understand relations and instances, yielding an initial relation set, and further supplements and filters relations via embedding-based similarity to ensure the target relation is included; (ii) relation prediction, which aims to predict the target relations from the above set via fine-grained comparative reasoning to better distinguish easily confused relations. Extensive experiments on two widely used OpenRE datasets demonstrate that ReaORE outperforms existing baselines.

[21] Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization cs.CLPDF

Zhenhua Xu, Dongsheng Chen, Jian Li, Yitong Lin, Zhebo Wang

TL;DR: 本文提出了一种名为Psy-CoT的心理学基础思维链框架，用于提升通用角色扮演智能体的性能。该框架将推理过程分解为交互感知、心理共情和逻辑构建三个角色特定步骤，使模型能够动态思考角色背景而非简单模仿。同时，针对强化学习训练中奖励模型易被通用短语‘欺骗’的问题，提出了角色感知策略优化（RAPO）方法，通过基于角色档案-词元互信息的梯度加权来不对称地放大角色特定词元的重要性。

Details

Motivation: 当前基于监督微调的主流方法仅鼓励行为模仿，缺乏类人的深层内部思维过程，导致在分布外场景下泛化能力差。因此，需要一种能更忠实、深入刻画角色的新方法。

Result: 在CoSER、CharacterBench和CharacterEval基准测试上的实验表明，Psy-CoT优于现有的角色扮演思维链方法，且RAPO在不同模型规模上均持续超越GRPO等基线方法。

Insight: 创新点在于将心理学原理（如共情）结构化地融入LLM的推理过程，以及提出了一种新颖的、基于互信息的梯度加权机制（RAPO）来解决强化学习训练中奖励模型被通用短语‘欺骗’的固有问题，从而更有效地对齐角色忠实度。

Abstract: Building general-purpose role-playing agents that faithfully portray any character from a natural-language profile remains challenging. The dominant paradigm – supervised fine-tuning – encourages behavioral mimicry without deep, human-like internal thought processes, resulting in poor out-of-distribution generalization. Therefore, we propose \textbf{Psy-CoT}, a psychology-grounded chain-of-thought framework that decomposes pre-response reasoning into three role-specific steps – \emph{Interaction Perception}, \emph{Psychological Empathy}, and \emph{Logical Construction} – so that the model \emph{thinks dynamically} from the profile rather than merely mimicking surface patterns. While structured reasoning provides a foundation, it alone is insufficient; reinforcement learning is essential to further align the model with character fidelity. However, we observe that under LLM-based reward models, both generic phrases that hack the reward model and genuinely role-specific phrases receive identical gradient signals – this hacking accumulates over training, misleading the model into treating both as equally optimal choices. To address this, we propose \textbf{Role-Aware Policy Optimization (RAPO)}, which uses profile–token mutual information to weight gradients asymmetrically – amplifying role-specific tokens under positive advantage while attenuating them under negative advantage. Experiments on CoSER, CharacterBench, and CharacterEval demonstrate that Psy-CoT outperforms existing role-playing CoT methods, and RAPO consistently surpasses GRPO across multiple model scales.

[22] NuclearQAv2: A Structured Benchmark for Evaluating Domain-Science Competence in Large Language Models cs.CL | cs.AIPDF

Henry Shaowu Yuchi, Michal Kucer, Benjamin H. Sims, Selma Peterson, Emily Taylor

TL;DR: 本文介绍了NuclearQAv2，一个用于评估大语言模型在核工程领域能力的结构化基准测试。该基准包含约1240个问题-答案对，涵盖布尔、数值和语言三类问题，通过结合专家编写、现有数据集和LLM辅助生成的方式构建。研究评估了多种LLM，发现它们在事实性问题表现良好，但在定量推理和概念理解方面仍面临挑战。

Details

Motivation: 大语言模型在广泛任务中表现出色，但在高度技术性领域（如核工程）的可靠性仍需验证。核工程问题解决不仅需要事实知识，还要求定量推理和概念理解，因此需要系统性的评估基准。

Result: 在NuclearQAv2基准上评估了多种LLM，观察到不同任务类型间存在显著性能差异：模型在事实性问题表现良好，但定量推理和概念理解任务更具挑战性。该基准为技术领域LLM能力评估提供了可扩展的框架。

Insight: 创新点在于采用混合流水线（专家编写、现有数据集和LLM辅助生成）构建领域特定基准，并利用结构化提示实现自动化问题生成和评估，提高了基准构建的可扩展性。这为其他技术领域创建类似评估工具提供了借鉴。

Abstract: Large language models (LLMs) have demonstrated strong performance across a wide range of tasks, but ensuring their reliability in highly technical domains remains a significant challenge. In nuclear engineering, problem solving often requires not only factual knowledge but also quantitative reasoning and conceptual understanding. To address the need for systematic evaluation in this domain, we introduce NuclearQAv2, a benchmark for assessing LLMs on nuclear engineering knowledge. The benchmark comprises approximately 1,240 question-answer pairs spanning three categories: boolean, numeric, and verbal. NuclearQAv2 is constructed using a hybrid pipeline that combines expert-authored questions, existing datasets, and LLM-assisted generation from domain-specific technical corpora. By leveraging structured prompting for both automated question generation and response evaluation, the proposed framework enables scalable benchmark construction and evaluation. We evaluate a diverse set of LLMs using NuclearQAv2 and observe substantial performance differences across task types. While the models generally perform well on factual questions, quantitative reasoning and conceptual understanding remain considerably more challenging. These results highlight the importance of multi-faceted evaluation frameworks and establish NuclearQAv2 as a scalable benchmark for assessing LLM capabilities in technical domains.

[23] The Riddle Riddle: Testing Flexible Reasoning in Large Language Models and Humans cs.CLPDF

Bella Fascendini, Kathryn McGregor, Max D. Gupta, Thomas L. Griffiths

TL;DR: 本文提出了一个名为‘谜语谜题’的新范式，用于测试大型语言模型是否具备灵活的推理能力，还是仅仅依赖于对训练数据的模式匹配。通过比较LLMs和人类在真实谜语与字面谜题上的表现差异，研究发现LLMs在真实谜语上表现更好，但在需要字面解释的谜题上准确率显著下降，而人类则相反。

Details

Motivation: 旨在探究大型语言模型的准确表现是源于灵活的推理策略，还是仅仅基于对训练数据中表面特征的模式匹配。

Result: 在九个SOTA LLMs和100名人类参与者的实验中，LLMs在真实谜语上的准确率为84.9%，但在谜语谜题上仅为50.7%；人类则相反，在真实谜语上为50.5%，在谜语谜题上为80.5%。错误分析显示，LLMs在谜语谜题上的错误有90.8%是由于不当使用创造性推理所致。

Insight: 创新点在于设计了‘谜语谜题’这一测试范式，能够有效区分记忆检索与灵活推理；客观分析表明，LLMs可能更依赖表面特征和记忆，而非真正的策略性推理，这提示在评估模型能力时需要设计针对性测试以避免混淆。

Abstract: Humans flexibly adapt their reasoning strategies to the requirements of a given problem. Large language models (LLMs) have performed well on many cognitive tasks, however, it is unclear whether this accuracy is a result of pattern matching from training data or flexible reasoning. Here, we introduce a novel paradigm to test this question: the riddle riddle paradigm. Riddle riddles are word problems written to mimic popular riddles, but altered so their answers only require literal interpretations. Identifying correct answers requires looking past the structure of each question and flexibly apply different reasoning strategies based on the content. If LLMs respond to surface features, such as form, a riddle-like structure should cause models to use an inventive reasoning strategy even when a literal interpretation suffices. Alternatively, if LLMs reason based on content, they should flexibly switch strategies when appropriate. Across two experiments with nine state-of-the-art LLMs and 100 human participants, we show humans and LLMs fail on this paradigm in opposite directions. LLMs were far more accurate on genuine riddles than on riddle riddles (84.9% vs. 50.7%); whereas humans showed the reverse effect (50.5% vs. 80.5%). Error analysis shows that 90.8% of LLM errors on riddle riddles (the condition where they show diminished performance) were due to inappropriate use of inventive reasoning while only 57.6% of human errors on genuine riddles were due to overextending literal reasoning. Thus, while both groups make mistakes, reasoning mistakes are made more often by LLMs than by humans. Overall, LLMs’ strong performance on genuine riddles may reflect memory retrieval rather than flexible strategy selection, and without stimuli designed to elicit this contrast, it becomes easy to conflate LLM-generated outputs that look like reasoning with genuine reasoning.

[24] Forecasting With LLMs: Improved Generalization Through Feature Steering cs.CL | cs.LGPDF

Humzah Merchant, Bradford Levy

TL;DR: 该论文探讨了使用大型语言模型（LLM）进行预测任务时，如何通过特征干预来提升其泛化能力。研究发现，通过稀疏自编码器识别并放大与时间感知推理相关的内部特征，可以有效减少预测中的前瞻性偏差，同时保持模型的一般推理性能。

Details

Motivation: 论文旨在解决LLM在预测任务中可能依赖时间特定知识而非可泛化模式的问题，特别是前瞻性偏差，以提升其基于历史数据的泛化推理能力。

Result: 在多个预测任务上，通过干预时间感知特征，显著减少了前瞻性偏差，同时保持了模型的一般推理性能，而干预候选前瞻性偏差特征则未产生效果。

Insight: 创新点在于利用稀疏自编码器对LLM内部状态进行可解释性分析，并因果性地干预时间感知特征以引导模型进行更基于历史的推理，这为提升LLM在预测任务中的泛化能力提供了新方法。

Abstract: Successful forecasting involves identifying patterns between historical and future states of the world which generalize to future observations. We apply LLMs to a variety of forecasting tasks and inspect their internal states using sparse autoencoders to understand whether they appear to rely on time-specific pieces of knowledge versus generalizable patterns. Our analyses identify features associated with both time-aware reasoning and look-ahead-biased reasoning. We then apply the LLMs to an entirely different domain and intervene on these features. We find that amplifying time-awareness features substantially reduces look-ahead bias on forecasting prompts while preserving general reasoning performance. In contrast, steering the candidate look-ahead-bias features does not produce an effect. These results suggest that interpretable temporal features can be used to causally shift LLMs toward more historically grounded reasoning.

[25] Paved with True Intents: Intent-Aware Training Improves LLM Safety Classification Across Training Regimes cs.CLPDF

Jeremias Ferrao, Niclas Müller-Hof, Iustin Sîrbu, Traian Rebedea, Yftah Ziser

TL;DR: 本文提出安全分类器应将用户意图作为提示词与最终标签之间的显式信号进行建模。为此，作者构建了AIMS数据集，包含1724个困难安全提示及其意图描述与伤害标签，并评估了意图感知训练在监督微调、偏好学习、推理蒸馏和强化学习等多种训练范式下的效果。结果表明，意图建模是一种紧凑且高质量的信号，能提升安全分类器的鲁棒性。

Details

Motivation: 论文的动机在于认为现有的安全分类器应显式地建模用户意图，以作为连接输入提示和最终安全标签的关键信号，从而提升分类器对困难安全案例的理解和判断能力。

Result: 在构建的AIMS数据集上评估表明，基于意图的DPO训练优于SFT，意图条件蒸馏在多数师生模型对中优于纯推理蒸馏。使用GRPO直接奖励意图忠实性在五个外部安全基准上取得了最强的平均性能，且意图感知模型在推理延迟与F1分数上形成了帕累托前沿。

Insight: 论文的创新点在于明确提出并验证了将用户意图作为显式监督信号用于安全分类器训练的有效性。从客观角度看，其构建的AIMS数据集及在多种主流训练范式中系统评估意图感知训练的方法，为提升LLM安全分类的鲁棒性提供了新的、可泛化的技术路径。

Abstract: We argue that safety classifiers should model user intent as an explicit signal between the prompt and the final label. To study this, we introduce AIMS, a human-annotated dataset of 1,724 difficult safety prompts, each paired with an intent description and harm label. We use AIMS to evaluate intent-aware training across supervised fine-tuning, preference learning, reasoning distillation, and reinforcement learning. Despite its size, AIMS enables competitive safety classifiers across training regimes: DPO from model-generated intent errors improves over SFT, and intent-conditioned distillation outperforms reasoning-only distillation in most teacher-student pairs. Most notably, directly rewarding intent faithfulness with GRPO yields the strongest average performance across five external safety benchmarks, while our intent-aware models form the inference latency-F1 Pareto frontier. These results show that faithful intent modeling is a compact, high-quality supervision signal for more robust safety classifiers.

[26] Bridging Talk and Thought: Understanding Dialogue Dynamics Across Collaborative Problem-Solving Contexts cs.CL | cs.AIPDF

Zhengyuan Liu, Stella Xin Yin, Min-Yen Kan, Nancy F. Chen

TL;DR: 本文提出了一个用于分析协作问题解决情境中对话的概念框架，重点关注人机协作和多智能体协作中涌现的动态交互。该框架通过一个整合了认知/非认知问题解决与元认知调节机制的双层分层编码方案，解决了现有分析方法的关键局限。

Details

Motivation: 随着智能系统成为能够自主推理和战略协作的主动智能体，理解协作问题解决过程中的对话交互对于优化和评估此类伙伴关系变得越来越重要。现有分析方法存在局限，需要新的框架来捕捉协作的动态本质。

Result: 该框架在涵盖多个领域的九个数据集上证明了其有效性和泛化能力。分析表明，元认知调节是区分更深层次协作的关键因素。

Insight: 创新点在于提出了一个整合认知、非认知与元认知调节机制的分层对话分析框架，为理解协作动态提供了新视角，并强调了元认知在深度协作中的鉴别作用。

Abstract: We present a conceptual framework for analyzing dialogue in collaborative problem-solving contexts, with an emphasis on the emerging dynamics of human-AI and multi-agent collaboration. As intelligent systems become active agents capable of autonomous reasoning and strategic cooperation, understanding the dialogic interaction during collaborative problem solving is increasingly important for optimizing and evaluating such partnerships. Our framework addresses key limitations in current analytical approaches through a hierarchical two-layer coding scheme that integrates cognitive and non-cognitive problem solving with metacognitive regulatory mechanisms. We demonstrate its effectiveness and generalizability across nine datasets spanning multiple domains, and provide insights into how humans and agents coordinate their knowledge, skills, and efforts to solve complex problems, showing in particular that metacognitive regulation can be an essential discriminator of deeper collaboration.

[27] CARVE: Content-Aware Recurrent with Value Efficiency for Chunk-Parallel Linear Attention cs.CL | cs.AI | cs.LG | cs.NEPDF

Sayak Dutta

TL;DR: 本文提出CARVE（内容感知的价值高效循环模型），一种改进的线性注意力机制，通过仅在关键轴上进行擦除操作，解决了现有delta-rule架构（GDN-2）的三个耦合缺陷。该方法在保持与GDN-2初始化时比特一致性的同时，实现了更好的性能、更低的计算开销和参数效率。

Details

Motivation: 现有循环模型（如GDN-2）的擦除门仅基于输入令牌而忽略内存内容，导致内存盲目的门控机制，同时存在价值轴擦除掩码参数浪费和阻碍WY形式三角块求解器有效性的数学限制。

Result: 在1.3B参数规模、100B令牌训练数据下，CARVE在WikiText上达到15.72困惑度（比GDN-2降低0.18，4.5σ显著效应），在九个常识推理基准测试中领先所有循环基线，并在所有RULER检索探针上达到SOTA，同时仅增加0.4%吞吐开销、降低13%峰值内存和19%参数数量。

Insight: 核心创新是提出’仅在关键轴擦除’原则，这被证明是WY形式求解器保持有效的充要条件；通过重用已写入GPU内存的循环输出张量作为擦除门的免费内容信号，并用每个注意力头的单个标量替代逐值的写门投影，实现了参数效率和计算优化。

Abstract: Recurrent models must forget in order to remember, yet the state of the art decides what to erase without consulting what is stored – the gate sees only the arriving token, not the memory it is about to modify. This memory-blind gating is one of three coupled defects in the leading delta-rule architecture (GDN-2): the value-axis erase mask wastes parameters at the scale of the value projection, and – as we prove – mathematically prevents the WY-form triangular chunk solver that makes recurrent training competitive with Transformers. We introduce CARVE (Content-Aware Recurrent with Value Efficiency), which resolves all three problems through one principle: erase only on the key axis. This is provably necessary and sufficient for the WY-form solver to remain valid. Within it, CARVE reuses the recurrent output tensor – already written to GPU memory – as a free content signal for the erase gate, and replaces the per-value write-gate projection with a single scalar per head. At initialisation CARVE is bit-identical to GDN-2; any quality difference emerges from what the content gate learns. At 1.3B parameters trained on 100B tokens, CARVE achieves WikiText perplexity 15.72 (minus 0.18 vs. GDN-2, a 4.5-sigma effect), leads every recurrent baseline on nine common-sense reasoning benchmarks, and sets state of the art on every RULER retrieval probe – at 0.4% throughput overhead, 13% lower peak memory, and 19% fewer parameters. Six formal theorems cover memory capacity, Lyapunov stability, gradient flow, expressivity separation, Pareto-optimal chunk size, and hybrid optimality.

[28] LMs as Task-Specific Knowledge Bases: An Interpretability Analysis cs.CLPDF

Amit Elhelo, Amir Globerson, Mor Geva

TL;DR: 本文通过行为和机制分析，探讨了语言模型是否像知识库一样对同一事实的不同查询能返回一致结果。研究发现，语言模型以任务特定的方式编码知识：不同任务训练获得的事实知识通常不会在其它任务中共现，且参数定位实验表明同一事实的不同任务由不同的参数子集支持。此外，思维链推理的部分有效性源于激活了超出评估任务的任务特定参数。

Details

Motivation: 动机在于验证语言模型是否具备知识库的关键特性——对同一事实的不同查询能基于单一真实来源返回一致结果，从而评估其作为知识库的可靠性。

Result: 行为分析显示，在训练中，一个任务获得的事实知识常无法在其它任务中共现；参数定位实验揭示了同一事实的不同任务由不同的参数子集支持。这些结果表明语言模型的知识编码是任务特定的。

Insight: 创新点在于通过行为和机制分析揭示了语言模型知识编码的任务特定性，挑战了将其参数视为统一知识库的类比，并强调了模型知识与查询方式在参数空间中的交织，对语言模型事实知识的可靠性和可控性具有启示意义。

Abstract: Language models (LMs) capture large amounts of factual knowledge applicable to a wide range of tasks, motivating the view of their parameters as a knowledge base. An important property of knowledge bases is that different queries for the same fact return consistent results, drawing on a single source of truth. We investigate whether LMs satisfy this property through behavioral and mechanistic analyses. Our results suggest that they encode knowledge in a task-specific manner. Behaviorally, facts acquired on one task frequently fail to co-emerge on others during training. Parameter localization experiments suggest a mechanistic explanation, revealing distinct parameter subsets underlying different tasks for the same fact. Finally, we show that chain-of-thought reasoning draws part of its effectiveness from engaging task-specific parameters beyond those tied to the evaluation task. Our findings suggest that what the model knows and how it is asked are intertwined in parameter space, undermining the “knowledge base” analogy and carrying implications for the reliability and controllability of factual knowledge in LMs.

[29] Multilingual Reasoning Cascades Need More Context cs.CLPDF

Arnav Mazumder, Dengjia Zhang, Shuyue Stella Li, Yulia Tsvetkov, Niyati Bafna

TL;DR: 本文研究多语言推理中的翻译级联方法，提出一种无需训练的上下文感知翻译级联策略，通过在最终翻译模块中保留原始问题、英文翻译和推理轨迹，显著提升多语言推理性能。

Details

Motivation: 传统翻译级联方法在推理过程中存在信息丢失问题，如文化背景、语域和歧义消解线索等，导致错误传播，需要改进信息流设计以缓解这一问题。

Result: 在九个多语言基准测试（涵盖285种高、中、低资源语言）和三种骨干模型上评估，该方法在开放生成任务中均表现出显著增益，原始语言问题贡献了大部分有益上下文。

Insight: 创新点在于简单且无需训练的上下文增强翻译级联，强调在机器翻译级联中保留原始用户问题至流程末端，可作为缓解错误传播的默认策略，对多语言推理系统设计具有借鉴意义。

Abstract: Translation cascades for reasoning translate the query from another language to English, reason in English, and translate the answer back to the original language. This is a competitive approach to multilingual reasoning, but structurally lossy, since each stage discards information later stages may need, including cues for cultural grounding, register, and disambiguation. We examine the benefits of a simple and training-free intervention: a context-aware translation cascade, which additionally provides the original question, the English translated question, and the reasoning trace to the context of the final translation module. We evaluate gains across nine multilingual benchmarks including various task types, three backbone models, and 285 high-, mid-, and low-resource languages, and demonstrate strong gains for open-ended generation across models and resource regimes. We show that the original language question carries most of the beneficial context. Our study emphasizes the need to better design information flow in machine translation cascades for mitigating error propagation, and provides a simple and actionable default strategy: preserve the original user question until the end of the pipeline.

[30] Empowering GUI Agents via Autonomous Experience Exploration and Hindsight Experience Utilization for Task Planning cs.CL | cs.AI | cs.CV | cs.LGPDF

Tianyi Men, Zhuoran Jin, Pengfei Cao, Yubo Chen, Kang Liu

TL;DR: 本文提出了一种名为PEEU（规划经验探索与利用）的方法，旨在提升小型开源多模态大语言模型（MLLM）在GUI代理任务规划中的能力。该方法通过自主探索环境获取经验，并利用事后经验合成严格对齐的高层次训练数据，以解决现有模型规划能力弱和跨网站泛化能力有限的问题。

Details

Motivation: 动机在于，虽然小型开源MLLM相比商业大模型具有成本效益和隐私保护优势，但其在任务规划和跨网站泛化方面存在明显不足，因此需要一种方法来增强其任务分解和规划能力。

Result: 在真实世界基准测试上的实验表明，PEEU方法效果显著：其7B参数模型达到了30.6%的准确率，超越了规模大得多的Qwen2.5-VL-32B模型，展现了优越的性能。

Insight: 核心创新点在于提出了PEEU方法，通过自主探索和事后经验利用来构建高质量的训练数据。客观分析认为，其提出的任务分解层次分析框架（TDHAF）为系统研究组合泛化提供了新视角，并揭示了掌握低层次原子技能不等于具备高层次规划能力，而高层次任务训练能带来更强的分布外（OOD）泛化能力，这对提升小型MLLM的规划能力至关重要。

Abstract: Multimodal web agents can assist humans in operating repetitive GUI tasks, where effective task planning is essential for decomposing complex tasks into executable actions. While small open source MLLMs are cost efficient and privacy preserving compared with commercial large models, they suffer from weak planning and limited cross website generalization. To address these limitations, we introduce the planning experience exploration and utilization (PEEU) method, which autonomously explores environments to discover experiences and utilizes hindsight experience to synthesize strictly aligned, high level training data. To quantitatively analyze the generalization behaviors driving this performance, we propose the task decomposition hierarchical analysis framework (TDHAF) to systematically study compositional generalization across three task granularities: low, middle and high levels. Our analysis reveals that mastering low level atomic skills does not guarantee high level planning competence, while high level task training yields stronger OOD generalization. Experiments on real world benchmarks demonstrate PEEU’s superior effectiveness: our 7B model achieves 30.6% accuracy, outperforming the much larger Qwen2.5-VL-32B model. These demonstrate constructing hindsight high level tasks and leveraging experiences is crucial for OOD planning abilities of small MLLMs.

cs.CV [Back]

[31] DocArena: Turning Raw Documents into Controllable Training Environments for Document Search Agents cs.CVPDF

Jiamian Wang, Ruiyi Zhang, Tong Yu, Jing Shi, Samyadeep Basu

TL;DR: DocArena提出了一种全自动的数据构建流程，能够将原始的多模态文档集合转化为可控、可扩展的训练环境，用于训练文档搜索智能体。该流程利用多模态大语言模型进行视觉感知和结构化索引，并基于跨页信息分布生成推理密集型的问答对，同时通过级联质量保证操作确保数据质量。

Details

Motivation: 现有方法主要基于文本环境训练搜索智能体，难以构建可控、可扩展且能处理多模态数据的训练环境。论文旨在解决这一问题，满足多模态文档搜索和问答的实际需求。

Result: 在仅策略模型不同的统一评估框架下，在六个多模态文档场景和七个基于文本的问答基准测试中，使用DocArena数据训练的智能体在检索准确性和问答质量上均取得了最佳性能。

Insight: 创新点在于将视觉感知与策略模型解耦的智能体架构，允许基于文本的LLM作为多模态文档检索和问答的推理主干；以及通过分析跨页信息分布来构建推理密集型QA对的数据生成方法，增强了训练环境的可控性和有效性。

Abstract: Recent methods train search agents via reinforcement learning from (question, answer, evidence) tuples without requiring expert trajectories. The tuples serve as the training environment, and whose properties directly shape what search strategies and generalization abilities the agent can develop. While prior works have made encouraging progress in improving training data quality, existing environments remain predominantly text-based and existing approaches can struggle to construct training environments that are controllable, scalable, and account for multimodal data. Given this, we propose DocArena, a fully automated data curation pipeline building on the practical need for multimodal document search and question-answering. It transforms raw document collections into training environments for search agents without any human annotation. The pipeline first structures and indexes documents through MLLM-based visual perception, then profiles and leverage the cross-page information distribution to construct reasoning-intensive QA pairs, as well as performs cascaded quality assurance operations via MLLM. We introduce DocArena-79K with QA pairs from 8,336 documents spanning 16 domains and 49 languages. We further design a Doc-Search agent infrastructure that decouples visual perception from the policy model, allowing text-based LLMs to serve as the reasoning backbone for multimodal document retrieval and QA. Under a unified evaluation framework where only the policy model differs, experiments on six multimodal document scenarios and seven text-based QA benchmarks show that agents trained on DocArena data achieve the best performance on both retrieval accuracy and QA quality. Further analysis on agent search behaviors confirms the effectiveness and controllability of the constructed training environment.

[32] Beyond Single-Source Cognitive Taskonomy:Multi-Source Task Relations through fMRI Transfer Learning cs.CV | q-bio.NCPDF

Junfeng Xia, Wendu Li, Mengjiao Zhang, Jie Guo

TL;DR: 本研究将基于fMRI掩码重建的认知任务分类学从单源任务迁移扩展到多源任务迁移，分析了23种人类连接组项目任务状态之间的迁移关系。通过训练1127个任务特定和迁移模型，并使用布尔整数规划分析预算约束下的任务分配，揭示了运动任务集群的跨范式局限性以及工作记忆任务在全局分配目标下的高优先级。

Details

Motivation: 现有基于fMRI重建的任务分类学研究主要关注单一源任务到目标任务的单向迁移，未能充分探索多源任务之间的复杂关系。本研究旨在扩展任务分类学，以量化多源任务迁移关系并分析预算约束下的最优任务分配策略。

Result: 在23个HCP任务状态上进行了大量实验，发现单源迁移具有方向性且受范式结构影响（如运动范式内迁移良好）。多源迁移效果取决于源任务集的组合，布尔整数规划在多种监督预算下反复将直接监督分配给多个工作记忆任务状态。

Insight: 创新点在于将fMRI任务分类学从一对一扩展到多对一迁移分析，并引入布尔整数规划解决预算约束下的任务分配问题。研究发现工作记忆任务整合了感知、注意和执行过程，在全局优化中具有特殊重要性，而仅靠成对任务关系无法完全捕捉多源任务间的复杂依赖。

Abstract: Cognitive tasks are organized by shared and specialized neural processes. Masked fMRI reconstruction provides a common self-supervised objective for quantifying transfer relations among task states, but existing reconstruction-based taskonomies mainly study one-to-one transfer from a single source task to a target. Here, we extend an fMRI cognitive taskonomy from single-source to multi-source transfer across 23 Human Connectome Project task states and use Boolean Integer Programming (BIP) to analyze budget-constrained task allocation. We train 1,127 task-specific and transfer models. Single-source transfer is directional and paradigm structured: motor states transfer well within the motor paradigm but provide limited support to most non-motor targets, consistent with a shared sensorimotor execution system and effector-specific representations. Multi-source transfer depends on the composition of the source set, suggesting that many-to-one task relations are not fully captured by pairwise taskonomy alone. Across supervision budgets, BIP repeatedly allocates direct supervision to several 0-back and 2-back working-memory states, although these states are not consistently the strongest individual sources. This pattern may reflect the integration of perceptual, attentional, and executive processes in working-memory tasks. Together, these findings reveal a cross-paradigm-limited motor cluster and working-memory states with high priority under the specified global allocation objective. Our study extends reconstruction-based fMRI taskonomy from one-to-one transfer to many-to-one task relations and budget-constrained task dependencies.

[33] Layer-Specific Prompt Fusion Discovery via Differentiable Search in Vision Foundation Models cs.CVPDF

Xi Xiao, Xingjian Li, Yunbei Zhang, Cheng Han, Tianming Liu

TL;DR: 本文研究了视觉提示调优中图像与提示令牌的融合方案问题，提出了一种通过可微分架构搜索联合优化可学习提示及其融合方案的混合融合方法，并在多个数据集上验证了其有效性。

Details

Motivation: 动机是探究视觉提示调优中单一融合方案（如拼接或相加）是否最优，以及开发混合融合方案是否能带来性能提升。

Result: 在VTAB-1k、FGVC和HTA的34个数据集上的广泛实验表明，该方法在保持ViT主干冻结的情况下，相比VPT-Deep等基线方法取得了性能提升，并在精度、延迟和参数量之间取得了有利的权衡。

Insight: 创新点在于将融合方案选择建模为双层优化问题，并通过可微分搜索探索了拼接、相加、仿射变换和交叉注意力四种融合方案，揭示了混合融合能更有效地利用ViT的层语义，为视觉提示调优提供了新视角。

Abstract: Visual prompt tuning has emerged as a parameter-efficient fine-tuning approach for adapting large-scale Vision Transformers (ViTs) to downstream tasks. As its learnable prompts are applied in input and feature spaces, prior to jointly going through attention in transformer layers, the most commonly used scheme for fusing image and prompt tokens is concatenation or addition. In this paper, we aim to study a fundamental yet essential problem in visual prompt tuning: whether a single fusion scheme tends to yield better results, and whether that would be beneficial to develop a hybrid fusion scheme. To this end, we formulate the task as a bi-level optimization problem, and solve it leveraging differentiable architecture search. In this context, the learnable prompts and their fusion schemes are jointly optimized. To enrich the search space in the architecture search, we propose two additional fusion schemes, namely, affine transformation and cross-attention, in addition to concatenation and addition. Extensive experiments on 34 datasets spanning VTAB-1k, FGVC, and HTA show consistent gains over prompt-tuning baselines. With a frozen ViT backbone, our method delivers a favorable accuracy–latency–parameter trade-off compared with VPT-Deep and recent variants. Our findings reveal that how prompts fuse with image tokens plays a significant role in visual prompt tuning, and a hybrid fusion fashion can more effectively leverage layer semantics of ViTs, contributing a novel perspective for visual prompt-tuning research.

[34] GeMoE: Gating Entropy is All You Need for Uncertainty-aware Adaptive Routing in MoE-based Large Vision-Language Models cs.CVPDF

Chaoxiang Cai, Minghe Weng, Jie Li, Yibo Jiang, Longrong Yang

TL;DR: 本文提出了一种基于门控熵的不确定性感知自适应路由方法GeMoE，用于改进基于混合专家（MoE）架构的大规模视觉语言模型中的路由策略。该方法将令牌路由视为信息编码任务，通过最小描述长度（MDL）框架来建模模型复杂度与性能之间的权衡，从而根据输入令牌的复杂性自适应地选择专家数量，提高资源利用率。

Details

Motivation: 传统MoE方法采用Top-k静态路由策略，无法考虑输入的变化并自适应选择专家数量，导致资源利用次优。本文旨在通过动态路由优化，解决静态路由在适应性和效率上的不足。

Result: 在多种骨干网络和基准测试上，GeMoE实现了与原始静态路由相比99.5%的平均性能保持，同时将平均专家激活稀疏性提高了36.5%。

Insight: 创新点在于将令牌路由形式化为信息编码任务，利用MDL框架和门控熵来量化不确定性，从而实现自适应路由；这提供了一种理论驱动的动态路由方法，可有效平衡模型复杂度和性能，提升MoE架构的效率。

Abstract: With the increase in model parameters and training data, the instruction following and generalization capabilities of Large VisionLanguage Models (LVLMs) have been significantly improved. Based on the Mixture of Experts (MoE) architecture, LVLMs expand their parameter capacity while maintaining the inference cost. However, traditional MoE methods employ a Top-k static routing strategy, which fails to account for variations in the input and adaptively select the number of experts, resulting in suboptimal resource utilization. In this paper, we propose viewing token routing as an information encoding task, framing dynamic routing as a Minimum Description Length (MDL) problem in encoding By validating the connection between MDL and gating entropy in the MoE scenario, we introduce Gating Entropy-based Uncertainty-aware Adaptive Routing (GeMoE) for MoE. Unlike traditional static or heuristic-based dynamic routing methods, GeMoE explicitly models the trade-off between model complexity and performance. By using gating entropy to assess the complexity of tokens, GeMoE adaptively determines the number of experts each token should engage. On a wide range of backbones and benchmarks, our method achieves 99.5% average performance retention compared to the original static routing, while improving average expert activation sparsity by 36.5%.

[35] Beyond Aesthetics: Quantifying Information Loss in Turbid Scenes cs.CVPDF

Vasiliki Ismiroglou, Stefan H. Bengtson, Tasos Benos, Thomas B. Moeslund, Malte Pedersen

TL;DR: 该论文针对水下浑浊环境中计算机视觉模型性能下降的问题，提出了一个名为Turbid Underwater Baseline (TUB)的真实浑浊水下图像数据集，并引入了一种基于相位一致性图、对对比度不变的度量PCD，用于量化浑浊场景中的结构信息损失。研究表明，PCD与实例分割模型在真实和合成浑浊图像上的性能高度相关，而现有常用指标则相关性很弱或没有。

Details

Motivation: 水下环境在浑浊条件下能见度迅速下降，但这对计算机视觉模型的影响尚不清楚，且现有研究过度依赖可能无法真实反映信息损失的合成浑浊数据集，因此需要量化真实浑浊场景中的信息损失。

Result: 在提出的TUB数据集上，PCD度量与实例分割模型（如Mask R-CNN）的性能表现出强相关性，而传统指标（如SSIM、PSNR）则相关性很弱或无相关性，这表明PCD能有效评估浑浊场景下的信息损失。

Insight: 创新点在于引入了对对比度不变的PCD度量来量化结构信息损失，并构建了大规模的真实浑浊水下图像数据集TUB，为评估模型在恶劣视觉条件下的鲁棒性提供了新的基准和工具。

Abstract: Visibility in underwater environments degrades rapidly under turbid conditions, yet the effects on computer-vision models remain unclear. This issue is compounded by reliance on synthetic turbidity datasets, which may misrepresent real-world information loss. To address this gap, we introduce the Turbid Underwater Baseline (TUB) dataset, comprising 1,320 images captured under extreme turbidity and over 16,000 high-confidence ground-truth segmentation masks. We additionally propose PCD, a metric derived from phase congruency maps that is invariant to contrast and aims to capture the loss of structural information in real turbidity. We show that PCD correlates strongly with the performance of instance segmentation models on both real and synthetic turbid images, whereas common metrics in the field show weak to no correlation at all. The dataset and relevant code can be found on the project page: https://vap.aau.dk/pcd

[36] What Do Deepfake Benchmarks Measure? An Audit Using Frozen Self-Supervised Representations cs.CVPDF

Samuel Pagon, Yixuan Shen, Vishal Asnani, Feng Liu

TL;DR: 这篇论文对深度伪造检测基准进行了审计，质疑当前基准测试的有效性。作者使用冻结的通用自监督表示上的线性探针作为诊断工具，发现其性能与专门设计的检测器相当，表明许多基准主要奖励通用的模态理解能力，而非真正的取证理解。

Details

Motivation: 当前深度伪造检测器在基准测试上表现良好，但在实际应用中经常失败，作者担心基准测试可能无法反映真实世界的深度伪造威胁，从而驱动了错误方向的复杂检测器设计。

Result: 在视频、图像和音频三种模态的深度伪造基准测试中，基于通用自监督表示的线性探针性能都接近专门设计的检测器。此外，生成器级别的难度部分可以通过相同表示空间中的Frechet几何来解释。

Insight: 论文的创新点在于提出了一种简单有效的基准审计方法，揭示了当前基准可能过度依赖通用特征而非取证特征。这提示在评估检测器时，需要区分其性能是源于对伪造痕迹的识别，还是仅仅利用了通用的数据分布知识。

Abstract: As deepfake generators approach perceptual indistinguishability, reliable detection becomes critical. Yet, detectors that score well on benchmarks routinely fail in the wild. A concerning feedback loop has emerged: benchmarks drive increasingly complex, engineered detectors, yet if those benchmarks do not reflect real-world deepfakes, this complexity may be solving the wrong problem entirely. This raises a prior question: what are these benchmarks actually measuring? We conduct an audit of video, image, and audio deepfake benchmarks using a deliberately simple diagnostic. If a linear probe on frozen, general-purpose self-supervised representations can approximate the performance of a bespoke detector, the benchmark is largely rewarding general modality understanding rather than forensic understanding. This has two implications: the benchmark may not reflect realistic threat models, and it raises the question of whether the bespoke detectors the probe approaches are truly learning forensic understanding. We observe, across three modalities, linear probes on general-purpose self-supervised representations closely approach the performance of bespoke detectors. We further show that generator-level difficulty is partly explained by Frechet geometry in the same representation space. Together, these results support a benchmark-audit view of deepfake detection: before high scores are read as evidence of forensic understanding, it is worth asking how much of the benchmark is already solved by general-purpose representations.

[37] Neural Voxel Dynamics: Learning Implicit 3D Physics via Volumetric Feature Advection cs.CVPDF

Zican Wang, Niloy Mitra

TL;DR: 本文提出了一种自监督框架Neural Voxel Dynamics，用于从视频中学习隐式的3D物理动力学。该方法通过将2D图像空间的特征提升到3D体素化潜在空间，并利用体素特征平流学习动作条件的状态转移算子，从而在单一统一流程中模拟异质物理现象（如刚体在流体中的运动）。

Details

Motivation: 当前生成式视频模型虽然视觉保真度高，但缺乏3D几何基础，导致物理不一致性和物体持久性缺失。本文旨在通过将预测瓶颈从2D图像空间转移到3D体素潜在空间来解决这一问题。

Result: 在多个基准测试（CLEVERER, PhysInOne, PhysGaia）上，仅通过端到端的视频信号和动作条件进行监督（无需物理引擎内部状态、标签或代理模型），该模型展示了良好的长期结构稳定性和物理合理性。

Insight: 核心创新在于将物理建模为时空状态平流问题，通过体素特征平流在隐式高维特征中学习3D物理动力学，实现了对异质物理现象的统一模拟，为仅通过单目视频被动观察构建通用动态世界模型提供了可扩展的途径。

Abstract: We present a self-supervised framework for learning implicit 3D physical dynamics directly from video-derived supervisory signals. While current generative video models achieve high visual fidelity, they lack a 3D geometric foundation, often resulting in physical inconsistencies and a failure to maintain object permanence. We address this by shifting the predictive bottleneck from 2D image space to a `lifted’ 3D Volumetric Latent Space. Our method unprojects semantic features from a Video Joint-Embedding Predictive Architecture (V-JEPA) into a voxelized grid, grounded by monocular depth priors. This lifting enables a Volumetric Feature Advection to learn an action-conditioned transition operator that treats physics as a spatio-temporal state advection problem, i.e., learn implicit 3D physics. Unlike state-of-the-art hybrid models that rely on explicit classical simulators for training and/or inference, our architecture tracks material states implicitly within high-dimensional V-JEPA features. This allows for the emergent simulation of heterogeneous phenomena (e.g., rigid body motion in fluid flow) within a single, unified pipeline. Supervised solely via end-to-end video-derived signal plus action conditions, without access to physics engine internal states, labels, or surrogate models, our model demonstrates good long-term structural stability and physical plausibility on multiple benchmarks (CLEVERER, PhysInOne, PhysGaia). We believe that this work opens a scalable pathway toward general-purpose dynamic world models that internalize the 3D invariants of the physical world solely through passive observation of monocular videos.

[38] Staying VIGILant: Mitigating Visual Laziness via Counterfactual Visual Alignment in MLLMs cs.CV | cs.CL | cs.LGPDF

Xi Xiao, Chen Liu, Chih-Ting Liao, Yunbei Zhang, Qizhen Lan

TL;DR: 该论文提出了一种名为VIGIL的强化学习后训练框架，旨在解决多模态大语言模型（MLLMs）中的视觉懒惰问题，即模型过度依赖语言先验而忽视视觉证据导致的幻觉。该方法通过引入几何约束，最大化视觉输入与生成响应之间的互信息，惩罚模型在反事实盲态下的不当置信。实验表明，VIGIL在幻觉和推理基准上优于现有对齐方法，且仅需25%的偏好数据即可达到SOTA方法的性能，并展现出无需显式监督的空间定位能力。

Details

Motivation: MLLMs虽然继承了LLMs的强大推理能力，但容易产生与视觉输入矛盾的幻觉。机制研究表明，这源于视觉懒惰：模型内部编码了正确的视觉证据，但在响应时过度依赖语言先验。现有的基于文本结果奖励的对齐方法（如DPO）会引入优化偏差，导致响应常与视觉证据矛盾。

Result: 在幻觉和推理基准（如MME、MMBench等）上的广泛实验表明，VIGIL一致优于近期对齐方法（如DPO、RLHF-V），且不损害纯文本能力。仅使用25%的偏好数据即可匹配SOTA方法的全数据性能，并在无显式边界框监督下展现出涌现的空间定位能力。

Insight: 核心创新点是将对齐焦点从数值奖励拟合转向因果视觉基础，通过最大化视觉-响应互信息的几何约束来直接解决视觉懒惰。具体通过惩罚反事实盲态下的“盲目置信”来实现，这是一种新颖的、基于信息论的强化学习训练目标，能有效促进模型对视觉证据的利用。

Abstract: Multimodal large language models (MLLMs) extend large language models (LLMs) with visual perception, enabling joint reasoning over images and text. Despite inheriting strong reasoning capabilities from LLMs, they remain prone to hallucinations that contradict their visual inputs. Mechanistic studies indicate that this weakness stems from visual laziness: MLLMs encode the correct visual evidence internally, but overly rely on strong language priors during response. Existing alignment methods, such as direct preference optimization, primarily optimize outcome-level rewards based on text. This introduces an optimization bias toward linguistic shortcuts, leading to responses that often contradict the visual evidence. To address this, we propose Visual Information Gain In aLignment (VIGIL), a reinforcement-learning (RL) post-training framework that shifts the focus from numerical reward fitting to causal visual grounding. VIGIL introduces a geometric constraint that explicitly maximizes the mutual information between the visual input and the generated response. We achieve this by penalizing “blind confidence” instances where the model remains improperly certain even when textual-visual attention is masked to create a counterfactual blind state. Extensive experiments show that VIGIL consistently outperforms recent alignment methods across hallucination and reasoning benchmarks without compromising text-only capabilities. Our approach matches the full-data performance of state-of-the-art methods using only 25% of the preference data and even demonstrates emergent spatial grounding capabilities without explicit bounding box supervision.

[39] Methane-Plume Segmentation From Hyperspectral Satellite Imagery Via Multimodal Deep Learning cs.CVPDF

Brayan Quintero, Jeferson Acevedo, Samuel Traslaviña, Hoover Rueda-Chacón

TL;DR: 本文提出了一种用于高光谱卫星图像中甲烷羽流分割的多模态深度学习模型，该模型通过特征引导的甲烷增强机制，在多个语义尺度上将物理上有意义的甲烷线索注入到基于Transformer的RGB表示中。

Details

Motivation: 高效检测甲烷羽流对于理解和缓解全球变暖至关重要，因为在地球观测图像中准确识别和分割甲烷羽流对于大规模监测是必不可少的。

Result: 该方法在MPDataset上进行了评估，在MIoU、MPrecision和Recall指标上分别比现有最佳方法提升了+0.92、+0.87和+1.01，并且计算成本显著低于其他高性能架构，实现了有利的精度-效率权衡。

Insight: 创新点在于提出了特征引导的甲烷增强机制，将物理先验知识（甲烷线索）与深度学习模型（Transformer）进行多尺度融合，这为遥感应用中实现准确且可扩展的甲烷羽流分割提供了一种高效的多模态融合策略。

Abstract: Efficient detection of methane plumes is crucial for understanding and mitigating global warming, as accurately identifying and segmenting them in earth observation imagery remain essential for large-scale monitoring. In this work, we propose a multimodal deep learning model that integrates a feature-guided methane enhancement (FGME) mechanism which injects physically meaningful methane cues into transformer-based RGB representations at multiple semantic scales. Our method is evaluated on the MPDataset, where it outperforms the state-of-the-art with improvements of +0.92 in MIoU, +0.87 in MPrecision and +1.01 in Recall. Notably, these gains are obtained with a substantially lower computational cost than other high-performing architectures, resulting in a favorable accuracy-efficiency trade-off for large-scale methane monitoring. These results highlight the potential of efficient multimodal fusion strategies for accurate and scalable methane plume segmentation in real-world remote sensing applications.

[40] Forget, Anticipate and Adapt: Test Time Training for Long Videos cs.CVPDF

Rajat Modi, Sebastian Noel, Xin Liang, Yogesh Singh Rawat

TL;DR: 本文提出了一种用于长视频的测试时训练（TTT）方法，称为帧遗忘网络（FFN）。该方法通过仅处理滑动窗口中的三个关键帧（即将离开的帧、当前帧和下一帧）来高效保留时序上下文，并引入一个惊喜度量来动态调整有效窗口大小，从而显著降低计算成本，使其能够处理长达数小时的视频。

Details

Motivation: 现有TTT方法在处理长视频时存在两大问题：一是使用包含过去帧的滑动窗口进行更新，计算量随窗口大小线性增长，对长视频计算不可行；二是即使时间上接近的帧相似时也进行TTT更新，浪费计算资源。

Result: FFN在密集分割和视频分类任务上展示了实证有效性，并能泛化到深度估计任务。在长达3小时的新数据集EpicTours上进行了评估，而此前该问题的数据集仅5分钟长，证明了其处理多小时长视频的能力。

Insight: 核心创新点在于设计了仅处理三个关键帧的高效滑动窗口机制，以及基于惊喜度量的自适应窗口算法，这大幅降低了长视频TTT的计算复杂度，同时保持了时序建模能力。新构建的长视频数据集EpicTours也为该领域研究提供了重要基准。

Abstract: Test Time Training (TTT) is a mechanism in which a model adapts to an incoming test-sample by performing some self-supervised (SSL) task and updating its weights even during inference. This procedure does not require labels at test-time. This paper focuses on TTT for long-videos. A major concern with existing approaches is: 1) they perform TTT updates using a sliding window containing frames in the past, whose compute increases linearly with the size of window. This becomes computationally intractable when the videos are hours long. 2) TTT is performed even when temporally close frames look similar, thereby consuming a lot of compute. We present the Frame Forgetting Network (FFN) that: 1) operates on only three frames within the sliding window, namely the frame that exits, the current frame and the frame after that. The model still manages to retain temporal context and work for hours long-videos; 2) mathematically define a surprise metric: how much new information the incoming frame contains with respect to the past seen frame. This facilitates determining how to modify the effective window size during TTT and constitutes the core mechanism of an adaptive windowing algorithm. Additionally, we curate a dataset EpicTours containing up to 3 hour long videos of walking city-tours, whereas earlier datasets on this problem were only 5 min long. We demonstrate FFNs empirical effectiveness on dense-segmentation, video classification tasks, generalization to depth-estimation, and multi-hour long videos.

[41] From Hallucination to Grounding: Diagnosing Visual Spatial Intelligence via CRISP cs.CV | cs.AIPDF

Zhixing Li, Yinan Yu

TL;DR: 本文提出了CRISP，一种新颖的结构化诊断评估范式，用于评估视觉空间智能，通过度量3D场景图和预言机干预协议来解耦潜在推理能力与感知瓶颈，揭示了现有视觉语言模型在感知与推理之间存在系统性脱节。

Details

Motivation: 当前视觉语言模型（VLM）评估常将语言先验与真正的空间推理混为一谈，CRISP旨在通过一致性（隐式感知与显式推理的对齐）来解决这一问题，提供更精细的诊断。

Result: 诊断发现，专有模型拥有强大的潜在推理引擎，但存在度量估计不准确和未能有效利用其隐式结构表示的关键失败；而开源模型则因缺乏多跳组合推理能力而受到根本性瓶颈。

Insight: CRISP的创新在于从依赖语言先验的“猜测正确”转向真正的“感知、验证和推理”，为超越端到端后训练的多模态对齐提供了严格路线图，其结构化诊断方法可借鉴用于评估模型的内在能力。

Abstract: Current VLM evaluations often conflate language priors with genuine spatial reasoning. To address this, we introduce CRISP, a novel structural-diagnostic evaluation paradigm that assesses visual spatial intelligence through consistency, the alignment between implicit perception and explicit reasoning. Unlike traditional black-box QA, CRISP utilizes metric 3D Scene Graphs and an oracle intervention protocol to decouple latent reasoning capabilities from perceptual bottlenecks. This granular diagnosis uncovers a systematic perception-reasoning disconnect. Crucially, we reveal that while proprietary models possess robust latent reasoning engines, they suffer from inaccurate metric estimation and a critical failure to leverage their implicit structural representations. Conversely, open-source models remain fundamentally bottlenecked by their lack of multi-hop compositional reasoning. By shifting the focus from merely guessing correctly'' via language priors to genuinely perceiving, verifying, and reasoning,’’ CRISP offers a rigorous roadmap for multimodal alignment beyond end-to-end post-training. The code and dataset are available at https://github.com/iiyamayuki/CRISP-Bench.

[42] Active Adversarial Perturbation-driven Associative Memory Retrieval for RGB-Event Visual Object Tracking cs.CV | cs.AI | cs.LGPDF

Xiao Wang, Xufeng Lou, Zikang Yan, Lan Chen, Sibao Chen

TL;DR: 本文提出了一种名为APRTrack的层次化扰动与检索框架，用于提升RGB-事件融合视觉目标跟踪在目标部分缺失和模态退化情况下的鲁棒性。该框架通过对抗性扰动模拟模态级和空间级的信号退化，并设计了一种分层路由机制来避免特征崩溃。同时，引入基于足迹引导和通道校准的Hopfield检索模块，以实现可控的历史特征补偿。

Details

Motivation: RGB-事件跟踪融合了RGB外观纹理和事件传感器的密集时序运动线索以提升定位鲁棒性，但在现实场景中，遮挡、边缘截断和前景杂波等结构化信号退化会损害传统多模态融合的可靠性，尤其是在恶劣环境下，任一模态都可能严重失效，导致目标不完整。

Result: 在FE108、COESOT、VisEvent和FELT等多个RGB-事件跟踪数据集上进行了广泛实验，验证了所提策略的有效性。

Insight: 创新点在于设计了两种对抗性扰动分支来模拟现实信号退化（全模态失效和局部目标区域缺失），并通过分层路由机制解耦其训练流程以防止特征崩溃；此外，提出了足迹引导的通道校准Hopfield检索模块，通过评估检索置信度和校准度量空间，实现了对目标区域边界内历史特征的可控补偿。

Abstract: RGB-Event tracking improves localization robustness by fusing RGB appearance textures and dense temporal motion cues from event sensors. While this multi-modal scheme broadens tracking applicability, real-world scenes suffer diverse structured signal degradations that hinder traditional multi-modal fusion. In harsh environments, either modality can lose reliability drastically, and targets frequently appear incomplete due to occlusion, edge truncation and foreground clutter.To tackle the above challenges, we present a hierarchical perturbation and retrieval framework tailored for RGB-Event tracking with robustness against partial target missing and modal degradation, termed APRTrack. To mimic real-world signal corruption, APRTrack constructs structured degradation via two adversarial perturbation branches at the modality and spatial levels, which separately simulate full-modal failure and localized target region absence. A hierarchical routing mechanism is designed to disentangle the training pipelines of the two perturbation types, effectively eliminating feature collapse induced by superimposed degradation constraints. Furthermore, we devise Footprint-guided Channel-calibrated Hopfield Retrieval (FCHR) for reliable historical information compensation. This module evaluates retrieval confidence based on association footprints between queries and memory banks, and calibrates the retrieval metric space prior to Hopfield matching, realizing controllable historical feature compensation bounded to target regions. Extensive experiments on FE108, COESOT, VisEvent, and FELT datasets demonstrate the effectiveness of our proposed strategies for the RGB-Event visual object tracking. The source code and pre-trained models will be released on https://github.com/Event-AHU/OpenEvTracking

[43] PhyEditBench: A Real-World Multi-Stage Benchmark for Physics-Aware Image Editing cs.CVPDF

Shengbin Guo, Shaokang He, Chaoyue Meng, Shengpeng Xiao, Xunzhi Xiang

TL;DR: 该论文提出了PhyEditBench，一个用于评估图像编辑模型物理理解能力的多阶段基准测试集，包含真实世界视频提取的238个实例和35个合成反物理实例，并揭示了当前SOTA编辑方法在物理推理方面的显著局限性。

Details

Motivation: 现有基于指令的图像编辑基准缺乏对物理推理能力的全面评估，而这是处理真实世界场景的关键能力，因此需要建立一个专门的基准来填补这一空白。

Result: 在PhyEditBench上对当前SOTA编辑方法的实证分析暴露了其在物理推理方面的重大缺陷；提出的免训练基线方法PhyWorld通过测试时缩放和潜在空间约简策略，性能优于同类模型。

Insight: 创新点在于构建了一个层次化分类（4个主类、12个子类）的真实世界物理感知编辑基准，并提出了视频生成过程可作为图像编辑推理机制的新见解，为模型物理理解能力的评估提供了新方向。

Abstract: While instruction-based image editing, enabled by multi-modal generative models, has advanced significantly, existing benchmarks lack a comprehensive evaluation of physics-based reasoning, a critical capability for handling real-world scenarios. To address this, we introduce PhyEditBench, a benchmark designed to assess the physical understanding of editing models. Guided by a hierarchical taxonomy, we establish 4 primary classes and 12 subclasses. It comprises 238 high-quality, high-resolution, real-world instances meticulously extracted from videos to capture authentic physical dynamics, alongside 35 synthetic Anti-Physics instances. Our empirical analysis of current SOTA editing methods exposes substantial limitations in their physics-based reasoning. We further propose a training-free baseline named PhyWorld that uses test-time scaling and a latent reduction strategy. PhyWorld outperforms comparable models and suggests that the video generation process can effectively serve as a reasoning mechanism for image editing. The project page is available at https://github.com/Previsior/PhyEditBench.

[44] Perception, Verdict, and Evolution: Hindsight-Driven Self-Refining Forensics Agent for AI-Generated Image Detection cs.CV | cs.AIPDF

Yangjun Wu, Keyu Yan, Yu Liu, Jingren Zhou, Fei Huang

TL;DR: 本文提出ForeAgent，一种用于AI生成图像检测的智能取证框架，采用感知-裁决架构聚合多视图线索，并引入后见之明驱动的自我精炼策略，通过采样-反思-进化范式实现持续自我进化。

Details

Motivation: 现有深度伪造检测方法面临两大局限：对细粒度取证伪影的敏感性不足，以及依赖前沿模型的静态合成监督，导致灵活性有限且成本高昂。

Result: 在Chameleon基准测试中达到82.18%的准确率（比AIDE提升16.41%），在AIGCDetect-Benchmark的16个生成器上平均准确率为93.3%，达到SOTA水平；外部评估显示其推理比GPT-5和GPT-5-mini更一致且因果基础更扎实。

Insight: 创新点包括感知-裁决架构融合语义、空间和频域特征，以及后见之明驱动的自我精炼策略，通过失败案例反思和双专家质量门控模块生成高质量训练数据，实现低成本持续进化。

Abstract: The rapid advancement of generative models presents a significant challenge to existing deepfake detection methods, particularly given the widespread dissemination of highly realistic AI-generated images. Although Multimodal Large Language Models (MLLMs) show strong potential for this task, existing approaches suffer from two key limitations: insufficient sensitivity to fine-grained forensic artifacts and reliance on static synthetic supervision from frontier models, leading to limited flexibility and high-cost. To address these issues, we propose ForeAgent, an agentic forensics framework for AI-generated image detection with iterative self-evolution. First, ForeAgent adopts a Perception-Verdict architecture that aggregates multi-view cues spanning semantic, spatial, and frequency-domain features, and leverages an MLLM as a verdict module to fuse these signals for a logical-grounded verdict. Second, to enable continual self-improvement, we introduce a Hindsight-Driven Self-Refining strategy following a Sampling-Reflection-Evolution paradigm. The agent performs inference rollouts on training instances. Guided by ground-truth labels as hindsight, it reflects on failure cases and low-quality reasoning trajectories to regenerate higher-quality reasoning traces. These synthesized samples are then strictly filtered through a dual-expert quality gating module. ForeAgent continuously evolves via fine-tuning on self-curated high-quality samples. Extensive experiments demonstrate that ForeAgent achieves state-of-the-art performance on the Chameleon benchmark, reaching 82.18% accuracy (+16.41% over AIDE), and achieves 93.3% mean accuracy on AIGCDetect-Benchmark across 16 generators. In addition, external evaluation shows that ForeAgent produces more consistent and causally grounded reasoning compared to GPT-5 and GPT-5-mini.

[45] Coarse-to-Fine: A Hybrid Self-Supervised Method for Non-rigid 3D Shape Matching cs.CVPDF

Feifan Luo, Ting Li, Zhao Li, Hongyang Chen

TL;DR: 本文提出了一种基于从粗到细策略的混合自监督方法，用于非刚性3D形状匹配。该方法采用双分支架构，结合拉普拉斯基和弹性基的功能映射学习流，确保粗映射与细化对应关系之间的一致性。

Details

Motivation: 解决非刚性3D形状匹配这一计算机视觉和图形学中的基础任务，特别是在非等距变形和拓扑噪声等挑战性场景下，现有方法可能存在的精度或效率不足的问题。

Result: 在多种挑战性场景（包括非等距变形和拓扑噪声）的实验中，该方法不仅保持了计算效率，还实现了最先进的性能（SOTA）。

Insight: 创新点包括：采用从粗到细的混合自监督策略和双分支（拉普拉斯基与弹性基）功能映射学习设计；通过对比能量提升特征判别性，且将其集成到现有方法中能带来一致改进，验证了方法的整体有效性。

Abstract: Non-rigid 3D shape matching is a fundamental task in computer vision and graphics. In this paper, we propose a hybrid self-supervised method based on a coarse-to-fine strategy, which ensures consistency between the coarse mapping and the refined correspondence produced by our refinement module. The architecture features a dual-branch design, consisting of two symmetric functional map learning streams: one based on the Laplacian basis and the other utilizing the elastic basis. Extensive experiments show that our approach not only maintains computational efficiency, but also achieves state-of-the-art performance across a variety of challenging scenarios, including non-isometric deformations and topological noise. Finally, we rigorously demonstrate that contrastive energies promote feature discrimination. Furthermore, integrating these energies with existing methods yields consistent improvements, validating the overall efficacy of our approach. Our code is available at https://github.com/LuoFeifan77/Coarse-to-Fine-Hybrid-Self-Supervised-Matching.

[46] DiCoBench: Benchmarking Multi-Image Fine-Grained Perception via Differential and Commonality Visual Cues cs.CVPDF

Geng Li, Yuxin Peng

TL;DR: 本文提出了DiCoBench基准测试，用于评估多模态大语言模型在高分辨率多图像细粒度感知方面的能力。该基准包含765个样本，分为差异性和共同性视觉线索两个渐进式任务，涵盖8种感知任务，采用高分辨率图像和多项选择题形式，旨在挑战现有MLLMs的自主感知能力。

Details

Motivation: 现有基准主要依赖显式文本线索或低分辨率输入，无法评估模型在高分辨率下自主感知隐式视觉线索的能力，因此需要构建一个更全面的多图像高分辨率基准来填补这一空白。

Result: 对18种不同MLLMs的广泛评估显示，与人类准确率（98.3%）相比存在显著性能差距，顶级模型在微尺度细节捕捉方面表现困难，表明当前SOTA模型在该基准上仍面临重大挑战。

Insight: 创新点在于通过差异性和共同性视觉线索的双轨设计，结合高分辨率图像和多项选择题形式，消除了评估指标偏差，为自主高分辨率多图像感知研究提供了具有挑战性的测试平台。

Abstract: Recent advancements in Multimodal Large Language Models (MLLMs) have demonstrated impressive fine-grained perception capabilities. However, existing benchmarks predominantly rely on explicit textual cues or low-resolution inputs, failing to evaluate a model’s ability to autonomously perceive implicit visual cues in high-resolution. To bridge this gap, we introduce DiCoBench, a comprehensive, multi-image high-resolution benchmark designed for cross-image fine-grained perception. DiCoBench consists of 765 meticulously curated samples categorized into two progressive tracks: Differential Visual Cues and Commonality Visual Cues, covering 8 distinct perception tasks. By formulating the benchmark as a multiple-choice question task and utilizing high-resolution imagery (approaching 2K), we eliminate evaluation metric bias and pose a substantial challenge to current state-of-the-art MLLMs. Our extensive evaluation of 18 diverse MLLMs reveals a striking performance gap compared to human accuracy (98.3%), with top-performing models struggling significantly with micro-scale detail capture. We believe DiCoBench will serve as a challenging testbed to drive future research in autonomous, high-resolution multi-image perception.

[47] TaskTok: Delving into Task Tokens for Task-driven Image Restoration cs.CV | eess.IVPDF

Hongjae Lee, Sojung Kang, Jaeseong Yu, Seung-Won Jung

TL;DR: 本文提出TaskTok框架，针对任务驱动的图像修复（TDIR）问题，通过分析潜在token空间发现任务相关线索在token序列中分布不均，因此设计可学习的token开关和轻量级token细化模块，仅选择性修复任务相关token，从而在提升下游高级视觉任务性能的同时保持高计算效率。

Details

Motivation: 传统图像修复关注感知质量，而任务驱动的图像修复（TDIR）旨在最大化下游高级视觉任务性能；现有基于生成先验的方法存在计算效率低和可能因无差别更新所有潜在token导致语义改变的问题。

Result: 在图像分类、语义分割和目标检测等任务上的广泛实验表明，TaskTok显著提升了任务性能，并具有高计算效率。

Insight: 创新点在于发现潜在token空间中任务相关线索的索引级专业化分布，从而提出选择性修复任务相关token的框架；客观分析认为其通过可学习token开关实现动态token选择，避免了不必要的计算，为TDIR提供了高效且专注的解决方案。

Abstract: While traditional image restoration focuses on perceptual quality, Task-Driven Image Restoration (TDIR) aims to maximize the performance of downstream high-level vision tasks. Recent approaches leveraging generative priors have shown promise for TDIR; however, they typically suffer from computational inefficiency and potential semantic alteration by indiscriminately updating all latent tokens. In this paper, we posit that not all visual information is equally important for machine perception. Through an analysis of the latent token space, we observe that task-relevant cues are unevenly distributed across the token sequence, exhibiting index-wise specialization. This suggests that selectively refining a subset of tokens can be sufficient for task-driven objectives. Leveraging this insight, we propose TaskTok, a novel framework that selectively restores only task-relevant tokens via a learnable token switch and a lightweight token refinement module. Extensive experiments across image classification, semantic segmentation, and object detection demonstrate that TaskTok significantly enhances task performance with high computational efficiency. The source code is available at https://github.com/jimmy9704/TaskTok

[48] Position Rebinding Cache Reuse: Replay-Free Visual Revisiting for Interleaved Multimodal Reasoning cs.CVPDF

Mengzhao Wang, Yanli Ji, Wangmeng Zuo, Peng Ye, Chongjun Tu

TL;DR: 本文提出了一种名为位置重绑定缓存重用（PRCR）的框架，用于解决交错多模态推理中视觉重访时的计算效率问题。该方法通过存储原始视觉键值缓存及其空间坐标，并在重用前重新分配位置兼容的坐标并重绑定键，从而避免了直接复制历史缓存导致的位置绑定失效和解码崩溃问题。

Details

Motivation: 现有方法在交错多模态推理中依赖令牌重放来重访视觉证据，这导致计算开销巨大；直接重用历史视觉键值缓存虽是一种捷径，但会因缓存键与原始位置上下文绑定而在后续解码中扭曲注意力并引发解码崩溃。

Result: 在多个多模态推理基准测试中，PRCR实现了与重放方法相当或更好的性能，平均准确率提升了5%，同时将视觉重访计算量减少了高达数万倍。

Insight: 创新点在于提出缓存级别的视觉重访框架，通过位置重绑定机制在重用历史视觉证据时保持文本位置连续性和相对视觉结构，从而在避免解码崩溃的同时显著提升计算效率。

Abstract: Interleaved multimodal reasoning improves visual grounding by revisiting visual evidence during multi-step generation, yet existing methods typically rely on token replay, repeatedly forwarding selected visual tokens. A natural shortcut is to reuse the historical visual key-value (KV) cache directly. However, we identify a critical failure mode of this strategy: cached visual keys are already bound to their original positional context. Such stale positional binding distorts attention under later decoding contexts and can trigger severe autoregressive decoding collapse. This failure suggests that effective cache reuse requires reconstructing visual evidence under positions compatible with the current decoding state, rather than directly copying position-bound historical cache entries. To this end, we propose Position Rebinding Cache Reuse (PRCR), a cache-level framework for replay-free visual revisiting. PRCR stores raw visual KV cache together with their original spatial coordinates, then reassigns position-compatible coordinates to select entries and rebinds their keys before injecting the reconstructed cache into the active decoder cache. This design reuses historical visual evidence while preserving textual positional continuity and relative visual structure. Experiments across multiple multimodal reasoning benchmarks show that PRCR achieves replay-level or better performance, improving average accuracy by 5 percent and reducing visual-revisiting computation by up to tens of thousands of times.

[49] Temporally Consistent Label Interpolation for Robust Surgical Multi-Task Learning under Challenging Conditions cs.CVPDF

Garam Kim, Juyoun Park

TL;DR: 本文提出了一种名为FAROS（Flow-guided Annotation for Robust Operating Scenes）的流引导标签插值框架，旨在解决手术场景多任务学习中因标注粒度不匹配（密集帧级时序任务与稀疏关键帧像素级空间任务）导致的监督不平衡问题。该方法结合零样本分割掩码传播与光流估计，在遮挡、烟雾等挑战性手术条件下生成时间一致性的密集伪标签，并集成到一个统一的基于Transformer的多任务框架中，联合学习手术阶段识别、步骤识别、预测、器械分割和动作识别。

Details

Motivation: 手术场景理解中的多任务学习因标注粒度不匹配而受到根本性阻碍：时序工作流任务（如阶段识别）需要密集帧级监督，而像素级空间任务（如器械分割）由于标注成本高昂仅稀疏标注于关键帧，这种监督不平衡破坏了共享表征学习并限制了跨异构任务的联合优化。

Result: 在DAVIS 2017基准测试中，采用稀疏真值协议验证了FAROS标签插值质量的鲁棒性，确认了其超越手术领域的传播能力。在GraSP、MISAW和AutoLaparo基准上的广泛实验进一步表明，FAROS显著改善了跨任务表征学习，并提升了跨时空任务的整体手术场景理解性能。

Insight: 论文的创新点在于提出了一种结合零样本分割与光流估计的流引导标签插值方法，以克服基于外观传播在挑战性手术条件下的局限性，从而生成时间一致的密集伪标签。从客观角度看，该方法通过平衡密集时序监督与稀疏空间监督，为多任务学习提供了一种有效的监督增强策略，有望推广到其他存在标注不匹配的视觉任务中。

Abstract: Effective multi-task learning for surgical scene understanding is fundamentally hindered by annotation granularity mismatch; temporal workflow tasks such as phase recognition, step recognition and anticipation benefit from dense frame-level supervision, whereas pixel-level spatial tasks including instrument segmentation and action recognition are only sparsely annotated on selected keyframes due to prohibitive labeling costs. This supervision imbalance undermines shared representation learning and limits joint optimization across heterogeneous surgical tasks. To address this, we propose Flow-guided Annotation for Robust Operating Scenes (FAROS), a flow-guided label interpolation framework, that combines zero-shot segmentation-based mask propagation with optical flow estimation to overcome the limitations of appearance-based propagation under challenging surgical conditions such as occlusion, smoke, and motion blur, generating temporally consistent dense pseudo labels from sparse keyframe annotations. The densified instrument masks and action labels are integrated into a unified Transformer-based multi-task framework that jointly learns surgical phase recognition, step recognition, anticipation, instrument segmentation, and action recognition, enabling balanced optimization between dense temporal supervision and sparse spatial supervision. The label interpolation quality of FAROS is first validated on the DAVIS 2017 benchmark under a sparse ground-truth protocol, confirming robust propagation beyond the surgical domain. Extensive experiments on GraSP, MISAW, and AutoLaparo benchmarks further demonstrate that FAROS significantly improves cross-task representation learning and enhances holistic surgical scene understanding performance across spatio-temporal tasks.

[50] FracEvent: Event-Camera Simulation via Fractional-Relaxation Pixel Dynamics cs.CV | cs.LG | eess.IVPDF

Langyi Chen, Chuanzhi Xu, Haoxian Zhou, Pengfei Ye, Ziyu Luo

TL;DR: FracEvent提出了一种基于分数松弛像素动力学的事件相机模拟器，通过建模像素级生命周期来改善生成事件的时间结构。该方法利用紧凑的松弛模式堆栈驱动电压状态，在连续电压轨迹上定位阈值交叉点来发射ON/OFF事件，并保留底层记忆模式以链接残余电压响应与后续事件时序。

Details

Motivation: 现有事件相机模拟器通常简化像素生命周期的时间结构，可能扭曲事件时序并削弱下游任务迁移效果，因此需要更精确的像素级动力学模型来提升模拟质量。

Result: 在多个数据集上，FracEvent通过事件流比较和下游任务（如图像重建与光流估计）迁移评估，改善了生成事件的时间结构，并取得了优于基线模拟器的下游迁移结果。

Insight: 创新点在于引入分数松弛电压动力学来建模像素生命周期，通过保留记忆模式链接残余电压与事件时序，从而更真实地模拟事件相机的时间响应特性。

Abstract: Event cameras asynchronously report brightness changes with microsecond-level temporal resolution, but real event data remain difficult to collect at scale because specialized sensors, careful synchronization, and task-specific annotations are required. Event-camera simulation is therefore important to event-based vision tasks. Most practical simulators build on contrast-threshold event generation, some with additional filtering, stochastic noise, or hand-tuned sensor parameters. While effective, such formulations often simplify the temporal structure produced by the lifecycle of each pixel, which can distort event timing and weaken downstream transfer. We introduce FracEvent, an event simulator that models this pixel-level lifecycle with fractional-relaxation voltage dynamics. Given a log-intensity trajectory, FracEvent drives a compact stack of relaxation modes, combines their responses into a voltage state, emits ON/OFF events by localizing threshold crossings on the continuous voltage trajectory, and updates the reference while retaining the underlying memory modes. This retained state links residual voltage response to later event timing. We evaluate FracEvent through event-stream comparison and downstream transfer on image reconstruction and optical flow estimation. Across multiple datasets, FracEvent improves the temporal structure of generated events and achieves stronger downstream-transfer results than competing simulator baselines, showing its practical value for event-camera simulation.

[51] LayersReg: A Layer-by-Layer Progressive Regressor for Reliable Intraoperative 3D/2D Registration cs.CVPDF

Xiyuan Wang, Zhenchao Wang, Xinran Chen, Junkai Liu, Chuan Chen

TL;DR: 本文提出了一种名为LayersReg的新型3D/2D配准方法，用于手术导航。该方法采用逐层渐进回归范式，赋予模型3D解剖结构感知能力，通过特征空间搜索移动图像与固定图像之间的相关性来迭代逼近正确空间位姿。实验表明，在大偏移和多模态条件下，该方法在X射线/CT配准和切片定位任务上均实现了高精度，超越了现有SOTA方法。

Details

Motivation: 传统迭代优化算法在术中配准中效率低、失败率高，而现有基于深度学习的方法本质上是记忆驱动的特定位姿特征检索，而非理解图像对齐任务本身，这限制了其在复杂场景下的泛化能力。

Result: 在X射线/CT配准任务上达到0.68°和1.41 mm的精度，在切片定位任务上达到0.73°和1.55 mm的精度，超越了现有最先进方法，满足了术中精度和实时性需求。

Insight: 核心创新在于将经典配准的迭代位姿搜索优化准则引入回归范式，通过逐层渐进的方式在特征空间搜索图像相关性，并耦合节点级回归以增强对空间位姿变化的感知，从而实现了对配准任务本身的理解而非单纯记忆。

Abstract: 3D/2D registration serves as a cornerstone technique in surgical navigation. Traditional iterative optimization algorithms suffer from low efficiency and high failure rates in intraoperative settings. Deep learning-based methods reformulate registration from iterative optimization to a regression problem that maps image appearance features to spatial pose, typically achieving improved real-time performance and accuracy. However, such learnable methods are confined to memory-driven retrieval of specific pose features rather than understanding the task of image alignment itself, which limits their generalization in complex scenarios. We propose LayersReg, a pioneering regression paradigm that endows the model with 3D anatomical awareness and searches for the correct pose in a progressive, layer-by-layer manner. Inspired by the iterative pose-searching optimization criterion of classical registration, LayersReg searches for correlations between the moving and fixed images in feature space, capturing the trend of pixel flow and thereby converging iteratively toward the correct spatial pose transformation. We further design a coupling of node-wise regression with the progressive registration framework to enhance the model’s perception of spatial pose changes. Experimental results demonstrate that under large offsets and multimodality conditions, LayersReg achieves high accuracy on both X-ray/CT registration (0.68°, 1.41 mm) and slice localization (0.73°, 1.55 mm) tasks, outperforming existing state-of-the-art methods while meeting the intraoperative demands for precision and real-time capability.

[52] Disco-LoRA: Disentangled Composition of Content, Style, and Motion for Multi-concept Video Customization cs.CV | cs.AIPDF

Xuancheng Xu, Gengyun Jia, Bing-Kun Bao

TL;DR: 本文提出Disco-LoRA，一个用于多概念视频定制的统一框架，旨在从参考数据中学习特定特征以生成可控视频。该框架通过两阶段方法解耦并灵活重组内容、风格和运动概念：首先将任务分解为内容-风格和内容-运动子任务，采用迭代双LoRA解缠框架；其次提出基于Z分数的统计正则化来协调LoRA权重，以保持层间趋势并最小化干扰。

Details

Motivation: 当前基于文本到视频（T2V）模型的视频定制在图像风格化和视频运动定制方面已取得进展，但同时对内容、风格和运动等多个概念进行联合控制仍是一个主要挑战。本文系统定义了多概念视频定制任务，旨在解决这一难题。

Result: 大量实验表明，Disco-LoRA在多概念视频定制任务中表现出色，能有效保留外观、风格和运动，实现可控的文本到视频生成。该研究构建了一个全面的基准测试来评估性能。

Insight: 创新点包括将多概念控制任务系统分解为两个子任务并采用迭代双LoRA解缠框架，以及提出基于Z分数的统计正则化来协调LoRA权重分布，从而在保持层间趋势的同时实现不同LoRA的灵活组合，减少干扰。

Abstract: Video customization based on Text-to-Video (T2V) models aims to learn specific features from reference data to generate controllable videos. While significant strides have been made in image stylization and video motion customization, simultaneously controlling multiple concepts, such as content, style, and motion, remains a major challenge. In this work, we systematically define the task of multi-concept video customization, which requires the joint control of content, style, and motion. To facilitate research in this area, we construct a comprehensive benchmark and propose Disco-LoRA, a unified framework designed to tackle this problem by disentangling and flexibly recombining different concepts in two stages: (1) We decompose the objective into two sub-tasks: Content-Style and Content-Motion. Each sub-task is addressed using our Iterative Dual-LoRA Disentanglement Framework, which effectively disentangles distinct concepts within the data. (2) We identify layer-wise weight trends as crucial for LoRA identity, while weight magnitudes dictate composability. To harmonize these scales, we propose a Z-score-based statistical regularization that aligns weight distributions, preserving layer-wise trends while minimizing interference between different LoRAs. Extensive experiments show that Disco-LoRA excels in multi-concept video customization, effectively preserving appearance, style, and motion for controllable text-to-video generation.

[53] PhysEditWorld: A Large-Scale Dataset Toward Physics-Editable World Models cs.CVPDF

Bin Hu, Yanwen Ma, Jiehui Huang, Ziliang Zhang, Haoning Wu

TL;DR: 论文提出了PhysEditWorld，一个专注于物理参数（特别是重力）的大规模多模态数据集，用于构建可物理编辑的世界模型。该数据集基于UE5的回放与渲染流水线构建，记录了多种重力配置下的游戏交互场景，包含超过6000万帧渲染画面和同步的多模态信号。初步实验表明，该数据集能提升生成视频模型和世界理解模型在物理动态建模与编辑一致性方面的性能。

Details

Motivation: 现有游戏世界模型虽能生成视觉上合理的动作序列，但其交互行为通常局限于探索或漫游轨迹，且物理动态多作为数据中的隐式关联学习，而非可控变量，这限制了其在需要显式操控物理规则的游戏创作环境中的应用。

Result: 在生成视频模型和世界理解模型上进行的初步效用研究表明，PhysEditWorld能够改进模型对重力的忠实动态建模，并增强在物理编辑下的生成一致性，为可控世界建模研究提供了可扩展的基础。

Insight: 核心创新在于通过标准化的回放范式构建了一个大规模、多模态、物理参数（重力）可控的数据集，实现了对同一初始状态和动作序列在不同物理配置下的可控、可归因的变体生成，为将物理规则作为显式、可控变量融入世界模型提供了数据基础。

Abstract: Recent game world models can synthesize visually plausible, action-conditioned rollouts. However, their interaction behaviors often remain limited to exploratory or wandering trajectories, and physical dynamics are typically learned as implicit correlations from data rather than as controllable variables. This limitation hinders their applicability to authored game environments, where physical rules are deliberately designed and require explicit manipulation. We introduce PhysEditWorld, a multimodal dataset with physical parameters, with a primary focus on gravity in this initial version. At its core, PhysEditWorld is built upon a replay paradigm implemented with a UE5 replay-and-rendering pipeline. Each scenario records a normalized action trace and replays the same initial state, character controller, action sequence, and camera policy under multiple gravity configurations, enabling controlled and attributable physical variation. PhysEditWorld contains 12 cinematic UE5 scenes, over 100 hours of gameplay interactions, and more than 60 million rendered rollout frames. Each sample provides synchronized multimodal signals, including RGB, depth, normals, audio, action traces, camera trajectory, engine states, semantic annotations, and explicit gravity labels. We further conduct initial utility studies on both generative video models and world understanding models, demonstrating that PhysEditWorld enables improved gravity-faithful dynamics modeling, enhances consistency under physical edits, and provides a scalable foundation for controllable world modeling research.

[54] Robust Onion: Peeling Open Vocab Object Detectors Under Noise cs.CV | cs.AIPDF

Priyank Pathak, Mukilan Karuppasamy, Aaditya Baranwal, Shruti Vyas, Yogesh S Rawat

TL;DR: 该论文提出了Robust Onion，一项针对开放词汇目标检测器在视觉噪声下鲁棒性的实证研究。通过使用可控的合成视觉退化，逐层剖析OV-ODs，揭示了鲁棒性如何、为何以及在何处退化，并系统分析了特征崩溃现象。研究发现，具有相似视觉主干网络的模型表现出可比的鲁棒性，其退化主要由相似层的特征崩溃驱动，而预训练策略、架构细节和标题监督等因素影响甚微。鲁棒性主要由图像域而非标注决定。最后，论文通过提出的轻量级即插即用方法NN&TK0，在真实数据集上验证了其见解，并解释了先前工作的鲁棒性观察。

Details

Motivation: 由于开放词汇目标检测器架构复杂，真实世界噪声对其影响尚不明确。本研究旨在系统分析OV-ODs在噪声下的鲁棒性退化机制。

Result: 研究揭示了模型鲁棒性与视觉主干网络强相关，在COCO和LVIS等基准上观察到相似的鲁棒性影响。提出的NN&TK0方法在BDD100K、WiderFace和VisDRONE等真实数据集上提升了鲁棒性，且可训练参数比端到端训练少96倍。

Insight: 核心创新在于通过可控合成噪声的逐层分析框架，揭示了OV-ODs鲁棒性退化的根本原因在于视觉主干网络的特征崩溃，而非下游任务设计。这为理解和提升模型鲁棒性提供了新的视角和轻量级解决方案。

Abstract: The impact of real-world noise on Open Vocabulary Object Detectors (OV-ODs) remains poorly understood due to their architectural complexity. We present our comprehensive analysis Robust Onion, an empirical study that uses controlled synthetic visual degradations to peel OV-ODs layer-by-layer, revealing how, why, and where robustness degrades, systematically analyzing feature collapse. Our findings reveal that models with similar vision backbones exhibit comparable robustness, driven by similar feature collapse at similar layers, while factors such as pretraining strategy, architectural nuances, and caption supervision contribute little. Robustness is primarily governed by the image domain rather than annotations, explaining the similar robustness impact on COCO and LVIS, and why datasets like ODinW-13 can give an impression of inflated robustness due to large, isolated objects. Finally, we validate our insights by improving robustness on real-world BDD100K, WiderFace, and VisDRONE via our lightweight plug-and-play NN & TK0 approach, using 96x fewer trainable parameters than end-to-end training. We also explain the prior works’ robustness observations.

[55] LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editing cs.CVPDF

Xinyu Wang, Chongbo Zhao, Fangneng Zhan, Yue Ma

TL;DR: 本文提出LiveEdit，一种面向实时交互场景的流式视频编辑框架，通过三阶段蒸馏流程将双向基础模型的编辑能力迁移至高效单向流式编辑器，并引入面向增强现实的掩码缓存机制以加速推理。该方法在保持编辑区域稳定性和背景一致性的同时，实现了12.66 FPS的实时处理速度，适用于增强现实等交互应用。

Details

Motivation: 现有流式视频编辑方法难以同时满足背景稳定性、非编辑区域保持以及低延迟实时交互的需求，且流式视频生成方法无法直接应用于编辑任务。

Result: 在专门构建的流式视频编辑基准测试中，该方法在流式基线中达到SOTA视觉质量，推理速度大幅提升至12.66 FPS。

Insight: 创新点包括：三阶段渐进式蒸馏实现长时程稳定编辑，面向AR的掩码缓存机制重用跨帧区域计算以减少冗余；从客观角度看，该方法将编辑任务与实时流式处理结合，为交互式视频编辑提供了新思路。

Abstract: Streaming video editing has made rapid progress, yet practical deployment is still limited by two core issues: maintaining stable backgrounds and non-edited regions over time, and achieving the low latency required for real-time interactive scenarios. Meanwhile, recent streaming video generation methods are mostly developed for synthesis and cannot be directly applied to editing due to the strict preservation requirement and region-specific control. In this work, we present a novel streaming video editing framework that performs causal, frame-by-frame editing with strong content preservation and real-time responsiveness. Our key design is a three-stage distillation pipeline that progressively transfers editing capability from a powerful bidirectional foundation model to an efficient unidirectional streaming editor, enabling stable long-horizon edits without sacrificing visual fidelity. To further support real-time deployment, we introduce an AR-oriented mask cache that reuses region-related computation across frames, substantially reducing redundant processing and accelerating inference. Finally, we establish a dedicated benchmark for streaming video editing. Extensive evaluations demonstrate that our method achieves state-of-the-art visual quality among streaming baselines while drastically boosting inference speed to 12.66 FPS, making it suitable for interactive and augmented reality applications.

[56] Do Image Editing Models Understand Lighting? cs.CVPDF

Tim Küchler, Johann-Friedrich Feiden, Matthias Nießner, Carsten Rother

TL;DR: 该论文针对生成式图像编辑模型是否真正理解真实世界光照的问题，提出了一个名为3D锚定光探针（3DLP）的新基准测试。作者通过采集一个包含1000对高动态范围（HDR）真实室内场景图像对的新数据集，其中光探针被物理性地打开和关闭，并标注了特定区域（如投射阴影、金属表面），以评估多种最先进的图像编辑模型在光照编辑任务上与真实物理的一致性。

Details

Motivation: 当前生成式图像编辑模型虽然视觉保真度高，但其是否具备对真实世界光照的内在理解仍是一个开放性问题。现有基准测试通常依赖于合成数据集或使用视觉语言模型（VLM）/人工判断来评估互联网图像的高层感知光传输合理性，缺乏基于真实世界光照变化的细粒度评估。

Result: 在提出的3DLP基准上评估了多种最先进的图像编辑模型，结果显示模型整体性能差异显著（镜面高光差异稍小）。最佳模型与真实物理一致性较高，但仍有改进空间；所有模型在接收光探针光照较少的区域都更容易出错。此外，测试发现视觉语言模型（VLM）不适用于像素级光传输分析。

Insight: 创新点在于构建了一个基于真实世界HDR图像对、具有细粒度区域标注（如投射阴影、金属表面）的新基准（3DLP）和数据集，并引入了两个新的评分来补偿AI生成的摄影效果（如白平衡调整），从而能够更精确地评估模型对光照物理的理解。从客观角度看，该工作为评估生成模型对底层物理属性（如光照）的理解提供了更可靠、可解释的基准。

Abstract: While recent advancements in generative image editing models have achieved stunning visual fidelity, it remains an open question whether these systems possess an intrinsic knowledge of real-world lighting. Existing benchmarks typically evaluate high-level plausibility of perceptual light transport on curated internet imagery, using VLMs or human judgement, or they rely on synthetically generated datasets. In this work, we introduce the 3D-anchored Light Probe (3DLP) benchmark, for which we have captured a new high-fidelity HDR dataset of real-world lighting changes. The dataset consists of 1K image pairs of diverse indoor scenery in which light probes are physically turned on and off. To allow for a granular performance analysis, we annotated specific image regions such as cast shadows or metallic surfaces. With this data, we evaluate a range of state-of-the-art image editing models by measuring how well their light probe edits align with reality. The evaluation uses two new scores to compensate for AI-generated photographic effects, such as adjusted white balance. Our results show that the overall performance of models differs considerably, with differences slightly less pronounced for specular highlights. The best image editing models are remarkably consistent with real-world physics, however, they still leave room for improvement. We observe that image regions that receive less light from the light probe are more prone to errors for all models. Furthermore, building on their success in evaluating macroscopic lighting plausibility, we test VLMs on our task but find that they are unsuitable for pixel-level light transport analysis. We will make the benchmark, together with the real-world dataset, publicly available to encourage future research on this topic.

[57] Depth-Semantic Alignment and Affinity-Guided Fusion for Structured Radar Point Cloud Generation cs.CVPDF

Amjad Hussain, Xin Qiu, Fuyuan Ai, Yuchen Tan, Zecheng Li

TL;DR: 本文提出了一种基于视觉-雷达融合的多模态点云生成方法，旨在解决毫米波雷达点云稀疏、噪声大和结构不完整的问题。该方法利用图像语义信息对雷达点云施加结构约束并实现空间对齐，同时结合稀疏补全策略以增强点密度并恢复缺失结构。生成的改进点云在目标检测和跟踪任务中进行了评估，结果表明其有效提升了感知模型在复杂环境下的检测精度和鲁棒性。

Details

Motivation: 毫米波雷达点云通常存在稀疏、噪声大和结构不完整的问题，这直接影响了下游感知任务（如目标检测和跟踪）的性能。因此，需要一种方法能够融合视觉信息来改善雷达点云的质量，以支持更可靠的智能感知系统。

Result: 实验结果表明，所提出的方法有效提升了点云质量，并在目标检测和跟踪任务中增强了感知模型的检测精度和鲁棒性。尽管摘要未明确提及具体基准或是否达到SOTA，但指出该方法为复杂环境下的感知提供了实用解决方案。

Insight: 论文的创新点在于结合图像语义信息对雷达点云进行结构约束和空间对齐，并引入稀疏补全策略来增强点密度和恢复结构。从客观角度看，这种跨模态融合与结构补全相结合的方法，为解决雷达点云固有缺陷提供了一种新颖且实用的技术路径。

Abstract: Point clouds are an important carrier of three-dimensional spatial information, and their quality directly affects the performance of downstream perception tasks such as object detection and tracking. However, millimeter-wave radar point clouds are typically sparse, noisy, and structurally incomplete. To address these limitations, this paper proposes a multimodal point cloud generation method based on vision-radar fusion. The proposed method leverages image semantic information to impose structural constraints and achieve spatial alignment for radar point clouds, while incorporating a sparse completion strategy to enhance point density and recover missing structures. The generated point clouds are further evaluated in object detection and tracking tasks. Experimental results demonstrate that the proposed method effectively improves point cloud quality and enhances the detection accuracy and robustness of perception models in complex environments, providing a practical solution for multisensor point cloud generation and intelligent perception systems.

[58] ProtoKV: Streaming Video Understanding under Delayed Query with Summary-State Memory cs.CV | cs.LGPDF

Le Tu Ngoc Minh, Jinyeong Lim, Dongsu Han

TL;DR: 本文提出了ProtoKV，一种用于流式视频理解（SVU）的恒定内存方法，通过将远历史信息聚合为固定容量的摘要状态而非保留原始token实例，解决了延迟查询下关键线索可能被驱逐或稀释的问题。该方法结合近窗口KV缓存和语义-空间原型库，在查询时通过伪token接口与标准注意力兼容，显著提升了长延迟场景下的准确性。

Details

Motivation: 流式视频理解面临延迟查询的挑战：关键视觉线索可能短暂出现，但在查询到达前经历多次更新，导致线索在有限内存中被驱逐或稀释，影响模型性能。

Result: 在SVU基准测试中，ProtoKV在长延迟场景下比基于token保留的基线方法准确率提升高达12.5个百分点，且延迟越长增益越明显，同时保持相近的查询时间成本和内存预算。

Insight: 创新点在于将远历史信息压缩为语义-空间原型库，并通过伪token接口实现与标准注意力的无缝集成，这为处理流式数据中的长期依赖提供了高效且可扩展的内存管理方案。

Abstract: Streaming video understanding (SVU) must answer queries that arrive asynchronously while visual tokens stream continuously under strict GPU-memory and query-time latency budgets. A key challenge is delayed query: decisive cues may appear briefly, yet many subsequent updates occur before the query arrives, increasing the risk that those cues are evicted or diluted under bounded memory. We propose ProtoKV, a constant-footprint SVU memory that represents far history as a fixed-capacity summary state rather than retaining token instances. ProtoKV keeps an exact near-window KV cache and aggregates older content into a semantic-spatial prototype bank with residual statistics. At query time, each prototype is exposed through a bounded pseudo-token interface that is drop-in compatible with standard attention. Under matched budgets and comparable query-time cost, ProtoKV improves accuracy by up to 12.5 points over token-retention baselines on SVU benchmarks in the long-delay regime, with gains that grow as query delay increases.

[59] Calibrated Harmonic Overlaid Implicit Neural Representations for Multi-Dimensional Data cs.CVPDF

Honghang Chen, Xiujun Zhang, Xiaoli Sun, Mingqing Xiao

TL;DR: 本文提出了一种新的校准谐波叠加隐式神经表示（CHOIR），用于多维数据恢复。该方法通过协调谐波叠加（CHS）替代传统函数组合，解决了网络深度增加时的优化不稳定问题，并引入感知频谱校准（PSC）来减轻频谱偏差。在多个多维数据恢复任务上的实验表明，该方法优于现有最先进方法。

Details

Motivation: 现有隐式神经表示（INR）方法主要依赖函数组合，随着网络深度增加会导致优化不稳定，且未能有效结合物理先验来缓解频谱偏差。

Result: 在多种多维数据恢复问题上的大量实验表明，该方法实现了优于最先进（SOTA）方法的性能。

Insight: 创新点在于用协调谐波叠加（CHS）机制替代传统函数组合以确保深度网络优化稳定性，并引入基于自然图像幂律谱先验的感知频谱校准（PSC）来物理地校准频谱分布。

Abstract: Implicit neural representation (INR) has emerged as a powerful prior for multi-dimensional data (e.g., multispectral images and videos). However, most INR methods employing periodic activation functions (e.g., Sine) predominantly rely on function composition. This mechanism introduces optimization instability as network depth increases, thereby limiting their performance. Meanwhile, these methods fail to incorporate proper physical priors to effectively alleviate spectrum bias. To address these issues, inspired by the commonalities between deep periodic networks and generalized Fourier series, we propose a novel Calibrated Harmonic Overlaid Implicit Neural Representation (CHOIR). Specifically, we utilize Coordinated Harmonic Superposition (CHS) to replace the conventional function composition used in most INRs, thereby ensuring optimization stability when scaling network depth. Furthermore, we introduce a Perceptual Spectrum Calibration (PSC) to mitigate spectrum bias. This calibration embeds the ubiquitous power-law spectrum prior of natural images and adjusts the globally fixed spectrum towards a physically plausible log-uniform distribution. Extensive experiments on various multidimensional data recovery problems demonstrate that our method achieves superior performance over state-of-the-art approaches. Code is available at https://github.com/chorl0229/CHOIR.

[60] LearniBridge: Learnable Calibration of Feature Caching for Diffusion Models Acceleration cs.CV | cs.LGPDF

Xuyue Huang, Zhe Chen, Wang Shen, Xiao-Ping Zhang

TL;DR: 本文提出了LearniBridge，一种用于加速扩散模型推理的可学习特征缓存校准机制。该方法通过分析发现最优校准更新存在于一个跨不同提示共享的低秩子空间中，并利用轻量级LoRA更新桥接多个时间步，仅需少量样本即可有效校准。实验表明，LearniBridge在图像和视频生成任务上实现了显著的加速效果，并在保持甚至提升生成质量的同时超越了之前的SOTA方法。

Details

Motivation: 扩散变换器（DiTs）在图像和视频生成方面取得了重大进展，但其推理过程计算成本高昂。现有的特征缓存加速方法虽然实现简单，但在高加速比下会因重用历史特征而导致严重的误差累积问题。

Result: 在图像生成模型FLUX、HunyuanVideo和WAN2.1上，LearniBridge分别实现了最高5.87倍、5.75倍和4.10倍的推理加速。在WAN2.1上，以4.10倍加速运行时，其在VBench基准上的得分比之前的SOTA方法提升了1.28%。

Insight: 核心创新点在于揭示了跨提示共享的低秩子空间是特征缓存最优校准的关键结构，并据此设计了一个基于轻量级LoRA更新的可学习校准机制。该方法仅需3-5个训练样本即可实现有效校准，为高效、高保真的扩散模型加速提供了一种新思路。

Abstract: Diffusion Transformers (DiTs) have driven substantial progress in image and video generation but suffer from prohibitive computational costs. Feature caching accelerates inference by reusing intermediate representations. Existing methods rely on historical features for implementation simplicity, yet suffer from severe error accumulation at high acceleration ratios. To address this limitation, we investigate the nature of the requisite feature correction. We demonstrate that the optimal calibration update is characterized by a shared low-rank subspace across diverse prompts. Guided by this structural insight, we propose LearniBridge, a learnable calibration mechanism for feature caching that bridges multiple timesteps through lightweight LoRA updates. This mechanism enables effective calibration requiring only 3-5 training samples. Extensive experiments on image and video generation show that LearniBridge achieves up to $5.87\times$, $5.75\times$, and $4.10\times$ acceleration on FLUX, HunyuanVideo, and WAN2.1, respectively. On WAN2.1, it improves VBench by 1.28% over the previous SOTA at $4.10\times$ acceleration. Our code is available at https://github.com/Iiiiiiirene/LearniBridge.

[61] Event-based Gaze Control System for Accurate Real-time Spin Estimation in Professional Ball Games cs.CV | eess.IVPDF

Yunpu Hu, Fabian Schilling, Valentina Cavinato, Asude Aydin, Agis Politis

TL;DR: 本文提出了一种基于事件相机的主动视觉系统，用于在专业球类比赛中实时、准确地估计球的旋转。该系统结合了事件相机、高速振镜和可调焦长焦镜头，通过混合跟踪方法（2D事件检测与3D定位系统结合）来跟踪球体，并提出了离线的球面对比度最大化方法（s-CMax）进行高精度旋转估计，以及一个用于实时应用的在线低延迟方法。

Details

Motivation: 在球类运动中，球的旋转对其轨迹有重要影响，但传统相机由于球体小、速度快、旋转快，难以在比赛中进行基于视觉的旋转估计。

Result: 离线方法s-CMax在静态球（乒乓球、棒球、网球、高尔夫球）上达到了SOTA精度，平均幅度和轴误差分别为2.1%和4.0度；在线方法在专业乒乓球比赛中实现了高精度（8.8%幅度和6.4度轴不匹配）、3毫秒延迟和750赫兹吞吐量。

Insight: 创新点包括：1) 结合事件相机、高速振镜和可调焦镜头的主动视觉系统设计，以应对高速运动挑战；2) 混合跟踪方法（2D事件检测与3D定位结合）确保鲁棒跟踪；3) 离线的球面对比度最大化方法（s-CMax）用于高精度旋转估计；4) 基于伪真值标签训练的CNN与GPU加速对比度最大化结合的在线低延迟方法，实现实时应用。

Abstract: Spin plays a crucial role in many ball sports due to its effect on the trajectory of the ball. Vision-based estimation of the ball’s spin during a game with conventional cameras is challenging due to the ball’s small size, high speed, and fast rotation. To address these challenges, we propose an event-based active vision system that can track unmodified balls and measure their spin in real-time. The system consists of an event camera for its high temporal resolution and minimal motion blur, high-speed pan/tilt galvanometer mirrors to keep the ball in the field of view, and a low-latency focus-tunable telephoto lens to increase the spatial resolution on the ball and keep it in focus. To track the ball, we use a hybrid approach that combines 2D event-based detection for centering and 3D positions from a ball localization system for re-initialization. For high-accuracy spin estimation, we propose an offline method that performs contrast maximization on the sphere (s-CMax). This method achieves state-of-the-art accuracy on static balls across multiple sports (table tennis, baseball, tennis, and golf), with mean magnitude and axis errors of 2.1% and 4.0 degrees, respectively. We then develop a low-latency online method for table tennis as a case study in real-time applications. This method uses an uncertainty-aware convolutional neural network trained on pseudo-ground-truth spin labels from the offline approach, combined with a GPU-accelerated batch implementation of contrast maximization for refinement. We demonstrate reliable tracking and spin estimation with a three-view setup during professional table tennis matches, with high accuracy (8.8% magnitude and 6.4 degrees axis mismatch), 3 ms latency, and 750 Hz throughput.

[62] ReasonCLIP-58M: Visually Grounded Commonsense Reasoning Supervision for CLIP cs.CV | cs.AIPDF

Sicheng Zhang, Muzammal Naseer, Binzhu Xie, Naufal Suryanto, Shi Qiu

TL;DR: 本文提出了ReasonCLIP-58M，一个为CLIP风格模型设计的持续预训练框架，旨在通过整合大规模推理监督来增强其视觉基础常识和组合推理能力。该框架采用两阶段策略，逐步融入推理信号并保持描述性对齐，同时构建了包含ReasonLite-42M和ReasonPro-16M的互补数据集以及RCLIP-Bench诊断评估基准。实验表明，ReasonCLIP在提升推理能力的同时，也改善了零样本检索性能，并能作为即插即用的视觉编码器为多模态大语言模型带来一致增益。

Details

Motivation: 当前CLIP及其变体的预训练主要依赖于描述性图文对齐，而下游应用日益需要视觉基础的常识推理和组合推理能力。本文旨在探索在不改变架构的情况下，CLIP风格的编码器是否能够支持此类推理。

Result: 训练得到的ReasonCLIP模型家族在视觉基础常识和组合推理方面得到改进，同时增强了零样本检索性能。当作为LLaVA-NeXT等多模态大语言模型的视觉编码器时，无需额外推理成本即可带来一致的性能提升。

Insight: 创新点在于提出了一个结构化的推理监督持续预训练框架，通过两阶段策略和类别结构化的监督，将大规模推理信号整合到CLIP模型中，从而增强了其视觉表征的表达能力，证明了推理监督的有效性。

Abstract: CLIP and its variants are widely adopted visual backbones in multimodal systems, but their pretraining remains dominated by descriptive image-text alignment. As downstream applications increasingly demand visually grounded commonsense inference and compositional reasoning, it remains unclear whether CLIP-style encoders can support such reasoning without architectural changes. To address this, we present ReasonCLIP-58M, a continual pretraining framework that integrates large-scale reasoning supervision into CLIP-style models through our two-stage strategy, which progressively integrates reasoning signals while preserving descriptive alignment, followed by category-structured reasoning supervision. To support this framework, we construct two complementary datasets and a benchmark: ReasonLite-42M, with open-form, visually verifiable reasoning captions; ReasonPro-16M, with category-specific reasoning supervision; and RCLIP-Bench for diagnostic evaluation of visually grounded reasoning. We train a family of ReasonCLIP that improves visually grounded commonsense and compositional reasoning while also enhancing zero-shot retrieval performance. As a drop-in visual encoder for multimodal large language models such as LLaVA-NeXT, ReasonCLIP delivers consistent gains without additional inference cost, demonstrating that structured reasoning supervision enhances the expressive capacity of CLIP-style visual representations. All datasets, models, and training code are available at https://github.com/RISys-Lab/ReasonCLIP.

[63] Multi-modality Image Fusion under Adverse Weather: Mask-Guided Feature Restoration and Interaction cs.CVPDF

Xilai Li, Xiaosong Li, Haishu Tan, Tao Ye, Huafeng Li

TL;DR: 本文提出了一种用于恶劣天气下的多模态图像融合方法，通过掩码引导的特征恢复与交互来增强场景表示。该方法引入了’伪真实值’简化训练，设计了基于融合结果与源图像映射关系的掩码生成机制，并采用掩码引导的跨模态交叉注意力机制，以选择性地关注信息丰富的特征。

Details

Motivation: 解决恶劣天气导致图像显著退化，破坏特征表示，并需要同时进行特征恢复和跨模态互补的问题。现有方法在此类条件下难以进行有效的表示学习，限制了实际性能。

Result: 在合成和真实世界数据集上的大量实验表明，该方法在视觉质量、定量指标和下游任务方面超越了最先进的方法。

Insight: 创新点包括：使用’伪真实值’简化训练流程；基于映射关系的掩码生成机制来量化各模态贡献；掩码引导的跨模态交叉注意力机制以避免对静态分布的过拟合；以及平衡特征恢复与交互的掩码引导和任务耦合退化感知学习策略。

Abstract: Multi-modality image fusion (MMIF) enhances scene representation by exploiting complementary cues from different modalities. Adverse weather, however, causes significant image degradation, disrupting feature representation and requiring simultaneous feature restoration and cross-modal complementarity. Existing methods often struggle with effective representation learning under such conditions, limiting their practical performance. To address these challenges, we propose a mask-guided MMIF method that integrates feature restoration and interaction. We first introduce “Pseudo Ground Truth” to simplify training, promoting faster and more effective feature learning. Then, we design a mask generation mechanism based on the mapping relationship between the fused result and the source images, quantifying the relative contribution of each modality during the fusion process. By incorporating the proposed mask-guided cross-modal cross-attention mechanism, the network is encouraged to selectively attend to informative features during modality interaction, mitigating the risk of overfitting to the static distribution of the “Pseudo Ground Truth”. Additionally, we propose a mask-guided learning strategy and a task-coupled degradation-aware learning strategy to balance feature restoration and interaction. Extensive experiments on synthetic and real-world datasets demonstrate that our method surpasses state-of-the-art approaches in visual quality, quantitative metrics, and downstream tasks. The source code is available at https://github.com/ixilai/AMG-Fuse.

[64] NaviCache: Test-Time Self-Calibration Caching for Video Generation cs.CV | cs.AI | cs.MMPDF

Zheqi Lv, Zhibo Zhu, Jinke Wang, Qi Tian, Shengyu Zhang

TL;DR: 本文提出NaviCache，一种用于视频生成模型的即插即用测试时自校准缓存方法，将特征演化重新概念化为惯性导航系统问题，以解决现有离线校准加速方法的数据依赖、耗时和对分布偏移敏感等问题，通过建模输入与输出变化的相对耦合来弥合领域差距和扩散过程的非平稳性。

Details

Motivation: 视频扩散模型受限于巨大的计算成本，现有离线校准加速方法存在校准数据依赖、校准时间过长以及对分布偏移敏感等问题，而离线免校准方法虽然消除了这些障碍，但依赖于瞬时零阶近似，容易受到观测噪声影响且忽略了扩散轨迹的内在动量。

Result: 在HunyuanVideo、Wan和Open-Sora系列上的大量实验表明，NaviCache在计算跳过的错误判断上更准确，并实现了出色的综合性能。

Insight: 创新点在于将特征演化重新概念化为惯性导航系统问题，引入双状态估计架构自适应跟踪特征变化率及其潜在漂移，并通过时间相关的噪声调度与不确定性感知的测量更新机制相结合，提供了一个理论上可靠的误差有界计算跳过机制。

Abstract: Video Diffusion Models (VDMs) is constrained by immense computational costs. While offline calibration-based acceleration suffers from calibration data dependency, prohibitive calibration duration, and susceptibility to distribution shifts, offline calibration-free methods eliminate these hurdles. However, since they rely on instantaneous zero-order approximations where the mapping between input and output differences varies in real-time, they are susceptible to observational noise and ignore the intrinsic momentum within the diffusion trajectory. In this paper, we propose NaviCache, a plug-and-play test-time self-calibration method re-conceptualizing feature evolution as an Inertial Navigation System (INS) problem. NaviCache bridges the fundamental domain gap and the non-stationary nature of diffusion by modeling the relative coupling between input and output variations. We introduce a dual-state estimation architecture that adaptively tracks the feature change ratio and its latent drift, initialized via a specialized Initial Alignment phase. By integrating a time-dependent noise schedule with an uncertainty-aware Measurement Update mechanism, NaviCache provides a theoretically grounded mechanism for error-bounded computation skipping. Extensive experiments on the HunyuanVideo, Wan, and Open-Sora series demonstrate that NaviCache exhibits more accurate error judgment for computation skipping and achieves outstanding comprehensive performance.

[65] Liquid Fusion of Heterogeneous Representations Towards General Salient Object Detection cs.CVPDF

Ke Chen, Ling Zhou, Guangqi Jiang, Gengshen Wu, Yi Liu

TL;DR: 本文提出了一种名为液态融合网络（LFNet）的新方法，用于通用显著目标检测（SOD）。该方法通过挖掘CNN和SSM在频谱偏好上的互补性，利用液态神经网络的动态信息传播思想，将VMamba和ConvNeXt的特征进行动态融合，并引入显著性引导上采样算子来优化特征传播。实验在五个任务上验证了其SOTA性能和优越的精度-效率权衡。

Details

Motivation: 当前基于状态空间模型（SSMs）的通用SOD方法忽视了不同神经网络范式（如CNN和SSM）固有的频谱偏差，而这些偏差导致它们的语义表征是互补的。本文旨在协调这些异构表征，以弥合其频谱偏差，提升通用SOD性能。

Result: 在RGB、RGB-D、RGB-T、VSOD和VDT五个不同的通用SOD任务上进行了广泛实验，结果表明LFNet实现了最先进的（SOTA）性能，并在检测精度和模型效率之间取得了优越的平衡。

Insight: 核心创新点在于通过数据集级别的频谱分析揭示了CNN与SSM表征的互补性，并受液态神经网络启发，提出了一个动态的状态-刺激范式（液态融合）来动态整合异构特征。此外，提出的显著性引导上采样算子通过频谱-空间协同设计来抑制上采样伪影，也是一个可借鉴的技术点。

Abstract: General Salient Object Detection (SOD) aims to identify and segment visually interesting objects from uni-modality or multi-modality scenes, recently advanced by cutting-edge State Space Models (SSMs). However, a critical limitation of current approaches is their neglect of the inherent spectral biases exhibited by different neural network paradigms. By digging to the dataset-level spectral analysis of Convolutional Neural Networks (CNNs) and SSMs, their semantic representations are inherently complementary based on their complementary frequency preferences. Inspired by this, we harmonize heterogeneous representations from SSMs and CNNs to bridge their spectral biases for general salient object detection. To this end, inspired by the dynamic information propagation of Liquid Neural Networks (LNNs), we introduce a liquid fusion to dynamically integrates features from two backbones, including VMamba and ConvNeXt, referred to Liquid Fusion Network (LFNet). Concretely, by treating the continuous VMamba features and ConvNeXt features as evolving states and exogenous stimulus, respectively, LFNet employs a dynamic gating mechanism for content-aware feature aggregation. Crucially, this state-stimulus paradigm enables to scale to multi-modal cues, resulting in flexibility in general SOD. Besides, a Saliency-Guided Upsampling (SGU) operator to propagate the features to the shallow layer, which leverages a spectral-spatial co-design to suppress upsampling artifacts while preserving semantics. Extensive experiments across five diverse tasks (RGB, RGB-D, RGB-T, VSOD, and VDT) demonstrate that LFNet achieves state-of-the-art performance, offering a superior trade-off between detection accuracy and model efficiency. Code has been released at https://github.com/cke520/LFNet.

[66] SpatialFlow-GRPO: Where Spatial Credit Drives Image Editing cs.CVPDF

Yankai Yang, Yancheng Long, Wei Chen, Xingyu Lu, Hongyang Wei

TL;DR: 本文提出了一种名为SpatialFlow-GRPO的图像编辑强化学习训练框架，旨在解决现有Flow-GRPO方法因依赖单一全图奖励而难以进行细粒度优化的问题。该框架通过引入空间细粒度的奖励反馈，将区域感知奖励转化为语义区域级别的优化信号，并在策略更新中对齐区域优势与相应的潜在位置。

Details

Motivation: 现有基于Flow-GRPO的图像编辑方法通常假设单一的全图奖励足以指导优化，但这种空间均匀性假设无法区分图像不同区域对整体质量的贡献，从而阻碍了精细编辑。

Result: 在OmniGen2和FLUX.2-klein-4B模型上，SpatialFlow-GRPO在GEdit-Bench、ImgEdit-Bench和新提出的MultiEditBench基准测试中均优于Flow-GRPO方法，展示了其在多区域编辑能力上的提升。

Insight: 核心创新在于打破了全图奖励的均匀性假设，引入了空间细粒度的奖励机制（通过训练区域感知奖励模型SFReward和构建SFReward-14K数据集），实现了编辑策略在语义区域级别的针对性优化，这为基于强化学习的生成模型精细控制提供了新思路。

Abstract: Recent online reinforcement learning has substantially improved image editing quality. However, existing Flow-GRPO-style methods usually rely on a single whole-image reward, which makes fine-grained editing optimization difficult. We observe that a key obstacle in image editing is this spatial uniformity assumption: a whole-image reward cannot distinguish how different spatial regions contribute to image quality. To address this issue, we propose SpatialFlow-GRPO, a training framework that introduces spatially fine-grained reward feedback. The framework converts region-aware rewards into semantic-region-level optimization signals and aligns region advantages with the corresponding latent positions during policy updates. We also train a region-aware reward model, SFReward, construct SFReward-14K with region-annotated editing samples, and introduce MultiEditBench to evaluate multi-region editing ability. On OmniGen2 and FLUX.2-klein-4B, SpatialFlow-GRPO outperforms Flow-GRPO on GEdit-Bench, ImgEdit-Bench, and MultiEditBench. The results show that SpatialFlow-GRPO converts local feedback into spatially aligned update signals and improves editing quality.

[67] Bridging Vision and Language Concepts through Optimal Transport Semantic Flow cs.CV | cs.AIPDF

Chenyang Zhang, Anqi Dong, Guangming Zhu, Nuoye Xiong, Siyuan Wang

TL;DR: 本文提出了一种基于最优传输流的概念瓶颈模型（OTF-CBM），用于改进视觉与语言概念的对齐。该方法将概念对齐重新构想为一个动态的跨模态传输过程，通过逆最优传输学习数据驱动的语义成本来衡量跨模态距离，并利用基于不平衡最优传输的流匹配来建模视觉图像块与文本概念之间的语义转换。

Details

Motivation: 现有视觉语言概念瓶颈模型（CBMs）通常依赖预对齐编码器或全局余弦相似度，这掩盖了细粒度的概念定位，且未能反映真实的语义几何结构。

Result: 实验表明，OTF-CBM在分类准确性和概念忠实度方面均取得了优越的性能。

Insight: 核心创新在于将概念对齐视为动态的跨模态最优传输过程，而非静态投影，并引入了基于速度的概念激活机制，从而在不进行ODE积分的情况下捕获可解释的几何关系，为可解释的跨模态推理提供了新的几何与动力学视角。

Abstract: Concept Bottleneck Models (CBMs) promise transparent reasoning by predicting through human-interpretable concepts, yet their effectiveness fundamentally depends on how well visual and textual representations are aligned or matched. Existing vision-language CBMs often rely on pre-aligned encoders or global cosine similarity, which obscures fine-grained concept localization and fails to reflect true semantic geometry. In this work, we rethink concept alignment as a dynamic cross-modal transport process instead of static projection and propose the Optimal Transport Flow Concept Bottleneck Model (OTF-CBM). It first learns a data-driven semantic cost via Inverse Optimal Transport to measure cross-modal distances, and then performs unbalanced optimal-transport-based flow matching to model semantic transitions between visual patches and textual concepts. With velocity-based concept activation, OTF-CBM captures interpretable geometric relations without ODE integration. Experiments further show that OTF-CBM achieves superior classification accuracy and concept faithfulness, offering a new geometric and dynamical perspective for interpretable cross-modal reasoning.

[68] RIS-Assisted Proactive Handover for Reliable mmWave Wireless Networks cs.CVPDF

Alaa Adnan, Mohammad Al-Quraan, Ahmed Zoha, M. Majid Butt, Sami Muhaidat

TL;DR: 本文提出了一种新颖的可重构智能表面（RIS）辅助的主动切换（PHO）方法，用于毫米波（mmWave）无线网络。该方法通过优化分配的RIS单元数量，在切换时序约束下平衡信号处理复杂度和链路质量，从而提高RIS辅助链路的能效。基于粒子群优化（PSO）的离线优化问题被用于确定最优的端到端RIS链路配置。

Details

Motivation: 毫米波网络极易受到视距（LoS）阻塞的影响。视觉辅助无线通信（VAWC）虽能实现主动切换以缓解阻塞，但当附近没有可用基站（BS）时，主动切换变得困难。此时，可重构智能表面（RIS）可用于恢复连接，但RIS的大量单元配置时间可能影响其在时敏场景中的响应能力。

Result: 结果表明，在保持信噪比（SNR）不变的情况下，将RIS单元数量减少12%可使能耗降低10%。此外，RIS辅助链路在阻塞区域实现了15-30 dB的性能提升，同时保持了准确的主动切换时序。

Insight: 创新点在于提出了一种在切换时序约束下优化RIS单元分配以权衡复杂度与链路质量的框架，并通过离线PSO优化绕过延迟限制。从客观角度看，该方法将RIS配置时间作为关键约束进行建模，为能量高效的可靠毫米波通信提供了新思路。

Abstract: Millimeter-wave (mmWave) networks are highly susceptible to line-of-sight (LoS) blockages. Vision-aided wireless communications (VAWC) enable proactive handovers (PHO) to mitigate such blockages; however, PHO becomes challenging when no nearby base station (BS) is available. In such cases, reconfigurable intelligent surfaces (RIS) can be used to restore connectivity. To ensure timely PHO, the RIS configuration time must be taken into account, as the large number of RIS elements can limit responsiveness in time-sensitive scenarios. This work proposes a novel RIS-assisted PHO approach that optimizes the number of allocated RIS elements to balance signal processing complexity and link quality under handover timing constraints, making the RIS-assisted link more energy-efficient. An optimization problem based on particle swarm optimization (PSO) is formulated to determine the optimal end-to-end RIS link setup that runs offline to bypass latency constraints. Results show that reducing the number of RIS elements by 12% leads to a 10% decrease in dissipated energy without compromising the signal-to-noise ratio (SNR). Moreover, the RIS-assisted link achieves a 15–30 dB improvement in blocked regions while maintaining accurate PHO timing.

Minh Duc Do, Tillmann Rheude, Noel Kronenberg, Roland Eils, Benjamin Wild

TL;DR: 本文提出了一种名为MICViT（多模态内部与跨上下文视觉Transformer）的3D视觉Transformer模型，用于处理多模态3D脑部MRI数据。该模型通过四种注意力机制（模态特定的局部与全局注意力、跨模态局部与全局注意力）显式建模模态内特征和模态间交互，以应对高维数据和小样本的挑战。在脑龄预测任务上，MICViT在三个异构数据集（UK Biobank、SOOP、Cam-CAN）上使用多种MRI模态（如T1、FLAIR、DWI、SWI）进行评估，并一致超越了现有的CNN和Transformer基线模型。

Details

Motivation: 脑部MRI数据具有高维、多模态且样本量有限的特点，同时存在解剖结构、病理和采集条件的多样性，这给机器学习模型带来了挑战。多模态成像虽能提供互补的临床信息，但有效整合这些信号仍很困难。

Result: MICViT在三个异构数据集（UK Biobank、SOOP、Cam-CAN）的脑龄预测任务中，使用多种MRI模态（如T1、FLAIR、DWI、SWI）进行评估，一致超越了最先进的CNN和Transformer基线模型，达到了SOTA水平。特别地，模型从多模态输入中获益更大，随着加入更多模态，性能提升更显著。

Insight: 论文的创新点在于显式建模模态内（局部与全局）和模态间（局部与全局）的交互，通过四种注意力机制整合多模态3D MRI信息。从客观角度看，这种结构化的跨模态注意力设计是有效利用多模态脑成像数据的关键，为神经影像表征学习提供了有前景的方向。

Abstract: Brain MRI poses a fundamental challenge for machine learning: models must learn from high-dimensional 3D data spanning multiple co-registered modalities, despite the limited sample sizes typical of neuroimaging studies relative to the diversity in anatomy, pathology, and acquisition conditions. While multimodal imaging provides complementary information critical for clinical interpretation, effectively integrating these signals remains difficult. We propose Multimodal Intra- and Cross-Context Vision Transformer (MICViT), a 3D vision transformer that explicitly models both modality-specific representations and cross-modal interactions across local and global contexts. Concretely, MICViT combines four attention mechanisms: modality-specific local and global attention for intra-modal feature learning, and cross-modal local and global attention to capture interactions between modalities. We evaluate MICViT on brain age prediction across three heterogeneous datasets (UK Biobank, n=41,404; SOOP, n=1,062; Cam-CAN, n=613) using multiple MRI modalities (e.g. T1, FLAIR, DWI, SWI). MICViT consistently outperforms state-of-the-art CNN and transformer baselines in 3D settings. Notably, it benefits more strongly from multimodal inputs, yielding larger performance gains as additional modalities are incorporated. These results demonstrate that explicitly modeling intra- and cross-modal interactions is key to unlocking the full potential of multimodal brain MRI, highlighting a promising direction for representation learning in neuroimaging.

[70] Tractography-Driven Synthetic Data Generation for Fiber Bundle Segmentation in Tracer Histology cs.CV | cs.LGPDF

Kyriaki-Margarita Bintsi, Sparsh Makharia, Yaël Balbastre, Joselyn Romero Avila, Julia F. Lehman

TL;DR: 本文提出了一种基于合成数据增强的框架，用于自动分割猕猴示踪剂组织学中的纤维束。该方法利用离体扩散MRI纤维追踪作为生成先验，合成2D图像块进行训练，并结合真实背景和领域随机化增强多样性。实验表明，仅使用真实数据训练时，该方法在跨大脑和纤维束密度方面具有更好的泛化能力，且仅需三分之一的手动标注数据即可达到与最先进方法相当的性能。

Details

Motivation: 扩散MRI纤维追踪能够无创重建白质通路，但其准确性受限于轴突组织的间接、低分辨率测量。非人灵长类动物的示踪剂注射研究为验证dMRI纤维追踪提供了金标准，但需要耗时的手动标注组织学切片中的纤维束。

Result: 在保留的大脑上进行实验，与仅使用真实数据训练相比，该方法在跨大脑和纤维束密度方面表现出更好的泛化能力。仅使用合成数据训练会导致性能较差，强调了真实监督的必要性。总体而言，该方法在仅需三分之一手动标注数据的情况下，达到了与最先进方法相当的性能。

Insight: 创新点在于利用dMRI纤维追踪作为生成先验合成训练数据，并结合真实背景和领域随机化增强多样性，从而减少对手动标注数据的依赖。从客观角度看，该方法通过合成数据增强策略，有效解决了组织学中纤维束分割的数据稀缺问题，并展示了合成与真实数据混合训练的重要性。

Abstract: Diffusion MRI (dMRI) tractography enables non-invasive reconstruction of white-matter pathways, but its accuracy is fundamentally limited by indirect, low-resolution measurements of axonal organization. Tracer injection studies in non-human primates provide a gold standard for validating dMRI tractography. This, however, requires time-consuming manual annotation of fiber bundles in histology sections. We propose a synthetic-data augmented framework for automated fiber bundle segmentation in macaque tracer histology. Our approach uses ex vivo dMRI tractography as a generative prior to synthesize 2D image patches for training. This provides us with sufficiently realistic foreground texture, which we compose with backgrounds from blockface photos and diversify via domain randomization. A 2D U-Net is trained on mixed real and synthetic patches. Experiments on held-out brains demonstrate improved generalization across brains and fiber bundle densities compared to training with real data only. Training with synthetic data only leads to poor performance, underscoring the need for real supervision. Overall, our approach achieves performance comparable to the state-of-the-art while requiring 3x less manually annotated data.

[71] PhysRAG: Enhancing Physics-Awareness in Video Generation via Retrieval-Augmented Generation cs.CVPDF

Kexu Cheng, Zicheng Liu, Mingju Gao, Chunhe Song, Hao Tang

TL;DR: 本文提出了PhysRAG，一种通过检索增强生成（RAG）来提升视频生成模型物理感知能力的新方法。该方法通过两阶段数据过滤从WISA-80K数据集中筛选出7K高质量视频用于训练，并构建物理视频数据库，利用可学习查询将物理知识注入视频扩散模型。

Details

Motivation: 解决现有视频生成模型难以捕捉热力学、力学和光学等多样化物理现象，从而缺乏物理真实性的核心挑战。

Result: 在PhyGenBench和VBench等基准测试中，该方法在视觉质量和物理规则遵循方面均达到了最先进的性能，超越了现有模型。

Insight: 创新点在于将RAG范式引入视频生成领域以增强物理感知，并设计了专门的数据过滤流程和基于可学习查询的物理知识注入机制，为数据稀缺下的物理建模提供了新思路。

Abstract: Developing physically aware video generation models remains a significant challenge due to the difficulty in capturing diverse physical phenomena, such as thermal dynamics, mechanics, and optics. In this work, we introduce PhysRAG, a novel pipeline that enhances physical awareness in video generation through Retrieval-Augmented Generation (RAG). To address the issue of limited high-quality data, we design a two-stage data filtering pipeline based on the WISA-80K dataset, resulting in a curated set of 7K high-quality videos for training. Furthermore, we construct a physical video database and develop a mechanism to inject physical knowledge into a video diffusion model using learnable queries. Our method achieves state-of-the-art performance in both visual quality and physical rule compliance, surpassing existing models in benchmarks such as PhyGenBench and VBench. We conduct extensive ablation studies to validate the effectiveness of our key components, including the data filtering pipeline, RAG mechanism, and method for physical information extraction. To facilitate future research, our code, data, and models are prepared for release at https://github.com/sediment1024/PhysRAG.

[72] Confidence-Aware Tool Orchestration for Robust Video Understanding cs.CV | cs.AIPDF

Yangfan He, Yujin Choi, Jaehong Yoon

TL;DR: 本文提出了Robust-TO框架，旨在解决视频推理语言模型中存在的‘盲目信任问题’。该框架通过一个统一的证据接口组织异构视觉感知工具，为每一帧计算可靠性-相关性分数，并利用校准后的可靠性分数在三级合成过程中指导证据加权，同时定义了联合优化正确性、证据可靠性和效率的置信度-成本GRPO奖励。

Details

Motivation: 解决现有视频推理语言模型隐含地假设每一输入帧都同等可靠的问题，即‘盲目信任问题’。该问题导致在现实扰动（如运动模糊、眩光、遮挡）下，前沿模型的准确性在真实世界具身基准上会大幅下降，且模型无法意识到其视觉证据已受损。

Result: 在两个涵盖八个任务的视频推理基准测试中，Robust-TO在干净输入上达到56.4%的平均准确率，比最强的开源基线高出10.6个百分点，并优于Gemini-2.5-Pro（46.2%）。在五种现实扰动类型下，Robust-TO保持了54.3%的平均准确率，比最强的开源基线高5.8个百分点，并且在所有对比方法中表现出最小的干净到扰动准确率下降。

Insight: 核心创新在于显式地将逐帧可信度整合到推理的每个阶段，通过统一的证据接口和校准的可靠性分数来系统性地管理和加权异构工具的证据。这为解决多模态模型中视觉输入质量不均导致的鲁棒性问题提供了一个可借鉴的代理框架和优化范式。

Abstract: Video reasoning language models implicitly assume that every input frame is equally reliable. This leads to what we term the Blind Trust Problem: under realistic perturbations such as motion blur, glare, or occlusion, frontier video reasoning models can suffer 15-30%p accuracy drops on real-world embodied benchmarks, while remaining unaware that their visual evidence has been degraded. To address this challenge, we propose Robust-TO, an agentic video understanding framework that explicitly integrates per-frame trustworthiness into every stage of reasoning. Robust-TO organizes heterogeneous visual perception tools under a unified evidence interface. Each tool receives a sub-query derived from the original question and a set of trustworthy frames selected by the reliability-relevance score. It returns evidence in a shared format: a concrete prediction (e.g., a bounding box, motion trajectory, recognized text, or action label), temporal grounding, and a calibrated reliability score. During reasoning, these calibrated scores guide evidence weighting in a three-tier synthesis process (high/medium/low) and define a confidence-cost GRPO reward that jointly optimizes correctness, evidence reliability, and efficiency. On two video reasoning benchmarks spanning eight tasks, Robust-TO achieves 56.4% average accuracy on clean inputs, surpassing the strongest open-source baseline by 10.6%p and outperforming Gemini-2.5-Pro (46.2%). Under five realistic corruption types, Robust-TO maintains 54.3% average accuracy, 5.8%p above the strongest open-source baseline, while exhibiting the smallest clean-to-corrupted accuracy drop among all compared methods.

[73] PortraitGen: Exemplar-Driven GRPO with Dual-Reward Guidance for Photorealistic Portrait Generation cs.CVPDF

Xiaomin Li, Qian Liang, Yinan Li, Ying Zhang, Chen Li

TL;DR: 本文提出PortraitGen框架，通过将真实图像引入GRPO采样组打破生成边界，并设计双奖励机制（OmniReward和AI-Portrait）优化人像照片真实感，在自建PortraitBench基准上显著超越现有基线。

Details

Motivation: 现有基于GRPO的文本到图像后训练方法过度追求表面美学（如色彩饱和度），未能解决AI伪影和生物不合理性等关键缺陷，主要因缺乏真实图像参与采样和针对细粒度伪影的奖励机制。

Result: 在PortraitBench基准上的大量实验表明，PortraitGen显著优于现有基线，有效抑制AI伪影并实现了前所未有的照片真实感。

Insight: 创新点包括：在GRPO中引入真实图像及其潜表示以突破生成分布限制；设计兼顾通用质量与人像保真度的双奖励机制；构建专用人像基准PortraitBench用于系统评估。

Abstract: Reinforcement Learning like Group Relative Policy Optimization (GRPO) has significantly advanced text-to-image post-training. However, current methods often favor superficial aesthetics, such as over-saturated colors, leaving critical flaws like AI artifacts and biological implausibilities unresolved. We attribute these limitations to two primary factors: (1) The absence of real images during post-training confines GRPO sampling to the original distribution, failing to break inherent generative boundaries; (2) the optimization process lacks specific rewards targeting fine-grained artifacts like overly oily skin and other AI artifacts. To address this, we propose PortraitGen, a novel framework tailored for photorealistic portrait generation. First, we break inherent generative boundaries by directly introducing real images into the GRPO sampling groups, where image inversion is employed to obtain their transition probabilities and latents. Second, to explicitly steer the model toward photorealism, we introduce a complementary dual-reward mechanism: OmniReward for general quality and AI-Portrait for human-centric fidelity. Furthermore, we curate PortraitBench, a comprehensive portrait-centric benchmark. Extensive experiments demonstrate that PortraitGen significantly outperforms existing baselines, effectively suppressing AI artifacts and achieving unprecedented photorealism.

[74] TraMP-LLaMA: Generative Interpretability with Decoupled Instruction Tuning for Facial Expression Quality Assessment cs.CVPDF

Shuchao Duan, Alan Whone, Hossein Rahmani, Jun Liu, Majid Mirmehdi

TL;DR: 本文提出了TraMP-LLaMA，一个用于面部表情质量评估（FEQA）的统一多模态框架。该框架不仅预测严重性分数，还能从面部运动线索生成结构化的文本报告，以增强模型的可解释性。它整合了RGB外观和关键点轨迹线索，并采用解耦的指令调优策略来减少严重性预测和语言生成之间的任务干扰。

Details

Motivation: 现有面部表情质量评估方法通常只输出严重性分数，缺乏对支持预测的可观察面部运动证据的明确解释，这限制了模型的可解释性，尤其在帕金森病评估中难以检查模型输出的依据。

Result: 在扩展的PFED5-plus数据集上的实验表明，TraMP-LLaMA在报告生成任务上优于竞争的视频-语言基线方法，并且在联合多表情训练下，其严重性预测性能在所有对比方法中达到最佳，将Spearman秩相关系数至少提高了4.39%。

Insight: 论文的创新点在于将面部表情质量评估从单一的分数预测扩展为“分数预测+结构化报告生成”的统一任务，并提出了解耦的指令调优策略来协调多任务学习。从客观角度看，其整合多模态线索（外观与轨迹）并生成可解释文本报告的方法，为医疗AI模型的透明化提供了新思路。

Abstract: Existing facial expression quality assessment (FEQA) methods typically produce only a severity score, without explicitly communicating the observable facial motion evidence that supports the prediction. This limits interpretability and makes it difficult to inspect the basis of model outputs in Parkinson’s disease assessment. To address this gap, we propose TraMP-LLaMA, a unified multimodal framework that jointly predicts severity scores and generates structured textual reports from facial motion cues. The framework integrates RGB appearance and landmark trajectory cues, and adopts a decoupled instruction-tuning strategy to reduce task interference between severity prediction and language generation. To support this task, we further extend the PFED5 dataset with expert-guided textual motion descriptions and construct PFED5-plus. Experiments on PFED5-plus show that TraMP-LLaMA outperforms competitive video-language baselines in report generation and achieves the best severity prediction performance among the compared methods under joint multi-expression training, improving Spearman’s rank correlation by at least 4.39 percent over all competing methods. The text annotations and code are available at https://github.com/shuchaoduan/TraMP-LLaMA.

[75] Computer Vision for MOBA Analytics: A Dataset and Baseline for Visibility Analysis in Dota 2 cs.CVPDF

Ricardo da Rocha Carvalho, Eloísa Oliveira, Luiz Bernardo Martins Kummer, Emerson Cabrera Paraiso, Rayson Laroca

TL;DR: 该论文提出了Dota2-Vis数据集和基线流程，用于分析《Dota 2》职业比赛中的视野可见性。数据集包含2025年国际邀请赛全部144场比赛的双视角高清视频及2477张手动标注的小地图图像，并评估了多种YOLO目标检测器来追踪玩家图标，以生成随时间变化的对手可见性曲线。

Details

Motivation: 现有MOBA分析主要依赖结构化数据，无法直接捕捉比赛中每支队伍的实际视野信息，因此需要基于视频数据来分析和补足这一关键维度。

Result: YOLO11l（大型）模型在玩家图标检测上取得了最佳整体性能，即使在密集和视觉杂乱的小地图场景中也能可靠识别。生成的可见性曲线揭示了玩家、英雄、角色和团队层面的行为模式，这些是传统结构化数据难以获取的。

Insight: 创新点在于首次为MOBA游戏引入了基于视频的视野分析公开数据集和基线流程，将计算机视觉技术应用于游戏分析，揭示了仅从结构化数据中无法观察到的团队行为差异。

Abstract: Introduction: Most Multiplayer Online Battle Arena (MOBA) analytics studies rely on structured data, which does not directly capture what each team could actually see during a match. Objective: This work introduces Dota2-Vis, a video-based dataset, and a baseline pipeline for visibility analysis in professional Dota 2 matches. Methodology: The dataset comprises all 144 matches from The International 2025, recorded from both team perspectives, totaling 288 Full HD videos, together with 2,477 manually annotated minimap images. We evaluate multiple variants of a modern object detector for player-icon detection and use the best-performing model to estimate opponent-visible player presence over time. Results: YOLO11l (large) achieved the best overall performance, reliably identifying player icons even in dense and visually cluttered minimap scenes. The resulting visibility curves reveal player, hero, role, and team-level patterns that complement conventional MOBA analytics, highlighting behavioral differences that are difficult to obtain from structured data alone. The dataset and code are publicly available at https://github.com/RicardoRCarvalho/dota2-vis/.

[76] Event-Aware Instructed Assistant for Referring Video Segmentation cs.CV | cs.AIPDF

Jinyu Liu, Henghui Ding, Shuting He, Yu-Gang Jiang

TL;DR: 本文提出EVIS（事件感知视频指令分割助手），一种用于指代视频分割的新方法。该方法通过可学习的事件查询将视频分解为多个简单事件，以逐事件、易于理解的方式处理复杂视频内容，并结合对象-像素混合学习实现长期视频中的目标跟踪。

Details

Motivation: 现有指代视频分割方法通常将视频视为由多幅图像组成的单一事件，忽略了视频通常包含多个不同事件的事实，这容易导致模型在处理复杂视频和文本内容时产生混淆和幻觉。

Result: 在5个公开基准测试上的广泛实验结果表明，EVIS在解决指代视频分割任务方面表现出强大的性能。

Insight: 核心创新在于提出通过文本引导的事件查询对视频进行事件级分解，实现分层理解，并引入对象-像素混合学习策略，将细粒度像素特征与先验对象查询相结合，以增强多模态大语言模型在长期视频中的目标跟踪能力。

Abstract: Existing referring video segmentation methods often treat a video as a single event consisting of multiple images, overlooking the fact that a video typically contains multiple distinct events. Under such a mechanism, the model needs to directly understand all the complex content in the video and text, which can easily lead to confusion and hallucinations. To address this issue, we propose to decompose a video to a set of simple events by learnable Event Query, and understand complex video content in an event-by-event, easy-to-understand manner. This is based on the observation that natural language expressions often divide a video into distinct, text-related segments, each representing a separate event within a compound event. We introduce EVIS, an Event-Aware Video Instructed Segmentation Assistant, which utilizes text-guided Event Queries to partition a video into simple events, extracting event-aware visual-text features to achieve a hierarchical understanding of the video. Additionally, we propose Object-Pixel-Hybrid Learning, which enables the MLLMs to track targets in long-term videos by integrating fine-grained pixel features with prior object queries. Extensive experimental results on 5 public benchmarks demonstrate EVIS’s strong performance in addressing the referring video segmentation task.

[77] Unison: Benchmarking Unified Multimodal Models via Synergistic Understanding and Generation cs.CVPDF

Jinyu Liu, Xincheng Shuai, Henghui Ding, Yu-Gang Jiang

TL;DR: 本文介绍了Unison，一个用于评估统一多模态模型联合理解与生成能力的综合性基准。该基准包含2,169个高质量任务样本，旨在克服现有评估方法孤立测试理解或生成、忽视两者协同作用的局限。Unison通过三个核心优势（全面维度、诊断性评估、人类对齐）对先进模型进行系统评估，揭示了当前系统的关键不足并指出了未来研究方向。

Details

Motivation: 现有统一多模态模型虽具备理解和生成双重能力，但评估方法通常将两者孤立测试，忽略了理解与生成之间的协同作用，这阻碍了对模型统一设计优势的全面认识。

Result: 论文在Unison基准上对最先进的模型进行了系统评估，揭示了当前统一多模态系统的关键局限性，并基于此提出了未来研究的潜在方向。

Insight: 论文的主要创新点在于提出了首个专注于评估理解与生成协同作用的综合性多模态基准（Unison），其设计包含内部一致性、理解引导生成等四个评估维度，并引入了与人类判断对齐的评估模型（Unison-Judge）以确保可靠性。从客观角度看，其诊断性评估设计（统一与解耦双轨）能对失败模式进行细粒度归因，量化统一建模的增益，这对理解和改进统一多模态模型架构具有重要价值。

Abstract: Unified multimodal models capable of both understanding and generation have achieved remarkable strides. However, despite their unified designs, existing evaluations typically assess understanding and generation capabilities in isolation, overlooking the synergy between comprehension and generation. To bridge this gap, we introduce Unison, a comprehensive benchmark comprising 2,169 high-quality unified task samples, designed to evaluate joint understanding and generation in unified multimodal models. Unison offers three key strengths: 1) Comprehensive Dimensions: Unison encompasses internal consistency, understanding-guided generation, generation-guided understanding, and mutual enhancement to enable holistic evaluation. 2) Diagnostic Evaluation: it provides both unified and decoupled tracks for understanding and generation, allowing fine-grained attribution of failure modes and quantitative analysis of the gains from unified modeling. 3) Human Alignment: we also introduce Unison-Judge, an evaluation model well aligned with human judgments to ensure reliable assessment. Based on systematic evaluations of state-of-the-art models on Unison, we uncover critical limitations in current unified multimodal systems and highlight promising directions for future research. Codes, Unison and Unison-Judge are publicly available at https://github.com/FudanCVL/Unison.

[78] SubdivAR: Autoregressive Next-Scale Prediction for Neural Mesh Subdivision cs.CVPDF

Huipeng Guo, Zikai Song, Hang Long, Jielei Zhang, Wenbing Li

TL;DR: 本文提出了SubdivAR，一种基于网格自回归表示（MAR）的神经网格细分框架。该框架将不同细分级别的网格组织成有序的尺度序列，将细分任务重新定义为自回归的下一尺度预测。通过引入混合拓扑感知Transformer和下一尺度坐标预测范式，该方法在保持细分拓扑的同时恢复精细几何细节。

Details

Motivation: 解决传统基于规则的网格细分方法产生过度平滑表面，以及现有神经细分方法受限于局部建模和泛化能力有限的问题。

Result: 在构建的FII-40K数据集上，SubdivAR超越了现有最先进（SOTA）基线方法，将Hausdorff距离和Chamfer距离分别降低了18.8%和14.2%，并在复杂的开放表面几何上表现出强大的鲁棒性。

Insight: 创新点在于将网格细分解构为自回归的尺度序列预测问题（MAR），并设计了结合全局语义注意力和拓扑约束局部特征聚合的混合Transformer架构。这为神经几何处理提供了新的序列建模视角和有效的拓扑感知机制。

Abstract: Mesh subdivision is a fundamental operation for converting coarse, editable meshes into high-resolution surfaces, with broad applications in digital asset creation. Classical rule-based schemes rely on fixed local refinement rules and often produce over-smoothed surfaces. Recent neural subdivision methods improve detail synthesis, but remain constrained by local modeling and exhibit limited generalizability. We present SubdivAR, a neural mesh subdivision framework based on our proposed Mesh Autoregressive Representation (MAR). MAR arranges meshes at different subdivision levels into an ordered scale sequence, reformulating subdivision as autoregressive next-scale prediction. To support this formulation, we introduce a Hybrid Topology-Aware Transformer that combines global semantic attention with topology-constrained local feature aggregation. SubdivAR adopts a next-scale coordinate prediction paradigm, regressing vertex offsets at each refinement stage to preserve subdivision topology while recovering fine-grained geometric details. To enable reliable learning, we construct FII-40K, a curated dataset of nearly 40,000 high-quality meshes with multi-level subdivision supervision. Experiments show that SubdivAR outperforms state-of-the-art baselines, reducing Hausdorff Distance and Chamfer Distance by 18.8% and 14.2%, respectively, and demonstrates strong robustness on complex open-surface geometries.

[79] Pseudo-Text-Conditioned 3D Grounding DINO for Organ Localization in Abdominal CT cs.CV | eess.IVPDF

Siqi Chen, Han Gong, Keyi Hou, Jingxuan Yang, Sheethal Bhat

TL;DR: 本文提出CT-3GDINO，一种轻量级3D检测器，用于腹部CT扫描中的器官定位。该方法基于Grounding DINO风格的查询架构，但使用冻结的伪文本类别标记替代真实文本编码器，结合Swin3D视觉骨干网络、双向特征增强和跨模态解码器来预测肝脏、脾脏、左右肾脏及肠道的三维边界框。

Details

Motivation: 解决腹部CT中可靠器官定位问题，为下游创伤分析提供空间先验信息。

Result: 在193个匹配的RSNA/RATIC CT数据集上，最佳多尺度模型从零开始训练，在3D IoU阈值0.1至0.7范围内取得了0.5830的总体top-1类别平均精度（mAP），优于固定骨干网络和可训练骨干网络的分类预训练变体（分别为0.5570和0.4657 mAP）。在粗定位（IoU 0.1）上表现优异（AP 0.9649），但在严格框对齐（IoU 0.7）上仍有局限（AP 0.1552）。

Insight: 创新点在于将基于文本的开放世界检测架构（Grounding DINO）适配到固定类别3D医学图像定位任务，通过伪文本条件化避免了真实文本编码器的需求。方法结合了轻量级设计、多尺度特征和跨模态解码，为伪文本条件化的3D器官定位建立了开源基线。

Abstract: Reliable organ localization in abdominal CT can provide spatial priors for downstream trauma analysis. We propose CT-3GDINO, a lightweight 3D detector that adapts a Grounding-DINO-style query-based architecture to fixed organ localization using frozen pseudo-text class tokens instead of a real text encoder. The model combines a Swin3D visual backbone, bidirectional feature enhancement, pseudo-text-guided query selection, and a cross-modality decoder to predict normalized 3D boxes for liver, spleen, left kidney, right kidney, and bowel. We train and evaluate on 193 matched RSNA/RATIC CT volumes with segmentation-derived boxes. The best multi-scale model, trained from scratch, achieves 0.5830 overall top-1 class-wise mAP over 3D IoU thresholds from 0.1 to 0.7, outperforming fixed- and trainable-backbone classification-pretrained variants with 0.5570 and 0.4657 mAP. Performance is strong for coarse localization, with 0.9649 AP at IoU 0.1, but remains limited for strict box alignment, with 0.1552 AP at IoU 0.7. These results establish CT-3GDINO as an open-source baseline for pseudo-text-conditioned 3D organ localization and motivate future work on localization-aware pretraining, richer multimodal conditioning, and injury-focused detection.

[80] FlameVQA: A Physically-Grounded UAV Wildfire VQA Benchmark with Radiometric Thermal Supervision cs.CV | cs.ROPDF

Mobin Habibpour, John Spodnik, Niloufar Alipour Talemi, Fatemeh Afghah

TL;DR: 本文提出了FlameVQA，一个基于无人机（UAV）的、用于野火监测的视觉问答（VQA）基准数据集。该数据集建立在FLAME 3数据集之上，利用配对的RGB图像和辐射热成像TIFF文件，通过多选问题形式评估模型在温度感知和安全关键推理方面的能力。

Details

Motivation: 现有的仅依赖RGB图像的无人机野火监测方法在烟雾、尺度变化和遮挡等复杂空中场景下可靠性不足，因此需要一个结合热辐射信息的、物理基础扎实的基准来推动该领域的发展。

Result: 在FlameVQA基准上评估了代表性多模态大语言模型（MLLMs），提供了基线结果。结果表明，当有明确的跨模态线索时模型表现良好，但在浓烟下的存在检测和覆盖范围估计任务上存在显著失败。

Insight: 主要创新点在于构建了一个结合辐射热成像物理信息的、面向安全关键任务的野火VQA基准，并采用了MLLM辅助标注与确定性热规则相结合的高可靠性标注流程。研究揭示了当前通用MLLMs在特定灾害监测领域需要针对性的领域适应。

Abstract: Wildfire monitoring from UAVs requires reliable reasoning over complex aerial scenes, where smoke, scale variation, and occlusions often limit RGB-only interpretation. We introduce FlameVQA, a multiple-choice visual question answering benchmark for UAV-based wildfire intelligence built on FLAME 3, leveraging paired RGB imagery and radiometric thermal TIFFs for temperature-grounded, safety-critical reasoning. FlameVQA includes 34 multiple-choice questions per image spanning six operational capability groups, covering tasks such as detection, localization, distribution/coverage estimation, cross-modal reasoning, and flight planning. To ensure label reliability, we combine MLLM-assisted annotation with deterministic thermal rules and cross-question consistency checks, followed by human auditing. We also evaluate representative MLLMs on FlameVQA to provide baselines for future work. Results show strong performance when explicit cross-modal cues are available, but notable failures on presence detection under heavy smoke and on coverage estimation. These findings suggest that current MLLMs require domain-specific adaptation to better support disaster and wildfire monitoring. The dataset and benchmark code are open-source at github.com/mobiiin/WildFire_VQA

[81] Safe Autoregressive Image Generation with Iterative Self-Improving Codebooks cs.CV | cs.AIPDF

Yunqi Xue, Zhijiang Li, Philip Torr, Jindong Gu

TL;DR: 本文提出了一种用于安全自回归图像生成的迭代自改进码本方法。该方法利用统一多模态模型自身的理解和判断能力，无需人工标注即可识别不安全生成的图像，并通过固定码本中的固有表示来消除有害映射。该方法包括两个步骤：首先，使用统一模型识别不安全生成并构建有害和安全的图文对，以构建有害空间并指导码本更新；其次，在无害空间内使用安全图文对进行自适应微调以确保生成图像质量，这两个步骤迭代进行直到模型安全不再提升。

Details

Motivation: 自回归统一多模态模型通过顺序预测离散视觉令牌来生成图像，这些令牌来自将嵌入映射到量化视觉模式的码本。这种类似语言的架构使模型能有效捕捉文本条件信息进行生成，但也引发了关于自回归方式生成图像安全性的问题。

Result: 该方法通过迭代自改进码本，在不依赖外部反馈的情况下，提升了模型的安全性，并确保了生成图像的质量。

Insight: 创新点在于利用模型自身能力进行安全评估和码本优化，实现了无需人工标注的迭代安全增强，为自回归图像生成提供了一种内置的安全改进机制。

Abstract: Unlike diffusion-based models that operate in continuous latent spaces, autoregressive unified multimodal models produce images by sequentially predicting discretized visual tokens. These tokens are derived from a codebook that maps embeddings to quantized visual patterns. The language-like architecture enables unified multimodal models to effectively capture text conditional information for generation, making them promising for text-to-image tasks. This also raises an interesting question: how safe are the images generated in such an autoregressive way? In this work, we propose iterative self-improving codebooks for safe autoregressive generation. We leverage the understanding and judgment capabilities of the unified multimodal model itself to identify unsafe generated images without human annotation. Subsequently, the inherent representations in the codebook are fixed to eliminate harmful mappings. Our method comprises two steps: first, we use the unified model to identify unsafe generations and construct corresponding harmful and safe image-text pairs. These pairs are used to construct the Harmful Space and guide updates to the codebook, thereby eliminating harmful outputs. Second, we perform adaptive fine-tuning on the codebook within the harmless space using safe image-text pairs to ensure the quality of generated images. These two steps are repeated until no further improvement is observed, producing a safety-enhanced model codebook. Without additional external feedback, the safety of models is improved iteratively.

[82] HarmVideoBench: Benchmarking Harmful Video Understanding in Large Multimodal Models cs.CV | cs.CLPDF

Jiajun Wu, Haoyu Kang, Yining Sun, Jiacheng Hou, Heng Zhang

TL;DR: 本文提出了HarmVideoBench，一个用于评估大型多模态模型对有害视频理解能力的多层次诊断基准，包含1,379个视频和4,137个多项选择题。该基准从可观察证据、片段内部含义和超越片段推理三个层次评估模型，并引入BCR方法以动态检索上下文，显著提升了基础模型的性能。

Details

Motivation: 现有有害视频基准存在两个主要局限：一是忽略了有害视频的多层次特性，仅将其视为二元分类任务；二是缺乏解释性理由，评估过程如同黑箱。

Result: 在HarmVideoBench上评估了19个领先模型，并提出的BCR方法将基础模型的宏观平均准确率从61.7%提升至84.4%，达到了最先进水平。

Insight: 创新点在于构建了一个多层次、注重深度上下文理解和解释性的有害视频理解基准，并提出了BCR这种动态检索上下文的方法来增强模型推理能力，避免了模型通过表面捷径获得高分。

Abstract: Large vision-language models (LVLMs) have recently shown immense potential in automated content moderation, sparking growing interest in developing harmful-video benchmarks. However, we identify two primary limitations in existing works: 1) The multi-layered characteristics of harmful videos are overlooked. Existing benchmarks predominantly formulate evaluation as a binary classification task, failing to capture implicit or deep contextual harms. 2) Explanatory rationales are completely absent. Current frameworks measure exclusively whether a model flags a video correctly rather than explaining why, turning evaluation into a black box where models can succeed through superficial shortcuts. To address these problems, we present HarmVideoBench, a multi-layered diagnostic benchmark comprising 1,379 videos paired with 4,137 multiple-choice questions. HarmVideoBench benchmarks three hierarchical dimensions: Observable Evidence, Clip-Internal Meaning, and Beyond-Clip Reasoning, aiming to evaluate models’ deep understanding beyond surface cues with carefully balanced and curated samples. We evaluate 19 leading models on HarmVideoBench to assess their multidimensional understanding of harmful videos. Moreover, we introduce BCR, a benchmark-aligned method that predicts reasoning boundaries and dynamically retrieves context only when needed. Experimental results show that BCR substantially improves the base model’s performance in harmful video understanding, raising the macro average from 61.7 percent to a state-of-the-art 84.4 percent.

[83] LISA: Likelihood Score Alignment for Visual-condition Controllable Generation cs.CVPDF

Yanghao Wang, Hongxu Chen, Jiazhen Liu, Zhenqi He, Rui Liu

TL;DR: 本文提出了一种名为LISA（Likelihood Score Alignment）的正则化方法，用于提升视觉条件可控生成任务中侧分支网络的训练效率和生成质量。该方法从基于分数的生成模型视角重新审视了主流双分支范式，通过显式对齐侧分支中间特征与近似似然分数，加速训练收敛并改善合成结果。

Details

Motivation: 当前主流的双分支范式（使用侧分支网络编码视觉条件并融合到冻结的预训练主网络）在视觉条件可控生成中虽成功，但其侧分支的作用和训练效率尚未被充分探索。

Result: 在多种图像/视频任务、架构以及扩散/流模型上的实验表明，LISA能一致地加速训练收敛、提升最终合成结果，并促使侧分支特征在条件建模中更解耦，且额外训练成本可忽略、推理成本为零。

Insight: 创新点在于从分数生成模型的理论视角将主网络解释为提供先验无条件分数，侧网络贡献似然分数，并提出了显式的似然分数对齐正则化方法。这为理解和改进条件生成模型提供了一个新的理论框架和高效的正则化技术。

Abstract: The prevalent dual-branch paradigm, i.e., training a side network to encode visual conditions and fusing its intermediate-layer features to a frozen pretrained main network, has shown remarkable success in visual-condition controllable generation. Despite its widespread adoption, the role of the side branch and its training efficiency remain underexplored. In this paper, we first revisit this mainstream paradigm through the lens of score-based generative modeling: 1) The main network preserves visual perceptual quality by providing a prior unconditional score. 2) The side network steers conditional control by implicitly contributing a likelihood score. Guided by this perspective, we propose LIkelihood Score Alignment (LISA), an effective regularization method that explicitly aligns the intermediate feature of the side network with an approximated likelihood score. Specifically, we first hook features from a designated layer of the side network and project them into the score latent space by a lightweight decoder. Then, we construct an approximated likelihood score target and calculate the distance between the decoder’s output and this target as an additional regularization loss. Finally, we jointly optimize the side network and decoder with both standard diffusion loss and our regularization loss. Experiments across various image/video tasks, architectures, and diffusion/flow models demonstrated that LISA can not only consistently accelerate the training convergence and improve final synthetic results, but also encourage the side network’s features to be more disentangled for conditional modeling with negligible additional training cost and zero extra inference cost.

Jiyong Kim, Shuang Song, Ronjgun Qin

TL;DR: SatSplatDiff是一种用于高保真卫星高斯溅射的几何保持生成细化方法。它通过引入单目深度监督、多尺度几何细化以及阴影引导的生成细化，解决了卫星图像因视角有限导致建筑物立面监督不足、表面空洞和视觉保真度下降的问题，同时最小化了生成细化过程中常见的几何退化。

Details

Motivation: 高斯溅射在卫星三维重建中展现出灵活性和效率，但卫星图像的有限顶部视角导致对建筑物立面的监督不足，产生表面空洞和视觉保真度下降。现有的生成细化方法通过预训练生成先验迭代优化渲染图像，但独立细化每个视图可能导致幻觉和破坏照片一致性，引发几何退化。

Result: 在IARPA2016和DFC2019数据集上的广泛评估表明，该方法实现了最先进的性能，与现有基线相比，几何MAE降低了高达18%，视觉保真度（FID-CLIP）提高了28-45%。它还实现了高达5倍的分辨率增强，幻觉最小且传感器外观一致，展现出无缝的跨瓦片一致性和大规模重建的强大可扩展性。

Insight: 创新点包括：基于先前工作SatSplat的光测DSM初始化和2DGS阴影投射，引入单目深度监督和多尺度几何细化以建立几何准确且正则化的表面表示；以及阴影引导的生成细化，其中几何计算的阴影图指导高斯保持与底层几何的一致性，提高视觉保真度并减少几何退化。从客观角度看，该方法通过结合几何约束和生成先验，有效平衡了视觉质量和几何精度，为卫星三维重建提供了可扩展的解决方案。

Abstract: Gaussian Splatting has been recently explored for satellite 3D reconstruction, demonstrating flexibility and efficiency in representing radiometrically diverse satellite scenes. However, the limited top viewpoint of satellite imagery results in insufficient supervision on building facades, leaving surface holes and degraded visual fidelity. Generative refinement, which leverages pretrained generative priors to iteratively refine and update the rendered images used as supervision targets, has recently been investigated to improve the visual fidelity of Gaussian-rendered images. However, since these models refine each view independently, the resulting images can generate hallucinations and break photo-consistency, leading to geometric degradation. To address these limitations, we propose SatSplatDiff, which aims to minimize geometric degradation prevalent in generative refinement. Building on photogrammetric DSM initialization and 2DGS-based shadow casting established in our prior work SatSplat, we first introduce monocular depth supervision and multi-scale geometric refinement to establish a geometrically accurate and well-regularized surface representation. We then apply shadow-guided generative refinement, where geometrically calculated shadow maps guide the Gaussians to maintain consistency with the underlying geometry, improving visual fidelity while reducing geometric degradation. Extensive evaluations on the IARPA2016 and DFC2019 datasets demonstrate state-of-the-art performance, reducing geometric MAE by up to 18% and improving visual fidelity (FID-CLIP) by 28-45% over existing baselines. Our method delivers up to 5x resolution enhancement with minimal hallucination and sensor-consistent appearance, demonstrating seamless cross-tile consistency and strong scalability for large-scale reconstruction. Source code is available at https://github.com/GDAOSU/SatSplatDiff

[85] CORTEX: A Structured Reasoning Benchmark for Trustworthy 3D Chest CT MLLMs cs.CVPDF

Hashmat Shadab Malik, Anees Ur Rehman Hashmi, Numan Saeed, Muzammal Naseer, Salman Khan

TL;DR: 本文提出了CORTEX，一个用于3D胸部CT的可信多模态大语言模型（MLLM）的结构化推理基准。它通过恢复缺失的诊断推理痕迹（包括任务理解、视觉观察、诊断推理和答案合成四个阶段），为训练和评估3D胸部CT MLLM提供结构化监督和分阶段评估协议。

Details

Motivation: 现有胸部CT问答数据集将专家放射学报告简化为仅答案对，丢弃了连接发现与结论的推理过程，且缺乏患者病史，导致难以解释和验证MLLM的推理，特别是在需要可追溯证据的3D放射学诊断中。

Result: 基于无推理标注的大型公开胸部CT数据集CT-RATE构建，CORTEX包含76,177个经过验证的推理痕迹，涵盖开放式VQA、封闭式VQA和报告生成任务，为构建可信的3D胸部CT推理模型提供了必要的监督和评估协议。

Insight: 创新点在于引入了一个模仿放射科医生工作流程的四阶段结构化推理框架，并通过前沿大语言模型生成、结合自动规则评分和专家评审的协议进行验证，其结构和评估标准均与临床医生紧密合作设计，增强了医学MLLM推理的可解释性和可信度。

Abstract: Reasoning in multimodal large language models (MLLMs) has shown strong promise in medical imaging. However, this reasoning is usually free-form text judged only by its final answer, making it hard to interpret and verify, especially in 3D radiology, where a diagnosis should be traceable to evidence in the scan. Existing chest CT question-answering datasets compound this by reducing expert radiology reports to answer-only pairs, dropping the reasoning that links findings to conclusions and omitting the patient history clinicians rely on. As a result, reasoning-capable 3D chest CT MLLMs remain out of reach, as neither the structured supervision needed to train them nor the protocol needed to verify their reasoning yet exists. We introduce CORTEX (Clinically Organized Reasoning and sTructured EXplanation), a structured reasoning benchmark for 3D chest CT. For each question, CORTEX restores the missing reasoning as a four-stage diagnostic trace mirroring a radiologist’s workflow: task understanding, visual observation, diagnostic reasoning, and answer synthesis. We generate these traces using frontier large language models with broad medical and general-domain knowledge, then filter and verify them with a stage-level evaluation protocol combining automated rubric scoring with expert radiologist review. Crucially, both the reasoning structure and evaluation rubrics are designed in close collaboration with clinicians. Built on CT-RATE, a large, publicly available chest CT dataset without reasoning annotations, CORTEX comprises 76,177 validated reasoning traces across open-ended VQA, closed-ended VQA, and report generation, providing both the structured supervision and the stage-level evaluation protocol needed to build and evaluate trustworthy reasoning models for 3D chest CT. Our dataset and evaluation code will be made publicly available upon acceptance.

[86] Sculpting NeRF Geometry: Human-Preference Fine-Tuning of a 3D-Aware Face GAN cs.CVPDF

Archer Moore, Mingming Gong, Liam Hodgkinson

TL;DR: 本文提出了一种基于人类反馈强化学习（RLHF）的方法，用于直接微调预训练的3D感知生成对抗网络（EG3D）的几何结构。该方法通过一个在辐射场密度值上学习的奖励模型，无需外部网格或形状先验，即可根据人类偏好重塑神经辐射场（NeRF）的3D几何，同时保持2D外观基本不变。

Details

Motivation: 现有3D生成RLHF方法通常优化显式表面表示（如网格），并依赖大量表面监督数据，本文旨在直接基于辐射场密度微调生成模型，避免网格提取和多视图渲染，简化流程。

Result: 在无条件3D人脸GAN（EG3D）上，微调后生成的人脸几何在用户两两比较中获得74.4%的偏好率；尽管FID-50k从4.09上升到6.66，但外观质量损失可控。

Insight: 创新点在于设计了一个直接读取NeRF连续密度场的奖励模型，提供纯几何学习信号，无需文本条件或网格转换；通过密度一致性约束在几何重塑时保持外观，为3D生成RLHF提供了更高效的端到端方案。

Abstract: Reinforcement learning from human feedback (RLHF) for 3D generation is now established across a number of works, but most existing pipelines optimise explicit surface representations, often by converting radiance fields into meshes and training heavily on surface-supervised data. We instead fine-tune a pretrained 3D-aware generative model directly from a learned reward over radiance-field density ($σ$) values, with no externally supplied mesh or shape prior. The reward model requires no pretraining, trains easily on a small set of preference samples, and yields robust improvement in 3D geometry. Working on an unconditional 3D-aware face GAN (EG3D), our reward reads the continuous 3D density field of the neural radiance field (NeRF) directly and supplies a geometry-only learning signal, requiring neither text conditioning, mesh extraction, nor multi-view rendering. A density-consistency constraint keeps the 2D appearance qualitatively similar while the geometry is reshaped, at a measurable but bounded distributional cost (FID-50k rises from 4.09 to 6.66): the fine-tuned generator, trained from the preferences of a single annotator as a proof of concept, produces face geometries preferred by users in 74.4% of pairwise comparisons.

[87] See & Sniff: Learning Visuo-Olfactory Representations cs.CVPDF

Seongyu Kim, Seungwoo Lee, Hyeonggon Ryu, Joon Son Chung, Arda Senocak

TL;DR: 本文提出了SmellNet-V数据集和See & Sniff自监督框架，用于学习视觉与嗅觉的联合表征。通过利用气味在语义类别内视觉不变性的洞察，将单模态嗅觉数据与网络图像合成配对，构建了可扩展的跨模态基准。该方法在仅凭气味的分类任务上超越基线7%，并能泛化到跨模态检索和气味源空间定位任务。

Details

Motivation: 现有多模态模型主要整合视觉与语言、音频或触觉，而嗅觉模态由于缺乏成对的视觉-嗅觉数据尚未被充分探索。本文旨在填补这一空白，推动视觉-嗅觉多模态感知这一新方向的发展。

Result: 在仅凭气味的分类任务上，该方法超越了纯嗅觉基线模型7%。此外，该方法在跨模态检索和本文新提出的像素级气味定位基准上均表现出良好的泛化能力，为视觉-嗅觉学习建立了新的评估标准。

Insight: 核心创新点在于利用‘气味身份在语义类别内对视觉变换具有不变性’这一洞察，低成本地将单模态嗅觉数据集转化为跨模态基准。提出的自监督框架通过密集局部对齐学习联合表征，并能自然地生成用于气味源空间定位的气味显著性图。

Abstract: While modern multimodal models integrate vision with language, audio, or touch, olfaction remains largely unexplored due to the lack of paired visuo-olfactory data. We introduce SmellNet-V, a scalable visuo-olfactory dataset built on the insight that odor identity is largely invariant to visual transformations within a semantic category. This allows us to synthetically pair smell-only samples with semantically aligned in-the-wild web images, converting a unimodal olfactory dataset into a cross-modal benchmark without costly co-collection. Building on this dataset, we propose See & Sniff, a self-supervised framework that learns joint visuo-olfactory representations via dense local alignment and naturally produces smell saliency maps for spatial grounding of odor sources. We further introduce pixel-level smell localization task and a benchmark for evaluation. Our method surpasses smell-only baselines by 7% in smell classification from smell alone and generalizes to cross-modal retrieval and smell localization, establishing visuo-olfactory learning as a new direction in multimodal perception.

[88] ViQ: Text-Aligned Visual Quantized Representations at Any Resolution cs.CVPDF

Xumin Yu, Zuyan Liu, Zhenyu Yang, Yuhao Dong, Shengsheng Qian

TL;DR: 本文提出了ViQ（视觉量化表示）框架，旨在为任意分辨率的视觉输入提供一种统一且通用的离散表示。该方法通过文本对齐预训练和特征离散化两阶段学习，平衡了语义信息和细节保留，并在多模态任务中实现了与最先进连续高维视觉特征编码器相当的性能，同时显著提升了训练效率。

Details

Motivation: 现有方法难以在离散表示中平衡低级细节和高级语义，重建导向的表示缺乏语义信息，而语义强的特征又损失细节。本文旨在解决这一平衡问题，并支持原生分辨率输入，以实现文本和视觉的统一表示。

Result: 在多模态任务上的大量实验表明，ViQ与最先进（SOTA）的连续高维视觉特征编码器相比具有竞争力，同时在低级重建中保持高精度。使用视觉量化表示进行多模态训练可大幅提升效率，在不同基础LLM和训练方案下实现20%-70%的加速。

Insight: 创新点包括：1）将量化学习结构化为文本对齐预训练和特征离散化两阶段；2）提出渐进紧凑特征空间的近端表示学习策略；3）引入支持任意分辨率灵活处理的位置感知头式量化机制。这为构建平衡语义与细节的统一视觉离散表示提供了新思路。

Abstract: A unified representation for text and vision is a natural pursuit, as it enables simpler multimodal modeling and more efficient training. However, representing images as discrete signals in the same way as text inevitably introduces severe information loss. Existing work struggles to balance low-level details and high-level semantics in discrete representations: reconstruction-oriented representations often lack semantic information, whereas semantically stronger features typically suffer from severe loss of detail. We present ViQ, a Visual Quantized Representations framework, which is designed to balance semantics and details in discrete representations while supporting inputs at native resolutions, thereby enabling it to serve as a unified and general discrete representation for arbitrary visual inputs. Our approach structures quantization learning into two stages: text-aligned pre-training and feature discretization. With text-aligned pre-training, we enhance the visual encoder semantic-rich supervision from the pretrained language model and enable it to process native-resolution visual inputs. During discretization, we propose a proximal representation learning strategy to progressively compact the feature space, along with a position-aware head-wise quantization mechanism that enables flexible processing of arbitrary resolutions. Extensive experiments on multimodal tasks demonstrate that ViQ achieves competitive performance compared to state-of-the-art multimodal vision encoders with continuous and high-dimensional visual features, while maintaining high precision in low-level reconstruction. We also show that multimodal training with visual quantized representations largely improves efficiency, yielding up to 20%-70% acceleration with different base LLMs and training recipes.

[89] Not All Actions Are Equal: Rethinking Conditioning for Dexterous World Model cs.CVPDF

Zizhao Yuan, Zhengtu Liang, Taowen Wang, Qiwei Liang, Yichi Wang

TL;DR: 本文提出DexAC-WM，一种针对高自由度灵巧操作的世界模型，通过结构化动作条件化和语义分支改进现有方法。该方法将动作条件化视为结构化过程而非全局压缩，并引入语义分支提供物体-场景先验，从而提升视频预测的视觉-时间真实性和动作跟随一致性。

Details

Motivation: 现有动作条件化世界模型通常将整个动作序列压缩为单一表示，这在低自由度控制中有效，但在高自由度场景下可靠性下降，因为灵巧动作具有异质性，均匀聚合会导致优化不平衡，影响细粒度效果建模和动作保真度。

Result: 在EgoDex和EgoVerse基准测试中，结合语义分支的DexAC显著提高了FID、FVD和PCK指标，证明了视觉-时间真实性和动作跟随一致性的提升。实验还验证了DexAC可扩展到其他骨干网络，展示了其可扩展性。

Insight: 创新点包括将动作条件化重新设计为结构化过程（通过动作标记化和局部细化-全局调制对齐视觉动态），以及引入语义分支提供丰富先验。客观分析认为，该研究强调了高自由度控制需要结构化动作建模和语义基础，为世界模型扩展提供了新思路。

Abstract: Recent advances in action-conditioned world models show promising progress in modeling complex interactions and forecasting future states under diverse action sequences. While these models are often driven by stronger visual representations and model capacity, action conditioning itself remains underexplored. Most existing approaches compress the entire action sequence into a single representation, which works well for low-DoF control but becomes less reliable in high-DoF scenarios. We observe that high-DoF dexterous actions are inherently heterogeneous, spanning multiple orders of magnitude, where large-scale motions coexist with subtle but important signals. When uniformly aggregated, optimization exhibits an imbalance across action components, which hinders the modeling of fine-grained effects and affects action fidelity. We therefore propose DexAC-WM, which treats action conditioning as a structured process rather than global compression. DexAC preserves dimension-level semantics via action tokenization and aligns action signals with visual dynamics through local refinement and global modulation. To address the limited high-level semantic grounding in existing world models, we further introduce a semantic branch that provides rich object-scene priors, which enables world model to capture dynamic visual details while supporting high-DoF action-conditioned video prediction. Experiments on EgoDex and EgoVerse show that combining the semantic branch with DexAC significantly improves FID, FVD, and PCK, demonstrating gains in visual-temporal realism and action-following consistency. We further verify that DexAC extends to other backbones, showing the scalability of our structured action-conditioning design. These results suggest that scaling world models to high-DoF control requires both structured action modeling and semantic grounding.

[90] OctoSense: Self-Supervised Learning for Multimodal Robot Perception cs.CV | cs.ROPDF

Anthony Bisulco, Jeremy Wang, Kostas Daniilidis, Randall Balestriero, Pratik Chaudhari

TL;DR: 本文提出了OctoSense，一个包含多种传感器（立体RGB相机、事件相机、LiDAR、热成像相机、IMU、RTK-GPS和本体感知）的开源机器人感知平台，并发布了包含59小时同步驾驶数据的同名数据集。论文还提出了一种用于多模态自监督学习的’后融合’掩码自编码器方法，该方法通过模态特定的分词器处理不同传感器的时空特性，并在推理时缓存令牌以实现高效处理。

Details

Motivation: 动机在于解决真实世界机器人感知中多模态传感器（具有不同表示、频率、延迟和噪声）数据的有效融合与表示学习问题，特别是在传感器性能退化（如夜间或恶劣条件）的挑战性场景下。

Result: 该方法在NVIDIA 5090和Orin NX上分别仅需6.68毫秒和112毫秒即可计算表示，速度快。在光流估计、深度估计、语义分割和自运动（平移、旋转、转向角）估计等任务上，其性能优于现有的仅基于图像的基础模型，并在夜间或传感器数据退化情况下表现出鲁棒性。

Insight: 创新点在于提出了一个针对异构机器人传感器数据的’后融合’掩码自编码器框架，其核心是使用模态特定的分词器来处理不同传感器的时空特性，以及在推理时缓存令牌以实现对新测量数据的流式高效处理，这为多模态机器人感知提供了一种有效的自监督学习方案。

Abstract: We present OctoSense, an open-source sensor platform with stereo RGB and event cameras, LiDAR, a thermal camera, an inertial measurement unit, RTK-corrected global positioning system, and proprioception (CAN bus data from a car, and joint angles for a quadruped robot). The eponymous OctoSense dataset contains 59 hours of time-synchronized driving data across different types of environments at different times of the day, including situations with highly degraded sensors. We demonstrate multi-modal self-supervised learning using such real-world robotics data, where sensors have different representations, frequencies, latencies and noise. Our approach, a “late-fusion” masked autoencoder, (i) uses modality-specific tokenizers to account for different spatiotemporal characteristics of these sensors, and (ii) caches modality-specific tokens at inference time to process new measurements as they come. This architecture (i) is fast (6.68 ms and 112 ms on NVIDIA 5090 and Orin NX respectively, to compute the representation), (ii) performs better than existing image-only foundation models on tasks such as estimation of optical flow, depth, semantic segmentation, and ego-motion (translation, rotation, and steering angle), and (iii) predicts robustly at nighttime or in situations where sensory data is degraded. See our project page for links to the dataset, code, and supplementary videos: https://abisulco.com/octosense/.

[91] RoPEMover: Depth-Aware Object Relocation via Positional Embeddings cs.CVPDF

Ipek Oztas, Duygu Ceylan, Aybars Bugra Aksoy, Aysegul Dundar

TL;DR: RoPEMover是一种基于扩散变换器位置嵌入的几何感知物体移动方法，通过操纵旋转位置嵌入（RoPE）实现单幅图像中物体的几何一致空间重排，包括处理遮挡、揭示未见区域并保持连贯的阴影和反射效果。

Details

Motivation: 现有方法在单图像物体移动任务中难以保持场景级一致性，如遮挡处理、新区域生成和光影连贯性，因此需要一种能直接操作扩散变换器位置表示以进行几何一致空间重排的新方法。

Result: 在标准物体运动基准测试中，该方法在所有评估指标上均达到了最先进的性能水平，展示了在保持物体身份、生成新区域合理内容及更新场景依赖效果方面的优越性。

Insight: 核心创新在于将2D旋转位置嵌入扩展为深度感知的3D空间结构编码，通过显式操纵RoPE的结构化空间场来实现可控运动；同时采用合成数据与少量真实图像的高效参数微调策略，在最小真实监督下实现了场景一致性保持。

Abstract: Moving an object in a single image requires geometry-consistent spatial rearrangement, including handling occlusions, revealing previously unseen regions, and maintaining coherent shadows and reflections. Existing approaches are not well suited to this setting and often fail to preserve such scene-level consistency. We address this problem by introducing a geometry-aware object motion method that operates directly on the positional representations of diffusion transformers. Our key insight is that rotary positional embeddings (RoPE) define a structured spatial field that can be explicitly manipulated to induce controlled motion. We extend 2D RoPE into a depth-aware formulation that encodes 3D spatial structure, enabling consistent object displacement and scene-aware updates. Our model is trained using synthetic data combined with a small set of real images via parameter-efficient fine-tuning. Despite minimal real supervision, it preserves object identity under large spatial displacements, generates plausible content in newly revealed regions, and consistently updates scene-dependent effects such as shadows and illumination. Experimental results on standard object motion benchmarks demonstrate state-of-the-art performance across all evaluation metrics.

[92] SAM2Matting: Generalized Image and Video Matting cs.CVPDF

Ruiqi Shen, Guangquan Jie, Chang Liu, Henghui Ding

TL;DR: SAM2Matting是一个将视觉目标跟踪器（如SAM2、SAM3）提升为高保真视频抠图系统的框架。它通过区域提议桥和专门的抠图头解耦任务，使跟踪器保持时间一致性，而抠图组件处理精细细节。该框架仅使用图像数据训练，却在视频抠图上达到SOTA，支持多种提示类型，并在人像和野外场景中展现出强大的泛化能力。

Details

Motivation: 解决视频抠图中高层跟踪（需要帧间理解）与低层抠图（关注极细粒度细节）之间的固有差距。现有方法依赖昂贵且范围狭窄的视频抠图数据集，限制了域外泛化并损害了跟踪鲁棒性。

Result: 在视频抠图任务上建立了新的最先进（SOTA）性能。尽管仅使用图像数据训练，但在人像和野外场景的基准测试中均表现出强大的泛化能力和时间一致性。

Insight: 核心创新在于提出一个‘跟踪器到抠图’的解耦框架，通过区域提议桥和专用抠图头，将成熟的VOS跟踪器的时序能力与精细抠图能力结合。这避免了直接训练视频抠图模型对大量视频数据的依赖，实现了仅用图像数据训练即可泛化至视频SOTA性能的突破。

Abstract: Despite impressive advances in image matting, video matting remains challenging due to the inherent gap between high-level tracking, which requires frame-wise understanding, and low-level matting, which focuses on extremely fine-grained details. Existing methods attempt this with expensive and narrowly-scoped video matting datasets, which may limit out-of-domain generalization and compromise tracking robustness. We rethink the paradigm with SAM2Matting, a tracker-to-matting framework that advances VOS trackers to high-fidelity video matting. Specifically, it decouples the task by enhancing a foundational tracker (e.g., SAM2, SAM3) with a region-proposal bridge and dedicated matting heads, enabling the uncompromised tracker to handle temporal consistency while the matting components resolve fine-grained details. Notably, despite being trained only on images, SAM2Matting establishes new state-of-the-art performance on video matting, supports diverse prompt types, maintains strong temporal consistency, and demonstrates robust generalization across both human-centric and in-the-wild scenarios.

[93] RayPE: Ray-Space Positional Encoding for 3D-Aware Video Generation cs.CVPDF

Minghao Yin, Jiahao Lu, Wenbo Hu, Wang Zhao, Shan Ying

TL;DR: 本文提出RayPE，一种用于3D感知视频生成的射线空间位置编码方法，通过将每个token的6D Plucker坐标注入自注意力机制，增强模型对3D场景结构的理解，从而提升相机控制性、跨帧3D一致性和视频质量。

Details

Motivation: 现有视频扩散变换器使用RoPE在(u,v,t)轴上定位token，但未考虑场景的3D结构，因此需要一种能编码3D几何关系的位置编码方法。

Result: 在包含四个数据集的混合训练中，RayPE以不到0.1%的额外参数，通过零初始化从预训练权重开始，显著提高了相机可控性、跨帧3D一致性和整体视频质量。

Insight: 创新点在于利用Plucker互易积的几何关系与注意力点积的代数形式相似性，设计可分解的注意力得分结构，并通过解耦射线方向与矩量级、门控机制和RMSNorm实现跨尺度稳定性。

Abstract: Modern video diffusion transformers position their tokens through RoPE on the (u,v,t) axes – a description of the camera’s sampling grid that says nothing about the 3D structure of the scene. We observe that the geometric relation between two camera rays is captured by the Plucker reciprocal product, which is bilinear in the two rays – the same algebraic form as the dot product in Transformer attention. Building on this analogy, we propose RayPE, a positional-encoding extension that injects per-token 6D Plucker coordinates additively into the queries and keys of self-attention, with a query/key flip arrangement under which the symmetric identity configuration coincides exactly with the reciprocal product. The injection is additive, the resulting attention score decomposes into a content term, a geometry term, and two content and geometry cross-terms – all of which our experiments find individually necessary. To make the encoding stable across video data with heterogeneous camera-translation scales (SfM, deep SLAM, metric), we further decouple ray direction from moment magnitude, gate the encoding by a learned function of the log-magnitude, and apply RMSNorm to align it with the QKNorm-normalized content branch. The full module adds less than 0.1% parameters to a pretrained video DiT, is zero-initialized to start from the pretrained weights, and improves camera controllability, cross-frame 3D consistency, and overall video quality on a four-dataset training mixture.

[94] PhysiFormer: Learning to Simulate Mechanics in World Space cs.CVPDF

Yiming Chen, Yushi Lan, Andrea Vedaldi

TL;DR: PhysiFormer是一种基于扩散变换器的模型，用于模拟物理上合理的3D物体运动。它直接在三维世界坐标系中表示物体网格，通过去噪扩散过程预测顶点轨迹，无需引入刚性或因果性等先验偏置。该模型支持刚性和弹性材料，能处理多物体场景，并泛化到未见过的几何形状和混合材料设置。

Details

Motivation: 现有视频世界模型通常在像素空间中操作，缺乏对物理一致性的显式建模；而现有神经物理方法依赖于特定潜在空间或强加刚性等归纳偏置。本文旨在直接在三维世界坐标中学习物理模拟，避免这些限制，以更通用地捕捉物体动力学。

Result: 在超过10万条模拟轨迹上训练后，PhysiFormer在轨迹精度、刚性保持和基于动量的物理一致性方面显著优于自回归基线模型。它能够泛化到混合材料、未见过的真实世界几何形状以及更多物体数量的场景。

Insight: 创新点在于将顶点轨迹预测建模为直接在三维世界坐标中的单一去噪扩散过程，无需引入刚性或因果性等归纳偏置，从而更灵活地学习物理动力学。模型还采用了在时间、空间和物体维度上分解的注意力机制，实现了高效的置换不变多物体推理。

Abstract: We present PhysiFormer, a diffusion transformer for physically-plausible 3D object motion. Unlike video world models that operate in view-dependent pixel space, PhysiFormer represents objects as 3D meshes expressed in world coordinates. Given the initial vertex positions and velocities, as well as object material type, rigid or elastic, the model samples future vertex trajectories. While related neural physics approaches build on ad-hoc latent spaces or explicitly enforce rigidity and causality, PhysiFormer shows that excellent results can be obtained without any such inductive biases, by casting vertex trajectory prediction as a single denoising diffusion process directly in world coordinates. The probabilistic formulation captures uncertainty in the learned dynamics, enabling diverse plausible futures from initial conditions, making this framework potentially useful for applications with unobserved uncertainty. The model features attention factorised over time, space, and objects for efficiency, enabling permutation-invariant multi-object reasoning without needing explicit object encoding. Trained on over 100k simulated trajectories, PhysiFormer generates rigid and elastic mechanics, and generalises to mixed-material settings, unseen real-world geometries, and larger object counts. It substantially outperforms autoregressive baselines in trajectory accuracy, rigidity preservation, and momentum-based physical consistency. Our results position coordinate-space diffusion as a promising step toward view-invariant, geometry-aware world modelling for robotics, graphics, and physical design. Visualisations, code, and models are available at https://yimingc9.github.io/physiformer.

[95] DnA: Denoising Attention for Visual Tasks cs.CVPDF

Ron Campos, Subhajit Maity, Xin Li, Srijan Das, Aritra Dutta

TL;DR: 本文提出了一种名为DnA（Denoising Attention）的新注意力机制，旨在解决视觉任务中多头注意力（MHA）的softmax激活函数可能产生噪声注意力模式的问题。DnA通过引入正负查询来分别识别正确类别的相关特征和紧密相关但不相关的特征，并将它们投影到具有较大主角度的不同子空间，以促进子空间分离和增强判别力。

Details

Motivation: 标准softmax在多头注意力中可能产生噪声注意力模式，这会稀释相关特征并降低模型性能。论文旨在通过设计一种去噪注意力机制来改善这一缺陷，提升视觉感知任务的性能。

Result: 使用ViT-B骨干网络，DnA在ImageNet-1K上相比基线实现了0.8%的绝对增益。在视频理解任务中，使用视频Transformer和视频LLM也分别取得了1.8%和0.5%的改进。

Insight: 创新点在于引入了正负查询的双查询机制和子空间投影策略，通过分离相关与不相关特征来去噪注意力，这是一种新颖的注意力结构设计，可借鉴用于增强视觉模型的判别能力。

Abstract: The softmax activation in multihead attention (MHA) is the de facto standard for attention-based models in visual perception tasks. However, standard softmax can produce noisy attention patterns that dilute relevant features and degrade its performance. In this paper, we propose Denoising Attention or DnA, in which, first, a positive query identifies which image features belong to the correct class, and a negative query identifies closely associated but irrelevant image features. DnA then projects these interactions into two distinct subspaces with larger principal angles, promoting subspace separation and improved discriminability. Using a ViT-B backbone, our proposed DnA achieves an absolute gain of 0.8% on ImageNet-1K compared to the baseline. We further show improvements across multiple visual understanding tasks, including video understanding with video transformers (1.8%) and video LLMs (0.5%). Our extensive empirical analyses justify the design choices involving two interacting subspaces and the denoising effect of DnA.

[96] Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards cs.CVPDF

Ritesh Thawkar, Shravan Venkatraman, Omkar Thawakar, Abdelrahman Shaker, Fahad Khan

TL;DR: 本文提出了一种自演化的统一多模态模型训练框架，通过内部角色（提议者、求解器、生成器）和自一致性奖励，仅使用未标注图像自主提升视觉理解和图像生成能力，无需人工标注或外部奖励模型。

Details

Motivation: 现有统一多模态模型依赖人工标注或外部奖励进行后训练，本文旨在探索模型能否仅通过未标注图像自主提升视觉理解和生成能力。

Result: 在八个理解指标上均超越基线模型，例如在BAGEL上MMMU提升3.5%，GenEval图像生成性能从82%提升至85%。

Insight: 创新点包括基于令牌熵的连续难度信号、求解器介导的多尺度内部评估机制，以及适用于不同架构（扩散、整流流、自回归）的统一训练框架。

Abstract: Most unified large multimodal models (LMMs) that support both visual understanding and image generation still rely on curated post-training supervision, such as human annotations, preference labels, or external reward models. We ask whether a unified LMM can improve both abilities autonomously using only unlabeled images. We propose a self-evolving training framework with three internal roles: a Proposer that generates visual questions, a Solver that answers and evaluates them, and a Generator that synthesizes images. Training uses only self-derived consistency signals, without human annotations, preference labels, or task-trained external reward/judge models. To stabilize learning, we introduce Solver Token Entropy (STE), a continuous difficulty signal based on token-level prediction uncertainty that remains useful even when sample-level consistency becomes unreliable. For image generation, we design a multi-scale internal evaluation scheme that combines question-answer fidelity scoring with cycle-consistent captioning. This creates a solver-mediated coupling, where better visual understanding enables more reliable generation assessment and stronger internal training signals. The framework preserves the same role decomposition, reward logic, and training schedule across diffusion-based BLIP3o, rectified-flow BAGEL, and autoregressive VARGPT-v1.1 architectures, requiring only each backbone’s native prompting and generation interface. Across eight understanding metrics, our method consistently improves over the corresponding base models. On BAGEL, it achieves a $+3.5%$ absolute gain on MMMU and improves GenEval image generation performance from $82%$ to $85%$. Code and models are publicly released.

[97] Paying More Attention to Visual Tokens in Self-Evolving Large Multimodal Models cs.CVPDF

Shravan Venkatraman, Ritesh Thawkar, Omkar Thawakar, Rao Muhammad Anwer, Hisham Cholakkal

TL;DR: 本文提出了一种名为VISE的无监督自进化框架，旨在解决现有自进化大型多模态模型中的视觉欠条件问题，即模型在生成过程中过度依赖语言先验而非视觉内容。VISE通过几何不变性和语义不变性两种奖励机制，直接正则化模型的视觉条件策略，从而提升模型在图像描述和视觉问答等任务上的性能。

Details

Motivation: 现有自进化大型多模态模型通过多角色自博弈和自一致性奖励方案优化答案一致性，但未能确保解码器关注视觉内容，导致视觉欠条件问题，即模型生成时依赖语言先验而非图像信息，从而在视觉语言理解任务上表现不佳。

Result: 在18个基准测试中，以Qwen3-VL-2B为基础模型，VISE在COCO上CIDEr得分提升+16.85，在TextCaps上提升+19.66，对象幻觉减少5.0 Chair-I点，并在四个模型家族和规模上展现出泛化能力。

Insight: 创新点在于引入几何不变性和语义不变性奖励来直接正则化视觉条件策略，无需专家角色、外部奖励模型或标注数据，仅使用原始未标注图像进行训练，有效缓解了视觉欠条件问题，提升了模型对视觉令牌的关注度。

Abstract: Recently, self-evolving large multimodal models (LMMs) have received attention for improving visual reasoning in a purely unsupervised setting. However, multi-role self-play and self-consistency reward schemes in existing self-evolving LMMs optimize answer agreement without ensuring the decoder attends to visual content, relying instead on statistical language priors to produce self consistent outputs. This leads to a persistent failure mode we term visual under-conditioning, where the decoder relies on language priors rather than the image during generation, manifesting as insufficient attention to visual tokens. As a result, current self-evolving LMMs struggle on vision–language understanding tasks such as image captioning and visual question answering. To address this, we propose VISE (Visual Invariance Self-Evolution), a purely unsupervised self-evolving framework that directly regularizes the model’s visual conditioning policy through two complementary invariance-based rewards: a geometric invariance reward that enforces spatial consistency under known transformations, and a semantic invariance reward that penalizes evidence-agnostic generation by requiring the model to recognize the absence of evidence when predicted regions are perturbed. VISE operates within a single model without specialist roles, external reward models, or annotations, and is trained on raw unlabeled images. Experiments on 18 benchmarks demonstrate the efficacy of our approach. Using Qwen3-VL-2B as the base model, VISE achieves gains of $+16.85$ CIDEr on COCO and $+19.66$ CIDEr on TextCaps, reduces object hallucination by $5.0$ Chair-I points, and generalizes across four model families and scales. Our code and models are available at https://mbzuai-oryx.github.io/VISE

cs.CY [Back]

[98] From Celebrities to Anyone: Characterizing AI Nudification Content, Technology, and Community Dynamics on 4chan cs.CY | cs.AI | cs.CV | cs.HCPDF

Chi Cui, Yixin Wu, Yang Zhang

TL;DR: 本研究对4chan上的AI裸体化内容进行了大规模分析，发现目标已从名人转向普通个人，开源模型主导生产，少数活跃生产者驱动整个生态系统。

Details

Motivation: 先前研究主要关注专门的裸体化平台和模型库，但匿名社区中SNEACI的请求、生成和交换行为尚未被探索，本研究旨在填补这一空白。

Result: 识别了24,105个SNEACI项目，其中55.8%的目标为非名人个体，Stable Diffusion系列生成42.7%的图像，Wan生成66.5%的视频，最活跃的生产者贡献了780个项目。

Insight: 揭示了AI裸体化从公众人物向社交圈内个体扩散的趋势，开源模型和微调教程降低了技术门槛，少数核心生产者通过内容和技术传播维持生态系统，强调了平台治理和技术保障的紧迫性。

Abstract: AI nudification uses generative models to create synthetic non-consensual sexually explicit imagery (SNEACI) of real individuals. Prior work has examined dedicated nudification platforms and model repositories, finding that most targets are female celebrities. However, the anonymous content community, where SNEACI is actively requested, generated, and exchanged, remains unexplored. In this work, we present a large-scale study of AI nudification in the wild, identifying 24,105 SNEACI items. We find a significant shift in target demographics: non-celebrity individuals now account for 55.8% of targets, compared to only 4.7% in prior studies, indicating that AI nudification has expanded from targeting public figures to increasingly harming individuals within users’ own social circles. Meanwhile, open-source models dominate production, with Stable Diffusion family generating 42.7% of images and Wan generating 66.5% of videos, all driven by thousands of shared fine-tuned models and accessible tutorials. Yet the ecosystem runs on a small cohort of active producers, with the most prolific producing 780 items, drives community engagement, shapes target demographics, and disseminates technical knowledge that lowers barriers for new producers. Our work provides an empirical understanding of how AI nudification operates in the wild, revealing the mechanisms that sustain this ecosystem and highlighting the urgent need for interventions in platform governance, technical safeguards, and affected individual protection.

cs.AI [Back]

[99] The Verification Horizon: No Silver Bullet for Coding Agent Rewards cs.AI | cs.CLPDF

Binghai Wang, Chenlong Zhang, Dayiheng Liu, Jiajun Zhang, Jiawei Chen

TL;DR: 本文探讨了当前编码智能体面临的核心挑战：随着基础模型推理能力增强和工程框架日益成熟，生成复杂候选解决方案已不再困难，而可靠验证这些方案成为更棘手的问题。论文从可扩展性、忠实性和鲁棒性三个维度分析验证信号质量，并通过四种奖励构建方式（测试验证器、规则验证器、用户验证器和自动化智能体验证器）在不同任务类型和能力水平下进行实验，证明针对性验证设计能有效抑制奖励攻击并提升任务完成质量。

Details

Motivation: 传统观点认为验证解决方案比生成更容易，但如今编码智能体却面临相反困境：生成复杂候选方案已不再困难，而可靠验证成为主要瓶颈，因为所有验证器都只是人类意图的代理，而非意图本身，这导致验证面临意图不明确和训练中代理与意图差距扩大的双重困难。

Result: 实验表明，针对性验证设计能有效抑制奖励攻击，提高任务完成质量，并在多个内部和公共基准测试中取得显著增益，验证了不同奖励构建方式在不同任务类型和策略能力水平下的有效性。

Insight: 论文的核心洞察是：随着策略能力持续增长，没有固定的奖励函数能保持长期有效，验证机制必须与生成器协同进化；同时，通过从可扩展性、忠实性和鲁棒性三个维度系统设计验证信号，可以更有效地利用奖励信号并缓解代理与意图的偏差问题。

Abstract: A classical intuition holds that verifying a solution is easier than producing one. For today’s coding agents, this intuition is being inverted: as foundation models develop stronger reasoning capabilities and engineering harnesses grow more sophisticated, generating complex candidate solutions is no longer difficult – reliably verifying them has become the harder problem. Every verifier we can build is only a proxy for human intent, never the intent itself. This makes verification subject to a twofold difficulty: first, intent is underspecified by nature, making it inherently hard to faithfully check whether it has been fulfilled; second, during model training, optimization widens the gap between proxy and intent – manifesting as reward hacking or signal saturation. To address this, we characterize the quality of verification signals along three dimensions – scalability, faithfulness, and robustness – and argue that achieving all three simultaneously is the central challenge. We further study four reward constructions: a test verifier for general coding tasks, a rubric verifier for frontend tasks, the user as verifier for real-world agent tasks, and an automated agent verifier for long-horizon tasks. Across different task types and policy capability levels, we conduct in-depth analysis and experiments on the core challenges of reward design and how to more effectively leverage reward signals. Experiments show that targeted verification design can effectively suppress reward hacking, improve task completion quality, and achieve significant gains across multiple internal and public benchmarks. These experiences collectively point to a core observation: no fixed reward function can remain effective as policy capability continues to grow; and verification must co-evolve with the generator.

[100] Narration-of-Thought: Inference-Time Scaffolding for Defeasible Ethical Reasoning in Large Language Models cs.AI | cs.CL | cs.CYPDF

Patrick Cooper, Alvaro Velasquez

TL;DR: 本文提出了Narration-of-Thought（NoT）方法，一种在推理时使用的结构化提示框架，旨在改善大型语言模型在道德困境推理中的缺陷。该方法将思维链分解为五个部分：主角、利益相关者、两步后果、不确定性和最终承诺，无需额外训练或微调。实验表明，NoT能显著减少利益相关者忽略和不确定性压制两种失败模式，并在多轮辩论协议中促进共识达成。

Details

Motivation: 解决标准思维链在道德困境推理中存在的两个主要失败模式：利益相关者忽略（仅考虑单一相关方）和不确定性压制（在做出行动承诺前未明确表达未知因素或保留意见）。

Result: 在来自三个供应商的四个生成模型上，对100个DailyDilemmas场景进行测试，NoT将利益相关者忽略率从最高31%降至1%以下，将不确定性压制率从最高72%降至1%-24%。在利益相关者数量和不确定性得分上，对四个模型中的三个取得了Cliff’s delta优势（分别为+0.79至+0.90和+0.65至+0.93）。在多轮辩论中，将6%的僵局转化为95%的完全共识。

Insight: 创新点在于通过结构化提示（分为五个明确部分）在推理时引导模型进行更全面、审慎的伦理推理，这是一种无需训练的参数高效方法。该方法将推理过程外部化，提供了可审计的决策依据，对于构建可靠的人工智能代理具有借鉴意义。

Abstract: Standard chain-of-thought on moral dilemmas exhibits two failure modes: stakeholder collapse (the trace names at most one party with a stake in the outcome) and uncertainty suppression (no explicit unknowns or hedges before committing to an action). We introduce narration-of-thought (NoT), a system prompt that structures chain-of-thought into five sections: protagonist, stakeholders, two-step consequences, uncertainty, then commitment. NoT adds no training, parameters, or fine-tuning. On 100 DailyDilemmas scenarios across four generators from three vendors, NoT cuts stakeholder collapse from up to 31% to under 1% and uncertainty suppression from up to 72% to 1-24% on every model. A matched-budget verbose-CoT control rules out token spend as the active ingredient; NoT retains Cliff’s delta advantages of +0.79 to +0.90 on stakeholder count and +0.65 to +0.93 on uncertainty score for three of four generators, and a section ablation attributes each shift to its specific sub-instruction. Textual-gradient descent initialised at NoT improves the scaffold further; a cross-family training judge (different vendor from the generator) dominates an in-family one on every measured axis. Extended to a five-round multi-stakeholder debate protocol, the scaffold converts a 6% standoff into 95% full consensus on a calibration set and 100% combined convergence on a DailyDilemmas replication. The resulting traces externalise the stakeholders, consequences, and uncertainty grounding each commitment, providing an auditable substrate for dependable agentic deployment.

[101] Humans Disengage, Reasoning Models Persist: Separating Difficulty Registration from Deliberation Allocation cs.AI | cs.CLPDF

Han-yu Wang

TL;DR: 该论文揭示了大型推理模型（LRMs）与人类在问题解决过程中时间分配模式的深层差异。研究发现，尽管两者在跨项目层面都表现出处理时间与问题难度正相关，但在同一项目内部，LRM在答错时会消耗更多计算资源（tokens），而人类则在答错时花费更少时间。

Details

Motivation: 研究旨在探究大型推理模型与人类在推理过程中时间分配策略的本质差异，特别是区分“难度感知”和“审议分配”这两个层面，以揭示表面相似性下的不同内在机制。

Result: 在公开的人类-LRM匹配语料库（如H-ARC）上，所有五个“思考型”LRM都表现出显著的“错误比正确消耗更多”效应（Cohen’s d = 1.47-3.13），而人类则表现出相反的效应。该结果在控制项目固定效应后依然稳健，并在多个数据集上可复现。

Insight: 核心创新在于将审议过程分解为“难度注册”和“分配”两个层面，并发现了LRM与人类在“分配”策略上的根本对立：人类倾向于“投入-放弃”策略，而LRM则遵循“不确定性驱动长度”策略。这提示仅用处理时长作为难度指标会掩盖关键的认知控制差异。

Abstract: Large reasoning models (LRMs) take longer on harder problems, just as humans do. This surface similarity hides an opposite pattern within items. When an LRM gets a problem wrong, it spends more tokens than when it gets the same problem right; humans do the reverse, spending less time on the trials they get wrong. We separate two levels of deliberation: how response time tracks difficulty across items (registration), and, with item identity held fixed, whether an agent spends more on its own failures or successes (allocation). On a public matched human-LRM corpus, humans and all five thinking LRMs reproduce the known cross-item alignment (registration) but diverge within items (allocation): every LRM shows a large wrong-vs-right effect (Cohen’s d = 1.47-3.13 on H-ARC) while humans show the opposite sign. The comparison stays inside each agent’s own scale; we never put seconds and tokens on one axis. The dissociation holds under item fixed effects, replicates across datasets, and is absent in a non-thinking baseline. We read the human pattern as engagement versus abandonment: people stay on items they expect to solve and give up on the rest. We read the LRM pattern as length driven by uncertainty: chains grow when the model is unsure, which is exactly when it tends to fail. Both policies produce the same cross-item correlation with difficulty, so they look aligned on the measure prior work has used; the divergence shows up only once item identity is fixed. Under resource-rational metareasoning, the split is between two stopping policies that share a difficulty signal but implement opposite control; trace length captures the signal and misses the control.

[102] Do Safety Guardrails Need to Reason? LeanGuard: A Fast and Light Approach for Robust Moderation cs.AI | cs.CLPDF

Dongbin Na

TL;DR: 本文质疑安全护栏是否需要推理步骤，提出了一种轻量级、快速的标签编码器LeanGuard。通过对比实验，发现去除推理链不会降低审核准确性，且LeanGuard在计算效率、鲁棒性和严格误报率下的召回率方面均优于基于推理的护栏方法。

Details

Motivation: 当前安全护栏方法通常依赖生成思维链（CoT）进行决策，导致模型沉重且推理缓慢，不适用于需快速响应或部署在资源受限设备（如机器人）的场景。本文旨在探究推理步骤对于内容审核是否必要。

Result: LeanGuard（一个395M参数的标签编码器）在公开基准测试中平均F1得分为82.90±0.26，与基于更大解码器的推理护栏性能相当，但推理计算量减少约100倍。在训练标签噪声下保持鲁棒性，且在严格误报率下召回率更高。

Insight: 创新点在于挑战了“推理链提升决策”的常见假设，通过控制变量实验证明推理对审核准确性无显著改善。LeanGuard采用轻量双向编码器，仅需单次前向传播，为高效、鲁棒的安全护栏设计提供了新思路，并指出当前基准可能不足以激励推理的必要性。

Abstract: In order to screen a prompt or a response, the recent guardrail methods generate a chain-of-thought (CoT) before they issue a verdict. This design follows a common belief that step-by-step reasoning improves a decision. However, CoT also makes the guard heavy and slow, because the model must generate many tokens before it decides. This may not match how guardrails are actually deployed. A guardrail sometimes should not be heavy and slow, and it often runs on-device, for example on an embodied robot. In this paper, we pose a question whether a safety guardrail really needs to reason. To answer this question, we train a lightweight bidirectional encoder and a reasoning guard on the same corpus, and we then remove only the reasoning while we keep everything else fixed. With this controlled same-base comparison, we show that the chain does not improve moderation accuracy. We name the resulting guard LeanGuard. A 395M label-only encoder reaches an average F1 of 82.90 $\pm$ 0.26 over public benchmarks. It matches a reasoning guard that is built on a much larger decoder, while it uses only a single forward pass over an input of at most 512 tokens. This is about a ~100x reduction in inference compute. We further show that this label-only encoder stays robust under training-label noise and retains far more recall at a strict false-positive rate than the reasoning guard, so a heavier reasoning guard is not the more robust choice either. Our finding suggests that the current guardrail benchmarks may not be hard enough to reward reasoning, and that the necessity of CoT for moderation is still not proven. We release all source codes and models including LeanGuard at https://github.com/ndb796/LeanGuard.

[103] KARLA: Knowledge-base Augmented Retrieval for Language Models cs.AI | cs.CLPDF

Francois Crespin, Fabian M. Suchanek, Nils Holzenberger

TL;DR: 本文提出了一种名为KARLA的新方法，使大型语言模型（LLM）在生成文本时能自动从知识库中检索事实知识。该方法无需重新训练模型即可更新输出中的事实，并能追溯事实来源以提高透明度和可解释性，同时让较小模型达到与较大模型相当的事实准确性。

Details

Motivation: 解决LLM输出中事实知识过时、不可追溯以及模型规模与事实准确性强耦合的问题，旨在实现无需重新训练即可更新知识、提高输出透明度和降低模型规模依赖。

Result: 实验表明，该方法在短文本和长文本生成任务中都改善了事实基础，通过知识库编辑而非参数更新即可实现事实修订。

Insight: 核心创新在于训练模型生成特殊标记来触发知识库查询，实现了知识检索与文本生成的动态、可解释性结合，为构建可更新、可追溯的知识增强型语言模型提供了新思路。

Abstract: We propose a new method that allows an LLM to automatically pull in factual knowledge from a knowledge base during token generation. This means that (1)~~factual knowledge in the LLM output can be updated without retraining the LLM, (2)~~facts in the LLM output can be traced to the knowledge base for transparency and explainability, and (3)~smaller models can achieve the same factual accuracy as larger models. Our core idea is to train the model to produce special tokens that trigger a query to the knowledge base. Our experiments show that our method improves factual grounding in both short and long-form generation, and allows factual revisions to take effect through KB edits rather than parameter updates.

[104] Einstein World Models cs.AI | cs.CL | cs.CVPDF

Munachiso Samuel Nwadike, Zangir Iklassov, Ali Mekky, Zayd M. Kawakibi Zuhri, Kentaro Inui

TL;DR: 本文提出爱因斯坦世界模型（EWM），一种将视觉-时间推演融入推理过程的LLM框架，旨在通过可视化反事实事件来增强大语言模型的复杂推理能力。EWM允许LLM调用世界模块生成场景的短时推演，并将其作为可检验的假设来支持后续推理。

Details

Motivation: 研究动机是探索智能体是否需要超越直接经验进行推理，特别是可视化反事实事件能否作为语言之外的机制来补充复杂思维，并训练LLM利用这种可视化机制提升推理能力。

Result: 论文未在摘要中提及具体实验结果或基准测试，但提出了EWM作为增强LLM推理能力的蓝图框架。

Insight: 创新点在于将工具调用（如网络搜索或代码执行）扩展到视觉思维实验领域，通过视觉-时间推演作为可检验假设来弥补纯文本推理的局限性，为LLM提供了新的推理范式。

Abstract: Does intelligence require the ability to reason about phenomena beyond direct experience? It is natural to suspect that some complex thought cannot be captured through language alone. However, of particular concern to this work, is whether visualising counterfactual events can complement language as a mechanism for complex thought. We ask whether LLMs can be trained to utilise such visualisation mechanisms, in a way that benefits their reasoning abilities. Motivated by this question, we propose Einstein World Models. EWMs are a blueprint for LLM-based reasoning systems that place visual-temporal rollouts inside the reasoning trace, allowing them to reason in ways that text alone may not support well. In an EWM, the LLM calls a world-module (not to be confused with a world model), to produce short rollouts of scenes under consideration. The returned rollout is treated not as the answer, but as an inspectable hypothesis that can support later reasoning. Einstein World Models extend the capability of LLMs for tool calling (such as web search or code execution), into the domain of visual thought experiments.

[105] ResilPhase: Plug-and-Play Phase Mapping and Noise-Resilient Macro-Trajectory Extrapolation for Diffusion Acceleration cs.AI | cs.CVPDF

Qicheng Zhao, Yu Li, Qi Sun, Zheyu Yan

TL;DR: 本文提出ResilPhase框架，通过将扩散模型加速问题重新定义为在ODE空间中进行稳定宏观轨迹外推，解决了现有‘缓存-预测’方法在高加速比下质量严重下降的问题。该方法通过全局漂移对齐、无导数重心拉格朗日外推器和有界相位映射，实现了噪声鲁棒的加速。

Details

Motivation: 现有基于导数多项式的扩散Transformer加速方法（如‘缓存-预测’方案）在高加速比下出现严重质量退化，其根本原因是离散外推与连续扩散轨迹未对齐且数值不稳定，导致空间误差累积、噪声导数放大和高阶不稳定性。

Result: 在FLUX.1-dev和HunyuanVideo基准测试中，ResilPhase在激进加速比下实现了最先进的保真度（SOTA）。

Insight: 创新点包括：将加速推理重新定义为ODE空间中的宏观轨迹外推以消除特征不一致；引入无导数重心拉格朗日外推器绕过导数不稳定性；提出有界相位映射正则化外推域以抑制振荡误差增长。这些构成了一个即插即用、噪声鲁棒的加速框架。

Abstract: The adoption of powerful diffusion models is hindered by their significant inference latency. Recent ``cache-then-forecast’’ schemes alleviate this issue by accelerating DiTs using derivative-based polynomials, but they suffer from severe quality degradation at high acceleration ratios. Our analysis reveals its root cause: the discrete extrapolation performed on representations that are misaligned with the continuous diffusion trajectory and are numerically unstable. Thus, accelerated DiTs suffer from accumulated spatial errors, noisy derivative amplification, and high-order instability. We therefore reformulate accelerated inference as stable macro-trajectory extrapolation in ordinary differential equation (ODE) space. Instead of predicting intermediate features, we align forecasting with the model’s Global Drift (GD), i.e., the end-to-end state evolution, thereby eliminating feature inconsistency and memory overhead. However, even this smooth macro-trajectory remains vulnerable to the derivative fallacy: its higher-order temporal derivatives are intrinsically noisy. Thus, we introduce a derivative-free barycentric Lagrange extrapolator to effectively bypass derivative instability and approximation error. We further propose a bounded Phase Mapping that regularizes the extrapolation domain, suppressing oscillatory error growth. These elements collectively constitute ResilPhase, a noise-resilient acceleration framework. Experiments on FLUX.1-dev and HunyuanVideo demonstrate state-of-the-art fidelity under aggressive acceleration ratios.

[106] EO-WM: A Physically Informed World Model for Probabilistic Earth Observation Forecasting cs.AI | cs.CVPDF

Junwei Luo, Shuai Yuan, Zhenya Yang, Yansheng Li, Zhe Liu

TL;DR: 本文提出EO-WM，一个用于多光谱地球观测概率性预测的视频扩散Transformer模型。该模型通过一个物理信息驱动的条件框架，将气象强迫分解为气候基线、天气异常和累积物理应力信号，以更好地捕捉天气变化对地表动态的影响。作者还引入了两个诊断性基准来评估模型对天气变化的响应能力，实验表明EO-WM在预测植被退化幅度和方向准确性方面优于现有方法。

Details

Motivation: 现有地球观测预测方法未能充分处理任务中的不确定性以及气象条件作为驱动信号的作用。确定性模型将不确定性坍缩为单一预测，而基于扩散的方法通常将天气变量视为未分化的条件信号，且现有基准主要关注重建精度而非对天气强迫变化的正确响应。

Result: 在提出的极端夏季基准和季节匹配对基准上，EO-WM将预测的归一化植被指数下降幅度的误差相对降低了5.63%，并将方向命中率相对提高了7.80%，同时在标准像素级指标上保持竞争力。

Insight: 创新点在于提出了一个物理信息驱动的条件框架，将气象强迫分解为不同成分并通过独立路径处理，以及引入了专门评估天气响应行为的诊断性基准。这为处理部分可观测、天气驱动的世界建模问题提供了更结构化的方法。

Abstract: Earth Observation (EO) forecasting aims to predict future Earth surface dynamics from satellite observations under changing meteorological conditions. In this paper, we view this task as a partially observed, weather-driven world modeling problem, in which weather acts as a conditioning signal, while forecasting remains uncertain due to sparse observations and unobserved land-surface states. However, existing methods do not fully capture this setting: deterministic models collapse uncertainty into a single future prediction, while diffusion-based methods typically treat weather variables as undifferentiated conditioning signals, and existing benchmarks focus mainly on reconstruction accuracy rather than whether forecasts respond correctly to changed weather forcing.We introduce EO-WM, a video diffusion transformer for multispectral EO forecasting. EO-WM incorporates a physically informed conditioning framework that represents meteorological forcing through a climatological baseline, weather anomalies, and cumulative physical stress signals. Specifically, it separates baseline and anomaly through distinct conditioning pathways, and accumulates anomalous forcing over time to capture sustained heat and drought stress. To evaluate weather-response behavior beyond standard metrics, we introduce two diagnostic benchmarks: an Extreme Summer Benchmark for severity-aware prediction of vegetation degradation under extreme weather, and a Seasonal Matched-Pair Benchmark for testing response fidelity under changed weather forcing. Experiments show that EO-WM reduces the error in predicted Normalized Difference Vegetation Index (NDVI) decline amplitude by a relative 5.63% and improves directional hit rate by a relative 7.80%, while remaining competitive on standard pixel-level metrics. The benchmarks and model will be made open-source at https://github.com/Luo-Z13/EO-WM.

cs.NI [Back]

[107] Dot-Flik: A Scalable Edge AI Architecture for Distributed Insect Monitoring cs.NI | cs.AI | cs.CV | cs.LGPDF

Mattia Consani, Denisa-Andreea Constantinescu, Åse Håtveit, Titus Venverloo, Fabio Duarte

TL;DR: 本文提出Dot-Flik，一种用于分布式昆虫监测的可扩展边缘AI架构。该架构通过运动信息帧过滤算法在边缘设备上预处理视频流，丢弃无关帧以降低数据传输和中央处理需求，并采用分层物联网设计实现数据采集与AI分类的解耦。

Details

Motivation: 现有基于视觉的昆虫监测系统存在硬件成本高、能耗大、依赖集中处理或云连接等问题，难以实现大规模、连续性的监测。本文旨在解决这些限制，为城市环境中的密集、低成本生物多样性监测网络奠定基础。

Result: 在真实户外部署中，系统在微风条件下实现了60-80%的帧率降低，维持了30 FPS的实时处理性能（计算余量12.8毫秒），节能高达22.6%，且每个中央节点可支持5-6个并发边缘流。

Insight: 创新点在于提出了一种无需在传感设备上进行深度学习推理的边缘预处理算法，以及一种解耦数据采集与AI分类的分层分布式架构，实现了处理需求的分数级扩展和监测覆盖率的显著提升。

Abstract: Global insect population declines necessitate scalable, continuous monitoring systems, yet existing vision-based solutions remain constrained by high hardware costs, energy demands, and reliance on centralized processing or cloud connectivity. This article presents three contributions to address these limitations. First, we propose a motion-informed frame filtering algorithm based on temporal differencing, gamma-corrected motion amplification, and block-based motion density analysis that discards irrelevant frames at the edge while preserving insect activity, without requiring deep learning inference on the sensing device. Second, we introduce a distributed, hierarchical IoT architecture that decouples data acquisition from AI classification through this edge-level preprocessing, projecting fractional scaling of central processing requirements and significantly increasing monitoring coverage compared to monolithic single-stream approaches. Third, we validate the complete system through real-world outdoor deployments on low-cost commodity hardware along four axes: real-time performance, network scalability, hardware cost, and energy efficiency under varying wind conditions. Results demonstrate 60-80% frame reduction under light-wind conditions, sustained real-time 30 FPS operation with 12.8 ms of computational headroom, up to 22.6% energy savings, and support for 5-6 concurrent edge streams per central node. These findings establish a practical foundation for dense, low-cost biodiversity monitoring networks in urban environments.

cs.LG [Back]

[108] Epiphany-Aware KV Cache Eviction Without the Attention Matrix cs.LG | cs.CLPDF

Steven Kolawole, Virginia Smith

TL;DR: 该论文提出了一种名为EpiKV的KV缓存淘汰方法，通过使用’顿悟分数’（epiphany score）来评估token重要性，无需计算注意力矩阵。该方法可直接集成到现有FlashAttention推理栈中，显著扩展可行上下文长度，在数学推理基准测试中达到或超越基于注意力的基线方法。

Details

Motivation: 解决长链思维推理中KV缓存成为部署瓶颈的问题。现有基于注意力权重的缓存淘汰方法在长推理轨迹中重要性评估不准确，且强制模型实现注意力矩阵会阻碍生产推理中融合内核的使用。

Result: 在MATH-500基准测试中，4096-token缓存下EpiKV达到72%准确率，匹配最强基于注意力的基线（ThinKV 71%）；在AIME-2024基准测试中，8192-token缓存下改进变体达到37%准确率，优于最佳基线（33%），推理速度提升最高达2.8倍。

Insight: 创新点在于提出’顿悟分数’作为token重要性度量，直接从前向传播中读取模型内部表示变化，无需注意力矩阵和额外状态。该方法无需训练、分类器或定制内核，可直接兼容现有推理框架，通过因果滚动z-score消除位置趋势，实现了更高效的缓存管理。

Abstract: As reasoning models emit chains of thought tens of thousands of tokens long, KV cache increasingly becomes a deployment bottleneck. Existing cache eviction methods rank tokens by attention weight, which is a noisy importance proxy in long reasoning traces, and prohibits the use of fused kernels in production inference by forcing the model to materialize the attention matrix. In this work, we instead score tokens with a metric we term the epiphany score: the change in the model’s internal representation, read directly from the forward pass with no attention matrix and negligible extra state. Our resulting cache eviction method, EpiKV, requires no training, classifier, or custom kernel, and can be used directly in FlashAttention inference stacks unchanged – scaling to a 16x longer feasible context than attention-based scoring. upper-mid layers negatively) and remove a positional trend with a causal rolling z-score. At a 4096-token cache EpiKV reaches 72% on MATH-500, matching the strongest attention-based baseline (ThinKV 71%, H2O 67%); a lag-normalized KV variant reaches 37% on AIME-2024 at 8192 tokens against the best of them (33%), at up to 2.8x the speed.

[109] HyperDFlash: MHC-Aligned Block Speculative Decoding with Gated Residual Reduction cs.LG | cs.CLPDF

Luxi Lin, Shuang Peng, Rui Ma, Junhao Hua, Shuwei Fan

TL;DR: HyperDFlash是一个针对DeepSeek-V4多超连接架构优化的块并行推测解码框架。它通过采用预折叠残差状态作为条件信号、使用轻量级门控残差缩减器以及引入针对性的KL蒸馏损失，解决了原生多令牌预测模块在后期位置精度下降以及DFlash方法无法直接适配多路径残差流的问题。

Details

Motivation: DeepSeek-V4的多超连接架构导致其多路径残差流与传统的推测解码设计存在特征错位，使得高效的块并行解码方法DFlash无法直接应用，同时其原生多令牌预测模块存在错误累积导致后期接受率下降的问题。

Result: 在数学推理、代码合成和对话基准测试中，HyperDFlash在平均接受的草稿长度和解码加速方面均优于原生MTP基线和普通DFlash适配，取得了显著的性能提升。

Insight: 创新点在于提出了与目标模型架构对齐的优化：使用预折叠残差状态保留多路径结构信息，以及利用内置超连接头参数构建轻量级门控残差缩减器进行输入感知的路径聚合，大幅减少了参数量并保持了架构一致性。

Abstract: We present HyperDFlash, a block-parallel speculative decoding framework tailored to the novel multi-hyper-connection (MHC) architecture proposed by DeepSeek-V4. Despite the strong initial-token drafting performance of the native Multi-Token Prediction (MTP) module in DeepSeek-V4, its draft accuracy degrades sharply at later positions, as error accumulation from unverified intermediate tokens harms acceptance rates. Although the original DFlash method supports efficient one-pass block drafting, it cannot be seamlessly adapted to the MHC paradigm, since the multi-path residual stream of DeepSeek-V4 induces feature misalignment with conventional drafting designs. To resolve this mismatch, we propose two model-aligned optimizations for MHC residual streams. First, we adopt pre-collapse residual states as the exclusive conditioning signal, preserving multi-path structural information and aligning the drafter with the native prediction pathway of the target model. Second, we replace the heavy generic linear compressor with a lightweight gated residual reducer, whose parameters are inherited from the built-in hyper-connection head. This design yields input-aware path aggregation with three orders of magnitude fewer parameters while maintaining architectural alignment. We further enhance training via a targeted KL distillation loss applied to the LM-head, which regularizes predictions against the full target probability distribution and improves draft quality at early training stages. Experiments across math reasoning, code synthesis, and conversational benchmarks show that HyperDFlash consistently outperforms both the native MTP baseline and vanilla DFlash adaptation. It achieves substantial gains in average accepted draft length and decoding speedup, validating the effectiveness of MHC alignment, gated reduction, and targeted distillation for high-performance speculative decoding.

[110] Structure Before Collapse: Transient semantic geometry in next-token prediction cs.LG | cs.CLPDF

Yize Zhao, Isabel Papadimitriou, Christos Thrampoulidis

TL;DR: 这篇论文研究了在下一词预测语言模型的训练中，尽管使用独热标签，梯度下降如何能够发现潜在的语义结构。作者通过三个受控合成实验发现，语义几何结构在训练早期出现，但最终会消失，模型会达到所有表征等距的对称状态。

Details

Motivation: 动机在于解决一个矛盾：下一词预测语言模型主要使用独热标签训练，这理论上应导致表征仅依赖输出标签而忽略输入语义相似性，但模型却明显学到了潜在的语义结构。论文旨在探究梯度下降如何在共现统计坍缩为独热稀疏性时发现这种分类语义结构。

Result: 在三个受控合成设置中，研究发现语义几何结构在训练早期出现，表征会根据共享属性聚类，尽管没有明确的监督信号。然而，这种结构是瞬态的；随着模型容量和训练时间的增加，最终会达到所有表征等距的预测对称状态。

Insight: 创新点在于揭示了在独热标签训练下，语义结构是训练动态中一个早期、瞬态的阶段，而非最终平衡态。这挑战了神经坍缩理论在语言模型中的直接适用性，并提出了对常用无约束特征模型的初步修改，以捕捉这种涌现的语义几何。

Abstract: Neural Collapse predicts that balanced one-hot classification pushes model representations to be equally far from each other; a symmetric configuration that depends only on the output label and ignores any semantic similarity in the inputs. This creates a puzzle: next-token prediction language models are trained predominantly (as context length increases) with one-hot labels: the same context is very unlikely to appear twice in training with different labels. However, they clearly learn latent structural features. That is, despite the one-hot training regime, a language model’s contextual embeddings represent the fact that the next word in ‘’Mary broke the ___’’ is likely to be filled by tokens in the latent classes of a) medium-sized, b) rigid, c) inanimate nouns. How does gradient descent find such categorical semantic structure when co-occurrence statistics collapse to one-hot sparsity, eliminating any shared next-tokens among different contexts? To investigate this tension we identify three synthetic controlled settings where inputs have latent semantic factors but are mapped to distinct one-hot labels. We find that semantic geometry emerges early in training, and that representations cluster by shared attributes despite receiving no explicit supervision to do so. This structure is transient: with sufficient capacity and time, the model eventually reaches the predicted symmetric state where all representations are equally separated. We study this phase transition through Gram matrix analysis and propose a preliminary modification to the commonly used unconstrained features model to capture the emergent semantic geometry.

[111] AIGP: An LLM-Based Framework for Long-Term Value Alignment in E-Commerce Pricing cs.LG | cs.AI | cs.CLPDF

Chennan Ma, Yanning Zhang, Siqi Hong, Xiuchong Wang, Fei Xiao

TL;DR: 该论文提出了一种名为AIGP的新型框架，利用大语言模型（LLM）结合领域知识、结构化数据和文本上下文，为电子商务定价做出可解释的、知识感知的决策，旨在解决传统动态定价模型在可解释性、非结构化信息利用以及与长期业务目标对齐方面的不足。

Details

Motivation: 传统的大规模电商动态定价模型存在可解释性有限、难以利用非结构化信息、以及与长期业务目标（如累计商品交易总额、投资回报率和里程碑达成率）不一致的问题。

Result: 在淘宝工厂平台上的大规模离线评估和在线A/B测试表明，与生产基线相比，AIGP在14天内实现了显著提升：商品交易总额（GMV）增长13.21%，投资回报率（ROI）增长7.59%，里程碑达成率增长8.20%。

Insight: 论文的核心创新点在于提出了一个结合LLM、离线强化学习（用于训练长期价值评估器LTVE）和直接偏好优化（DPO）的端到端框架，以实现与长期业务目标对齐的可解释定价决策，并通过监督微调进行知识蒸馏以实现高效部署。

Abstract: Traditional dynamic pricing models in large-scale e-commerce suffer from limited interpretability, poor utilization of unstructured information, and misalignment with long-term business objectives such as cumulative Gross Merchandise Value (GMV), Return on Investment (ROI) and milestone achievement. We propose AIGP, a novel framework that leverages a Large Language Model (LLM) prompted with domain knowledge, structured data and textual context to make interpretable, knowledge-aware pricing decisions. For efficient deployment while maintaining high-quality outputs, we employ supervised fine-tuning for knowledge distillation. Central to AIGP is the Long-Term Value Estimator (LTVE), trained via offline reinforcement learning on historical data, which serves as a reward model to score candidate pricing actions and select preference pairs for Direct Preference Optimization (DPO), thereby aligning the pricing policy with long-term business objectives. Extensive offline evaluations and large-scale online A/B tests on Tao Factory demonstrate that AIGP achieves significant improvements: +13.21% in GMV, +7.59% in ROI, and +8.20% in milestone achievement rate over 14 days compared to the production baseline, while simultaneously providing interpretable and transparent pricing rationales.

[112] Just how sure are you? Improving Verbalized Uncertainty Calibration in Medical VQA cs.LG | cs.CL | cs.CVPDF

Eren Senoglu, Federico Toschi, Nicolo Brunello, Andrea Sassella, Mark James Carman

TL;DR: 本文提出了一种基于训练的框架，用于改善多模态大语言模型在医学视觉问答任务中的口头不确定性校准。该方法通过组合损失函数（包括Brier校准项、锚点正则化、对比图像文本对齐项和KL模型稳定项）对模型进行微调，以减少校准误差并提高判别力，同时保持预测准确性。

Details

Motivation: 现有的口头置信度校准方法主要针对纯文本LLM开发，未考虑医学图像理解的多模态特性，导致MLLM在医学VQA中往往产生过度自信的输出，无论实际正确与否。

Result: 在三个医学VQA基准测试和两种架构（MedGemma 4B IT和Qwen2 VL 7B Instruct）上，该方法将校准误差降低了60%以上，并将判别力提高了26%以上，同时保持了预测准确性，平均性能优于基于提示、采样和训练的方法。

Insight: 创新点包括：1) 采用基于2x2因子扰动设计的对比图像文本对齐信号，以探究模型对视觉模态输入与语言先验的依赖；2) 使用组合损失函数，其中每个组件对改善校准都是必要的；3) 引入top K KL散度正则化器以保护微调过程中的模型回答能力。

Abstract: Multimodal large language models (MLLMs) applied to Medical Visual Question Answering (VQA) tend to produce overconfident outputs regardless of actual correctness, and existing verbalized confidence calibration methods, developed primarily for text only LLMs, do not account for the multimodal nature of medical image understanding. This work proposes a training based framework that finetunes MLLMs to improve their calibration using a composite loss function combining a Brier style calibration term, an anchor regularizer that prevents confidence collapse toward extreme values, a contrastive image text alignment term, and a KL based model stabilization term. The alignment signal is derived from a $2 \times 2$ factorial perturbation design that crosses image presence with text integrity, probing the reliance of the model on visual modality input versus language priors. Finally, a top K KL divergence regularizer is used to protect the answering ability of the model during finetuning. Across three Medical VQA benchmarks and two architectures (MedGemma 4B IT and Qwen2 VL 7B Instruct), our method reduces calibration error by 60% or more, and improves discrimination by 26% or more, while preserving predictive accuracy. On average across benchmarks, the technique outperforms prompting based, sampling based, and training based approaches, and ablation experiments confirm that each component of the loss function is indeed necessary for improving the calibration. All code for the experiments is publicly available.

[113] Fast LeWorldModel cs.LG | cs.CV | cs.ROPDF

Yuntian Gao, Xiangyu Xu

TL;DR: 本文提出Fast LeWorldModel (Fast-LeWM)，一种快速的潜在世界模型，用于视觉规划。它通过动作前缀预测替代了LeWM中自回归的局部单步潜在状态转移，从而直接建模不同时间跨度下累积的动作效果，显著降低了规划计算成本并减少了长时域预测中的潜在误差累积。

Details

Motivation: 现有LeWorldModel (LeWM)等联合嵌入预测架构在视觉规划时，需要通过自回归的单步潜在转移模型反复评估候选动作序列，导致计算开销大，且随着预测时域增长，潜在误差会不断累积。

Result: 在多个任务上，Fast-LeWM相比LeWM提高了平均成功率，同时大幅减少了规划时间，并实现了更低的开环潜在损失，且该损失随预测时域增长的速率显著变慢。

Insight: 核心创新在于将动作前缀作为基本预测单元，通过前缀级监督使模型学习状态在不同动作前缀下的连续演化，而非仅拟合单步状态转移；规划时可直接利用编码动作序列的最后一个前缀令牌评估未来潜在状态，无需显式模拟中间状态。

Abstract: Joint-Embedding Predictive Architectures (JEPAs), including recent LeWorldModel (LeWM), have become a promising foundation for reconstruction-free visual world models. For visual planning, however, LeWM evaluates candidate action sequences by repeatedly applying a local one-step latent transition model. This autoregressive rollout makes planning computationally expensive and exposes the predicted trajectory to accumulated latent errors as the horizon grows. We propose Fast LeWorldModel (Fast-LeWM), a fast latent world model that replaces repeated local rollout with action-prefix prediction. Given the current latent and a candidate action sequence, Fast-LeWM encodes its prefixes and predicts the future latents reached after executing those prefixes in parallel. By making action prefixes the basic prediction unit, Fast-LeWM directly models action effects accumulated to different extents over multiple horizons. This prefix-level supervision forces the model to learn how states continuously evolve under different action prefixes, rather than only fitting one-step state transitions. During planning, the predictor can use the last prefix token from the encoded action sequence to evaluate the corresponding future latent without explicitly rolling through each intermediate imagined state. Across multiple tasks, Fast-LeWM improves average success over LeWM while substantially reducing planning time, achieving lower open-loop latent loss whose growth becomes significantly slower as the rollout horizon increases.

cs.HC [Back]

[114] DanceDuo: Bridging Human Movement and AI Choreography cs.HC | cs.CVPDF

Gia-Cat Bui-Le, Tuong-Vy Truong-Thuy, Hai-Dang Nguyen, Trung-Nghia Le

TL;DR: 本文提出了一种名为DanceDuo的新型平台，该平台利用扩散模型生成与多种音乐流派同步的AI编舞序列，旨在鼓励舞蹈练习。该系统允许用户通过选择音乐曲目、人形模型以及导入个人舞蹈视频进行比较来与AI互动，从而提供丰富且引人入胜的用户体验。DanceDuo不仅提供舞蹈生成，还集成了人体姿态估计模型，为用户提供其自身表演与AI生成序列的深入比较。

Details

Motivation: 解决音乐驱动舞蹈生成领域的问题，旨在通过AI编舞平台鼓励舞蹈练习，并整合用户交互与比较功能以提升体验。

Result: 通过全面的用户研究，表明用户认为界面直观，尤其赞赏舞蹈比较功能，但摘要未提及具体的定量基准测试结果或SOTA比较。

Insight: 创新点在于将扩散模型用于音乐驱动的AI编舞生成，并整合人体姿态估计以实现用户表演与AI序列的实时比较，为娱乐和专业应用提供了新途径。

Abstract: In recent years, advancements in deep learning and generative models have revolutionized music-driven dance generation. This paper introduces a novel platform, namely DanceDuo, leveraging diffusion models to generate AI-choreographed dance sequences synchronized with a variety of music genres, to encourage dancing practice. The system allows users to interact with AI by selecting music tracks, humanoid models, and importing personal dance videos for comparison, fostering a rich and engaging user experience. DanceDuo not only offers dance generation but also integrates human pose estimation models to provide users with insightful comparisons of their own performances with AI-generated sequences. We conducted a comprehensive user study, revealing that users found the interface intuitive, with particular praise for the dance comparison feature. Our DanceDuo contributes significantly to the integration of AI in dance choreography, offering novel avenues for both recreational and professional applications.

[115] Budget-Aware Keyboardless Interaction cs.HC | cs.CVPDF

Quang-Thang Nguyen, Gia-Phuc Song-Dong, Minh-Triet Tran, Trung-Nghia Le

TL;DR: 本文提出了一种仅使用标准摄像头和打印键盘布局纸张的新型虚拟键盘系统，旨在解决传统输入设备移动性差的问题。该方法结合现代分割检测模型与传统图像处理算法，在标准环境下高效识别键盘区域，并通过分析用户指甲颜色实现触摸检测。

Details

Motivation: 传统键盘、鼠标等输入设备限制了用户移动性，现有虚拟键盘方案通常需要复杂设置或昂贵设备，因此本文旨在开发一种低成本、易部署的键盘替代方案。

Result: 实验表明，所提出的键盘和击键检测方法在实际应用中取得了有希望的结果，用户研究参与者认为该系统具有吸引力。

Insight: 创新点在于仅利用普通摄像头和打印纸张实现虚拟键盘交互，无需复杂校准或特殊光照条件，通过结合深度学习分割检测与传统图像处理（如指甲颜色分析）降低了系统成本与部署难度。

Abstract: Interacting with computers typically relies on traditional input devices such as keyboards, mice, and monitors, which can be cumbersome for users seeking greater mobility. Virtual keyboards have been explored to address these limitations, but they often involve complex setups or expensive equipment. This paper proposes a novel virtual keyboard system that leverages only a standard camera and a paper with a printed keyboard layout. Unlike previous methods requiring complex calibration or special lighting conditions, our approach can work on standard environment using modern computer vision technologies. Combining modern segmentation and detection models with traditional image processing algorithms, we efficiently identify the keyboard region. Touch detection is performed using an algorithm analyzing the color of the user’s fingernail. Experiments demonstrated a promising results our proposed solution of keyboard and keystroke detection for practical applications. Participants attended our user study also found the proposed system interesting.

cs.RO [Back]

[116] TaskNPoint: How to Teach Your Humanoid to Hit a Backhand in Minutes cs.RO | cs.CVPDF

Blake Werner, Ilona Demler, Pietro Perona, Aaron D. Ames

TL;DR: 论文提出TaskNPoint训练协议，用于快速教会人形机器人动态技能（如网球反手击球）。该方法结合人类教练的少量输入（如技能演示和交互窗口识别）与物理模拟环境中的强化学习，能在单GPU上不到一小时内实现零样本泛化到新目标位置。

Details

Motivation: 解决如何高效教授人形机器人复杂动态技能的问题，其动机源于人类学习此类技能（如网球反手）的模式——依赖教练指导和针对性练习，而非海量无监督数据。

Result: 在Unitree G1人形机器人上测试，成功实现了对人工投掷网球的正反手击球、踢 incoming 足球以及从新位置抓放盒子等任务，仅需短视频演示和单GPU短时训练，无需针对每个任务进行奖励函数调优。

Insight: 核心创新在于将技能学习分解为人类教练提供高层语义（技能集、演示、交互窗口、目标）与模拟环境学习填充具体轨迹的分工范式，并通过训练中的随机目标采样实现零样本泛化，这为数据高效的机器人技能学习提供了新思路。

Abstract: How do we learn to hit a tennis backhand? Not from a thousand hours of tennis tournaments on TV - we work with a coach and practice. We argue this is also the right recipe for teaching dynamic skills to humanoid robots. This follows from a structural property of dynamic skills: the outcome is decided by a short, crucial portion of the trajectory - for a backhand, the ~20cm of racket travel around ball contact. Getting this interaction window right requires coordinating the whole motion, so that control, physics, and morphology act in concert. Learning thus reduces to mastering a handful of distinct actions and, for each, practicing until the window comes out right. To this end, we introduce TaskNPoint, a training protocol which makes the coach-learner division of labor explicit. The human coach contributes four inputs: a discrete set of skills (e.g. different shots), one demonstration per skill, identification of the interaction window, and the goal. Learning in a physically realistic simulation environment fills in each action trajectory and provides robustness to unmodeled events. Crucially, randomized target sampling during training lets a single demonstration generalize zero-shot to unseen goal locations. We test this approach on a Unitree G1 humanoid that hits forehands and backhands against balls thrown by a human, kicks incoming soccer balls, and picks and places boxes from novel locations. We find that learning is successful from short human video demonstrations and under an hour of training on a single GPU, with no per-task reward tuning.

[117] WatchAct: A Benchmark for Behavior-Grounded Robot Manipulation cs.RO | cs.AI | cs.CVPDF

Baiqi Li, Ce Zhang, Yu Fang, Yue Yang, Shangzhe Li

TL;DR: 论文提出了WatchAct基准测试，用于评估机器人基于观察人类行为视频进行操作的推理能力。该基准包含3000个长时程实例，涵盖四个认知能力领域，并设计了分离式评估协议来分别测试视频到计划的推理、策略执行和完整任务完成能力。实验表明当前系统在模拟和真实机器人上的表现都远未达到解决该基准的水平。

Details

Motivation: 解决现有机器人操作基准测试的局限性——它们通常只将指令与单张当前图像配对，无法评估机器人对观察到的连续人类行为的推理能力。视频包含了语言未明确表达的空间布局、物体历史状态和手势信息，这对机器人理解人类行为顺序和意图至关重要。

Result: 在模拟环境中，最佳系统（Gemini-3.1-Pro与π₀.₅）仅达到16.3%的成功率；在真实Franka Research 3机器人上为14.0%。视频到计划推理方面，Gemini-3.1-Pro的计划成功率仅为36.8%（人类为97.1%）；在完美计划下，π₀.₅的任务成功率为21.5%，在领域外场景中降至10.6%。

Insight: 创新点在于构建了首个基于真实人类行为视频的机器人操作基准，将视频、语言指令、模拟场景和可执行任务对齐。提出了分离式评估协议，能诊断系统在视频理解、计划生成和策略执行各环节的瓶颈。基准设计聚焦于事件解析、程序推理、隐含意图推断和情景推理四个认知维度，更贴近实际协作需求。

Abstract: A robot working alongside people must reason about what they have done, in what order, and with what intent. Video carries the spatial layouts, object histories, and gestures that language leaves underspecified, yet today’s manipulation benchmarks pair an instruction with a single current image, offering no way to evaluate reasoning over observed human behavior. We introduce WatchAct, a benchmark for robot manipulation grounded in observed human behavior. Each instance pairs a real-world human-action video and a language instruction with an aligned simulator scene and an executable LIBERO task, enabling scalable and reproducible evaluation. WatchAct comprises 3,000 long-horizon instances across 14 tasks in four capability domains drawn from the cognitive demands of watching another agent: parsing events (Event Grounding), recovering procedural structure (Procedural Reasoning), inferring unstated intent (Implicit Intent Inference), and tracking how the scene was changed (Episodic Reasoning). We further propose a disentangled evaluation protocol that separately measures (i)~~video-to-plan reasoning by vision-language models, (ii)~~policy execution under oracle plans, and (iii)~full task completion by integrated planner–policy pipelines. In both simulation and on a Franka Research 3 robot, current systems remain far from solving WatchAct. The best pipeline, Gemini-3.1-Pro with $π_{0.5}$, reaches only 16.3% Success Rate (SR) in simulation and 14.0% on the real robot. Gemini-3.1-Pro attains just 36.8% Plan SR (vs. 97.1% for humans), while $π_{0.5}$ reaches only 21.5% Task SR under oracle plans and drops to 10.6% on out-of-domain scenarios. Dataset and code are available at https://baiqi-li.github.io/watchact_page/.

[118] PressMimic: Pressure-Guided Motion Capture and Control for Humanoid Robot Imitation cs.RO | cs.CVPDF

Yi Lu, Shenghao Ren, Tianyu Xiong, Zhaoxiang Li, Jiaqi Li

TL;DR: 本文提出PressMimic框架，通过引入压力这一物理接地信号，从感知到控制全流程改进人形机器人运动模仿。在感知阶段，利用融合RGB与压力的多模态模型FRAPPE++联合估计3D姿态与全局运动；在控制阶段，提出压力监督策略（PSP）将压力信号融入强化学习，确保物理一致的接触模式。

Details

Motivation: 现有的人形运动模仿流程主要依赖基于视觉的运动捕捉和运动学模仿，忽略了接触动力学，导致脚部滑动、地面穿透和不稳定行为等问题。本文旨在从物理接地的角度重新审视该任务，利用压力作为统一模态来弥合感知与控制之间的鸿沟。

Result: 实验表明，压力信号的引入提高了运动估计精度、轨迹一致性和执行稳定性。论文构建了包含同步RGB、压力与动作捕捉数据的大规模数据集MotionPRO用于验证。

Insight: 核心创新在于将压力作为跨感知与控制阶段的统一物理接地信号，通过多模态融合（FRAPPE++）和压力监督强化学习（PSP）来共同解决运动估计的模糊性并强制执行物理一致的接触模式，从而提升模仿的物理真实性与鲁棒性。

Abstract: Humanoid motion imitation requires not only accurate perception of human kinematics but also faithful reproduction of physical interactions with the environment. However, existing pipelines rely primarily on vision-based motion capture and kinematic imitation, largely ignoring contact dynamics, leading to artifacts such as foot sliding, floor penetration, and unstable behaviors. In this work, we revisit humanoid motion imitation from the perspective of physical grounding and leverage pressure as a unified modality across perception and control. We present PressMimic, a framework that integrates pressure into the full pipeline from motion capture to humanoid control. In the perception stage, we introduce FRAPPE++, a multimodal model that fuses RGB and pressure to jointly estimate 3D pose and global motion, where pressure provides explicit contact and support constraints to resolve ambiguity in vision-based estimation. In the control stage, we propose a pressure-supervised policy (PSP) that incorporates pressure-derived signals into reinforcement learning, enabling physically consistent contact patterns during execution. We further construct MotionPRO, a large-scale dataset with synchronized RGB, pressure, and motion capture data. Experiments show that pressure improves motion estimation accuracy, trajectory consistency, and execution stability. These results demonstrate that pressure serves as an effective physical grounding signal, bridging perception and control for physically consistent humanoid motion imitation.

[119] Improving Vision-Language-Action Model Fine-Tuning with Structured Stage and Keyframe Supervision cs.RO | cs.CVPDF

Yuan Xu, Yixiang Chen, Kai Wang, Jiabing Yang, Peiyan Li

TL;DR: 本文提出了一种名为StaKe的辅助监督框架，用于改进视觉-语言-动作（VLA）模型在机器人操作任务中的微调。该框架通过从演示的夹爪状态自动推导出阶段分类器和关键帧预测器两种互补信号，为模型提供结构化监督，从而专注于具有挑战性的夹爪事件转换。实验表明，该方法在双臂仿真和单臂真实机器人任务中均能显著提升成功率。

Details

Motivation: 现有VLA模型在微调时，动作监督在所有时间步上均匀施加，缺乏对机器人当前操作阶段或下一个夹爪事件目标的结构化监督，导致失败集中在具有挑战性的夹爪事件转换上。

Result: 在双臂仿真和单臂Franka真实机器人任务上的实验表明，StaKe能持续提升成功率（相对增益分别为14%和56%），且在涉及更多夹爪事件转换的长时程任务上改进更大。消融研究验证了每个设计选择的有效性。

Insight: 创新点在于提出了一种无需人工标注、从夹爪状态自动推导结构化监督信号（阶段分类和关键帧预测）的轻量级辅助框架。客观来看，这是一种通过引入任务阶段和关键动作的显式监督来增强VLA模型表示学习、提升长时程操作鲁棒性的通用策略。

Abstract: Vision-Language-Action (VLA) models have shown strong potential for generalizable robotic manipulation. During fine-tuning, however, action supervision applies equally across all timesteps, without structured supervision on which manipulation stage the robot is in or what the next gripper-event target should be. This causes failures to concentrate around challenging gripper-event transitions. To address this, we propose StaKe, a plug-in auxiliary supervision framework that automatically derives two complementary signals from demonstration gripper states without manual annotation: a stage classifier that identifies the current manipulation stage, and a keyframe predictor that estimates the target joint action at the next gripper transition. Both are modeled as lightweight auxiliary heads that enrich the learned representations during training, while leaving the base VLA policy architecture and inference loop unchanged. Experiments on bimanual simulation and single-arm Franka real-robot tasks show that StaKe consistently improves success rates (relative gains of 14% and 56%, respectively), with larger improvements on longer-horizon tasks that involve more gripper-event transitions. Ablation studies validate each design choice, and qualitative analysis confirms that the learned representations faithfully track manipulation stages. These results indicate that structured supervision is an effective and general strategy for enhancing VLA fine-tuning in long-horizon manipulation. Project website: https://hi-yuanxu.github.io/StaKe-Web/

[120] Proposal-Conditioned Latent Diffusion for Closed-Loop Traffic Scenario Generation cs.RO | cs.CVPDF

Shubham Vaijanath Phoolari, Aleyna Kara, Christoph Lauer, Steven Peters

TL;DR: 本文提出了一种基于扩散模型的闭环交通场景生成框架，该框架以实例为中心的场景上下文和多模态提议先验为条件，并通过紧凑的动作潜在表示和基于提议的初始化来提高采样效率。实验表明，该方法在真实感、安全性和可控性之间取得了良好平衡，并允许通过测试时引导在竞争目标之间进行系统权衡。

Details

Motivation: 解决闭环交通仿真中生成场景一致、可控且交互式多智能体行为的挑战，同时克服现有基于扩散模型方法计算成本高、难以部署在时间受限的自动驾驶规划与仿真重规划循环中的问题。

Result: 在Waymo Open Motion Dataset上的实验表明，该方法在多样化的交互场景中，在真实感、安全性和可控性方面取得了有利的平衡，并且测试时引导能够实现竞争目标之间的系统权衡。

Insight: 创新点在于提出了一个以实例为中心场景上下文和多模态提议先验为条件的扩散框架，并引入了紧凑的动作潜在表示和基于提议的初始化以提高效率，无需重新训练即可减少每步运行时间；其测试时引导机制为实现目标权衡提供了灵活手段。

Abstract: Closed-loop traffic simulation remains challenging because it must generate interactive multi-agent behaviors that are scene-consistent and controllable throughout rollout. Prior diffusion-based approaches achieve strong realism, but their computational cost can hinder deployment in time-constrained replanning loops for autonomous vehicle planning and simulation. We present a diffusion-based scenario generation framework conditioned on instance-centric scene context and multimodal proposal priors, with optional test-time guidance for shaping safety-critical behaviors. A compact action-latent representation and proposal-based initialization improve sampling efficiency and reduce per-step runtime without retraining. Experiments on the Waymo Open Motion Dataset demonstrate a favorable balance among realism, safety, and controllability across diverse interactive scenarios, while showing that test-time guidance enables systematic trade-offs among competing objectives.

E-In Son, Jung-Taak Kim, Seung-Woo Seo

TL;DR: 本文提出ORION方法，通过引入序数神经坍缩作为视觉导航的表征先验，以解决端到端模仿学习中视觉编码器因间接监督而学习到模糊、与动作无关的表征问题。该方法根据导航动作的序数结构（如从极左到极右）显式组织表征空间，使类表示沿单一判别轴顺序排列，并抑制类内离轴方差。预训练编码器随后被集成到基于扩散的导航框架中，并进行端到端微调。

Details

Motivation: 动机在于解决视觉导航中，端到端模仿学习因仅使用单一动作损失进行间接监督，导致编码器学习到模糊、动作无关的表征，从而在复杂多变的环境和视觉干扰下，导航策略在模糊决策点产生不一致动作并导致失败的问题。

Result: 在仿真和真实世界环境中的大量实验表明，ORION在导航成功率和目标进度上持续优于端到端和神经坍缩基线方法，在视觉挑战性场景（如复杂多路交叉口）中取得了显著提升。

Insight: 创新点在于将导航动作的固有序数关系（如控制类别的空间顺序）作为先验，引导编码器学习结构化的表征空间，通过序数神经坍缩使类表示沿单一轴顺序排列并压缩类内方差，从而增强表征的判别性和鲁棒性，可借鉴于其他需要结构化表征的序数或多类分类任务。

Abstract: Learning robust navigation policies directly from visual observations remains a fundamental challenge in vision-based robotic navigation. In end-to-end imitation learning approaches, the visual encoder and action decoder are jointly optimized using a single action loss, which provides only an indirect supervisory signal to the encoder. This indirect supervision frequently results in the encoder learning ambiguous, action-agnostic representations. The problem is further complicated by substantial variations in scene structure and appearance across diverse environments, as well as the prevalence of visual distractors inherent to real-world navigation settings. Such action-agnostic features cause the navigation policy to produce inconsistent actions at ambiguous decision points, leading to navigation failure. To overcome these limitations, we propose ORION (Ordinal Neural Collapse for Visual Navigation), a method that explicitly organizes the encoder’s representation space according to the ordinal structure of navigation actions. In the context of goal-directed navigation, ego-centric control categories from Far Left to Far Right exhibit a natural ordinal relationship in which neighboring classes share similar visual contexts, while semantically opposing classes differ substantially in appearance. We encourage class representations to be arranged sequentially along a single discriminative axis, while suppressing off-axis variance within each class. The pretrained encoder is then integrated into a diffusion-based navigation framework, and the full pipeline is fine-tuned end-to-end. Extensive experiments in both simulation and real-world settings show that ORION consistently outperforms end-to-end and neural collapse baselines in navigation success rate and goal progress, with notable gains in visually challenging scenarios such as complex multi-way intersections.

cs.CR [Back]

[122] Adversarial Diffusion Across Modalities: A Fusion Survey of Attacks, Defenses, and Evaluation for Text, Vision, and Vision-Language Models cs.CR | cs.CLPDF

Abrar Alotaibi, Moataz Ahmed

TL;DR: 本文是一篇关于对抗性扩散攻击与防御的跨模态融合综述，系统整合了文本、视觉和视觉语言模型中的扩散攻击、防御与评估方法。作者提出了统一的分类框架和评估标准，并指出了当前LLM侧文献的五大弱点及未来研究方向。

Details

Motivation: 当前AI系统的对抗性评估在文本/LLM攻击、图像分类器攻击、视觉语言模型越狱以及扩散净化防御四个领域各自独立发展，缺乏统一的概念框架和评估标准。本文旨在通过信息融合，将这些分散的研究整合到一个统一的分类和评估体系中。

Result: 综述涵盖了四个领域共50篇已发表论文，提出了基于扩散角色的六类分类法，并应用了攻击成功率、可转移性、查询预算、困惑度和防御规避五个维度的统一评估框架。同时指出了当前LLM侧文献的五大常见弱点。

Insight: 创新点在于首次将跨模态的对抗性扩散研究整合到统一的概念框架中，提出了系统化的分类法和多维度评估标准。从客观角度看，该工作为未来攻击与防御研究提供了重要的基准比较基础和实验设计指导。

Abstract: Adversarial evaluation of AI systems has matured along four largely disconnected tracks: diffusion-based attacks on text and large language models (LLMs), diffusion-based attacks on image classifiers, jailbreak pipelines against vision-language models, and diffusion-based input purification defenses. Each has developed its own vocabulary, threat models, and benchmarks, with denoising diffusion models emerging as a shared generative mechanism whose recipes are now actively ported between communities. This survey performs an information-fusion exercise at the meta-research level: we integrate these four tracks into a single conceptual framework with a unified taxonomy, evaluation criteria, and research agenda, focusing on the LLM-side slice. We catalog fifty published papers across four scope areas (text/LLM, image classifier, vision-language model, defense), plus four diffusion-LLM-as-victim entries and ten non-diffusion baselines against which any new attack must be compared. We propose a six-class taxonomy of diffusion roles in adversarial pipelines, augmented by a threat-model axis recording attacker knowledge, query budget, and target accessibility, and apply a five-dimension framework (attack success rate, transferability, query budget, perplexity, defense-evasion) uniformly across modalities. The review adopts a dual attacker-defender perspective: alongside the attack catalog we cover four diffusion-based defenses that form the natural evaluation backdrop for new attacks. Our critical analysis identifies five recurring weaknesses of the current LLM-side literature, and we close with a research agenda of open questions and concrete experimental designs. The companion catalog and spreadsheet are released with the paper. We are explicit that this is a narrative review with quality assessment, not a PRISMA-compliant systematic review, and discuss the implications for replication.

cs.IR [Back]

[123] From Clicks to Intent: Cross-Platform Session Embeddings with LLM-Distilled Taxonomy for Financial Services Recommendations cs.IR | cs.AI | cs.CE | cs.CL | cs.LGPDF

Dianjing Fan, Yao Li, Kyaw Hpone Myint, Dwipam Katariya, Alexandre G. R. Day

TL;DR: 本文提出了一种用于金融服务推荐的可扩展双用途意图预测框架，旨在解决金融场景中登录前网页交互与登录后应用体验差异巨大、跨渠道实体匹配困难导致网页意图信号未被充分利用的问题。该框架将原始网页点击流转化为两个输出：一个自监督Transformer编码多模态点击流生成紧凑的会话嵌入，同时一个基于LLM的分类法生成与蒸馏管道产生可解释的意图标签。

Details

Motivation: 动机在于金融服务推荐系统中，登录前网页用户（探索新产品）与登录后应用用户（关注账户服务）的行为差异显著，且跨渠道实体解析（如匿名网页会话与认证移动账户的匹配）困难，导致基于网页的意图信号在认证后个性化推荐中未被充分利用。现有方法通常是临时且局限的，缺乏同时支持定量下游推荐和定性大规模理解的灵活性。

Result: 在移动主页磁贴排名任务中，会话嵌入相比生产基线将宏观Recall@1提升了1.88%，并将Log Loss降低了13.38%。在用户转化预测任务中，该嵌入在微观F1上优于LLM标签4.3%，而蒸馏层以超低延迟提供可解释标签，性能仅下降7%。

Insight: 创新点在于结合自监督点击流表示与LLM蒸馏的分类法，共同服务于生产中的定量任务（如推荐排名）和定性理解（如意图解释）。具体而言，利用Transformer编码多模态点击流生成通用会话嵌入，并设计LLM驱动的分类法生成与蒸馏流程，以可扩展方式产生可解释标签，实现了性能与可解释性的平衡。

Abstract: Sequential user behavior modeling is widely adopted in industrial recommender systems; however, significant gaps remain in financial services, where pre-login web interactions and authenticated in-app experiences differ drastically. Specifically, pre-login web users typically explore new products, whereas logged-in app users focus on account servicing. Due to the challenge of cross-channel entity resolution (e.g., matching anonymous web sessions to authenticated mobile accounts), web-based intent signals remain underutilized for post-authentication personalization. Existing methods for capturing web-based intent are often ad-hoc and narrow, lacking the flexibility to support both quantitative downstream recommendations and qualitative understanding at scale. In this work, we propose a scalable and dual-purpose intent prediction framework for web-based interactions and demonstrate its applicability for personalization. Our approach transforms raw web clickstreams into two outputs: a self-supervised Transformer encodes multi-modal clickstreams into a compact session embedding, while an LLM-based taxonomy generation and distillation pipeline produces interpretable intent labels. Our system demonstrates that self-supervised clickstream representations combined with LLM-distilled taxonomies can jointly serve quantitative tasks and qualitative understanding in production: on the mobile homepage tile ranking task, the session embedding improves macro Recall@1 by 1.88% and reduces Log Loss by 13.38% over production baselines. On the user conversion prediction task, the embedding outperforms the LLM labels by 4.3% on micro F1, while the distillation layer delivers interpretable labels at ultra-low latency with only a 7% performance drop.

Table of Contents

cs.CL [Back]

[1] Assert, don’t describe: Linguistic features that shift LLM reasoning about animal welfare cs.CL | cs.AIPDF

[2] Investigating LLM’s Problem Solving Capability – a Study on Statics Questions cs.CL | cs.AIPDF

[3] Helpfulness Hurts: Domain-Dependent Degradation of Mid-Trained Compassion Values Under Post-Training cs.CL | cs.AI | cs.CYPDF

[4] Low Resource Multimodal Translation of Nepali Spoken Words into Emotion-Conditioned Sign Language Avatars cs.CL | cs.AIPDF

[5] Context Recycling for Long-Horizon LLM Inference cs.CL | cs.AI | cs.LGPDF

[6] Where Larger Models Excel: The Primacy of Constraint-Guided Reasoning cs.CLPDF

[7] From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Models cs.CL | cs.AI | cs.CV | cs.LG | cs.MMPDF

[8] Soft Token Alignment for Cross-Lingual Reasoning cs.CLPDF

[9] Utilizing Cognitive Signals Generated during Human Reading to Enhance Keyphrase Extraction from Microblogs cs.CL | cs.DL | cs.HC | cs.IRPDF

[10] Speaking Numbers to LLMs: Multi-Wavelet Number Embeddings for Time Series Forecasting cs.CL | cs.AIPDF

[11] Comparing BERT Sentence-Pair Classification and Few-Shot LLM Prompting for Detecting Threat and Solution Framing in German Climate News cs.CLPDF

[12] The Inattentional Gap: Task-Conditioned Language and Vision Models Omit the Safety-Critical Signals They Can Otherwise Report cs.CL | cs.AI | cs.CVPDF

[13] \textsc{DiARC}: Distinguishing Positive and Negative Samples Helps Improving ARC-like Reasoning Ability of Large Language Models cs.CL | cs.AIPDF

[14] Zero-shot Tweet-Level Stance Detection Enhanced by External Knowledge and Reflective Chain-of-Thought Reasoning cs.CLPDF

[15] SocialPersona: Benchmarking Personalized Profiling and Response with Multimodal Social-Media Context cs.CL | cs.IRPDF

[16] Beyond Logical Forms: LLM-Extracted Patterns for Fallacy Classification cs.CL | cs.AIPDF

[17] OPID: On-Policy Skill Distillation for Agentic Reinforcement Learning cs.CLPDF

[18] Information-Aware KV Cache Compression for Long Reasoning cs.CL | cs.AIPDF

[19] GAVEL: Grounded Caption Error Verification and Localization cs.CLPDF

[20] ReaORE: Reasoning-Guided Progressive Open Relation Extraction Empowered by Large Reasoning Models cs.CL | cs.AIPDF

[21] Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization cs.CLPDF

[22] NuclearQAv2: A Structured Benchmark for Evaluating Domain-Science Competence in Large Language Models cs.CL | cs.AIPDF

[23] The Riddle Riddle: Testing Flexible Reasoning in Large Language Models and Humans cs.CLPDF

[24] Forecasting With LLMs: Improved Generalization Through Feature Steering cs.CL | cs.LGPDF

[25] Paved with True Intents: Intent-Aware Training Improves LLM Safety Classification Across Training Regimes cs.CLPDF

[26] Bridging Talk and Thought: Understanding Dialogue Dynamics Across Collaborative Problem-Solving Contexts cs.CL | cs.AIPDF

[27] CARVE: Content-Aware Recurrent with Value Efficiency for Chunk-Parallel Linear Attention cs.CL | cs.AI | cs.LG | cs.NEPDF

[28] LMs as Task-Specific Knowledge Bases: An Interpretability Analysis cs.CLPDF

[29] Multilingual Reasoning Cascades Need More Context cs.CLPDF

[30] Empowering GUI Agents via Autonomous Experience Exploration and Hindsight Experience Utilization for Task Planning cs.CL | cs.AI | cs.CV | cs.LGPDF

cs.CV [Back]

[31] DocArena: Turning Raw Documents into Controllable Training Environments for Document Search Agents cs.CVPDF

[32] Beyond Single-Source Cognitive Taskonomy:Multi-Source Task Relations through fMRI Transfer Learning cs.CV | q-bio.NCPDF

[33] Layer-Specific Prompt Fusion Discovery via Differentiable Search in Vision Foundation Models cs.CVPDF

[34] GeMoE: Gating Entropy is All You Need for Uncertainty-aware Adaptive Routing in MoE-based Large Vision-Language Models cs.CVPDF

[35] Beyond Aesthetics: Quantifying Information Loss in Turbid Scenes cs.CVPDF

[36] What Do Deepfake Benchmarks Measure? An Audit Using Frozen Self-Supervised Representations cs.CVPDF

[37] Neural Voxel Dynamics: Learning Implicit 3D Physics via Volumetric Feature Advection cs.CVPDF

[38] Staying VIGILant: Mitigating Visual Laziness via Counterfactual Visual Alignment in MLLMs cs.CV | cs.CL | cs.LGPDF

[39] Methane-Plume Segmentation From Hyperspectral Satellite Imagery Via Multimodal Deep Learning cs.CVPDF

[40] Forget, Anticipate and Adapt: Test Time Training for Long Videos cs.CVPDF

[41] From Hallucination to Grounding: Diagnosing Visual Spatial Intelligence via CRISP cs.CV | cs.AIPDF

[42] Active Adversarial Perturbation-driven Associative Memory Retrieval for RGB-Event Visual Object Tracking cs.CV | cs.AI | cs.LGPDF

[43] PhyEditBench: A Real-World Multi-Stage Benchmark for Physics-Aware Image Editing cs.CVPDF

[44] Perception, Verdict, and Evolution: Hindsight-Driven Self-Refining Forensics Agent for AI-Generated Image Detection cs.CV | cs.AIPDF

[45] Coarse-to-Fine: A Hybrid Self-Supervised Method for Non-rigid 3D Shape Matching cs.CVPDF

[46] DiCoBench: Benchmarking Multi-Image Fine-Grained Perception via Differential and Commonality Visual Cues cs.CVPDF

[47] TaskTok: Delving into Task Tokens for Task-driven Image Restoration cs.CV | eess.IVPDF

[48] Position Rebinding Cache Reuse: Replay-Free Visual Revisiting for Interleaved Multimodal Reasoning cs.CVPDF

[49] Temporally Consistent Label Interpolation for Robust Surgical Multi-Task Learning under Challenging Conditions cs.CVPDF

[50] FracEvent: Event-Camera Simulation via Fractional-Relaxation Pixel Dynamics cs.CV | cs.LG | eess.IVPDF

[51] LayersReg: A Layer-by-Layer Progressive Regressor for Reliable Intraoperative 3D/2D Registration cs.CVPDF

[52] Disco-LoRA: Disentangled Composition of Content, Style, and Motion for Multi-concept Video Customization cs.CV | cs.AIPDF

[53] PhysEditWorld: A Large-Scale Dataset Toward Physics-Editable World Models cs.CVPDF

[54] Robust Onion: Peeling Open Vocab Object Detectors Under Noise cs.CV | cs.AIPDF

[55] LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editing cs.CVPDF

[56] Do Image Editing Models Understand Lighting? cs.CVPDF

[57] Depth-Semantic Alignment and Affinity-Guided Fusion for Structured Radar Point Cloud Generation cs.CVPDF

[58] ProtoKV: Streaming Video Understanding under Delayed Query with Summary-State Memory cs.CV | cs.LGPDF

[59] Calibrated Harmonic Overlaid Implicit Neural Representations for Multi-Dimensional Data cs.CVPDF

[60] LearniBridge: Learnable Calibration of Feature Caching for Diffusion Models Acceleration cs.CV | cs.LGPDF

[61] Event-based Gaze Control System for Accurate Real-time Spin Estimation in Professional Ball Games cs.CV | eess.IVPDF

[62] ReasonCLIP-58M: Visually Grounded Commonsense Reasoning Supervision for CLIP cs.CV | cs.AIPDF

[63] Multi-modality Image Fusion under Adverse Weather: Mask-Guided Feature Restoration and Interaction cs.CVPDF

[64] NaviCache: Test-Time Self-Calibration Caching for Video Generation cs.CV | cs.AI | cs.MMPDF

[65] Liquid Fusion of Heterogeneous Representations Towards General Salient Object Detection cs.CVPDF

[66] SpatialFlow-GRPO: Where Spatial Credit Drives Image Editing cs.CVPDF

[67] Bridging Vision and Language Concepts through Optimal Transport Semantic Flow cs.CV | cs.AIPDF

[68] RIS-Assisted Proactive Handover for Reliable mmWave Wireless Networks cs.CVPDF

[69] Modeling Local, Global, and Cross-Modal Context in Multimodal 3D MRI cs.CVPDF

[70] Tractography-Driven Synthetic Data Generation for Fiber Bundle Segmentation in Tracer Histology cs.CV | cs.LGPDF

[71] PhysRAG: Enhancing Physics-Awareness in Video Generation via Retrieval-Augmented Generation cs.CVPDF

[72] Confidence-Aware Tool Orchestration for Robust Video Understanding cs.CV | cs.AIPDF

[73] PortraitGen: Exemplar-Driven GRPO with Dual-Reward Guidance for Photorealistic Portrait Generation cs.CVPDF

[74] TraMP-LLaMA: Generative Interpretability with Decoupled Instruction Tuning for Facial Expression Quality Assessment cs.CVPDF

[75] Computer Vision for MOBA Analytics: A Dataset and Baseline for Visibility Analysis in Dota 2 cs.CVPDF

[76] Event-Aware Instructed Assistant for Referring Video Segmentation cs.CV | cs.AIPDF

[77] Unison: Benchmarking Unified Multimodal Models via Synergistic Understanding and Generation cs.CVPDF