Table of Contents

cs.CL [Back]

[1] AccessEval: Benchmarking Disability Bias in Large Language Models

Srikant Panda,Amit Agarwal,Hitesh Laxmichand Patel

Main category: cs.CL

TL;DR: AccessEval是一个用于评估大型语言模型在残疾情境下偏见的基准测试，涵盖21个模型、6个领域和9种残疾类型。研究发现，模型对残疾相关查询的回答通常更负面、刻板且准确率更低。

Details Motivation: 大型语言模型在多样化的应用中可能表现出对残疾群体的偏见，但缺乏系统性评估工具。本文旨在填补这一空白，通过真实情境下的查询揭示模型的偏见。

Contribution: 提出了AccessEval基准测试，系统评估了21个模型在残疾相关的6个领域和9种残疾类型中的表现，揭示了模型偏见的具体形式和影响。

Method: 通过中性查询和残疾相关查询的配对比较，使用情感分析、社会感知和事实准确性指标评估模型输出。

Result: 模型对残疾相关查询的回答更负面、刻板且错误率高，尤其在听力、言语和行动不便领域表现明显。

Insight: AccessEval揭示了模型行为中隐含的偏见，强调了在技术评估中纳入用户实际影响的重要性，为未来的偏见缓解提供了方向。

Abstract: Large Language Models (LLMs) are increasingly deployed across diverse domains but often exhibit disparities in how they handle real-life queries. To systematically investigate these effects within various disability contexts, we introduce \textbf{AccessEval (Accessibility Evaluation)}, a benchmark evaluating 21 closed- and open-source LLMs across 6 real-world domains and 9 disability types using paired Neutral and Disability-Aware Queries. We evaluated model outputs with metrics for sentiment, social perception, and factual accuracy. Our analysis reveals that responses to disability-aware queries tend to have a more negative tone, increased stereotyping, and higher factual error compared to neutral queries. These effects show notable variation by domain and disability type, with disabilities affecting hearing, speech, and mobility disproportionately impacted. These disparities reflect persistent forms of ableism embedded in model behavior. By examining model performance in real-world decision-making contexts, we better illuminate how such biases can translate into tangible harms for disabled users. This framing helps bridges the gap between technical evaluation and user impact, reinforcing importance of bias mitigation in day-to-day applications. Our dataset is publicly available at: https://huggingface.co/datasets/Srikant86/AccessEval

[2] RAR$^2$: Retrieval-Augmented Medical Reasoning via Thought-Driven Retrieval

Kaishuai Xu,Wenjun Hou,Yi Cheng,Wenjie Li

Main category: cs.CL

TL;DR: RAR$^2$是一个联合学习框架，通过构建思维过程揭示隐式知识需求，指导检索和答案生成，提升了医学推理任务的性能。

Details Motivation: 大型语言模型（LLM）在医学任务中表现潜力，但传统检索增强生成（RAG）对需要复杂推理的问题效果有限，未能明确建模推理过程。

Contribution: 提出了RAR$^2$框架，结合Reasoning-Augmented Retrieval和Retrieval-Augmented Reasoning，显式建模推理过程以提升检索和生成质量。

Method: 利用混合偏好对训练数据，应用Direct Preference Optimization（DPO）训练模型，并设计两种测试时扩展策略。

Result: 在多个生物医学问答数据集上，RAR$^2$显著优于传统RAG方法（无论是否微调）。

Insight: 显式建模推理过程对提升医学任务的检索和生成效果至关重要，混合偏好对和DPO优化是关键创新点。

Abstract: Large Language Models (LLMs) have shown promising performance on diverse medical benchmarks, highlighting their potential in supporting real-world clinical tasks. Retrieval-Augmented Generation (RAG) has emerged as a key approach for mitigating knowledge gaps and hallucinations by incorporating external medical information. However, RAG still struggles with complex medical questions that require intensive reasoning, as surface-level input often fails to reflect the true knowledge needs of the task. Existing methods typically focus on refining queries without explicitly modeling the reasoning process, limiting their ability to retrieve and integrate clinically relevant knowledge. In this work, we propose RAR$^2$, a joint learning framework that improves both Reasoning-Augmented Retrieval and Retrieval-Augmented Reasoning. RAR$^2$ constructs a thought process to uncover implicit knowledge requirements and uses it to guide retrieval and answer generation. We build a training dataset of mixed preference pairs and apply Direct Preference Optimization (DPO) to train the model. Moreover, we design two test-time scaling strategies to explore the boundaries of our framework. Experiments demonstrate the effectiveness of RAR$^2$ across several biomedical question answering datasets, outperforming RAG baselines with or without fine-tuning.

[3] Multi-Modal Sentiment Analysis with Dynamic Attention Fusion

Sadia Abdulhalim,Muaz Albaghdadi,Moshiur Farazi

Main category: cs.CL

TL;DR: 该论文提出了一种动态注意力融合（DAF）框架，结合预训练语言模型的文本嵌入和语音编码器的声学特征，通过自适应注意力机制动态加权多模态信息，显著提升了情感分析的性能。

Details Motivation: 传统的情感分析仅依赖文本，忽略了声音等非语言线索的重要性，导致情感捕捉不全面。因此，作者提出了一种多模态动态融合方法，以更全面地建模情感。

Contribution: 1. 提出了动态注意力融合（DAF）框架；2. 在不微调底层编码器的情况下，实现了多模态信息的有效融合；3. 在大规模多模态基准测试中显著优于静态融合和单模态基线。

Method: 使用预训练语言模型的冻结文本嵌入和语音编码器的声学特征，通过自适应注意力机制动态分配权重，融合多模态信息。

Result: DAF模型在F1分数上显著提升，预测误差降低，并通过消融实验验证了动态加权策略的重要性。

Insight: 动态权重分配是多模态情感分析的关键，能够更好地建模情感复杂的输入，为情感计算应用提供了更鲁棒的基础。

Abstract: Traditional sentiment analysis has long been a unimodal task, relying solely on text. This approach overlooks non-verbal cues such as vocal tone and prosody that are essential for capturing true emotional intent. We introduce Dynamic Attention Fusion (DAF), a lightweight framework that combines frozen text embeddings from a pretrained language model with acoustic features from a speech encoder, using an adaptive attention mechanism to weight each modality per utterance. Without any finetuning of the underlying encoders, our proposed DAF model consistently outperforms both static fusion and unimodal baselines on a large multimodal benchmark. We report notable gains in F1-score and reductions in prediction error and perform a variety of ablation studies that support our hypothesis that the dynamic weighting strategy is crucial for modeling emotionally complex inputs. By effectively integrating verbal and non-verbal information, our approach offers a more robust foundation for sentiment prediction and carries broader impact for affective computing applications – from emotion recognition and mental health assessment to more natural human computer interaction.

[4] Painless Activation Steering: An Automated, Lightweight Approach for Post-Training Large Language Models

Sasha Cui,Zhongren Chen

Main category: cs.CL

TL;DR: 该论文提出了一种名为PAS（Painless Activation Steering）的新型后训练方法，用于轻量级、自动化地调整大语言模型的行为，无需手工提示或特征标注，显著提升了可控性和效率。

Details Motivation: 现有的权重调整和提示调整方法各有缺点：前者耗时耗力，后者不够精确且需要大量手工尝试。尽管激活导向（AS）是一种更廉价、快速且可控的替代方案，但其依赖手工提示对或特征标注，限制了实用性。

Contribution: 提出了PAS，一种完全自动化的激活导向方法，无需手工干预即可应用于任何标注数据集；还提出了其自省变体iPAS，在某些任务中表现出最强的因果导向效果。

Method: PAS通过自动化构造轻量级的激活向量，实现了对大语言模型的快速后训练调整。该方法可直接利用标注数据，无需额外手工步骤。

Result: 在18个任务上的实验表明，PAS能有效提升行为类任务的性能（如偏差、道德和对齐任务），但对智能导向任务效果有限。iPAS在部分任务中提升显著（如34.8%的对齐任务）。

Insight: AS技术在后训练中具有潜力，但其适用性因任务类型而异；PAS为AS的实用化提供了自动化解决方案，同时可以与ICL和SFT等方法结合使用。

Abstract: Language models (LMs) are typically post-trained for desired capabilities and behaviors via weight-based or prompt-based steering, but the former is time-consuming and expensive, and the latter is not precisely controllable and often requires manual trial-and-error. While activation steering (AS) promises a cheap, fast, and controllable alternative to the two existing post-training methods, current AS techniques require hand-crafted prompt pairs or labor-intensive feature annotation, making them more inconvenient than the plug-and-play methods such as Reinforcement Learning (RL) and Supervised Fine-Tuning (SFT). We introduce Painless Activation Steering (PAS), a family of fully automated methods that make AS readily usable with any given labeled dataset, with no need for prompt construction, feature labeling, or human intervention. We evaluate PAS on three open-weight models (Llama3.1-8B-Instruct, DeepSeek-R1-Distill-8B, and Nous-Hermes-2) and 18 tasks; we find that PAS reliably improves performance for behavior tasks, but not for intelligence-oriented tasks. The introspective variant (iPAS) delivers the strongest causal steering effects (10.1% on Bias, 5.2% on Morality, and 34.8% on Alignment). We also show PAS delivers additional gains on top of In-Context Learning (ICL) and SFT. PAS constructs a fast, lightweight activation vector that can be cheaply trained, easily stored, and activated at will. Our results provide a characterization of where AS helps, where it fails, and how to deploy it as a practical, automated LM post-training option.

[5] MIRAGE: Multi-hop Reasoning with Ambiguity Evaluation for Illusory Questions

Jeonghyun Park,Ingeol Baek,Seunghyun Yoon,Haeun Jang,Aparna Garimella,Akriti Jain,Nedim Lipka,Hwanhee Lee

Main category: cs.CL

TL;DR: 论文提出了一种名为MIRAGE的基准测试，用于评估多跳推理中的模糊性问题。当前的LLMs在该问题上表现不佳，作者提出了CLARION框架作为基线方法。

Details Motivation: 现实中的多跳问答（QA）常涉及模糊性，现有的大型语言模型（LLMs）在处理此类问题时表现不佳。作者希望通过MIRAGE基准推动对模糊性与多跳推理结合的研究。

Contribution: 1. 提出了MIRAGE基准，包含1,142个高质量模糊性多跳问题；2. 分析了LLMs在此类问题上的薄弱表现；3. 提出了CLARION框架作为解决方案。

Method: 作者设计了MIRAGE基准，并通过多LLM验证流程对其质量进行保障。此外，提出了CLARION，一个多智能体框架，用于解决模糊性和多步推理问题。

Result: 实验表明，现有LLMs在MIRAGE上表现不佳，CLARION框架显著优于现有方法。

Insight: 模糊性与多跳推理的结合是一个独特且重要的挑战，需要设计更自适应和鲁棒的推理系统。

Abstract: Real-world Multi-hop Question Answering (QA) often involves ambiguity that is inseparable from the reasoning process itself. This ambiguity creates a distinct challenge, where multiple reasoning paths emerge from a single question, each requiring independent resolution. Since each sub-question is ambiguous, the model must resolve ambiguity at every step. Thus, answering a single question requires handling multiple layers of ambiguity throughout the reasoning chain. We find that current Large Language Models (LLMs) struggle in this setting, typically exploring wrong reasoning paths and producing incomplete answers. To facilitate research on multi-hop ambiguity, we introduce MultI-hop Reasoning with AmbiGuity Evaluation for Illusory Questions (MIRAGE), a benchmark designed to analyze and evaluate this challenging intersection of ambiguity interpretation and multi-hop reasoning. MIRAGE contains 1,142 high-quality examples of ambiguous multi-hop questions, categorized under a taxonomy of syntactic, general, and semantic ambiguity, and curated through a rigorous multi-LLM verification pipeline. Our experiments reveal that even state-of-the-art models struggle on MIRAGE, confirming that resolving ambiguity combined with multi-step inference is a distinct and significant challenge. To establish a robust baseline, we propose CLarifying Ambiguity with a Reasoning and InstructiON (CLARION), a multi-agent framework that significantly outperforms existing approaches on MIRAGE, paving the way for more adaptive and robust reasoning systems.

[6] ML2B: Multi-Lingual ML Benchmark For AutoML

Ekaterina Trofimova,Zosia Shamina,Maria Selifanova,Artem Zaitsev,Remi Savchuk,Maxim Minets,Daria Ozerova,Emil Sataev,Denis Zuenko,Andrey E. Ustyuzhanin

Main category: cs.CL

TL;DR: ML2B是多语言ML代码生成的第一个基准测试，包含30个Kaggle竞赛的13种语言翻译，揭示了非英语任务性能下降15-45%的问题。

Details Motivation: 现有ML代码生成基准主要限于英语，忽略了ML研究和实践的全球性与多语言需求，需要填补这一空白。

Contribution: 提出了ML2B，首个多语言ML代码生成基准，附带自动化评估框架AIDE，揭示了跨语言表现的关键挑战。

Method: ML2B包含30个Kaggle竞赛的13种语言翻译，使用AIDE框架进行端到端评估。

Result: 结果显示非英语任务性能下降15-45%，突显多语言表示学习的挑战。

Insight: 多语言ML代码生成中，跨语言表现存在显著差异，需进一步研究改进表示学习。

Abstract: Large language models (LLMs) have recently demonstrated strong capabilities in generating machine learning (ML) code, enabling end-to-end pipeline construction from natural language instructions. However, existing benchmarks for ML code generation are mainly restricted to English, overlooking the global and multilingual nature of ML research and practice. To address this gap, we present ML2B, the first benchmark for evaluating multilingual ML code generation. ML2B consists of 30 Kaggle competitions translated into 13 natural languages, covering tabular, text, and image data types, with structured metadata and validated human-reviewed translations. For evaluation, we employ AIDE, an automated framework for end-to-end assessment of data science pipelines, and provide insights into cross-lingual model performance. Our results reveal substantial 15-45% performance degradation on non-English tasks, highlighting critical challenges in multilingual representation learning for code generation. The benchmark, evaluation framework, and comprehensive results are made available through our GitHub repository to facilitate future research in multilingual ML code generation: https://github.com/enaix/ml2b.

[7] EditGRPO: Reinforcement Learning with Post -Rollout Edits for Clinically Accurate Chest X-Ray Report Generation

Kai Zhang,Christopher Malon,Lichao Sun,Martin Renqiang Min

Main category: cs.CL

TL;DR: EditGRPO是一种混合策略强化学习算法，通过临床效用的奖励优化胸部X光报告生成，结合在线探索与离线修正，在多个数据集上优于传统方法。

Details Motivation: 现有MLLM在监督微调时未明确对齐临床效用，需要一种方法优化生成的临床准确性。

Contribution: 提出EditGRPO，结合在线策略探索和离线修正，显著提升报告生成的临床相关性和泛化能力。

Method: 混合策略RL算法，在训练过程中注入句子级修正，解决探索与采样效率问题。

Result: 在CheXbert、GREEN等指标上平均提升3.4%，未见过数据集上平均增益5.9%。

Insight: 结合在线探索与离线修正的混合策略RL能有效提升临床报告的生成质量与泛化性。

Abstract: Radiology report generation requires advanced medical image analysis, effective temporal reasoning, and accurate text generation. Although recent innovations, particularly multimodal large language models (MLLMs), have shown improved performance, their supervised fine-tuning (SFT) objective is not explicitly aligned with clinical efficacy. In this work, we introduce EditGRPO, a mixed-policy reinforcement learning (RL) algorithm designed specifically to optimize the generation through clinically motivated rewards. EditGRPO integrates on-policy exploration with off-policy guidance by injecting sentence-level detailed corrections during training rollouts. This mixed-policy approach addresses the exploration dilemma and sampling efficiency issues typically encountered in RL. Applied to a Qwen2.5-VL-3B MLLM initialized with supervised fine-tuning (SFT), EditGRPO outperforms both SFT and vanilla GRPO baselines, achieving an average improvement of 3.4% in CheXbert, GREEN, Radgraph, and RATEScore metrics across four major chest X-ray report generation datasets. Notably, EditGRPO also demonstrates superior out-of-domain generalization, with an average performance gain of 5.9% on unseen datasets.

[8] Critique-Coder: Enhancing Coder Models by Critique Reinforcement Learning

Chi Ruan,Dongfu Jiang,Yubo Wang,Wenhu Chen

Main category: cs.CL

TL;DR: 论文提出了一种名为Critique Reinforcement Learning (CRL)的新方法，通过结合强化学习和批判性学习，提升了编码模型的性能。

Details Motivation: 现有的强化学习方法主要关注生成响应，缺乏明确的批判或反思机制。研究希望通过引入CRL，增强模型的批判能力和推理能力。

Contribution: 提出了Critique Reinforcement Learning (CRL)框架，并将CRL与标准强化学习结合，训练出性能更优的Critique-Coder模型。

Method: CRL要求模型生成对（问题，解决方案）对的批判，并通过判断批判标签与真实标签是否一致来提供奖励。将20%的标准RL数据替换为CRL数据进行混合训练。

Result: Critique-Coder在多个基准测试中表现优于仅使用RL的模型，尤其在LiveCodeBench（v5）和BBEH逻辑推理任务中表现突出。

Insight: CRL不仅可以提升编码能力，还能增强模型的通用推理能力，表明其在广泛任务中具有迁移性。CRL是标准RL的有力补充。

Abstract: Reinforcement Learning (RL) has emerged as a popular training paradigm, particularly when paired with reasoning models. While effective, it primarily focuses on generating responses and lacks mechanisms to explicitly foster critique or reflection. Several recent studies, like Critique-Fine-Tuning (CFT) and Critique-Guided-Distillation (CGD) have shown the benefits of explicitly teaching LLMs how to critique. Motivated by them, we propose Critique Reinforcement Learning (CRL), where the model is tasked with generating a critique for a given (question, solution) pair. The reward is determined solely by whether the final judgment label $c \in {\texttt{True}, \texttt{False}}$ of the generated critique aligns with the ground-truth judgment $c^*$. Building on this point, we introduce \textsc{Critique-Coder}, which is trained on a hybrid of RL and CRL by substituting 20% of the standard RL data with CRL data. We fine-tune multiple models (\textsc{Critique-Coder}) and evaluate them on different benchmarks to show their advantages over RL-only models. We show that \textsc{Critique-Coder} consistently outperforms RL-only baselines on all the evaluated benchmarks. Notably, our \textsc{Critique-Coder-8B} can reach over 60% on LiveCodeBench (v5), outperforming other reasoning models like DeepCoder-14B and GPT-o1. Beyond code generation, \textsc{Critique-Coder} also demonstrates enhanced general reasoning abilities, as evidenced by its better performance on logic reasoning tasks from the BBEH dataset. This indicates that the application of CRL on coding datasets enhances general reasoning and critique abilities, which are transferable across a broad range of tasks. Hence, we believe that CRL works as a great complement to standard RL for LLM reasoning.

[9] ChatInject: Abusing Chat Templates for Prompt Injection in LLM Agents

Hwan Chang,Yonghyun Jun,Hwanhee Lee

Main category: cs.CL

TL;DR: ChatInject是一种通过模仿结构化聊天模板的恶意负载攻击LLM代理的方法，利用模型的指令跟随特性，展示了比传统提示注入更高的攻击成功率。

Details Motivation: 随着基于LLM的代理与外部环境交互的增多，间接提示注入成为一种新的攻击面。现有研究集中于纯文本攻击，但对结构化聊天模板的利用尚未充分探索。

Contribution: 提出了ChatInject攻击方法，利用聊天模板和说服性多轮对话操纵LLM代理，展示更高的攻击成功率和跨模型的可迁移性。

Method: 通过格式化恶意负载以模仿原生聊天模板，提出多轮对话变体（Multi-turn）逐步诱导代理执行可疑操作。

Result: 实验表明，ChatInject的平均攻击成功率显著提升（AgentDojo从5.18%到32.05%），多轮对话成功率更高（InjecAgent达到52.33%）。现有防御方法对其无效。

Insight: 揭示了当前LLM代理系统在结构化聊天模板和多轮对话场景下的脆弱性，强调了针对此类攻击的新防御需求。

Abstract: The growing deployment of large language model (LLM) based agents that interact with external environments has created new attack surfaces for adversarial manipulation. One major threat is indirect prompt injection, where attackers embed malicious instructions in external environment output, causing agents to interpret and execute them as if they were legitimate prompts. While previous research has focused primarily on plain-text injection attacks, we find a significant yet underexplored vulnerability: LLMs’ dependence on structured chat templates and their susceptibility to contextual manipulation through persuasive multi-turn dialogues. To this end, we introduce ChatInject, an attack that formats malicious payloads to mimic native chat templates, thereby exploiting the model’s inherent instruction-following tendencies. Building on this foundation, we develop a persuasion-driven Multi-turn variant that primes the agent across conversational turns to accept and execute otherwise suspicious actions. Through comprehensive experiments across frontier LLMs, we demonstrate three critical findings: (1) ChatInject achieves significantly higher average attack success rates than traditional prompt injection methods, improving from 5.18% to 32.05% on AgentDojo and from 15.13% to 45.90% on InjecAgent, with multi-turn dialogues showing particularly strong performance at average 52.33% success rate on InjecAgent, (2) chat-template-based payloads demonstrate strong transferability across models and remain effective even against closed-source LLMs, despite their unknown template structures, and (3) existing prompt-based defenses are largely ineffective against this attack approach, especially against Multi-turn variants. These findings highlight vulnerabilities in current agent systems.

[10] Learning to Detect Relevant Contexts and Knowledge for Response Selection in Retrieval-based Dialogue Systems

Kai Hua,Zhiyuan Feng,Chongyang Tao,Rui Yan,Lu Zhang

Main category: cs.CL

TL;DR: 该论文提出了一种多轮响应选择模型RSM-DCK，用于检测上下文和知识库中相关的部分，以提高检索式对话系统的性能。通过预选和后选机制，模型能够更有效地匹配响应候选。

Details Motivation: 现有的检索式对话系统在处理上下文和知识库时，往往将所有内容用于匹配响应候选，忽略了不同部分的差异性重要性，导致性能下降。

Contribution: 提出了一种能够动态检测上下文和知识库中相关部分的多轮响应选择模型RSM-DCK。

Method: 使用最近上下文作为查询，在单词和话语级别预选相关内容。响应候选与选定的上下文和知识库交互，并用融合表示后选更相关的知识部分。

Result: 在两个基准数据集上测试，模型性能优于现有方法，并能有效检测相关内容。

Insight: 动态选择和融合上下文与知识库的相关部分可以显著提升对话系统的响应选择能力。

Abstract: Recently, knowledge-grounded conversations in the open domain gain great attention from researchers. Existing works on retrieval-based dialogue systems have paid tremendous efforts to utilize neural networks to build a matching model, where all of the context and knowledge contents are used to match the response candidate with various representation methods. Actually, different parts of the context and knowledge are differentially important for recognizing the proper response candidate, as many utterances are useless due to the topic shift. Those excessive useless information in the context and knowledge can influence the matching process and leads to inferior performance. To address this problem, we propose a multi-turn \textbf{R}esponse \textbf{S}election \textbf{M}odel that can \textbf{D}etect the relevant parts of the \textbf{C}ontext and \textbf{K}nowledge collection (\textbf{RSM-DCK}). Our model first uses the recent context as a query to pre-select relevant parts of the context and knowledge collection at the word-level and utterance-level semantics. Further, the response candidate interacts with the selected context and knowledge collection respectively. In the end, The fused representation of the context and response candidate is utilized to post-select the relevant parts of the knowledge collection more confidently for matching. We test our proposed model on two benchmark datasets. Evaluation results indicate that our model achieves better performance than the existing methods, and can effectively detect the relevant context and knowledge for response selection.

[11] HEART: Emotionally-driven test-time scaling of Language Models

Gabriela Pinto,Palash Goyal,Yiwen Song,Souradip Chakraborty,Zifeng Wang,Tomas Pfister,Hamid Palangi

Main category: cs.CL

TL;DR: HEART是一种新颖的框架，通过情感驱动的提示改进语言模型在推理任务中的表现，利用六种普遍情感引导模型自我校正，显著提升准确性，但在无验证器场景下效果不稳定。

Details Motivation: 当前的语言模型测试时缩放策略主要依赖逻辑或结构优化，忽略了情感反馈的潜力。心理学研究表明情绪可以调节认知表现，因此研究者希望通过情感驱动的提示改进模型的推理能力。

Contribution: 提出了HEART框架，首次利用情感驱动的提示（基于Ekman的六种普遍情感）引导语言模型自我校正，显著提升了推理任务的准确性。

Method: HEART通过情感化的短语对模型的错误回答提供反馈，系统地调整情感语调，帮助模型跳出错误的推理路径并探索更好的解决方案。使用了OlympiadBench等基准进行评估。

Result: 在验证器辅助下，HEART显著提升了模型的推理深度和准确性。然而在无验证器场景下，效果不稳定，表明这是未来研究的重点挑战。

Insight: 机器推理的下一个前沿可能是不仅优化逻辑，还需理解和利用模型的‘情感’（HEART），情感驱动的提示为改进推理能力提供了新方向。

Abstract: Test-time scaling has shown considerable success in improving the performance of language models on complex reasoning tasks without requiring fine-tuning. However, current strategies such as self-reflection primarily focus on logical or structural refinement. They do not leverage the guiding potential of affective feedback. Inspired by psychological research showing that emotions can modulate cognitive performance, we introduce HEART–a novel framework that uses emotionally-driven prompts for iterative self-correction. HEART provides feedback on a model’s incorrect response using a curated set of concise, emotionally charged phrases based on the six universal emotions categorized by Dr. Paul Ekman. By systematically varying the emotional tone of the feedback across iterations, our method guides the model to escape flawed reasoning paths and explore more promising alternatives. We evaluate our framework on challenging reasoning benchmarks including OlympiadBench, Humanity’s Last Exam, and SimpleQA. Our results reveal a significant new phenomenon: when guided by an oracle verifier, this affective iteration protocol unlocks significantly deeper reasoning, leading to consistent and substantial increases in accuracy over state-of-the-art baselines with the same verifier. However, we also identify a critical bottleneck for practical deployment. In a verifier-free setting, it struggles to harness these gains consistently, highlighting as a key challenge for future work. Our findings suggest that the next frontier in machine reasoning may lie not just in refining logic, but also in understanding and leveraging the `HEART’ of the models.

[12] Infusing Theory of Mind into Socially Intelligent LLM Agents

EunJeong Hwang,Yuwei Yin,Giuseppe Carenini,Peter West,Vered Shwartz

Main category: cs.CL

TL;DR: 论文提出了一种将心智理论（ToM）注入大型语言模型（LLM）社会智能代理的方法，通过显式生成对话中的心理状态来提升模型的目标达成能力和对话效果。

Details Motivation: 心智理论（ToM）是人类社交智能的核心，但现有的聊天机器人和LLM代理通常缺乏这种能力。作者希望通过引入ToM，提升LLM代理在社交对话中的表现和目标达成能力。

Contribution: 1. 首次展示通过显式生成心理状态可以显著提升LLM的对话效果；2. 提出了ToMAgent（ToMA），一种结合心理状态生成和对话前瞻训练的社会智能代理；3. 在Sotopia社交评估基准上验证了方法的有效性。

Method: 1. 通过提示LLM生成对话中的心理状态；2. 提出ToMAgent，结合ToM和对话前瞻训练，最大化心理状态对目标达成的帮助；3. 使用Sotopia基准进行实验分析。

Result: ToMA在实验中表现出更强的策略性和目标导向推理能力，能够实现长期适应性并维持更好的伙伴关系，显著优于基线方法。

Insight: 显式建模心理状态有助于LLM代理更有效地达成社交目标，同时保持良好的社交关系，这一方法为构建更社会智能的LLM代理提供了新方向。

Abstract: Theory of Mind (ToM)-an understanding of the mental states of others-is a key aspect of human social intelligence, yet, chatbots and LLM-based social agents do not typically integrate it. In this work, we demonstrate that LLMs that explicitly use ToM get better at dialogue, achieving goals more effectively. After showing that simply prompting models to generate mental states between dialogue turns already provides significant benefit, we further introduce ToMAgent (ToMA), a ToM-focused dialogue agent. ToMA is trained by pairing ToM with dialogue lookahead to produce mental states that are maximally useful for achieving dialogue goals. Experiments on the Sotopia interactive social evaluation benchmark demonstrate the effectiveness of our method over a range of baselines. Comprehensive analysis shows that ToMA exhibits more strategic, goal-oriented reasoning behaviors, which enable long-horizon adaptation, while maintaining better relationships with their partners. Our results suggest a step forward in integrating ToM for building socially intelligent LLM agents.

[13] Extract-0: A Specialized Language Model for Document Information Extraction

Henrique Godoy

Main category: cs.CL

TL;DR: Extract-0是一个7B参数的专用语言模型，专注于文档信息提取任务，性能超过更大参数量的通用模型。其训练方法结合合成数据生成、LoRA微调和GRPO强化学习，在多样化的文档提取任务中表现优异。

Details Motivation: 文档信息提取任务需要高效且精确的模型，但通用的大语言模型往往计算资源消耗巨大且性能不一定最优。为此，研究者提出了一种专门优化的模型和方法来解决这一问题。

Contribution: 1. 提出Extract-0，一个7B参数的专用语言模型，性能超过更大参数量的通用模型（如GPT-4.1）。2. 创新性地结合合成数据生成、LoRA微调和GRPO强化学习方法来训练模型。

Method: 1. 使用合成数据生成管道生成28万训练样本。2. 采用LoRA进行高效微调（仅修改0.53%的权重）。3. 引入基于语义相似度的奖励函数和GRPO强化学习方法来优化模型。

Result: Extract-0在1000个多样化文档提取任务中的平均奖励为0.573，优于GPT-4.1（0.457）、o3（0.464）和GPT-4.1-2025（0.459）。

Insight: 任务专用的优化可以显著提升性能并减少计算资源需求，表明在特定领域开发专用模型的潜力。

Abstract: This paper presents Extract-0, a 7-billion parameter language model specifically optimized for document information extraction that achieves performance exceeding models with parameter counts several orders of magnitude larger. Through a novel combination of synthetic data generation, supervised fine-tuning with Low-Rank Adaptation (LoRA), and reinforcement learning via Group Relative Policy Optimization (GRPO), Extract-0 achieves a mean reward of 0.573 on a benchmark of 1,000 diverse document extraction tasks, outperforming GPT-4.1 (0.457), o3 (0.464), and GPT-4.1-2025 (0.459). The training methodology employs a memory-preserving synthetic data generation pipeline that produces 280,128 training examples from diverse document sources, followed by parameterefficient fine-tuning that modifies only 0.53% of model weights (40.4M out of 7.66B parameters). The reinforcement learning phase introduces a novel semantic similarity-based reward function that handles the inherent ambiguity in information extraction tasks. This research demonstrates that task-specific optimization can yield models that surpass general-purpose systems while requiring substantially fewer computational resource.

[14] Large language models management of medications: three performance analyses

Kelli Henry,Steven Xu,Kaitlin Blotske,Moriah Cargile,Erin F. Barreto,Brian Murray,Susan Smith,Seth R. Bauer,Yanjun Gao,Tianming Liu,Andrea Sikora

Main category: cs.CL

TL;DR: 本文评估了GPT-4o在药物管理三个任务中的表现：药物名称与配方的匹配、药物相互作用识别以及药物订单句子的生成。结果表明，GPT-4o在这些任务中表现不佳，强调了领域特定训练和评估框架的重要性。

Details Motivation: 尽管大语言模型（LLMs）在医疗诊断中有潜在用途，但其在药物管理任务中的一致性和准确性尚未得到充分评估。本文旨在填补这一空白。

Contribution: 本文首次对GPT-4o在药物配方匹配、药物相互作用识别和药物订单生成三个关键任务中的表现进行了系统性评估，揭示了模型的局限性。

Method: 研究通过三个实验评估GPT-4o的性能，使用TF-IDF向量的余弦相似度、标准化Levenshtein相似度和ROUGE-1/ROUGE-L F1等指标量化准确性，并结合临床医生的手动评估。

Result: GPT-4o在药物配方匹配中表现不佳（仅49%正确），药物相互作用识别的准确性在搜索增强评估中有所提升（54.7% vs. 69.2%），但在无相互作用时表现更差。药物订单生成任务中错误率为34.2%。

Insight: 研究表明，通用大语言模型在特定领域（如药物管理）中表现有限，需要通过领域专用数据和评估框架进行优化。

Abstract: Background: Large language models (LLMs) can be useful in diagnosing medical conditions, but few studies have evaluated their consistency in recommending appropriate medication regimens. The purpose of this evaluation was to test GPT-4o on three medication benchmarking tests including mapping a drug name to its correct formulation, identifying drug-drug interactions using both its internal knowledge and using a web search, and preparing a medication order sentence after being given the medication name. Methods: Using GTP-4o three experiments were completed. Accuracy was quantified by computing cosine similarity on TF-IDF vectors, normalized Levenshtein similarity, and ROUGE-1/ROUGE-L F1 between each response and its reference string or by manual evaluation by clinicians. Results: GPT-4o performed poorly on drug-formulation matching, with frequent omissions of available drug formulations (mean 1.23 per medication) and hallucinations of formulations that do not exist (mean 1.14 per medication). Only 49% of tested medications were correctly matched to all available formulations. Accuracy was decreased for medications with more formulations (p<0.0001). GPT-4o was also inconsistent at identifying drug-drug-interactions, although it had better performance with the search-augmented assessment compared to its internal knowledge (54.7% vs. 69.2%, p=0.013). However, allowing a web-search worsened performance when there was no drug-drug interaction (median % correct 100% vs. 40%, p<0.001). Finally, GPT-4o performed moderately with preparing a medication order sentence, with only 65.8% of medication order sentences containing no medication or abbreviation errors. Conclusions: Model performance was overall poor for all tests. This highlights the need for domain-specific training through clinician-annotated datasets and a comprehensive evaluation framework for benchmarking performance.

[15] ADAM: A Diverse Archive of Mankind for Evaluating and Enhancing LLMs in Biographical Reasoning

Jasin Cekinmez,Omid Ghahroodi,Saad Fowad Chandle,Dhiman Gupta,Ehsaneddin Asgari

Main category: cs.CL

TL;DR: ADAM框架首次系统性评估和提升大语言模型在传记推理中的能力，通过多语言、多模态数据集AdamDB和基于Bloom分类法的评估基准AdamBench，并提出AdamRAG解决幻觉问题。

Details Motivation: 传记推理是大语言模型的关键能力，但此前未被系统研究。ADAM旨在填补这一空白，提供多语言、多模态和文化多样性的评估框架。

Contribution: 1) 提出首个系统性评估传记推理能力的框架ADAM；2) 构建多语言、多模态数据集AdamDB；3) 设计基于Bloom分类法的评估基准AdamBench；4) 提出针对传记内容的检索增强生成系统AdamRAG。

Method: ADAM结合多模态数据集AdamDB和评估基准AdamBench，通过AdamRAG系统（检索增强生成）解决幻觉问题，并在跨语言和跨模态任务中验证效果。

Result: AdamRAG显著提升开源模型的性能，对闭源模型也有一定帮助；流行度强烈影响准确性，多模态输入的改进较小且不稳定。

Insight: 1) 检索增强生成对传记推理尤其重要；2) 流行度是模型准确性的关键因素；3) 多模态输入的效果有限。

Abstract: We introduce ADAM (A Diverse Archive of Mankind), a framework for evaluating and improving multimodal large language models (MLLMs) in biographical reasoning. To the best of our knowledge, this is the first work to systematically examine LLM capabilities in biography, a critical yet underexplored dimension of factual knowledge. At its core, AdamDB is a multilingual and multimodal dataset covering over 4 million individuals across geography, time, and profession, while AdamBench provides cognitively structured evaluations based on Bloom’s taxonomy, spanning six reasoning levels in both English and native languages. To address hallucinations, particularly for lesser-known individuals, we propose AdamRAG, a retrieval-augmented generation system tailored to biographical contexts. Experiments show that AdamRAG substantially improves open-source models and modestly benefits closed-source ones, with the largest gains on lower-order reasoning. Popularity strongly mediates accuracy, and multimodal input via face images offers smaller, less consistent improvements than retrieval. ADAM establishes the first benchmark and framework for cognitively, culturally, and multimodally grounded biographical evaluation, advancing the development of multilingual, accurate, and hallucination-resistant MLLMs.

[16] Look Back to Reason Forward: Revisitable Memory for Long-Context LLM Agents

Yaorui Shi,Yuxin Chen,Siyuan Wang,Sihang Li,Hengxing Cai,Qi Gu,Xiang Wang,An Zhang

Main category: cs.CL

TL;DR: ReMemR1是一种针对长上下文问题设计的记忆增强型LLM智能体，通过回调增强的记忆和强化学习多级奖励（RLMLR）方法，解决了现有‘边读边记’方法中存在的信息丢失和非线性推理问题。

Details Motivation: 解决大型语言模型在处理长上下文问题时因单向处理和记忆覆盖导致的信息丢失和非线性推理困难。

Contribution: 提出了ReMemR1框架，结合回调增强的记忆机制和RLMLR方法，提升了长上下文推理能力。

Method: 使用选择性检索和回放机制实现非线性推理，并通过多级奖励信号强化训练过程。

Result: 在长文档QA任务中表现显著优于现有记忆方法。

Insight: 非线性记忆访问和多级奖励信号是提升长上下文推理效果的关键。

Abstract: Large language models face challenges in long-context question answering, where key evidence of a query may be dispersed across millions of tokens. Existing works equip large language models with a memory corpus that is dynamically updated during a single-pass document scan, also known as the “memorize while reading” methods. While this approach scales efficiently, it suffers from irreversible forward-only processing, information loss through overwriting, and sparse reinforcement learning signals. To tackle these challenges, we present ReMemR1, a memory-augmented agent with callback-enhanced memory that allows selective retrieval from the entire memory history and allows non-linear reasoning and revisiting of early evidence. To further strengthen training, we propose Reinforcement Learning with Multi-Level Rewards (RLMLR), which combines final-answer rewards with dense, step-level signals that guide effective memory use. Together, these contributions mitigate information degradation, improve supervision, and support multi-hop memory utilizing. Experiments on long-document QA show significant gains over existing memory-based approaches, which validates ReMemR1 as an effective solution for long-context reasoning agents.

[17] From Evidence to Trajectory: Abductive Reasoning Path Synthesis for Training Retrieval-Augmented Generation Agents

Muzhi Li,Jinhu Qi,Yihong Wu,Minghao Zhao,Liheng Ma,Yifan Li,Xinyu Wang,Yingxue Zhang,Ho-fung Leung,Irwin King

Main category: cs.CL

TL;DR: 该论文提出了EviPath范式，通过合成证据锚定的推理路径，提升检索增强生成（RAG）代理的开发效果。

Details Motivation: 现有方法在训练RAG代理时缺乏过程级监督，难以指导任务分解、检索器调用和逐步决策。强化学习的稀疏奖励和LLM推理能力有限，现有数据合成方法也无法建模环境交互。

Contribution: 提出了EviPath范式，包含：（1）溯因子任务规划；（2）忠实子问题回答；（3）对话式微调，实现了从合成数据中学习复杂推理和工具使用能力。

Method: EviPath通过子任务规划、基于证据的子问题回答和对话式微调，合成完整的代理-环境交互轨迹。

Result: 实验表明，使用EviPath训练的8B参数模型在开放域问答任务中显著优于现有方法，EM得分绝对提升了14.7%。

Insight: 合成数据可以显式建模推理路径和环境交互，弥补LLM在复杂任务中的能力不足。

Abstract: Retrieval-augmented generation agents development is hindered by the lack of process-level supervision to effectively guide agentic capabilities like task decomposition, retriever invocation, and stepwise decision-making. While reinforcement learning offers a potential solution, it suffers from sparse rewards and the limited reasoning capabilities of large language models (LLMs). Meanwhile, existing data synthesis methods only produce chain-of-thought rationales and fail to model environmental interactions. In this paper, we propose EviPath, an evidence-anchored reasoning path synthesis paradigm for RAG agent development. EviPath comprises: (i) Abductive Subtask Planning, which decomposes the problem into sub-questions and iteratively plans an optimal solution path based on the dependencies between them; (ii) Faithful Sub-question Answering, which uses supporting evidence to construct a proxy environment to generate reasoning thoughts and answers for each sub-question; and (iii) Conversational Fine-Tuning, which formats the complete agent-environment interaction trajectory into a dialogue format suitable for Supervised Fine-Tuning. EviPath allows LLMs to learn complex reasoning and tool-use capabilities directly from synthesized data. Extensive experiments on widely-used question-answering benchmarks show that an 8B parameter model trained with EviPath-synthesized data significantly and consistently outperforms state-of-the-art baselines with a double-digit absolute EM gain of 14.7% in open-domain question answering.

[18] The Geometry of Creative Variability: How Credal Sets Expose Calibration Gaps in Language Models

Esteban Garces Arias,Julian Rodemann,Christian Heumann

Main category: cs.CL

TL;DR: 论文提出了一种基于credal sets的几何框架，用于量化语言模型在创造性任务中的不确定性，揭示其在捕捉人类创造性变化时的不足。

Details Motivation: 理解大型语言模型在创造性任务中的不确定性是一个重要挑战，尤其是当存在多个有效输出时。

Contribution: 1. 提出了基于credal sets的几何框架，用于分解语言模型的不确定性。2. 通过实验揭示了模型在捕捉人类创造性变化时的校准差距。3. 分析了解码策略和模型规模对不确定性的影响。

Method: 使用credal sets量化不确定性，并将其分解为认知和随机成分。实验基于WritingPrompts数据集，评估了4个语言模型和5种解码策略生成的10万篇故事。

Result: 最佳模型与人类校准的最高值为0.434（Gemma-2B，温度0.7）。解码策略对总认知不确定性的贡献为39.4%到72.0%。模型规模与校准质量相关性较弱。

Insight: 解码策略对不确定性影响显著，而模型规模和调优方式对校准质量影响有限。该框架为提升人类-AI创造性对齐提供了实用建议。

Abstract: Understanding uncertainty in large language models remains a fundamental challenge, particularly in creative tasks where multiple valid outputs exist. We present a geometric framework using credal sets - convex hulls of probability distributions - to quantify and decompose uncertainty in neural text generation, calibrated against human creative variation. Analyzing 500 creative writing prompts from the WritingPrompts dataset with 10 unique human continuations each, we evaluate four language models across five decoding strategies, generating 100,000 stories. Our credal set analysis reveals substantial gaps in capturing human creative variation, with the best model-human calibration reaching only 0.434 (Gemma-2B with temperature 0.7). We decompose total uncertainty into epistemic and aleatoric components, finding that the choice of decoding strategy contributes 39.4% to 72.0% of total epistemic uncertainty. Model scale shows weak correlation with calibration quality and no significant difference exists between base and instruction-tuned models in calibration quality. Our geometric framework provides actionable insights for improving generation systems for human-AI creative alignment. We release our complete experimental framework.

[19] d$^2$Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching

Yuchu Jiang,Yue Cai,Xiangzhong Luo,Jiale Fu,Jiarui Wang,Chonghan Liu,Xu Yang

Main category: cs.CL

TL;DR: d$^2$Cache是一种训练免费的近似KV缓存框架，旨在加速扩散式大语言模型（dLLMs）的推理效率。它通过双阶段细粒度选择策略和自适应KV状态更新，显著提升了推理速度和生成质量。

Details Motivation: 扩散式大语言模型（dLLMs）因其双向注意力机制，无法直接受益于标准的KV缓存技术，导致推理效率低下。d$^2$Cache通过创新的缓存机制解决了这一问题。

Contribution: 提出了d$^2$Cache框架，通过双阶段细粒度选择和自适应KV状态更新，显著提升了dLLMs的推理效率和生成质量。

Method: d$^2$Cache采用两阶段细粒度选择策略，动态更新关键token的KV状态，同时缓存其他token的KV状态以供重用。此外，它还支持准左到右生成，缓解序列尾部token的过度自信问题。

Result: 在LLaDA和Dream等代表性dLLMs上的实验表明，d$^2$Cache不仅实现了显著的推理加速，还一致提升了生成质量。

Insight: d$^2$Cache的创新在于将KV缓存技术扩展到双向注意力模型，解决了传统KV缓存不适用于dLLMs的问题，同时通过动态更新和缓存机制提升了模型的整体效率。

Abstract: Diffusion-based large language models (dLLMs), despite their promising performance, still suffer from inferior inference efficiency. This is because dLLMs rely on bidirectional attention and cannot directly benefit from the standard key-value (KV) cache as autoregressive models (ARMs) do. To tackle this issue, we introduce \textit{Dual aDaptive Cache} (d$^2$Cache), which is a training-free approximate KV cache framework for accelerating dLLM inference. d$^2$Cache features a two-stage fine-grained selection strategy to identify tokens and adaptively update their KV states at each decoding step, while caching the KV states of the remaining tokens for reuse. Furthermore, d$^2$Cache naturally offers a more reliable decoding alternative, which can enable quasi left-to-right generation and mitigate premature overconfidence in tokens at the end of the sequence. Extensive experimental results on two representative dLLMs (\ie, LLaDA and Dream) demonstrate that d$^2$Cache not only achieves substantial inference speedups, but also yields consistent improvements in generation quality. The code is available at https://github.com/Kamichanw/d2Cache.

[20] Non-Collaborative User Simulators for Tool Agents

Jeonghoon Shim,Woojung Song,Cheyon Jin,Seungwon KooK,Yohan Jo

Main category: cs.CL

TL;DR: 本文提出了一种非协作用户模拟器架构，用于模拟真实世界中非协作用户的四种行为类别，以帮助开发和测试工具代理在多轮对话中的性能。

Details Motivation: 现有用户模拟器通常只模拟协作行为，无法训练和测试工具代理应对真实世界中的非协作用户行为。

Contribution: 1. 提出了一种新型用户模拟器架构，模拟四种非协作行为；
2. 在MultiWOZ和$\tau$-bench数据集上展示了现有工具代理的性能下降；
3. 提供了一个可扩展的框架，帮助社区开发工具代理并预诊断其在非协作条件下的表现。

Method: 设计了一种用户模拟器，模拟四种非协作行为：请求不可用服务、偏离主题对话、表达不耐烦和不完整表达。

Result: 实验中，现有工具代理在非协作用户条件下性能显著下降，出现幻觉和对话题破裂等问题。

Insight: 非协作用户行为的模拟对工具代理的鲁棒性测试至关重要，可帮助改进其在真实世界中的表现。

Abstract: Non-Collaborative User Simulators for Tool Agents Download PDF Jeonghoon Shim, Woojung Song, Cheyon Jin, Seungwon KooK, Yohan Jo 19 Sept 2025 (modified: 25 Sept 2025)ICLR 2026 Conference SubmissionConference, AuthorsRevisionsCC BY 4.0 Keywords: Tool Agent, User Simulator, Non-collaborative User, Dialogue Simulation TL;DR: A non-collaborative user simulation method for tool agent. Abstract: Tool agents interact with users through multi-turn dialogues to accomplish various tasks. Recent studies have adopted user simulation methods to develop these agents in multi-turn settings. However, existing user simulators tend to be agent-friendly, exhibiting only cooperative behaviors, which fails to train and test agents against non-collaborative users in the real world. To address this, we propose a novel user simulator architecture that simulates four categories of non-collaborative behaviors: requesting unavailable services, digressing into tangential conversations, expressing impatience, and providing incomplete utterances. Our user simulator can simulate challenging and natural non-collaborative behaviors while reliably delivering all intents and information necessary to accomplish the task. Our experiments on MultiWOZ and $\tau$-bench reveal significant performance degradation in state-of-the-art tool agents when encountering non-collaborative users. We provide detailed analyses of agents’ weaknesses under each non-collaborative condition, such as escalated hallucinations and dialogue breakdowns. Ultimately, we contribute an easily extensible user simulation framework to help the research community develop tool agents and preemptively diagnose them under challenging real-world conditions within their own services.

[21] Tagging the Thought: Unlocking Personalization Reasoning via Reinforcement Learning

Song Jin,Juntian Zhang,Yong Liu,Xun Zhang,Yufei Zhang,Fei Jiang,Guojun Yin,Wei Lin,Rui Yan

Main category: cs.CL

TL;DR: 论文提出了TagPR框架，通过强化学习增强大语言模型的个性化推理能力，结合语义标签和复合奖励信号，在公开基准上取得了显著提升。

Details Motivation: 当前大语言模型在通用推理能力上表现优异，但在个性化推理（如分析用户历史、推断偏好）方面仍有不足，亟需改进。

Contribution: 1. 提出了TagPR框架，结合语义标签和复合奖励信号（PRMU）强化个性化推理能力；2. 设计了数据驱动的标注流程和多阶段强化学习策略。

Method: 1. 自动生成并标注推理链，构建结构化数据集；2. 采用监督微调（SFT）和多阶段强化学习，结合复合奖励信号（标签约束和PRMU）。

Result: 在LaMP基准和自建数据集上平均提升32.65%，验证了结构化推理对个性化能力的有效性。

Insight: 结构化且可解释的推理路径是解锁大语言模型真正个性化能力的关键路径。

Abstract: Recent advancements have endowed Large Language Models (LLMs) with impressive general reasoning capabilities, yet they often struggle with personalization reasoning - the crucial ability to analyze user history, infer unique preferences, and generate tailored responses. To address this limitation, we introduce TagPR, a novel training framework that significantly enhances an LLM’s intrinsic capacity for personalization reasoning through a tagging the thought approach. Our method first develops a data-driven pipeline to automatically generate and semantically label reasoning chains, creating a structured dataset that fosters interpretable reasoning. We then propose a synergistic training strategy that begins with Supervised Fine-Tuning (SFT) on this tagged data to establish foundational reasoning patterns, followed by a multi-stage reinforcement learning (RL) process. This RL phase is guided by a unique composite reward signal, which integrates tag-based constraints and a novel Personalization Reward Model with User Embeddings (PRMU) to achieve fine-grained alignment with user-specific logic. Extensive experiments on the public LaMP benchmark and a self-constructed dataset demonstrate that our approach achieves state-of-the-art results, delivering an average improvement of 32.65% over the base model across all tasks. Our work validates that structured, interpretable reasoning is a highly effective pathway to unlocking genuine personalization capabilities in LLMs.

[22] Diagnose, Localize, Align: A Full-Stack Framework for Reliable LLM Multi-Agent Systems under Instruction Conflicts

Guancheng Wan,Leixin Sun,Longxu Dou,Zitong Shi,Fang Wu,Eric Hanchen Jiang,Wenke Huang,Guibin Zhang,Hejia Geng,Xiangru Tang,Zhenfei Yin,Yizhou Sun,Wei Wang

Main category: cs.CL

TL;DR: 该论文提出了一个全栈框架，通过诊断、定位和对齐三个阶段，解决多智能体系统中指令冲突导致的可靠性问题。

Details Motivation: LLM驱动的多智能体系统在复杂任务中表现出色，但在指令冲突下（如系统-用户或智能体间冲突）容易失效，现有宏观指标难以发现问题。

Contribution: 提出CRAS评分指标用于诊断角色遵从性，发现注意力漂移现象，并设计SAIL方法在局部层进行对齐优化。

Method: 框架分为三阶段：(1)诊断：用CRAS分解角色遵从性；(2)定位：分析注意力漂移；(3)对齐：在关键层使用LoRA和DPO优化。

Result: 在标准基准测试中，SAIL显著提升指令遵从性（如AutoGen在MedQA上提升5.60%），无需全模型微调。

Insight: 注意力头在中间层集中解决冲突，表明局部优化是提升系统可靠性的有效途径。

Abstract: Large Language Model (LLM)-powered multi-agent systems (MAS) have rapidly advanced collaborative reasoning, tool use, and role-specialized coordination in complex tasks. However, reliability-critical deployment remains hindered by a systemic failure mode: hierarchical compliance under instruction conflicts (system-user, peer-peer), where agents misprioritize system-level rules in the presence of competing demands. Moreover, widely used macro-level metrics (e.g., pass@k) obscure these micro-level violations and offer little actionable guidance for remedy. In this work, we present a full-stack, three-stage framework: (1) Diagnose - Contextualized Role Adherence Score (CRAS), a query-wise, context-aware scoring metric that decomposes role adherence into four measurable dimensions; (2) Localize - attention drift analysis revealing that instruction conflicts are resolved by attention heads that are largely concentrated in middle layers; (3) Align - Surgical Alignment of Instruction Layers (SAIL), which installs LoRA only on the localized focal layers and optimizes a token-weighted DPO-style preference objective that credits tokens by their focal attentional contribution. Across standard benchmarks and MAS frameworks, our surgical approach improves instruction hierarchy compliance (e.g., +5.60% with AutoGen on MedQA) without full-model finetuning.

[23] From Harm to Help: Turning Reasoning In-Context Demos into Assets for Reasoning LMs

Haonan Wang,Weida Liang,Zihang Fu,Nie Zheng,Yifan Zhang,Yao Tong,Tongyao Zhu,Hao Jiang,Chuang Li,Jiaying Wu,Kenji Kawaguchi

Main category: cs.CL

TL;DR: 论文探讨了推理大语言模型（RLMs）在few-shot CoT（思维链）下性能下降的问题，并提出了一种新方法I2S，通过将演示转化为显式的、可重用的洞察力来提升推理性能。

Details Motivation: 研究发现RLMs在few-shot CoT下性能反而下降，尤其是通过验证器强化学习训练的模型。作者旨在揭示这一问题背后的机制并提出解决方案。

Contribution: 提出了Insight-to-Solve（I2S）方法，将演示转化为目标问题的推理轨迹，并通过自我精炼（I2S+）提升一致性和正确性。

Method: I2S将演示转化为显式洞察力并生成目标问题的推理轨迹；I2S+进一步通过自我精炼优化推理。

Result: 实验表明I2S和I2S+在多个基准测试中优于直接回答和测试扩展基线，甚至在GPT模型上也表现出显著提升（如GPT-4.1在AIME’25上提升14.0%）。

Insight: 发现few-shot CoT性能下降的两种机制：语义误导和策略迁移失败，并提出通过洞察力提取和精炼有效利用演示。

Abstract: Recent reasoning LLMs (RLMs), especially those trained with verifier-based reinforcement learning, often perform worse with few-shot CoT than with direct answering. We revisit this paradox using high-quality reasoning traces from DeepSeek-R1 as demonstrations and find that adding more exemplars consistently degrades accuracy, even when demonstrations are optimal. A detailed analysis reveals two mechanisms behind this decline: (i) semantic misguidance, where high textual similarity leads the model to treat the target as the same as the exemplar and to copy intermediate steps verbatim; and (ii) strategy transfer failure, where the model struggles to extract useful reasoning strategies and apply them to target questions. Guided by these, we introduce Insight-to-Solve (I2S), a sequential test-time procedure that turns demonstrations into explicit, reusable insights and derives a target-specific reasoning trace; optionally, the reasoning is self-refined for coherence and correctness (I2S+). Extensive experiments on diverse benchmarks show that I2S and I2S+ consistently outperform both direct answering and test-time scaling baselines across open- and closed-source models. Even for GPT models, our method helps: on AIME’25, GPT-4.1 rises by +14.0%, and o1-mini improves by +2.7% on AIME and +1.7% on GPQA, indicating that in-context demonstrations can be harnessed effectively via insight-refine-solve framework.

[24] Global Beats, Local Tongue: Studying Code Switching in K-pop Hits on Billboard Charts

Aditya Narayan Sankaran,Reza Farahbakhsh,Noel Crespi

Main category: cs.CL

TL;DR: 这篇论文研究了K-pop歌曲中的语码转换现象，特别分析了英语和韩语的使用情况，揭示了全球市场压力对歌词语言选择的影响，以及表演者身份和榜单背景对风格模式的作用。

Details Motivation: K-pop在全球市场的成功与其歌词中的语码转换现象密切相关。论文旨在探究这一现象，揭示语言策略如何在全球榜单成功的K-pop歌曲中发挥作用。

Contribution: 论文的主要贡献包括构建了一个包含榜单热门K-pop歌曲的数据集，分析了英语和韩语的占比及语码转换频率，并探讨了性别和榜单类型对语言使用的影响。

Method: 研究使用了数据集统计分析的方法，量化了英语和韩语的使用比例及语码转换频率。此外，通过分类任务预测表演者性别，并对比了不同榜单的语言使用差异。

Result: 研究发现英语在K-pop歌曲中占主导地位，且语码转换频繁。性别差异不明显，但女性独唱歌手更倾向于使用英语。美国榜单Hot 100对英语的依赖更强。

Insight: 语言选择不仅反映了K-pop的全球市场策略，也揭示了表演者身份和榜单背景对歌词风格的影响。这种策略有助于增强歌曲的国际吸引力。

Abstract: Code switching, particularly between Korean and English, has become a defining feature of modern K-pop, reflecting both aesthetic choices and global market strategies. This paper is a primary investigation into the linguistic strategies employed in K-pop songs that achieve global chart success, with a focus on the role of code-switching and English lyric usage. A dataset of K-pop songs that appeared on the Billboard Hot 100 and Global 200 charts from 2017 to 2025, spanning 14 groups and 8 solo artists, was compiled. Using this dataset, the proportion of English and Korean lyrics, the frequency of code-switching, and other stylistic features were analysed. It was found that English dominates the linguistic landscape of globally charting K-pop songs, with both male and female performers exhibiting high degrees of code-switching and English usage. Statistical tests indicated no significant gender-based differences, although female solo artists tend to favour English more consistently. A classification task was also performed to predict performer gender from lyrics, achieving macro F1 scores up to 0.76 using multilingual embeddings and handcrafted features. Finally, differences between songs charting on the Hot 100 versus the Global 200 were examined, suggesting that, while there is no significant gender difference in English, higher English usage may be more critical for success in the US-focused Hot 100. The findings highlight how linguistic choices in K-pop lyrics are shaped by global market pressures and reveal stylistic patterns that reflect performer identity and chart context.

[25] PARL-MT: Learning to Call Functions in Multi-Turn Conversation with Progress Awareness

Huacan Chai,Zijie Cao,Maolin Ran,Yingxuan Yang,Jianghao Lin,pengxin,Hairui Wang,Renjie Ding,Ziyu Wan,Muning Wen,Weiwen Liu,Weinan Zhang,Fei Huang,Ying Wen

Main category: cs.CL

TL;DR: PARL-MT是一个新框架，通过显式整合进度感知来改进大型语言模型在多轮对话中的函数调用能力，结合了进度感知生成和强化学习算法，显著优于现有方法。

Details Motivation: 现实应用（如旅行规划或多阶段数据分析）通常涉及多轮对话，现有方法要么忽略了任务级规划，要么缺乏进度感知的显式整合。PARL-MT旨在解决这些问题。

Contribution: 提出了PARL-MT框架，显式引入进度感知，包括进度感知生成（PAG）管道和进度感知引导的强化学习（PAG-RL）算法。

Method: 1. PAG管道自动构建数据集，结合对话摘要与未来任务规划；2. PAG-RL算法将进度感知融入RL训练，减少冗余并提升全局任务对齐。

Result: 在两个公开基准测试中，PARL-MT显著优于现有方法，证明了进度感知的有效性。

Insight: 进度感知是多轮函数调用的关键，显式建模任务规划和上下文摘要能显著提升模型的鲁棒性和效率。

Abstract: Large language models (LLMs) have achieved impressive success in single-turn function calling, yet real-world applications such as travel planning or multi-stage data analysis typically unfold across multi-turn conversations. In these settings, LLMs must not only issue accurate function calls at each step but also maintain progress awareness, the ability to summarize past interactions and plan future actions to ensure coherent, long-horizon task execution. Existing approaches, however, either reduce multi-turn training to isolated single-turn samples, which neglects task-level planning, or employ end-to-end reinforcement learning (RL) that struggles with redundancy and lacks explicit integration of progress awareness. To overcome these limitations, we introduce PARL-MT, a framework that explicitly incorporates progress awareness into LLM training for multi-turn function calling. PARL-MT combines (i) a Progress Awareness Generation (PAG) pipeline, which automatically constructs datasets coupling conversation summaries with future task planning, and (ii) a Progress Awareness-Guided Reinforcement Learning (PAG-RL) algorithm, which integrates progress awareness into RL training to reduce contextual redundancy and improve alignment between local actions and global task completion. Empirical results on two public benchmarks demonstrate that PARL-MT significantly outperforms existing methods, highlighting the effectiveness of progress awareness in enabling robust and efficient multi-turn function calling.

[26] A Structured Framework for Evaluating and Enhancing Interpretive Capabilities of Multimodal LLMs in Culturally Situated Tasks

Haorui Yu,Ramon Ruiz-Dolz,Qiufeng Yi

Main category: cs.CL

TL;DR: 论文提出了一个量化评估框架，测试主流视觉语言模型（VLMs）在传统中国画评论任务中的表现，并通过实验揭示了其优势和局限性。

Details Motivation: 研究目标是为了评估VLMs在复杂文化任务（如中国画评论）中的表现，探索其语义理解与内容生成能力。

Contribution: 1) 设计了一个量化框架，提取多维评估特征；2) 定义了多种评论角色；3) 通过实验评估了主流VLMs的表现。

Method: 1) 从专家评论中提取多维特征；2) 定义量化角色；3) 使用角色引导提示评估VLMs。

Result: 实验揭示了VLMs在艺术评论任务中的表现水平、优势和改进空间。

Insight: VLMs在复杂语义理解与文化任务中存在潜力，但需进一步提升多样性和精准性。

Abstract: This study aims to test and evaluate the capabilities and characteristics of current mainstream Visual Language Models (VLMs) in generating critiques for traditional Chinese painting. To achieve this, we first developed a quantitative framework for Chinese painting critique. This framework was constructed by extracting multi-dimensional evaluative features covering evaluative stance, feature focus, and commentary quality from human expert critiques using a zero-shot classification model. Based on these features, several representative critic personas were defined and quantified. This framework was then employed to evaluate selected VLMs such as Llama, Qwen, or Gemini. The experimental design involved persona-guided prompting to assess the VLM’s ability to generate critiques from diverse perspectives. Our findings reveal the current performance levels, strengths, and areas for improvement of VLMs in the domain of art critique, offering insights into their potential and limitations in complex semantic understanding and content generation tasks. The code used for our experiments can be publicly accessed at: https://github.com/yha9806/VULCA-EMNLP2025.

[27] Detecting Corpus-Level Knowledge Inconsistencies in Wikipedia with Large Language Models

Sina J. Semnani,Jirayu Burapacheep,Arpandeep Khatua,Thanawan Atchariyachanvanit,Zheng Wang,Monica S. Lam

Main category: cs.CL

TL;DR: 该论文提出了一个称为CLAIRE的系统，结合LLM推理和检索技术，用于检测Wikipedia中的知识不一致性，并通过用户研究和基准测试验证其有效性。

Details Motivation: Wikipedia是重要的知识库，但其准确性可能存在问题。研究团队关注知识不一致性这一特定类型的错误，以提升Wikipedia的质量。

Contribution: 1.引入了语料库级别的不一致性检测任务；2.提出了CLAIRE系统，结合LLM和检索技术；3.贡献了首个真实Wikipedia不一致性基准WIKICOLLIDE。

Method: CLAIRE系统利用LLM推理和检索技术，生成潜在不一致性声明及上下文证据，供人工审核。

Result: 用户研究中，87.5%的编辑信心提升，检测效率提高了64.7%。基准测试显示，3.3%的英文Wikipedia事实存在不一致性。

Insight: 知识不一致性是Wikipedia的显著问题，LLM辅助系统如CLAIRE能有效帮助人工编辑大规模提升知识一致性。

Abstract: Wikipedia is the largest open knowledge corpus, widely used worldwide and serving as a key resource for training large language models (LLMs) and retrieval-augmented generation (RAG) systems. Ensuring its accuracy is therefore critical. But how accurate is Wikipedia, and how can we improve it? We focus on inconsistencies, a specific type of factual inaccuracy, and introduce the task of corpus-level inconsistency detection. We present CLAIRE, an agentic system that combines LLM reasoning with retrieval to surface potentially inconsistent claims along with contextual evidence for human review. In a user study with experienced Wikipedia editors, 87.5% reported higher confidence when using CLAIRE, and participants identified 64.7% more inconsistencies in the same amount of time. Combining CLAIRE with human annotation, we contribute WIKICOLLIDE, the first benchmark of real Wikipedia inconsistencies. Using random sampling with CLAIRE-assisted analysis, we find that at least 3.3% of English Wikipedia facts contradict another fact, with inconsistencies propagating into 7.3% of FEVEROUS and 4.0% of AmbigQA examples. Benchmarking strong baselines on this dataset reveals substantial headroom: the best fully automated system achieves an AUROC of only 75.1%. Our results show that contradictions are a measurable component of Wikipedia and that LLM-based systems like CLAIRE can provide a practical tool to help editors improve knowledge consistency at scale.

[28] Scaling Policy Compliance Assessment in Language Models with Policy Reasoning Traces

Joseph Marvin Imperial,Harish Tayyar Madabushi

Main category: cs.CL

TL;DR: 论文提出了一种称为政策推理痕迹（PRT）的方法，通过生成专业化的推理链来提升语言模型在政策合规性评估中的表现，显著提高了HIPAA和GDPR政策的评估准确性。

Details Motivation: 政策合规性评估是一项重要任务，但专家级别的黄金标准推理过程获取成本高。为解决这一问题，论文提出了PRT方法。

Contribution: 提出了Policy Reasoning Traces（PRT），通过专业化推理链提升语言模型的政策合规性评估能力，并在HIPAA和GDPR政策中实现了新的SOTA。

Method: PRT作为推理桥梁，作用于推断时和训练时，通过生成推理链明确违规行为，从而提高模型的评估能力。

Result: 实验表明，PRT显著提升了开放权重和商业模型的表现，并在政策条款引用和合规决策中展现了高利用率。

Insight: PRT不仅提高了评估准确性，还增强了模型对政策条款的引证能力，展示了推理链在任务中的关键作用。

Abstract: Policy compliance assessment is a fundamental task of evaluating whether an input case strictly complies with a set of human-defined rules, more generally known as policies. In practice, human experts follow a systematic, step-by-step process to identify violations with respect to specific stipulations outlined in the policy. However, such documentation of gold-standard, expert-level reasoning processes is costly to acquire. In this paper, we introduce Policy Reasoning Traces (PRT), a form of specialized generated reasoning chains that serve as a reasoning bridge to improve an LLM’s policy compliance assessment capabilities. Our empirical evaluations demonstrate that the use of PRTs for both inference-time and training-time scenarios significantly enhances the performance of open-weight and commercial models, setting a new state-of-the-art for HIPAA and GDPR policies. Beyond accuracy gains, we also highlight how PRTs can improve an LLM’s ability to accurately cite policy clauses, as well as influence compliance decisions through their high utilization from the raw chains of thought.

[29] Learning to Reason in Structured In-context Environments with Reinforcement Learning

Peng Yu,Zeyuan Zhao,Shao Zhang,Luoyi Fu,Xinbing Wang,Ying Wen

Main category: cs.CL

TL;DR: 论文提出了SIE框架，通过结构化数据自动构建推理环境，提升语言模型的泛化推理能力和可验证性。

Details Motivation: 现有推理环境难以扩展且泛化能力有限，需要一种能支持规模化、泛化性强且可验证的环境。

Contribution: 引入了SIE框架，利用结构化数据构建推理环境，支持规模化、泛化推理和规则可验证性。

Method: SIE框架通过结构化数据自动生成推理环境，利用其组合模式和显式模式支持推理技能的泛化和验证。

Result: 实验表明SIE显著提升结构化推理能力，并在跨领域数学和逻辑任务中有效泛化。

Insight: 在信息有限的SIE环境中，语言模型可通过探索推断缺失信息，进一步提升推理鲁棒性和泛化能力。

Abstract: Large language models (LLMs) have achieved significant advancements in reasoning capabilities through reinforcement learning (RL) via environmental exploration. As the intrinsic properties of the environment determine the abilities that LLMs can learn, the environment plays a important role in the RL finetuning process. An ideal LLM reasoning environment should possess three core characteristics: scalability, generalizable reasoning, and verifiability. However, existing mathematical and coding environments are difficult to scale due to heavy reliance on expert annotation, while the skills learned in game-based environments are too specialized to generalize. To bridge this gap, we introduce the \textbf{S}tructured \textbf{I}n-context \textbf{E}nvironment (SIE) framework. SIE achieves scalability by automatically constructing reasoning environments from large-scale structured data, where the rich compositional patterns naturally support generalizable reasoning. Moreover, the explicit schemas and reasoning chains in structured data provide a foundation for rule-based verifiability. Experimental results show that SIE framework not only achieves substantial improvements in in-domain structured reasoning, but also enables the learned compositional reasoning skills to generalize effectively to out-of-domain mathematical and logical reasoning tasks. We further explored learning in information-limited partial SIEs and found that LLMs can infer the missing information through exploring the environment, leading to robust reasoning improvements and generalization performance.

[30] MedCritical: Enhancing Medical Reasoning in Small Language Models via Self-Collaborative Correction

Xinchun Su,Chunxu Luo,Yixuan Li,Weidong Yang,Lipeng Ma

Main category: cs.CL

TL;DR: MedCritical提出了一种新颖的自协作校正框架，通过两阶段方法增强小型语言模型在医学推理任务中的表现，避免了传统知识蒸馏的高成本和低效率问题。

Details Motivation: 小型语言模型在复杂医学推理任务中表现不佳，而现有基于知识蒸馏的方法依赖大型教师模型，导致成本高、效率低。

Contribution: 提出MedCritical框架，通过自协作校正（两阶段：长链思维模板提取和模型自迭代协作优化）提升小型模型的推理能力，无需依赖大型教师模型的高成本指导。

Method: 1. 提取教师模型的长链思维模板指导学生模型生成复杂推理；2. 引入直接偏好优化（DPO），通过模型自迭代协作和校正轨迹优化学生模型的推理能力。

Result: MedCritical 7B模型在CMExam基准测试中分别超越Taiyi和Huatuo-o1-7B模型3.04%和10.12%，取得了7B级小模型的SOTA性能。

Insight: 模型可以通过自协作和自我迭代优化提升能力，减少对大型教师模型的依赖，同时保持高性能。

Abstract: In the field of medicine, complex reasoning tasks such as clinical diagnosis, treatment planning, and medical knowledge integration pose significant challenges, where small language models often underperform compared to large language models like GPT-4 and Deepseek. Recent knowledge distillation-based methods aim to address these issues through teacher-guided error correction, but this LLM as judge approach remains challenging in terms of cost, time, and efficiency. To circumvent this issue, we propose a novel two-stage framework, MedCritical, which uses a small language model fine-tuned by a large teacher model to play against itself. In the first stage, we extract high-level and detailed long-chain thought templates from the teacher model to guide the student model to generate more complex reasoning thoughts. In the second stage, we introduce direct preference optimization (DPO) through model self-iteration collaboration to enhance the reasoning ability of the student model by playing against the correction trajectory of the fine-tuned model during training. This model self-learning DPO approach teaches the student model to use its own error-driven insights to consolidate its skills and knowledge to solve complex problems, and achieves comparable results to traditional knowledge distillation methods using teacher models at a lower cost. Notably, our MedCritical 7B model outperforms the Taiyi and Huatuo-o1-7B models by 3.04% and 10.12% respectively on the CMExam benchmark, achieving new SOTA performance among 7B-class small models.

[31] CCD: Mitigating Hallucinations in Radiology MLLMs via Clinical Contrastive Decoding

Xi Zhang,Zaiqiao Meng,Jake Lever,Edmond S. L. Ho

Main category: cs.CL

TL;DR: 该论文提出了CCD（Clinical Contrastive Decoding）方法，通过双阶段对比机制减少放射学多模态大语言模型（MLLMs）中的医疗幻觉问题，无需额外训练或检索即可提升临床描述的准确性。

Details Motivation: 放射学MLLMs在生成临床描述时易产生不支持的幻觉内容，这对医疗应用的准确性构成严重风险，亟需解决方法。

Contribution: 提出CCD框架，利用放射学专家模型的结构化临床信号，通过双阶段对比机制优化生成过程中的令牌级逻辑，从而减少幻觉。

Method: CCD采用训练和检索无关的推断框架，整合任务特异性放射学专家模型的结构化信号，通过对比机制调整生成时的令牌级输出概率。

Result: 在多个数据集和模型上的实验表明，CCD能显著提升放射学报告生成的性能，如在MIMIC-CXR数据集上，RadGraph-F1指标提升达17%。

Insight: CCD提供了一种轻量化的通用方法，通过结合专家模型和MLLMs的力量，有效减少医疗幻觉，为实际应用提供了可行的技术路径。

Abstract: Multimodal large language models (MLLMs) have recently achieved remarkable progress in radiology by integrating visual perception with natural language understanding. However, they often generate clinically unsupported descriptions, known as medical hallucinations, which pose serious risks in medical applications that demand accuracy and image-grounded outputs. Through empirical analysis, we find that prompt-induced hallucinations remain prevalent in radiology MLLMs, largely due to over-sensitivity to clinical sections. To address this, we introduce Clinical Contrastive Cecoding (CCD), a training-free and retrieval-free inference framework that integrates structured clinical signals from task-specific radiology expert models. CCD introduces a dual-stage contrastive mechanism to refine token-level logits during generation, thereby enhancing clinical fidelity without modifying the base MLLM. Experiments on three datasets and multiple models demonstrate that CCD consistently improves overall performance on radiology report generation (RRG). On the MIMIC-CXR dataset, it yields up to a 17% improvement in RadGraph-F1 when applied to state-of-the-art RRG models. Our approach provides a lightweight and generalisable solution for mitigating medical hallucinations, effectively bridging expert models and MLLMs in radiology.

[32] Train Once, Answer All: Many Pretraining Experiments for the Cost of One

Sebastian Bordt,Martin Pawelczyk

Main category: cs.CL

TL;DR: 该论文提出一种高效的方法，通过单次训练同时进行多个预训练实验，显著降低大型语言模型（LLM）的实验成本。

Details Motivation: 当前预训练模型实验的计算成本高昂，限制了科学研究的广度和深度，作者希望通过单次训练运行实现多实验并行。

Contribution: 提出一种单次训练运行中同时进行多个预训练实验的方法，并通过实验验证其可行性和有效性。

Method: 在1.5B参数模型的训练过程中，同时运行十个实验，动态调整训练数据以研究不同问题（如知识获取、数学推理等），并通过持续预训练实验测试交互影响。

Result: 实验表明，多个实验对模型训练动态和性能影响极小，交互效应可以忽略，验证了方法的有效性。

Insight: 该方法为在有限计算资源下进行大规模的模型研究提供了新的可能性，但仍需注意实验中潜在的交互影响。

Abstract: Recent work has demonstrated that controlled pretraining experiments are a powerful tool for understanding learning, reasoning, and memorization in large language models (LLMs). However, the computational cost of pretraining presents a significant constraint. To overcome this constraint, we propose to conduct multiple pretraining experiments simultaneously during a single training run. We demonstrate the feasibility of this approach by conducting ten experiments during the training of a 1.5B parameter model on 210B tokens. Although we only train a single model, we can replicate the results from multiple previous works on data contamination, poisoning, and memorization. We also conduct novel investigations into knowledge acquisition, mathematical reasoning, and watermarking. For example, we dynamically update the training data until the model acquires a particular piece of knowledge. Remarkably, the influence of the ten experiments on the model’s training dynamics and overall performance is minimal. However, interactions between different experiments may act as a potential confounder in our approach. We propose to test for interactions with continual pretraining experiments, finding them to be negligible in our setup. Overall, our findings suggest that performing multiple pretraining experiments in a single training run can enable rigorous scientific experimentation with large models on a compute budget.

[33] No Loss, No Gain: Gated Refinement and Adaptive Compression for Prompt Optimization

Wenhang Shi,Yiren Chen,Shuqing Bian,Xinyi Zhang,Kai Tang,Pengfei Hu,Zhe Zhao,Wei Lu,Xiaoyong Du

Main category: cs.CL

TL;DR: GRACE通过门控细化和自适应压缩策略，实现了高效且稳定的提示优化，显著提升了LLM的性能和效率。

Details Motivation: 当前的自动提示优化方法存在不稳定生成和易陷入局部最优的问题，提出GRACE框架以解决这些挑战。

Contribution: 提出GRACE框架，结合门控细化和自适应压缩策略，显著优化提示生成的稳定性和效率。

Method: 1. 门控细化：通过反馈调节门和更新拒绝门稳定提示改进；2. 自适应压缩：在优化停滞时提炼核心概念以重启优化路径。

Result: 在11个任务上，GRACE平均性能提升4.7%、4.4%和2.7%，且仅需25%的生成预算。

Insight: 通过可控信息损失（细化与压缩）可以高效解决提示优化的局部最优问题。

Abstract: Prompt engineering is crucial for leveraging the full potential of large language models (LLMs). While automatic prompt optimization offers a scalable alternative to costly manual design, generating effective prompts remains challenging. Existing methods often struggle to stably generate improved prompts, leading to low efficiency, and overlook that prompt optimization easily gets trapped in local optima. Addressing this, we propose GRACE, a framework that integrates two synergistic strategies: Gated Refinement and Adaptive Compression, achieving Efficient prompt optimization. The gated refinement strategy introduces a feedback regulation gate and an update rejection gate, which refine update signals to produce stable and effective prompt improvements. When optimization stagnates, the adaptive compression strategy distills the prompt’s core concepts, restructuring the optimization trace and opening new paths. By strategically introducing information loss through refinement and compression, GRACE delivers substantial gains in performance and efficiency. In extensive experiments on 11 tasks across three practical domains, including BIG-Bench Hard (BBH), domain-specific, and general NLP tasks, GRACE achieves significant average relative performance improvements of 4.7%, 4.4% and 2.7% over state-of-the-art methods, respectively. Further analysis shows that GRACE achieves these gains using only 25% of the prompt generation budget required by prior methods, highlighting its high optimization efficiency and low computational overhead. Our code is available at https://github.com/Eric8932/GRACE.

[34] Liaozhai through the Looking-Glass: On Paratextual Explicitation of Culture-Bound Terms in Machine Translation

Sherrie Shen,Weixuan Wang,Alexandra Birch

Main category: cs.CL

TL;DR: 该论文研究了机器翻译中文化绑定词的副文本显化任务，通过构建数据集并评估大语言模型的性能，发现尽管LLM生成的副文本能提升理解，但仍不及专业翻译的效果。研究还揭示了文化中介的开放性和非规范性。

Details Motivation: 机器翻译在处理文化绑定词时存在挑战，现有方法忽略了副文本（如脚注）的作用。本文旨在填补这一空白，探索副文本显化的可能性。

Contribution: 1. 形式化了副文本显化任务；2. 构建了专家对齐的副文本数据集；3. 评估了LLM在副文本生成中的表现；4. 揭示了文化中介的开放性。

Method: 1. 基于Genette的副文本理论；2. 构建来自《聊斋志异》英译本的560条副文本数据集；3. 评估LLM在显化选择和内容上的表现，包括提示和检索方法。

Result: LLM生成的副文本提升了受众理解，但仍显著弱于专业翻译。统计分析显示专业翻译在副文本使用上也存在较大差异。

Insight: 文化中介具有开放性，副文本显化在MT中有潜力超越语言对等，扩展到单语解释和个性化适应。

Abstract: The faithful transfer of contextually-embedded meaning continues to challenge contemporary machine translation (MT), particularly in the rendering of culture-bound terms–expressions or concepts rooted in specific languages or cultures, resisting direct linguistic transfer. Existing computational approaches to explicitating these terms have focused exclusively on in-text solutions, overlooking paratextual apparatus in the footnotes and endnotes employed by professional translators. In this paper, we formalize Genette’s (1987) theory of paratexts from literary and translation studies to introduce the task of paratextual explicitation for MT. We construct a dataset of 560 expert-aligned paratexts from four English translations of the classical Chinese short story collection Liaozhai and evaluate LLMs with and without reasoning traces on choice and content of explicitation. Experiments across intrinsic prompting and agentic retrieval methods establish the difficulty of this task, with human evaluation showing that LLM-generated paratexts improve audience comprehension, though remain considerably less effective than translator-authored ones. Beyond model performance, statistical analysis reveals that even professional translators vary widely in their use of paratexts, suggesting that cultural mediation is inherently open-ended rather than prescriptive. Our findings demonstrate the potential of paratextual explicitation in advancing MT beyond linguistic equivalence, with promising extensions to monolingual explanation and personalized adaptation.

[35] Comparison of Scoring Rationales Between Large Language Models and Human Raters

Haowei Hua,Hong Jiao,Dan Song

Main category: cs.CL

TL;DR: 本文比较了人类评分员与大型语言模型（LLMs）在自动评分中的评分理由，研究了评分不一致的原因，并通过聚类和相似性分析评估了LLMs的评分准确性。

Details Motivation: 随着LLMs在自动评分中的应用增加，研究其评分理由与人类评分的差异有助于提高对两者评分逻辑的理解，并优化自动评分的可靠性。

Contribution: 1. 比较了人类和LLMs的评分理由；2. 使用多种指标（如二次加权Kappa、余弦相似度）评估评分一致性；3. 通过主成分分析研究了评分理由的聚类模式。

Method: 1. 使用GPT-4o和Gemini等LLMs为大规模测试的论文评分；2. 通过二次加权Kappa和归一化互信息评估评分准确性；3. 使用余弦相似度比较评分理由的相似性；4. 通过主成分分析对评分理由嵌入进行聚类。

Result: 研究发现LLMs在评分中表现出一定的准确性，但其评分理由与人类评分员存在差异，揭示了评分不一致的潜在原因。

Insight: LLMs在自动评分中表现出强大的潜力，但其评分逻辑与人类不同，需要通过进一步研究优化其评分理由的一致性和可解释性。

Abstract: Advances in automated scoring are closely aligned with advances in machine-learning and natural-language-processing techniques. With recent progress in large language models (LLMs), the use of ChatGPT, Gemini, Claude, and other generative-AI chatbots for automated scoring has been explored. Given their strong reasoning capabilities, LLMs can also produce rationales to support the scores they assign. Thus, evaluating the rationales provided by both human and LLM raters can help improve the understanding of the reasoning that each type of rater applies when assigning a score. This study investigates the rationales of human and LLM raters to identify potential causes of scoring inconsistency. Using essays from a large-scale test, the scoring accuracy of GPT-4o, Gemini, and other LLMs is examined based on quadratic weighted kappa and normalized mutual information. Cosine similarity is used to evaluate the similarity of the rationales provided. In addition, clustering patterns in rationales are explored using principal component analysis based on the embeddings of the rationales. The findings of this study provide insights into the accuracy and ``thinking’’ of LLMs in automated scoring, helping to improve the understanding of the rationales behind both human scoring and LLM-based automated scoring.

[36] Retrieval-Constrained Decoding Reveals Underestimated Parametric Knowledge in Language Models

Rajaa El Hamdani,Samy Haffoudhi,Nils Holzenberger,Fabian Suchanek,Thomas Bonald,Fragkiskos D. Malliaros

Main category: cs.CL

TL;DR: 论文提出了一种名为检索约束解码（RCD）的策略，通过限制模型输出为唯一的表面形式，揭示了语言模型（LM）中被低估的参数知识。实验表明，RCD显著提高了LM在YAGO-QA数据集上的表现。

Details Motivation: 现有的语言模型虽然编码了大量事实知识，但由于严格的评估标准，许多正确的回答被误判为错误。作者认为这是低估了模型的参数知识。

Contribution: 提出了检索约束解码（RCD）方法，并引入了YAGO-QA数据集，展示了RCD如何显著提升语言模型的性能。

Method: RCD是一种解码策略，限制模型输出为唯一的表面形式，避免因表达多样性导致的误判。

Result: 实验结果显示，RCD显著提高了模型的F1分数，例如Llama-3.1-70B的F1从32.3%提升到46.0%。

Insight: 严格的评估标准可能低估了语言模型的知识能力，RCD为解码策略提供了一种新思路。

Abstract: Language models (LMs) encode substantial factual knowledge, but often produce answers judged as incorrect. We hypothesize that many of these answers are actually correct, but are expressed in alternative surface forms that are dismissed due to an overly strict evaluation, leading to an underestimation of models’ parametric knowledge. We propose Retrieval-Constrained Decoding (RCD), a decoding strategy that restricts model outputs to unique surface forms. We introduce YAGO-QA, a dataset of 19,137 general knowledge questions. Evaluating open-source LMs from 135M to 70B parameters, we show that standard decoding undervalues their knowledge. For instance, Llama-3.1-70B scores only 32.3% F1 with vanilla decoding but 46.0% with RCD. Similarly, Llama-3.1-8B reaches 33.0% with RCD, outperforming the larger model under vanilla decoding. We publicly share the code and dataset at https://github.com/Rajjaa/disambiguated-LLM.

[37] Cognition-of-Thought Elicits Social-Aligned Reasoning in Large Language Models

Xuanming Zhang,Yuxuan Chen,Min-Hsuan Yeh,Yixuan Li

Main category: cs.CL

TL;DR: 论文提出了一种名为Cognition-of-Thought（CooT）的解码时框架，通过显式认知自监控循环增强大语言模型的对齐能力，实现了动态、可审计的安全性控制。

Details Motivation: 现有的大语言模型对齐方法通常将安全性嵌入模型权重，导致控制隐式、静态且难以修改。CooT旨在通过显式的动态监控机制改进这一问题。

Contribution: CooT框架通过结合生成器和感知器，引入了动态的认知自监控机制，利用优先级原则检测并纠正生成过程中的对齐问题。

Method: CooT将标准文本生成器与认知感知器结合，感知器基于分层原则（如安全性优于服从性）实时监控生成内容，并在检测到问题时回滚并重新生成。

Result: 实验表明，CooT在多基准和模型家族中一致提升了安全性和社会推理表现。

Insight: 对齐可以从固定的权重属性转变为动态、显式的推理过程，通过解码时干预实现灵活的策略更新。

Abstract: Large language models (LLMs) excel at complex reasoning but can still exhibit harmful behaviors. Current alignment strategies typically embed safety into model weights, making these controls implicit, static, and difficult to modify. This paper introduces Cognition-of-Thought (CooT), a novel decoding-time framework that equips LLMs with an explicit cognitive self-monitoring loop. CooT couples a standard text Generator with a cognitive Perceiver that continuously monitors the unfolding sequence. The Perceiver uses a structured, precedence-based hierarchy of principles (e.g., safety over obedience) to detect potential misalignments as they arise. When violations are flagged, CooT intervenes by rolling back the generation to the point of error and regenerating under injected guidance that combines universal social priors with context-specific warnings. CooT thus transforms alignment from a fixed property into an explicit, dynamic, and auditable process active during inference, allowing for flexible policy updates without retraining the model. Extensive experiments across multiple benchmarks and model families confirm that CooT consistently improves safety and social reasoning performance.

[38] Text-Based Approaches to Item Difficulty Modeling in Large-Scale Assessments: A Systematic Review

Sydney Peters,Nan Zhang,Hong Jiao,Ming Li,Tianyi Zhou,Robert Lissitz

Main category: cs.CL

TL;DR: 本文系统综述了37篇关于在大规模评估中使用基于文本的方法建模题目难度的研究，总结了数据集、方法、模型表现等，发现语言模型尤其是基于Transformer的架构在自动化难度预测中表现优异。

Details Motivation: 传统题目难度建模依赖现场测试和经典测量理论（CTT）或项目反应理论（IRT），成本高且耗时。基于文本的机器学习方法提供了高效且经济的替代方案。

Contribution: 系统性总结了37篇相关研究，为未来研究提供了基准，并展示了语言模型（尤其是Transformer架构）在题目难度预测中的潜力。

Method: 综述了37篇论文，分析了数据集、难度参数、输入特征、模型架构（包括经典机器学习和Transformer模型）及评价指标。

Result: 基于文本的方法在RMSE（0.165）、Pearson相关性（0.87）和准确率（0.806）上表现优异，语言模型尤其突出。

Insight: 语言模型能自动捕捉句法和语义特征，避免了手工特征工程，经典模型则因其可解释性仍有价值。未来需进一步探索模型的通用性和公平性。

Abstract: Item difficulty plays a crucial role in test performance, interpretability of scores, and equity for all test-takers, especially in large-scale assessments. Traditional approaches to item difficulty modeling rely on field testing and classical test theory (CTT)-based item analysis or item response theory (IRT) calibration, which can be time-consuming and costly. To overcome these challenges, text-based approaches leveraging machine learning and language models, have emerged as promising alternatives. This paper reviews and synthesizes 37 articles on automated item difficulty prediction in large-scale assessment settings published through May 2025. For each study, we delineate the dataset, difficulty parameter, subject domain, item type, number of items, training and test data split, input, features, model, evaluation criteria, and model performance outcomes. Results showed that although classic machine learning models remain relevant due to their interpretability, state-of-the-art language models, using both small and large transformer-based architectures, can capture syntactic and semantic patterns without the need for manual feature engineering. Uniquely, model performance outcomes were summarized to serve as a benchmark for future research and overall, text-based methods have the potential to predict item difficulty with root mean square error (RMSE) as low as 0.165, Pearson correlation as high as 0.87, and accuracy as high as 0.806. The review concludes by discussing implications for practice and outlining future research directions for automated item difficulty modeling.

[39] The Impact of Role Design in In-Context Learning for Large Language Models

Hamidreza Rouzegar,Masoud Makrehchi

Main category: cs.CL

TL;DR: 本文研究了在大语言模型（LLM）的上下文学习（ICL）中，提示中角色设计的影响，发现角色配置可以提升模型性能。

Details Motivation: 尽管提示工程已被广泛研究，但提示中的角色设计对模型性能的影响尚未充分探索。

Contribution: 分析了角色设计在零样本和小样本学习场景中对LLM性能的影响，并验证了其在多个任务中的效果。

Method: 在GPT-3.5、GPT-4o、Llama2-7b和Llama2-13b上进行了实验，评估了角色配置在不同任务（如情感分析、文本分类等）中的表现。

Result: 研究结果表明，基于角色的提示结构设计可以提高LLM的性能。

Insight: 角色设计是提示工程中一个未被充分挖掘的因素，可能为LLM的性能优化提供新思路。

Abstract: In-context learning (ICL) enables Large Language Models (LLMs) to generate predictions based on prompts without additional fine-tuning. While prompt engineering has been widely studied, the impact of role design within prompts remains underexplored. This study examines the influence of role configurations in zero-shot and few-shot learning scenarios using GPT-3.5 and GPT-4o from OpenAI and Llama2-7b and Llama2-13b from Meta. We evaluate the models’ performance across datasets, focusing on tasks like sentiment analysis, text classification, question answering, and math reasoning. Our findings suggest the potential of role-based prompt structuring to enhance LLM performance.

[40] Towards Efficient CoT Distillation: Self-Guided Rationale Selector for Better Performance with Fewer Rationales

Jianzhi Yan,Le Liu,Youcheng Pan,Shiwei Chen,Yang Xiang,Buzhou Tang

Main category: cs.CL

TL;DR: MoRSD通过选择和优化高质量推理链（rationales）提升小语言模型的推理能力，减少了噪声和错误信息的传递，实现了性能提升和效率优化。

Details Motivation: 现有CoT蒸馏方法过于注重数据量而忽视推理链质量，导致噪声或错误信息传递给学生模型，影响了小模型的推理能力。

Contribution: 提出了MoRSD方法，通过模型导向的推理链选择和Rationale Difficulty度量标准，优选出高质量推理链进行蒸馏。

Method: 设计了Rationale Difficulty（RD）指标衡量学生模型在给定推理链下生成正确答案的能力，并基于此选择高准确、多样且难度适当的推理链用于蒸馏。

Result: 在七个数据集上平均提升了4.6%的性能，同时减少了推理链数量，验证了少量高质量推理链的有效性。

Insight: 研究发现高质量推理链比大规模数据更能提升学生模型的推理能力，为高效CoT蒸馏提供了新思路。

Abstract: Chain-of-thought (CoT) distillation aims to enhance small language models’ (SLMs) reasoning by transferring multi-step reasoning capability from the larger teacher models. However, existing work underestimates rationale quality, focusing primarily on data quantity, which may transfer noisy or incorrect information to the student model. To address the above issues, we proposed \textbf{M}odel-\textbf{O}riented \textbf{R}ationale \textbf{S}election \textbf{D}istillation (MoRSD), which can discern and select high quality rationales for distillation to improve performance further. We further propose a Rationale Difficulty (RD) metric to measure the ability of the student model to generate the correct answer under a given rationale. Compared to the baseline, we achieved 4.6$%$ average improvement on seven datasets over three tasks, using fewer rationales by controlling their accuracy, diversity, and difficulty. Our results reveal that a small portion of the high quality rationales can enhance the reasoning ability of student models than the entire dataset. Our method promises to be a possible solution for efficient CoT distillation. Our code will be released in https://github.com/Leon221220/MoRSD.

[41] Jackal: A Real-World Execution-Based Benchmark Evaluating Large Language Models on Text-to-JQL Tasks

Kevin Frank,Anmol Gulati,Elias Lumer,Sindy Campagna,Vamse Kumar Subbiah

Main category: cs.CL

TL;DR: Jackal 是一个新颖的大规模文本到 JQL 的基准测试，评估大型语言模型（LLMs）在自然语言查询到 JQL 查询转换任务上的表现。它包含 100,000 对自然语言请求和已验证的 JQL 查询，并在一个拥有超过 200,000 个问题的实时 Jira 实例上进行执行验证。

Details Motivation: 目前缺乏一个开放的、基于执行的基准测试来评估从自然语言到 JQL 查询的映射能力，尤其是在企业环境中。Jackal 的引入填补了这一空白。

Contribution: 1. 提出了 Jackal，首个大规模、基于执行的文本到 JQL 基准测试；2. 提供了执行评分工具和 Jira 实例快照以确保重现性；3. 评估了 23 种 LLMs 的性能，揭示了当前技术的局限性。

Method: Jackal 包含四种用户请求类型（长自然语言、短自然语言、语义相似、语义精确）的 JQL 查询对，并通过执行验证评分工具评估模型的执行准确性、精确匹配和规范精确匹配。

Result: 在 Jackal-5K 子集上，最佳模型 Gemini 2.5 Pro 的平均执行准确率仅为 60.3%，不同请求类型之间的表现差异显著（长自然语言 86.0%，短自然语言 35.7%，语义相似 22.7%，语义精确 99.3%）。

Insight: 当前 LLMs 在文本到 JQL 任务上的表现存在显著不足，尤其在复杂或语义模糊的查询上表现较差。Jackal 为未来研究提供了新的挑战和方向。

Abstract: Enterprise teams rely on the Jira Query Language (JQL) to retrieve and filter issues from Jira. Yet, to our knowledge, there is no open, real-world, execution-based benchmark for mapping natural language queries to JQL. We introduce Jackal, a novel, large-scale text-to-JQL benchmark comprising 100,000 natural language (NL) requests paired with validated JQL queries and execution-based results on a live Jira instance with over 200,000 issues. To reflect real-world usage, each JQL query is associated with four types of user requests: (i) Long NL, (ii) Short NL, (iii) Semantically Similar, and (iv) Semantically Exact. We release Jackal, a corpus of 100,000 text-to-JQL pairs, together with an execution-based scoring toolkit, and a static snapshot of the evaluated Jira instance for reproducibility. We report text-to-JQL results on 23 Large Language Models (LLMs) spanning parameter sizes, open and closed source models, across execution accuracy, exact match, and canonical exact match. In this paper, we report results on Jackal-5K, a 5,000-pair subset of Jackal. On Jackal-5K, the best overall model (Gemini 2.5 Pro) achieves only 60.3% execution accuracy averaged equally across four user request types. Performance varies significantly across user request types: (i) Long NL (86.0%), (ii) Short NL (35.7%), (iii) Semantically Similar (22.7%), and (iv) Semantically Exact (99.3%). By benchmarking LLMs on their ability to produce correct and executable JQL queries, Jackal exposes the limitations of current state-of-the-art LLMs and sets a new, execution-based challenge for future research in Jira enterprise data.

[42] LLM Hallucination Detection: HSAD

JinXin Li,Gang Tu,JunJie Hu

Main category: cs.CL

TL;DR: HSAD提出了一种基于隐藏层时域信号频域分析的大语言模型幻觉检测方法，通过模拟人类信号感知和判别过程，结合频域特征提取，显著提升了检测准确性和鲁棒性。

Details Motivation: 大语言模型在生成过程中频繁出现的幻觉问题成为其关键应用场景部署的阻碍，现有方法在知识覆盖范围和推理偏差检测上存在局限。

Contribution: 1. 首次将LLM推理过程建模为随时间展开的认知旅程；2. 提出基于频域分析的隐藏层信号特征提取方法；3. 设计了高准确性和鲁棒性的幻觉检测算法HSAD。

Method: 1. 模拟人类信号感知过程，建模隐藏层时域信号；2. 使用快速傅里叶变换提取频域特征；3. 基于频域特征设计幻觉检测算法。

Result: HSAD在检测准确性和鲁棒性上优于现有方法，有效克服了知识覆盖和推理偏差检测的局限。

Insight: 频域分析为LLM幻觉检测提供了新视角，揭示了推理过程中的异常信号特点，推动了领域发展。

Abstract: Although Large Language Models have demonstrated powerful capabilities in a wide range of tasks such as language understanding and code generation, the frequent occurrence of hallucinations during the generation process has become a significant impediment to their deployment in critical application scenarios. Current mainstream hallucination detection methods rely on factual consistency verification or static hidden layer features. The former is constrained by the scope of knowledge coverage, while the latter struggles to capture reasoning biases during the inference process. To address these issues, and inspired by signal analysis methods in cognitive neuroscience, this paper proposes a hallucination detection method based on the frequency-domain analysis of hidden layer temporal signals, named HSAD (\textbf{H}idden \textbf{S}ignal \textbf{A}nalysis-based \textbf{D}etection). First, by treating the LLM’s reasoning process as a cognitive journey that unfolds over time, we propose modeling and simulating the human process of signal perception and discrimination in a deception-detection scenario through hidden layer temporal signals. Next, The Fast Fourier Transform is applied to map these temporal signals into the frequency domain to construct spectral features, which are used to capture anomalies that arise during the reasoning process; analysis experiments on these spectral features have proven the effectiveness of this approach. Finally, a hallucination detection algorithm is designed based on these spectral features to identify hallucinations in the generated content. By effectively combining the modeling of the reasoning process with frequency-domain feature extraction, the HSAD method overcomes the limitations of existing approaches in terms of knowledge coverage and the detection of reasoning biases, demonstrating higher detection accuracy and robustness.

[43] Timber: Training-free Instruct Model Refining with Base via Effective Rank

Taiqiang Wu,Runming Yang,Tao Liu,Jiahao Wang,Zenan Xu,Ngai Wong

Main category: cs.CL

TL;DR: Timber是一种无需训练的方法，通过部分恢复Instruct模型到Base模型的权重差异，提升探索能力同时保持其利用能力。实验验证其在Pass@k性能上的有效性。

Details Motivation: 现有的后训练方法通常被认为表面化，导致Instruct模型在牺牲探索能力的情况下提升利用能力。Timber旨在解决这一权衡问题，无需额外训练即可优化模型。

Contribution: 1. 提供了权重层面有效秩（eRank）不变的新证据；2. 提出了训练免费的Timber方法，通过微调权重差异改进Instruct模型的探索能力；3. 在Llama和Qwen系列上验证了方法的有效性。

Method: Timber通过有针对性地调整权重差异（weight deltas），部分将Instruct模型恢复到Base模型的状态，从而提升探索能力。

Result: 实验表明，Timber显著提升了Pass@k性能，尤其在Instruct模型的探索能力方面表现突出。

Insight: 后训练阶段在权重层面的分析揭示了模型行为的变化，Timber提供了一种无需训练的实用优化策略。

Abstract: Post-training, which elicits a pretrained Base model into the corresponding Instruct model, is widely considered to be superficial. In this work, we first reinforce this hypothesis by providing novel quantitative evidence from the weight level that the effective rank (eRank) remains negligibly changed. However, this superficiality also suffers a critical trade-off, improving the exploitation capabilities at the cost of limiting its exploration. To tackle this issue, we propose Timber, a simple yet effective training-free method that enhances the exploration capability of the Instruct model while preserving its exploitation. The key insight is to partially revert Instruct towards the paired Base model by subtle yet targeted refinement of the weight deltas. Extensive experiments on Llama and Qwen series demonstrate that Timber consistently improves vanilla Instruct models, particularly on Pass@k performance. Our findings offer new insights into the post-training stage at the weight level and practical strategies to refine the Instruct model without training.

[44] Fast Thinking for Large Language Models

Haoyu Zheng,Zhuonan Wang,Yuqian Yuan,Tianwei Lin,Wenqiao Zhang,Zheqi Lv,Juncheng Li,Siliang Tang,Yueting Zhuang,Hongyang He

Main category: cs.CL

TL;DR: 本文提出了一种名为Fast-Thinking的框架，通过训练时学习离散策略先验的代码本，在推理时仅使用少量连续的思维向量，从而避免生成显式的推理标记，显著降低了推理成本。同时引入了GainRouter机制，自适应地在快速推理和显式推理间切换，提高了效率。

Details Motivation: 现有的Chain-of-Thought (CoT)方法虽然在复杂推理任务中表现优异，但由于需要生成冗长的推理步骤，导致延迟高且标记使用量大。作者旨在通过减少显式推理标记的生成，提高推理效率。

Contribution: 1) 提出了Fast-Thinking框架，利用离散策略先验的代码本在推理时避免显式推理标记的生成；2) 设计了GainRouter机制，动态选择快速或慢速推理模式；3) 在多个基准测试中实现高效且可控的推理。

Method: 1) 训练时使用简洁的CoT草图学习离散策略代码本；2) 推理时通过少量连续思维向量引导策略；3) GainRouter机制动态权衡推理速度和准确性。

Result: 实验表明，该方法在保持或提升准确性的同时，显著降低了推理成本。

Insight: 通过隐式策略引导和动态选择推理模式，可以有效平衡推理效率与模型性能。

Abstract: Reasoning-oriented Large Language Models (LLMs) often rely on generating explicit tokens step by step, and their effectiveness typically hinges on large-scale supervised fine-tuning or reinforcement learning. While Chain-of-Thought (CoT) techniques substantially enhance performance on complex reasoning tasks, they remain inefficient, requiring long reasoning traces that increase latency and token usage. In this work, we introduce Latent Codebooks for Fast Thinking, a framework that uses concise CoT sketches only during training to learn a codebook of discrete strategy priors. At inference, the model conditions on a handful of continuous thinking vectors distilled from the codebook in a single pass, enabling strategy-level guidance without producing explicit reasoning tokens. To complement this design, we propose GainRouter, a lightweight routing mechanism that adaptively switches between fast codebook guided inference and slow explicit reasoning, thereby suppressing overthinking and reducing unnecessary token generation. Experiments across multiple reasoning benchmarks show that our approach achieves competitive or superior accuracy while substantially lowering inference cost, offering a practical path toward efficient and controllable reasoning in large language models.

[45] Don’t Settle Too Early: Self-Reflective Remasking for Diffusion Language Models

Zemin Huang,Yuhang Wang,Zhiyang Chen,Guo-Jun Qi

Main category: cs.CL

TL;DR: 论文提出了RemeDi模型，通过引入remasking机制和联合预测token分布与置信度分数，提升了基于扩散的语言模型的文本生成质量。

Details Motivation: 现有基于掩码的扩散语言模型（DLMs）难以修正错误的token，生成后token通常固定不变，导致文本质量受限。

Contribution: 提出了RemeDi模型，引入了remasking机制，允许模型在生成过程中动态识别并修正低质量token。

Method: 联合预测token分布和置信度分数，设计了一个包含监督微调和强化学习的训练流程，前者用于检测和修正错误token，后者优化生成轨迹。

Result: RemeDi在多个数据集上取得了开源DLMs中的最优性能。

Insight: 动态remasking机制能够显著提升扩散语言模型生成文本的灵活性和质量。

Abstract: Mask-based Diffusion Language Models (DLMs) struggle to revise incorrect tokens: once a token is generated, it typically remains fixed. The key challenge is to identify potential errors in the inputs. In this paper, we propose \emph{\underline{Rem}asking-\underline{e}nabled \underline{Di}ffusion Language Model (RemeDi}, a mask-based DLM that introduces \emph{remasking} as another fundamental mechanism, enabling more flexible text refinement in diffusion-based text generation. To achieve this, RemeDi jointly predicts token distributions and per-token confidence scores at each step. The confidence scores determine which tokens to be unmasked after the current step, allowing the model to identify tokens with low quality and remask them. These remasked tokens can be resampled with richer context in subsequent steps. We design a remask-aware pipeline to train this ability, including supervised fine-tuning which teaches the model to detect and remask incorrect tokens in addition to predict mask tokens, and reinforcement learning which optimizes full generation trajectories toward higher rewards. Experiments show that RemeDi achieves the state-of-the-art results among open-source DLMs on multiple datasets.

[46] Beyond English-Centric Training: How Reinforcement Learning Improves Cross-Lingual Reasoning in LLMs

Shulin Huang,Yiran Ding,Junshu Pan,Yue Zhang

Main category: cs.CL

TL;DR: 论文探讨了强化学习（RL）与监督微调（SFT）在多语言复杂推理任务中的表现差异，发现RL在多语言推理中表现更优且泛化能力更强，尤其是在非英语数据上训练时。

Details Motivation: 增强大型语言模型（LLMs）的复杂推理能力是研究热点，但RL在多语言推理中的泛化能力尚未被系统研究。

Contribution: 首次系统比较了RL和SFT在多语言推理中的表现，揭示了RL在多语言泛化上的优势及其机制。

Method: 以Qwen2.5-3B-Base为基础模型，在多语言推理基准（数学、常识、科学推理）上对比RL和SFT的效果，并通过机制分析探索RL的优势原因。

Result: RL在多语言推理任务中不仅准确率更高，且泛化能力显著优于SFT；在非英语数据上训练的RL表现更优。

Insight: RL可能使模型学到更鲁棒的推理策略，为多语言推理的公平性和有效性提供了新方向。

Abstract: Enhancing the complex reasoning capabilities of Large Language Models (LLMs) attracts widespread attention. While reinforcement learning (RL) has shown superior performance for improving complex reasoning, its impact on cross-lingual generalization compared to Supervised Fine-Tuning (SFT) remains unexplored. We present the first systematic investigation into cross-lingual reasoning generalization of RL and SFT. Using Qwen2.5-3B-Base as our foundation model, we conduct experiments on diverse multilingual reasoning benchmarks, including math reasoning, commonsense reasoning, and scientific reasoning. Our investigation yields two significant findings: (1) Tuning with RL not only achieves higher accuracy but also demonstrates substantially stronger cross-lingual generalization capabilities compared to SFT. (2) RL training on non-English data yields better overall performance and generalization than training on English data, which is not observed with SFT. Furthermore, through comprehensive mechanistic analyses, we explore the underlying factors of RL’s superiority and generalization across languages. Our results provide compelling evidence that RL enables the model with more robust reasoning strategies, offering crucial guidance for more equitable and effective multilingual reasoning.

[47] Aligning LLMs for Multilingual Consistency in Enterprise Applications

Amit Agarwal,Hansa Meghwani,Hitesh Laxmichand Patel,Tao Sheng,Sujith Ravi,Dan Roth

Main category: cs.CL

TL;DR: 论文提出一种批量对齐策略，通过多语言语义等价数据微调LLMs，显著提升非英语语言的准确性，同时保持英语性能和模型推理能力。

Details Motivation: 大型语言模型（LLMs）在多语言企业应用中存在性能差距，尤其是非英语语言准确性显著低于英语，影响客户体验和操作可靠性。

Contribution: 提出了一种实用的批量对齐方法，利用多语言语义等价数据直接对齐模型输出，提升非英语语言准确性，且易于实现和扩展。

Method: 采用批量训练策略，每个训练批次包含语义等价的多语言数据，通过微调LLMs直接对齐不同语言的输出。

Result: 非英语语言准确性提升高达23.9%，且未损害英语性能、模型推理能力或检索质量。

Insight: 直接对齐多语言输出是一种简单有效的策略，可显著改善企业应用中的多语言模型一致性。

Abstract: Large language models (LLMs) remain unreliable for global enterprise applications due to substantial performance gaps between high-resource and mid/low-resource languages, driven by English-centric pretraining and internal reasoning biases. This inconsistency undermines customer experience and operational reliability in multilingual settings such as customer support, content moderation, and information retrieval. Even with advanced Retrieval-Augmented Generation (RAG) systems, we observe up to an 29% accuracy drop in non-English languages compared to English. We propose a practical, batch-wise alignment strategy for fine-tuning LLMs, leveraging semantically equivalent multilingual data in each training batch to directly align model outputs across languages. This approach improves non-English accuracy by up to 23.9% without compromising English performance, model reasoning, or retrieval quality. Our method is simple to implement, scalable, and integrates seamlessly with existing LLM training & deployment pipelines, enabling more robust and equitable multilingual AI solutions in industry.

[48] TF-Bench: Evaluating Program Semantics Reasoning with Type Inference in System F

Yifeng He,Luning Yang,Christopher Castro Gaw Gonzalo,Hao Chen

Main category: cs.CL

TL;DR: TF-Bench是一个新型基准测试，旨在评估大型语言模型（LLMs）在System F类型推断任务中的程序语义推理能力，揭示了当前最优模型的显著局限性。

Details Motivation: 现有基准测试缺乏形式化的程序逻辑评估框架，无法区分LLMs是真正理解程序语义还是依赖自然语言与代码的浅层关联。

Contribution: 提出了TF-Bench及其实证严格性更强的变体TF-Bench_pure，设计了两项新指标评估模型的推理稳健性和测试时推理效果。

Method: 利用System F的类型推断任务构造纯语义驱动的评估数据集，并通过已验证的转换去除语义无关的自然语言信息。

Result: 最佳模型Claude-3.7-sonnet在TF-Bench_pure上的准确率仅为55.85%，突显了LLMs在程序语义推理上的不足。

Insight: 研究揭示了当前LLMs在程序语义理解上的局限性，为未来研究方向提供了重要启示。

Abstract: Large Language Models (LLMs) are increasingly integrated into the software engineering ecosystem. Their test-time compute (TTC) reasoning capabilities show significant potential for understanding program logic and semantics beyond mere token recognition. However, current benchmarks for code reasoning lack a formal, program-centric deductive framework to ensure sound evaluation, and are incapable of assessing whether models genuinely reason about program semantics or merely exploit superficial associations between natural language and code tokens. To bridge this gap, we introduce TF-Bench, a benchmark designed to evaluate LLM reasoning based on type inference in System F, a task we refer to as program semantics reasoning. By employing verified transformations to remove semantically irrelevant natural language, we construct TF-Bench_pure, a purely semantics-driven variant of TF-Bench. Our analysis reveals substantial limitations in state-of-the-art LLMs, with the best-performing LLM (Claude-3.7-sonnet) achieving only 55.85% accuracy on TF-Bench_pure. Additionally, we propose two novel metrics to assess robustness and the effectiveness of test-time reasoning, underscoring critical limitations in current LLM capabilities and highlighting essential directions for future research.

[49] VIVA+: Human-Centered Situational Decision-Making

Zhe Hu,Yixiao Ren,Guanzhong Liu,Jing Li,Yu Yin

Main category: cs.CL

TL;DR: VIVA+是一个基于认知科学的基准测试，旨在评估多模态大型语言模型（MLLMs）在复杂人类环境中的决策和推理能力，包含1,317个真实场景和6,373个选择题，重点关注情境理解、行动理由和反思推理三个核心能力。

Details Motivation: 当前MLLMs在人类中心环境中的表现缺乏系统和深入的评估，尤其在细微推理和决策能力方面存在挑战。VIVA+填补了这一空白，提供了一个标准化的评估框架。

Contribution: 提出VIVA+基准测试，系统地评估MLLMs的三项核心能力；揭示了现有模型的性能差异与挑战，并提出了改进策略。

Method: 设计了1,317个真实场景和6,373个选择题，通过多步推理和针对性训练策略提升模型性能。

Result: 实验表明商业化模型和开源模型在VIVA+上表现差异显著，多步推理和训练策略能显著提升性能。

Insight: MLLMs在情境感知和社会适应性决策方面仍有较大局限，未来的改进方向包括提升上下文理解和多模态能力。

Abstract: Multimodal Large Language Models (MLLMs) show promising results for embodied agents in operating meaningfully in complex, human-centered environments. Yet, evaluating their capacity for nuanced, human-like reasoning and decision-making remains challenging. In this work, we introduce VIVA+, a cognitively grounded benchmark for evaluating the reasoning and decision-making of MLLMs in human-centered situations. VIVA+ consists of 1,317 real-world situations paired with 6,373 multiple-choice questions, targeting three core abilities for decision-making: (1) Foundational Situation Comprehension, (2) Context-Driven Action Justification, and (3) Reflective Reasoning. Together, these dimensions provide a systematic framework for assessing a model’s ability to perceive, reason, and act in socially meaningful ways. We evaluate the latest commercial and open-source models on VIVA+, where we reveal distinct performance patterns and highlight significant challenges. We further explore targeted training and multi-step reasoning strategies, which yield consistent performance improvements. Finally, our in-depth analysis highlights current model limitations and provides actionable insights for advancing MLLMs toward more robust, context-aware, and socially adept decision-making in real-world settings.

[50] Collaboration of Fusion and Independence: Hypercomplex-driven Robust Multi-Modal Knowledge Graph Completion

Zhiqiang Liu,Yichi Zhang,Mengshu Sun,Lei Liang,Wen Zhang

Main category: cs.CL

TL;DR: 论文提出了一种新的多模态知识图谱补全方法M-Hyper，通过融合与独立模态表征的协同工作，结合超复数代数（四元数）建模多模态交互，实现了显著的性能提升。

Details Motivation: 现有MMKGC方法分为融合型和集成型，前者丢失模态特异性信息且缺乏灵活性，后者难以捕捉模态间上下文相关的语义交互。M-Hyper旨在解决这两类方法的局限性。

Contribution: 1）提出M-Hyper方法，实现融合与独立模态表征的协同；2）引入四元数代数建模多模态交互；3）设计FERF和R2MF模块分别生成鲁棒的独立和融合模态表征。

Method: 1）FERF模块：细粒度实体表征分解，生成三种独立模态表征；2）R2MF模块：鲁棒的关系感知模态融合，生成一种融合模态表征；3）将四种表征映射到超复数空间（双四元数）建模交互。

Result: 实验表明M-Hyper在性能、鲁棒性和计算效率上均达到SOTA。

Insight: 超复数代数在多模态任务中具有潜力，能够自然建模正交模态间的交互，同时保持模态独立性。

Abstract: Multi-modal knowledge graph completion (MMKGC) aims to discover missing facts in multi-modal knowledge graphs (MMKGs) by leveraging both structural relationships and diverse modality information of entities. Existing MMKGC methods follow two multi-modal paradigms: fusion-based and ensemble-based. Fusion-based methods employ fixed fusion strategies, which inevitably leads to the loss of modality-specific information and a lack of flexibility to adapt to varying modality relevance across contexts. In contrast, ensemble-based methods retain modality independence through dedicated sub-models but struggle to capture the nuanced, context-dependent semantic interplay between modalities. To overcome these dual limitations, we propose a novel MMKGC method M-Hyper, which achieves the coexistence and collaboration of fused and independent modality representations. Our method integrates the strengths of both paradigms, enabling effective cross-modal interactions while maintaining modality-specific information. Inspired by ``quaternion’’ algebra, we utilize its four orthogonal bases to represent multiple independent modalities and employ the Hamilton product to efficiently model pair-wise interactions among them. Specifically, we introduce a Fine-grained Entity Representation Factorization (FERF) module and a Robust Relation-aware Modality Fusion (R2MF) module to obtain robust representations for three independent modalities and one fused modality. The resulting four modality representations are then mapped to the four orthogonal bases of a biquaternion (a hypercomplex extension of quaternion) for comprehensive modality interaction. Extensive experiments indicate its state-of-the-art performance, robustness, and computational efficiency.

[51] Do LLMs Understand Romanian Driving Laws? A Study on Multimodal and Fine-Tuned Question Answering

Eduard Barbu,Adrian Marius Dumitran

Main category: cs.CL

TL;DR: 该论文评估了大型语言模型（LLMs）在处理罗马尼亚驾驶法律问答任务中的表现，研究了多模态和领域特定微调的效果，并揭示了文本描述优于直接视觉输入的发现。

Details Motivation: 确保驾驶员掌握交通规则对道路安全至关重要，但针对低资源语言（如罗马尼亚语）的问答系统研究较少。

Contribution: 1.发布了包含1,208个问题的数据集（387个多模态）；2.比较了文本和多模态SOTA系统；3.研究了领域特定微调对LLMs的影响；4.揭示了文本描述优于视觉输入的发现；5.通过LLM-as-a-Judge评估解释质量。

Method: 1.构建罗马尼亚驾驶法律问答数据集；2.对比文本和多模态SOTA模型；3.微调Llama和RoLlama模型；4.使用LLM-as-a-Judge评估生成解释的质量。

Result: 1.SOTA模型表现良好，微调后的8B模型具有竞争力；2.文本描述优于直接视觉输入；3.LLM-as-a-Judge揭示了自偏好偏差。

Insight: 多模态任务中文本描述可能比直接视觉输入更有效；LLM-as-a-Judge可用于评估解释质量，但需注意偏差。

Abstract: Ensuring that both new and experienced drivers master current traffic rules is critical to road safety. This paper evaluates Large Language Models (LLMs) on Romanian driving-law QA with explanation generation. We release a 1{,}208-question dataset (387 multimodal) and compare text-only and multimodal SOTA systems, then measure the impact of domain-specific fine-tuning for Llama 3.1-8B-Instruct and RoLlama 3.1-8B-Instruct. SOTA models perform well, but fine-tuned 8B models are competitive. Textual descriptions of images outperform direct visual input. Finally, an LLM-as-a-Judge assesses explanation quality, revealing self-preference bias. The study informs explainable QA for less-resourced languages.

[52] Compose and Fuse: Revisiting the Foundational Bottlenecks in Multimodal Reasoning

Yucheng Wang,Yifan Hou,Aydin Javadov,Mubashara Akhtar,Mrinmaya Sachan

Main category: cs.CL

TL;DR: 本文提出了一个基于逻辑的多模态推理评估框架，揭示了模态交互中的核心瓶颈，并提出简单直接的方法缓解这些问题。

Details Motivation: 多模态大语言模型（MLLMs）在多模态推理中存在不一致的表现，缺乏对模态交互机制的系统分析，作者希望通过控制实验揭示核心问题。

Contribution: 1. 提出了一个逻辑驱动的多模态推理评估框架；2. 识别了任务组合瓶颈和融合瓶颈两大核心问题；3. 提出了简单的两步提示方法（先识别再推理）和改进早期融合的策略。

Method: 通过逻辑框架将多模态推理分为六种交互模式，分析事实分布和逻辑组合对性能的影响，实验验证了两大瓶颈的存在并提出解决方案。

Result: 额外模态仅在其提供独立且充分的推理路径时有益，冗余或链式支持则有害；两步提示和改进早期融合显著提升性能。

Insight: 多模态推理的主要障碍是模态整合而非感知能力，任务组合感知训练和融合控制是未来研究方向。

Abstract: Multimodal large language models (MLLMs) promise enhanced reasoning by integrating diverse inputs such as text, vision, and audio. Yet cross-modal reasoning remains underexplored, with conflicting reports on whether added modalities help or harm performance. These inconsistencies stem from a lack of controlled evaluation frameworks and analysis of models’ internals to isolate when and why modality interactions support or undermine reasoning. We address this gap through a logic-grounded evaluation framework that categorizes multimodal reasoning into six interaction patterns, varying how facts are distributed across modalities and logically combined. Empirically, additional modalities enhance reasoning only when they provide independent and sufficient reasoning paths, while redundant or chained entailment support often hurts performance. Moreover, reasoning degrades in three systematic ways: weaker modalities drag down overall performance, conflicts bias preference toward certain modalities, and joint signals from different modalities fail to be integrated effectively. Therefore, we identify two core failures: task-composition bottleneck, where recognition and reasoning cannot be jointly executed in one pass, and fusion bottleneck, where early integration introduces bias. For further investigation, we find that attention patterns fail to encode fact usefulness, but a simple two-step prompting (recognize then reason) restores performance, confirming the task-composition bottleneck. Moreover, modality identity remains recoverable in early layers, and softening attention in early fusion improves reasoning, highlighting biased fusion as another failure mode. Overall, our findings show that integration, not perception, is the main barrier to multimodal reasoning, suggesting composition-aware training and early fusion control as promising directions.

[53] Understanding Textual Capability Degradation in Speech LLMs via Parameter Importance Analysis

Chao Wang,Rui-Chen Zheng,Yang Ai,Zhen-Hua Ling

Main category: cs.CL

TL;DR: 这篇论文分析了语音增强的大型语言模型（LLMs）中文本能力退化的原因，并提出了一种基于参数重要性分析的框架，揭示了微调过程对文本推理关键参数的分布扰乱，并通过分层学习率调度和低秩适应（LoRA）两种方法缓解这一问题。

Details Motivation: 语音功能的引入扩展了LLMs的能力，但也导致了其核心文本能力的退化。论文旨在理解并解决这一问题，以充分发挥语音增强LLMs的潜力。

Contribution: 1. 提出了一个基于参数重要性分析的框架，揭示了语音微调对文本推理关键参数分布的扰乱机制。2. 探讨了两种缓解策略（分层学习率调度和LoRA），并实验验证了它们的有效性。

Method: 1. 使用参数重要性估计技术分析语音微调对LLMs的影响。2. 提出分层学习率调度和LoRA两种方法，以减少对文本推理关键参数的干扰。

Result: 实验表明，两种缓解策略在保持文本能力的同时，提升了语音问答任务的性能。这为LLMs的结构化知识保存提供了原则性解释。

Insight: 语音微调导致的文本能力退化与关键参数的分布偏移密切相关，针对性地调整学习率或使用LoRA可以有效缓解这一问题。

Abstract: The integration of speech into Large Language Models (LLMs) has substantially expanded their capabilities, but often at the cost of weakening their core textual competence. This degradation limits the ability of speech-enabled LLMs to fully exploit their pre-trained text-based knowledge. In this work, we analyze the underlying mechanisms of this issue through a focused study of the widely used encoder-adaptor paradigm. We propose an analytical framework based on parameter importance estimation, which reveals that fine-tuning for speech introduces a textual importance distribution shift: the layer-wise allocation of parameters critical to textual reasoning is disrupted. Building on this insight, we investigate two mitigation strategies: layer-wise learning rate scheduling and Low-Rank Adaptation (LoRA), both aim to preserve the original parameter distribution. Experimental results show that both approaches better maintain textual competence than full fine-tuning, while also improving downstream spoken question answering performance. Furthermore, our analysis offers a principled explanation for the effectiveness of the proposed mitigation strategies, linking their benefits to the structural properties of textual knowledge in LLMs.

[54] Knowledge-Level Consistency Reinforcement Learning: Dual-Fact Alignment for Long-Form Factuality

Junliang Li,Yucheng Wang,Yan Chen,Yu Ran,Ruiqing Zhang,Jing Liu,Hua Wu,Haifeng Wang

Main category: cs.CL

TL;DR: 论文提出了一种名为KLCF的新颖强化学习框架，通过双事实对齐机制优化长文本生成的事实性和一致性，解决了现有RLHF方法中的幻觉问题。

Details Motivation: 长文本生成中的幻觉和事实性缺陷是LLM可靠性的主要障碍。现有RLHF框架依赖偏好奖励，但忽略了模型内部的知识边界，加剧了幻觉问题。

Contribution: 提出了KLCF框架，通过知识一致性强化学习和双事实对齐机制，优化事实召回率和精确度。

Method: 利用预训练的知识边界构建事实清单指导在线强化学习，同时训练基于基础模型内部知识的自评估模块以提高事实精确度。

Result: 实验表明，KLCF显著提升了多长文本基准的事实性指标，有效缓解了幻觉问题。

Insight: KLCF通过轻量级、无需外部知识的设计，实现了高效且可扩展的事实性优化。

Abstract: Hallucination and factuality deficits remain key obstacles to the reliability of large language models (LLMs) in long-form generation. Existing reinforcement learning from human feedback (RLHF) frameworks primarily rely on preference rewards, yet they often overlook the model’s internal knowledge boundaries, exacerbating the so-called “hallucination tax”. To address this challenge, we propose Knowledge-Level Consistency Reinforcement Learning Framework (KLCF), a novel framework that focuses on the knowledge consistency between the policy model’s expressed knowledge and the base model’s parametric knowledge, and introduces a Dual-Fact Alignment mechanism to jointly optimize factual recall and precision. Specifically, KLCF leverages pretrained knowledge boundaries to construct fact checklist, guiding online reinforcement learning to improve factual coverage and recall; simultaneously, it trains a self-assessment module based on the base model’s internal knowledge to enhance factual precision during generation. Unlike prior methods that rely on external retrieval or heavy verification, our reward design is fully external-knowledge-free and lightweight, making KLCF efficient and easily scalable to large-scale training. Experimental results demonstrate that KLCF substantially improves factuality metrics across multiple long-form benchmarks and effectively alleviates model hallucinations.

[55] Transformer Tafsir at QIAS 2025 Shared Task: Hybrid Retrieval-Augmented Generation for Islamic Knowledge Question Answering

Muhammad Abu Ahmad,Mohamad Ballout,Raia Abu Ahmad,Elia Bruni

Main category: cs.CL

TL;DR: 该论文提出了一种混合检索增强生成（RAG）系统，用于伊斯兰知识问答任务，结合稀疏和稠密检索方法以及交叉编码器重排序，显著提升大型语言模型（LLM）的性能。

Details Motivation: 解决伊斯兰知识问答任务中LLM性能不足的问题，通过混合检索和重排序方法优化结果。

Contribution: 1. 提出三阶段RAG管道（BM25检索、稠密嵌入检索、交叉编码器重排序）；2. 在两子任务中验证性能提升（准确率提高达25%）。

Method: 1. BM25稀疏检索；2. 稠密嵌入模型语义匹配；3. 交叉编码器重排序；4. 结合LLM（Fanar和Mistral）生成答案。

Result: 最佳配置（Fanar）在两子任务中分别达到45%和80%的准确率。

Insight: 混合检索和重排序能显著提升LLM在特定领域问答任务中的表现。

Abstract: This paper presents our submission to the QIAS 2025 shared task on Islamic knowledge understanding and reasoning. We developed a hybrid retrieval-augmented generation (RAG) system that combines sparse and dense retrieval methods with cross-encoder reranking to improve large language model (LLM) performance. Our three-stage pipeline incorporates BM25 for initial retrieval, a dense embedding retrieval model for semantic matching, and cross-encoder reranking for precise content retrieval. We evaluate our approach on both subtasks using two LLMs, Fanar and Mistral, demonstrating that the proposed RAG pipeline enhances performance across both, with accuracy improvements up to 25%, depending on the task and model configuration. Our best configuration is achieved with Fanar, yielding accuracy scores of 45% in Subtask 1 and 80% in Subtask 2.

[56] Open-DeBias: Toward Mitigating Open-Set Bias in Language Models

Arti Rani,Shweta Singh,Nihar Ranjan Sahoo,Gaurav Kumar Nayak

Main category: cs.CL

TL;DR: 该论文提出了Open-DeBias方法，用于解决语言模型中的开放集偏见问题，并引入了OpenBiasBench基准测试。通过高效的数据和参数微调，该方法在偏见检测和缓解方面表现出色，且具备跨语言泛化能力。

Details Motivation: 现有偏见缓解方法仅针对预定义类别，无法处理新颖或上下文相关的偏见。论文旨在解决开放集偏见的检测与缓解问题。

Contribution: 1. 提出OpenBiasBench基准，涵盖已知和未见偏见类别；2. 开发Open-DeBias方法，高效缓解偏见并支持开放集泛化。

Method: 利用适配器模块（adapter modules）进行数据高效和参数高效的微调，缓解社会与刻板印象偏见，并泛化至未见偏见。

Result: 在BBQ数据集上，Open-DeBias在模糊子集上提升QA准确率48%，在明确子集上提升6%；在零样本迁移至韩语BBQ时达到84%准确率。

Insight: Open-DeBias展示了偏见缓解方法的可扩展性和跨语言泛化能力，适用于开放域和多语言场景。

Abstract: Large Language Models (LLMs) have achieved remarkable success on question answering (QA) tasks, yet they often encode harmful biases that compromise fairness and trustworthiness. Most existing bias mitigation approaches are restricted to predefined categories, limiting their ability to address novel or context-specific emergent biases. To bridge this gap, we tackle the novel problem of open-set bias detection and mitigation in text-based QA. We introduce OpenBiasBench, a comprehensive benchmark designed to evaluate biases across a wide range of categories and subgroups, encompassing both known and previously unseen biases. Additionally, we propose Open-DeBias, a novel, data-efficient, and parameter-efficient debiasing method that leverages adapter modules to mitigate existing social and stereotypical biases while generalizing to unseen ones. Compared to the state-of-the-art BMBI method, Open-DeBias improves QA accuracy on BBQ dataset by nearly $48%$ on ambiguous subsets and $6%$ on disambiguated ones, using adapters fine-tuned on just a small fraction of the training data. Remarkably, the same adapters, in a zero-shot transfer to Korean BBQ, achieve $84%$ accuracy, demonstrating robust language-agnostic generalization. Through extensive evaluation, we also validate the effectiveness of Open-DeBias across a broad range of NLP tasks, including StereoSet and CrowS-Pairs, highlighting its robustness, multilingual strength, and suitability for general-purpose, open-domain bias mitigation. The project page is available at: https://sites.google.com/view/open-debias25

[57] SPELL: Self-Play Reinforcement Learning for evolving Long-Context Language Models

Ziyi Yang,Weizhou Shen,Ruijun Chen,Chenliang Li,Fanqi Wan,Ming Yan,Xiaojun Quan,Fei Huang

Main category: cs.CL

TL;DR: SPELL是一个多角色自博弈强化学习框架，通过整合提问者、回答者和验证者三种角色，实现了无需标注的长上下文推理优化，显著提升了长上下文推理能力。

Details Motivation: 长上下文推理在大型语言模型中进展缓慢，主要因为长文本处理的固有难度以及缺乏可靠的人工标注和可编程奖励信号。

Contribution: 提出了SPELL框架，通过多角色自博弈强化学习实现了可扩展的无标注优化，显著提升了长上下文推理能力。

Method: 整合了提问者、回答者和验证者三种角色，通过自博弈生成问题和奖励信号，逐步提升模型能力。

Result: 在六个长上下文基准测试中表现优异，平均提升了7.6个点的pass@8指标。

Insight: SPELL展示了自博弈强化学习在长上下文推理中的潜力，尤其在没有大量标注数据的情况下仍能显著提升性能。

Abstract: Progress in long-context reasoning for large language models (LLMs) has lagged behind other recent advances. This gap arises not only from the intrinsic difficulty of processing long texts, but also from the scarcity of reliable human annotations and programmatically verifiable reward signals. In this paper, we propose SPELL, a multi-role self-play reinforcement learning framework that enables scalable, label-free optimization for long-context reasoning. SPELL integrates three cyclical roles-questioner, responder, and verifier-within a single model to enable continual self-improvement. The questioner generates questions from raw documents paired with reference answers; the responder learns to solve these questions based on the documents; and the verifier evaluates semantic equivalence between the responder’s output and the questioner’s reference answer, producing reward signals to guide continual training. To stabilize training, we introduce an automated curriculum that gradually increases document length and a reward function that adapts question difficulty to the model’s evolving capabilities. Extensive experiments on six long-context benchmarks show that SPELL consistently improves performance across diverse LLMs and outperforms equally sized models fine-tuned on large-scale annotated data. Notably, SPELL achieves an average 7.6-point gain in pass@8 on the strong reasoning model Qwen3-30B-A3B-Thinking, raising its performance ceiling and showing promise for scaling to even more capable models.

[58] Winning the Pruning Gamble: A Unified Approach to Joint Sample and Token Pruning for Efficient Supervised Fine-Tuning

Shaobo Wang,Jiaming Wang,Jiajun Zhang,Cong Wang,Yue Min,Zichen Wen,Fei Huang,Huiqiang Jiang,Junyang Lin,Dayiheng Liu,Linfeng Zhang

Main category: cs.CL

TL;DR: 该论文提出了一种名为Q-Tuning的统一框架，用于联合优化样本和token级别的剪枝策略，通过Error-Uncertainty (EU) Plane诊断数据效用，显著提升了监督微调（SFT）的数据效率。

Details Motivation: 随着监督微调（SFT）的计算成本增加，传统剪枝方法（单一样本或token级别）存在效率低下的问题，未能充分利用数据潜力。

Contribution: 1) 提出EU Plane框架，联合分析样本和token的效用；2) 设计Q-Tuning方法，分阶段动态剪枝，实现数据高效利用。

Method: 1) 样本级筛选保留高价值样本；2) 非对称token剪枝策略，仅裁剪低效token。

Result: 在SmolLM2-1.7B上，仅用12.5%数据即超越完整数据训练38%。

Insight: 联合样本和token剪枝能更精细地利用数据，显著提升模型性能，尤其在有限预算下表现出优越性。

Abstract: As supervised fine-tuning (SFT) evolves from a lightweight post-training step into a compute-intensive phase rivaling mid-training in scale, data efficiency has become critical for aligning large language models (LLMs) under tight budgets. Existing data pruning methods suffer from a fragmented design: they operate either at the sample level or the token level in isolation, failing to jointly optimize both dimensions. This disconnect leads to significant inefficiencies–high-value samples may still contain redundant tokens, while token-level pruning often discards crucial instructional or corrective signals embedded in individual examples. To address this bottleneck, we introduce the Error-Uncertainty (EU) Plane, a diagnostic framework that jointly characterizes the heterogeneous utility of training data across samples and tokens. Guided by this insight, we propose Quadrant-based Tuning (Q-Tuning), a unified framework that strategically coordinates sample pruning and token pruning. Q-Tuning employs a two-stage strategy: first, it performs sample-level triage to retain examples rich in informative misconceptions or calibration signals; second, it applies an asymmetric token-pruning policy, using a context-aware scoring mechanism to trim less salient tokens exclusively from misconception samples while preserving calibration samples in their entirety. Our method sets a new state of the art across five diverse benchmarks. Remarkably, on SmolLM2-1.7B, Q-Tuning achieves a +38% average improvement over the full-data SFT baseline using only 12.5% of the original training data. As the first dynamic pruning approach to consistently outperform full-data training, Q-Tuning provides a practical and scalable blueprint for maximizing data utilization in budget-constrained LLM SFT.

[59] DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruning

Yibo Yan,Guangwei Xu,Xin Zou,Shuliang Liu,James Kwok,Xuming Hu

Main category: cs.CL

TL;DR: DocPruner提出了一个存储高效的多向量视觉文档检索框架，通过自适应剪枝减少存储开销。

Details Motivation: 当前基于多向量范式的视觉文档检索方法虽然效果显著，但存储开销巨大，难以大规模部署，亟需解决方案。

Contribution: DocPruner首次引入自适应块级嵌入剪枝机制，显著减少存储需求（50-60%），同时保持检索性能。

Method: 利用文档内部块的注意力分布动态识别并剪枝冗余嵌入，实现高效存储。

Result: 在十多个数据集上的实验表明，DocPruner显著减少存储，性能损失可忽略。

Insight: 自适应剪枝是解决多向量范式存储问题的有效途径，为大规模VDR系统提供了实用的解决方案。

Abstract: Visual Document Retrieval (VDR), the task of retrieving visually-rich document pages using queries that combine visual and textual cues, is crucial for numerous real-world applications. Recent state-of-the-art methods leverage Large Vision-Language Models (LVLMs) in a multi-vector paradigm, representing each document as patch-level embeddings to capture fine-grained details. While highly effective, this approach introduces a critical challenge: prohibitive storage overhead, as storing hundreds of vectors per page makes large-scale deployment costly and impractical. To address this, we introduce DocPruner, the first framework to employ adaptive patch-level embedding pruning for VDR to effectively reduce the storage overhead. DocPruner leverages the intra-document patch attention distribution to dynamically identify and discard redundant embeddings for each document. This adaptive mechanism enables a significant 50-60% reduction in storage for leading multi-vector VDR models with negligible degradation in document retrieval performance. Extensive experiments across more than ten representative datasets validate that DocPruner offers a robust, flexible, and effective solution for building storage-efficient, large-scale VDR systems.

[60] Taming Masked Diffusion Language Models via Consistency Trajectory Reinforcement Learning with Fewer Decoding Step

Jingyi Yang,Guanxu Chen,Xuhao Hu,Jing Shao

Main category: cs.CL

TL;DR: 本文提出了一种名为EOSER和ASS的解码调度器，以及CJ-GRPO强化学习方法，用于优化掩码扩散语言模型（MDLMs）的解码和训练过程，减少解码步骤并提高性能。

Details Motivation: 当前掩码扩散语言模型的解码策略和强化学习方法研究不足，直接迁移自回归模型技术可能不最优。本文旨在解决MDLMs解码和训练中的不一致性问题。

Contribution: 1) 提出EOSER和ASS解码调度器，支持MDLMs实现全扩散式解码；2) 提出CJ-GRPO方法，优化训练与解码轨迹一致性。

Method: 结合EOSER和ASS解码调度器减少解码步骤，利用CJ-GRPO方法确保训练和解码轨迹一致，提升MDLM性能。

Result: 在数学和规划任务上实验，基于LLaDA-8B-Instruct模型，验证了方法的有效性和高效性。

Insight: 通过优化解码和强化学习方法，MDLMs能更高效地实现并行解码，减少推理步骤，适用于复杂推理任务。

Abstract: Masked diffusion language models (MDLMs) have recently emerged as a promising alternative to autoregressive (AR) language models, offering properties such as parallel decoding, flexible generation orders, and the potential for fewer inference steps. Despite these advantages, decoding strategies and reinforcement learning (RL) algorithms tailored for MDLMs remain underexplored. A naive approach is to directly transfer techniques well-established for AR models to MDLMs. However, this raises an immediate question: Is such a naive transfer truly optimal? For example, 1) Block-wise and semi-AR decoding strategies are not employed during the training of MDLMs, so why do they outperform full diffusion-style decoding during inference? 2) Applying RL algorithms designed for AR models directly to MDLMs exhibits a training-inference inconsistency, since MDLM decoding are non-causal (parallel). This results in inconsistencies between the rollout trajectory and the optimization trajectory. To address these challenges, we propose EOS Early Rejection (EOSER) and Ascending Step-Size (ASS) decoding scheduler, which unlock the potential of MDLMs to perform full diffusion-style decoding, achieving competitive performance with fewer decoding steps. Additionally, we introduce Consistency Trajectory Group Relative Policy Optimization (CJ-GRPO) for taming MDLMs, which emphasizes the consistency between rollout trajectory and optimization trajectory, and reduces the optimization errors caused by skip-step optimization. We conduct extensive experiments on reasoning tasks, such as mathematical and planning benchmarks, using LLaDA-8B-Instruct. The results demonstrate that the proposed EOSER and ASS mechanisms, together with CJ-GRPO, hold significant promise for effectively and efficiently taming MDLMs. Code: https://github.com/yjyddq/EOSER-ASS-RL.

[61] Assessing Large Language Models in Updating Their Forecasts with New Information

Zhangdie Yuan,Zifeng Ding,Andreas Vlachos

Main category: cs.CL

TL;DR: EVOLVECAST框架评估大语言模型（LLMs）在新信息出现时如何调整预测和置信度，发现其更新行为不一致且保守，置信度校准也未达人类水平。

Details Motivation: 现有研究将事件预测视为静态任务，忽视了新证据如何动态影响预测和置信度。本文提出评估LLMs在新信息下更新预测的能力。

Contribution: 提出EVOLVECAST框架，首次系统评估LLMs在新信息下的预测更新行为，并对比人类数据揭示其不一致性和保守偏差。

Method: 使用EVOLVECAST框架，对比LLMs和人类在新信息下的预测调整与置信度校准，分析语言化置信度和对数置信度的表现差异。

Result: LLMs对新信息有一定响应，但更新行为不一致且保守，置信度校准未显著优于对方，且均远低于人类参考标准。

Insight: LLMs在新信息下的预测更新能力存在局限，需进一步研究更稳健的信念更新机制，其保守偏差可能影响实际决策可靠性。

Abstract: Prior work has largely treated future event prediction as a static task, failing to consider how forecasts and the confidence in them should evolve as new evidence emerges. To address this gap, we introduce EVOLVECAST, a framework for evaluating whether large language models appropriately revise their predictions in response to new information. In particular, EVOLVECAST assesses whether LLMs adjust their forecasts when presented with information released after their training cutoff. We use human forecasters as a comparative reference to analyze prediction shifts and confidence calibration under updated contexts. While LLMs demonstrate some responsiveness to new information, their updates are often inconsistent or overly conservative. We further find that neither verbalized nor logits-based confidence estimates consistently outperform the other, and both remain far from the human reference standard. Across settings, models tend to express conservative bias, underscoring the need for more robust approaches to belief updating.

[62] Vision-Grounded Machine Interpreting: Improving the Translation Process through Visual Cues

Claudio Fantinuoli

Main category: cs.CL

TL;DR: 该论文提出了视觉接地机器口译（VGI），通过引入视觉线索改进单模态机器口译的性能。

Details Motivation: 当前机器口译系统仅依赖语音信号，限制了在多模环境下（如需要视觉上下文）的翻译效果。

Contribution: 提出了VGI系统原型，集成视觉语言模型，利用视觉输入提升翻译过程中的歧义消解能力。

Method: 构建了一个原型系统，结合语音和摄像头视觉输入，并使用诊断语料库评估视觉接地效果。

Result: 视觉接地显著改进了词汇歧义消解，但对性别解析的提升有限，对句法歧义无帮助。

Insight: 多模态（视觉+语音）是提升机器口译质量的必要方向，但在不同歧义类型上效果差异显著。

Abstract: Machine Interpreting systems are currently implemented as unimodal, real-time speech-to-speech architectures, processing translation exclusively on the basis of the linguistic signal. Such reliance on a single modality, however, constrains performance in contexts where disambiguation and adequacy depend on additional cues, such as visual, situational, or pragmatic information. This paper introduces Vision-Grounded Interpreting (VGI), a novel approach designed to address the limitations of unimodal machine interpreting. We present a prototype system that integrates a vision-language model to process both speech and visual input from a webcam, with the aim of priming the translation process through contextual visual information. To evaluate the effectiveness of this approach, we constructed a hand-crafted diagnostic corpus targeting three types of ambiguity. In our evaluation, visual grounding substantially improves lexical disambiguation, yields modest and less stable gains for gender resolution, and shows no benefit for syntactic ambiguities. We argue that embracing multimodality represents a necessary step forward for advancing translation quality in machine interpreting.

[63] HiPO: Hybrid Policy Optimization for Dynamic Reasoning in LLMs

Ken Deng,Zizheng Zhan,Wen Xiang,Wenqiang Zhu,Tianhao Peng,Xinping Lei,Weihao Li,Jingxuan Xu,Kun Wu,Yifan Yao,Haoyang Huang,Huaixi Tang,Kepeng Lei,Zhiyi Lai,Songwei Yu,Zongxian Feng,Zuchen Gao,Weihao Xie,Chenchen Zhang,Yanan Wu,Yuanxing Zhang,Lecheng Huang,Yuqun Zhang,Jie Liu,Zhaoxiang Zhang,Haotian Zhang,Bin Chen,Jiaheng Liu

Main category: cs.CL

TL;DR: HiPO提出了一种混合策略优化框架，用于动态控制LLM的推理过程，选择性决定何时详细推理（Think-on）或直接回应（Think-off），从而在保持精度的同时减少计算开销。

Details Motivation: 现有的链式推理（CoT）方法虽能提升LLM在复杂任务的精度，但始终生成冗长推理代价高昂，导致不必要的token开销和高推理成本。

Contribution: 1. 提出HiPO框架，实现自适应推理控制；2. 结合混合数据管道和强化学习奖励机制，平衡精度与效率；3. 实验证明其可在数学和编程任务中减少token使用且不损失精度。

Method: 1. 构建包含Think-on和Think-off配对的混合数据管道；2. 设计混合强化学习奖励系统，避免过度依赖详细推理；3. 通过动态决策选择最佳推理模式。

Result: 在数学和编程基准测试中，HiPO显著减少了token使用量，同时维持或提升了任务精度。

Insight: HiPO为资源敏感场景下的高效推理提供了原则性解决方案，推动了面向推理的LLM的实际部署。

Abstract: Large Language Models (LLMs) increasingly rely on chain-of-thought (CoT) reasoning to improve accuracy on complex tasks. However, always generating lengthy reasoning traces is inefficient, leading to excessive token usage and higher inference costs. This paper introduces the Hybrid Policy Optimization (i.e., HiPO), a framework for adaptive reasoning control that enables LLMs to selectively decide when to engage in detailed reasoning (Think-on) and when to respond directly (Think-off). Specifically, HiPO combines a hybrid data pipelineproviding paired Think-on and Think-off responseswith a hybrid reinforcement learning reward system that balances accuracy and efficiency while avoiding over-reliance on detailed reasoning. Experiments across mathematics and coding benchmarks demonstrate that HiPO can substantially reduce token length while maintaining or improving accuracy. Finally, we hope HiPO a can be a principled approach for efficient adaptive reasoning, advancing the deployment of reasoning-oriented LLMs in real-world, resource-sensitive settings.

[64] Toward Preference-aligned Large Language Models via Residual-based Model Steering

Lucio La Cava,Andrea Tagarelli

Main category: cs.CL

TL;DR: PaLRS提出了一种无需训练、即插即用的方法，通过提取LLM残差流中的偏好信号，生成轻量级的转向向量，实现在推理时对齐模型偏好。相比现有方法，PaLRS更高效、灵活且节省时间。

Details Motivation: 现有偏好对齐方法（如RLHF或DPO）需要大量标注数据和昂贵的参数优化，且生成的任务特定模型难以复用。PaLRS旨在解决这些问题，提供一种数据需求低、无需训练的高效替代方案。

Contribution: 1. 提出PaLRS方法，利用LLM残差流中的偏好信号生成转向向量。2. 仅需少量偏好对即可实现偏好对齐，且无需训练。3. 在数学推理和代码生成任务中表现优于DPO对齐模型。

Method: PaLRS通过提取LLM残差流中的偏好信号，生成轻量级转向向量，并在推理时应用这些向量调整模型行为。方法仅需100对偏好数据，无需额外优化。

Result: 实验证明，PaLRS对齐的模型在数学推理和代码生成任务中表现一致提升，同时保持基础模型的通用性能。相比DPO，PaLRS节省大量时间且效果更好。

Insight: LLM残差流中隐含了丰富的偏好信号，可通过轻量级方法提取并应用于模型对齐，这为高效偏好对齐提供了新方向。

Abstract: Preference alignment is a critical step in making Large Language Models (LLMs) useful and aligned with (human) preferences. Existing approaches such as Reinforcement Learning from Human Feedback or Direct Preference Optimization typically require curated data and expensive optimization over billions of parameters, and eventually lead to persistent task-specific models. In this work, we introduce Preference alignment of Large Language Models via Residual Steering (PaLRS), a training-free method that exploits preference signals encoded in the residual streams of LLMs. From as few as one hundred preference pairs, PaLRS extracts lightweight, plug-and-play steering vectors that can be applied at inference time to push models toward preferred behaviors. We evaluate PaLRS on various small-to-medium-scale open-source LLMs, showing that PaLRS-aligned models achieve consistent gains on mathematical reasoning and code generation benchmarks while preserving baseline general-purpose performance. Moreover, when compared to DPO-aligned models, they perform better with huge time savings. Our findings highlight that PaLRS offers an effective, much more efficient and flexible alternative to standard preference optimization pipelines, offering a training-free, plug-and-play mechanism for alignment with minimal data.

[65] GEAR: A General Evaluation Framework for Abductive Reasoning

Kaiyu He,Peilin Wu,Mian Zhang,Kun Wan,Wentian Zhao,Xinya Du,Zhiyu Chen

Main category: cs.CL

TL;DR: 论文提出了GEAR框架，用于评估大型语言模型（LLMs）的溯因推理能力，通过一致性、泛化性和多样性三个指标自动评分，无需人工标注。GEAR还能为模型提供无监督的训练信号。

Details Motivation: 现有的研究主要集中在指令跟随和演绎推理，而忽略了LLMs发现新知识的能力。因此，作者希望通过溯因推理（生成解释观察结果的合理假设）来填补这一空白，并开发一个无需标注、可扩展的评估框架。

Contribution: 1. 提出GEAR框架，通过一致性、泛化性和多样性自动评估溯因推理；2. 设计了基于动量的课程学习策略，利用GEAR生成的信号优化LLMs；3. 在四个基准测试上评估了九种LLMs，揭示了传统评估方法掩盖的差异。

Method: 1. GEAR框架通过三部分评分：一致性、泛化性和多样性；2. 动量课程学习策略动态调整训练数据，从易到难优化模型；3. 在1,500个问题和50,000个假设上进行实验。

Result: 实验表明，GEAR能够可靠地评估LLMs的溯因能力，且基于GEAR的训练策略显著提升了所有指标的得分，并能迁移到传统基准测试中。

Insight: 1. 无需人工标注的评估框架可以更灵活地适应模型能力的提升；2. 动态课程学习策略有助于模型逐步掌握复杂任务；3. GEAR揭示了LLMs在溯因推理中的潜在改进空间。

Abstract: Since the advent of large language models (LLMs), research has focused on instruction following and deductive reasoning. A central question remains: can these models discover new knowledge, and how can we evaluate this ability? We address this by studying abductive reasoning-the generation of plausible hypotheses to explain observations-and introduce GEAR (General Evaluation for Abductive Reasoning), a general-purpose, fully automated, transparent, and label-free evaluation paradigm. GEAR scores hypothesis sets by three metrics: consistency (each hypothesis explains the observations), generalizability (consistent hypotheses make meaningful predictions on unseen inputs), and diversity (the set covers distinct predictions and patterns). Built this way, GEAR is scalable (no human gold answers), reliable (deterministic scoring aligned with classical abduction), and open-ended (scores improve only when models produce new plausible hypotheses, unlike static benchmarks that saturate once accuracy is high). Using GEAR, we conduct a fine-grained study of nine LLMs on four abduction benchmarks with 1,500 problems, generating over 50,000 candidate hypotheses and revealing model differences obscured by gold-answer or purely human evaluations. We further propose a momentum-based curriculum that adjusts GEAR-derived training data by learning velocity: it starts with what the model learns quickly and shifts toward harder objectives such as generating diverse hypotheses once the model is confident on foundational objectives. Without gold-label supervision, this strategy improves all GEAR objectives and these gains transfer to established abductive reasoning benchmarks. Taken together, GEAR provides a principled framework that evaluates abduction and supplies label-free, scalable training signals that help LLMs produce more diverse and reliable hypotheses.

[66] BTC-SAM: Leveraging LLMs for Generation of Bias Test Cases for Sentiment Analysis Models

Zsolt T. Kardkovács,Lynda Djennane,Anna Field,Boualem Benatallah,Yacine Gaci,Fabio Casati,Walid Gaaloul

Main category: cs.CL

TL;DR: BTC-SAM提出了一种利用LLMs生成高质量测试用例的新框架，用于检测情感分析模型的偏见问题，覆盖范围更广且语言多样性更好。

Details Motivation: 情感分析模型中存在社会偏见，传统测试方法依赖专家或众包，成本高且覆盖有限。BTC-SAM旨在通过LLMs低成本生成高质量测试用例，解决这一问题。

Contribution: 提出了BTC-SAM框架，利用LLMs生成多样化和高覆盖度的测试用例，检测情感分析模型的偏见问题。

Method: 基于LLMs的可控生成方法，生成自然、语言丰富的测试句子，覆盖多种偏见类型。

Result: 实验表明，BTC-SAM生成的测试用例具有更好的语言多样性和覆盖范围，优于传统提示方法。

Insight: LLMs可以高效生成高质量测试用例，为模型偏见检测提供了一种低成本、可扩展的新方法。

Abstract: Sentiment Analysis (SA) models harbor inherent social biases that can be harmful in real-world applications. These biases are identified by examining the output of SA models for sentences that only vary in the identity groups of the subjects. Constructing natural, linguistically rich, relevant, and diverse sets of sentences that provide sufficient coverage over the domain is expensive, especially when addressing a wide range of biases: it requires domain experts and/or crowd-sourcing. In this paper, we present a novel bias testing framework, BTC-SAM, which generates high-quality test cases for bias testing in SA models with minimal specification using Large Language Models (LLMs) for the controllable generation of test sentences. Our experiments show that relying on LLMs can provide high linguistic variation and diversity in the test sentences, thereby offering better test coverage compared to base prompting methods even for previously unseen biases.

[67] Pragmatic Inference for Moral Reasoning Acquisition: Generalization via Distributional Semantics

Guangliang Liu,Xi Chen,Bocheng Chen,Xitong Zhang,Kristen Johnson

Main category: cs.CL

TL;DR: 论文探讨了如何通过语用推理方法提升大语言模型在道德推理中的泛化能力，填补语用与语义间的差距。

Details Motivation: 大语言模型依赖分布语义，但在道德推理中泛化能力不足，需解决语用层面的挑战。

Contribution: 提出基于道德基础理论的语用推理方法，通过上下文信息连接道德基础与推理目标，提升泛化能力。

Method: 采用语用推理方法，结合道德基础理论，逐步利用上下文信息指导模型推理。

Result: 实验证明方法显著提升大语言模型的道德推理泛化能力。

Insight: 研究为基于道德基础理论的未来工作奠定基础，揭示了语用信息在道德推理中的重要性。

Abstract: Moral reasoning has emerged as a promising research direction for Large Language Models (LLMs), yet achieving generalization remains a central challenge. From a linguistic standpoint, this difficulty arises because LLMs are adept at capturing distributional semantics, which fundamentally differs from the morals which operate at the pragmatic level. This paper investigates how LLMs can achieve generalized moral reasoning despite their reliance on distributional semantics. We propose pragmatic inference methods grounded in moral foundations theory, which leverage contextual information at each step to bridge the pragmatic gap and guide LLMs in connecting moral foundations with moral reasoning objectives. Experimental results demonstrate that our approach significantly enhances LLMs’ generalization in moral reasoning, providing a foundation for future research grounded in moral foundations theory.

[68] Dual-Scale World Models for LLM Agents Towards Hard-Exploration Problems

Minsoo Kim,Seung-won Hwang

Main category: cs.CL

TL;DR: GLoW提出了一种双尺度世界模型方法，通过全局轨迹前沿和局部试错学习解决LLM智能体在硬探索任务中的局限性，并在Jericho基准测试中取得SOTA性能。

Details Motivation: LLM智能体在需要探索学习新知识的硬探索任务中表现受限，因此提出GLoW方法以提升其探索能力。

Contribution: 提出双尺度世界模型（全局和局部）和Multi-path Advantage Reflection机制，指导探索并减少环境交互需求。

Method: 结合全局高价值轨迹前沿和局部试错学习，利用Multi-path Advantage Reflection生成优势信号引导探索。

Result: 在Jericho基准测试中，GLoW表现优异，性能与SOTA强化学习方法相当，但环境交互需求减少100-800倍。

Insight: 双尺度建模和优势信号引导能显著提升LLM智能体的探索效率，降低对高交互成本的需求。

Abstract: LLM-based agents have seen promising advances, yet they are still limited in “hard-exploration” tasks requiring learning new knowledge through exploration. We present GLoW, a novel approach leveraging dual-scale world models, maintaining a trajectory frontier of high-value discoveries at the global scale, while learning from local trial-and-error in exploration through a Multi-path Advantage Reflection mechanism which infers advantage-based progress signals to guide exploration. To evaluate our framework for hard-exploration, we tackle the Jericho benchmark suite of text-based games, where GLoW achieves a new state-of-theart performance for LLM-based approaches. Compared to state-of-the-art RLbased methods, our approach achieves comparable performance while requiring 100-800x fewer environment interactions.

[69] EduVidQA: Generating and Evaluating Long-form Answers to Student Questions based on Lecture Videos

Sourjyadip Ray,Shubham Sharma,Somak Aditya,Pawan Goyal

Main category: cs.CL

TL;DR: 该论文提出了EduVidQA数据集和任务，利用多模态大语言模型（MLLMs）自动生成并评估学生对在线讲座视频的提问的长篇回答。数据集包含5252个问题-答案对，来自296个计算机科学视频，涵盖多种主题和难度。作者通过实证研究学生偏好，并评估了6种先进MLLM的表现。

Details Motivation: 随着数字平台重塑教育模式，确保互动性对有效学习至关重要。论文探索了MLLMs在自动回答学生在线讲座问题中的应用，填补了这一实际重要任务的研究空白。

Contribution: 1. 提出了EduVidQA数据集；2. 通过学生偏好研究提供了任务设计的见解；3. 评估了6种MLLM的性能，揭示了任务的挑战性。

Method: 1. 构建了包含合成和真实问题的EduVidQA数据集；2. 通过学生实证研究定性偏好；3. 使用文本和定性指标评估多种MLLM。

Result: 实验表明合成数据对微调MLLMs有效，但任务本身仍具挑战性。评估结果显示性能的复杂性，为未来研究提供了基准。

Insight: 1. 学生偏好研究为任务设计提供了重要指导；2. 多模态任务需结合文本和定性指标全面评估；3. EduVidQA为教育领域的NLP研究开辟了新方向。

Abstract: As digital platforms redefine educational paradigms, ensuring interactivity remains vital for effective learning. This paper explores using Multimodal Large Language Models (MLLMs) to automatically respond to student questions from online lectures - a novel question answering task of real world significance. We introduce the EduVidQA Dataset with 5252 question-answer pairs (both synthetic and real-world) from 296 computer science videos covering diverse topics and difficulty levels. To understand the needs of the dataset and task evaluation, we empirically study the qualitative preferences of students, which we provide as an important contribution to this line of work. Our benchmarking experiments consist of 6 state-of-the-art MLLMs, through which we study the effectiveness of our synthetic data for finetuning, as well as showing the challenging nature of the task. We evaluate the models using both text-based and qualitative metrics, thus showing a nuanced perspective of the models’ performance, which is paramount to future work. This work not only sets a benchmark for this important problem, but also opens exciting avenues for future research in the field of Natural Language Processing for Education.

[70] Beyond Magic Words: Sharpness-Aware Prompt Evolving for Robust Large Language Models with TARE

Guancheng Wan,Lucheng Fu,Haoxin Liu,Yiqiao Jin,Hui Yi Leong,Eric Hanchen Jiang,Hejia Geng,Jinhe Bi,Yunpu Ma,Xiangru Tang,B. Aditya Prakash,Yizhou Sun,Wei Wang

Main category: cs.CL

TL;DR: 论文提出TARE框架，通过对抗搜索和鲁棒选择优化提示词，减少其在语义空间中的文本锐度，从而提高大型语言模型的鲁棒性。

Details Motivation: 现有提示词优化方法主要关注点对点准确性，忽视了语义不变性和搜索稳定性，导致提示词在小幅度语义保持的改写下性能波动较大。

Contribution: 1. 首次形式化定义了提示词在离散语义空间中的文本锐度；2. 提出了无需梯度更新的黑盒框架TARE；3. 进一步引入ATARE，通过学习各向异性权重动态调整语义邻域。

Method: TARE框架分为基于采样的对抗搜索（内部）和鲁棒选择（外部）。ATARE通过学习权重和动态调整邻域半径实现搜索平衡。

Result: 实验表明，TARE和ATARE能够在多样化任务中生成对改写鲁棒的提示词，性能优于仅优化准确性的方法且计算高效。

Insight: 提示词的鲁棒性优化需结合对抗搜索和动态语义邻域调整，文本锐度的形式化为解决该问题提供了新思路。

Abstract: The performance of Large Language Models (LLMs) hinges on carefully engineered prompts. However, prevailing prompt optimization methods, ranging from heuristic edits and reinforcement learning to evolutionary search, primarily target point-wise accuracy. They seldom enforce paraphrase invariance or searching stability, and therefore cannot remedy this brittleness in practice. Automated prompt search remains brittle: small, semantically preserving paraphrases often cause large performance swings. We identify this brittleness as the textual sharpness of the prompt landscape. In this work, we provide the first formal treatment of textual sharpness in the discrete, semantic space of prompts, together with an operational robustness criterion over a semantic neighborhood; the design is black-box or API-only, requiring no gradients to update the model’s parameters. Then we introduce TARE (Textual Sharpness-Aware Evolving), a derivative-free framework that alternates between an inner, sampling-based adversarial search that stresses a prompt with hard paraphrases and an outer, robust selection that prefers candidates whose neighborhoods remain strong. We further propose ATARE, which learns anisotropic weights to shape the semantic neighborhood and adapts its radius over time to balance exploration and fidelity. Diverse tasks evaluate our methods, whose design for minimizing textual sharpness gap leads to prompts that preserve accuracy under paraphrasing, outperforming accuracy-only prompt search while remaining computationally practical.

[71] Your thoughts tell who you are: Characterize the reasoning patterns of LRMs

Yida Chen,Yuning Mao,Xianjun Yang,Suyu Ge,Shengjie Bi,Lijuan Liu,Saghar Hosseini,Liang Tan,Yixin Nie,Shaoliang Nie

Main category: cs.CL

TL;DR: 该论文提出了一种名为LOT的分类方法，通过生成式语言模型比较不同大推理模型（LRM）的推理轨迹，并用自然语言描述其差异。LOT能够以高精度区分不同LRM的推理风格，并揭示了这些风格对任务性能的影响。

Details Motivation: 目前对大推理模型（LRM）的比较主要基于宏观统计（如任务准确率或推理长度），但不同LRM是否具有不同的推理模式仍是一个未解问题。因此，作者希望填补这一空白。

Contribution: 1. 提出了LLM提出的开放分类法（LOT），通过自然语言比较和描述LRM的推理差异；2. 展示了LOT能够高精度（80-100%）区分不同LRM的推理风格；3. 揭示了推理风格对任务性能的影响（例如通过对齐推理风格提升小模型的性能）。

Method: 1. 使用生成式语言模型分析推理轨迹，生成描述性特征；2. 通过统计建模预测推理轨迹的来源LRM；3. 迭代应用于数据集中，形成人类可读的分类法。

Result: LOT在数学、科学和编程任务上对12种开源LRM的推理进行了比较，发现它们在规模和目标领域上有系统性差异。此外，通过对齐推理风格，小模型在GPQA任务上的准确率提升了3.3-5.7%。

Insight: 1. LRM的推理风格存在显著差异，且这些差异可以量化；2. 推理风格的调整可以提升模型性能，表明推理模式对任务解决具有实际影响。

Abstract: Current comparisons of large reasoning models (LRMs) focus on macro-level statistics such as task accuracy or reasoning length. Whether different LRMs reason differently remains an open question. To address this gap, we introduce the LLM-proposed Open Taxonomy (LOT), a classification method that uses a generative language model to compare reasoning traces from two LRMs and articulate their distinctive features in words. LOT then models how these features predict the source LRM of a reasoning trace based on their empirical distributions across LRM outputs. Iterating this process over a dataset of reasoning traces yields a human-readable taxonomy that characterizes how models think. We apply LOT to compare the reasoning of 12 open-source LRMs on tasks in math, science, and coding. LOT identifies systematic differences in their thoughts, achieving 80-100% accuracy in distinguishing reasoning traces from LRMs that differ in scale, base model family, or objective domain. Beyond classification, LOT’s natural-language taxonomy provides qualitative explanations of how LRMs think differently. Finally, in a case study, we link the reasoning differences to performance: aligning the reasoning style of smaller Qwen3 models with that of the largest Qwen3 during test time improves their accuracy on GPQA by 3.3-5.7%.

[72] Retrieval-augmented GUI Agents with Generative Guidelines

Ran Xu,Kaixin Ma,Wenhao Yu,Hongming Zhang,Joyce C. Ho,Carl Yang,Dong Yu

Main category: cs.CL

TL;DR: RAG-GUI是一种轻量级视觉语言模型（VLM），通过结合检索增强技术和生成指导原则，显著提升了GUI代理在复杂数字任务中的表现。

Details Motivation: 由于训练数据稀缺和复杂任务的罕见场景需求，现有的GUI代理在实际应用中表现受限。RAG-GUI旨在通过检索增强和生成指导解决这些问题。

Contribution: 提出RAG-GUI，首次结合检索增强技术和生成指导原则；通过监督微调和自我引导拒绝采样微调优化模型，具有模型无关性和即插即用能力。

Method: RAG-GUI采用监督微调（SFT）预热模型，并通过自我引导拒绝采样微调（RSF）进一步优化，结合检索增强技术动态利用外部知识。

Result: 在三个任务中，RAG-GUI均优于基线代理，并在两种模型规模下表现提升2.6%到13.3%，展示了强泛化能力和实用性。

Insight: RAG-GUI的成功表明，动态外部知识检索与模型微调的结合能显著提升GUI代理的任务完成能力，为复杂任务的自动化提供了新思路。

Abstract: GUI agents powered by vision-language models (VLMs) show promise in automating complex digital tasks. However, their effectiveness in real-world applications is often limited by scarce training data and the inherent complexity of these tasks, which frequently require long-tailed knowledge covering rare, unseen scenarios. We propose RAG-GUI , a lightweight VLM that leverages web tutorials at inference time. RAG-GUI is first warm-started via supervised finetuning (SFT) and further refined through self-guided rejection sampling finetuning (RSF). Designed to be model-agnostic, RAG-GUI functions as a generic plug-in that enhances any VLM-based agent. Evaluated across three distinct tasks, it consistently outperforms baseline agents and surpasses other inference baselines by 2.6% to 13.3% across two model sizes, demonstrating strong generalization and practical plug-and-play capabilities in real-world scenarios.

[73] Beyond Overall Accuracy: A Psychometric Deep Dive into the Topic-Specific Medical Capabilities of 80 Large Language Models

Zhimeng Luo,Lixin Wu,Adam Frisch,Daqing He

Main category: cs.CL

TL;DR: 该论文提出了一个基于项目反应理论（IRT）的评估框架MedIRT，用于更精确地评估大型语言模型（LLMs）在医学领域的主题特定能力。通过对80个LLMs在1,100道USMLE题目上的评测，揭示了模型能力的多样性和复杂性，并提出了一个实用的决策支持框架。

Details Motivation: 随着LLMs在医疗领域应用的增加，传统的准确性评估方法无法捕捉问题的特性和主题特定的能力差异，亟需一种更可靠的评估方法。

Contribution: 1）提出了基于IRT的MedIRT评估框架；2）揭示了LLMs在医学不同主题下的能力差异；3）提供了实用的决策支持工具。

Method: 使用IRT对80个LLMs在1,100道USMLE题目上的表现进行建模，估计模型能力、题目难度和区分度。

Result: 发现LLMs的能力分布不均匀，GPT-5在多数领域表现最佳，但在某些主题上被Claude-3-opus超越；同时识别出有缺陷的题目。

Insight: 1）总体排名可能掩盖模型在特定领域的优势；2）IRT不仅能评估模型表现，还能用于题目审计；3）主题特异性评估对医疗应用至关重要。

Abstract: As Large Language Models (LLMs) are increasingly proposed for high-stakes medical applications, there has emerged a critical need for reliable and accurate evaluation methodologies. Traditional accuracy metrics fail inadequately as they neither capture question characteristics nor offer topic-specific insights. To address this gap, we introduce \textsc{MedIRT}, a rigorous evaluation framework grounded in Item Response Theory (IRT), the gold standard in high-stakes educational testing. Unlike previous research relying on archival data, we prospectively gathered fresh responses from 80 diverse LLMs on a balanced, 1,100-question USMLE-aligned benchmark. Using one unidimensional two-parameter logistic IRT model per topic, we estimate LLM’s latent model ability jointly with question difficulty and discrimination, yielding more stable and nuanced performance rankings than accuracy alone. Notably, we identify distinctive ``spiky’’ ability profiles, where overall rankings can be misleading due to highly specialized model abilities. While \texttt{GPT-5} was the top performer in a majority of domains (8 of 11), it was outperformed in Social Science and Communication by \texttt{Claude-3-opus}, demonstrating that even an overall 23rd-ranked model can hold the top spot for specific competencies. Furthermore, we demonstrate IRT’s utility in auditing benchmarks by identifying flawed questions. We synthesize these findings into a practical decision-support framework that integrates our multi-factor competency profiles with operational metrics. This work establishes a robust, psychometrically grounded methodology essential for the safe, effective, and trustworthy deployment of LLMs in healthcare.

[74] PET: Preference Evolution Tracking with LLM-Generated Explainable Distribution

Luyang Zhang,Siyuan Peng,Jialu Wang,Shichao Zhu,Beibei Li,Zhongcun Wang,Guangmou Pan,Yan Li,Song Yang

Main category: cs.CL

TL;DR: PET框架通过动态概率分布推断用户偏好演化，提升排名质量40%（NDCG），并在长尾内容排序中显著优于现有生产模型7倍。

Details Motivation: 现代数字生态系统中，用户偏好演化理解是关键挑战。现有LLM直接生成偏好列表的方法缺乏透明性和可解释性，且容易导致流行度偏差。

Contribution: 提出PET框架，将用户偏好建模为动态概率分布，通过可解释的偏好簇格结构实现透明偏好学习。

Method: 结合logit-probing和生成分类技术，推断用户的动态偏好分布，避免直接生成偏好列表的局限性。

Result: 在公开基准（Yelp、MovieLens）上NDCG提升40%，在短视频平台数据集上长尾内容排序表现优于生产模型7倍。

Insight: 通过分布化偏好映射取代直接列表生成，PET实现了更可解释、公平且多样化的个性化系统。

Abstract: Understanding how user preference evolves over time is a fundamental challenge central to modern digital ecosystems, for which Large Language Models (LLMs) are an increasingly prominent and popular approach due to their ability to comprehend the rich semantic context within behavioral data. A common practice is to use LLMs to predict a user’s next action by directly generating a ranked list of preferred items. Although effective for short-term prediction, the end-to-end generation paradigm inherently limits personalization. Its opaque decision-making process obscures holistic user profiling and exacerbates popularity bias. To address these limitations, we propose Preference Evolution Tracking (PET), a framework that reframes the task as inferring a dynamic probability distribution over a stable and interpretable lattice of preference clusters. By applying logit-probing and generative classification techniques, PET infers a user’s preference as a probability distribution, enabling transparent preference learning. On public benchmarks (Yelp, MovieLens), PET improves ranking quality by up to 40% in NDCG over direct generation baselines. On a large-scale, real-world dataset from a short-video platform, it excels at ranking long-tail contents, significantly outperforming a SOTA production model by 7 times in the NDCG score. Ultimately, PET transforms the user profile model from direct preference list generation to a transparent distributional preference mapping, paving the way for more explainable, fair, and diverse personalization systems.

[75] AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforced Self-Play

Ran Xu,Yuchen Zhuang,Zihan Dong,Jonathan Wang,Yue Yu,Joyce C. Ho,Linjun Zhang,Haoyu Wang,Wenqi Shi,Carl Yang

Main category: cs.CL

TL;DR: AceSearcher是一个基于强化自博弈的框架，通过训练单个LLM在分解复杂查询和生成答案两个角色间切换，解决了搜索增强型LLM在多跳检索和推理能力上的不足，显著提升了复杂推理任务的性能。

Details Motivation: 现有搜索增强型LLM在多跳检索和复杂推理任务中表现不佳，主要原因是无效的多跳检索和推理能力有限。AceSearcher旨在通过协同自博弈框架解决这些问题。

Contribution: 提出了AceSearcher框架，结合监督微调和强化微调，无需中间标注即可提升LLM在复杂推理任务中的性能。在多个数据集上的实验显示其显著优于现有方法，且模型效率极高。

Method: AceSearcher训练LLM在两个角色间切换：分解器（分解复杂查询）和求解器（整合检索上下文生成答案），并通过监督和强化微调优化最终答案准确性。

Result: 在三个推理密集型任务的10个数据集中，AceSearcher平均精确匹配率提升7.6%。较小的模型（1.5B和8B）也能超越参数多9倍的现有搜索增强型LLM。

Insight: AceSearcher展示了自博弈和角色交替在提升LLM复杂推理能力中的潜力，同时证明了小模型通过高效设计也能胜任大模型的任务。

Abstract: Search-augmented LLMs often struggle with complex reasoning tasks due to ineffective multi-hop retrieval and limited reasoning ability. We propose AceSearcher, a cooperative self-play framework that trains a single large language model (LLM) to alternate between two roles: a decomposer that breaks down complex queries and a solver that integrates retrieved contexts for answer generation. AceSearcher couples supervised fine-tuning on a diverse mixture of search, reasoning, and decomposition tasks with reinforcement fine-tuning optimized for final answer accuracy, eliminating the need for intermediate annotations. Extensive experiments on three reasoning-intensive tasks across 10 datasets show that AceSearcher outperforms state-of-the-art baselines, achieving an average exact match improvement of 7.6%. Remarkably, on document-level finance reasoning tasks, AceSearcher-32B matches the performance of the DeepSeek-V3 model using less than 5% of its parameters. Even at smaller scales (1.5B and 8B), AceSearcher often surpasses existing search-augmented LLMs with up to 9x more parameters, highlighting its exceptional efficiency and effectiveness in tackling complex reasoning tasks. Our code will be published at https://github.com/ritaranx/AceSearcher and https://huggingface.co/AceSearcher.

[76] Can Large Language Models Express Uncertainty Like Human?

Linwei Tao,Yi-Fan Yeh,Bo Kai,Minjing Dong,Tao Huang,Tom A. Lamb,Jialin Yu,Philip H. S. Torr,Chang Xu

Main category: cs.CL

TL;DR: 论文探讨大型语言模型(LLM)如何通过语言表达不确定性(如可能、或许)，提出了一种轻量级且符合人类习惯的方法，并发布首个大规模数据集和改进框架。

Details Motivation: 在高风险场景中，LLM的过度自信可能误导用户。现有方法(如对数概率或多采样)存在计算成本高或不符合自然语言习惯的问题，需要更高效且贴近人类的解决方案。

Contribution: 1. 发布首个多样性的大规模语言不确定性标注数据集；2. 提出轻量级映射器，低成本将语言不确定性转化为置信分数；3. 首次系统研究LLM的语言不确定性表达能力；4. 提出微调框架以提升可靠性。

Method: 通过提示工程(prompting)和微调(fine-tuning)优化LLM的语言不确定性表达，并使用映射器将其量化为置信分数。

Result: 研究表明，多数LLM在语言不确定性表达上表现不佳，但经过设计的提示和微调可显著提升校准性和区分能力。

Insight: 语言不确定性表达是一种轻量化、高效且符合人类习惯的LLM不确定性评估方法，值得进一步探索。

Abstract: Large language models (LLMs) are increasingly used in high-stakes settings, where overconfident responses can mislead users. Reliable confidence estimation has been shown to enhance trust and task accuracy. Yet existing methods face practical barriers: logits are often hidden, multi-sampling is computationally expensive, and verbalized numerical uncertainty (e.g., giving a 0-100 score) deviates from natural communication. We revisit linguistic confidence (LC), where models express uncertainty through hedging language (e.g., probably, might), offering a lightweight and human-centered alternative. To advance this direction, we (1) release the first diverse, large-scale dataset of hedging expressions with human-annotated confidence scores, and (2) propose a lightweight mapper that converts hedges into confidence scores at near-zero cost. Building on these resources, we (3) conduct the first systematic study of LC across modern LLMs and QA benchmarks, revealing that while most LLMs underperform in expressing reliable LC, carefully designed prompting achieves competitive calibration and discriminability. Finally, we (4) introduce a fine-tuning framework that further improves LC reliability. Taken together, our work positions linguistic confidence as a scalable, efficient, and human-aligned approach to LLM uncertainty estimation, and calls for deeper exploration of this promising yet underexplored direction.

[77] BeyondBench: Benchmark-Free Evaluation of Reasoning in Language Models

Gaurav Srivastava,Aafiya Hussain,Zhenyu Bi,Swastik Roy,Priya Pitre,Meng Lu,Morteza Ziyadi,Xuan Wang

Main category: cs.CL

TL;DR: BeyondBench 是一个不使用静态基准测试的语言模型评估框架，通过动态生成基于算法的任务来解决训练数据污染问题，确保公平评估模型的推理能力。

Details Motivation: 传统基准测试因训练数据污染问题，难以区分模型的推理能力与记忆能力。

Contribution: 提出 BeyondBench 框架，动态生成数学任务以避免数据污染问题，覆盖 44 个算法任务和 117 种变体。

Method: 基于算法动态生成问题，问题空间超过 10^15 种实例，并通过数学证明验证解决方案。

Result: 评估 101 个语言模型，显示在复杂任务上推理能力显著下降，大型模型在 Hard Suite 中表现差异显著。

Insight: 模型在多项式到指数复杂度任务上的性能急剧下降，工具使用对性能有显著影响。

Abstract: Evaluating language models fairly is becoming harder as static benchmarks available on the internet risk contamination by training data. This makes it unclear whether models are truly reasoning or just recalling answers. In this paper, we introduce BeyondBench, an evaluation framework that avoids this problem by using algorithmic problem generation. Unlike traditional benchmarks that risk contamination from internet-scale training data, BeyondBench creates mathematically grounded problems on the fly, ensuring each test remains fresh and uncontaminated. Our framework covers 44 algorithmic tasks with a total of 117 variations, grouped into three difficulty levels: the Easy Suite (29 tasks) for basic arithmetic and statistics, the Medium Suite (5 tasks, 49 variations) for sequence patterns and reasoning, and the Hard Suite (10 tasks, 68 variations) tackling NP-complete and constraint satisfaction problems. Each task generates problems from a combinatorial space larger than 10^15 unique instances, with solutions verified deterministically by mathematical proofs. We evaluated 101 language models, including 85 open-source and 16 closed-source models, spanning sizes from 0.5B to 141B parameters and multiple quantization schemes. Our results show consistent reasoning deficiencies across model families, with performance degrading sharply as problem complexity increases from polynomial to exponential. In our Hard Suite evaluations, models such as Gemini-2.5-pro, Llama-3.3-70B, and Qwen2.5-72B achieved average accuracies of 56.38%, 26.91%, and 33.60%, respectively. Moreover, we observe that performance drops drastically without tool usage, with GPT-5, GPT-5-mini, and GPT-5-nano showing a decline of 16.81%, 28.05%, and 47.59% accuracy on the hard suite. Our leaderboard is publicly available at https://ctrl-gaurav.github.io/BeyondBench/

[78] MRAG-Suite: A Diagnostic Evaluation Platform for Visual Retrieval-Augmented Generation

Yuelyu Ji

Main category: cs.CL

TL;DR: MRAG-Suite是一个诊断性评估平台，专注于解决视觉检索增强生成（Visual RAG）中查询难度和模糊性的系统评估问题。

Details Motivation: 当前的多模态检索增强生成系统在评估中缺乏对查询难度和模糊性的系统性考量，亟需一个更全面的诊断工具。

Contribution: 提出了MRAG-Suite平台，整合了多种多模态基准测试，并引入了基于难度和模糊性的过滤策略及MM-RAGChecker诊断工具。

Method: 通过整合WebQA、Chart-RAG等基准数据集，结合难度和模糊性过滤策略，利用MM-RAGChecker进行逐项诊断。

Result: 结果显示，在困难和模糊查询下，准确率显著下降，且MM-RAGChecker能有效诊断出幻觉问题。

Insight: 系统揭示了Visual RAG在复杂场景中的局限性，并为未来改进提供了方向。

Abstract: Multimodal Retrieval-Augmented Generation (Visual RAG) significantly advances question answering by integrating visual and textual evidence. Yet, current evaluations fail to systematically account for query difficulty and ambiguity. We propose MRAG-Suite, a diagnostic evaluation platform integrating diverse multimodal benchmarks (WebQA, Chart-RAG, Visual-RAG, MRAG-Bench). We introduce difficulty-based and ambiguity-aware filtering strategies, alongside MM-RAGChecker, a claim-level diagnostic tool. Our results demonstrate substantial accuracy reductions under difficult and ambiguous queries, highlighting prevalent hallucinations. MM-RAGChecker effectively diagnoses these issues, guiding future improvements in Visual RAG systems.

[79] LOGOS: LLM-driven End-to-End Grounded Theory Development and Schema Induction for Qualitative Research

Xinyu Pi,Qisen Yang,Chuong Nguyen

Main category: cs.CL

TL;DR: LOGOS是一个端到端框架，通过LLM驱动的编码、语义聚类、图推理和迭代优化过程，实现了扎根理论的全面自动化，显著提升了定性研究的可扩展性和效率。

Details Motivation: 扎根理论虽然能从定性数据中获得深刻见解，但其依赖专家密集型的手工编码导致难以规模化。现有工具无法实现完全自动化，LOGOS旨在解决这一问题。

Contribution: LOGOS首次实现了扎根理论的端到端自动化，提出了LLM驱动的编码方法、语义聚类和图推理技术，以及5维评价指标和标准化的训练-测试分割协议。

Method: LOGOS结合LLM驱动的编码、语义聚类和图推理，通过迭代优化过程生成结构化、分层次的理论模式。

Result: 在五个不同数据集上，LOGOS优于现有基线方法，并在复杂数据集上与专家开发的模式对齐率达到88.2%。

Insight: LOGOS展示了通过自动化技术在不牺牲理论深度的前提下，如何实现定性研究的规模化和民主化。

Abstract: Grounded theory offers deep insights from qualitative data, but its reliance on expert-intensive manual coding presents a major scalability bottleneck. Current computational tools stop short of true automation, keeping researchers firmly in the loop. We introduce LOGOS, a novel, end-to-end framework that fully automates the grounded theory workflow, transforming raw text into a structured, hierarchical theory. LOGOS integrates LLM-driven coding, semantic clustering, graph reasoning, and a novel iterative refinement process to build highly reusable codebooks. To ensure fair comparison, we also introduce a principled 5-dimensional metric and a train-test split protocol for standardized, unbiased evaluation. Across five diverse corpora, LOGOS consistently outperforms strong baselines and achieves a remarkable $88.2%$ alignment with an expert-developed schema on a complex dataset. LOGOS demonstrates a powerful new path to democratize and scale qualitative research without sacrificing theoretical nuance.

[80] DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Models

Zherui Li,Zheng Nie,Zhenhong Zhou,Yufei Guo,Yue Liu,Yitong Zhang,Yu Cheng,Qingsong Wen,Kun Wang,Jiaheng Zhang

Main category: cs.CL

TL;DR: DiffuGuard提出了一种针对扩散大语言模型（dLLMs）的安全防御框架，通过动态随机重掩码和块级审计修复机制，有效降低攻击成功率，同时保持模型性能。

Details Motivation: 扩散大语言模型因其迭代和平行生成机制，存在独特的漏洞，需要一种针对性的防御方法以提升其安全性。

Contribution: 1. 揭示了标准贪心重掩码策略的偏差和去噪路径依赖性现象；2. 提出了DiffuGuard框架，动态引入随机性和块级审计修复机制。

Method: 采用双阶段方法：1. 随机退火重掩码（Stochastic Annealing Remasking）减轻贪心选择偏差；2. 块级审计修复（Block-level Audit and Repair）自主检测和修正风险。

Result: 在四种dLLMs上测试，DiffuGuard将六种攻击方法的成功率从47.9%降至14.7%，且不影响模型效率和实用性。

Insight: dLLMs具备潜在的安全能力，但需要通过解码策略的优化和动态修复机制来激活。

Abstract: The rapid advancement of Diffusion Large Language Models (dLLMs) introduces unprecedented vulnerabilities that are fundamentally distinct from Autoregressive LLMs, stemming from their iterative and parallel generation mechanisms. In this paper, we conduct an in-depth analysis of dLLM vulnerabilities to jailbreak attacks across two distinct dimensions: intra-step and inter-step dynamics. Experimental results reveal a harmful bias inherent in the standard greedy remasking strategy and identify a critical phenomenon we term Denoising-path Dependence, where the safety of early-stage tokens decisively influences the final output. These findings also indicate that while current decoding strategies constitute a significant vulnerability, dLLMs possess a substantial intrinsic safety potential. To unlock this potential, we propose DiffuGuard, a training-free defense framework that addresses vulnerabilities through a dual-stage approach: Stochastic Annealing Remasking dynamically introduces controlled randomness to mitigate greedy selection bias, while Block-level Audit and Repair exploits internal model representations for autonomous risk detection and guided correction. Comprehensive experiments on four dLLMs demonstrate DiffuGuard’s exceptional effectiveness, reducing Attack Success Rate against six diverse jailbreak methods from 47.9% to 14.7% while preserving model utility and efficiency. Our code is available at: https://github.com/niez233/DiffuGuard.

[81] Q-Mirror: Unlocking the Multi-Modal Potential of Scientific Text-Only QA Pairs

Junying Wang,Zicheng Zhang,Ye Shen,Yalun Wu,Yingji Liang,Yijin Guo,Farong Wen,Wenzhe Li,Xuezhi Zhao,Qi Jia,Guangtao Zhai

Main category: cs.CL

TL;DR: Q-Mirror提出了一种将纯文本问答对(TQA)转化为高质量多模态问答对(MMQA)的框架，并通过闭环迭代优化提升生成质量，为解决科学领域多模态基准构建的成本和规模问题提供了可行方案。

Details Motivation: 高质量多模态基准对科学推理模型的进步至关重要，但手动构建成本高且难以扩展。因此，探索如何将现有纯文本问答对转化为多模态问答对具有实际意义。

Contribution: 1) 提出了TQA-to-MMQA的转换框架和评估标准；2) 构建了两个大规模基准测试；3) 开发了闭环代理系统Q-Mirror，显著提升了MMQA的生成质量和评估效率。

Method: 1) 定义转换任务和评估标准；2) 构建多模态基准；3) 使用Q-Mirror系统，集成生成与评估模块，通过闭环迭代优化方法提升质量。

Result: 实验表明，Q-Mirror将平均分从78.90提升到85.22，通过率从72%提升到95%，且顶级理解模型在多模态评估中与人工判断高度一致。

Insight: 1) 多模态生成的质量仍需提升；2) 闭环迭代优化是提高生成质量的有效方法；3) 高质量评估标准对生成结果的可靠性至关重要。

Abstract: High-quality, multi-modal benchmarks are crucial for advancing scientific reasoning in large models yet their manual creation is costly and unscalable. To address this bottleneck, we explore the potential for transforming Text-Only QA Pairs (TQAs) into high-quality Multi-Modal QA Pairs (MMQAs), which include three parts: 1) Task Definition & Evaluation Rubric: We develop a TQA-to-MMQA framework and establish a comprehensive, multi-dimensional MMQA quality rubric that provides principles for the transformation. 2) Benchmark Construction: Then we construct two extensive benchmarks to rigorously evaluate state-of-the-art generation & understanding models on the distinct tasks of MMQA generation & MMQA quality evaluation. 3) Preliminary Solution: We develop an agentic system (Q-Mirror), which operationalizes our framework by integrating MMQA generation and evaluation into a closed loop for iterative refinement. Our experiments show that while state-of-the-art models can generate MMQAs, their outputs still leave substantial gaps, underscoring the need for reliable evaluation. We further demonstrate that top-tier understanding models align closely with human judgment in MMQA quality assessment. Leveraging both insights, the Q-Mirror agent raises average scores from 78.90 to 85.22 and pass rates from 72% to 95%, offering a practical path to large-scale scientific benchmarks.

[82] Multimodal Large Language Models Meet Multimodal Emotion Recognition and Reasoning: A Survey

Yuntao Shou,Tao Meng,Wei Ai,Keqin Li

Main category: cs.CL

TL;DR: 这篇综述文章总结了多模态大语言模型（MLLMs）在多模态情感识别与推理领域的最新进展，包括模型架构、数据集和性能基准，并指出了关键挑战和未来研究方向。

Details Motivation: 随着对更高级语义和跨模态融合需求的增加，MLLMs的兴起为复杂场景中的情感识别与推理提供了新的可能性。然而，该领域缺乏系统性综述，本文旨在填补这一空白。

Contribution: 本文首次全面综述了MLLMs在多模态情感识别与推理中的应用，整理了模型架构、数据集和性能基准，并总结了关键挑战和未来方向。

Method: 论文采用了文献综述的方法，系统地整理了相关研究成果，并分类总结了模型架构、数据集和性能指标。

Result: 文章总结了MLLMs在情感识别与推理中的表现，指出了现有方法的优势和局限。

Insight: MLLMs在该领域展现出巨大潜力，但仍需解决数据稀缺性和多模态对齐等挑战。

Abstract: In recent years, large language models (LLMs) have driven major advances in language understanding, marking a significant step toward artificial general intelligence (AGI). With increasing demands for higher-level semantics and cross-modal fusion, multimodal large language models (MLLMs) have emerged, integrating diverse information sources (e.g., text, vision, and audio) to enhance modeling and reasoning in complex scenarios. In AI for Science, multimodal emotion recognition and reasoning has become a rapidly growing frontier. While LLMs and MLLMs have achieved notable progress in this area, the field still lacks a systematic review that consolidates recent developments. To address this gap, this paper provides a comprehensive survey of LLMs and MLLMs for emotion recognition and reasoning, covering model architectures, datasets, and performance benchmarks. We further highlight key challenges and outline future research directions, aiming to offer researchers both an authoritative reference and practical insights for advancing this domain. To the best of our knowledge, this paper is the first attempt to comprehensively survey the intersection of MLLMs with multimodal emotion recognition and reasoning. The summary of existing methods mentioned is in our Github: \href{https://github.com/yuntaoshou/Awesome-Emotion-Reasoning}{https://github.com/yuntaoshou/Awesome-Emotion-Reasoning}.

[83] Beyond Repetition: Text Simplification and Curriculum Learning for Data-Constrained Pretraining

Matthew Theodore Roque,Dan John Velasco

Main category: cs.CL

TL;DR: 论文研究了在数据受限环境下，通过文本简化和课程学习优化语言模型预训练的效果，发现简化文本和复杂度排序能提升模型性能。

Details Motivation: 探索在小规模数据集上如何通过数据增强（如文本简化）和课程学习（数据排序）提升语言模型的预训练效果。

Contribution: 1) 比较简化文本与重复使用原始数据的效果；2) 验证文本复杂度排序对表示质量的影响；3) 提出四种数据调度策略并分析其对不同规模模型的适用性。

Method: 1) 构建并行语料库（原始文本与LLM简化版）；2) 设计四种数据调度策略（重复暴露、低到高复杂度、高到低、交替）；3) 通过微调和零样本任务评估模型表示质量。

Result: 简化数据能提升微调和零样本性能：小模型受益于低到高复杂度排序，大模型在交替排序下表现更佳。

Insight: 1) 数据增强对小数据集预训练有益；2) 模型规模影响最优课程学习策略的选择；3) 简化文本可作为一种高效的数据扩充方法。

Abstract: Most studies on language model pretraining focus on large datasets, leaving open questions about optimization in data-constrained settings. In such settings, the effects of training data order and of including alternative versions of the same text remain underexplored. We address this by studying curriculum learning in pretraining, focusing on text-complexity ordering and data augmentation via simplification. We ask: (1) Does simplifying texts enhance representation quality more than reusing the original data? and (2) Does ordering data by text complexity yield better representations? To answer, we build on a pair of parallel corpora where human-written paragraphs are aligned with LLM-simplified variants, and test four data schedules: repeated exposure, low-to-high complexity, high-to-low, and interleaved. We analyze models’ representation quality from a sample efficiency perspective via fine-tuning, as well as its zero-shot performance on linguistic knowledge, entity tracking, world knowledge, and commonsense reasoning. Our findings show that adding simplified data improves fine-tuning and zero-shot performance over a repeated-exposure baseline: smaller models benefit from low-to-high complexity, while larger models perform better with interleaved ordering.

[84] Reinforcement Mid-Training

Yijun Tian,Shaoyu Chen,Zhichao Xu,Yawei Wang,Jinhe Bi,Peng Han,Wei Wang

Main category: cs.CL

TL;DR: 论文提出了一种新的中间训练阶段——强化中间训练（RMT），解决了大型语言模型中推理步骤冗余、标记熵分布不平衡和标记信息利用不足的问题。

Details Motivation: 现有的大型语言模型训练分为预训练和后训练两阶段，但忽略了中间阶段的潜力，导致推理效率低和性能瓶颈。

Contribution: 1. 提出了RMT框架；2. 设计了动态标记预算机制；3. 开发了基于课程的适应性采样方法；4. 提出了双重训练策略。

Method: 1. 动态标记预算限制冗余推理；2. 课程适应性采样渐进学习标记；3. 双重训练结合强化学习和下一标记预测。

Result: RMT在语言建模中性能提升64.91%，推理长度减少79%，并在数学领域后训练中提升18.76%。

Insight: 强化中间训练是提升模型效率和性能的关键阶段，动态预算和适应性采样可有效优化训练过程。

Abstract: The development of state-of-the-art large language models is commonly understood as a two-stage process involving pre-training and post-training. We point out the need for an additional intermediate stage called reinforcement mid-training with potential for strong performance gains. In this paper, we formally define the problem and identify three key challenges: (1) inefficient training due to excessive reasoning steps, (2) disregard of the imbalanced token entropy distribution, and (3) underutilization of token information. To address these challenges, we propose RMT, a framework for efficient, adaptive, and unified reinforcement mid-training with various innovative components. In particular, we first introduce a dynamic token budget mechanism that constrains unnecessary reasoning steps and mitigates model overthinking. Next, we design a curriculum-based adaptive sampling method that fosters a progressive learning trajectory from easy to hard tokens. Finally, we present a dual training strategy that combines reinforcement learning with next-token prediction, ensuring targeted learning on key tokens and full exploitation of all token information. Extensive experiments demonstrate the superiority of RMT over state-of-the-art methods, achieving up to +64.91% performance improvement with only 21% of the reasoning length in language modeling. We also show that checkpoints obtained after reinforcement mid-training can benefit the subsequent post-training, yielding up to +18.76% improvement in the mathematical domain.

[85] LLaDA-MoE: A Sparse MoE Diffusion Language Model

Fengqi Zhu,Zebin You,Yipeng Xing,Zenan Huang,Lin Liu,Yihong Zhuang,Guoshan Lu,Kangyu Wang,Xudong Wang,Lanning Wei,Hongrui Guo,Jiaqi Hu,Wentao Ye,Tieyuan Chen,Chenchen Li,Chengfu Tang,Haibo Feng,Jun Hu,Jun Zhou,Xiaolu Zhang,Zhenzhong Lan,Junbo Zhao,Da Zheng,Chongxuan Li,Jianguo Li,Ji-Rong Wen

Main category: cs.CL

TL;DR: LLaDA-MoE是一种基于稀疏Mixture-of-Experts（MoE）架构的大型语言扩散模型，训练了约20T tokens。它在推理阶段仅激活1.4B参数，保持7B参数容量，取得了与更大参数模型竞争的性能。

Details Motivation: 现有扩散语言模型通常需要大量计算资源，研究者希望通过稀疏MoE架构降低计算开销，同时保持高性能。

Contribution: 主要贡献是提出了LLaDA-MoE，验证了稀疏MoE架构在扩散语言模型中的有效性，并在推理阶段显著减少了激活参数的数量。

Method: 采用Mixture-of-Experts架构，训练了一个7B参数的模型，推理时仅激活1.4B参数。

Result: LLaDA-MoE在多个基准测试中超越了之前的扩散语言模型（如LLaDA、Dream等），并展示了与Qwen2.5-3B-Instruct相当的能力。

Insight: 研究表明，稀疏MoE架构能够在减少计算开销的同时保持高性能，这为扩散语言模型的进一步探索提供了空间。

Abstract: We introduce LLaDA-MoE, a large language diffusion model with the Mixture-of-Experts (MoE) architecture, trained from scratch on approximately 20T tokens. LLaDA-MoE achieves competitive performance with significantly reduced computational overhead by maintaining a 7B-parameter capacity while activating only 1.4B parameters during inference. Our empirical evaluation reveals that LLaDA-MoE achieves state-of-the-art performance among diffusion language models with larger parameters, surpassing previous diffusion language models LLaDA, LLaDA 1.5, and Dream across multiple benchmarks. The instruct-tuned model LLaDA-MoE-7B-A1B-Instruct demonstrates capabilities comparable to Qwen2.5-3B-Instruct in knowledge understanding, code generation, mathematical reasoning, agent and alignment tasks, despite using fewer active parameters. Our results show that integrating a sparse MoE architecture into the training objective of masked diffusion language models still brings out MoE’s strengths under efficient inference with few active parameters, and opens ample room for further exploration of diffusion language models. LLaDA-MoE models are available at Huggingface.

[86] Agentar-Scale-SQL: Advancing Text-to-SQL through Orchestrated Test-Time Scaling

Pengfei Wang,Baolin Sun,Xuemei Dong,Yaxun Dai,Hongwei Yuan,Mengdie Chu,Yingqi Gao,Xiang Qi,Peng Zhang,Ying Yan

Main category: cs.CL

TL;DR: Agentar-Scale-SQL通过协调测试时扩展（Orchestrated Test-Time Scaling），在多视角协同下提升Text-to-SQL性能，达到了BIRD基准的最优结果。

Details Motivation: 现有Text-to-SQL方法在复杂基准（如BIRD）上远不及人类专家，且测试时扩展缺乏协调策略，忽略了模型的内部推理过程。

Contribution: 提出了Agentar-Scale-SQL框架，结合内部扩展、顺序扩展和并行扩展的多视角协同策略，显著提升了Text-to-SQL的性能。

Method: 采用三种扩展策略：1）基于强化学习的内部推理扩展（Internal Scaling），2）迭代优化的顺序扩展（Sequential Scaling），3）多样合成与锦标赛选择的并行扩展（Parallel Scaling）。

Result: 在BIRD测试集上达到81.67%的执行准确率，排名第一，接近人类专家水平。

Insight: 协调多视角扩展策略是提升Text-to-SQL性能的有效路径，且框架易于适配新数据库和更强语言模型。

Abstract: State-of-the-art (SOTA) Text-to-SQL methods still lag significantly behind human experts on challenging benchmarks like BIRD. Current approaches that explore test-time scaling lack an orchestrated strategy and neglect the model’s internal reasoning process. To bridge this gap, we introduce Agentar-Scale-SQL, a novel framework leveraging scalable computation to improve performance. Agentar-Scale-SQL implements an Orchestrated Test-Time Scaling strategy that synergistically combines three distinct perspectives: i) Internal Scaling via RL-enhanced Intrinsic Reasoning, ii) Sequential Scaling through Iterative Refinement, and iii) Parallel Scaling using Diverse Synthesis and Tournament Selection. Agentar-Scale-SQL is a general-purpose framework designed for easy adaptation to new databases and more powerful language models. Extensive experiments show that Agentar-Scale-SQL achieves SOTA performance on the BIRD benchmark, reaching 81.67% execution accuracy on the test set and ranking first on the official leaderboard, demonstrating an effective path toward human-level performance.

[87] Multilingual Text-to-SQL: Benchmarking the Limits of Language Models with Collaborative Language Agents

Khanh Trinh Pham,Thu Huong Nguyen,Jun Jo,Quoc Viet Hung Nguyen,Thanh Tam Nguyen

Main category: cs.CL

TL;DR: 论文提出了多语言Text-to-SQL基准MultiSpider 2.0，扩展了Spider 2.0到8种语言，揭示了当前语言模型在多语言任务上的局限性，并提出协作式语言代理方法提升性能。

Details Motivation: 现有Text-to-SQL基准多为英语，限制了多语言进展。扩展多语言任务有助于推动跨语言系统的开发。

Contribution: 1）推出MultiSpider 2.0基准，覆盖8种语言；2）揭示LLMs在多语言任务上的性能差距；3）提出协作式语言代理方法提升SQL生成准确性。

Method: 通过扩展Spider 2.0数据集，构建多语言基准；采用协作式语言代理迭代优化SQL查询。

Result: 主流LLMs在多语言任务上执行准确率仅4%（单语言60%），协作方法将其提升至15%。

Insight: 多语言Text-to-SQL任务仍需更多研究，当前模型对语言多样性适应性不足，协作方法展现了改进潜力。

Abstract: Text-to-SQL enables natural access to databases, yet most benchmarks are English-only, limiting multilingual progress. We introduce MultiSpider 2.0, extending Spider 2.0 to eight languages (English, German, French, Spanish, Portuguese, Japanese, Chinese, Vietnamese). It preserves Spider 2.0’s structural difficulty while adding linguistic and dialectal variability, demanding deeper reasoning for complex SQL. On this benchmark, state-of-the-art LLMs (such as DeepSeek-R1 and OpenAI o1) reach only 4% execution accuracy when relying on intrinsic reasoning, versus 60% on MultiSpider 1.0. Therefore, we provide a collaboration-driven language agents baseline that iteratively refines queries, improving accuracy to 15%. These results reveal a substantial multilingual gap and motivate methods that are robust across languages and ready for real-world enterprise deployment. Our benchmark is available at https://github.com/phkhanhtrinh23/Multilingual_Text_to_SQL.

[88] CDT: A Comprehensive Capability Framework for Large Language Models Across Cognition, Domain, and Task

Haosi Mo,Xinyu Ma,Xuebo Liu,Derek F. Wong,Yu Li,Jie Liu,Min Zhang

Main category: cs.CL

TL;DR: 论文提出了一个全面的评估框架CDT（Cognition-Domain-Task），用于衡量大语言模型在认知、领域和任务三个维度的能力，弥补了现有基准的不足。

Details Motivation: 现有的大语言模型评测基准通常关注孤立的能力，缺乏一个全面的评估框架。因此，作者提出了CDT框架，以更全面地评估模型能力。

Contribution: 提出了CDT框架，结合Cattell-Horn-Carroll认知理论扩展了模型能力的定义，应用于数据集能力评估和数据选择。

Method: CDT框架通过三个维度（认知、领域和任务）全面评估模型能力，并基于此进行数据集分析和数据选择。

Result: 实验表明，CDT的能力指标与下游性能高度相关，且在数据选择任务中显著提升了基准分数（分别提高了1.6和2.2分）。

Insight: CDT框架不仅提供了更全面的模型能力评估方式，还能有效指导数据集的分析和构建。

Abstract: Recent advances in Large Language Models (LLMs) have significantly enhanced their capabilities, highlighting the need for comprehensive evaluation frameworks that extend beyond task-specific benchmarks. However, existing benchmarks often focus on isolated abilities, lacking a holistic framework for assessing LLM capabilities. To address this gap, we propose the Cognition-Domain-Task (CDT) framework, which comprehensively measures a model’s capabilities across three dimensions. We expand the scope of model capability definitions at the cognitive level by incorporating the Cattell-Horn-Carroll cognitive theory, refining the categorization of model capabilities. We apply CDT in two directions: dataset capability evaluation and data selection. Experiments show that our capability metrics correlate well with downstream performance and can support effective dataset analysis and construction. The experiments on data selection also show significant improvements in both general and specific benchmarks, achieving scores of 44.3 and 45.4, with an increase of 1.6 and 2.2 points over the baselines, respectively. These results validate the effectiveness and practicality of CDT. Source code and models are available at https://github.com/Alessa-mo/CDT.

[89] Alternatives To Next Token Prediction In Text Generation – A Survey

Charlie Wyatt,Aditya Joshi,Flora Salim

Main category: cs.CL

TL;DR: 这篇综述探讨了替代Next Token Prediction（NTP）的文本生成方法，将其分为五类：多标记预测、计划后生成、潜在推理、连续生成方法及非Transformer架构，旨在解决NTP的局限性。

Details Motivation: NTP范式虽然在大型语言模型中取得成功，但也导致长期规划不足、错误累积和计算效率低下等问题，因此需要探索替代方法。

Contribution: 提出了五种替代NTP的方法分类，并提供了研究指南以解决NTP的局限性。

Method: 将替代方法系统分为五类：多标记预测、计划后生成、潜在推理、连续生成方法及非Transformer架构。

Result: 综述了现有研究并提出分类法，为开发更具变革性的自然语言处理模型提供方向。

Insight: 替代NTP的方法可能在长期规划和计算效率方面优于传统方法，但仍需进一步研究验证。

Abstract: The paradigm of Next Token Prediction (NTP) has driven the unprecedented success of Large Language Models (LLMs), but is also the source of their most persistent weaknesses such as poor long-term planning, error accumulation, and computational inefficiency. Acknowledging the growing interest in exploring alternatives to NTP, the survey describes the emerging ecosystem of alternatives to NTP. We categorise these approaches into five main families: (1) Multi-Token Prediction, which targets a block of future tokens instead of a single one; (2) Plan-then-Generate, where a global, high-level plan is created upfront to guide token-level decoding; (3) Latent Reasoning, which shifts the autoregressive process itself into a continuous latent space; (4) Continuous Generation Approaches, which replace sequential generation with iterative, parallel refinement through diffusion, flow matching, or energy-based methods; and (5) Non-Transformer Architectures, which sidestep NTP through their inherent model structure. By synthesizing insights across these methods, this survey offers a taxonomy to guide research into models that address the known limitations of token-level generation to develop new transformative models for natural language processing.

[90] GRPO-MA: Multi-Answer Generation in GRPO for Stable and Efficient Chain-of-Thought Training

Hongcheng Wang,Yinuo Huang,Sukai Wang,Guanghui Ren,Hao Dong

Main category: cs.CL

TL;DR: GRPO-MA改进了GRPO算法，通过多答案生成解决了梯度耦合、稀疏奖励信号和优势估计不稳定三大问题，提升了训练稳定性和效率。

Details Motivation: GRPO在训练Chain-of-Thought推理时存在梯度耦合、稀疏奖励信号和不稳定优势估计的问题，限制了模型性能和训练效率。

Contribution: 提出了GRPO-MA，利用多答案生成降低优势估计方差，提升训练稳定性和效率，并通过理论分析和实验验证其有效性。

Method: GRPO-MA通过在每次思维过程中生成多个答案，减少梯度耦合和稀疏奖励问题，同时稳定优势估计。

Result: 在数学、代码和多模态任务上，GRPO-MA显著提升性能与训练效率，且增加答案数量能持续改进模型表现。

Insight: 多答案生成能有效缓解RL训练中的不稳定问题，且理论支持其在降低方差方面的优势。

Abstract: Recent progress, such as DeepSeek-R1, has shown that the GRPO algorithm, a Reinforcement Learning (RL) approach, can effectively train Chain-of-Thought (CoT) reasoning in Large Language Models (LLMs) and Vision-Language Models (VLMs). In this paper, we analyze three challenges of GRPO: gradient coupling between thoughts and answers, sparse reward signals caused by limited parallel sampling, and unstable advantage estimation. To mitigate these challenges, we propose GRPO-MA, a simple yet theoretically grounded method that leverages multi-answer generation from each thought process, enabling more robust and efficient optimization. Theoretically, we show that the variance of thought advantage decreases as the number of answers per thought increases. Empirically, our gradient analysis confirms this effect, showing that GRPO-MA reduces gradient spikes compared to GRPO. Experiments on math, code, and diverse multimodal tasks demonstrate that GRPO-MA substantially improves performance and training efficiency. Our ablation studies further reveal that increasing the number of answers per thought consistently enhances model performance.

[91] AdaThink-Med: Medical Adaptive Thinking with Uncertainty-Guided Length Calibration

Shaohao Rui,Kaitao Chen,Weijie Ma,Xiaosong Wang

Main category: cs.CL

TL;DR: AdaThink-Med是一个端到端框架，旨在通过不确定性引导的长度校准，增强医疗推理模型的自适应思维能力，动态调整思考长度以平衡性能和计算成本。

Details Motivation: 现有医疗LLMs在处理简单和复杂问题时均采用冗长的推理过程，增加了实际应用中的推理成本。需要一种自适应方法，动态调整思考长度以提高效率。

Contribution: 提出首个端到端框架AdaThink-Med，通过不确定性引导的长度校准模块，动态调整推理长度，显著减少冗余计算，同时保持性能。

Method: 1) 生成多个候选输出；2) 评估每个输出的正确性和不确定性；3) 通过不确定性校准模块估计问题难度，动态调整推理长度。

Result: 在六个医疗QA基准测试中，平均减少6.4倍推理长度，性能仅轻微下降，并自发形成“非思考”和“思考”两种推理模式。

Insight: 不确定性是动态调整推理长度的有效指标；自适应思维能显著提升效率，同时保持模型性能。

Abstract: Recent advances in inference time scaling with extended long chain-of thought have significantly improved the reasoning capabilities of both general and medical large language models (LLMs). However, these models tend to engage in lengthy reasoning processes regardless of the difficulty of the input question, leading to increased inference costs in real-world applications. Therefore, enabling adaptive thinking where models think less for simpler questions and think more for complex ones is critical for the effective use of medical LLMs in practice. Despite its importance, there is a lack of end-to-end approaches designed to enhance the adaptive thinking capabilities of medical LLMs while providing a comprehensive examination of the trade-off between performance and computational cost. To bridge this gap, we propose AdaThink-Med, the first end-to-end framework designed to enhance adaptive thinking ability in medical reasoning models with uncertainty-guided length calibration. AdaThink-Med first generates multiple candidate outputs for each question, evaluates the correctness and uncertainty of each candidate, and then estimates problem difficulty via an uncertainty-guided length calibration module. For outputs with low difficulty and correct answers, the framework penalizes longer reasoning paths; whereas for those with high difficulty and incorrect answers, it encourages extending the chain of thought to explore alternative solutions. On six public medical QA benchmarks, AdaThink-Med achieves up to 6.4x length reduction on average while retaining performance with only minimal degradation. Intriguingly, we observe that AdaThink-Med spontaneously develops two distinct reasoning modes, which we characterize as “non-thinking” and “thinking”, demonstrating the model’s ability to suppress redundant reasoning processes dynamically.

[92] Inducing Dyslexia in Vision Language Models

Melika Honarmand,Ayati Sharma,Badr AlKhamissi,Johannes Mehrer,Martin Schrimpf

Main category: cs.CL

TL;DR: 该论文通过在视觉语言模型（VLM）中模拟阅读障碍（dyslexia），通过选择性扰动模型中的视觉单词处理单元，复制了人类阅读障碍的关键特征，为研究阅读障碍提供了一种新的计算框架。

Details Motivation: 传统研究阅读障碍的方法（如行为学和神经影像学）无法验证因果假设，因此作者利用VLM模拟阅读障碍，以更灵活地探索其机制。

Contribution: 论文的主要贡献是通过功能性扰动VLM中的视觉单词处理单元，成功模拟了人类阅读障碍的典型特征（如语音缺陷），同时保持了其他认知能力的完整。

Method: 作者在VLM中识别了选择性处理视觉单词的单元，并通过靶向扰动这些单元（而非随机单元）来模拟阅读障碍。实验使用了认知神经科学的刺激材料。

Result: 结果显示，靶向扰动导致了选择性阅读任务受损，但其他视觉和语言理解能力未受影响，这与人类阅读障碍的表现一致。

Insight: 这项工作表明，VLM可以作为研究阅读障碍的工具，为未来的理论研究提供了新的模拟和测试平台。

Abstract: Dyslexia, a neurodevelopmental disorder characterized by persistent reading difficulties, is often linked to reduced activity of the visual word form area in the ventral occipito-temporal cortex. Traditional approaches to studying dyslexia, such as behavioral and neuroimaging methods, have provided valuable insights but remain limited in their ability to test causal hypotheses about the underlying mechanisms of reading impairments. In this study, we use large-scale vision-language models (VLMs) to simulate dyslexia by functionally identifying and perturbing artificial analogues of word processing. Using stimuli from cognitive neuroscience, we identify visual-word-form-selective units within VLMs and demonstrate that targeted ablation of these units, unlike ablation of random units, leads to selective impairments in reading tasks while general visual and language comprehension abilities remain intact. In particular, the resulting model matches dyslexic humans’ phonological deficits without a significant change in orthographic processing. Taken together, our modeling results replicate key characteristics of dyslexia and establish a computational framework for investigating reading disorders.

[93] InfLLM-V2: Dense-Sparse Switchable Attention for Seamless Short-to-Long Adaptation

Weilin Zhao,Zihan Zhou,Zhou Su,Chaojun Xiao,Yuxuan Li,Yanghao Li,Yudi Zhang,Weilun Zhao,Zhen Li,Yuxiang Huang,Ao Sun,Xu Han,Zhiyuan Liu

Main category: cs.CL

TL;DR: InfLLM-V2提出了一种密集-稀疏可切换注意力框架，解决了传统Transformer在长序列处理中的计算和内存瓶颈问题，同时保持了短序列到长序列的无缝适应。

Details Motivation: 标准Transformer的自注意力机制在处理长序列时面临严重的计算和内存瓶颈。现有的稀疏注意力方法虽然提供了解决方案，但带来了额外的参数和训练流程中断的问题。

Contribution: InfLLM-V2的主要贡献包括：1）密集-稀疏可切换注意力框架，无缝适应短序列到长序列；2）参数复用架构，减少额外参数；3）高效的实现，显著降低计算开销。

Method: InfLLM-V2通过参数复用和无架构修改，实现密集注意力与稀疏注意力的平滑切换。短序列使用密集注意力，长序列自动切换到稀疏注意力，保证计算效率。

Result: 实验表明，InfLLM-V2在处理长上下文理解和链式推理任务时，速度比密集注意力快4倍，同时保持了98.1%和99.7%的性能。

Insight: InfLLM-V2提供了一种高效且灵活的长序列处理方案，解决了稀疏注意力方法中参数冗余和训练流程不连贯的问题，适用于实际部署。

Abstract: Long-sequence processing is a critical capability for modern large language models. However, the self-attention mechanism in the standard Transformer architecture faces severe computational and memory bottlenecks when processing long sequences. While trainable sparse attention methods offer a promising solution, existing approaches such as NSA introduce excessive extra parameters and disrupt the conventional \textit{pretrain-on-short, finetune-on-long} workflow, resulting in slow convergence and difficulty in acceleration. To overcome these limitations, we introduce dense-sparse switchable attention framework, termed as InfLLM-V2. InfLLM-V2 is a trainable sparse attention that seamlessly adapts models from short to long sequences. Specifically, InfLLM-V2 reuses dense attention parameters through parameter-free architecture modification, maintaining consistency between short and long sequence processing. Additionally, InfLLM-V2 ensures computational efficiency across all sequence lengths, by using dense attention for short inputs and smoothly transitioning to sparse attention for long sequences. To achieve practical acceleration, we further introduce an efficient implementation of InfLLM-V2 that significantly reduces the computational overhead. Our experiments on long-context understanding and chain-of-thought reasoning demonstrate that InfLLM-V2 is 4$\times$ faster than dense attention while retaining 98.1% and 99.7% of the performance, respectively. Based on the InfLLM-V2 framework, we have trained and open-sourced MiniCPM4.1 (https://huggingface.co/openbmb/MiniCPM4.1-8B), a hybrid reasoning model, providing a reproducible implementation for the research community.

[94] MemGen: Weaving Generative Latent Memory for Self-Evolving Agents

Guibin Zhang,Muxin Fu,Shuicheng Yan

Main category: cs.CL

TL;DR: MemGen提出了一种动态生成记忆框架，通过内存触发器和内存编织器，实现了LLM-powered agents的记忆与推理的紧密结合，显著提升了性能，并展示了自发形成人类记忆能力的潜力。

Details Motivation: 现有代理记忆范式（参数化记忆和检索记忆）无法捕捉人类认知中推理与记忆的动态结合，MemGen旨在填补这一空白。

Contribution: 1) 提出MemGen框架，结合内存触发器和内存编织器；2) 展示了代理在无监督下自发形成多种人类记忆能力；3) 在多个基准上显著超越现有记忆系统。

Method: MemGen由内存触发器（监控推理状态以决定显式调用记忆）和内存编织器（根据当前状态生成潜在的token序列作为记忆）组成，实现记忆与推理的动态结合。

Result: MemGen在8个基准上优于ExpeL、AWM和GRPO，性能提升最高达38.22%，并展示了跨域泛化能力和自发记忆能力的形成。

Insight: MemGen的自发记忆形成表明，机器学习可实现更自然的人类认知机制，具备潜在的自我演化能力。

Abstract: Agent memory shapes how Large Language Model (LLM)-powered agents, akin to the human brain, progressively refine themselves through environment interactions. Existing paradigms remain constrained: parametric memory forcibly adjusts model parameters, and retrieval-based memory externalizes experience into structured databases, yet neither captures the fluid interweaving of reasoning and memory that underlies human cognition. To address this gap, we propose MemGen, a dynamic generative memory framework that equips agents with a human-esque cognitive faculty. It consists of a \textit{memory trigger}, which monitors the agent’s reasoning state to decide explicit memory invocation, and a \textit{memory weaver}, which takes the agent’s current state as stimulus to construct a latent token sequence as machine-native memory to enrich its reasoning. In this way, MemGen enables agents to recall and augment latent memory throughout reasoning, producing a tightly interwoven cycle of memory and cognition. Extensive experiments across eight benchmarks show that MemGen surpasses leading external memory systems such as ExpeL and AWM by up to $38.22%$, exceeds GRPO by up to $13.44%$, and exhibits strong cross-domain generalization ability. More importantly, we find that without explicit supervision, MemGen spontaneously evolves distinct human-like memory faculties, including planning memory, procedural memory, and working memory, suggesting an emergent trajectory toward more naturalistic forms of machine cognition.

[95] Socratic-Zero : Bootstrapping Reasoning via Data-Free Agent Co-evolution

Shaobo Wang,Zhengbo Jiao,Zifan Zhang,Yilang Peng,Xu Ze,Boyu Yang,Wei Wang,Hu Wei,Linfeng Zhang

Main category: cs.CL

TL;DR: Socratic-Zero提出了一种完全自主的框架，通过教师、求解器和生成器的协同进化，从少量种子示例生成高质量训练数据，显著提升了模型在数学推理任务中的表现。

Details Motivation: 现有大规模高质量数据集依赖人工标注，难以扩展；数据合成方法存在质量不一致和无法动态适应模型能力的问题。

Contribution: 设计了三个智能体协同进化的闭环系统，能够自动生成高质量训练数据；实验表明其在多个数学推理基准上的显著提升。

Method: 通过教师生成挑战性问题、求解器优化推理能力、生成器扩展课程设计策略，实现闭环的自适应课程生成。

Result: 仅用100种子问题，Socratic-Solver-8B在七项数学推理基准上平均提升20.2个百分点；合成数据还使其他LLM超越SOTA商业模型。

Insight: 闭环的协同进化机制能够有效解决数据合成的质量和适应性难题，为少样本或零样本学习提供了新思路。

Abstract: Recent breakthroughs in large language models (LLMs) on reasoning tasks rely heavily on massive, high-quality datasets-typically human-annotated and thus difficult to scale. While data synthesis or distillation offers a promising alternative, existing methods struggle with inconsistent data quality and an inability to dynamically adapt to the evolving capabilities of the model, leading to suboptimal training signals. To address these limitations, we introduce Socratic-Zero, a fully autonomous framework that generates high-quality training data from minimal seed examples through the co-evolution of three agents: the Teacher, the Solver, and the Generator. The Solver continuously refines its reasoning by learning from preference feedback on both successful and failed trajectories; the Teacher adaptively crafts increasingly challenging questions based on the Solver’s weaknesses; and the Generator distills the Teacher’s question-design strategy to enable scalable, high-fidelity curriculum generation. This closed-loop system produces a self-improving curriculum-requiring no pre-existing tasks or labels. Remarkably, starting from only 100 seed questions, our Socratic-Solver-8B achieves an average gain of +20.2 percentage points over prior data synthesis methods across seven mathematical reasoning benchmarks (AMC23, AIME24-25, Olympiad, MATH-500, Minerva, and GSM8K), with consistent gains on both Qwen3 and GLM4 series models. Even more surprisingly, synthetic data from Socratic-Generator-32B enables student LLMs to achieve superior performance compared to other state-of-the-art (SOTA) commercial LLMs on these benchmarks, including Qwen3-235B-A22B, DeepSeek-V3.1-671B, GPT-5, Gemini-2.5-Pro, Grok-4, and Claude-4.1-Opus.

[96] ProxyAttn: Guided Sparse Attention via Representative Heads

Yixuan Wang,Huang He,Siqi Bao,Hua Wu,Haifeng Wang,Qingfu Zhu,Wanxiang Che

Main category: cs.CL

TL;DR: ProxyAttn是一种训练免费的稀疏注意力算法，利用注意力头之间的相似性，通过代表性头代理所有头的注意力分数，并结合动态预算估计，实现了更精细的块重要性评估，显著提升了长文本任务的效率和性能。

Details Motivation: 传统注意力机制的二次复杂度限制了大型语言模型（LLM）在长文本任务中的效率。现有方法通过动态估计块重要性实现了块稀疏注意力，但其粗粒度估计在高稀疏率下会导致性能下降。

Contribution: 1. 提出了ProxyAttn，一种无需训练的稀疏注意力算法；2. 通过压缩注意力头维度实现了更精确的块估计；3. 提出了基于多头动态预算的块感知动态预算估计方法。

Method: 1. 观察到多头注意力之间的相似性，用代表性头的分数近似所有头的分数；2. 结合多头动态预算，实现低计算成本的精细块重要性评估。

Result: 在多种主流模型和基准测试中，ProxyAttn实现了高达10.3倍的注意力加速和2.4倍的预填充加速，且无明显性能损失。

Insight: 注意力头之间存在高度相似性，可以通过少量代表性头代理其他头的行为，从而在高稀疏率下实现高效且精确的注意力计算。

Abstract: The quadratic complexity of attention mechanisms limits the efficiency of Large Language Models (LLMs) on long-text tasks. Recently, methods that dynamically estimate block importance have enabled efficient block sparse attention, leading to significant acceleration in long-text pre-filling of LLMs. However, their coarse-grained estimation inevitably leads to performance degradation at high sparsity rates. In this work, we propose ProxyAttn, a training-free sparse attention algorithm that achieves more precise block estimation by compressing the dimension of attention heads. Based on our observation of the similarity among multiple attention heads, we use the scores of pooled representative heads to approximate the scores for all heads. To account for the varying sparsity among heads, we also propose a block-aware dynamic budget estimation method. By combining the scores from representative proxy heads with multi-head dynamic budgets, we achieve a more fine-grained block importance evaluation at low computational cost. Experiments on a variety of mainstream models and extensive benchmarks confirm the underlying similarity among attention heads. Leveraging a fine-grained estimation, the proposed method achieves substantial gains in performance and efficiency compared to existing methods. More precisely, ProxyAttn can achieve up to 10.3x attention acceleration and 2.4x prefilling acceleration without significant performance loss. Our code is available at https://github.com/wyxstriker/ProxyAttn.

[97] LatentEvolve: Self-Evolving Test-Time Scaling in Latent Space

Guibin Zhang,Fanci Meng,Guancheng Wan,Zherui Li,Kun Wang,Zhenfei Yin,Lei Bai,Shuicheng Yan

Main category: cs.CL

TL;DR: LatentEvolve提出了一种自进化的测试时缩放框架，通过模拟人脑的双系统学习机制，提升了大型语言模型在推理阶段的性能表现，无需修改模型参数。

Details Motivation: 现有测试时缩放方法独立且缺乏渐进学习能力，限制了大型语言模型在推理阶段的潜力。受人类大脑互补学习系统启发，作者希望通过模拟快慢双系统的学习机制，改进测试时缩放的效果。

Contribution: 提出了LatentEvolve框架，通过白天快速检索历史潜表示和夜间整合潜优化，实现了自进化的测试时缩放，显著提升了模型的推理能力。

Method: LatentEvolve包含两个进化组件：白天缩放（快速检索历史潜表示）和夜间缩放（整合历史潜优化），模拟了人类大脑的快慢双系统学习机制。

Result: 在八个基准数据集和五种模型骨干上的实验显示，LatentEvolve超越了现有的测试时缩放方法（如LatentSeek和TTRL），最高提升13.33%，并展现出出色的跨领域和跨骨干泛化能力。

Insight: 通过模拟人类认知的动态学习机制（快慢结合），可以在完全无监督的情况下显著提升大型语言模型的推理能力，为测试时缩放提供了新的设计思路。

Abstract: Test-time Scaling (TTS) has been demonstrated to significantly enhance the reasoning capabilities of Large Language Models (LLMs) during the inference phase without altering model parameters. However, existing TTS methods are largely independent, implying that LLMs have not yet evolved to progressively learn how to scale more effectively. With the objective of evolving LLMs to learn ``how to scale test-time computation,’’ we propose LatentEvolve, a self-evolving latent TTS framework inspired by the complementary learning system (CLS) theory. Analogous to the human brain’s dual system of a fast-recall hippocampus and a slow-consolidating neocortex, LatentEvolve comprises two evolutionary components: \textit{daytime scaling}, which rapidly retrieves historical latent representations to better guide current LLM reasoning; and \textit{nighttime scaling}, which integrates past latent optimizations in a manner akin to the human brain’s consolidation of experiences during sleep. The alternation of daytime and nighttime processes facilitates a fast and slow evolution of LLM TTS, mirroring human cognitive dynamics in a fully unsupervised manner. Extensive experiments across eight benchmarks and five model backbones demonstrate that our LatentEvolve surpasses state-of-the-art TTS methods such as LatentSeek and TTRL by up to $13.33%$ and exhibits exceptional cross-domain and cross-backbone generalization.

[98] KnowGuard: Knowledge-Driven Abstention for Multi-Round Clinical Reasoning

Xilin Dang,Kexin Chen,Xiaorui Su,Ayush Noori,Iñaki Arango,Lucas Vittor,Xinyi Long,Yuyang Du,Marinka Zitnik,Pheng Ann Heng

Main category: cs.CL

TL;DR: KnowGuard提出了一种基于知识驱动的多轮临床推理中的‘暂不决策’机制，通过知识图谱探索和证据评估，显著提升了诊断准确性并减少了不必要的交互。

Details Motivation: 临床实践中，医生在面对信息不足时会选择暂不决策以避免误诊，但现有大型语言模型（LLMs）难以实现这一行为，常因过度自信导致错误。KnowGuard旨在解决这一问题。

Contribution: 1）提出了‘先探索再暂不决策’的新范式；2）设计了基于知识图谱的系统性证据探索和评估方法；3）在开放的临床基准测试中验证了方法的有效性。

Method: 方法分为两阶段：1）证据发现阶段，通过图谱扩展和直接检索探索医学知识；2）证据评估阶段，基于患者背景和对话历史对证据排序，动态调整探索策略。

Result: KnowGuard在诊断准确性上提升了3.93%，同时平均减少了7.27次不必要的交互，优于现有技术。

Insight: 结合外部知识图谱的方法能够更准确地识别知识边界，显著提升临床决策的可靠性。

Abstract: In clinical practice, physicians refrain from making decisions when patient information is insufficient. This behavior, known as abstention, is a critical safety mechanism preventing potentially harmful misdiagnoses. Recent investigations have reported the application of large language models (LLMs) in medical scenarios. However, existing LLMs struggle with the abstentions, frequently providing overconfident responses despite incomplete information. This limitation stems from conventional abstention methods relying solely on model self-assessments, which lack systematic strategies to identify knowledge boundaries with external medical evidences. To address this, we propose \textbf{KnowGuard}, a novel \textit{investigate-before-abstain} paradigm that integrates systematic knowledge graph exploration for clinical decision-making. Our approach consists of two key stages operating on a shared contextualized evidence pool: 1) an evidence discovery stage that systematically explores the medical knowledge space through graph expansion and direct retrieval, and 2) an evidence evaluation stage that ranks evidence using multiple factors to adapt exploration based on patient context and conversation history. This two-stage approach enables systematic knowledge graph exploration, allowing models to trace structured reasoning paths and recognize insufficient medical evidence. We evaluate our abstention approach using open-ended multi-round clinical benchmarks that mimic realistic diagnostic scenarios, assessing abstention quality through accuracy-efficiency trade-offs beyond existing closed-form evaluations. Experimental evidences clearly demonstrate that KnowGuard outperforms state-of-the-art abstention approaches, improving diagnostic accuracy by 3.93% while reducing unnecessary interaction by 7.27 turns on average.

[99] DiaCDM: Cognitive Diagnosis in Teacher-Student Dialogues using the Initiation-Response-Evaluation Framework

Rui Jia,Yuang Wei,Ruijia Li,Yuang-Hao Jiang,Xinyu Xie,Yaomin Shen,Min Zhang,Bo Jiang

Main category: cs.CL

TL;DR: DiaCDM是一个基于对话的认知诊断模型，首次将认知诊断应用于师生对话场景，通过改进的IRE框架和图编码方法解决了传统CD模型在动态和非结构化对话中的问题，显著提升了诊断准确性和结果可解释性。

Details Motivation: 传统认知诊断模型（CD）依赖结构化测试数据，无法直接应用于动态、非结构化的师生对话。此外，从长对话中准确提取诊断语义也存在挑战。因此，研究需要一种适合对话场景的诊断框架和方法。

Contribution: 1. 提出首个基于师生对话的认知诊断模型DiaCDM；2. 结合教育理论设计了基于IRE框架的诊断方法；3. 提出一种图编码方法，整合教师问题与相关知识组件，精准捕捉关键信息。

Method: 1. 使用改进的IRE框架（发起-回应-评估）设计对话诊断流程；2. 开发图编码方法，关联教师提问和学生回答中的知识点；3. 基于真实对话数据集验证模型效果。

Result: 在三个真实对话数据集上的实验表明，DiaCDM显著提升了诊断准确性，同时增强了结果的可解释性，为教师评估学生认知状态提供了新工具。

Insight: 1. 对话场景的认知诊断需要动态和非结构化数据处理能力；2. 结合教育理论（如IRE框架）能有效提升模型适应性；3. 图编码方法可以有效捕捉对话中的核心语义关系。

Abstract: While cognitive diagnosis (CD) effectively assesses students’ knowledge mastery from structured test data, applying it to real-world teacher-student dialogues presents two fundamental challenges. Traditional CD models lack a suitable framework for handling dynamic, unstructured dialogues, and it’s difficult to accurately extract diagnostic semantics from lengthy dialogues. To overcome these hurdles, we propose DiaCDM, an innovative model. We’ve adapted the initiation-response-evaluation (IRE) framework from educational theory to design a diagnostic framework tailored for dialogue. We also developed a unique graph-based encoding method that integrates teacher questions with relevant knowledge components to capture key information more precisely. To our knowledge, this is the first exploration of cognitive diagnosis in a dialogue setting. Experiments on three real-world dialogue datasets confirm that DiaCDM not only significantly improves diagnostic accuracy but also enhances the results’ interpretability, providing teachers with a powerful tool for assessing students’ cognitive states. The code is available at https://github.com/Mind-Lab-ECNU/DiaCDM/tree/main.

[100] Hierarchical Error Correction for Large Language Models: A Systematic Framework for Domain-Specific AI Quality Enhancement

Zhilong Zhao,Yindi Liu

Main category: cs.CL

TL;DR: 该论文提出了一种分层纠错（HEC）框架，用于解决大语言模型在专业领域中的性能问题，通过系统化的错误分析和有针对性的干预策略，显著提升了模型的准确性。

Details Motivation: 大语言模型在专业领域中表现不佳，例如医疗编码任务中仅达到45.9%的准确率。为了解决这一问题，研究者分析了错误的层次结构，并提出了一个系统化的纠错框架。

Contribution: 主要贡献是提出了HEC框架，包括错误分层分析（知识层、推理层和复杂度层）和三阶段纠错方法，证明了其在多个专业领域中的有效性。

Method: 方法包括分析错误模式的分层结构（知识层58.4%、推理层39.6%、复杂度层2.0%），并根据分层重要性设计了三阶段纠错框架。实验验证了其在医疗、法律等领域的有效性。

Result: 实验结果表明，HEC框架在多个领域的任务中平均提升了11.2个百分点（p < 0.001），但在高基线任务（>75%准确率）中效果有限。

Insight: 系统化的错误分析可以指导有效的AI增强策略，尤其在中等基线任务中表现突出，但也揭示了在高基线和复杂推理任务中的局限性。

Abstract: Large Language Models face significant performance challenges in specialized domains, with state-of-the-art models achieving only 45.9% accuracy on medical coding tasks. This study proposes a Hierarchical Error Correction (HEC) framework that addresses domain-specific AI limitations through systematic error analysis and targeted intervention strategies. We analyze error patterns across four specialized domains and find that AI errors follow consistent hierarchical structures: Knowledge-layer errors (58.4%), Reasoning-layer errors (39.6%), and Complexity-layer errors (2.0%). Based on these patterns, we develop a three-stage correction framework that addresses errors according to their hierarchical importance and demonstrates that framework effectiveness correlates inversely with baseline task performance. Experimental validation across medical transcription (4,921 cases), legal document classification (1,000 cases), political bias detection (645 cases), and legal reasoning (1,000 cases) shows consistent improvements. Cross-model validation across five LLM architectures demonstrates average improvements of 11.2 percentage points (p < 0.001). However, analysis reveals framework limitations in high-baseline tasks (>75% accuracy), where hierarchical intervention may interfere with effective reasoning processes. The results suggest that systematic error analysis can guide effective AI enhancement strategies in specialized domains, particularly for moderate-baseline tasks, while highlighting the importance of understanding framework boundaries for optimal deployment.

[101] Expanding Computation Spaces of LLMs at Inference Time

Yoonna Jang,Kisu Yang,Isabelle Augenstein

Main category: cs.CL

TL;DR: 本文研究了在推理阶段为语言模型插入填充标记（filler tokens）以扩展其计算空间的有效性，揭示了标记类型、数量和插入位置的影响，并发现较小模型从中受益更大。

Details Motivation: 通过扩展语言模型的计算空间，提升其在推理阶段的性能，尤其是在任务相关的详细推理步骤中。

Contribution: 1. 确定了填充标记的有效类型、数量和插入位置；2. 研究了模型训练阶段对计算空间的利用能力；3. 通过注意力图分析计算空间的动态行为。

Method: 在推理阶段插入填充标记，并通过实验在不同规模的模型（1.7B到32B）上进行评估，关注标记类型、数量和位置的影响。

Result: 适当插入填充标记能显著提升模型性能（SmolLM2-1.7B-Instruct最高提升12.372%），尤其对小模型效果显著。注意力图显示填充标记能延续原有注意力机制。

Insight: 填充标记为语言模型提供了额外的计算容量，而非冗余输入，小模型更能从中受益。

Abstract: Chain-of-thought (CoT) rationale enables language models to use additional task-related text for problem-solving, benefiting not only from detailed reasoning steps but also from the expanded computational space of longer inputs. Prior work has trained filler or special tokens to serve as additional computation spaces. In this study, we investigate whether language models can leverage artificially inserted sequences of filler tokens solely at inference. We first identify effective token types, numbers, and insertion locations, then examine at what stage of training models begin to exploit the expanded computation space, and finally analyze dynamics within these spaces via attention maps. Experiments on models ranging from 1.7B to 32B across open-domain QA and math tasks show that appropriate token types and counts vary, but placing filler tokens directly before the final ‘Answer:’ token is most effective. Smaller models benefit most, up to 12.372 percentage points in SmolLM2-1.7B-Instruct, indicating that these spaces act as additional computational capacity rather than redundant input. Attention maps reveal that expanded spaces often continue the original attention mechanism and sometimes focus on questions or answer options, suggesting meaningful computation for problem-solving.

[102] MobileLLM-R1: Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes

Changsheng Zhao,Ernie Chang,Zechun Liu,Chia-Jung Chang,Wei Wen,Chen Lai,Rick Cao,Yuandong Tian,Raghuraman Krishnamoorthi,Yangyang Shi,Vikas Chandra

Main category: cs.CL

TL;DR: MobileLLM-R1挑战了大型语言模型需要海量数据和参数才能具备推理能力的假设，展示了通过高质量数据精选和重采样，小规模模型也能实现强大推理能力。

Details Motivation: 传统观点认为语言模型的推理能力需要大规模模型和海量数据支持，但近期研究表明小规模模型也能具备推理能力。本文进一步探讨数据规模和质量的必要性。

Contribution: 提出了MobileLLM-R1系列模型，证明仅需2T高质量数据和精选训练方法，小规模模型也能超越更大规模的同类模型；并开源了完整训练方案和数据。

Method: 通过设计的数据质量评估指标，精选和重采样开源数据集，结合4.2T token的训练和标准后训练流程，开发了MobileLLM-R1系列小规模推理模型。

Result: MobileLLM-R1-950M在AIME评分达15.5，远超同类开源模型，且在推理任务中媲美或超越更大规模的Qwen3-0.6B，仅需后者11.7%的训练数据。

Insight: 高质量数据和小规模模型的精心设计训练方法可以替代海量数据和参数规模，为资源有限的推理模型开发提供了新思路。

Abstract: The paradigm shift in large language models (LLMs) from instinctive responses to chain-of-thought (CoT) reasoning has fueled two prevailing assumptions: (1) reasoning capabilities only emerge in sufficiently large models, and (2) such capabilities require training on massive datasets. While the first assumption has already been challenged by recent sub-billion-parameter reasoning models such as Qwen3-0.6B and DeepSeek distilled variants, the second remains largely unquestioned. In this work, we revisit the necessity of scaling to extremely large corpora (>10T tokens) for reasoning emergence. By carefully curating and resampling open-source datasets that we identify as beneficial under our designed metrics, we demonstrate that strong reasoning abilities can emerge with far less data. Specifically, we show that only ~2T tokens of high-quality data are sufficient, and pre-training with 4.2T tokens on the dataset resampled from these ~2T tokens, followed by a established post-training procedure, enables the development of MobileLLM-R1, a series of sub-billion-parameter reasoning models that substantially outperform prior models trained on fully open-sourced data. For example, MobileLLM-R1-950M achieves an AIME score of 15.5, compared to just 0.6 for OLMo-2-1.48B and 0.3 for SmolLM-2-1.7B. Remarkably, despite being trained on only 11.7% of the tokens compared to Qwen3’s proprietary 36T-token corpus for pretraining, MobileLLM-R1-950M matches or surpasses Qwen3-0.6B across multiple reasoning benchmarks. To facilitate further research in this direction, we have released the complete training recipe, data sources, data mixing ratio, and model checkpoints, together with the key insights obtained throughout this study.

[103] The Dialogue That Heals: A Comprehensive Evaluation of Doctor Agents’ Inquiry Capability

Linlu Gong,Ante Wang,Yunghwei Lai,Weizhi Ma,Yang Liu

Main category: cs.CL

TL;DR: 论文提出了MAQuE基准，用于全面评估医疗多轮问答能力，涵盖任务成功、询问熟练度等多项指标，并揭示了现有AI医生在询问能力上的不足以及对患者行为变化的敏感性。

Details Motivation: 当前AI医生虽具备诊断能力，但忽略了其他关键品质（如沟通能力）。研究旨在填补这一空白，通过大规模基准和评估框架全面衡量AI医生的多轮询问能力。

Contribution: 1) 提出MAQuE基准，包含3000个模拟患者代理；2) 设计了涵盖多方面的评估框架；3) 揭示了现有LLM在询问能力上的局限性和改进空间。

Method: 1) 构建多样化的模拟患者代理；2) 提出多维度评估指标（任务成功、询问熟练度等）；3) 对不同LLM进行实验分析。

Result: 实验显示，即使最先进的LLM在询问能力上仍有显著不足，且对患者行为变化高度敏感。精细指标还暴露了不同评估视角间的权衡问题。

Insight: 1) AI医生的询问能力需进一步提升；2) 现实患者行为的多样性对诊断准确性影响重大；3) 实际临床应用中需平衡性能与实用性。

Abstract: An effective physician should possess a combination of empathy, expertise, patience, and clear communication when treating a patient. Recent advances have successfully endowed AI doctors with expert diagnostic skills, particularly the ability to actively seek information through inquiry. However, other essential qualities of a good doctor remain overlooked. To bridge this gap, we present MAQuE(Medical Agent Questioning Evaluation), the largest-ever benchmark for the automatic and comprehensive evaluation of medical multi-turn questioning. It features 3,000 realistically simulated patient agents that exhibit diverse linguistic patterns, cognitive limitations, emotional responses, and tendencies for passive disclosure. We also introduce a multi-faceted evaluation framework, covering task success, inquiry proficiency, dialogue competence, inquiry efficiency, and patient experience. Experiments on different LLMs reveal substantial challenges across the evaluation aspects. Even state-of-the-art models show significant room for improvement in their inquiry capabilities. These models are highly sensitive to variations in realistic patient behavior, which considerably impacts diagnostic accuracy. Furthermore, our fine-grained metrics expose trade-offs between different evaluation perspectives, highlighting the challenge of balancing performance and practicality in real-world clinical settings.

[104] SemanticShield: LLM-Powered Audits Expose Shilling Attacks in Recommender Systems

Kaihong Li,Huichi Zhou,Bin Ma,Fangjun Huang

Main category: cs.CL

TL;DR: 提出了SemanticShield框架，通过LLM检测推荐系统中的虚假行为攻击，结合行为特征和语义分析，提高了识别恶意用户的能力。

Details Motivation: 推荐系统易受虚假行为攻击（shilling attacks），现有防御主要关注用户行为，忽略了物品特征（如标题和描述）中的恶意语义。

Contribution: 1. 提出两阶段检测框架，结合行为特征和LLM语义分析；2. 设计了强化微调方法优化轻量级LLM，提升语义一致性检测能力；3. 验证了SemanticShield在多样攻击策略下的有效性。

Method: 1. 第一阶段通过低成本行为标准预筛可疑用户；2. 第二阶段用LLM审计语义一致性；3. 通过强化微调优化轻量级LLM。

Result: 在六种代表性攻击策略上表现优异，对未知攻击方法也展现出强泛化能力。

Insight: 物品特征中的语义信息能有效暴露恶意意图，轻量级LLM结合强化学习可显著提升检测性能。

Abstract: Recommender systems (RS) are widely used in e-commerce for personalized suggestions, yet their openness makes them susceptible to shilling attacks, where adversaries inject fake behaviors to manipulate recommendations. Most existing defenses emphasize user-side behaviors while overlooking item-side features such as titles and descriptions that can expose malicious intent. To address this gap, we propose a two-stage detection framework that integrates item-side semantics via large language models (LLMs). The first stage pre-screens suspicious users using low-cost behavioral criteria, and the second stage employs LLM-based auditing to evaluate semantic consistency. Furthermore, we enhance the auditing model through reinforcement fine-tuning on a lightweight LLM with carefully designed reward functions, yielding a specialized detector called SemanticShield. Experiments on six representative attack strategies demonstrate the effectiveness of SemanticShield against shilling attacks, and further evaluation on previously unseen attack methods shows its strong generalization capability. Code is available at https://github.com/FrankenstLee/SemanticShield.

[105] GateMABSA: Aspect-Image Gated Fusion for Multimodal Aspect-based Sentiment Analysis

Adamu Lawan,Haruna Yunusa

Main category: cs.CL

TL;DR: GateMABSA是一个新颖的多模态门控架构，用于解决多模态基于方面的情感分析中视觉噪声和跨模态对齐问题，显著优于基线模型。

Details Motivation: 现有MABSA模型难以过滤视觉噪声并有效对齐文本和图像中的方面与情感内容，因此GateMABSA提出了一种改进方案。

Contribution: 提出了GateMABSA架构，结合了语法、语义和融合感知的mLSTM，通过三种专门设计的mLSTM实现了更优的多模态融合。

Method: 设计了Syn-mLSTM、Sem-mLSTM和Fuse-mLSTM，分别用于整合语法结构、强调方面语义相关性以及选择性地融合多模态信息。

Result: 在两个Twitter基准数据集上的实验表明，GateMABSA显著优于多个基线模型。

Insight: 通过分离语法、语义和融合的多模态处理，GateMABSA有效提升了跨模态对齐能力和噪声过滤效果。

Abstract: Aspect-based Sentiment Analysis (ABSA) has recently advanced into the multimodal domain, where user-generated content often combines text and images. However, existing multimodal ABSA (MABSA) models struggle to filter noisy visual signals, and effectively align aspects with opinion-bearing content across modalities. To address these challenges, we propose GateMABSA, a novel gated multimodal architecture that integrates syntactic, semantic, and fusion-aware mLSTM. Specifically, GateMABSA introduces three specialized mLSTMs: Syn-mLSTM to incorporate syntactic structure, Sem-mLSTM to emphasize aspect–semantic relevance, and Fuse-mLSTM to perform selective multimodal fusion. Extensive experiments on two benchmark Twitter datasets demonstrate that GateMABSA outperforms several baselines.

[106] An empirical study on the limitation of Transformers in program trace generation

Simeng Sun

Main category: cs.CL

TL;DR: 该论文实证研究了Transformer模型在程序追踪生成任务中的局限性，发现虽然模型在分布内数据上表现良好，但在泛化到不同因素时存在系统性失败。

Details Motivation: 研究动机是探索Transformer在程序追踪生成（PTG）任务中的表现，尤其是在外部化推理的长追踪任务中，验证其泛化能力。

Contribution: 主要贡献是通过实验揭示Transformer在小规模程序追踪生成任务中的局限性，并提出了一些改进设计（如替代位置编码、混合模型等）以提升泛化能力。

Method: 方法包括训练小型Transformer模型，并测试多种修改方案（如替代softmax、短卷积等），以评估其在PTG任务中的表现。

Result: 结果显示，尽管模型在分布内数据上表现良好，但在泛化性测试（如程序长度、追踪步骤）中表现不佳，部分设计显著提升了泛化能力。

Insight: 研究发现，Transformer在处理长追踪任务时的泛化能力有限，但通过模型设计的调整可以部分缓解这一问题。

Abstract: We study Transformers on the task \emph{program trace generation} (PTG), where models produce step-by-step execution traces for synthetic programs. Unlike existing algorithmic problems, PTG externalizes reasoning through long traces where each step is trivial. We train small Transformers with diverse modifications, including alternative position encodings, softmax replacements, hybrid model, and short convolutions. While these models achieve strong in-distribution accuracy, they exhibit systematic failures when generalizing to various factors (e.g., program length, trace steps), though some designs significantly improve generalization.

[107] Scaling Generalist Data-Analytic Agents

Shuofei Qiao,Yanqiu Zhao,Zhisong Qiu,Xiaobin Wang,Jintian Zhang,Zhao Bin,Ningyu Zhang,Yong Jiang,Pengjun Xie,Fei Huang,Huajun Chen

Main category: cs.CL

TL;DR: 该论文介绍了DataMind，一种可扩展的数据合成和智能体训练方法，用于构建通用的数据分析智能体。DataMind解决了开源数据分析智能体面临的三大挑战：数据资源不足、训练策略不当和不稳定的多轮代码执行。通过一系列创新技术，DataMind在多个数据分析基准测试中取得了最优性能。

Details Motivation: 现有方法依赖专有模型和提示工程，而开源模型在面对多样化格式和大规模数据文件时表现不佳。DataMind旨在解决这些问题，推动开源数据分析智能体的发展。

Contribution: 1) 提出DataMind，一种可扩展的数据合成和智能体训练方法；2) 构建高质量轨迹数据集DataMind-12K；3) DataMind-14B和DataMind-7B在多个基准测试中表现优异。

Method: 1) 细化任务分类和递归从易到难的任务组合机制；2) 知识增强轨迹采样策略与模型/规则过滤；3) 动态调整的SFT和RL损失训练目标；4) 内存高效且稳定的多轮代码执行框架。

Result: DataMind-14B在多个数据分析基准测试中平均得分71.16%，优于最强的专有基线DeepSeek-V3.1和GPT-5；DataMind-7B在开源模型中表现最佳，得分68.10%。

Insight: 论文展示了高质量数据合成和训练策略对提升智能体性能的重要性，为开源社区提供了可借鉴的方法和数据集。

Abstract: Data-analytic agents are emerging as a key catalyst for automated scientific discovery and for the vision of Innovating AI. Current approaches, however, rely heavily on prompt engineering over proprietary models, while open-source models struggle to face diverse-format, large-scale data files and long-horizon, multi-step reasoning that real-world analytics demands. This paper introduces DataMind, a scalable data synthesis and agent training recipe designed to build generalist data-analytic agents. DataMind tackles three key challenges in building open-source data-analytic agents, including insufficient data resources, improper training strategy, and unstable code-based multi-turn rollout. Concretely, DataMind applies 1) a fine-grained task taxonomy and a recursive easy-to-hard task composition mechanism to increase the diversity and difficulty of synthesized queries; 2) a knowledge-augmented trajectory sampling strategy followed by model-based and rule-based filtering; 3) a dynamically adjustable training objective combining both SFT and RL losses; 4) a memory-frugal and stable code-based multi-turn rollout framework. Built on DataMind, we curate DataMind-12K, a high-quality trajectory set spanning diverse domains, task categories, and data file formats for data-analytic tasks. Trained on DataMind-12K, our DataMind-14B achieves state-of-the-art with an average score of 71.16% on multiple data analysis benchmarks, outperforming the strongest proprietary baselines DeepSeek-V3.1 and GPT-5. Our DataMind-7B also performs best among all open-source models with a score of 68.10%. We also incorporate some empirical insights gained from our exploratory trials into the analysis experiments, aiming to provide actionable insights about agentic training for the community. We will release DataMind-12K and DataMind-7B,14B for the community’s future research.

[108] jina-reranker-v3: Last but Not Late Interaction for Document Reranking

Feng Wang,Yuqing Li,Han Xiao

Main category: cs.CL

TL;DR: jina-reranker-v3是一种0.6B参数的多语言文档重排序模型，通过引入‘最后但不延迟的交互’机制，显著提升了性能。

Details Motivation: 现有方法（如ColBERT）采用延迟交互，导致效率较低。研究旨在通过更紧密的交互机制提升性能。

Contribution: 提出了一种新颖的‘最后但不延迟的交互’机制，实现了查询和文档在同一上下文窗口内的因果自注意力交互。

Method: 模型在同一上下文窗口内对查询和文档执行因果自注意力，并从每个文档的最后一个标记提取上下文嵌入。

Result: 在BEIR基准测试中达到61.94 nDCG@10，性能优于当前最优模型，且体积小十倍。

Insight: 紧密的交互机制可以显著提升重排序任务的性能，同时保持模型的紧凑性。

Abstract: jina-reranker-v3 is a 0.6B parameter multilingual document reranker that introduces a novel last but not late interaction. Unlike late interaction models such as ColBERT that perform separate encoding followed by multi-vector matching, our approach conducts causal self-attention between query and documents within the same context window, enabling rich cross-document interactions before extracting contextual embeddings from the last token of each document. This compact architecture achieves state-of-the-art BEIR performance with 61.94 nDCG@10 while being ten times smaller than generative listwise rerankers.

[109] InfoAgent: Advancing Autonomous Information-Seeking Agents

Gongrui Zhang,Jialiang Zhu,Ruiqi Yang,Kai Qiu,Miaosen Zhang,Zhirong Wu,Qi Dai,Bei Liu,Chong Luo,Zhengyuan Yang,Linjie Li,Lijuan Wang,Weizhu Chen,Yuan Zhang,Xin Li,Zhaoyi Liu,Xin Geng,Baining Guo

Main category: cs.CL

TL;DR: 本文介绍了InfoAgent，一种基于创新数据合成流程和网络搜索工具的自主信息检索代理，通过实体树和子树采样提高问题难度，并通过自托管搜索基础设施增强透明度。

Details Motivation: 构建能够通过外部工具扩展能力的LLM代理是AI研究的新前沿，但目前方法依赖商业工具且透明度不足。

Contribution: 1. 提出创新数据合成流程（实体树和子树采样）；2. 开发自托管搜索基础设施；3. 两阶段训练方法（SFT+RL）显著提升代理性能。

Method: 1. 构建实体树并采样复杂查询；2. 自托管搜索工具；3. 两阶段训练（SFT和RL）。

Result: InfoAgent在BrowseComp、BrowseComp-ZH和Xbench-DS上分别达到15.3%、29.2%和40.4%的准确率，优于现有开源代理。

Insight: 自托管工具和系统性数据合成是提升代理能力的关键，RL阶段显著优化了工具使用能力。

Abstract: Building Large Language Model agents that expand their capabilities by interacting with external tools represents a new frontier in AI research and applications. In this paper, we introduce InfoAgent, a deep research agent powered by an innovative data synthesis pipeline and orchestrated web search tools. To construct challenging, hard-to-find queries,we build entity trees and apply sub-tree sampling with entity fuzzification to systematically increase question difficulty. Unlike prior work that relies heavily on commercial search tools, we develop a dedicated self-hosted search infrastructure, enhancing transparency of agent environments and facilitating further advancement of agent capacity. We evaluate the effectiveness of our data pipeline by measuring the average number of tool calls required to correctly answer a question, and also show that our agent yields better performance when equipped with our tools. Our \mbox{InfoAgent} is post-trained from Qwen3-14B using a two-stage recipe: cold-start supervised finetuning to instill long-horizon search behaviors, followed by reinforcement learning which significantly improves reasoning-driven tool use. With our methods, InfoAgent achieves 15.3% accuracy on BrowseComp, 29.2% on BrowseComp-ZH, and 40.4% on Xbench-DS, outperforming prior open-source deep research agents such as WebSailor-72B and DeepDive-32B.

cs.CV [Back]

[110] Pathological Truth Bias in Vision-Language Models

Yash Thube

Main category: cs.CV

TL;DR: MATS方法是一种多模态行为审核工具，用于评估视觉语言模型是否拒绝与视觉矛盾的陈述，揭示了生成式模型在空间一致性上的缺陷和对比模型的鲁棒性。

Details Motivation: 标准的视觉语言模型评估基准可能掩盖系统性错误，降低实际应用中的信任度。

Contribution: 提出了MATS方法和两个新指标SCS与IAR，用于量化模型在面对视觉矛盾时的表现，并定位了模型的失效原因。

Method: MATS通过实验指令调优的生成式模型和对比式模型，使用激活补丁技术定位失效原因。

Result: 生成式VLMs（如LLaVA 1.5）在SCS上表现差且IAR高，而对比模型（如CLIP）更稳健。激活补丁技术揭示了失效的具体模块。

Insight: 生成式模型的失效可能与跨注意力机制的晚期模块有关，而对比模型的失效则集中在池化投影组件。

Abstract: Vision Language Models (VLMs) are improving quickly, but standard benchmarks can hide systematic failures that reduce real world trust. We introduce MATS (Multimodal Audit for Truthful Spatialization), a compact behavioral audit that measures whether models reject visually contradicted statements, and two metrics Spatial Consistency Score (SCS) and Incorrect Agreement Rate (IAR). Instruction tuned generative VLMs (LLaVA 1.5, QwenVLchat) exhibit very low SCS and high IAR, while contrastive encoders (CLIP, SigLIP) are far more robust. Activation patching causally localizes failure loci (mid to late cross attention for generative models, pooled projection components for contrastive models) and suggests concrete repair paths.

[111] Scale and Rotation Estimation of Similarity-Transformed Images via Cross-Correlation Maximization Based on Auxiliary Function Method

Shinji Yamashita,Yuma Kinoshita,Hitoshi Kiya

Main category: cs.CV

TL;DR: 本文提出了一种高效算法，通过基于辅助函数方法的互相关系数最大化，实现子像素级的尺度和旋转联合估计。

Details Motivation: 传统的相位相关技术在处理平移偏移时效果良好，但无法有效解决因相机缩放或旋转引起的尺度和旋转变化问题。

Contribution: 提出了一种新颖算法，结合对数极坐标下的傅里叶变换和互相关系数最大化策略，实现了高精度的尺度和旋转联合估计。

Method: 基于对数极坐标的傅里叶变换和辅助函数方法，通过子像素级互相关系数最大化完成估计。

Result: 实验表明，该方法在尺度和旋转估计上的均值误差低于传统基于离散互相关系数的傅里叶变换技术。

Insight: 该方法为图像对齐问题提供了一种高效且精确的解决方案，尤其在需要处理尺度和旋转变化的场景中表现优异。

Abstract: This paper introduces a highly efficient algorithm capable of jointly estimating scale and rotation between two images with sub-pixel precision. Image alignment serves as a critical process for spatially registering images captured from different viewpoints, and finds extensive use in domains such as medical imaging and computer vision. Traditional phase-correlation techniques are effective in determining translational shifts; however, they are inadequate when addressing scale and rotation changes, which often arise due to camera zooming or rotational movements. In this paper, we propose a novel algorithm that integrates scale and rotation estimation based on the Fourier transform in log-polar coordinates with a cross-correlation maximization strategy, leveraging the auxiliary function method. By incorporating sub-pixel-level cross-correlation our method enables precise estimation of both scale and rotation. Experimental results demonstrate that the proposed method achieves lower mean estimation errors for scale and rotation than conventional Fourier transform-based techniques that rely on discrete cross-correlation.

[112] Robust Object Detection for Autonomous Driving via Curriculum-Guided Group Relative Policy Optimization

Xu Jia

Main category: cs.CV

TL;DR: 本文提出了一种基于强化学习的框架，通过课程引导的Group Relative Policy Optimization（GRPO）方法，解决了自动驾驶中目标检测任务中的稀疏奖励和噪声问题，显著提升了检测精度和鲁棒性。

Details Motivation: 当前的多模态大语言模型（MLLMs）在视觉语言推理任务中表现优异，但在需要精确定位和鲁棒性的结构化感知任务中表现不佳。为解决这一问题，作者提出了一种结合课程学习和强化学习的优化方法。

Contribution: 主要贡献包括：（1）提出了结合课程学习和GRPO的强化学习框架；（2）通过难度感知过滤和课程数据调度，优化学习过程；（3）在自动驾驶基准测试中验证了方法的有效性。

Method: 方法的核心是Group Relative Policy Optimization（GRPO），结合课程学习和难度感知过滤。通过设计稀疏奖励和KL正则化，稳定优化过程并提升泛化能力。

Result: 在自动驾驶基准测试中，该方法显著提升了目标检测的准确性和鲁棒性。消融实验验证了奖励设计、KL正则化和课程调度的重要性。

Insight: 研究表明，结合结构化数据课程的强化学习是一种可扩展且鲁棒的方法，适用于复杂的多模态检测任务。

Abstract: Multimodal Large Language Models (MLLMs) excel in vision-language reasoning but often struggle with structured perception tasks requiring precise localization and robustness. We propose a reinforcement learning framework that augments Group Relative Policy Optimization (GRPO) with curriculum-based data scheduling and difficulty-aware filtering. This approach stabilizes optimization under sparse, noisy rewards and enables progressive adaptation to complex samples. Evaluations on autonomous driving benchmarks demonstrate substantial improvements in detection accuracy and robustness. Ablation studies confirm the importance of reward design, KL regularization, and curriculum pacing for convergence stability and generalization. Our findings highlight reinforcement-driven optimization with structured data curricula as a scalable path toward robust and interpretable multimodal detection.

[113] Graph-Theoretic Consistency for Robust and Topology-Aware Semi-Supervised Histopathology Segmentation

Ha-Hieu Pham,Minh Le,Han Huynh,Nguyen Quoc Khanh Le,Huy-Hieu Pham

Main category: cs.CV

TL;DR: 论文提出了一种拓扑图一致性（TGC）框架，通过整合图论约束，显著提升半监督病理分割的精度和拓扑有效性。

Details Motivation: 病理图像的分割需要密集标注，但成本高昂且数据有限。现有方法依赖像素级一致性，容易传播噪声伪标签并产生碎片化或不合理的拓扑结构。

Contribution: 提出TGC框架，通过对齐预测图与参考图的拉普拉斯谱、连通分量数和邻接统计量，强制全局拓扑一致性。

Method: 利用图论约束（拉普拉斯谱、连通分量数、邻接统计量）优化预测图与参考图的一致性，提升了分割的拓扑有效性。

Result: 在GlaS和CRAG数据集上，TGC在5-10%监督下达到SOTA性能，显著缩小了与全监督的差距。

Insight: 全局拓扑约束可以有效弥补半监督分割中像素级一致性的不足，显著提升分割质量和拓扑合理性。

Abstract: Semi-supervised semantic segmentation (SSSS) is vital in computational pathology, where dense annotations are costly and limited. Existing methods often rely on pixel-level consistency, which propagates noisy pseudo-labels and produces fragmented or topologically invalid masks. We propose Topology Graph Consistency (TGC), a framework that integrates graph-theoretic constraints by aligning Laplacian spectra, component counts, and adjacency statistics between prediction graphs and references. This enforces global topology and improves segmentation accuracy. Experiments on GlaS and CRAG demonstrate that TGC achieves state-of-the-art performance under 5-10% supervision and significantly narrows the gap to full supervision. Code is available at https://github.com/hieuphamha19/TGC.

[114] Sequential Token Merging: Revisiting Hidden States

Yan Wen,Peng Ye,Lin Zhang,Baopu Li,Jiakang Yuan,Yaoxin Yang,Tao Chen

Main category: cs.CV

TL;DR: 论文提出了一种称为Sequential Token Merging (STM)的方法，通过分析Vision Mambas (ViMs)中的Limited Directional Sequential Dependence (LDSD)机制，优化了隐藏状态的聚合方式，显著提升了模型效率，且精度损失极小。

Details Motivation: Vision Mambas虽然实现了次二次复杂度，但图像分辨率带来的二次复杂度问题仍限制了其效率。现有方法未能充分利用ViMs中的隐藏状态信息流机制。

Contribution: 1) 揭示了ViMs中的Limited Directional Sequential Dependence (LDSD)机制；2) 提出了Sequential Token Merging (STM)方法，通过双向最近邻合并和隐藏状态保护，显著提升效率。

Method: 1) 双向最近邻合并以保持序列依赖性；2) 隐藏状态保护机制以稳定类标记周围的隐藏状态。

Result: ViM-Ti在20%标记减少时精度仅下降1.0%，ViM-S在40%减少时仅下降1.4%，实现了高效的SOTA表现。

Insight: Mamba的选择性扫描机制可用于隐藏状态的渐进信息聚合，为状态空间模型动力学提供了新见解。

Abstract: Vision Mambas (ViMs) achieve remarkable success with sub-quadratic complexity, but their efficiency remains constrained by quadratic token scaling with image resolution. While existing methods address token redundancy, they overlook ViMs’ intrinsic Limited Directional Sequential Dependence (LDSD) - a critical information flow mechanism revealed in our analysis. We further identify Mamba’s selective scan enables gradual information aggregation in hidden states. Based on these insights, we propose Sequential Token Merging (STM), featuring: 1) Bidirectional nearest neighbor merging to preserve sequential dependencies through symmetric spatial aggregation, and 2) Hidden states protection to stabilize the hidden states around the class token. STM strategically leverages Mamba’s layer-wise loss convergence to convert temporal forgetfulness into stability. Experiments demonstrate STM’s superiority: 1.0% accuracy drop for ViM-Ti at 20% token reduction, and only 1.4% degradation for ViM-S at 40% reduction. Our method achieves state-of-the-art efficiency with minimal complexity, while providing new insights into state-space model dynamics. Codes will be released soon.

[115] Deep Learning Empowered Super-Resolution: A Comprehensive Survey and Future Prospects

Le Zhang,Ao Li,Qibin Hou,Ce Zhu,Yonina C. Eldar

Main category: cs.CV

TL;DR: 这篇论文是对深度学习驱动的超分辨率技术的全面综述，涵盖了单图像、视频、立体和光场超分辨率方法，分析了超过150种技术，并探讨了未来研究方向。

Details Motivation: 超分辨率技术在计算机视觉领域备受关注，现有综述多集中于特定领域，缺乏全面性。本文旨在填补这一空白，提供全面的技术分析和未来展望。

Contribution: 论文贡献包括：1）覆盖多种超分辨率任务的技术综述；2）超过150种方法的系统分类与分析；3）开放问题和未来方向的探讨；4）公开的数据资源库。

Method: 研究采用了系统综述的方法，对单图像超分辨率（SISR）、视频超分辨率（VSR）、立体超分辨率（SSR）和光场超分辨率（LFSR）进行了详细分类和技术分析，并基于不同目的进行结构化梳理。

Result: 论文总结了现有技术的优缺点，提供了方法论、数据集、评估协议和复杂性分析，同时提出了潜在研究方向和开放问题。

Insight: 超分辨率技术仍面临挑战，如复杂场景下的鲁棒性、计算效率与实际应用的平衡。未来的研究应关注多模态融合和轻量化设计。

Abstract: Super-resolution (SR) has garnered significant attention within the computer vision community, driven by advances in deep learning (DL) techniques and the growing demand for high-quality visual applications. With the expansion of this field, numerous surveys have emerged. Most existing surveys focus on specific domains, lacking a comprehensive overview of this field. Here, we present an in-depth review of diverse SR methods, encompassing single image super-resolution (SISR), video super-resolution (VSR), stereo super-resolution (SSR), and light field super-resolution (LFSR). We extensively cover over 150 SISR methods, nearly 70 VSR approaches, and approximately 30 techniques for SSR and LFSR. We analyze methodologies, datasets, evaluation protocols, empirical results, and complexity. In addition, we conducted a taxonomy based on each backbone structure according to the diverse purposes. We also explore valuable yet under-studied open issues in the field. We believe that this work will serve as a valuable resource and offer guidance to researchers in this domain. To facilitate access to related work, we created a dedicated repository available at https://github.com/AVC2-UESTC/Holistic-Super-Resolution-Review.

[116] Learning Hyperspectral Images with Curated Text Prompts for Efficient Multimodal Alignment

Abhiroop Chatterjee,Susmita Ghosh

Main category: cs.CV

TL;DR: 该论文提出了一种高效的多模态对齐方法，通过学习高光谱图像（HSI）并结合精心设计的文本提示，显著减少了参数更新量（仅0.07%），同时在高光谱场景理解任务中取得了最佳性能。

Details Motivation: 高光谱图像（HSI）的高维3D体素结构使得跨模态对齐成为一项具有挑战性的问题。尽管视觉和语言模型在自然图像或文本任务中表现良好，但在高光谱领域的跨模态对齐研究仍不足。本文旨在通过优化视觉语言模型（VLM）来解决这一问题。

Contribution: 1. 提出了一种基于CLIP风格的对比训练框架，用于高光谱图像与文本的多模态对齐；2. 引入了描述性文本提示作为结构化锚点，增强对齐效果；3. 仅更新总参数的0.07%，实现了高性能和小模型尺寸的双重优势。

Method: 1. 使用视觉主干网络提取HSI的体素级嵌入；2. 将这些嵌入映射到冻结的大型嵌入模型（LEM）的潜在空间；3. 通过对比损失对齐视觉特征和文本标记表示，仅限于精选的困难负样本和半困难负样本；4. 引入描述性提示以编码类别语义。

Result: 在Indian Pines（IP）数据集上，OA和Kappa分别提升了0.92和1.60；在Pavia University（PU）数据集上，分别提升了0.69和0.90。模型参数比基线方法DCTN少50倍，比SS-TMNet少90倍。

Insight: 通过精心设计的文本提示和对比学习，可以在极少量参数更新的情况下实现高效的多模态对齐，为高光谱图像理解提供了一种轻量化且高性能的解决方案。

Abstract: As data requirements continue to grow, efficient learning increasingly depends on the curation and distillation of high-value data rather than brute-force scaling of model sizes. In the case of a hyperspectral image (HSI), the challenge is amplified by the high-dimensional 3D voxel structure, where each spatial location is associated with hundreds of contiguous spectral channels. While vision and language models have been optimized effectively for natural image or text tasks, their cross-modal alignment in the hyperspectral domain remains an open and underexplored problem. In this article, we make an attempt to optimize a Vision-Language Model (VLM) for hyperspectral scene understanding by exploiting a CLIP-style contrastive training framework. Our framework maps voxel-level embeddings from a vision backbone onto the latent space of a frozen large embedding model (LEM), where a trainable probe aligns vision features with the model’s textual token representations. The two modalities are aligned via a contrastive loss restricted to a curated set of hard (closest wrong classes) and semi-hard (random distractors) negatives, along with positive pairs. To further enhance alignment, descriptive prompts that encode class semantics are introduced and act as structured anchors for the HSI embeddings. It is seen that the proposed method updates only 0.07 percent of the total parameters, yet yields state-of-the-art performance. For example, on Indian Pines (IP) the model produces better results over unimodal and multimodal baselines by +0.92 Overall Accuracy (OA) and +1.60 Kappa ($\kappa$), while on Pavia University (PU) data it provides gains of +0.69 OA and +0.90 $\kappa$. Moreover, this is achieved with the set of parameters, nearly 50$\times$ smaller than DCTN and 90$\times$ smaller than SS-TMNet.

[117] IBiT: Utilizing Inductive Biases to Create a More Data Efficient Attention Mechanism

Adithya Giri

Main category: cs.CV

TL;DR: IBiT通过引入归纳偏置的学习掩码，使视觉Transformer在小数据集上表现更好，同时保留了Transformer的可解释性。

Details Motivation: 尽管Transformer在计算机视觉中表现优异，但它们缺乏卷积神经网络的归纳偏置。本文旨在通过引入这些偏置，提升Transformer在小数据集上的性能。

Contribution: 提出了Inductively Biased Image Transformers (IBiT)，通过学习掩码引入归纳偏置，显著提升了Transformer在小数据集上的准确性。

Method: 设计了一种学习掩码机制，将卷积网络的归纳偏置引入视觉Transformer中，无需依赖知识蒸馏。

Result: IBiT在小数据集上表现优于传统Transformer，同时保持了可解释性。

Insight: 通过显式引入归纳偏置，可以弥补Transformer在小数据集上的不足，同时保留其优势。

Abstract: In recent years, Transformer-based architectures have become the dominant method for Computer Vision applications. While Transformers are explainable and scale well with dataset size, they lack the inductive biases of Convolutional Neural Networks. While these biases may be learned on large datasets, we show that introducing these inductive biases through learned masks allow Vision Transformers to learn on much smaller datasets without Knowledge Distillation. These Transformers, which we call Inductively Biased Image Transformers (IBiT), are significantly more accurate on small datasets, while retaining the explainability Transformers.

[118] LayoutAgent: A Vision-Language Agent Guided Compositional Diffusion for Spatial Layout Planning

Zezhong Fan,Xiaohan Li,Luyi Ma,Kai Zhao,Liang Peng,Topojoy Biswas,Evren Korpeoglu,Kaushiki Nag,Kannan Achan

Main category: cs.CV

TL;DR: LayoutAgent结合视觉语言推理与组合扩散方法，生成多目标场景的空间布局，解决了传统扩散模型缺乏空间推理能力的问题。

Details Motivation: 传统扩散模型在图像生成方面表现优异，但缺乏空间推理能力，导致生成的对象布局不现实；而传统空间规划方法虽强调几何一致性，却无法捕捉视觉场景的语义丰富性。因此，需要一种新方法，既能生成高质量图像，又能规划符合语义关系和物理合理性的空间布局。

Contribution: 提出了LayoutAgent框架，通过视觉语言模型预处理输入，结合组合扩散方法生成符合对象关系的空间布局，并利用前景条件图像生成器完成场景合成。

Method: 方法分为三步：1）视觉语言模型预处理输入（分割、对象大小估计、场景图构建和提示语重写）；2）组合扩散方法合成边界框，生成符合场景图的布局；3）前景条件图像生成器渲染对象到布局中。

Result: 实验证明，LayoutAgent在布局一致性、空间真实性和美学对齐方面优于其他现有布局生成模型。

Insight: LayoutAgent的创新在于将视觉语言推理与组合扩散方法结合，统一了语义丰富性与空间一致性，为多目标场景生成提供了新思路。

Abstract: Designing realistic multi-object scenes requires not only generating images, but also planning spatial layouts that respect semantic relations and physical plausibility. On one hand, while recent advances in diffusion models have enabled high-quality image generation, they lack explicit spatial reasoning, leading to unrealistic object layouts. On the other hand, traditional spatial planning methods in robotics emphasize geometric and relational consistency, but they struggle to capture semantic richness in visual scenes. To bridge this gap, in this paper, we propose LayoutAgent, an agentic framework that unifies vision-language reasoning with compositional diffusion for layout generation. Given multiple input images with target objects in them, our method first employs visual-language model to preprocess the inputs through segmentation, object size estimation, scene graph construction, and prompt rewriting. Then we leverage compositional diffusion-a method traditionally used in robotics-to synthesize bounding boxes that respect object relations encoded in the scene graph for spatial layouts. In the end, a foreground-conditioned image generator composes the complete scene by rendering the objects into the planned layout guided by designed prompts. Experiments demonstrate that LayoutAgent outperforms other state-of-the-art layout generation models in layout coherence, spatial realism and aesthetic alignment.

[119] CompareBench: A Benchmark for Visual Comparison Reasoning in Vision-Language Models

Jie Cai,Kangning Yang,Lan Fu,Jiaming Ding,Jinlong Li,Huiming Sun,Daitao Xing,Jinglin Shen,Zibo Meng

Main category: cs.CV

TL;DR: CompareBench是一个用于评估视觉-语言模型（VLMs）视觉比较推理能力的基准测试，包含1000个QA对，涵盖四个任务：数量、时间、几何和空间关系。测试发现，即使是先进模型也存在显著缺陷。

Details Motivation: 视觉比较推理是视觉-语言模型中的基础能力，但之前缺乏系统性评测。CompareBench填补了这一空白，旨在推动更可靠的多模态推理研究。

Contribution: 1. 提出了CompareBench基准测试；2. 构建了辅助数据集TallyBench和HistCaps；3. 评测了开源和闭源模型的能力局限性。

Method: 设计了四个任务（数量、时间、几何、空间），共计1000个QA对，并基于TallyBench和HistCaps数据集生成测试数据。评测了多种VLMs的表现。

Result: 所有模型在时间排序和空间关系任务上表现较差，且在基础数量和几何比较任务中也常犯错，显示当前模型在视觉比较推理中存在系统性盲点。

Insight: 视觉比较推理需要更多研究，尤其在时间动态性和空间关系建模方面，CompareBench为后续改进提供了诊断工具。

Abstract: We introduce CompareBench, a benchmark for evaluating visual comparison reasoning in vision-language models (VLMs), a fundamental yet understudied skill. CompareBench consists of 1000 QA pairs across four tasks: quantity (600), temporal (100), geometric (200), and spatial (100). It is derived from two auxiliary datasets that we constructed: TallyBench (2000 counting images with QA) and HistCaps (515 historical images with bilingual captions). We evaluate both closed-source APIs (OpenAI, Gemini, Claude) and open-source models (Qwen2.5-VL and Qwen3-VL series). Results show clear scaling trends but also reveal critical limitations: even the strongest models consistently fail at temporal ordering and spatial relations, and they often make mistakes in basic counting and geometric comparisons that are trivial for humans. These findings demonstrate that visual comparison remains a systematic blind spot for current VLMs. By providing controlled, diverse, and diagnostic evaluation, CompareBench establishes a foundation for advancing more reliable multimodal reasoning.

[120] MILR: Improving Multimodal Image Generation via Test-Time Latent Reasoning

Yapeng Mi,Hengli Li,Yanpeng Zhao,Chenxi Li,Huimin Wu,Xiaojian Ma,Song-Chun Zhu,Ying Nian Wu,Qing Li

Main category: cs.CV

TL;DR: MILR提出了一种在测试时联合推理图像和文本的统一潜在向量空间方法，显著提升了多模态图像生成的性能。

Details Motivation: 现有基于推理的图像生成方法通常局限于单模态或依赖高质量推理数据进行微调，限制了其性能和应用范围。

Contribution: MILR通过统一潜在空间的联合推理，首次实现了测试时的多模态（图像和文本）推理，无需依赖外部数据微调。

Method: MILR通过在离散图像和文本标记的向量表示中进行搜索，利用策略梯度方法和图像质量评价器实现推理，嵌入到MUG框架中。

Result: 在GenEval、T2I-CompBench和WISE基准测试中，MILR实现了SOTA性能，尤其在WISE上提升了80%。

Insight: 统一潜在空间的联合推理是关键优势，MILR还展现了在时间和文化推理上的强大能力。

Abstract: Reasoning-augmented machine learning systems have shown improved performance in various domains, including image generation. However, existing reasoning-based methods for image generation either restrict reasoning to a single modality (image or text) or rely on high-quality reasoning data for fine-tuning. To tackle these limitations, we propose MILR, a test-time method that jointly reasons over image and text in a unified latent vector space. Reasoning in MILR is performed by searching through vector representations of discrete image and text tokens. Practically, this is implemented via the policy gradient method, guided by an image quality critic. We instantiate MILR within the unified multimodal understanding and generation (MUG) framework that natively supports language reasoning before image synthesis and thus facilitates cross-modal reasoning. The intermediate model outputs, which are to be optimized, serve as the unified latent space, enabling MILR to operate entirely at test time. We evaluate MILR on GenEval, T2I-CompBench, and WISE, achieving state-of-the-art results on all benchmarks. Notably, on knowledge-intensive WISE, MILR attains an overall score of 0.63, improving over the baseline by 80%. Our further analysis indicates that joint reasoning in the unified latent space is the key to its strong performance. Moreover, our qualitative studies reveal MILR’s non-trivial ability in temporal and cultural reasoning, highlighting the efficacy of our reasoning method.

[121] VideoScore2: Think before You Score in Generative Video Evaluation

Xuan He,Dongfu Jiang,Ping Nie,Minghao Liu,Zhengxuan Jiang,Mingyi Su,Wentao Ma,Junru Lin,Chun Ye,Yi Lu,Keming Wu,Benjamin Schneider,Quy Duc Do,Zhuofeng Li,Yiming Jia,Yuxuan Zhang,Guo Cheng,Haozhe Wang,Wangchunshu Zhou,Qunshu Lin,Yuanxing Zhang,Ge Zhang,Wenhao Huang,Wenhu Chen

Main category: cs.CV

TL;DR: VideoScore2是一个多维度、可解释且与人类对齐的视频生成评估框架，专注于视觉质量、文本视频对齐和物理一致性，并通过详细的思考链提供评估依据。

Details Motivation: 现有视频评估方法多为单一不透明评分，缺乏可解释性或仅提供粗略分析，无法全面评估视频质量。

Contribution: 提出VideoScore2框架，包含三个维度评估（视觉质量、文本对齐、物理一致性）和详细的思考链；构建大规模数据集VideoFeedback2；采用两阶段训练方法（监督微调+强化学习）。

Method: 使用大规模数据集VideoFeedback2（27,168条人类标注视频），通过监督微调和GRPO强化学习两阶段训练模型。

Result: 在VideoScore-Bench-v2上达到44.35（+5.94）准确率，在四个跨域基准上平均提升4.32分。

Insight: VideoScore2的详细评估维度为可控视频生成提供有效奖励建模，并增强了评估与生成之间的桥梁。

Abstract: Recent advances in text-to-video generation have produced increasingly realistic and diverse content, yet evaluating such videos remains a fundamental challenge due to their multi-faceted nature encompassing visual quality, semantic alignment, and physical consistency. Existing evaluators and reward models are limited to single opaque scores, lack interpretability, or provide only coarse analysis, making them insufficient for capturing the comprehensive nature of video quality assessment. We present VideoScore2, a multi-dimensional, interpretable, and human-aligned framework that explicitly evaluates visual quality, text-to-video alignment, and physical/common-sense consistency while producing detailed chain-of-thought rationales. Our model is trained on a large-scale dataset VideoFeedback2 containing 27,168 human-annotated videos with both scores and reasoning traces across three dimensions, using a two-stage pipeline of supervised fine-tuning followed by reinforcement learning with Group Relative Policy Optimization (GRPO) to enhance analytical robustness. Extensive experiments demonstrate that VideoScore2 achieves superior performance with 44.35 (+5.94) accuracy on our in-domain benchmark VideoScore-Bench-v2 and 50.37 (+4.32) average performance across four out-of-domain benchmarks (VideoGenReward-Bench, VideoPhy2, etc), while providing interpretable assessments that bridge the gap between evaluation and controllable generation through effective reward modeling for Best-of-N sampling. Project Page: https://tiger-ai-lab.github.io/VideoScore2/

[122] TRUST: Test-Time Refinement using Uncertainty-Guided SSM Traverses

Sahar Dastani,Ali Bahri,Gustavo Adolfo Vargas Hakim,Moslem Yazdanpanah,Mehrdad Noori,David Osowiechi,Samuel Barbeau,Ismail Ben Ayed,Herve Lombaert,Christian Desrosiers

Main category: cs.CV

TL;DR: TRUST提出了一种新颖的测试时间适应方法，利用SSM的独特架构特性，通过多样化的遍历排列生成多视角输入，以伪标签指导参数更新，显著提升了模型的鲁棒性。

Details Motivation: State Space Models（SSMs）在视觉任务中表现出色，但在分布偏移下泛化性能显著下降。TRUST旨在利用SSMs的独特结构特性，通过测试时间适应提升模型鲁棒性。

Contribution: TRUST是首个利用SSMs架构特性进行测试时间适应的方法，提出了不确定性指导的遍历策略和多视角伪标签更新机制。

Method: TRUST通过多样化的SSM遍历排列生成多视角输入，利用模型预测作为伪标签更新Mamba专用参数，并通过权重平均整合学习信息。

Result: 在七个基准测试中，TRUST显著提升了模型的鲁棒性，并优于现有的TTA方法。

Insight: TRUST展示了SSMs在测试时间适应中的潜力，通过其独特的结构特性（如遍历排列）提升模型对分布偏移的适应能力。

Abstract: State Space Models (SSMs) have emerged as efficient alternatives to Vision Transformers (ViTs), with VMamba standing out as a pioneering architecture designed for vision tasks. However, their generalization performance degrades significantly under distribution shifts. To address this limitation, we propose TRUST (Test-Time Refinement using Uncertainty-Guided SSM Traverses), a novel test-time adaptation (TTA) method that leverages diverse traversal permutations to generate multiple causal perspectives of the input image. Model predictions serve as pseudo-labels to guide updates of the Mamba-specific parameters, and the adapted weights are averaged to integrate the learned information across traversal scans. Altogether, TRUST is the first approach that explicitly leverages the unique architectural properties of SSMs for adaptation. Experiments on seven benchmarks show that TRUST consistently improves robustness and outperforms existing TTA methods.

[123] MMPB: It’s Time for Multi-Modal Personalization

Jaeik Kim,Woojin Kim,Woohyeon Park,Jaeyoung Do

Main category: cs.CV

TL;DR: MMPB是一个用于评估视觉语言模型（VLM）个性化能力的首个大规模基准测试，包含10k图像-查询对和111个个性化概念。研究发现现有VLM在个性化任务中表现不佳，尤其是在对话一致性和视觉适应能力方面。

Details Motivation: 个性化在用户导向的AI系统中至关重要，但现有VLM在这方面未得到充分探索。通过MMPB基准测试，填补了这一研究空白。

Contribution: 提出了首个全面的VLM个性化基准测试MMPB，并揭示了现有模型在个性化任务中的主要挑战和改进空间。

Method: 设计了三个任务类型评估VLM的个性化能力，并采用概念注入、多轮对话和个性化查询的三阶段评估协议。

Result: 实验显示大多数VLM（包括闭源模型）在个性化任务中表现不佳，特别是在对话一致性和用户偏好处理方面存在显著问题。

Insight: MMPB不仅提供了可扩展的基准测试，还为未来实现真正个性化的多模态AI指明了方向。

Abstract: Visual personalization is essential in user-facing AI systems such as smart homes and healthcare, where aligning model behavior with user-centric concepts is critical. However, recent large Vision-Language Models (VLMs), despite their broad applicability, remain underexplored in their ability to adapt to individual users. In this paper, we introduce MMPB, the first extensive benchmark for evaluating VLMs on personalization. MMPB comprises 10k image-query pairs and includes 111 personalizable concepts across four categories: humans, animals, objects, and characters, with the human category enriched with preference-grounded queries. We structure personalization into three main task types, each highlighting a different key property of VLMs. Using 23 widely used VLMs including both open- and closed-source models, we evaluate personalization performance via a three-stage protocol: concept injection, multi-turn dialogue, and personalized querying. Our findings indicate that most VLMs (including some closed-source models) struggle with personalization, particularly in maintaining consistency over dialogue, handling user preferences, and adapting to visual cues. Our analysis reveals that the challenges in VLM personalization (such as refusal behaviors and long-context forgetting) highlight substantial room for improvement. By identifying these limitations and offering a scalable benchmark, MMPB offers valuable insights and a solid foundation for future research toward truly personalized multi-modal AI. Project Page: aidaslab.github.io/MMPB

[124] Seeing Isn’t Believing: Context-Aware Adversarial Patch Synthesis via Conditional GAN

Roie Kazoom,Alon Goldberg,Hodaya Cohen,Ofer Hadar

Main category: cs.CV

TL;DR: 该论文提出了一种条件GAN框架，用于合成上下文感知的对抗性补丁，能够在指定目标类别的同时保持视觉真实性和黑盒攻击的有效性。

Details Motivation: 现有的对抗性补丁攻击方法多依赖于不切实际的白盒假设、非针对性目标，或生成视觉上显眼的补丁，限制了实际应用。

Contribution: 提出了首个全可控的对抗性补丁生成框架，支持目标类别的自由选择，并结合Grad-CAM引导的补丁布局，实现了语义感知的定位。

Method: 采用生成式U-Net设计和Grad-CAM引导的补丁放置，结合条件GAN生成具有上下文感知的对抗性补丁。

Result: 在多种卷积网络和视觉Transformer上实现了SOTA性能，攻击成功率（ASR）和目标类别成功率（TCS）均超过99%。

Insight: 该方法在视觉真实性、目标控制和黑盒适用性三方面取得平衡，为对抗性鲁棒性研究树立了新标杆。

Abstract: Adversarial patch attacks pose a severe threat to deep neural networks, yet most existing approaches rely on unrealistic white-box assumptions, untargeted objectives, or produce visually conspicuous patches that limit real-world applicability. In this work, we introduce a novel framework for fully controllable adversarial patch generation, where the attacker can freely choose both the input image x and the target class y target, thereby dictating the exact misclassification outcome. Our method combines a generative U-Net design with Grad-CAM-guided patch placement, enabling semantic-aware localization that maximizes attack effectiveness while preserving visual realism. Extensive experiments across convolutional networks (DenseNet-121, ResNet-50) and vision transformers (ViT-B/16, Swin-B/16, among others) demonstrate that our approach achieves state-of-the-art performance across all settings, with attack success rates (ASR) and target-class success (TCS) consistently exceeding 99%. Importantly, we show that our method not only outperforms prior white-box attacks and untargeted baselines, but also surpasses existing non-realistic approaches that produce detectable artifacts. By simultaneously ensuring realism, targeted control, and black-box applicability-the three most challenging dimensions of patch-based attacks-our framework establishes a new benchmark for adversarial robustness research, bridging the gap between theoretical attack strength and practical stealthiness.

[125] Learning Temporal Saliency for Time Series Forecasting with Cross-Scale Attention

Ibrahim Delibasoglu,Fredrik Heintz

Main category: cs.CV

TL;DR: 本文提出了CrossScaleNet，一种结合跨尺度注意力机制的创新架构，用于时间序列预测和时间显著性检测，同时提升模型的可解释性和预测性能。

Details Motivation: 时间序列预测中可解释性的缺失限制了模型的透明度和决策支持能力。传统的时间显著性检测方法计算成本高且效果不佳，亟需一种高效且透明的解决方案。

Contribution: 提出了CrossScaleNet，结合跨尺度注意力机制和多尺度处理，实现了高性能和时间显著性检测的可解释性；实验表明其在预测和显著性检测方面均优于现有方法。

Method: 采用基于补丁（patch-based）的跨注意力机制和多尺度处理，将注意力嵌入训练过程，提供内在的时间显著性解释。

Result: 在合成数据集和公开基准上验证了模型的鲁棒性，并在真实数据集上超过了大多数基于Transformer的模型，同时保持了预测准确性和更好的可解释性。

Insight: 现有的可解释性模型往往在性能上表现不佳，而CrossScaleNet成功填补了这一空白，实现了性能和可解释性的平衡。

Abstract: Explainability in time series forecasting is essential for improving model transparency and supporting informed decision-making. In this work, we present CrossScaleNet, an innovative architecture that combines a patch-based cross-attention mechanism with multi-scale processing to achieve both high performance and enhanced temporal explainability. By embedding attention mechanisms into the training process, our model provides intrinsic explainability for temporal saliency, making its decision-making process more transparent. Traditional post-hoc methods for temporal saliency detection are computationally expensive, particularly when compared to feature importance detection. While ablation techniques may suffice for datasets with fewer features, identifying temporal saliency poses greater challenges due to its complexity. We validate CrossScaleNet on synthetic datasets with known saliency ground truth and on established public benchmarks, demonstrating the robustness of our method in identifying temporal saliency. Experiments on real-world datasets for forecasting task show that our approach consistently outperforms most transformer-based models, offering better explainability without sacrificing predictive accuracy. Our evaluations demonstrate superior performance in both temporal saliency detection and forecasting accuracy. Moreover, we highlight that existing models claiming explainability often fail to maintain strong performance on standard benchmarks. CrossScaleNet addresses this gap, offering a balanced approach that captures temporal saliency effectively while delivering state-of-the-art forecasting performance across datasets of varying complexity.

[126] Multimodal Slice Interaction Network Enhanced by Transfer Learning for Precise Segmentation of Internal Gross Tumor Volume in Lung Cancer PET/CT Imaging

Yi Luo,Yike Guo,Hamed Hooshangnejad,Rui Zhang,Xue Feng,Quan Chen,Wil Ngwa,Kai Ding

Main category: cs.CV

TL;DR: 该论文提出了一种基于迁移学习的多模态交互感知网络方法，结合切片交互模块（SIM），用于在PET/CT影像中精确分割肺癌的内部总肿瘤体积（IGTV），显著提升了分割性能。

Details Motivation: 肺癌是全球癌症相关死亡的主要原因，PET/CT影像中IGTV的精确分割对放射治疗至关重要，但受限于标注数据稀缺和肿瘤边界PET信号弱的问题。

Contribution: 1）提出了一种迁移学习框架，利用大规模GTV数据集预训练模型；2）设计了切片交互模块（SIM）以建模切片间依赖关系；3）在私有IGTV数据集上取得了显著优于基线的分割效果。

Method: 采用预训练的MAMBA网络，结合2.5D分割框架和SIM模块，SIM通过通道与空间注意力分支及深度卷积建模切片间依赖关系。

Result: 在私有IGTV数据集上，Dice分数达到0.609，显著超过传统基线方法的0.385。

Insight: 迁移学习和多模态交互的有效结合能够显著提升IGTV分割的精度，为临床放射治疗计划提供了更可靠的工具。

Abstract: Lung cancer remains the leading cause of cancerrelated deaths globally. Accurate delineation of internal gross tumor volume (IGTV) in PET/CT imaging is pivotal for optimal radiation therapy in mobile tumors such as lung cancer to account for tumor motion, yet is hindered by the limited availability of annotated IGTV datasets and attenuated PET signal intensity at tumor boundaries. In this study, we present a transfer learningbased methodology utilizing a multimodal interactive perception network with MAMBA, pre-trained on extensive gross tumor volume (GTV) datasets and subsequently fine-tuned on a private IGTV cohort. This cohort constitutes the PET/CT subset of the Lung-cancer Unified Cross-modal Imaging Dataset (LUCID). To further address the challenge of weak PET intensities in IGTV peripheral slices, we introduce a slice interaction module (SIM) within a 2.5D segmentation framework to effectively model inter-slice relationships. Our proposed module integrates channel and spatial attention branches with depthwise convolutions, enabling more robust learning of slice-to-slice dependencies and thereby improving overall segmentation performance. A comprehensive experimental evaluation demonstrates that our approach achieves a Dice of 0.609 on the private IGTV dataset, substantially surpassing the conventional baseline score of 0.385. This work highlights the potential of transfer learning, coupled with advanced multimodal techniques and a SIM to enhance the reliability and clinical relevance of IGTV segmentation for lung cancer radiation therapy planning.

[127] ControlEvents: Controllable Synthesis of Event Camera Datawith Foundational Prior from Image Diffusion Models

Yixuan Hu,Yuxuan Xue,Simon Klenk,Daniel Cremers,Gerard Pons-Moll

Main category: cs.CV

TL;DR: ControlEvents提出了一种基于扩散模型的生成方法，用于合成高质量的事件相机数据，通过利用Stable Diffusion的先验知识，显著降低了标记数据的成本，并展示了在多个任务中的有效性。

Details Motivation: 事件相机因其高时间分辨率和高动态范围特性受到关注，但获取大规模标记数据成本高昂且困难。本文旨在通过生成合成数据解决这一问题。

Contribution: 1. 提出了ControlEvents，一种基于扩散模型的生成方法，支持多样化的控制信号（如文本标签、2D骨架和3D姿态）生成事件数据。2. 利用Stable Diffusion的先验知识，仅需少量标记数据和微调即可生成高质量数据。3. 展示了生成数据在多个任务中提升模型性能的能力，并具备基于未见文本标签的生成能力。

Method: 基于扩散模型的方法，结合Stable Diffusion的先验知识，通过多样化的控制信号（文本标签、2D骨架、3D姿态）生成事件数据。利用少量标记数据和最小化微调实现高质量生成。

Result: 生成的合成数据在视觉识别、2D骨架估计和3D姿态估计任务中均提升了模型性能。同时展示了基于未见文本标签生成事件的能力。

Insight: 利用基础模型的扩散先验知识可以有效降低事件相机数据生成的成本和难度，同时通过多样化控制信号实现灵活的数据生成。

Abstract: In recent years, event cameras have gained significant attention due to their bio-inspired properties, such as high temporal resolution and high dynamic range. However, obtaining large-scale labeled ground-truth data for event-based vision tasks remains challenging and costly. In this paper, we present ControlEvents, a diffusion-based generative model designed to synthesize high-quality event data guided by diverse control signals such as class text labels, 2D skeletons, and 3D body poses. Our key insight is to leverage the diffusion prior from foundation models, such as Stable Diffusion, enabling high-quality event data generation with minimal fine-tuning and limited labeled data. Our method streamlines the data generation process and significantly reduces the cost of producing labeled event datasets. We demonstrate the effectiveness of our approach by synthesizing event data for visual recognition, 2D skeleton estimation, and 3D body pose estimation. Our experiments show that the synthesized labeled event data enhances model performance in all tasks. Additionally, our approach can generate events based on unseen text labels during training, illustrating the powerful text-based generation capabilities inherited from foundation models.

[128] Learning KAN-based Implicit Neural Representations for Deformable Image Registration

Nikita Drozdov,Marat Zinovev,Dmitry Sorokin

Main category: cs.CV

TL;DR: 该论文提出了KAN-IDIR和RandKAN-IDIR，首次将Kolmogorov-Arnold Networks (KANs)与隐式神经表示(INRs)相结合，用于变形图像配准。通过随机基函数采样策略，降低了计算成本，同时保持了配准质量，并在多种数据集上表现出优异的精度和学习稳定性。

Details Motivation: 传统学习方法的变形图像配准需要大量训练数据且在精度上难以匹敌经典迭代方法，而隐式神经表示(INRs)虽具备潜力但计算效率和学习稳定性不足。论文旨在通过KANs与INRs的结合解决这些问题。

Contribution: 1. 首次将KANs引入INRs用于变形图像配准；2. 提出随机基函数采样策略，降低计算成本；3. 在多模态数据集上验证了方法的优越性和稳定性。

Method: 采用Kolmogorov-Arnold Networks (KANs)与隐式神经表示(INRs)结合，并通过随机基函数采样减少基函数数量，降低计算复杂度。

Result: KAN-IDIR和RandKAN-IDIR在所有评估模态和解剖结构中均达到INR方法中最高的配准精度，计算开销低且学习稳定性强。RandKAN-IDIR的表现略优于可学习基函数索引的模型。

Insight: 随机基函数采样策略不仅能降低成本，还能略微提升性能，揭示了学习稳定性与计算效率的平衡可能成为未来INR研究的关键方向。

Abstract: Deformable image registration (DIR) is a cornerstone of medical image analysis, enabling spatial alignment for tasks like comparative studies and multi-modal fusion. While learning-based methods (e.g., CNNs, transformers) offer fast inference, they often require large training datasets and struggle to match the precision of classical iterative approaches on some organ types and imaging modalities. Implicit neural representations (INRs) have emerged as a promising alternative, parameterizing deformations as continuous mappings from coordinates to displacement vectors. However, this comes at the cost of requiring instance-specific optimization, making computational efficiency and seed-dependent learning stability critical factors for these methods. In this work, we propose KAN-IDIR and RandKAN-IDIR, the first integration of Kolmogorov-Arnold Networks (KANs) into deformable image registration with implicit neural representations (INRs). Our proposed randomized basis sampling strategy reduces the required number of basis functions in KAN while maintaining registration quality, thereby significantly lowering computational costs. We evaluated our approach on three diverse datasets (lung CT, brain MRI, cardiac MRI) and compared it with competing instance-specific learning-based approaches, dataset-trained deep learning models, and classical registration approaches. KAN-IDIR and RandKAN-IDIR achieved the highest accuracy among INR-based methods across all evaluated modalities and anatomies, with minimal computational overhead and superior learning stability across multiple random seeds. Additionally, we discovered that our RandKAN-IDIR model with randomized basis sampling slightly outperforms the model with learnable basis function indices, while eliminating its additional training-time complexity.

[129] Convolutional Set Transformer

Federico Chinello,Giacomo Boracchi

Main category: cs.CV

TL;DR: 论文提出了卷积集变换器（CST），一种直接处理3D图像张量的神经网络架构，避免了传统方法需要先通过CNN提取特征的局限，同时在特征提取和上下文建模中实现了协同作用。

Details Motivation: 现有集合输入网络（如Deep Sets和Set Transformer）仅支持向量输入，无法直接处理3D图像张量，需要额外借助CNN提取特征。CST旨在解决这一问题，直接处理图像集合并实现更高性能。

Contribution: 1. 提出CST，支持直接处理3D图像张量；2. 实现特征提取与上下文建模的协同；3. 支持CNN解释方法（如Grad-CAM）；4. 开源预训练模型CST-15。

Method: CST结合卷积操作与集合变换器，直接在图像张量上操作，同时完成特征提取和集合内关系建模。模型支持预训练和迁移学习。

Result: 实验表明，CST在集合分类和集合异常检测任务中性能优于现有方法，且支持模型解释。

Insight: CST的设计避免了传统方法的分步处理，展现了端到端的优势；其兼容解释方法的特点提升了模型透明性。

Abstract: We introduce the Convolutional Set Transformer (CST), a novel neural architecture designed to process image sets of arbitrary cardinality that are visually heterogeneous yet share high-level semantics - such as a common category, scene, or concept. Existing set-input networks, e.g., Deep Sets and Set Transformer, are limited to vector inputs and cannot directly handle 3D image tensors. As a result, they must be cascaded with a feature extractor, typically a CNN, which encodes images into embeddings before the set-input network can model inter-image relationships. In contrast, CST operates directly on 3D image tensors, performing feature extraction and contextual modeling simultaneously, thereby enabling synergies between the two processes. This design yields superior performance in tasks such as Set Classification and Set Anomaly Detection and further provides native compatibility with CNN explainability methods such as Grad-CAM, unlike competing approaches that remain opaque. Finally, we show that CSTs can be pre-trained on large-scale datasets and subsequently adapted to new domains and tasks through standard Transfer Learning schemes. To support further research, we release CST-15, a CST backbone pre-trained on ImageNet (https://github.com/chinefed/convolutional-set-transformer).

[130] TY-RIST: Tactical YOLO Tricks for Real-time Infrared Small Target Detection

Abdulkarim Atrash,Omar Moured,Yufan Chen,Jiaming Zhang,Seyda Ertekin,Omur Ugur

Main category: cs.CV

TL;DR: TY-RIST 是一种优化的 YOLOv12n 架构，专注于实时红外小目标检测，通过改进主干网络、检测头、注意力模块和剪枝策略，显著提升了检测性能和计算效率。

Details Motivation: 红外小目标检测在国防和监控中至关重要，但面临目标特征少、环境噪声多、检测遗漏和计算成本高的挑战。

Contribution: 提出了 TY-RIST，改进了 YOLOv12n 架构，引入步幅感知主干、高分辨率检测头、级联坐标注意力块和剪枝策略，同时提出 NWD 增强回归稳定性。

Method: 结合步幅感知主干和高分辨率检测头优化特征提取，使用级联坐标注意力提升目标定位能力，通过剪枝减少计算成本。

Result: 在四个基准测试中，mAP@0.5IoU 提升 7.9%，精确度提升 3%，召回率提升 10.2%，同时达到 123 FPS 的实时性能。

Insight: 剪枝策略在显著降低计算成本的同时略微提升精度，表明模型优化可以在速度和性能之间取得平衡。

Abstract: Infrared small target detection (IRSTD) is critical for defense and surveillance but remains challenging due to (1) target loss from minimal features, (2) false alarms in cluttered environments, (3) missed detections from low saliency, and (4) high computational costs. To address these issues, we propose TY-RIST, an optimized YOLOv12n architecture that integrates (1) a stride-aware backbone with fine-grained receptive fields, (2) a high-resolution detection head, (3) cascaded coordinate attention blocks, and (4) a branch pruning strategy that reduces computational cost by about 25.5% while marginally improving accuracy and enabling real-time inference. We also incorporate the Normalized Gaussian Wasserstein Distance (NWD) to enhance regression stability. Extensive experiments on four benchmarks and across 20 different models demonstrate state-of-the-art performance, improving mAP at 0.5 IoU by +7.9%, Precision by +3%, and Recall by +10.2%, while achieving up to 123 FPS on a single GPU. Cross-dataset validation on a fifth dataset further confirms strong generalization capability. Additional results and resources are available at https://www.github.com/moured/TY-RIST

[131] FishAI 2.0: Marine Fish Image Classification with Multi-modal Few-shot Learning

Chenghan Yang,Peng Zhou,Dong-Sheng Zhang,Yueyun Wang,Hong-Bin Shen,Xiaoyong Pan

Main category: cs.CV

TL;DR: FishAI 2.0结合多模态少样本学习与图像生成技术，提出了一种海洋鱼类智能识别框架，解决了传统方法在数据稀缺条件下的性能问题，并通过实验验证了其高效性和实用性。

Details Motivation: 传统海洋生物图像识别在数据稀缺条件下（尤其是稀有物种）表现不佳，为解决这一问题，FishAI 2.0融合多模态学习和图像生成技术。

Contribution: 1. 提出FishAI 2.0框架，结合多模态少样本学习；2. 利用DeepSeek和Stable Diffusion 2生成高质量图像以增强数据；3. 在多个分类级别上显著提升了识别准确率。

Method: 1. 构建分层海洋鱼类基准数据集；2. 使用DeepSeek生成文本描述，并用Stable Diffusion 2进行图像增强；3. 训练基于CLIP的模型实现少样本识别。

Result: 在家、属、种级别上，Top-1准确率分别达到91.67%、87.58%和85.42%，显著优于基线模型。

Insight: 多模态学习和数据增强可有效解决少样本学习中的数据稀缺问题，为生态监测提供了可扩展的技术方案。

Abstract: Traditional marine biological image recognition faces challenges of incomplete datasets and unsatisfactory model accuracy, particularly for few-shot conditions of rare species where data scarcity significantly hampers the performance. To address these issues, this study proposes an intelligent marine fish recognition framework, FishAI 2.0, integrating multimodal few-shot deep learning techniques with image generation for data augmentation. First, a hierarchical marine fish benchmark dataset, which provides a comprehensive data foundation for subsequent model training, is utilized to train the FishAI 2.0 model. To address the data scarcity of rare classes, the large language model DeepSeek was employed to generate high-quality textual descriptions, which are input into Stable Diffusion 2 for image augmentation through a hierarchical diffusion strategy that extracts latent encoding to construct a multimodal feature space. The enhanced visual-textual datasets were then fed into a Contrastive Language-Image Pre-Training (CLIP) based model, enabling robust few-shot image recognition. Experimental results demonstrate that FishAI 2.0 achieves a Top-1 accuracy of 91.67 percent and Top-5 accuracy of 97.97 percent at the family level, outperforming baseline CLIP and ViT models with a substantial margin for the minority classes with fewer than 10 training samples. To better apply FishAI 2.0 to real-world scenarios, at the genus and species level, FishAI 2.0 respectively achieves a Top-1 accuracy of 87.58 percent and 85.42 percent, demonstrating practical utility. In summary, FishAI 2.0 improves the efficiency and accuracy of marine fish identification and provides a scalable technical solution for marine ecological monitoring and conservation, highlighting its scientific value and practical applicability.

[132] Brain Tumor Classification from MRI Scans via Transfer Learning and Enhanced Feature Representation

Ahta-Shamul Hoque Emran,Hafija Akter,Abdullah Al Shiam,Abu Saleh Musa Miah,Anichur Rahman,Fahmid Al Farid,Hezerul Abdul Karim

Main category: cs.CV

TL;DR: 该论文提出了一个基于迁移学习的深度学习框架，用于从MRI扫描中自动检测脑肿瘤，并引入了新的Dense-Dropout模块以增强特征学习。同时，作者贡献了一个新的脑肿瘤MRI数据集（MMCB T）。

Details Motivation: 脑肿瘤的及时检测对患者治疗至关重要，但现有数据集可靠性不足。本研究旨在提供一个高效、自动化的检测框架，并填补数据空白。

Contribution: 1. 提出了一种结合ResNet50特征提取和Dense-Dropout序列的深度学习框架，增强非线性特征学习并减少过拟合。2. 发布了MMCBT数据集，包含临床验证的MRI扫描图像，解决了数据稀缺问题。

Method: 1. 使用预训练的ResNet50进行特征提取，通过GAP和线性投影获取紧凑的高层图像表示。2. 设计了Dense-Dropout序列，通过多样化特征变换提升鲁棒性。3. 采用数据增强平衡数据集。

Result: 该方法通过增强的特征表示和数据平衡优化了脑肿瘤分类性能。

Insight: 迁移学习结合新型Dropout模块能有效提升医学图像分类任务的效果。高质量、平衡的数据集对模型性能至关重要。

Abstract: Brain tumors are abnormal cell growths in the central nervous system (CNS), and their timely detection is critical for improving patient outcomes. This paper proposes an automatic and efficient deep-learning framework for brain tumor detection from magnetic resonance imaging (MRI) scans. The framework employs a pre-trained ResNet50 model for feature extraction, followed by Global Average Pooling (GAP) and linear projection to obtain compact, high-level image representations. These features are then processed by a novel Dense-Dropout sequence, a core contribution of this work, which enhances non-linear feature learning, reduces overfitting, and improves robustness through diverse feature transformations. Another major contribution is the creation of the Mymensingh Medical College Brain Tumor (MMCBT) dataset, designed to address the lack of reliable brain tumor MRI resources. The dataset comprises MRI scans from 209 subjects (ages 9 to 65), including 3671 tumor and 13273 non-tumor images, all clinically verified under expert supervision. To overcome class imbalance, the tumor class was augmented, resulting in a balanced dataset well-suited for deep learning research.

[133] ARSS: Taming Decoder-only Autoregressive Visual Generation for View Synthesis From Single View

Wenbin Teng,Gonglin Chen,Haiwei Chen,Yajie Zhao

Main category: cs.CV

TL;DR: ARSS是一个基于解码器-自回归模型的框架，用于从单一图像生成新视角，解决了扩散模型在多视角一致性上的不足。

Details Motivation: 扩散模型在生成质量上表现出色，但在世界建模任务（如稀疏输入的新视角生成）中存在非因果生成的缺点，导致视角间的不一致性。AR模型通过因果生成更适合此类任务。

Contribution: 提出ARSS框架，利用GPT风格的自回归模型生成新视角；设计了视频分词器和相机编码器，并提出空间随机排列的自回归变换模块，以提升生成质量。

Method: 使用视频分词器将连续图像序列转换为离散token；相机编码器将相机轨迹转化为3D位置引导；通过空间随机排列的自回归变换模块增强一致性。

Result: 在公开数据集上，ARSS的性能与或优于基于扩散模型的SOTA视角合成方法。

Insight: 因果自回归模型在视角合成任务中具有潜在优势，尤其是在需要视角一致性的场景下，可以通过结构调整提升生成质量。

Abstract: Despite their exceptional generative quality, diffusion models have limited applicability to world modeling tasks, such as novel view generation from sparse inputs. This limitation arises because diffusion models generate outputs in a non-causal manner, often leading to distortions or inconsistencies across views, and making it difficult to incrementally adapt accumulated knowledge to new queries. In contrast, autoregressive (AR) models operate in a causal fashion, generating each token based on all previously generated tokens. In this work, we introduce \textbf{ARSS}, a novel framework that leverages a GPT-style decoder-only AR model to generate novel views from a single image, conditioned on a predefined camera trajectory. We employ a video tokenizer to map continuous image sequences into discrete tokens and propose a camera encoder that converts camera trajectories into 3D positional guidance. Then to enhance generation quality while preserving the autoregressive structure, we propose a autoregressive transformer module that randomly permutes the spatial order of tokens while maintaining their temporal order. Extensive qualitative and quantitative experiments on public datasets demonstrate that our method performs comparably to, or better than, state-of-the-art view synthesis approaches based on diffusion models. Our code will be released upon paper acceptance.

[134] Disentangling Static and Dynamic Information for Reducing Static Bias in Action Recognition

Masato Kobayashi,Ning Ding,Toru Tamaki

Main category: cs.CV

TL;DR: 该论文提出了一种通过分离静态场景信息和动态时间信息来减少动作识别中静态偏置的方法，结合统计独立性损失和场景预测损失，实验证明其有效性。

Details Motivation: 动作识别模型过度依赖静态线索而非动态人体运动（静态偏置），这在实际应用和零样本动作识别中表现不佳，因此需要减少这种偏置。

Contribution: 提出了一种分离静态和动态信息的方法，通过统计独立性损失和场景预测损失减少静态偏置，提升动作识别性能。

Method: 使用统计独立性损失分离偏向静态信息的流和动态信息的流，并结合场景预测损失，确保静态信息不被过度依赖。

Result: 实验表明该方法能有效减少静态偏置，并验证了场景预测损失的重要性。

Insight: 动作识别中，分离静态和动态信息是关键，场景预测损失有助于防止模型过度依赖静态线索。

Abstract: Action recognition models rely excessively on static cues rather than dynamic human motion, which is known as static bias. This bias leads to poor performance in real-world applications and zero-shot action recognition. In this paper, we propose a method to reduce static bias by separating temporal dynamic information from static scene information. Our approach uses a statistical independence loss between biased and unbiased streams, combined with a scene prediction loss. Our experiments demonstrate that this method effectively reduces static bias and confirm the importance of scene prediction loss.

[135] Desensitizing for Improving Corruption Robustness in Point Cloud Classification through Adversarial Training

Zhiqiang Tian,Weigang Li,Chunhua Deng,Junwei Hu,Yongqiang Wang,Wenping Liu

Main category: cs.CV

TL;DR: 论文提出了一种名为Desensitized Adversarial Training（DesenAT）的方法，通过特征脱敏和自蒸馏框架提升点云分类任务中模型的鲁棒性。

Details Motivation: 点云数据因场景复杂性、传感器不精确性和处理误差不可避免地存在噪声。传统方法训练的DNN对点云特征过度依赖，导致模型在噪声数据上表现不佳。研究尝试量化这种依赖并探索降低依赖是否可以提高模型鲁棒性。

Contribution: 1. 量化了DNN对点云特征的敏感度，发现高敏感特征对模型性能影响更大；2. 提出DesenAT方法，通过特征脱敏对抗训练和自蒸馏框架减少对特征的依赖。

Method: 1. 使用Shapley值量化特征敏感度，生成对抗样本；2. 通过自蒸馏框架补偿对抗样本的信息损失，提升模型鲁棒性。

Result: 在ModelNet-C和PointCloud-C数据集上的实验表明，DesenAT在不降低干净数据集性能的情况下显著提升了模型鲁棒性。

Insight: 减少DNN对特定特征的过度依赖可以有效提升其在噪声数据上的表现，自蒸馏框架是平衡鲁棒性和性能的有效手段。

Abstract: Due to scene complexity, sensor inaccuracies, and processing imprecision, point cloud corruption is inevitable. Over-reliance on input features is the root cause of DNN vulnerabilities. It remains unclear whether this issue exists in 3D tasks involving point clouds and whether reducing dependence on these features can enhance the model’s robustness to corrupted point clouds. This study attempts to answer these questions. Specifically, we quantified the sensitivity of the DNN to point cloud features using Shapley values and found that models trained using traditional methods exhibited high sensitivity values for certain features. Furthermore, under an equal pruning ratio, prioritizing the pruning of highly sensitive features causes more severe damage to model performance than random pruning. We propose `Desensitized Adversarial Training’ (DesenAT), generating adversarial samples using feature desensitization and conducting training within a self-distillation framework, which aims to alleviate DNN’s over-reliance on point clouds features by smoothing sensitivity. First, data points with high contribution components are eliminated, and spatial transformation is used to simulate corruption scenes, generate adversarial samples, and conduct adversarial training on the model. Next, to compensate for information loss in adversarial samples, we use the self-distillation method to transfer knowledge from clean samples to adversarial samples, and perform adversarial training in a distillation manner.Extensive experiments on ModelNet-C and PointCloud-C demonstrate show that the propose method can effectively improve the robustness of the model without reducing the performance of clean data sets. This code is publicly available at \href{https://github.com/JerkyT/DesenAT/tree/master}{https://github.com/JerkyT/DesenAT}.

[136] Geometry-Aware Losses for Structure-Preserving Text-to-Sign Language Generation

Zetian Wu,Tianshuo Zhou,Stefan Lee,Liang Huang

Main category: cs.CV

TL;DR: 该论文提出了一种基于几何约束的方法，用于从文本生成手语的视频，重点解决传统方法忽略人体解剖学约束的问题，显著提升了动作的自然性和结构的合理性。

Details Motivation: 传统文本到手语视频生成方法忽略了人体骨骼运动的解剖学约束和协调模式，导致生成的姿势僵硬或不符合生物力学原理。因此，作者提出了一种新方法，通过几何约束显式建模骨骼关节之间的关系。

Contribution: 1. 提出了一种几何感知的损失函数，显式建模关节位置、骨骼长度和运动动态的关系；2. 设计了父节点相对加权机制，增强手指灵活性；3. 引入了骨骼姿态损失和骨骼长度约束，确保解剖学结构的合理性。

Method: 通过在训练过程中结合几何约束，包括关节位置、骨骼长度和运动动态的建模，并采用父节点相对加权机制和骨骼姿态损失，增强了生成动作的自然性和结构的合理性。

Result: 相比于之前的最佳方法，该方法将性能差距缩小了56.51%，同时在骨骼长度和运动方差上分别减少了18.76%和5.48%，显著提升了生成结果的解剖学合理性和动作自然性。

Insight: 显式建模几何约束可以有效解决人体动作生成的解剖学不合理问题，尤其是在复杂的联合运动（如手指动作）中，几何感知的损失函数能够显著提升生成质量。

Abstract: Sign language translation from text to video plays a crucial role in enabling effective communication for Deaf and hard–of–hearing individuals. A major challenge lies in generating accurate and natural body poses and movements that faithfully convey intended meanings. Prior methods often neglect the anatomical constraints and coordination patterns of human skeletal motion, resulting in rigid or biomechanically implausible outputs. To address this, we propose a novel approach that explicitly models the relationships among skeletal joints–including shoulders, arms, and hands–by incorporating geometric constraints on joint positions, bone lengths, and movement dynamics. During training, we introduce a parent-relative reweighting mechanism to enhance finger flexibility and reduce motion stiffness. Additionally, bone-pose losses and bone-length constraints enforce anatomically consistent structures. Our method narrows the performance gap between the previous best and the ground-truth oracle by 56.51%, and further reduces discrepancies in bone length and movement variance by 18.76% and 5.48%, respectively, demonstrating significant gains in anatomical realism and motion naturalness.

[137] Planning with Unified Multimodal Models

Yihao Sun,Zhilong Zhang,Yang Yu,Pierre-Luc Bacon

Main category: cs.CV

TL;DR: 论文提出了Uni-Plan框架，基于统一多模态模型（UMMs），通过结合语言和视觉推理能力提升决策任务的成功率。

Details Motivation: 现有决策方法主要依赖语言推理，限制了其能力和准确性，而多模态输入输出的统一模型具有更强的潜力。

Contribution: 1. 提出Uni-Plan框架，利用UMM实现策略、动态模型和价值函数的统一；2. 提出自区分过滤方法防止动态预测的幻觉问题。

Method: Uni-Plan框架采用单一UMM模型同时执行多任务，并通过自区分过滤技术筛选无效预测。

Result: 在长时程规划任务中，Uni-Plan显著优于VLM方法，且无需专家演示，数据扩展性更强。

Insight: 结合多模态生成的视觉内容可以更好地支持复杂决策任务，未来研究有望进一步发展UMM的推理能力。

Abstract: With the powerful reasoning capabilities of large language models (LLMs) and vision-language models (VLMs), many recent works have explored using them for decision-making. However, most of these approaches rely solely on language-based reasoning, which limits their ability to reason and make informed decisions. Recently, a promising new direction has emerged with unified multimodal models (UMMs), which support both multimodal inputs and outputs. We believe such models have greater potential for decision-making by enabling reasoning through generated visual content. To this end, we propose Uni-Plan, a planning framework built on UMMs. Within this framework, a single model simultaneously serves as the policy, dynamics model, and value function. In addition, to avoid hallucinations in dynamics predictions, we present a novel approach self-discriminated filtering, where the generative model serves as a self-discriminator to filter out invalid dynamics predictions. Experiments on long-horizon planning tasks show that Uni-Plan substantially improves success rates compared to VLM-based methods, while also showing strong data scalability, requiring no expert demonstrations and achieving better performance under the same training-data size. This work lays a foundation for future research in reasoning and decision-making with UMMs.

[138] Copyright Infringement Detection in Text-to-Image Diffusion Models via Differential Privacy

Xiafeng Man,Zhipeng Wei,Jingjing Chen

Main category: cs.CV

TL;DR: 本文提出了一种基于差分隐私的文本到图像扩散模型版权侵权检测方法，提出了DPM框架和CIDD数据集，解决了现有方法缺乏鲁棒性和理论依据的问题。

Details Motivation: 大规模视觉模型（如Stable Diffusion）可能未经授权记忆并重现受版权保护的内容，现有检测方法缺乏鲁棒性和理论支持，亟需一种新方法。

Contribution: 1. 从差分隐私角度形式化版权侵权及其检测；2. 提出条件敏感性度量（类似于DP中的敏感性）；3. 提出DPM检测框架和CIDD数据集。

Method: DPM通过微调模型在两个相反方向（学习与遗忘）模拟包含和排除过程，并使用统计指标在正交提示分布上计算置信分数。

Result: DPM在不依赖原始训练数据或文本提示的情况下，可靠地检测侵权内容，提供了可解释的解决方案。

Insight: 差分隐私为版权侵权检测提供了理论基础，DPM的提出为生成式AI时代的知识产权保护提供了实用工具。

Abstract: The widespread deployment of large vision models such as Stable Diffusion raises significant legal and ethical concerns, as these models can memorize and reproduce copyrighted content without authorization. Existing detection approaches often lack robustness and fail to provide rigorous theoretical underpinnings. To address these gaps, we formalize the concept of copyright infringement and its detection from the perspective of Differential Privacy (DP), and introduce the conditional sensitivity metric, a concept analogous to sensitivity in DP, that quantifies the deviation in a diffusion model’s output caused by the inclusion or exclusion of a specific training data point. To operationalize this metric, we propose D-Plus-Minus (DPM), a novel post-hoc detection framework that identifies copyright infringement in text-to-image diffusion models. Specifically, DPM simulates inclusion and exclusion processes by fine-tuning models in two opposing directions: learning or unlearning. Besides, to disentangle concept-specific influence from the global parameter shifts induced by fine-tuning, DPM computes confidence scores over orthogonal prompt distributions using statistical metrics. Moreover, to facilitate standardized benchmarking, we also construct the Copyright Infringement Detection Dataset (CIDD), a comprehensive resource for evaluating detection across diverse categories. Our results demonstrate that DPM reliably detects infringement content without requiring access to the original training dataset or text prompts, offering an interpretable and practical solution for safeguarding intellectual property in the era of generative AI.

[139] Sensor-Adaptive Flood Mapping with Pre-trained Multi-Modal Transformers across SAR and Multispectral Modalities

Tomohiro Tanaka,Narumasa Tsutsumida

Main category: cs.CV

TL;DR: 该论文提出了一种基于预训练多模态变换器的传感器自适应洪水监测方法，能够在SAR和多光谱数据中灵活切换，实现快速准确的洪水范围绘制。

Details Motivation: 洪水是频繁发生的自然灾害，现有监测方法受限于单一传感器的天气依赖性和多传感器融合的计算成本，亟需一种灵活高效的解决方案。

Contribution: 1. 提出了基于轻量级多模态变换器Presto的传感器自适应方法；2. 支持SAR、多光谱或二者结合的输入；3. 在Sen1Floods11数据集上表现优于基线模型。

Method: 通过微调预训练的Presto变换器（约0.4M参数），实现对SAR和多光谱数据的像素级处理，适应不同传感器输入。

Result: 在最优传感器融合场景下，F1分数为0.896，mIoU为0.886；在多光谱单独输入时F1为0.893，SAR单独输入时为0.718。

Insight: 多模态预训练提升了模型的鲁棒性和灵活性，使其在现实灾害场景中更具实用性。

Abstract: Floods are increasingly frequent natural disasters causing extensive human and economic damage, highlighting the critical need for rapid and accurate flood inundation mapping. While remote sensing technologies have advanced flood monitoring capabilities, operational challenges persist: single-sensor approaches face weather-dependent data availability and limited revisit periods, while multi-sensor fusion methods require substantial computational resources and large-scale labeled datasets. To address these limitations, this study introduces a novel sensor-flexible flood detection methodology by fine-tuning Presto, a lightweight ($\sim$0.4M parameters) multi-modal pre-trained transformer that processes both Synthetic Aperture Radar (SAR) and multispectral (MS) data at the pixel level. Our approach uniquely enables flood mapping using SAR-only, MS-only, or combined SAR+MS inputs through a single model architecture, addressing the critical operational need for rapid response with whatever sensor data becomes available first during disasters. We evaluated our method on the Sen1Floods11 dataset against the large-scale Prithvi-100M baseline ($\sim$100M parameters) across three realistic data availability scenarios. The proposed model achieved superior performance with an F1 score of 0.896 and mIoU of 0.886 in the optimal sensor-fusion scenario, outperforming the established baseline. Crucially, the model demonstrated robustness by maintaining effective performance in MS-only scenarios (F1: 0.893) and functional capabilities in challenging SAR-only conditions (F1: 0.718), confirming the advantage of multi-modal pre-training for operational flood mapping. Our parameter-efficient, sensor-flexible approach offers an accessible and robust solution for real-world disaster scenarios requiring immediate flood extent assessment regardless of sensor availability constraints.

[140] GeLoc3r: Enhancing Relative Camera Pose Regression with Geometric Consistency Regularization

Jingxing Li,Yongjae Lee,Deliang Fan

Main category: cs.CV

TL;DR: GeLoc3r提出了一种通过几何一致性正则化增强相对相机位姿估计的回归方法，在不增加推理时间的情况下显著提高了精度。

Details Motivation: 现有的ReLoc3R在速度和回归精度上表现优异，但其内部表示的几何不一致性限制了其达到基于对应方法（如MASt3R）的精度上限。

Contribution: 引入了几何一致性正则化（GCR），训练回归网络生成几何一致的位姿，同时保持推理速度。

Method: 利用地面真实深度生成稠密3D-2D对应，通过FusionTransformer学习对应重要性，并使用加权RANSAC计算几何一致位姿，生成一致性损失以指导回归网络训练。

Result: 在多个基准测试中显著优于ReLoc3R，例如CO3Dv2数据集的AUC@5°提升16%。

Insight: 通过在训练中注入几何知识而非推理时强制计算，GeLoc3r实现了回归速度和几何理解的平衡。

Abstract: Prior ReLoc3R achieves breakthrough performance with fast 25ms inference and state-of-the-art regression accuracy, yet our analysis reveals subtle geometric inconsistencies in its internal representations that prevent reaching the precision ceiling of correspondence-based methods like MASt3R (which require 300ms per pair). In this work, we present GeLoc3r, a novel approach to relative camera pose estimation that enhances pose regression methods through Geometric Consistency Regularization (GCR). GeLoc3r overcomes the speed-accuracy dilemma by training regression networks to produce geometrically consistent poses without inference-time geometric computation. During training, GeLoc3r leverages ground-truth depth to generate dense 3D-2D correspondences, weights them using a FusionTransformer that learns correspondence importance, and computes geometrically-consistent poses via weighted RANSAC. This creates a consistency loss that transfers geometric knowledge into the regression network. Unlike FAR method which requires both regression and geometric solving at inference, GeLoc3r only uses the enhanced regression head at test time, maintaining ReLoc3R’s fast speed and approaching MASt3R’s high accuracy. On challenging benchmarks, GeLoc3r consistently outperforms ReLoc3R, achieving significant improvements including 40.45% vs. 34.85% AUC@5{\deg} on the CO3Dv2 dataset (16% relative improvement), 68.66% vs. 66.70% AUC@5{\deg} on RealEstate10K, and 50.45% vs. 49.60% on MegaDepth1500. By teaching geometric consistency during training rather than enforcing it at inference, GeLoc3r represents a paradigm shift in how neural networks learn camera geometry, achieving both the speed of regression and the geometric understanding of correspondence methods.

[141] MMeViT: Multi-Modal ensemble ViT for Post-Stroke Rehabilitation Action Recognition

Ye-eun Kim,Suhyeon Lim,Andrew J. Choi

Main category: cs.CV

TL;DR: 该论文提出了一种多模态集成视觉Transformer（MMeViT）模型，用于中风患者的康复动作识别，结合IMU传感器与RGB-D摄像头的多模态数据，填补了现有HAR技术在中风患者中的空白。

Details Motivation: 由于中风康复治疗需求增加而医疗资源短缺，远程监控系统成为缓解医疗人员负担的有效方案。现有HAR技术多针对健康人群，缺乏对中风患者动作的适应性。

Contribution: 1) 设计了一个针对中风患者上肢日常活动（ADL）的监测系统；2) 直接采集多模态数据集并研究预处理方法；3) 提出适合多模态数据的深度学习模型。

Method: 使用IMU传感器和RGB-D摄像头采集数据，提出MMeViT模型（基于视觉Transformer）处理多模态动作数据，并分析其聚类特性。

Result: 发现中风患者的动作数据聚类性较弱；模型能学习难以聚类的数据特征，为未来扩展至康复评估反馈提供可能。

Insight: 深度学习模型可从中风患者的动作特征中学习，未来可能用于复杂任务（如康复评估），而不仅是简单的动作分类。

Abstract: Rehabilitation therapy for stroke patients faces a supply shortage despite the increasing demand. To address this issue, remote monitoring systems that reduce the burden on medical staff are emerging as a viable alternative. A key component of these remote monitoring systems is Human Action Recognition (HAR) technology, which classifies actions. However, existing HAR studies have primarily focused on non-disable individuals, making them unsuitable for recognizing the actions of stroke patients. HAR research for stroke has largely concentrated on classifying relatively simple actions using machine learning rather than deep learning. In this study, we designed a system to monitor the actions of stroke patients, focusing on domiciliary upper limb Activities of Daily Living (ADL). Our system utilizes IMU (Inertial Measurement Unit) sensors and an RGB-D camera, which are the most common modalities in HAR. We directly collected a dataset through this system, investigated an appropriate preprocess and proposed a deep learning model suitable for processing multimodal data. We analyzed the collected dataset and found that the action data of stroke patients is less clustering than that of non-disabled individuals. Simultaneously, we found that the proposed model learns similar tendencies for each label in data with features that are difficult to clustering. This study suggests the possibility of expanding the deep learning model, which has learned the action features of stroke patients, to not only simple action recognition but also feedback such as assessment contributing to domiciliary rehabilitation in future research. The code presented in this study is available at https://github.com/ye-Kim/MMeViT.

[142] Mask What Matters: Controllable Text-Guided Masking for Self-Supervised Medical Image Analysis

Ruilang Wang,Shuotong Xu,Bowen Liu,Runlin Huang,Donglong Chen,Weifeng Su

Main category: cs.CV

TL;DR: 论文提出了一种名为Mask What Matters的可控文本引导掩码框架，用于自监督医学图像分析。通过结合视觉语言模型区分诊断相关区域与背景，降低了冗余掩码，显著提升了语义对齐性和任务通用性，在分类、检测任务中表现优于现有方法。

Details Motivation: 医学图像标注数据稀缺，现有自监督掩码图像建模方法因随机高比例掩码导致效率低下且语义对齐不佳。需要一种更高效、可控的方法来提升模型性能。

Contribution: 1. 提出了一种可控文本引导的掩码框架，通过视觉语言模型实现诊断相关区域的动态掩码；2. 减少了冗余掩码比例（如40% vs 70%），提升了语义对齐性和任务通用性。

Method: 利用视觉语言模型进行基于提示的区域定位，对诊断相关区域进行差异化掩码，减少背景区域的冗余掩码。

Result: 在脑MRI、胸部CT和肺部X光等多种医学图像模态上，分类准确率提升+3.1%，检测任务中BoxAP和MaskAP分别提升+1.3和+1.1。

Insight: 可控的文本驱动掩码可以显著提升自监督学习的效果，适用于医学图像分析的语义对齐和模型鲁棒性。

Abstract: The scarcity of annotated data in specialized domains such as medical imaging presents significant challenges to training robust vision models. While self-supervised masked image modeling (MIM) offers a promising solution, existing approaches largely rely on random high-ratio masking, leading to inefficiency and poor semantic alignment. Moreover, region-aware variants typically depend on reconstruction heuristics or supervised signals, limiting their adaptability across tasks and modalities. We propose Mask What Matters, a controllable text-guided masking framework for self-supervised medical image analysis. By leveraging vision-language models for prompt-based region localization, our method flexibly applies differentiated masking to emphasize diagnostically relevant regions while reducing redundancy in background areas. This controllable design enables better semantic alignment, improved representation learning, and stronger cross-task generalizability. Comprehensive evaluation across multiple medical imaging modalities, including brain MRI, chest CT, and lung X-ray, shows that Mask What Matters consistently outperforms existing MIM methods (e.g., SparK), achieving gains of up to +3.1 percentage points in classification accuracy, +1.3 in box average precision (BoxAP), and +1.1 in mask average precision (MaskAP) for detection. Notably, it achieves these improvements with substantially lower overall masking ratios (e.g., 40% vs. 70%). This work demonstrates that controllable, text-driven masking can enable semantically aligned self-supervised learning, advancing the development of robust vision models for medical image analysis.

[143] FMC-DETR: Frequency-Decoupled Multi-Domain Coordination for Aerial-View Object Detection

Ben Liang,Yuan Liu,Bingwen Qiu,Yihong Wang,Xiubao Sui,Qian Chen

Main category: cs.CV

TL;DR: FMC-DETR提出了一种频率解耦的多域协调框架，用于解决航空图像中小物体检测的挑战，通过WeKat骨干网和MDFC模块实现了全局上下文建模与细粒度细节的平衡。

Details Motivation: 航空图像中小物体检测因视觉线索有限和复杂场景下全局上下文建模困难而极具挑战性，现有方法的延迟上下文融合和非线性建模不足导致性能瓶颈。

Contribution: 1. 提出WeKat骨干网，结合小波变换增强浅层特征的全局低频上下文感知；2. 引入轻量级CPF模块改善多尺度特征交互；3. 设计MDFC模块融合空间、频率和结构先验。

Method: 1. WeKat骨干网通过级联小波变换和Kolmogorov-Arnold网络实现非线性多尺度依赖建模；2. CPF模块减少冗余；3. MDFC模块协调多域特征。

Result: 在VisDrone数据集上，AP和AP50分别提升6.5%和8.2%，性能优于基线，参数量更少。

Insight: 频率解耦和多域协调是提升小物体检测性能的关键，全局低频信息与局部高频细节的平衡对模型有效。

Abstract: Aerial-view object detection is a critical technology for real-world applications such as natural resource monitoring, traffic management, and UAV-based search and rescue. Detecting tiny objects in high-resolution aerial imagery presents a long-standing challenge due to their limited visual cues and the difficulty of modeling global context in complex scenes. Existing methods are often hampered by delayed contextual fusion and inadequate non-linear modeling, failing to effectively use global information to refine shallow features and thus encountering a performance bottleneck. To address these challenges, we propose FMC-DETR, a novel framework with frequency-decoupled fusion for aerial-view object detection. First, we introduce the Wavelet Kolmogorov-Arnold Transformer (WeKat) backbone, which applies cascaded wavelet transforms to enhance global low-frequency context perception in shallow features while preserving fine-grained details, and employs Kolmogorov-Arnold networks to achieve adaptive non-linear modeling of multi-scale dependencies. Next, a lightweight Cross-stage Partial Fusion (CPF) module reduces redundancy and improves multi-scale feature interaction. Finally, we introduce the Multi-Domain Feature Coordination (MDFC) module, which unifies spatial, frequency, and structural priors to to balance detail preservation and global enhancement. Extensive experiments on benchmark aerial-view datasets demonstrate that FMC-DETR achieves state-of-the-art performance with fewer parameters. On the challenging VisDrone dataset, our model achieves improvements of 6.5% AP and 8.2% AP50 over the baseline, highlighting its effectiveness in tiny object detection. The code can be accessed at https://github.com/bloomingvision/FMC-DETR.

[144] Follow-Your-Preference: Towards Preference-Aligned Image Inpainting

Yutao Shen,Junkun Yuan,Toru Aonishi,Hideki Nakayama,Yue Ma

Main category: cs.CV

TL;DR: 该论文研究了基于偏好对齐的图像修复技术，通过直接偏好优化方法和公共奖励模型构建训练数据集，实验表明多模型集成能有效缓解偏差并提升性能。

Details Motivation: 图像修复领域缺乏偏好对齐的研究，作者希望通过基本方法重新审视这一问题，并为该领域提供一个简单可靠的基线。

Contribution: 主要贡献包括验证奖励模型的有效性、揭示偏好数据的鲁棒性趋势、发现奖励模型的偏差问题，并提出简单集成方法以提升性能。

Method: 利用直接偏好优化方法和公共奖励模型构建偏好对齐的训练数据集，并通过多模型集成缓解偏差。

Result: 实验显示，提出的对齐模型在标准指标、GPT-4评估和人工评估中均显著优于之前的方法。

Insight: 奖励模型尽管存在偏差，但仍可用于偏好对齐；多模型集成是提升鲁棒性和泛化能力的有效策略。

Abstract: This paper investigates image inpainting with preference alignment. Instead of introducing a novel method, we go back to basics and revisit fundamental problems in achieving such alignment. We leverage the prominent direct preference optimization approach for alignment training and employ public reward models to construct preference training datasets. Experiments are conducted across nine reward models, two benchmarks, and two baseline models with varying structures and generative algorithms. Our key findings are as follows: (1) Most reward models deliver valid reward scores for constructing preference data, even if some of them are not reliable evaluators. (2) Preference data demonstrates robust trends in both candidate scaling and sample scaling across models and benchmarks. (3) Observable biases in reward models, particularly in brightness, composition, and color scheme, render them susceptible to cause reward hacking. (4) A simple ensemble of these models yields robust and generalizable results by mitigating such biases. Built upon these observations, our alignment models significantly outperform prior models across standard metrics, GPT-4 assessments, and human evaluations, without any changes to model structures or the use of new datasets. We hope our work can set a simple yet solid baseline, pushing this promising frontier. Our code is open-sourced at: https://github.com/shenytzzz/Follow-Your-Preference.

[145] CoPatch: Zero-Shot Referring Image Segmentation by Leveraging Untapped Spatial Knowledge in CLIP

Na Min An,Inha Kang,Minhyun Lee,Hyunjung Shim

Main category: cs.CV

TL;DR: CoPatch通过利用CLIP中未开发的空间知识，提出了一种零样本Referring Image Segmentation方法，显著提升了空间定位能力。

Details Motivation: 现有的基础视觉语言模型（如CLIP）虽然在图像和文本对齐方面表现优异，但在理解空间关系上存在不足，影响了Referring Image Segmentation任务的性能。

Contribution: 1. 提出了一种零样本RIS框架CoPatch，通过增强文本和图像模态的空间表示能力；2. 设计了混合文本特征和新的patch级图像特征提取方法，提升了空间结构的敏感性；3. 引入了聚类图像-文本相似度图CoMap，实现了精确的掩模选择。

Method: 1. 在语言模态中，引入带有空间线索的上下文token构建混合文本特征；2. 在视觉模态中，从中层特征中新发现的路径提取patch级图像特征；3. 通过CoMap融合增强特征，进行掩模选择。

Result: 在RefCOCO、RefCOCO+、RefCOCOg和PhraseCut数据集上，CoPatch在不需额外训练的情况下，mIoU提升了2–7个点。

Insight: 研究表明，挖掘和利用视觉语言模型中潜藏的空间知识对于提升零样本RIS任务的性能至关重要。

Abstract: Spatial grounding is crucial for referring image segmentation (RIS), where the goal of the task is to localize an object described by language. Current foundational vision-language models (VLMs), such as CLIP, excel at aligning images and text but struggle with understanding spatial relationships. Within the language stream, most existing methods often focus on the primary noun phrase when extracting local text features, undermining contextual tokens. Within the vision stream, CLIP generates similar features for images with different spatial layouts, resulting in limited sensitivity to spatial structure. To address these limitations, we propose \textsc{CoPatch}, a zero-shot RIS framework that leverages internal model components to enhance spatial representations in both text and image modalities. For language, \textsc{CoPatch} constructs hybrid text features by incorporating context tokens carrying spatial cues. For vision, it extracts patch-level image features using our novel path discovered from intermediate layers, where spatial structure is better preserved. These enhanced features are fused into a clustered image-text similarity map, \texttt{CoMap}, enabling precise mask selection. As a result, \textsc{CoPatch} significantly improves spatial grounding in zero-shot RIS across RefCOCO, RefCOCO+, RefCOCOg, and PhraseCut (+ 2–7 mIoU) without requiring any additional training. Our findings underscore the importance of recovering and leveraging the untapped spatial knowledge inherently embedded in VLMs, thereby paving the way for opportunities in zero-shot RIS.

[146] Deep Learning for Oral Health: Benchmarking ViT, DeiT, BEiT, ConvNeXt, and Swin Transformer

Ajo Babu George,Sadhvik Bathini,Niranjana S R

Main category: cs.CV

TL;DR: 本研究系统地评估并比较了五种先进的基于Transformer的架构（ViT、DeiT、ConvNeXt、Swin Transformer和BEiT）在多类牙齿疾病分类中的表现，特别关注数据不平衡的实际挑战。ConvNeXt表现最佳，验证准确率达到81.06%，其次是BEiT和Swin Transformer。ViT和DeiT在龋齿相关类别上表现较差。

Details Motivation: 现有文献中常忽视数据不平衡等实际问题，本研究旨在填补这一空白，为临床牙齿疾病诊断工具提供模型选择的依据。

Contribution: 本研究首次系统地比较了多种Transformer架构在牙齿疾病分类任务中的性能，尤其是对数据不平衡问题的处理能力，为实际应用提供了参考。

Method: 使用Oral Diseases数据集训练和验证五种模型（ViT、DeiT、ConvNeXt、Swin Transformer、BEiT），并通过验证准确率、精确率、召回率和F1分数等指标评估性能。

Result: ConvNeXt表现最佳（验证准确率81.06%），BEiT和Swin Transformer次之（80.00%和79.73%），ViT和DeiT在龋齿相关类别上表现较弱。

Insight: ConvNeXt、Swin Transformer和BEiT在牙齿疾病分类中表现可靠，适合临床影像诊断。数据不平衡问题对模型性能有显著影响，需要在未来研究中重点关注。

Abstract: Objective: The aim of this study was to systematically evaluate and compare the performance of five state-of-the-art transformer-based architectures - Vision Transformer (ViT), Data-efficient Image Transformer (DeiT), ConvNeXt, Swin Transformer, and Bidirectional Encoder Representation from Image Transformers (BEiT) - for multi-class dental disease classification. The study specifically focused on addressing real-world challenges such as data imbalance, which is often overlooked in existing literature. Study Design: The Oral Diseases dataset was used to train and validate the selected models. Performance metrics, including validation accuracy, precision, recall, and F1-score, were measured, with special emphasis on how well each architecture managed imbalanced classes. Results: ConvNeXt achieved the highest validation accuracy at 81.06, followed by BEiT at 80.00 and Swin Transformer at 79.73, all demonstrating strong F1-scores. ViT and DeiT achieved accuracies of 79.37 and 78.79, respectively, but both struggled particularly with Caries-related classes. Conclusions: ConvNeXt, Swin Transformer, and BEiT showed reliable diagnostic performance, making them promising candidates for clinical application in dental imaging. These findings provide guidance for model selection in future AI-driven oral disease diagnostic tools and highlight the importance of addressing data imbalance in real-world scenarios

[147] HTMA-Net: Towards Multiplication-Avoiding Neural Networks via Hadamard Transform and In-Memory Computing

Emadeldeen Hamdan,Ahmet Enis Cetin

Main category: cs.CV

TL;DR: HTMA-Net通过结合哈达玛变换和免乘法SRAM内存计算，减少了深度神经网络的乘法运算，同时保持精度，适用于边缘设备。

Details Motivation: 减少乘法运算是边缘设备上高效部署深度神经网络的关键，HTMA-Net旨在降低计算复杂度和参数数量。

Contribution: HTMA-Net提出了一种结合哈达玛变换和免乘法内存计算的新框架，显著减少了乘法运算和计算复杂度。

Method: 通过选择性替换中间卷积层为哈达玛变换层，并结合免乘法内存计算实现内卷积运算。

Result: 在ResNet-18等模型上，HTMA-Net消除了52%的乘法运算，同时保持精度。

Insight: 结构化哈达玛变换与免乘法内存计算的结合是实现高效深度学习架构的有效途径。

Abstract: Reducing the cost of multiplications is critical for efficient deep neural network deployment, especially in energy-constrained edge devices. In this work, we introduce HTMA-Net, a novel framework that integrates the Hadamard Transform (HT) with multiplication-avoiding (MA) SRAM-based in-memory computing to reduce arithmetic complexity while maintaining accuracy. Unlike prior methods that only target multiplications in convolutional layers or focus solely on in-memory acceleration, HTMA-Net selectively replaces intermediate convolutions with Hybrid Hadamard-based transform layers whose internal convolutions are implemented via multiplication-avoiding in-memory operations. We evaluate HTMA-Net on ResNet-18 using CIFAR-10, CIFAR-100, and Tiny ImageNet, and provide a detailed comparison against regular, MF-only, and HT-only variants. Results show that HTMA-Net eliminates up to 52% of multiplications compared to baseline ResNet-18, ResNet-20, and ResNet-50 models, while achieving comparable accuracy in evaluation and significantly reducing computational complexity and the number of parameters. Our results demonstrate that combining structured Hadamard transform layers with SRAM-based in-memory computing multiplication-avoiding operators is a promising path towards efficient deep learning architectures.

[148] Towards Comprehensive Interactive Change Understanding in Remote Sensing: A Large-scale Dataset and Dual-granularity Enhanced VLM

Junxiao Xue,Quan Deng,Xuecheng Wu,Kelu Yao,Xinyi Yin,Fei Yu,Wei Zhou,Yanfei Zhong,Yang Liu,Dingkang Yang

Main category: cs.CV

TL;DR: 论文提出了一种新的大规模交互式多任务数据集ChangeIMTI，并设计了一种具有双粒度感知能力的视觉引导视觉语言模型ChangeVG，用于双时相遥感图像的变化理解任务。

Details Motivation: 现有数据集在遥感变化理解任务（如变化描述、分类、计数与定位）中缺乏深度理解与交互性，限制了模型的性能。

Contribution: 1. 构建了ChangeIMTI数据集，覆盖四种互补任务；2. 提出了双粒度感知的视觉引导模型ChangeVG，结合空间特征提取与高层语义总结；3. 在多项任务中验证了方法的优越性。

Method: 设计了双分支架构ChangeVG，分别提取细粒度空间特征与高层语义信息，并通过辅助提示引导视觉语言模型进行指令调优。

Result: 在变化描述任务中，方法在S*m指标上超越最强基线1.39分。

Insight: 双粒度特征和高层语义的协同结合对遥感变化理解任务至关重要，且大规模多任务数据集有助于提升模型性能。

Abstract: Remote sensing change understanding (RSCU) is essential for analyzing remote sensing images and understanding how human activities affect the environment. However, existing datasets lack deep understanding and interactions in the diverse change captioning, counting, and localization tasks. To tackle these gaps, we construct ChangeIMTI, a new large-scale interactive multi-task instruction dataset that encompasses four complementary tasks including change captioning, binary change classification, change counting, and change localization. Building upon this new dataset, we further design a novel vision-guided vision-language model (ChangeVG) with dual-granularity awareness for bi-temporal remote sensing images (i.e., two remote sensing images of the same area at different times). The introduced vision-guided module is a dual-branch architecture that synergistically combines fine-grained spatial feature extraction with high-level semantic summarization. These enriched representations further serve as the auxiliary prompts to guide large vision-language models (VLMs) (e.g., Qwen2.5-VL-7B) during instruction tuning, thereby facilitating the hierarchical cross-modal learning. We extensively conduct experiments across four tasks to demonstrate the superiority of our approach. Remarkably, on the change captioning task, our method outperforms the strongest method Semantic-CC by 1.39 points on the comprehensive S*m metric, which integrates the semantic similarity and descriptive accuracy to provide an overall evaluation of change caption. Moreover, we also perform a series of ablation studies to examine the critical components of our method.

[149] Stochastic Interpolants via Conditional Dependent Coupling

Chenrui Ma,Xi Xiao,Tianyang Wang,Xiao Wang,Yanning Shen

Main category: cs.CV

TL;DR: 这篇论文提出了一种基于条件依赖耦合的统一多阶段生成框架，解决了现有图像生成模型在计算成本和保真度之间的权衡问题。

Details Motivation: 现有图像生成模型（如基于VAE或像素空间的模型）在信息丢失、计算成本和多阶段优化方面存在局限性。本文旨在实现高保真度和高效率的统一生成框架。

Contribution: 1) 提出了一种基于条件依赖耦合的多阶段生成框架；2) 将生成过程分解为多阶段的插值轨迹；3) 整个模型通过统一的Diffusion Transformer实现端到端优化。

Method: 1) 利用条件依赖耦合策略分解生成过程；2) 设计统一的Diffusion Transformer模型，支持多阶段知识共享和端到端训练。

Result: 实验表明，该方法在多个分辨率下实现了高保真度和高效率。

Insight: 通过统一的Diffusion Transformer和多阶段插值策略，可以在不牺牲保真度的情况下显著降低计算成本。

Abstract: Existing image generation models face critical challenges regarding the trade-off between computation and fidelity. Specifically, models relying on a pretrained Variational Autoencoder (VAE) suffer from information loss, limited detail, and the inability to support end-to-end training. In contrast, models operating directly in the pixel space incur prohibitive computational cost. Although cascade models can mitigate computational cost, stage-wise separation prevents effective end-to-end optimization, hampers knowledge sharing, and often results in inaccurate distribution learning within each stage. To address these challenges, we introduce a unified multistage generative framework based on our proposed Conditional Dependent Coupling strategy. It decomposes the generative process into interpolant trajectories at multiple stages, ensuring accurate distribution learning while enabling end-to-end optimization. Importantly, the entire process is modeled as a single unified Diffusion Transformer, eliminating the need for disjoint modules and also enabling knowledge sharing. Extensive experiments demonstrate that our method achieves both high fidelity and efficiency across multiple resolutions.

[150] Benchmarking DINOv3 for Multi-Task Stroke Analysis on Non-Contrast CT

Donghao Zhang,Yimin Chen,Kauê TN Duarte,Taha Aslan,Mohamed AlShamrani,Brij Karmur,Yan Wan,Shengcai Chen,Bo Hu,Bijoy K Menon,Wu Qiu

Main category: cs.CV

TL;DR: 论文利用DINOv3，一种先进的自监督视觉转换器，解决非对比CT（NCCT）影像在卒中分析中的低对比度和信噪比问题，建立了多任务卒中分析的基准。

Details Motivation: NCCT是卒中快速诊断的重要工具，但其低对比度和信噪比限制了分析效果。本研究旨在探索DINOv3在卒中多任务分析中的潜力。

Contribution: 1. 首次将DINOv3应用于NCCT影像的多任务卒中分析；2. 为多种卒中任务（分割、分类等）建立了权威基准；3. 揭示了自监督模型在卒中诊断中的优势和局限。

Method: 使用DINOv3生成NCCT的强大特征表示，并在多个公开和私有数据集上评估其性能，涵盖卒中分割、分类等多种任务。

Result: 研究表明DINOv3在多任务卒中分析中表现优异，为未来研究提供了可靠的基准，并展示了自监督模型在医学影像中的潜力。

Insight: 自监督视觉转换器（如DINOv3）可以显著提升NCCT影像的卒中分析效果，但其性能仍需结合特定任务优化。医学影像的低信噪比问题需要进一步研究。

Abstract: Non-contrast computed tomography (NCCT) is essential for rapid stroke diagnosis but is limited by low image contrast and signal to noise ratio. We address this challenge by leveraging DINOv3, a state-of-the-art self-supervised vision transformer, to generate powerful feature representations for a comprehensive set of stroke analysis tasks. Our evaluation encompasses infarct and hemorrhage segmentation, anomaly classification (normal vs. stroke and normal vs. infarct vs. hemorrhage), hemorrhage subtype classification (EDH, SDH, SAH, IPH, IVH), and dichotomized ASPECTS classification (<=6 vs. >6) on multiple public and private datasets. This study establishes strong benchmarks for these tasks and demonstrates the potential of advanced self-supervised models to improve automated stroke diagnosis from NCCT, providing a clear analysis of both the advantages and current constraints of the approach. The code is available at https://github.com/Zzz0251/DINOv3-stroke.

[151] Earth-Agent: Unlocking the Full Landscape of Earth Observation with Agents

Peilin Feng,Zhutao Lv,Junyan Ye,Xiaolei Wang,Xinjie Huo,Jinhua Yu,Wanghan Xu,Wenlong Zhang,Lei Bai,Conghui He,Weijia Li

Main category: cs.CV

TL;DR: Earth-Agent是一个新型的基于代理的框架，整合了RGB和多光谱地球观测数据，通过多模态工具生态系统实现复杂任务的跨模态、多步推理。它还提出了Earth-Bench基准测试，用于全面评估代理的性能。

Details Motivation: 现有的MLLMs在多模态地球观测任务中存在局限性，无法完成复杂的多步推理和领域工具的使用。代理方法为解决这些问题提供了潜力，但当前研究仍局限于RGB感知和浅层推理。

Contribution: 提出了第一个整合RGB和多光谱数据的代理框架Earth-Agent，支持跨模态和定量时空推理；设计了Earth-Bench基准测试，包含248个专家标注的任务和13,729张图像。

Method: Earth-Agent基于MCP工具生态系统，支持动态调用跨模态专家工具和模型，实现复杂科学任务的多步推理。

Result: 实验表明，Earth-Agent在不同LLM骨干、通用代理框架和MLLMs的比较中表现出色，展示了其在EO领域的潜力。

Insight: Earth-Agent为地球观测分析设定了新范式，推动了LLMs在科学领域的应用。

Abstract: Earth observation (EO) is essential for understanding the evolving states of the Earth system. Although recent MLLMs have advanced EO research, they still lack the capability to tackle complex tasks that require multi-step reasoning and the use of domain-specific tools. Agent-based methods offer a promising direction, but current attempts remain in their infancy, confined to RGB perception, shallow reasoning, and lacking systematic evaluation protocols. To overcome these limitations, we introduce Earth-Agent, the first agentic framework that unifies RGB and spectral EO data within an MCP-based tool ecosystem, enabling cross-modal, multi-step, and quantitative spatiotemporal reasoning beyond pretrained MLLMs. Earth-Agent supports complex scientific tasks such as geophysical parameter retrieval and quantitative spatiotemporal analysis by dynamically invoking expert tools and models across modalities. To support comprehensive evaluation, we further propose Earth-Bench, a benchmark of 248 expert-curated tasks with 13,729 images, spanning spectrum, products and RGB modalities, and equipped with a dual-level evaluation protocol that assesses both reasoning trajectories and final outcomes. We conduct comprehensive experiments varying different LLM backbones, comparisons with general agent frameworks, and comparisons with MLLMs on remote sensing benchmarks, demonstrating both the effectiveness and potential of Earth-Agent. Earth-Agent establishes a new paradigm for EO analysis, moving the field toward scientifically grounded, next-generation applications of LLMs in Earth observation. Our code and dataset will be publicly released.

[152] Sparse2Dense: A Keypoint-driven Generative Framework for Human Video Compression and Vertex Prediction

Bolin Chen,Ru-Ling Liao,Yan Ye,Jie Chen,Shanzhi Yin,Xinrui Ju,Shiqi Wang,Yibo Fan

Main category: cs.CV

TL;DR: Sparse2Dense利用稀疏3D关键点作为传输符号，实现超低码率人体视频压缩和精确顶点预测，支持动态运动建模、外观合成与几何一致性。

Details Motivation: 在带宽受限的多媒体应用中，同时实现超低码率人体视频压缩和精确顶点预测是一项关键挑战，需动态运动建模、外观合成与几何一致性协调。

Contribution: 提出Sparse2Dense框架，通过稀疏3D关键点驱动生成模型，实现视频压缩与顶点预测的多任务学习，确保时序一致性和几何对齐。

Method: 采用基于关键点的深度生成模型，编码复杂运动为稀疏关键点，并预测稠密运动以合成视频；结合顶点预测器通过联合优化学习几何结构。

Result: 实验表明Sparse2Dense在压缩性能上优于传统/生成式视频编解码器，同时实现精确顶点预测，适用于实时运动分析、虚拟动画等场景。

Insight: 稀疏关键点是高效表征人体运动的紧凑符号，结合生成模型和几何预测可提升带宽受限场景下的多媒体传输效率。

Abstract: For bandwidth-constrained multimedia applications, simultaneously achieving ultra-low bitrate human video compression and accurate vertex prediction remains a critical challenge, as it demands the harmonization of dynamic motion modeling, detailed appearance synthesis, and geometric consistency. To address this challenge, we propose Sparse2Dense, a keypoint-driven generative framework that leverages extremely sparse 3D keypoints as compact transmitted symbols to enable ultra-low bitrate human video compression and precise human vertex prediction. The key innovation is the multi-task learning-based and keypoint-aware deep generative model, which could encode complex human motion via compact 3D keypoints and leverage these sparse keypoints to estimate dense motion for video synthesis with temporal coherence and realistic textures. Additionally, a vertex predictor is integrated to learn human vertex geometry through joint optimization with video generation, ensuring alignment between visual content and geometric structure. Extensive experiments demonstrate that the proposed Sparse2Dense framework achieves competitive compression performance for human video over traditional/generative video codecs, whilst enabling precise human vertex prediction for downstream geometry applications. As such, Sparse2Dense is expected to facilitate bandwidth-efficient human-centric media transmission, such as real-time motion analysis, virtual human animation, and immersive entertainment.

[153] TRAX: TRacking Axles for Accurate Axle Count Estimation

Avinash Rai,Sandeep Jana,Vishal Vijay

Main category: cs.CV

TL;DR: 提出了一种基于视频的端到端系统TRAX，用于准确估计车辆轴数，解决了密集环境中轴数统计的局限性，并通过创新的TRAX算法提升了长车辆和遮挡情况下的准确性。

Details Motivation: 车辆轴数的准确统计对交通管理、收费和基础设施建设至关重要。现有方法在密集环境、长车辆和遮挡场景中表现不佳，亟需改进。

Contribution: 1) 提出了一种端到端的轴数统计系统；2) 结合YOLO-OBB和YOLO检测车辆和轮胎，并通过智能关联提升准确性；3) 设计了TRAX算法，显著减少了误检并提升了长车辆的轴数统计精度。

Method: 1) 使用YOLO-OBB检测和分类车辆；2) 用YOLO检测轮胎；3) 提出TRAX算法跟踪帧之间的轮胎和轴特征，解决遮挡和长车辆的挑战。

Result: 在真实交通视频中表现出较强的鲁棒性，显著减少了误检并提升了长车辆的轴数统计准确性。

Insight: 机器视觉可以替代传统路边基础设施，为智能交通系统提供可扩展的解决方案，尤其是在复杂环境中表现出色。

Abstract: Accurate counting of vehicle axles is essential for traffic control, toll collection, and infrastructure development. We present an end-to-end, video-based pipeline for axle counting that tackles limitations of previous works in dense environments. Our system leverages a combination of YOLO-OBB to detect and categorize vehicles, and YOLO to detect tires. Detected tires are intelligently associated to their respective parent vehicles, enabling accurate axle prediction even in complex scenarios. However, there are a few challenges in detection when it comes to scenarios with longer and occluded vehicles. We mitigate vehicular occlusions and partial detections for longer vehicles by proposing a novel TRAX (Tire and Axle Tracking) Algorithm to successfully track axle-related features between frames. Our method stands out by significantly reducing false positives and improving the accuracy of axle-counting for long vehicles, demonstrating strong robustness in real-world traffic videos. This work represents a significant step toward scalable, AI-driven axle counting systems, paving the way for machine vision to replace legacy roadside infrastructure.

[154] Patch Rebirth: Toward Fast and Transferable Model Inversion of Vision Transformers

Seongsoo Heo,Dong-Wan Choi

Main category: cs.CV

TL;DR: 该论文提出了一种名为Patch Rebirth Inversion (PRI)的新方法，用于解决Vision Transformers (ViTs)在模型反转中的计算效率问题。PRI通过逐步分离重要补丁并允许其余补丁继续优化，显著提升了反转效率和质量。

Details Motivation: 模型反转技术在无数据学习中应用广泛，但ViTs因其昂贵的自注意力机制导致计算效率低下。此前提出的Sparse Model Inversion (SMI)通过剪枝补丁提升效率，但研究发现丢弃补丁会抑制知识的迁移。

Contribution: 1. 提出了Patch Rebirth Inversion (PRI)，通过渐进式策略平衡效率和知识迁移；2. 揭示了补丁的”Re-Birth效应”，即初始不重要补丁可通过持续优化积累知识；3. PRI在效率和性能上均优于SMI，且接近Dense Model Inversion (DMI)。

Method: PRI在反转过程中渐进式分离最重要补丁构建稀疏合成图像，同时允许剩余补继续优化以备未来选择。这种方法利用了补丁的动态优化特性。

Result: 实验表明，PRI比标准DMI快10倍，比SMI快2倍，且在精度上优于SMI，与DMI相当。

Insight: 1. 补丁的丢弃可能抑制知识的迁移；2. 渐进式优化策略可有效平衡效率和知识提取；3. “Re-Birth效应”为模型反转提供了新的优化方向。

Abstract: Model inversion is a widely adopted technique in data-free learning that reconstructs synthetic inputs from a pretrained model through iterative optimization, without access to original training data. Unfortunately, its application to state-of-the-art Vision Transformers (ViTs) poses a major computational challenge, due to their expensive self-attention mechanisms. To address this, Sparse Model Inversion (SMI) was proposed to improve efficiency by pruning and discarding seemingly unimportant patches, which were even claimed to be obstacles to knowledge transfer. However, our empirical findings suggest the opposite: even randomly selected patches can eventually acquire transferable knowledge through continued inversion. This reveals that discarding any prematurely inverted patches is inefficient, as it suppresses the extraction of class-agnostic features essential for knowledge transfer, along with class-specific features. In this paper, we propose Patch Rebirth Inversion (PRI), a novel approach that incrementally detaches the most important patches during the inversion process to construct sparse synthetic images, while allowing the remaining patches to continue evolving for future selection. This progressive strategy not only improves efficiency, but also encourages initially less informative patches to gradually accumulate more class-relevant knowledge, a phenomenon we refer to as the Re-Birth effect, thereby effectively balancing class-agnostic and class-specific knowledge. Experimental results show that PRI achieves up to 10x faster inversion than standard Dense Model Inversion (DMI) and 2x faster than SMI, while consistently outperforming SMI in accuracy and matching the performance of DMI.

[155] Self-Consistency as a Free Lunch: Reducing Hallucinations in Vision-Language Models via Self-Reflection

Mingfei Han,Haihong Hao,Jinxing Zhou,Zhihui Li,Yuhui Zheng,Xueqing Deng,Linjie Yang,Xiaojun Chang

Main category: cs.CV

TL;DR: 本文提出了一种利用视觉语言模型的自一致性（长回答与短回答的对比）来自动生成训练数据的方法，以减少幻觉现象（如虚构物体或不准确属性），无需人工标注或外部监督。

Details Motivation: 视觉语言模型常因幻觉问题生成不可靠的输出，现有方法依赖人工标注或外部监督，成本高且不易扩展。本文旨在通过自一致性检测和自动训练数据生成，降低成本并提升可靠性。

Contribution: 提出了一个基于自一致性的自反思框架，自动生成高质量训练数据；证明了仅依赖模型内部信号即可有效减少幻觉现象。

Method: 设计了一种自反思流程，通过比较模型的长回答和短答案检测不一致性，并利用这些信号自动生成训练数据。完全依赖模型内部信号，无需外部监督。

Result: 在多个基准测试（如AMBER、MMHal-Bench等）上显著提升了事实基础和可靠性，同时保持了指令跟随能力。

Insight: 模型的短回答（如二元问题）通常更可靠，可作为自一致性检测的参考信号；自一致性可以作为减少幻觉的低成本且高效的解决方案。

Abstract: Vision-language models often hallucinate details, generating non-existent objects or inaccurate attributes that compromise output reliability. Existing methods typically address these issues via extensive human annotations or external supervision from more powerful models. In this work, we present a novel framework that leverages the model’s self-consistency between long responses and short answers to generate preference pairs for training. We observe that short binary questions tend to yield highly reliable responses, which can be used to query the target model to evaluate and rank its generated responses. Specifically, we design a self-reflection pipeline where detailed model responses are compared against concise binary answers, and inconsistency signals are utilized to automatically curate high-quality training data without human annotations or external model-based supervision. By relying solely on self-consistency rather than external supervision, our method offers a scalable and efficient solution that effectively reduces hallucinations using unlabeled data. Extensive experiments on multiple benchmarks, i.e., AMBER, MultiObject-Hal (ROPE), Object HalBench, and MMHal-Bench, demonstrate significant improvements in factual grounding and reliability. Moreover, our approach maintains robust instruction-following ability, as evidenced by enhanced performance on LLaVA-Bench and MMBench.

[156] TATTOO: Training-free AesTheTic-aware Outfit recOmmendation

Yuntian Wu,Xiaonan Hu,Ziqi Zhou,Hao Lu

Main category: cs.CV

TL;DR: 本文提出了TATTOO，一种无需训练的美学感知穿搭推荐方法，利用多模态大语言模型（MLLMs）生成目标物品描述和美学链式思考，从而实现高效的穿搭推荐。

Details Motivation: 现有的穿搭推荐方法通常需要大规模标注数据进行任务特定训练，且缺乏对人类美学的显式引导。本文旨在通过训练自由范式简化传统流程，同时提升推荐效果和美学感知。

Contribution: 1. 提出TATTOO方法，首次将训练自由的范式引入美学感知穿搭推荐；2. 利用MLLMs生成目标物品描述和美学特征；3. 设计动态熵门控机制融合视觉与文本特征。

Method: 1. 使用MLLMs生成目标物品描述；2. 通过美学链式思考将图像提炼为结构化美学特征；3. 动态熵门控机制融合多模态特征，生成共享嵌入空间进行排序。

Result: 在Aesthetic-100数据集上表现优于现有训练方法，在Polyvore数据集上展示了先进的零样本检索能力。

Insight: 训练自由范式结合MLLMs可以显著简化传统推荐系统流程，同时利用美学特征提升推荐效果。

Abstract: The global fashion e-commerce market relies significantly on intelligent and aesthetic-aware outfit-completion tools to promote sales. While previous studies have approached the problem of fashion outfit-completion and compatible-item retrieval, most of them require expensive, task-specific training on large-scale labeled data, and no effort is made to guide outfit recommendation with explicit human aesthetics. In the era of Multimodal Large Language Models (MLLMs), we show that the conventional training-based pipeline could be streamlined to a training-free paradigm, with better recommendation scores and enhanced aesthetic awareness. We achieve this with TATTOO, a Training-free AesTheTic-aware Outfit recommendation approach. It first generates a target-item description using MLLMs, followed by an aesthetic chain-of-thought used to distill the images into a structured aesthetic profile including color, style, occasion, season, material, and balance. By fusing the visual summary of the outfit with the textual description and aesthetics vectors using a dynamic entropy-gated mechanism, candidate items can be represented in a shared embedding space and be ranked accordingly. Experiments on a real-world evaluation set Aesthetic-100 show that TATTOO achieves state-of-the-art performance compared with existing training-based methods. Another standard Polyvore dataset is also used to measure the advanced zero-shot retrieval capability of our training-free method.

[157] Increasing the Diversity in RGB-to-Thermal Image Translation for Automotive Applications

Kaili Wang,Leonardo Ravaglia,Roberto Longo,Lore Goetschalckx,David Van Hamme,Julie Moeyersoms,Ben Stoffelen,Tom De Schepper

Main category: cs.CV

TL;DR: 提出了一种通过多模态翻译框架和CoAdaIN技术实现RGB到热图像的多样化转换，以解决自动驾驶中热成像数据集不足的问题。

Details Motivation: 热成像在ADAS中具有优势，但现有数据和方法限制了多样性，需要更真实的转换方法。

Contribution: 提出了CoAdaIN技术，实现对图像组件的自适应风格调整，生成多样化和更真实的热图像。

Method: 使用多模态翻译框架结合CoAdaIN，取代传统的全局AdaIN，逐组件调整风格。

Result: 生成的转换热图像更真实且多样化，优于现有方法。

Insight: 逐组件风格调整是实现多样化图像转换的有效途径。

Abstract: Thermal imaging in Advanced Driver Assistance Systems (ADAS) improves road safety with superior perception in low-light and harsh weather conditions compared to traditional RGB cameras. However, research in this area faces challenges due to limited dataset availability and poor representation in driving simulators. RGB-to-thermal image translation offers a potential solution, but existing methods focus on one-to-one mappings. We propose a one-to-many mapping using a multi-modal translation framework enhanced with our Component-aware Adaptive Instance Normalization (CoAdaIN). Unlike the original AdaIN, which applies styles globally, CoAdaIN adapts styles to different image components individually. The result, as we show, is more realistic and diverse thermal image translations. This is the accepted author manuscript of the paper published in IEEE Sensors Conference 2024. The final published version is available at 10.1109/SENSORS60989.2024.10785056.

[158] LiDAR-based Human Activity Recognition through Laplacian Spectral Analysis

Sasan Sharifipour,Constantino Álvarez Casado,Le Nguyen,Tharindu Ekanayake,Manuel Lage Cañellas,Nhi Nguyen,Miguel Bordallo López

Main category: cs.CV

TL;DR: 本文提出了一种基于LiDAR点云和拉普拉斯谱分析的人类活动识别方法，通过图谱特征提取和时间统计分析实现高效分类，在MM-Fi数据集上表现优于骨架基线方法。

Details Motivation: 基于LiDAR的人类活动识别（HAR）因其隐私保护和光照鲁棒性优于摄像头，但需要开发高效且可解释的特征提取方法，避免依赖复杂的端到端深度学习。

Contribution: 提出了一种直接从点云几何中提取紧凑且可解释的图谱特征的方法，通过拉普拉斯谱分析生成姿态描述符，并在时间窗口中进行统计分析，实现高效分类。

Method: 将LiDAR帧映射为邻近图（epsilon-graph），计算拉普拉斯谱，利用特征值和特征向量的统计信息形成姿态描述符，滑动窗口的时间统计分析生成固定长度的分类向量，使用支持向量机和随机森林进行分类。

Result: 在MM-Fi数据集上，该方法在13类康复活动上达到94.4%的准确率，27类活动上达到90.3%，优于骨架基线方法。

Insight: 拉普拉斯谱分析提供了一种直接从点云几何中提取高效特征的途径，避免了深度学习的黑盒问题，同时保持了高准确率。

Abstract: Human Activity Recognition supports applications in healthcare, manufacturing, and human-machine interaction. LiDAR point clouds offer a privacy-preserving alternative to cameras and are robust to illumination. We propose a HAR method based on graph spectral analysis. Each LiDAR frame is mapped to a proximity graph (epsilon-graph) and the Laplacian spectrum is computed. Eigenvalues and statistics of eigenvectors form pose descriptors, and temporal statistics over sliding windows yield fixed vectors for classification with support vector machines and random forests. On the MM-Fi dataset with 40 subjects and 27 activities, under a strict subject-independent protocol, the method reaches 94.4% accuracy on a 13-class rehabilitation set and 90.3% on all 27 activities. It also surpasses the skeleton-based baselines reported for MM-Fi. The contribution is a compact and interpretable feature set derived directly from point cloud geometry that provides an accurate and efficient alternative to end-to-end deep learning.

[159] Learning Regional Monsoon Patterns with a Multimodal Attention U-Net

Swaib Ilias Mazumder,Manish Kumar,Aparajita Khan

Main category: cs.CV

TL;DR: 本文提出了一种多模态深度学习框架（注意力引导的U-Net），用于高分辨率的印度季风降水分类，结合多种地理空间数据，在极端降雨类别中表现优于现有方法。

Details Motivation: 准确的季风降雨预测对印度农业和水资源管理至关重要，但现有模型受限于粗分辨率数据和复杂的区域变化性。

Contribution: 1) 提出了一个多模态深度学习框架；2) 构建了一个新的1公里分辨率数据集；3) 在极端降雨预测中取得了先进的结果。

Method: 使用注意力引导的U-Net架构结合七种地理空间模态数据，并采用焦点损失和Dice损失函数处理类别不平衡问题。

Result: 该方法在多模态数据上表现优于单模态基准和现有深度学习方法，尤其是在极端降雨类别中。

Insight: 结合多种地理空间数据和注意力机制可以有效捕捉区域季风模式，提升降水预测精度。

Abstract: Accurate monsoon rainfall prediction is vital for India’s agriculture, water management, and climate risk planning, yet remains challenging due to sparse ground observations and complex regional variability. We present a multimodal deep learning framework for high-resolution precipitation classification that leverages satellite and Earth observation data. Unlike previous rainfall prediction models based on coarse 5-50 km grids, we curate a new 1 km resolution dataset for five Indian states, integrating seven key geospatial modalities: land surface temperature, vegetation (NDVI), soil moisture, relative humidity, wind speed, elevation, and land use, covering the June-September 2024 monsoon season. Our approach uses an attention-guided U-Net architecture to capture spatial patterns and temporal dependencies across modalities, combined with focal and dice loss functions to handle rainfall class imbalance defined by the India Meteorological Department (IMD). Experiments demonstrate that our multimodal framework consistently outperforms unimodal baselines and existing deep learning methods, especially in extreme rainfall categories. This work contributes a scalable framework, benchmark dataset, and state-of-the-art results for regional monsoon forecasting, climate resilience, and geospatial AI applications in India.

[160] SynDoc: A Hybrid Discriminative-Generative Framework for Enhancing Synthetic Domain-Adaptive Document Key Information Extraction

Yihao Ding,Soyeon Caren Han,Yanbei Jiang,Yan Li,Zechuan Li,Yifan Peng

Main category: cs.CV

TL;DR: SynDoc是一个混合判别式-生成式框架，通过合成数据生成和自适应指令调优提升领域文档关键信息抽取的性能。

Details Motivation: 现有大型（多模态）语言模型在领域适应的视觉丰富文档理解中存在幻觉、领域适应性不足和依赖大量微调数据的问题。

Contribution: 提出SynDoc框架，结合判别式和生成式模型的优势，通过合成数据生成和自适应指令调优改进领域知识抽取。

Method: 1. 使用结构信息抽取和领域特定查询生成高质量合成数据；2. 自适应指令调优优化判别模型；3. 递归推理机制迭代优化结果。

Result: SynDoc在领域适应的文档关键信息抽取任务中表现出高效、精确和可扩展的性能。

Insight: 混合判别式-生成式方法能够有效结合领域知识和通用世界知识，提升文档理解的稳定性和准确性。

Abstract: Domain-specific Visually Rich Document Understanding (VRDU) presents significant challenges due to the complexity and sensitivity of documents in fields such as medicine, finance, and material science. Existing Large (Multimodal) Language Models (LLMs/MLLMs) achieve promising results but face limitations such as hallucinations, inadequate domain adaptation, and reliance on extensive fine-tuning datasets. This paper introduces SynDoc, a novel framework that combines discriminative and generative models to address these challenges. SynDoc employs a robust synthetic data generation workflow, using structural information extraction and domain-specific query generation to produce high-quality annotations. Through adaptive instruction tuning, SynDoc improves the discriminative model’s ability to extract domain-specific knowledge. At the same time, a recursive inferencing mechanism iteratively refines the output of both models for stable and accurate predictions. This framework demonstrates scalable, efficient, and precise document understanding and bridges the gap between domain-specific adaptation and general world knowledge for document key information extraction tasks.

[161] Vid-Freeze: Protecting Images from Malicious Image-to-Video Generation via Temporal Freezing

Rohit Chowdhury,Aniruddha Bala,Rohan Jaiswal,Siddharth Roheda

Main category: cs.CV

TL;DR: Vid-Freeze是一种新型对抗攻击方法，通过精心设计的对抗扰动冻结图像到视频（I2V）模型的运动合成，防止恶意视频生成。

Details Motivation: 随着图像到视频生成模型的快速发展，恶意内容合成的风险增加。现有防御方法如I2VGuard效果有限，亟需一种更有效的方法来阻断运动合成。

Contribution: 提出Vid-Freeze方法，通过针对性抑制I2V模型的注意力机制，完全阻断运动合成，同时保持输入图像的语义保真度。

Method: 采用注意力抑制的对抗攻击，添加精心设计的扰动，使I2V模型生成的视频静止或接近静态。

Result: 实验证明Vid-Freeze能有效保护图像，阻断恶意视频生成，展示了注意力攻击在防御I2V滥用中的潜力。

Insight: 注意力机制是I2V模型的关键弱点，针对它的对抗攻击为防御恶意视频生成提供了一种新思路。

Abstract: The rapid progress of image-to-video (I2V) generation models has introduced significant risks, enabling video synthesis from static images and facilitating deceptive or malicious content creation. While prior defenses such as I2VGuard attempt to immunize images, effective and principled protection to block motion remains underexplored. In this work, we introduce Vid-Freeze - a novel attention-suppressing adversarial attack that adds carefully crafted adversarial perturbations to images. Our method explicitly targets the attention mechanism of I2V models, completely disrupting motion synthesis while preserving semantic fidelity of the input image. The resulting immunized images generate stand-still or near-static videos, effectively blocking malicious content creation. Our experiments demonstrate the impressive protection provided by the proposed approach, highlighting the importance of attention attacks as a promising direction for robust and proactive defenses against misuse of I2V generation models.

[162] Balanced Diffusion-Guided Fusion for Multimodal Remote Sensing Classification

Hao Liu,Yongjie Zheng,Yuhan Kang,Mingyang Zhang,Maoguo Gong,Lorenzo Bruzzone

Main category: cs.CV

TL;DR: 本文提出了一种平衡扩散引导融合（BDGF）框架，利用多模态扩散特征指导多分支网络进行土地覆盖分类，解决了多模态DDPM的模态不平衡问题，并通过自适应掩码策略和层次特征引导提升了分类性能。

Details Motivation: 多模态遥感数据分类中，现有DDPMs存在模态不平衡问题，且如何有效利用扩散特征引导多样性特征提取尚不明确。

Contribution: 1. 提出了BDGF框架，通过自适应模态掩码策略平衡数据分布；2. 设计了层次特征引导机制，融合CNN、Mamba和Transformer网络；3. 提出互学习策略增强分支间协作。

Method: 采用自适应模态掩码策略平衡DDPMs的数据分布，通过特征融合、组通道注意力和交叉注意力机制实现层次特征引导。

Result: 在四个多模态遥感数据集上验证了BDGF的优越分类性能。

Insight: 平衡数据分布和层次特征引导是多模态遥感分类的关键，互学习策略能有效提升模型协作能力。

Abstract: Deep learning-based techniques for the analysis of multimodal remote sensing data have become popular due to their ability to effectively integrate complementary spatial, spectral, and structural information from different sensors. Recently, denoising diffusion probabilistic models (DDPMs) have attracted attention in the remote sensing community due to their powerful ability to capture robust and complex spatial-spectral distributions. However, pre-training multimodal DDPMs may result in modality imbalance, and effectively leveraging diffusion features to guide complementary diversity feature extraction remains an open question. To address these issues, this paper proposes a balanced diffusion-guided fusion (BDGF) framework that leverages multimodal diffusion features to guide a multi-branch network for land-cover classification. Specifically, we propose an adaptive modality masking strategy to encourage the DDPMs to obtain a modality-balanced rather than spectral image-dominated data distribution. Subsequently, these diffusion features hierarchically guide feature extraction among CNN, Mamba, and transformer networks by integrating feature fusion, group channel attention, and cross-attention mechanisms. Finally, a mutual learning strategy is developed to enhance inter-branch collaboration by aligning the probability entropy and feature similarity of individual subnetworks. Extensive experiments on four multimodal remote sensing datasets demonstrate that the proposed method achieves superior classification performance. The code is available at https://github.com/HaoLiu-XDU/BDGF.

[163] Seeing Symbols, Missing Cultures: Probing Vision-Language Models’ Reasoning on Fire Imagery and Cultural Meaning

Haorui Yu,Qiufeng Yi,Yijia Chu,Yang Zhao

Main category: cs.CV

TL;DR: 论文研究了视觉语言模型（VLMs）在文化图像理解中的局限性，发现模型倾向于依赖表面模式匹配而非真正的文化理解。通过诊断框架，揭示了模型在处理西方文化与非西方文化图像时的系统性偏见。

Details Motivation: 尽管VLMs在多模态任务中表现出色，但其文化理解能力存在明显缺陷，尤其是在处理非西方文化或紧急场景时。研究旨在揭示这些模型的局限性，强调文化理解和公平性的重要性。

Contribution: 论文的主要贡献是提出了一个诊断框架，用于评估VLMs在文化图像理解上的表现，揭示了模型在处理西方与非西方文化图像时的偏见和误判（如将紧急场景误分类为庆祝活动）。

Method: 研究通过分类和解释分析，测试了多个VLMs在处理西方节日、非西方传统和紧急场景时的表现。

Result: 实验表明，VLMs能够正确识别西方节日，但在处理非西方文化活动和紧急场景时表现不佳，经常产生模糊标签或严重误分类。

Insight: 研究揭示VLMs依赖符号化捷径而非深层文化理解，强调了在多模态系统中融入文化评估的必要性，以确保模型的解释性和公平性。

Abstract: Vision-Language Models (VLMs) often appear culturally competent but rely on superficial pattern matching rather than genuine cultural understanding. We introduce a diagnostic framework to probe VLM reasoning on fire-themed cultural imagery through both classification and explanation analysis. Testing multiple models on Western festivals, non-Western traditions, and emergency scenes reveals systematic biases: models correctly identify prominent Western festivals but struggle with underrepresented cultural events, frequently offering vague labels or dangerously misclassifying emergencies as celebrations. These failures expose the risks of symbolic shortcuts and highlight the need for cultural evaluation beyond accuracy metrics to ensure interpretable and fair multimodal systems.

[164] C3-OWD: A Curriculum Cross-modal Contrastive Learning Framework for Open-World Detection

Siheng Wang,Zhengdao Li,Yanshu Li,Canran Xiao,Haibo Zhan,Zhengtao Yao,Xuzhi Zhang,Jiale Kang,Linshan Li,Weiming Liu,Zhikang Dong,Jifeng Shen,Junhao Dong,Qiang Sun,Piotr Koniusz

Main category: cs.CV

TL;DR: C3-OWD提出了一种课程跨模态对比学习框架，同时解决了开放世界目标检测的鲁棒性和多样性问题。

Details Motivation: 现有方法在开放世界目标检测中要么缺乏鲁棒性，要么多样性不足，无法同时应对未见类别和极端环境。

Contribution: 提出了C3-OWD框架，通过两阶段课程学习和EMA机制，统一了鲁棒性和多样性目标。

Method: 包括两个阶段：第一阶段通过RGBT数据预训练增强鲁棒性；第二阶段通过视觉-语言对齐提升泛化能力。使用EMA机制防止灾难性遗忘。

Result: 在FLIR、OV-COCO和OV-LVIS上表现优异，分别达到80.1 AP50、48.6 AP50-Novel和35.7 mAPr。

Insight: 通过课程学习和EMA机制，可以同时优化鲁棒性和多样性，为开放世界检测提供了新思路。

Abstract: Object detection has advanced significantly in the closed-set setting, but real-world deployment remains limited by two challenges: poor generalization to unseen categories and insufficient robustness under adverse conditions. Prior research has explored these issues separately: visible-infrared detection improves robustness but lacks generalization, while open-world detection leverages vision-language alignment strategy for category diversity but struggles under extreme environments. This trade-off leaves robustness and diversity difficult to achieve simultaneously. To mitigate these issues, we propose \textbf{C3-OWD}, a curriculum cross-modal contrastive learning framework that unifies both strengths. Stage1 enhances robustness by pretraining with RGBT data, while Stage2 improves generalization via vision-language alignment. To prevent catastrophic forgetting between two stages, we introduce an Exponential Moving Average (EMA) mechanism that theoretically guarantees preservation of pre-stage performance with bounded parameter lag and function consistency. Experiments on FLIR, OV-COCO, and OV-LVIS demonstrate the effectiveness of our approach: C3-OWD achieves $80.1$ AP$^{50}$ on FLIR, $48.6$ AP$^{50}_{\text{Novel}}$ on OV-COCO, and $35.7$ mAP$_r$ on OV-LVIS, establishing competitive performance across both robustness and diversity evaluations. Code available at: https://github.com/justin-herry/C3-OWD.git.

[165] Decoupling Reasoning and Perception: An LLM-LMM Framework for Faithful Visual Reasoning

Hongrui Jia,Chaoya Jiang,Shikun Zhang,Wei Ye

Main category: cs.CV

TL;DR: 该论文提出了一种无需训练的视觉推理框架，通过解耦推理和感知过程，利用LLM负责高级推理，LMM仅作为视觉问答引擎，显著减少了视觉无根据的推理步骤并提高了推理保真度。

Details Motivation: 当前大型多模态模型（LMMs）在扩展推理链时过度依赖文本逻辑，逐渐脱离视觉信息，导致推理错误。为了解决这一问题，作者提出解耦推理和感知过程。

Contribution: 1) 提出了一种无需训练的视觉推理框架；2) 通过LLM和LMM的分工协作，分别处理推理和感知任务；3) 显著减少了视觉无根据的推理步骤并提高了推理保真度。

Method: 1) 使用LLM负责高级推理；2) LMM仅作为视觉问答引擎，提供所需视觉信息；3) 框架轻量化且无需额外训练或架构修改。

Result: 综合评估表明，该框架有效控制了视觉推理过程，显著减少了视觉无根据的推理步骤，并大幅提高了推理保真度。

Insight: 解耦推理和感知任务是提高多模态模型推理保真度的有效方法，无需复杂训练或架构调整即可实现性能提升。

Abstract: Significant advancements in the reasoning capabilities of Large Language Models (LLMs) are now driven by test-time scaling laws, particularly those leveraging extended Chain-of-Thought (CoT) reasoning. Inspired by these breakthroughs, researchers have extended these paradigms to Large Multimodal Models (LMMs). However, a critical limitation emerges: as their reasoning chains extend, LMMs increasingly rely on textual logic, progressively losing grounding in the underlying visual information. This leads to reasoning paths that diverge from the image content, culminating in erroneous conclusions. To address this, we introduce a strikingly simple yet effective training-free visual-reasoning pipeline. The core concept is to decouple the reasoning and perception processes. A powerful LLM orchestrates the high-level reasoning, strategically interrogating a LMM to extract specific visual information required for its logical chain. The LMM, in turn, functions exclusively as a visual question-answering engine, supplying the necessary perceptual details on demand. This lightweight, plug-and-play approach requires no additional training or architectural changes. Comprehensive evaluations validate that our framework effectively governs the visual reasoning process, leading to a significant reduction in visually-unfounded reasoning steps and a substantial improvement in reasoning fidelity.

[166] LRPO: Enhancing Blind Face Restoration through Online Reinforcement Learning

Bin Wu,Yahui Liu,Chi Zhang,Yao Zhao,Wei Wang

Main category: cs.CV

TL;DR: LRPO首次将在线强化学习应用于盲人脸恢复任务，通过采样候选者的奖励优化策略网络，提升恢复质量。

Details Motivation: 盲人脸恢复（BFR）的大解空间导致常见伪影（如细节缺失和身份模糊），传统方法难以解决。

Contribution: 1) 提出LRPO框架，首次将在线RL用于BFR；2) 设计复合奖励函数、真实引导的似然正则化和噪声级别优势分配策略。

Method: 通过在线RL优化策略网络，结合复合奖励函数、似然正则化和噪声级别优势分配。

Result: LRPO显著超越基线方法，实现最先进的恢复质量。

Insight: 平衡感知质量与保真度是BFR的关键，RL提供了一种细化解空间的灵活方法。

Abstract: Blind Face Restoration (BFR) encounters inherent challenges in exploring its large solution space, leading to common artifacts like missing details and identity ambiguity in the restored images. To tackle these challenges, we propose a Likelihood-Regularized Policy Optimization (LRPO) framework, the first to apply online reinforcement learning (RL) to the BFR task. LRPO leverages rewards from sampled candidates to refine the policy network, increasing the likelihood of high-quality outputs while improving restoration performance on low-quality inputs. However, directly applying RL to BFR creates incompatibility issues, producing restoration results that deviate significantly from the ground truth. To balance perceptual quality and fidelity, we propose three key strategies: 1) a composite reward function tailored for face restoration assessment, 2) ground-truth guided likelihood regularization, and 3) noise-level advantage assignment. Extensive experiments demonstrate that our proposed LRPO significantly improves the face restoration quality over baseline methods and achieves state-of-the-art performance.

[167] DentVLM: A Multimodal Vision-Language Model for Comprehensive Dental Diagnosis and Enhanced Clinical Practice

Zijie Meng,Jin Hao,Xiwei Dai,Yang Feng,Jiaxiang Liu,Bin Feng,Huikai Wu,Xiaotang Gai,Hengchuan Zhu,Tianxiang Hu,Yangyang Wu,Hongxia Xu,Jin Li,Jun Xiao,Xiaoqiang Liu,Joey Tianyi Zhou,Fudong Zhu,Zhihe Zhao,Lunguo Xia,Bing Fang,Jimeng Sun,Jian Wu,Zuozhu Liu

Main category: cs.CV

TL;DR: 这篇论文介绍了DentVLM，一个多模态视觉-语言模型，用于全面的口腔疾病诊断和临床实践增强。

Details Motivation: 当前AI模型在口腔疾病诊断中往往只能处理单一任务，无法满足临床复杂的多模态需求，因此需要开发一个更全面的解决方案。

Contribution: DentVLM是一个基于大规模双语数据集的多模态模型，能够在36个诊断任务中显著优于现有模型，提升诊断准确性并减少临床诊断时间。

Method: DentVLM采用了大规模的双语数据集（110,447张图像和246万VQA对），支持7种2D口腔成像模态的诊断任务。

Result: DentVLM在临床研究中表现优异，超越初级和高级牙医的诊断能力，并显著减少诊断时间（15-22%）。

Insight: DentVLM不仅提升了诊断效率，还展示了在家庭、医院和多代理协作等场景中的应用潜力，有助于改善医疗资源不均。

Abstract: Diagnosing and managing oral diseases necessitate advanced visual interpretation across diverse imaging modalities and integrated information synthesis. While current AI models excel at isolated tasks, they often fall short in addressing the complex, multimodal requirements of comprehensive clinical dental practice. Here we introduce DentVLM, a multimodal vision-language model engineered for expert-level oral disease diagnosis. DentVLM was developed using a comprehensive, large-scale, bilingual dataset of 110,447 images and 2.46 million visual question-answering (VQA) pairs. The model is capable of interpreting seven 2D oral imaging modalities across 36 diagnostic tasks, significantly outperforming leading proprietary and open-source models by 19.6% higher accuracy for oral diseases and 27.9% for malocclusions. In a clinical study involving 25 dentists, evaluating 1,946 patients and encompassing 3,105 QA pairs, DentVLM surpassed the diagnostic performance of 13 junior dentists on 21 of 36 tasks and exceeded that of 12 senior dentists on 12 of 36 tasks. When integrated into a collaborative workflow, DentVLM elevated junior dentists’ performance to senior levels and reduced diagnostic time for all practitioners by 15-22%. Furthermore, DentVLM exhibited promising performance across three practical utility scenarios, including home-based dental health management, hospital-based intelligent diagnosis and multi-agent collaborative interaction. These findings establish DentVLM as a robust clinical decision support tool, poised to enhance primary dental care, mitigate provider-patient imbalances, and democratize access to specialized medical expertise within the field of dentistry.

[168] Dynamic-TreeRPO: Breaking the Independent Trajectory Bottleneck with Structured Sampling

Xiaolong Fu,Lichen Ma,Zipeng Guo,Gaojing Zhou,Chongxiao Wang,ShiPing Dong,Shizhe Zhou,Shizhe Zhou,Ximan Liu,Jingling Fu,Tan Lit Sin,Yu Shi,Zhen Chen,Junshi Huang,Jason Li

Main category: cs.CV

TL;DR: 本文提出Dynamic-TreeRPO方法，通过树结构采样和动态噪声强度改进文本到图像生成的轨迹搜索效率，并结合LayerTuning-RL范式优化训练。

Details Motivation: 现有RL方法在文本到图像生成中存在探索低效和采样冗余问题。

Contribution: 1. 提出Dynamic-TreeRPO，结合树结构采样和动态噪声强度；2. 设计LayerTuning-RL范式，动态加权损失函数；3. 显著提升生成质量和训练效率。

Method: 1. 滑动窗口采样策略转化为树结构搜索；2. GRPO优化和约束SDE采样；3. 动态噪声强度设计；4. LayerTuning-RL范式动态加权PRM损失。

Result: 在HPS-v2.1、PickScore和ImageReward基准上分别提升4.9%、5.91%和8.66%，训练效率提高近50%。

Insight: 树结构采样共享前缀路径可降低计算开销，动态噪声强度提升探索多样性。

Abstract: The integration of Reinforcement Learning (RL) into flow matching models for text-to-image (T2I) generation has driven substantial advances in generation quality. However, these gains often come at the cost of exhaustive exploration and inefficient sampling strategies due to slight variation in the sampling group. Building on this insight, we propose Dynamic-TreeRPO, which implements the sliding-window sampling strategy as a tree-structured search with dynamic noise intensities along depth. We perform GRPO-guided optimization and constrained Stochastic Differential Equation (SDE) sampling within this tree structure. By sharing prefix paths of the tree, our design effectively amortizes the computational overhead of trajectory search. With well-designed noise intensities for each tree layer, Dynamic-TreeRPO can enhance the variation of exploration without any extra computational cost. Furthermore, we seamlessly integrate Supervised Fine-Tuning (SFT) and RL paradigm within Dynamic-TreeRPO to construct our proposed LayerTuning-RL, reformulating the loss function of SFT as a dynamically weighted Progress Reward Model (PRM) rather than a separate pretraining method. By associating this weighted PRM with dynamic-adaptive clipping bounds, the disruption of exploration process in Dynamic-TreeRPO is avoided. Benefiting from the tree-structured sampling and the LayerTuning-RL paradigm, our model dynamically explores a diverse search space along effective directions. Compared to existing baselines, our approach demonstrates significant superiority in terms of semantic consistency, visual fidelity, and human preference alignment on established benchmarks, including HPS-v2.1, PickScore, and ImageReward. In particular, our model outperforms SoTA by $4.9%$, $5.91%$, and $8.66%$ on those benchmarks, respectively, while improving the training efficiency by nearly $50%$.

[169] Test-time Uncertainty Estimation for Medical Image Registration via Transformation Equivariance

Lin Tian,Xiaoling Hu,Juan Eugenio Iglesias

Main category: cs.CV

TL;DR: 该论文提出了一种基于变换等变性的测试时不确定性估计框架，适用于任何预训练的医学图像配准网络，无需修改架构或重新训练。通过分析输入空间扰动下网络预测的方差，分解出不确定性为内在噪声和系统误差漂移两部分，实验表明不确定性图与配准误差高度相关。

Details Motivation: 当前深度配准网络缺乏对其预测可靠性的明确指示，限制了其在临床应用中的安全性。现有不确定性估计方法需修改网络或重新训练，不够灵活。

Contribution: 1. 提出了一种兼容预训练网络的测试时不确定性估计框架。2. 基于变换等变性理论，分解不确定性为内在噪声和系统误差漂移。

Method: 利用变换等变性原理，通过在输入图像上施加空间扰动并分析预测方差来估计不确定性，无需修改网络或重新训练。

Result: 在多种解剖结构和配准模型上验证，不确定性图与配准误差显著相关，能有效标识需注意的区域。

Insight: 变换等变性为不确定性估计提供了理论依据，框架的灵活性使其可直接应用于预训练网络，推动医学图像配准的临床安全部署。

Abstract: Accurate image registration is essential for downstream applications, yet current deep registration networks provide limited indications of whether and when their predictions are reliable. Existing uncertainty estimation strategies, such as Bayesian methods, ensembles, or MC dropout, require architectural changes or retraining, limiting their applicability to pretrained registration networks. Instead, we propose a test-time uncertainty estimation framework that is compatible with any pretrained networks. Our framework is grounded in the transformation equivariance property of registration, which states that the true mapping between two images should remain consistent under spatial perturbations of the input. By analyzing the variance of network predictions under such perturbations, we derive a theoretical decomposition of perturbation-based uncertainty in registration. This decomposition separates into two terms: (i) an intrinsic spread, reflecting epistemic noise, and (ii) a bias jitter, capturing how systematic error drifts under perturbations. Across four anatomical structures (brain, cardiac, abdominal, and lung) and multiple registration models (uniGradICON, SynthMorph), the uncertainty maps correlate consistently with registration errors and highlight regions requiring caution. Our framework turns any pretrained registration network into a risk-aware tool at test time, placing medical image registration one step closer to safe deployment in clinical and large-scale research settings.

[170] GRAPE: Let GPRO Supervise Query Rewriting by Ranking for Retrieval

Zhaohua Zhang,Jianhuan Zhuo,Muxi Chen,Chenchen Zhao,Wenyu Jiang,Tianwen Jiang,Mingyang Chen,Yu Tang,Qiuyong Xiao,Jihong Zhang,Zhixun Su

Main category: cs.CV

TL;DR: GRAPE通过引入GRPO和排序奖励信号，优化基于LLM的查询重写策略，显著提升了检索系统在分布偏移（如多语言、长文本和多模态）下的性能，Recall@10平均提高4.9%。

Details Motivation: CLIP模型在大规模检索系统中表现优异，但在输入分布与训练数据不同的任务（如多语言、长文本或多模态查询）中效果不佳。传统方法依赖LLM的查询重写，但缺乏监督信号导致效果不理想。GRAPE旨在通过排序信号优化查询重写以解决这一问题。

Contribution: 1.提出了GRAPE框架，将GRPO和排序奖励信号引入LLM的查询重写中；2.设计了抑制分数膨胀的语料相关排序奖励；3.在多种分布偏移（多语言、长文本、多模态）下验证了有效性。

Method: 1.利用GRPO优化查询重写，使查询更符合检索器训练分布；2.提出语料相关排序奖励，解决相似度分数膨胀问题；3.通过多任务实验验证方法有效性。

Result: 在多语言（Flickr30k-CN等）、长文本（Wikipedia）和多模态（CIRR）数据集上，Recall@10平均提升4.9%。

Insight: 排序信号能有效引导LLM生成更优查询，语料相关奖励设计是关键，能避免分数膨胀问题，提升检索性能。

Abstract: The CLIP model has become a cornerstone of large-scale retrieval systems by aligning text and image data in a unified embedding space. Despite its simplicity and efficiency, CLIP struggles when applied to tasks whose input distributions diverge from its training corpus, such as queries with multilingual, long-form, or multimodal differences. To avoid costly retraining, existing methods mainly adopt query-rewriting strategies with large language models (LLMs), aiming to mitigate distribution gaps at the query level. However, due to the lack of supervision signals, LLMs fail to generate the optimal one that fits the training distribution. We address this challenge with GRAPE (Grouped Ranking-Aware Policy Optimization Enhancement), a plug-and-play enhancement approach that incorporates ranking signals into retrieval-guided query rewriting with LLMs. Intuitively, GRAPE proposes to leverage GRPO to bridge distributional differences – including length, multilingual, and modality shifts – by transforming queries into forms better aligned with the retriever’s training distribution. However, our preliminary experiment finds that naively finetuning LLM with similarity scores can lead to score inflation, where nearly all candidates are assigned unexpectedly high scores regardless of their true relevance. To address score inflation, we propose a corpus-relative ranking-based reward, which explicitly aligns optimization with ranking metrics while suppressing spurious score inflation. Extensive experiments demonstrate that GRAPE consistently improves retrieval performance under distributional shifts – including multilingual differences (Flickr30k-CN, CVLUE, XM3600), length differences (Wikipedia), and multimodal differences (CIRR) – achieving an average improvement of 4.9% in Recall@10. The code is available at https://github.com/Chinese0123456/GRAPE.git

[171] UniPose: Unified Cross-modality Pose Prior Propagation towards RGB-D data for Weakly Supervised 3D Human Pose Estimation

Jinghong Zheng,Changlong Jiang,Jiaqi Li,Haohong Kuang,Hang Xu,Tingbing Yan

Main category: cs.CV

TL;DR: UniPose提出了一种统一的跨模态姿态先验传播方法，用于弱监督的3D人体姿态估计，通过自监督学习将2D姿态标注从RGB数据集迁移到3D域，避免了多视角校准或合成-真实数据偏移问题。

Details Motivation: 传统3D姿态估计依赖大量3D关键点标注，标注成本高昂。UniPose旨在利用易获取的RGB-D序列和现成的2D姿态标注，实现弱监督的3D姿态估计。

Contribution: 1）提出UniPose方法，通过自监督学习将2D姿态标注迁移到3D域；2）引入跨模态交互和空间-时间约束提升3D姿态估计；3）在CMU Panoptic和ITOP数据集上表现优异，接近全监督方法。

Method: 1）利用2D姿态标注作为弱监督；2）通过自监督学习和空间-时间约束（如身体对称性和关节运动）训练点云网络；3）提出2D到3D的反投影损失和跨模态交互机制；4）采用锚点到关节的预测方法进行3D提升。

Result: 在CMU Panoptic和ITOP数据集上，UniPose性能接近全监督方法，加入无标注数据（如NTU RGB+D 60）后表现更优。

Insight: UniPose通过跨模态学习和自监督机制，显著减少了3D姿态估计对标注数据的依赖，展示了在弱监督条件下的高效性和实用性。

Abstract: In this paper, we present UniPose, a unified cross-modality pose prior propagation method for weakly supervised 3D human pose estimation (HPE) using unannotated single-view RGB-D sequences (RGB, depth, and point cloud data). UniPose transfers 2D HPE annotations from large-scale RGB datasets (e.g., MS COCO) to the 3D domain via self-supervised learning on easily acquired RGB-D sequences, eliminating the need for labor-intensive 3D keypoint annotations. This approach bridges the gap between 2D and 3D domains without suffering from issues related to multi-view camera calibration or synthetic-to-real data shifts. During training, UniPose leverages off-the-shelf 2D pose estimations as weak supervision for point cloud networks, incorporating spatial-temporal constraints like body symmetry and joint motion. The 2D-to-3D back-projection loss and cross-modality interaction further enhance this process. By treating the point cloud network’s 3D HPE results as pseudo ground truth, our anchor-to-joint prediction method performs 3D lifting on RGB and depth networks, making it more robust against inaccuracies in 2D HPE results compared to state-of-the-art methods. Experiments on CMU Panoptic and ITOP datasets show that UniPose achieves comparable performance to fully supervised methods. Incorporating large-scale unlabeled data (e.g., NTU RGB+D 60) enhances its performance under challenging conditions, demonstrating its potential for practical applications. Our proposed 3D lifting method also achieves state-of-the-art results.

[172] WorldSplat: Gaussian-Centric Feed-Forward 4D Scene Generation for Autonomous Driving

Ziyue Zhu,Zhanqian Wu,Zhenxin Zhu,Lijun Zhou,Haiyang Sun,Bing Wan,Kun Ma,Guang Chen,Hangjun Ye,Jin Xie,jian Yang

Main category: cs.CV

TL;DR: WorldSplat提出了一种前馈框架，用于生成4D自动驾驶场景，结合了生成与重建的优势，通过4D高斯模型和多模态信息实现了高质量的新视角合成。

Details Motivation: 现有的驾驶场景生成方法在3D一致性和多视角覆盖上表现不足，而3D/4D重建方法缺乏生成能力。WorldSplat旨在结合两者的优势，解决生成与重构之间的矛盾。

Contribution: 1) 提出了一个前馈框架WorldSplat，用于4D驾驶场景生成；2) 引入了4D感知的潜在扩散模型，生成像素对齐的4D高斯；3) 通过增强的视频扩散模型优化新视角视频。

Method: 采用两步法：1) 使用多模态信息集成的4D潜在扩散模型生成4D高斯；2) 通过视频扩散模型对高斯渲染的新视角视频进行优化。

Result: 在基准数据集上的实验表明，WorldSplat能够生成高保真、时空一致的多视角驾驶视频。

Insight: WorldSplat展示了生成与重建技术的结合潜力，为自动驾驶系统提供了更具可扩展性和可控性的训练数据。

Abstract: Recent advances in driving-scene generation and reconstruction have demonstrated significant potential for enhancing autonomous driving systems by producing scalable and controllable training data. Existing generation methods primarily focus on synthesizing diverse and high-fidelity driving videos; however, due to limited 3D consistency and sparse viewpoint coverage, they struggle to support convenient and high-quality novel-view synthesis (NVS). Conversely, recent 3D/4D reconstruction approaches have significantly improved NVS for real-world driving scenes, yet inherently lack generative capabilities. To overcome this dilemma between scene generation and reconstruction, we propose \textbf{WorldSplat}, a novel feed-forward framework for 4D driving-scene generation. Our approach effectively generates consistent multi-track videos through two key steps: ((i)) We introduce a 4D-aware latent diffusion model integrating multi-modal information to produce pixel-aligned 4D Gaussians in a feed-forward manner. ((ii)) Subsequently, we refine the novel view videos rendered from these Gaussians using a enhanced video diffusion model. Extensive experiments conducted on benchmark datasets demonstrate that \textbf{WorldSplat} effectively generates high-fidelity, temporally and spatially consistent multi-track novel view driving videos.

[173] SPIKE-RL: Video-LLMs meet Bayesian Surprise

Sahithya Ravi,Aditya Chinchure,Raymond T. Ng,Leonid Sigal,Vered Shwartz

Main category: cs.CV

TL;DR: SPIKE-RL是一个通过贝叶斯惊喜量化视频中关键时刻的框架，结合强化学习优化视频帧采样策略，从而提升Video-LLMs在下游任务中的表现。

Details Motivation: 现有Video-LLMs通常均匀采样视频帧，容易错过关键叙事时刻，希望通过量化视觉证据引发的信念更新，识别视频中的关键时刻。

Contribution: 提出SPIKE框架量化贝叶斯惊喜，并开发SPIKE-RL通过强化学习优化信念假设，实现基于惊喜加权的帧采样策略。

Method: SPIKE利用视觉证据触发信念更新定位惊喜时刻；SPIKE-RL结合GRPO优化信念假设，以视频标题为奖励信号指导帧采样。

Result: 在五个下游任务中，惊喜加权采样优于均匀采样，显著提升了模型性能。

Insight: 通过跟踪信念更新和量化惊喜，Video-LLMs可以动态调整理解，增强对新信息的适应能力。

Abstract: Real-world videos often show routine activities punctuated by memorable, surprising events. However, most Video-LLMs process videos by sampling frames uniformly, likely missing critical moments that define a video’s narrative. We introduce SPIKE, an inference-time framework that quantifies Bayesian Surprise as the belief update triggered by new visual evidence in the video stream, identifying moments where new visual evidence conflicts with prior beliefs. SPIKE effectively localizes surprise in videos, strongly correlated with humans on positive (FunQA) and negative (Oops!) surprise benchmarks. Since the beliefs of zero-shot Video-LLMs are often suboptimal, we develop SPIKE-RL, which leverages GRPO to optimize belief hypotheses based on a reward signal from the video caption. SPIKE and SPIKE-RL guide query-agnostic surprise-weighted frame sampling, which allocates more frames to interesting moments in the video. With this strategy, we achieve consistent performance gains on five downstream benchmarks over uniform sampling. By enabling Video-LLMs to track beliefs and register surprise, our work paves the way for more robust models that can revise their understanding in response to new information.

[174] FM-SIREN & FM-FINER: Nyquist-Informed Frequency Multiplier for Implicit Neural Representation with Periodic Activation

Mohammed Alsakabi,Wael Mobeirek,John M. Dolan,Ozan K. Tonguz

Main category: cs.CV

TL;DR: FM-SIREN & FM-FINER提出了一种新的频率乘数设计，解决了周期性激活网络中神经元冗余问题，提升了隐式神经表示的表达能力。

Details Motivation: 现有的基于周期性激活的隐式神经表示网络（如SIREN和FINER）存在特征冗余问题，固定频率乘数导致神经元捕获的重叠频率分量限制了多层感知机的表达能力。

Contribution: 1.提出了FM-SIREN和FM-FINER，通过Nyquist-informed神经元特异性频率乘数减少特征冗余；2.无需额外超参数调优或网络深度即可引入频率多样性。

Method: 从离散正弦变换（DST）中获取灵感，为周期性激活函数分配Nyquist-informed频率乘数，确保神经元间的频率多样性。

Result: 该方法在1D音频、2D图像、3D形状拟合及神经辐射场（NeRF）合成任务中表现出色，特征冗余减少近50%，显著优于基线模型。

Insight: 频率多样性的引入是提升隐式神经表示表达能力的关键，且无需增加网络复杂度即可实现显著改进。

Abstract: Existing periodic activation-based implicit neural representation (INR) networks, such as SIREN and FINER, suffer from hidden feature redundancy, where neurons within a layer capture overlapping frequency components due to the use of a fixed frequency multiplier. This redundancy limits the expressive capacity of multilayer perceptrons (MLPs). Drawing inspiration from classical signal processing methods such as the Discrete Sine Transform (DST), we propose FM-SIREN and FM-FINER, which assign Nyquist-informed, neuron-specific frequency multipliers to periodic activations. Unlike existing approaches, our design introduces frequency diversity without requiring hyperparameter tuning or additional network depth. This simple yet principled modification reduces the redundancy of features by nearly 50% and consistently improves signal reconstruction across diverse INR tasks, including fitting 1D audio, 2D image and 3D shape, and synthesis of neural radiance fields (NeRF), outperforming their baseline counterparts while maintaining efficiency.

[175] FoR-SALE: Frame of Reference-guided Spatial Adjustment in LLM-based Diffusion Editing

Tanawan Premsri,Parisa Kordjamshidi

Main category: cs.CV

TL;DR: FoR-SALE 是一种基于 FoR（参考框架）的扩散编辑方法，通过空间调整提升文本到图像生成的空间一致性，显著改善现有模型的性能。

Details Motivation: 目前的文本到图像生成模型在多模态空间中缺乏对人类空间描述（如不同视角）的准确理解能力，FoR-SALE 旨在解决这一问题。

Contribution: 提出 FoR-SALE，扩展了 SLD 框架，通过 FoR 指导的空间调整改进文本到图像生成的空间一致性，并设计了新的潜在空间操作。

Method: 利用视觉模块提取图像空间配置，将空间描述映射到相机视角，检测语言与视觉的不对齐后生成编辑操作。

Result: 在两个空间理解基准测试中，FoR-SALE 将 SOTA 模型的性能提升高达 5.3%，仅需一轮校正。

Insight: FoR 的引入显著提升了模型对复杂空间描述的理解能力，潜在空间操作为未来的图像编辑任务提供了新思路。

Abstract: Frame of Reference (FoR) is a fundamental concept in spatial reasoning that humans utilize to comprehend and describe space. With the rapid progress in Multimodal Language models, the moment has come to integrate this long-overlooked dimension into these models. In particular, in text-to-image (T2I) generation, even state-of-the-art models exhibit a significant performance gap when spatial descriptions are provided from perspectives other than the camera. To address this limitation, we propose Frame of Reference-guided Spatial Adjustment in LLM-based Diffusion Editing (FoR-SALE), an extension of the Self-correcting LLM-controlled Diffusion (SLD) framework for T2I. For-Sale evaluates the alignment between a given text and an initially generated image, and refines the image based on the Frame of Reference specified in the spatial expressions. It employs vision modules to extract the spatial configuration of the image, while simultaneously mapping the spatial expression to a corresponding camera perspective. This unified perspective enables direct evaluation of alignment between language and vision. When misalignment is detected, the required editing operations are generated and applied. FoR-SALE applies novel latent-space operations to adjust the facing direction and depth of the generated images. We evaluate FoR-SALE on two benchmarks specifically designed to assess spatial understanding with FoR. Our framework improves the performance of state-of-the-art T2I models by up to 5.3% using only a single round of correction.

[176] 3DPCNet: Pose Canonicalization for Robust Viewpoint-Invariant 3D Kinematic Analysis from Monocular RGB cameras

Tharindu Ekanayake,Constantino Álvarez Casado,Miguel Bordallo López

Main category: cs.CV

TL;DR: 3DPCNet是一个紧凑的、与姿态估计器无关的模块，用于将3D姿态从相机坐标系转换到一致的身体中心坐标系，从而消除视角依赖性。

Details Motivation: 单目3D姿态估计器生成的相机中心骨架会产生视角依赖的运动信号，这在健康和体育科学等应用中的比较分析中带来困难。

Contribution: 提出了3DPCNet，一种混合编码器架构，通过自监督学习将姿态对齐到一致的身体中心坐标系，显著提升了姿态对齐的精度。

Method: 采用图卷积网络和Transformer的混合编码器，通过门控交叉注意力融合局部骨骼特征和全局上下文，预测6D旋转并将其映射到SO(3)矩阵。

Result: 在MM-Fi数据集上，3DPCNet将平均旋转误差从20°降低到3.4°，关节位置误差从64mm降低到47mm。在TotalCapture数据集上，生成的加速度信号与IMU地面真实数据表现出良好的视觉一致性。

Insight: 3DPCNet通过消除视角依赖性，显著提高了运动分析的物理合理性，为单目3D姿态估计的实际应用提供了有力工具。

Abstract: Monocular 3D pose estimators produce camera-centered skeletons, creating view-dependent kinematic signals that complicate comparative analysis in applications such as health and sports science. We present 3DPCNet, a compact, estimator-agnostic module that operates directly on 3D joint coordinates to rectify any input pose into a consistent, body-centered canonical frame. Its hybrid encoder fuses local skeletal features from a graph convolutional network with global context from a transformer via a gated cross-attention mechanism. From this representation, the model predicts a continuous 6D rotation that is mapped to an $SO(3)$ matrix to align the pose. We train the model in a self-supervised manner on the MM-Fi dataset using synthetically rotated poses, guided by a composite loss ensuring both accurate rotation and pose reconstruction. On the MM-Fi benchmark, 3DPCNet reduces the mean rotation error from over 20$^{\circ}$ to 3.4$^{\circ}$ and the Mean Per Joint Position Error from ~64 mm to 47 mm compared to a geometric baseline. Qualitative evaluations on the TotalCapture dataset further demonstrate that our method produces acceleration signals from video that show strong visual correspondence to ground-truth IMU sensor data, confirming that our module removes viewpoint variability to enable physically plausible motion analysis.

[177] No Concept Left Behind: Test-Time Optimization for Compositional Text-to-Image Generation

Mohammad Hossein Sameti,Amir M. Mansourian,Arash Marioriyad,Soheil Fadaee Oshyani,Mohammad Hossein Rohban,Mahdieh Soleymani Baghshah

Main category: cs.CV

TL;DR: 本文提出了一种细粒度的测试时优化框架，通过分解输入提示为语义概念，在全局和概念级别评估对齐，提升文本到图像生成的组合忠实度。

Details Motivation: 现有的文本到图像模型在处理复杂提示时，常遗漏或错误表示特定对象和属性，需要一种无需重新训练的方法改进生成效果。

Contribution: 提出一种基于概念分解的测试时优化框架，利用细粒度CLIP评估概念对齐，并通过迭代提示优化改进生成结果。

Method: 分解输入提示为语义概念，使用细粒度CLIP计算概念级对齐反馈，结合大语言模型迭代优化提示。

Result: 在DrawBench和CompBench数据集上，方法显著提升了概念覆盖和人类评估的生成忠实度。

Insight: 细粒度的概念对齐评估是提升文本到图像生成效果的关键，测试时优化可以在不重新训练模型的情况下改进生成质量。

Abstract: Despite recent advances in text-to-image (T2I) models, they often fail to faithfully render all elements of complex prompts, frequently omitting or misrepresenting specific objects and attributes. Test-time optimization has emerged as a promising approach to address this limitation by refining generation without the need for retraining. In this paper, we propose a fine-grained test-time optimization framework that enhances compositional faithfulness in T2I generation. Unlike most of prior approaches that rely solely on a global image/text similarity score, our method decomposes the input prompt into semantic concepts and evaluates alignment at both the global and concept levels. A fine-grained variant of CLIP is used to compute concept-level correspondence, producing detailed feedback on missing or inaccurate concepts. This feedback is fed into an iterative prompt refinement loop, enabling the large language model to propose improved prompts. Experiments on DrawBench and CompBench prompts demonstrate that our method significantly improves concept coverage and human-judged faithfulness over both standard test-time optimization and the base T2I model. Code is available at: https://github.com/AmirMansurian/NoConceptLeftBehind

[178] Robust Multi-Modal Face Anti-Spoofing with Domain Adaptation: Tackling Missing Modalities, Noisy Pseudo-Labels, and Model Degradation

Ming-Tsung Hsu,Fang-Yu Hsu,Yi-Ting Lin,Kai-Heng Chien,Jun-Ren Chen,Cheng-Hsiang Su,Yi-Chen Ou,Chiou-Ting Hsu,Pei-Kai Huang

Main category: cs.CV

TL;DR: 该论文提出了一种名为MFAS-DANet的新框架，解决了多模态人脸防伪（FAS）在领域适应场景下的三个主要问题：缺失模态、噪声伪标签和模型退化。

Details Motivation: 现有的多模态FAS方法在处理新领域攻击时表现不佳，且尚未探索领域适应在多模态FAS中的应用。

Contribution: 提出了MFAS-DANet框架，解决了缺失模态、噪声伪标签和模型退化问题。

Method: 提取互补特征代替缺失模态、利用预测不确定性生成可靠伪标签、设计自适应机制调整损失权重。

Result: 通过大量实验证明了MFAS-DANet的有效性和领先性能。

Insight: 该研究为多模态FAS领域适应提供了新的解决方案，突出了互补特征和预测不确定性的重要性。

Abstract: Recent multi-modal face anti-spoofing (FAS) methods have investigated the potential of leveraging multiple modalities to distinguish live and spoof faces. However, pre-adapted multi-modal FAS models often fail to detect unseen attacks from new target domains. Although a more realistic domain adaptation (DA) scenario has been proposed for single-modal FAS to learn specific spoof attacks during inference, DA remains unexplored in multi-modal FAS methods. In this paper, we propose a novel framework, MFAS-DANet, to address three major challenges in multi-modal FAS under the DA scenario: missing modalities, noisy pseudo labels, and model degradation. First, to tackle the issue of missing modalities, we propose extracting complementary features from other modalities to substitute missing modality features or enhance existing ones. Next, to reduce the impact of noisy pseudo labels during model adaptation, we propose deriving reliable pseudo labels by leveraging prediction uncertainty across different modalities. Finally, to prevent model degradation, we design an adaptive mechanism that decreases the loss weight during unstable adaptations and increasing it during stable ones. Extensive experiments demonstrate the effectiveness and state-of-the-art performance of our proposed MFAS-DANet.

[179] RestoRect: Degraded Image Restoration via Latent Rectified Flow & Feature Distillation

Shourya Verma,Mengbo Wang,Nadia Atallah Lanman,Ananth Grama

Main category: cs.CV

TL;DR: RestoRect提出了一种基于潜在修正流特征蒸馏的图像恢复方法，解决了高性能模型速度慢与快速模型效果差的权衡问题。

Details Motivation: 现有图像恢复方法在性能和速度之间存在明显权衡，知识蒸馏的传统静态特征匹配方法无法捕捉现代transformer架构的动态特征生成过程。

Contribution: 提出了一种新颖的潜在修正流特征蒸馏方法RestoRect，结合Retinex理论、可学习各向异性扩散和三角色空间极化，实现了高效的图像恢复。

Method: 将特征蒸馏重新建模为生成过程，学生模型通过学习潜在空间中的可学习轨迹合成教师级别的特征；并引入特征层提取损失和交叉归一化特征对齐。

Result: 在15个图像恢复数据集、4个任务和8个指标上取得了优越的结果，训练稳定性、收敛速度和推理速度均有显著提升。

Insight: RestoRect的动态特征学习方法突破了传统静态匹配的局限性，为跨架构知识迁移提供了新思路。

Abstract: Current approaches for restoration of degraded images face a critical trade-off: high-performance models are too slow for practical use, while fast models produce poor results. Knowledge distillation transfers teacher knowledge to students, but existing static feature matching methods cannot capture how modern transformer architectures dynamically generate features. We propose ‘RestoRect’, a novel Latent Rectified Flow Feature Distillation method for restoring degraded images. We apply rectified flow to reformulate feature distillation as a generative process where students learn to synthesize teacher-quality features through learnable trajectories in latent space. Our framework combines Retinex theory for physics-based decomposition with learnable anisotropic diffusion constraints, and trigonometric color space polarization. We introduce a Feature Layer Extraction loss for robust knowledge transfer between different network architectures through cross-normalized transformer feature alignment with percentile-based outlier detection. RestoRect achieves better training stability, and faster convergence and inference while preserving restoration quality. We demonstrate superior results across 15 image restoration datasets, covering 4 tasks, on 8 metrics.

[180] Orientation-anchored Hyper-Gaussian for 4D Reconstruction from Casual Videos

Junyi Wu,Jiachen Tao,Haoxuan Wang,Gaowen Liu,Ramana Rao Kompella,Yan Yan

Main category: cs.CV

TL;DR: OriGS提出了一种基于场景方向的超维度表示方法，通过全局方向场（Global Orientation Field）和定向感知超高斯（Orientation-aware Hyper-Gaussian）实现了高质量的四维重建。

Details Motivation: 现有方法通常依赖于低秩假设，难以建模非约束动态场景中复杂的、区域特定的变形。OriGS旨在通过引入方向信息的超维度表示来解决这一问题。

Contribution: 1. 提出了全局方向场，为动态建模提供稳定的结构指导；2. 提出了定向感知超高斯，统一嵌入时间、空间、几何和方向信息，实现了区域特定的变形推断。

Method: 1. 估计全局方向场，传播主方向信息；2. 设计定向感知超高斯，通过条件切片推断局部动态。

Result: 实验表明，OriGS在复杂动态场景中的重建质量优于主流方法。

Insight: 通过方向信息引导动态建模，可以更好地捕捉局部动态与全局运动意图的一致性。

Abstract: We present Orientation-anchored Gaussian Splatting (OriGS), a novel framework for high-quality 4D reconstruction from casually captured monocular videos. While recent advances extend 3D Gaussian Splatting to dynamic scenes via various motion anchors, such as graph nodes or spline control points, they often rely on low-rank assumptions and fall short in modeling complex, region-specific deformations inherent to unconstrained dynamics. OriGS addresses this by introducing a hyperdimensional representation grounded in scene orientation. We first estimate a Global Orientation Field that propagates principal forward directions across space and time, serving as stable structural guidance for dynamic modeling. Built upon this, we propose Orientation-aware Hyper-Gaussian, a unified formulation that embeds time, space, geometry, and orientation into a coherent probabilistic state. This enables inferring region-specific deformation through principled conditioned slicing, adaptively capturing diverse local dynamics in alignment with global motion intent. Experiments demonstrate the superior reconstruction fidelity of OriGS over mainstream methods in challenging real-world dynamic scenes.

[181] Multi-modal Data Spectrum: Multi-modal Datasets are Multi-dimensional

Divyam Madaan,Varshan Muhunthan,Kyunghyun Cho,Sumit Chopra

Main category: cs.CV

TL;DR: 该论文通过大规模实证研究，量化了23个视觉问答基准中视觉和文本模态及其交互作用的依赖程度，揭示了当前基准评估中存在的一些问题，为多模态基准设计和评估提供了定量分析方法。

Details Motivation: 理解单模态依赖（个体模态对目标任务的贡献）和多模态依赖（模态与目标任务之间的关系）的相互作用是推动多模态学习的关键，但目前对这些依赖关系的本质和在基准评估中的交互作用了解甚少。

Contribution: 论文的主要贡献是通过实证研究量化了多模态基准中的依赖关系，并揭示了现有基准在设计上的偏差问题，为多模态数据集的分析和评估提供了新的视角。

Method: 使用多模态大语言模型（MLLMs）对23个视觉问答基准进行了大规模分析，涵盖了通用知识推理、专业知识推理、光学字符识别和文档理解等领域。

Result: 研究发现，视觉、文本及其交互作用的依赖程度在不同基准和同一基准内差异显著，许多基准在试图减少文本偏向时无意中放大了图像依赖性。大型模型常通过单模态依赖掩盖多模态推理能力的不足。

Insight: 论文揭示了多模态基准设计中的潜在问题，强调了定量分析的重要性，并为未来的多模态学习和基准设计提供了指导。

Abstract: Understanding the interplay between intra-modality dependencies (the contribution of an individual modality to a target task) and inter-modality dependencies (the relationships between modalities and the target task) is fundamental to advancing multi-modal learning. However, the nature of and interaction between these dependencies within current benchmark evaluations remains poorly characterized. In this work, we present a large-scale empirical study to quantify these dependencies across 23 visual question-answering benchmarks using multi-modal large language models (MLLMs) covering domains such as general and expert knowledge reasoning, optical character recognition, and document understanding. Our findings show that the reliance on vision, question (text), and their interaction varies significantly, both across and within benchmarks. We discover that numerous benchmarks intended to mitigate text-only biases have inadvertently amplified image-only dependencies. This characterization persists across model sizes, as larger models often use these intra-modality dependencies to achieve high performance that mask an underlying lack of multi-modal reasoning. We provide a quantitative characterization of multi-modal datasets, enabling a principled approach to multi-modal benchmark design and evaluation.

[182] Enhancing Polyp Segmentation via Encoder Attention and Dynamic Kernel Update

Fatemeh Salahi Chashmi,Roya Sotoudeh

Main category: cs.CV

TL;DR: 该论文提出了一种结合动态核机制和全局编码器注意力模块的新框架，用于提高息肉分割的准确性和效率。通过动态核迭代优化分割预测，并结合全局注意力捕捉关键病灶特征，同时采用统一通道适应在解码器中标准化特征维度。实验表明，该方法在多个基准数据集上优于现有技术，并在计算效率和准确性上取得了平衡。

Details Motivation: 息肉分割在结直肠癌检测中至关重要，但由于息肉形状多样、大小不一且边界对比度低，分割任务具有挑战性。现有方法在处理复杂边界和计算效率上仍有改进空间。

Contribution: 1. 提出动态核机制，通过全局上下文初始化并迭代优化分割预测。2. 引入全局编码器注意力模块，增强对多尺度病灶特征的捕捉能力。3. 采用统一通道适应标准化解码器特征维度，简化结构并降低计算成本。

Method: 1. 动态核（DK）机制：利用全局上下文初始化，逐阶段迭代更新分割预测。2. 编码器注意力（EA）模块：聚合多尺度编码器信息，增强特征提取。3. 统一通道适应（UCA）：在解码器中标准化特征维度，提升计算效率。

Result: 在KvasirSEG和CVC ClinicDB数据集上，该方法在Dice和交并比（IoU）指标上优于现有技术，同时简化了解码器结构，降低了计算成本。

Insight: 结合注意力机制和动态优化的设计可以有效提升分割任务的性能，尤其是在处理复杂边界和多样化形状的目标时。未来可扩展应用于其他医学图像分割任务。

Abstract: Polyp segmentation is a critical step in colorectal cancer detection, yet it remains challenging due to the diverse shapes, sizes, and low contrast boundaries of polyps in medical imaging. In this work, we propose a novel framework that improves segmentation accuracy and efficiency by integrating a Dynamic Kernel (DK) mechanism with a global Encoder Attention module. The DK mechanism, initialized by a global context vector from the EA module, iteratively refines segmentation predictions across decoding stages, enabling the model to focus on and accurately delineate complex polyp boundaries. The EA module enhances the network’s ability to capture critical lesion features by aggregating multi scale information from all encoder layers. In addition, we employ Unified Channel Adaptation (UCA) in the decoder to standardize feature dimensions across stages, ensuring consistent and computationally efficient information fusion. Our approach extends the lesion-aware kernel framework by introducing a more flexible, attention driven kernel initialization and a unified decoder design. Extensive experiments on the KvasirSEG and CVC ClinicDB benchmark datasets demonstrate that our model outperforms several state of the art segmentation methods, achieving superior Dice and Intersection over Union scores. Moreover, UCA simplifies the decoder structure, reducing computational cost without compromising accuracy. Overall, the proposed method provides a robust and adaptable solution for polyp segmentation, with promising applications in clinical and automated diagnostic systems.

[183] Evaluating point-light biological motion in multimodal large language models

Akila Kadambi,Marco Iacoboni,Lisa Aziz-Zadeh,Srini Narayanan

Main category: cs.CV

TL;DR: 论文提出了第一个用于评估多模态大语言模型（MLLMs）处理稀疏点光源生物运动能力的基准ActPLD，发现当前模型的性能普遍较低，揭示了其在动作和时空理解方面的根本缺陷。

Details Motivation: 人类能从稀疏的点光源运动（PLDs）中提取丰富的语义信息，这是基于人类的具体经验。研究PLDs的处理能力有助于测试MLLMs在动作理解方面的局限性。

Contribution: 提出了首个评估MLLMs处理PLDs动作能力的基准ActPLD，测试了包括最先进的开源和专有系统的性能。

Method: 利用单人和社交互动的PLDs作为测试刺激，评估MLLMs的动作理解和时空推理能力。

Result: 结果显示各模型在PLDs任务上的性能普遍较低，突显了其在动作和时空理解方面的不足。

Insight: 研究表明，当前MLLMs在缺乏丰富视觉信息的情况下，难以有效处理动作语义，强调了其在多模态理解中的局限性。

Abstract: Humans can extract rich semantic information from minimal visual cues, as demonstrated by point-light displays (PLDs), which consist of sparse sets of dots localized to key joints of the human body. This ability emerges early in development and is largely attributed to human embodied experience. Since PLDs isolate body motion as the sole source of meaning, they represent key stimuli for testing the constraints of action understanding in these systems. Here we introduce ActPLD, the first benchmark to evaluate action processing in MLLMs from human PLDs. Tested models include state-of-the-art proprietary and open-source systems on single-actor and socially interacting PLDs. Our results reveal consistently low performance across models, introducing fundamental gaps in action and spatiotemporal understanding.

[184] Imaging-Based Mortality Prediction in Patients with Systemic Sclerosis

Alec K. Peltekian,Karolina Senkow,Gorkem Durak,Kevin M. Grudzinski,Bradford C. Bemiss,Jane E. Dematte,Carrie Richardson,Nikolay S. Markov,Mary Carns,Kathleen Aren,Alexandra Soriano,Matthew Dapas,Harris Perlman,Aaron Gundersheimer,Kavitha C. Selvan,John Varga,Monique Hinchcliff,Krishnan Warrior,Catherine A. Gao,Richard G. Wunderink,GR Scott Budinger,Alok N. Choudhary,Anthony J. Esposito,Alexander V. Misharin,Ankit Agrawal,Ulas Bagci

Main category: cs.CV

TL;DR: 该研究提出了一个基于影像的大规模纵向分析框架，结合放射组学和深度学习技术，预测系统性硬化（SSc）患者的死亡率。

Details Motivation: SSc相关的间质性肺病（ILD）是高死亡率的主要原因，但现有CT影像的诊断和预测能力尚未充分挖掘。

Contribution: 提出了一个新的框架，首次大规模结合放射组学和深度学习预测SSc患者的死亡率，填补了文献中的空白。

Method: 使用ResNet-18、DenseNet-121和Swin Transformer等预训练模型，并在2,125例SSc患者的CT扫描上进行微调，进行1年、3年和5年的死亡率预测。

Result: 模型在1年、3年和5年的死亡率预测中分别取得了0.769、0.801和0.709的AUC值。

Insight: 研究表明，放射组学和深度学习能够显著提升SSc患者的早期风险预测能力，为临床决策提供了新的工具。

Abstract: Interstitial lung disease (ILD) is a leading cause of morbidity and mortality in systemic sclerosis (SSc). Chest computed tomography (CT) is the primary imaging modality for diagnosing and monitoring lung complications in SSc patients. However, its role in disease progression and mortality prediction has not yet been fully clarified. This study introduces a novel, large-scale longitudinal chest CT analysis framework that utilizes radiomics and deep learning to predict mortality associated with lung complications of SSc. We collected and analyzed 2,125 CT scans from SSc patients enrolled in the Northwestern Scleroderma Registry, conducting mortality analyses at one, three, and five years using advanced imaging analysis techniques. Death labels were assigned based on recorded deaths over the one-, three-, and five-year intervals, confirmed by expert physicians. In our dataset, 181, 326, and 428 of the 2,125 CT scans were from patients who died within one, three, and five years, respectively. Using ResNet-18, DenseNet-121, and Swin Transformer we use pre-trained models, and fine-tuned on 2,125 images of SSc patients. Models achieved an AUC of 0.769, 0.801, 0.709 for predicting mortality within one-, three-, and five-years, respectively. Our findings highlight the potential of both radiomics and deep learning computational methods to improve early detection and risk assessment of SSc-related interstitial lung disease, marking a significant advancement in the literature.

[185] Calibrated and Resource-Aware Super-Resolution for Reliable Driver Behavior Analysis

Ibne Farabi Shihab,Weiheng Chai,Jiyang Wang,Sanjeda Akter,Senem Velipasalar Gursoy,Anuj Sharma

Main category: cs.CV

TL;DR: 该论文提出了一种资源感知的自适应超分辨率框架，旨在优化驾驶员行为分析的模型校准和关键事件的精准召回，取得了最佳校准性能和安全性指标。

Details Motivation: 驾驶员监控系统在安全关键场景中不仅需要高精度，还需要可靠的置信度评分，而直接低分辨率训练虽然整体精度高，但预测校准差，存在安全隐患。

Contribution: 提出了资源感知的自适应超分辨率框架，优化了模型校准和关键事件的精准召回；设计了一个轻量级伪影检测器，过滤超分辨率引起的幻觉。

Method: 采用自适应超分辨率技术，结合轻量级伪影检测器（0.3M参数，5.2ms开销），提升模型校准和安全性。

Result: 在安全关键指标上表现最佳：校准误差（ECE）5.8%，AUPR（嗜睡检测）0.78，精准召回（手机使用检测）0.74。

Insight: 直接低分辨率训练的模型虽通用性强，但校准性能差；该框架为安全关键应用提供了可靠解决方案。

Abstract: Driver monitoring systems require not just high accuracy but reliable, well-calibrated confidence scores for safety-critical deployment. While direct low-resolution training yields high overall accuracy, it produces poorly calibrated predictions that can be dangerous in safety-critical scenarios. We propose a resource-aware adaptive super-resolution framework that optimizes for model calibration and high precision-recall on critical events. Our approach achieves state-of-the-art performance on safety-centric metrics: best calibration (ECE of 5.8% vs 6.2% for LR-trained baselines), highest AUPR for drowsiness detection (0.78 vs 0.74), and superior precision-recall for phone use detection (0.74 vs 0.71). A lightweight artifact detector (0.3M parameters, 5.2ms overhead) provides additional safety by filtering SR-induced hallucinations. While LR-trained video models serve as strong general-purpose baselines, our adaptive framework represents the state-of-the-art solution for safety-critical applications where reliability is paramount.

[186] OVSeg3R: Learn Open-vocabulary Instance Segmentation from 2D via 3D Reconstruction

Hongyang Li,Jinyuan Qu,Lei Zhang

Main category: cs.CV

TL;DR: OVSeg3R通过3D重建从2D感知模型中学习开放词汇的3D实例分割，避免了手动标注的成本。该方法利用2D到3D的对应关系生成3D注释，并提出View-wise Instance Partition和2D Instance Boundary-aware Superpoint算法以优化训练过程，显著提升了ScanNet200基准的性能。

Details Motivation: 现有的3D实例分割方法通常依赖封闭词汇表或需要昂贵的标注成本，限制了其在实际应用中的广泛性和可扩展性。提出的OVSeg3R方法旨在通过利用2D模型的开放词汇能力和3D重建技术，降低标注成本并提升分割性能。

Contribution: 1. 提出一种新的训练方案OVSeg3R，结合2D开放词汇模型和3D重建技术学习开放词汇3D实例分割。2. 设计了View-wise Instance Partition算法，避免2D到3D投影中的假阳性监督问题。3. 引入2D Instance Boundary-aware Superpoint，利用2D掩码约束超级点聚类，提升分割效果。

Method: 1. 使用3D重建模型从2D视频重建场景作为输入。2. 通过2D到3D的对应关系将2D实例掩码预测投影到3D中以生成注释。3. 提出View-wise Instance Partition算法和2D Instance Boundary-aware Superpoint方法优化训练。

Result: 在ScanNet200基准测试中，OVSeg3R显著提升了性能（+2.3 mAP），并在开放词汇设置下，对未见类别的性能提升了约+7.1 mAP。

Insight: 通过结合2D开放词汇模型和3D重建技术，可以在降低标注成本的同时显著提升3D实例分割的性能。设计针对性的算法（如监督分区和边界感知聚类）可以帮助解决2D到3D迁移中的关键问题。

Abstract: In this paper, we propose a training scheme called OVSeg3R to learn open-vocabulary 3D instance segmentation from well-studied 2D perception models with the aid of 3D reconstruction. OVSeg3R directly adopts reconstructed scenes from 2D videos as input, avoiding costly manual adjustment while aligning input with real-world applications. By exploiting the 2D to 3D correspondences provided by 3D reconstruction models, OVSeg3R projects each view’s 2D instance mask predictions, obtained from an open-vocabulary 2D model, onto 3D to generate annotations for the view’s corresponding sub-scene. To avoid incorrectly introduced false positives as supervision due to partial annotations from 2D to 3D, we propose a View-wise Instance Partition algorithm, which partitions predictions to their respective views for supervision, stabilizing the training process. Furthermore, since 3D reconstruction models tend to over-smooth geometric details, clustering reconstructed points into representative super-points based solely on geometry, as commonly done in mainstream 3D segmentation methods, may overlook geometrically non-salient objects. We therefore introduce 2D Instance Boundary-aware Superpoint, which leverages 2D masks to constrain the superpoint clustering, preventing superpoints from violating instance boundaries. With these designs, OVSeg3R not only extends a state-of-the-art closed-vocabulary 3D instance segmentation model to open-vocabulary, but also substantially narrows the performance gap between tail and head classes, ultimately leading to an overall improvement of +2.3 mAP on the ScanNet200 benchmark. Furthermore, under the standard open-vocabulary setting, OVSeg3R surpasses previous methods by about +7.1 mAP on the novel classes, further validating its effectiveness.

[187] VividFace: High-Quality and Efficient One-Step Diffusion For Video Face Enhancement

Shulian Zhang,Yong Guo,Long Peng,Ziyang Wang,Ye Chen,Wenbo Li,Xiao Zhang,Yulun Zhang,Jian Chen

Main category: cs.CV

TL;DR: VividFace是一个高效的一步扩散框架，用于视频人脸增强，解决了现有方法在纹理建模、训练数据不足和推理效率低的问题。

Details Motivation: 视频人脸增强在许多应用中至关重要，但现有方法面临纹理建模不准确、训练数据不足和推理效率低的问题，需要改进。

Contribution: 1. 提出VividFace，基于预训练WANX模型的一步扩散框架；2. 联合潜空间-像素的人脸聚焦训练策略；3. MLLM驱动的数据筛选流程。

Method: 结合单步流匹配范式，从降级输入直接映射到高质量输出；采用两阶段训练策略，联合优化潜空间和像素空间。

Result: 在感知质量、身份保持和时间稳定性方面达到SOTA，同时显著提升推理效率。

Insight: 联合潜空间-像素的训练策略和数据筛选流程能显著提升模型性能和泛化能力。

Abstract: Video Face Enhancement (VFE) seeks to reconstruct high-quality facial regions from degraded video sequences, a capability that underpins numerous applications including video conferencing, film restoration, and surveillance. Despite substantial progress in the field, current methods that primarily rely on video super-resolution and generative frameworks continue to face three fundamental challenges: (1) faithfully modeling intricate facial textures while preserving temporal consistency; (2) restricted model generalization due to the lack of high-quality face video training data; and (3) low efficiency caused by repeated denoising steps during inference. To address these challenges, we propose VividFace, a novel and efficient one-step diffusion framework for video face enhancement. Built upon the pretrained WANX video generation model, our method leverages powerful spatiotemporal priors through a single-step flow matching paradigm, enabling direct mapping from degraded inputs to high-quality outputs with significantly reduced inference time. To further boost efficiency, we propose a Joint Latent-Pixel Face-Focused Training strategy that employs stochastic switching between facial region optimization and global reconstruction, providing explicit supervision in both latent and pixel spaces through a progressive two-stage training process. Additionally, we introduce an MLLM-driven data curation pipeline for automated selection of high-quality video face datasets, enhancing model generalization. Extensive experiments demonstrate that VividFace achieves state-of-the-art results in perceptual quality, identity preservation, and temporal stability, while offering practical resources for the research community.

[188] VAMamba: An Efficient Visual Adaptive Mamba for Image Restoration

Han Hu,Zhuoran Zheng,Liang Li,Chen Lyu

Main category: cs.CV

TL;DR: VAMamba提出了一种高效的视觉自适应Mamba框架，通过QCLAM和GPS-SS2D两项创新，解决了现有Mamba方法中固定扫描模式和特征利用效率低的问题。

Details Motivation: 现有Mamba方法在图像修复任务中表现受限，主要原因是固定扫描模式和低效的特征利用，无法适应多样化的退化情况。

Contribution: 1. 提出QCLAM，通过FIFO缓存历史特征并智能融合，动态重用特征；2. 提出GPS-SS2D，基于视觉Transformer生成像素重要性得分图，动态优化扫描路径。

Method: 1. QCLAM利用低秩自适应内存动态融合历史特征；2. GPS-SS2D通过贪心策略生成自适应扫描路径。

Result: 在多种图像修复任务中，VAMamba在修复质量和计算效率上均优于现有方法。

Insight: 动态特征重用和自适应扫描路径能显著提升图像修复任务的性能，同时保持较高的计算效率。

Abstract: Recent Mamba-based image restoration methods have achieved promising results but remain limited by fixed scanning patterns and inefficient feature utilization. Conventional Mamba architectures rely on predetermined paths that cannot adapt to diverse degradations, constraining both restoration performance and computational efficiency. To overcome these limitations, we propose VAMamba, a Visual Adaptive Mamba framework with two key innovations. First, QCLAM(Queue-basedCacheLow-rankAdaptiveMemory)enhancesfeaturelearningthrougha FIFO cache that stores historical representations. Similarity between current LoRA-adapted and cached features guides intelligent fusion, enabling dynamic reuse while effectively controlling memorygrowth.Second, GPS-SS2D(GreedyPathScanSS2D)introducesadaptive scanning. A Vision Transformer generates score maps to estimate pixel importance, and a greedy strategy de termines optimal forward and backward scanning paths. These learned trajectories replace rigid patterns, enabling SS2D to perform targeted feature extraction. The integration of QCLAM and GPS-SS2D allows VAMamba to adaptively focus on degraded regions while maintaining high computational efficiency. Extensive experiments across diverse restoration tasks demonstrate that VAMamba consistently outperforms existing approaches in both restoration quality and efficiency, establishing new benchmarks for adaptive image restoration. Our code is available at https://github.com/WaterHQH/VAMamba.

[189] Deep Taxonomic Networks for Unsupervised Hierarchical Prototype Discovery

Zekun Wang,Ethan Haarer,Zhiyi Dai,Tianyi Zhu,Christopher J. MacLellan

Main category: cs.CV

TL;DR: 该论文提出了深度分类网络（deep taxonomic networks），一种基于变分推断的深度潜变量方法，用于从未标注数据中自动发现层次化的原型簇，填补了现有深度层次聚类方法的不足。

Details Motivation: 受到人类将知识组织为层次化分类的启发，论文旨在解决现有深度层次聚类方法的局限性，如结构固定为类别数量且未充分利用中间层次的原型信息。

Contribution: 1. 提出深度分类网络，通过变分推断优化一个完整的二元树结构的混合高斯先验；2. 无需标注数据即可自动发现层次化关系和原型簇；3. 提出新的评估机制，利用所有层次的原型簇。

Method: 1. 采用变分推断框架；2. 优化一个二元树结构的混合高斯先验；3. 通过最大化ELBO鼓励发现原型间的层次关系。

Result: 在多个图像分类数据集上表现出色，优于基线方法；定性结果显示方法能够发现丰富且可解释的层次化分类。

Insight: 该方法不仅能够发现粗粒度的语义类别，还能捕捉细粒度的视觉差异，展示了层次化原型发现的潜力。

Abstract: Inspired by the human ability to learn and organize knowledge into hierarchical taxonomies with prototypes, this paper addresses key limitations in current deep hierarchical clustering methods. Existing methods often tie the structure to the number of classes and underutilize the rich prototype information available at intermediate hierarchical levels. We introduce deep taxonomic networks, a novel deep latent variable approach designed to bridge these gaps. Our method optimizes a large latent taxonomic hierarchy, specifically a complete binary tree structured mixture-of-Gaussian prior within a variational inference framework, to automatically discover taxonomic structures and associated prototype clusters directly from unlabeled data without assuming true label sizes. We analytically show that optimizing the ELBO of our method encourages the discovery of hierarchical relationships among prototypes. Empirically, our learned models demonstrate strong hierarchical clustering performance, outperforming baselines across diverse image classification datasets using our novel evaluation mechanism that leverages prototype clusters discovered at all hierarchical levels. Qualitative results further reveal that deep taxonomic networks discover rich and interpretable hierarchical taxonomies, capturing both coarse-grained semantic categories and fine-grained visual distinctions.

[190] MAN: Latent Diffusion Enhanced Multistage Anti-Noise Network for Efficient and High-Quality Low-Dose CT Image Denoising

Tangtangfang Fang,Jingxi Hu,Xiangjian He,Jiaqi Yang

Main category: cs.CV

TL;DR: MAN是一种基于潜在扩散的多阶段抗噪网络，用于高效高质量的低剂量CT图像去噪，通过压缩潜在空间和注意力机制显著降低计算成本，同时保持高质量的去噪效果。

Details Motivation: 尽管扩散模型在低剂量CT去噪中表现出色，但其极高的计算成本阻碍了临床应用。本文旨在解决这一问题，提出高效且高质量的解决方案。

Contribution: 提出MAN模型，结合潜在扩散和多阶段抗噪网络，在压缩潜在空间中实现快速确定性去噪，显著减少计算开销，同时保持高质量。

Method: 采用感知优化的自动编码器压缩图像到潜在空间，利用基于注意力的条件U-Net进行快速扩散去噪，减少计算复杂度。

Result: 在低剂量CT数据集上，MAN在PSNR/SSIM得分上与DDPM和Dn-Dp等计算密集型模型相当，但推理速度快60倍以上。

Insight: 通过压缩潜在空间和注意力机制，MAN在保持高质量的同时大幅提升效率，为生成模型在医学影像中的实际应用提供了可行路径。

Abstract: While diffusion models have set a new benchmark for quality in Low-Dose Computed Tomography (LDCT) denoising, their clinical adoption is critically hindered by extreme computational costs, with inference times often exceeding thousands of seconds per scan. To overcome this barrier, we introduce MAN, a Latent Diffusion Enhanced Multistage Anti-Noise Network for Efficient and High-Quality Low-Dose CT Image Denoising task. Our method operates in a compressed latent space via a perceptually-optimized autoencoder, enabling an attention-based conditional U-Net to perform the fast, deterministic conditional denoising diffusion process with drastically reduced overhead. On the LDCT and Projection dataset, our model achieves superior perceptual quality, surpassing CNN/GAN-based methods while rivaling the reconstruction fidelity of computationally heavy diffusion models like DDPM and Dn-Dp. Most critically, in the inference stage, our model is over 60x faster than representative pixel space diffusion denoisers, while remaining competitive on PSNR/SSIM scores. By bridging the gap between high fidelity and clinical viability, our work demonstrates a practical path forward for advanced generative models in medical imaging.

[191] RIV: Recursive Introspection Mask Diffusion Vision Language Model

YuQian Li,Limeng Qiao,Lin Ma

Main category: cs.CV

TL;DR: 论文提出了RIV模型，通过自省训练和递归推断机制，赋予掩码扩散视觉语言模型自校正能力，显著提升了多模态理解任务的性能。

Details Motivation: 现有掩码扩散视觉语言模型（MDVLM）缺乏对生成错误的校正能力，影响了模型的可靠性和准确性。

Contribution: 1. 引入自省训练机制，通过自省模型检测生成序列中的错误（包括语法、拼写和逻辑错误）；2. 提出递归推断机制，通过交替的解掩码、自省和重掩码步骤优化输出结果。

Method: 1. 自省训练：训练自省模型来识别生成序列中的错误；2. 递归推断：在推断过程中，交替执行解掩码、自省和重掩码步骤，直到生成可靠结果。

Result: 在多个基准测试中，RIV模型表现优异，优于现有大部分MDVLM模型。

Insight: 通过引入自省和递归机制，可以显著提升生成模型的自我校正能力，从而在多模态任务中实现更高的准确性和可靠性。

Abstract: Mask Diffusion-based Vision Language Models (MDVLMs) have achieved remarkable progress in multimodal understanding tasks. However, these models are unable to correct errors in generated tokens, meaning they lack self-correction capability. In this paper, we propose Recursive Introspection Mask Diffusion Vision Language Model (RIV), which equips the model with self-correction ability through two novel mechanisms. The first is Introspection Training, where an Introspection Model is introduced to identify errors within generated sequences. Introspection Training enables the model to detect not only grammatical and spelling mistakes, but more importantly, logical errors. The second is Recursive Inference. Beginning with the standard unmasking step, the learned Introspection Model helps to identify errors in the output sequence and remask them. This alternating ($\text{unmask}\rightarrow\text{introspection}\rightarrow\text{remask}$) process is repeated recursively until reliable results are obtained. Experimental results on multiple benchmarks demonstrate that the proposed RIV achieves state-of-the-art performance, outperforming most existing MDVLMs.

[192] Efficient Domain-Adaptive Multi-Task Dense Prediction with Vision Foundation Models

Beomseok Kang,Niluthpol Chowdhury Mithun,Mikhail Sizintsev,Han-Pang Chiu,Supun Samarasekera

Main category: cs.CV

TL;DR: FAMDA提出了一种利用视觉基础模型（VFMs）生成高质量伪标签的自训练框架，用于多任务密集预测的领域自适应，在合成到真实和多场景任务中取得了SOTA效果。

Details Motivation: 多任务密集预测在机器人应用中很重要，但领域迁移问题限制了其在新环境中的表现。现有方法主要依赖对抗学习，效果不如自训练技术。

Contribution: 提出了FAMDA框架，利用VFMs作为教师模型生成高质量伪标签，通过自训练将通用能力迁移到高效的单一学生网络中。

Method: 结合分割和深度基础模型，通过自训练范式生成目标域伪标签，蒸馏到轻量级学生网络中。

Result: 在合成到真实和多场景任务中达到SOTA性能，轻量版本比基础模型小10倍以上。

Insight: 视觉基础模型的能力可以通过自训练高效迁移到轻量网络中，适合资源受限的应用。

Abstract: Multi-task dense prediction, which aims to jointly solve tasks like semantic segmentation and depth estimation, is crucial for robotics applications but suffers from domain shift when deploying models in new environments. While unsupervised domain adaptation (UDA) addresses this challenge for single tasks, existing multi-task UDA methods primarily rely on adversarial learning approaches that are less effective than recent self-training techniques. In this paper, we introduce FAMDA, a simple yet effective UDA framework that bridges this gap by leveraging Vision Foundation Models (VFMs) as powerful teachers. Our approach integrates Segmentation and Depth foundation models into a self-training paradigm to generate high-quality pseudo-labels for the target domain, effectively distilling their robust generalization capabilities into a single, efficient student network. Extensive experiments show that FAMDA achieves state-of-the-art (SOTA) performance on standard synthetic-to-real UDA multi-task learning (MTL) benchmarks and a challenging new day-to-night adaptation task. Our framework enables the training of highly efficient models; a lightweight variant achieves SOTA accuracy while being more than 10$\times$ smaller than foundation models, highlighting FAMDA’s suitability for creating domain-adaptive and efficient models for resource-constrained robotics applications.

[193] MotionVerse: A Unified Multimodal Framework for Motion Comprehension, Generation and Editing

Ruibing Hou,Mingshuang Luo,Hongyu Pan,Hong Chang,Shiguang Shan

Main category: cs.CV

TL;DR: MotionVerse是一个统一的多模态框架，利用大型语言模型（LLMs）理解和编辑单人与多人场景中的运动数据。通过运动分词器和延迟并行建模策略，实现了高效的运动表示和依赖捕捉，同时采用双塔架构避免模态干扰。

Details Motivation: 传统运动理解、生成和编辑方法通常依赖特定任务的设计，缺乏统一性和灵活性。MotionVerse旨在通过LLMs的多模态能力，实现一个通用的运动数据处理框架。

Contribution: 1. 提出了MotionVerse统一框架；2. 设计了运动分词器和延迟并行建模策略；3. 引入双塔架构以缓解模态干扰。

Method: 1. 使用运动分词器将连续运动序列转为多流离散token；2. 延迟并行建模策略分时编码token流；3. 双塔架构分离运动与语言模态的参数。

Result: 实验表明，MotionVerse在各运动相关任务中表现优异，消融研究验证了各模块的有效性。

Insight: 通过结合LLMs和运动数据的离散表示，MotionVerse展示了多模态框架在运动任务中的潜力，同时为高效建模提供了新思路。

Abstract: This paper proposes MotionVerse, a unified framework that harnesses the capabilities of Large Language Models (LLMs) to comprehend, generate, and edit human motion in both single-person and multi-person scenarios. To efficiently represent motion data, we employ a motion tokenizer with residual quantization, which converts continuous motion sequences into multi-stream discrete tokens. Furthermore, we introduce a \textit{Delay Parallel} Modeling strategy, which temporally staggers the encoding of residual token streams. This design enables LLMs to effectively capture inter-stream dependencies while maintaining computational efficiency comparable to single-stream modeling. Moreover, to alleviate modality interference between motion and language, we design a \textit{dual-tower architecture} with modality-specific parameters, ensuring stable integration of motion information for both comprehension and generation tasks. Comprehensive ablation studies demonstrate the effectiveness of each component in MotionVerse, and extensive experiments showcase its superior performance across a wide range of motion-relevant tasks.

[194] LightFair: Towards an Efficient Alternative for Fair T2I Diffusion via Debiasing Pre-trained Text Encoders

Boyu Han,Qianqian Xu,Shilong Bao,Zhiyong Yang,Kangli Zi,Qingming Huang

Main category: cs.CV

TL;DR: LightFair提出了一种轻量级方法来提高文本到图像扩散模型（T2I DMs）的公平性，通过专注于微调文本编码器的嵌入信息，减少了对全参数训练或辅助网络的依赖。

Details Motivation: 现有的T2I DMs公平性方法通常需要高成本的训练或采样负担，且性能不佳。LightFair旨在通过优化文本编码器的嵌入来高效解决这一问题。

Contribution: 1）揭示了文本编码器输出的中性嵌入在CLIP空间中存在显著偏差；2）提出了协作距离约束的去偏策略；3）设计了双阶段文本引导采样策略以平衡去偏与生成质量。

Method: LightFair通过微调文本编码器的嵌入信息，结合协作距离约束的去偏策略和双阶段采样策略，实现了高效的公平性优化。

Result: 在Stable Diffusion v1.5上，LightFair仅需四分之一的训练负担，取得了SOTA的去偏效果，且采样负担几乎不增加。

Insight: 文本编码器是T2I DMs中易于微调的核心模块，优化其嵌入信息可以高效提升模型的公平性，同时减少额外的计算负担。

Abstract: This paper explores a novel lightweight approach LightFair to achieve fair text-to-image diffusion models (T2I DMs) by addressing the adverse effects of the text encoder. Most existing methods either couple different parts of the diffusion model for full-parameter training or rely on auxiliary networks for correction. They incur heavy training or sampling burden and unsatisfactory performance. Since T2I DMs consist of multiple components, with the text encoder being the most fine-tunable and front-end module, this paper focuses on mitigating bias by fine-tuning text embeddings. To validate feasibility, we observe that the text encoder’s neutral embedding output shows substantial skewness across image embeddings of various attributes in the CLIP space. More importantly, the noise prediction network further amplifies this imbalance. To finetune the text embedding, we propose a collaborative distance-constrained debiasing strategy that balances embedding distances to improve fairness without auxiliary references. However, mitigating bias can compromise the original generation quality. To address this, we introduce a two-stage text-guided sampling strategy to limit when the debiased text encoder intervenes. Extensive experiments demonstrate that LightFair is effective and efficient. Notably, on Stable Diffusion v1.5, our method achieves SOTA debiasing at just $1/4$ of the training burden, with virtually no increase in sampling burden. The code is available at https://github.com/boyuh/LightFair.

[195] EfficientMIL: Efficient Linear-Complexity MIL Method for WSI Classification

Chengying She,Ben Wang,Xinran Zhang,Dongjie Fan,Jialu Zhang,Chengwei Chen,Lizhuang Liu

Main category: cs.CV

TL;DR: EfficientMIL是一种线性复杂度的多示例学习（MIL）方法，用于全切片图像（WSI）分类，通过设计自适应补丁选择器（APS）和采用高效序列模型（如GRU、LSTM和Mamba），解决了基于注意力机制的MIL方法的计算瓶颈。

Details Motivation: 当前基于注意力机制的MIL方法在WSI分类中表现优异，但计算复杂度高，限制了其实际应用。因此，需要一种更高效的方法来提升计算效率。

Contribution: 提出了EfficientMIL，一种线性复杂度的MIL方法，通过APS补丁选择器和高效序列模型取代了传统的自注意力机制，显著降低了计算成本。

Method: 设计了APS补丁选择器，并采用RNN-based GRU、LSTM和SSM Mamba等高效序列模型，避免了传统MIL方法的二次复杂度问题。

Result: 在TCGA-Lung和CAMELYON16数据集上表现优于现有方法，例如EfficientMIL-Mamba在TCGA-Lung上的AUC为0.976，EfficientMIL-GRU在CAMELYON16上的AUC为0.990。

Insight: 线性复杂度的序列模型可以替代高复杂度的注意力机制，同时保持良好的性能，为大规模WSI分类提供了新的解决方案。

Abstract: Whole slide images (WSIs) classification represents a fundamental challenge in computational pathology, where multiple instance learning (MIL) has emerged as the dominant paradigm. Current state-of-the-art (SOTA) MIL methods rely on attention mechanisms, achieving good performance but requiring substantial computational resources due to quadratic complexity when processing hundreds of thousands of patches. To address this computational bottleneck, we introduce EfficientMIL, a novel linear-complexity MIL approach for WSIs classification with the patches selection module Adaptive Patch Selector (APS) that we designed, replacing the quadratic-complexity self-attention mechanisms in Transformer-based MIL methods with efficient sequence models including RNN-based GRU, LSTM, and State Space Model (SSM) Mamba. EfficientMIL achieves significant computational efficiency improvements while outperforming other MIL methods across multiple histopathology datasets. On TCGA-Lung dataset, EfficientMIL-Mamba achieved AUC of 0.976 and accuracy of 0.933, while on CAMELYON16 dataset, EfficientMIL-GRU achieved AUC of 0.990 and accuracy of 0.975, surpassing previous state-of-the-art methods. Extensive experiments demonstrate that APS is also more effective for patches selection than conventional selection strategies.

[196] From Static to Dynamic: a Survey of Topology-Aware Perception in Autonomous Driving

Yixiao Chen,Ruining Yang,Xin Chen,Jia He,Dongliang Xu,Yue Yao

Main category: cs.CV

TL;DR: 这篇综述系统地回顾了自动驾驶中拓扑感知感知的四个核心研究方向，强调了从静态地图到动态感知的范式转变，并探讨了其对自动驾驶系统的适应性、可扩展性和可解释性的影响。

Details Motivation: 传统的静态地图虽然为自动驾驶系统提供了语义上下文，但其构建成本高、难以实时更新且缺乏跨区域的泛化能力。这促使研究者转向利用车载传感器数据进行实时地图构建和拓扑推理的动态表示。

Contribution: 1. 系统综述了自动驾驶中拓扑感知感知的四个核心研究方向；2. 揭示了从静态地图到动态感知的范式转变趋势；3. 强调了紧凑空间建模、语义关系推理、领域知识集成和多模态场景理解的贡献。

Method: 研究了四个方向的方法：向量化地图构建、拓扑结构建模、先验知识融合和基于语言模型的感知。

Result: 动态表示方法能够实现实时地图构建和拓扑推理，提升了自动驾驶系统的适应性、可扩展性和可解释性。

Insight: 动态感知方法通过结合多模态数据和语言模型，能够更好地理解复杂的驾驶环境，为未来的自动驾驶系统提供了新的发展方向。

Abstract: The key to achieving autonomous driving lies in topology-aware perception, the structured understanding of the driving environment with an emphasis on lane topology and road semantics. This survey systematically reviews four core research directions under this theme: vectorized map construction, topological structure modeling, prior knowledge fusion, and language model-based perception. Across these directions, we observe a unifying trend: a paradigm shift from static, pre-built maps to dynamic, sensor-driven perception. Specifically, traditional static maps have provided semantic context for autonomous systems. However, they are costly to construct, difficult to update in real time, and lack generalization across regions, limiting their scalability. In contrast, dynamic representations leverage on-board sensor data for real-time map construction and topology reasoning. Each of the four research directions contributes to this shift through compact spatial modeling, semantic relational reasoning, robust domain knowledge integration, and multimodal scene understanding powered by pre-trained language models. Together, they pave the way for more adaptive, scalable, and explainable autonomous driving systems.

[197] Griffin: Generative Reference and Layout Guided Image Composition

Aryan Mikaeili,Amirhossein Alimohammadi,Negar Hassanpour,Ali Mahdavi-Amiri,Andrea Tagliasacchi

Main category: cs.CV

TL;DR: 这篇论文提出了一种无需训练的方法，名为Griffin，通过参考图像和布局引导实现更精确的图像合成控制。

Details Motivation: 现有文本到图像模型在生成高度真实图像方面表现出色，但文本控制的局限性限制了需要更精确布局的场景。论文旨在解决多图像布局控制的挑战，通过参考图像而非文本来指定内容，并提供明确的布局指导。

Contribution: 提出了Griffin方法，使用生成参考和布局引导的图像合成技术，实现无需训练、单参考图像的精确控制和灵活的对象及部分级合成。

Method: 采用生成参考和布局引导的方法，允许用户通过图像指定内容并提供明确的布局指导，从而实现对图像合成的精确控制。

Result: 在多种图像合成任务中证明了该方法的有效性，展示了其在对象和部分级合成上的灵活性和控制能力。

Insight: 通过参考图像和布局引导的结合，可以在无需额外训练的情况下实现更精确的图像合成控制，为生成式模型的实际应用提供了新思路。

Abstract: Text-to-image models have achieved a level of realism that enables the generation of highly convincing images. However, text-based control can be a limiting factor when more explicit guidance is needed. Defining both the content and its precise placement within an image is crucial for achieving finer control. In this work, we address the challenge of multi-image layout control, where the desired content is specified through images rather than text, and the model is guided on where to place each element. Our approach is training-free, requires a single image per reference, and provides explicit and simple control for object and part-level composition. We demonstrate its effectiveness across various image composition tasks.

[198] Sparse-Up: Learnable Sparse Upsampling for 3D Generation with High-Fidelity Textures

Lu Xiao,Jiale Zhang,Yang Liu,Taicheng Huang,Xin Tian

Main category: cs.CV

TL;DR: Sparse-Up提出了一种高效的3D纹理建模框架，通过稀疏体素引导纹理重建并保持多视角一致性，同时利用表面锚定和视角分区突破分辨率限制。

Details Motivation: 现有3D生成方法在保留高频细节和多视角一致性之间存在权衡，导致纹理撕裂或分辨率受限。

Contribution: 提出了Sparse-Up框架，通过稀疏体素和优化策略实现高效的高保真纹理建模，显著减少内存占用。

Method: 1. 表面锚定：使用可学习的上采样策略约束体素到网格表面；2. 视角分区：通过图像块引导体素分区，仅对可见局部块监督梯度。

Result: 减少了70%以上的冗余体素，突破了分辨率限制，同时保持了高频细节和多视角一致性。

Insight: 稀疏性和局部化是高效高保真3D生成的关键，可学习的上采样策略显著优化了内存和细节的平衡。

Abstract: The creation of high-fidelity 3D assets is often hindered by a ‘pixel-level pain point’: the loss of high-frequency details. Existing methods often trade off one aspect for another: either sacrificing cross-view consistency, resulting in torn or drifting textures, or remaining trapped by the resolution ceiling of explicit voxels, forfeiting fine texture detail. In this work, we propose Sparse-Up, a memory-efficient, high-fidelity texture modeling framework that effectively preserves high-frequency details. We use sparse voxels to guide texture reconstruction and ensure multi-view consistency, while leveraging surface anchoring and view-domain partitioning to break through resolution constraints. Surface anchoring employs a learnable upsampling strategy to constrain voxels to the mesh surface, eliminating over 70% of redundant voxels present in traditional voxel upsampling. View-domain partitioning introduces an image patch-guided voxel partitioning scheme, supervising and back-propagating gradients only on visible local patches. Through these two strategies, we can significantly reduce memory consumption during high-resolution voxel training without sacrificing geometric consistency, while preserving high-frequency details in textures.

[199] ReWatch-R1: Boosting Complex Video Reasoning in Large Vision-Language Models through Agentic Data Synthesis

Congzhi Zhang,Zhibin Wang,Yinchao Ma,Jiawei Peng,Yihan Wang,Qiang Zhou,Jun Song,Bo Zheng

Main category: cs.CV

TL;DR: 论文提出了ReWatch数据集和ReWatch-R1方法，通过多阶段合成流程和Multi-Agent ReAct框架解决了复杂视频推理中的数据瓶颈问题，并结合SFT和RLVR显著提升了LVLM的视频推理性能。

Details Motivation: 当前的大型视觉语言模型（LVLM）在复杂视频推理中的应用因缺乏高质量、多跳问题和视频基础的CoT数据而受限，论文旨在解决这一数据瓶颈问题。

Contribution: 1) 提出了ReWatch数据集，包含ReWatch-Caption、ReWatch-QA和ReWatch-CoT三部分；2) 开发了Multi-Agent ReAct框架用于生成视频基础的CoT数据；3) 设计了Observation & Reasoning奖励机制，结合SFT和RLVR训练了ReWatch-R1模型。

Method: 1) 多阶段数据合成流程构建数据集；2) Multi-Agent ReAct框架模拟人类‘重看’过程生成CoT数据；3) 结合SFT和RLVR训练模型，引入O&R奖励机制评估推理对齐性。

Result: ReWatch-R1在5个视频推理基准测试中达到了最先进的平均性能。

Insight: 通过模拟人类‘重看’过程和显式建模信息检索与验证，可以有效提升复杂视频推理的数据质量和模型性能。

Abstract: While Reinforcement Learning with Verifiable Reward (RLVR) significantly advances image reasoning in Large Vision-Language Models (LVLMs), its application to complex video reasoning remains underdeveloped. This gap stems primarily from a critical data bottleneck: existing datasets lack the challenging, multi-hop questions and high-quality, video-grounded Chain-of-Thought (CoT) data necessary to effectively bootstrap RLVR. To address this, we introduce ReWatch, a large-scale dataset built to foster advanced video reasoning. We propose a novel multi-stage synthesis pipeline to synthesize its three components: ReWatch-Caption, ReWatch-QA, and ReWatch-CoT. A core innovation is our Multi-Agent ReAct framework for CoT synthesis, which simulates a human-like “re-watching” process to generate video-grounded reasoning traces by explicitly modeling information retrieval and verification. Building on this dataset, we develop ReWatch-R1 by post-training a strong baseline LVLM with Supervised Fine-Tuning (SFT) and our RLVR framework. This framework incorporates a novel Observation & Reasoning (O&R) reward mechanism that evaluates both the final answer’s correctness and the reasoning’s alignment with video content, directly penalizing hallucination. Our experiments show that ReWatch-R1 achieves state-of-the-art average performance on five challenging video reasoning benchmarks.

[200] LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Training

Xiang An,Yin Xie,Kaicheng Yang,Wenkang Zhang,Xiuwei Zhao,Zheng Cheng,Yirui Wang,Songcen Xu,Changrui Chen,Chunsheng Wu,Huajie Tan,Chunyuan Li,Jing Yang,Jie Yu,Xiyao Wang,Bin Qin,Yumeng Wang,Zizhen Yan,Ziyong Feng,Ziwei Liu,Bo Li,Jiankang Deng

Main category: cs.CV

TL;DR: LLaVA-OneVision-1.5是一个开放、高效的多模态训练框架，通过大规模数据集和低成本训练方法实现了领先性能。

Details Motivation: 现有的大规模多模态模型计算和成本高昂，LLaVA-OneVision-1.5旨在提供一个开放且低成本的解决方案，推动多模态研究的民主化。

Contribution: 1) 构建了85M的预训练数据集和26M的指令数据集；2) 提出了高效的端到端训练框架，成本仅16,000美元；3) 在多个基准任务上实现了SOTA性能。

Method: 1) 使用大规模、平衡的数据集；2) 采用离线并行数据打包策略优化训练效率；3) 设计了低成本训练流程。

Result: LLaVA-OneVision-1.5-8B在27个基准任务中的18个上优于Qwen2.5-VL-7B，4B版本在所有任务上均超越Qwen2.5-VL-3B。

Insight: 通过数据优化和高效训练框架，低成本也能实现高性能的多模态模型，为社区提供了可复现的开放方案。

Abstract: We present LLaVA-OneVision-1.5, a novel family of Large Multimodal Models (LMMs) that achieve state-of-the-art performance with significantly reduced computational and financial costs. Different from the existing works, LLaVA-OneVision-1.5 provides an open, efficient, and reproducible framework for building high-quality vision-language models entirely from scratch. The LLaVA-OneVision-1.5 release comprises three primary components: (1) Large-Scale Curated Datasets: We construct an 85M concept-balanced pretraining dataset LLaVA-OneVision-1.5-Mid-Traning and a meticulously curated 26M instruction dataset LLaVA-OneVision-1.5-Instruct, collectively encompassing 64B compressed multimodal tokens. (2) Efficient Training Framework: We develop a complete end-to-end efficient training framework leveraging an offline parallel data packing strategy to facilitate the training of LLaVA-OneVision-1.5 within a $16,000 budget. (3) State-of-the-art Performance: Experimental results demonstrate that LLaVA-OneVision1.5 yields exceptionally competitive performance across a broad range of downstream tasks. Specifically, LLaVA-OneVision-1.5-8B outperforms Qwen2.5-VL-7B on 18 of 27 benchmarks, and LLaVA-OneVision-1.5-4B surpasses Qwen2.5-VL-3B on all 27 benchmarks. We anticipate releasing LLaVA-OneVision-1.5-RL shortly and encourage the community to await further updates.

[201] HIVTP: A Training-Free Method to Improve VLMs Efficiency via Hierarchical Visual Token Pruning Using Middle-Layer-Based Importance Score

Jingqi Xu,Jingxi Lu,Chenghao Li,Sreetama Sarkar,Peter A. Beerel

Main category: cs.CV

TL;DR: HIVTP是一种无需训练的方法，通过基于中间层的重要性分数进行分层视觉令牌剪枝，显著提高了视觉语言模型（VLMs）的推理效率。

Details Motivation: 视觉语言模型的视觉编码器输出的令牌数量庞大，严重影响了推理效率，但许多令牌并不重要，可以被安全剪枝。

Contribution: 提出了HIVTP方法，利用中间层的注意力图估计令牌重要性，并设计分层剪枝策略保留全局和局部重要令牌。

Method: 通过中间层注意力图计算令牌重要性，分两阶段剪枝：全局阶段保留区域高分令牌，局部阶段保留窗口最重要令牌。

Result: HIVTP将LLaVA-v1.5-7B和LLaVA-Next-7B的TTFT分别降低50.0%和55.1%，生成吞吐量提升60.9%和47.3%，且精度未受损。

Insight: 中间层注意力图能更精准地反映令牌重要性，分层剪枝策略平衡了全局和局部信息的保留。

Abstract: Vision-Language Models (VLMs) have shown strong capabilities on diverse multimodal tasks. However, the large number of visual tokens output by the vision encoder severely hinders inference efficiency, and prior studies have shown that many of these tokens are not important and can therefore be safely pruned. In this work, we propose HIVTP, a training-free method to improve VLMs efficiency via hierarchical visual token pruning using a novel middle-layer-based importance score. Specifically, we utilize attention maps extracted from the middle layers of the vision encoder, which better reflect fine-grained and object-level attention, to estimate visual token importance. Based on this, we propose a hierarchical visual token pruning method to retain both globally and locally important visual tokens. Specifically, we reshape the 1-D visual token sequence output by the vision encoder into a 2-D spatial layout. In the global retaining stage, we divide the image into regions and retain tokens with higher importance scores in each region; in the local retaining stage, we then divide the image into small windows and retain the most important token in each local window. Experimental results show that our proposed method, HIVTP, can reduce the time-to-first-token (TTFT) of LLaVA-v1.5-7B and LLaVA-Next-7B by up to 50.0% and 55.1%, respectively, and improve the token generation throughput by up to 60.9% and 47.3%, without sacrificing accuracy, and even achieving improvements on certain benchmarks. Compared with prior works, HIVTP achieves better accuracy while offering higher inference efficiency.

[202] Token Merging via Spatiotemporal Information Mining for Surgical Video Understanding

Xixi Jiang,Chen Yang,Dong Zhang,Pingcheng Dong,Xin Yang,Kwang-Ting Cheng

Main category: cs.CV

TL;DR: 该论文提出了一种用于手术视频理解的时空信息挖掘令牌合并方法（STIM-TM），首次实现了时空维度独立的令牌合并，显著降低了计算成本（减少65% GFLOPs），同时保持了高精度。

Details Motivation: 现有Transformer模型在处理手术视频时存在高计算成本问题，且未充分利用视频的时空结构和信息分布的异构性。

Contribution: 提出STIM-TM方法，通过解耦时空维度的令牌合并策略，独立减少冗余令牌；首次针对手术视频理解设计了专用方法。

Method: 1. 时间维度：通过显著性权重合并连续帧中空间对应的令牌，保留关键序列信息；2. 空间维度：通过时间稳定性分析优先合并静态令牌，保护动态区域。该方法无需训练即可实现高效处理。

Result: 显著提升效率（减少65% GFLOPs），在多种手术视频任务中保持竞争力；支持长序列手术视频的高效训练。

Insight: 时空维度的解耦分析和信息异构性挖掘是提升手术视频理解效率的关键。

Abstract: Vision Transformer models have shown impressive effectiveness in the surgical video understanding tasks through long-range dependency modeling. However, current methods suffer from prohibitive computational costs due to processing massive spatiotemporal tokens across video frames. While prior work on token merging has advanced model efficiency, they fail to adequately consider the inherent spatiotemporal structure of video data and overlook the heterogeneous nature of information distribution, leading to suboptimal performance. In this paper, we propose a spatiotemporal information mining token merging (STIM-TM) method, representing the first dedicated approach for surgical video understanding. STIM-TM introduces a decoupled strategy that reduces token redundancy along temporal and spatial dimensions independently. Specifically, the temporal component merges spatially corresponding tokens from consecutive frames using saliency weighting, preserving critical sequential information and maintaining continuity. Meanwhile, the spatial component prioritizes merging static tokens through temporal stability analysis, protecting dynamic regions containing essential surgical information. Operating in a training-free manner, STIM-TM achieves significant efficiency gains with over $65%$ GFLOPs reduction while preserving competitive accuracy across comprehensive surgical video tasks. Our method also supports efficient training of long-sequence surgical videos, addressing computational bottlenecks in surgical applications.

[203] RCI: A Score for Evaluating Global and Local Reasoning in Multimodal Benchmarks

Amit Agarwal,Hitesh Laxmichand Patel,Srikant Panda,Hansa Meghwani,Jyotika Singh,Karan Dua,Paul Li,Tao Sheng,Sujith Ravi,Dan Roth

Main category: cs.CV

TL;DR: 该论文提出了区域理解指数（RCI），用于评估多模态基准中对全局和局部视觉信息的依赖程度，揭示了当前大多数基准偏向局部推理的空间偏差。

Details Motivation: 现有的多模态大语言模型（MLLMs）在视觉语言基准上表现出色，但缺乏对是否真正需要全局推理的评估。当前的评估方法无法区分全局和局部推理，限制了数据集优化和实际应用模型的开发。

Contribution: 提出了RCI分数，首次直接量化数据集对全局与局部视觉信息的依赖。该方法可帮助诊断和缓解基准中的空间偏差，推动更健壮的多模态系统发展。

Method: RCI通过系统比较参考模型在图像块和完整图像上的表现，判断任务是依赖于全局理解还是可通过局部线索解决。

Result: 应用于13个多模态基准时，发现大多数基准倾向于局部推理，并表现出显著的空间偏差。

Insight: RCI为研究者和实践者提供了诊断工具，有助于构建更均衡的数据集和基准，从而促进面向实际应用的多模态系统开发。

Abstract: Multimodal Large Language Models (MLLMs) have achieved impressive results on vision-language benchmarks, yet it remains unclear whether these benchmarks assess genuine global reasoning or allow success via localized visual cues. Existing evaluation methods do not explicitly measure this distinction, hindering effective dataset curation and real-world focused model development. We introduce Region Comprehension Index (RCI), the first model-based score to directly quantify a dataset’s reliance on global versus local visual information. RCI systematically compares reference-model performance on image patches versus full images, revealing if tasks require holistic image understanding or can be solved with partial or localized visual cues. When applying RCI to 13 widely used multimodal benchmarks, we observed that most of them favor localized reasoning and exhibit significant spatial biases, indicating potential risks in real-world applications. RCI equips researchers & practitioners with an actionable tool for diagnosing & mitigating these biases, enabling the construction of datasets and benchmarks to foster the development of robust, enterprise-ready multimodal systems.

[204] MSD-KMamba: Bidirectional Spatial-Aware Multi-Modal 3D Brain Segmentation via Multi-scale Self-Distilled Fusion Strategy

Dayu Tan,Ziwei Zhang,Yansan Su,Xin Peng,Yike Dai,Chunhou Zheng,Weimin Zhong

Main category: cs.CV

TL;DR: 论文提出了MSD-KMamba框架，结合双向空间感知和多尺度自蒸馏策略，解决了CNN-Transformer混合模型在长距离依赖和高计算复杂度上的问题，显著提升了3D多模态脑部分割的精度和效率。

Details Motivation: 传统的CNN-Transformer混合模型依赖高复杂度的全局注意力机制，导致计算资源消耗大，而知识蒸馏和稀疏注意力机制在复杂任务中难以兼顾分割精度和效率。

Contribution: 1. 提出双向空间感知分支，捕捉长距离空间上下文依赖和非线性特征；2. 设计多尺度自蒸馏融合策略，增强分层特征表示；3. 在多个基准数据集上验证了框架的高精度、鲁棒性和高效性。

Method: 1. 双向空间感知分支捕获脑区长距离依赖；2. 多尺度自蒸馏融合策略优化特征传递；3. 结合两种机制降低计算复杂度，提升全局感知能力。

Result: 在多个标准数据集上，MSD-KMamba在分割精度、鲁棒性和泛化性上优于现有方法，同时保持高计算效率。

Insight: 通过双向空间感知和多尺度自蒸馏的结合，可以有效平衡模型的性能和计算复杂度，为3D多模态分割任务提供了新的解决方案。

Abstract: Numerous CNN-Transformer hybrid models rely on high-complexity global attention mechanisms to capture long-range dependencies, which introduces non-linear computational complexity and leads to significant resource consumption. Although knowledge distillation and sparse attention mechanisms can improve efficiency, they often fall short of delivering the high segmentation accuracy necessary for complex tasks. Balancing model performance with computational efficiency remains a critical challenge. In this work, we propose a novel 3D multi-modal image segmentation framework, termed MSD-KMamba, which integrates bidirectional spatial perception with multi-scale self-distillation. The bidirectional spatial aware branch effectively captures long-range spatial context dependencies across brain regions, while also incorporating a powerful nonlinear feature extraction mechanism that further enhances the model’s ability to learn complex and heterogeneous patterns. In addition, the proposed multi-scale self-distilled fusion strategy strengthens hierarchical feature representations and improves the transfer of semantic information at different resolution levels. By jointly leveraging the bidirectional spatial perception branch and the multi-scale self-distilled fusion strategy, our framework effectively mitigates the bottleneck of quadratic computational complexity in volumetric segmentation, while simultaneously addressing the limitation of insufficient global perception. Extensive experiments on multiple standard benchmark datasets demonstrate that MSD-KMamba consistently outperforms state-of-the-art methods in segmentation accuracy, robustness, and generalization, while maintaining high computational efficiency and favorable scalability. The source code of MSD-KMamba is publicly available at https://github.com/daimao-zhang/MSD-KMamba.

[205] QuantSparse: Comprehensively Compressing Video Diffusion Transformer with Model Quantization and Attention Sparsification

Weilun Feng,Chuanguang Yang,Haotong Qin,Mingqiang Wu,Yuqi Li,Xiangqi Li,Zhulin An,Libo Huang,Yulun Zhang,Michele Magno,Yongjun Xu

Main category: cs.CV

TL;DR: QuantSparse是一个统一的框架，通过结合模型量化和注意力稀疏化，显著降低了视频扩散变换器的计算和存储成本，同时保持了高性能。

Details Motivation: 尽管扩散变换器在视频生成方面表现出色，但其高昂的计算和内存成本限制了实际应用。现有方法（如量化和稀疏化）在极端压缩下性能下降严重。

Contribution: 提出了QuantSparse，首次将模型量化与注意力稀疏化统一到一个框架中，并通过多尺度显著注意力蒸馏（MSAD）和二阶稀疏注意力重参数化（SSAR）缓解压缩带来的信息损失。

Method: 1. 多尺度显著注意力蒸馏（MSAD）：通过全局结构引导和局部显著监督减轻量化偏差。2. 二阶稀疏注意力重参数化（SSAR）：利用二阶残差的时序稳定性高效恢复稀疏化丢失的信息。

Result: 在HunyuanVideo-13B数据集上，QuantSparse的PSNR达到20.88，显著优于基线Q-VDiT（16.85），同时存储和推理速度分别提升了3.68倍和1.88倍。

Insight: 通过多尺度蒸馏和二阶稀疏化的结合，QuantSparse展示了在极端压缩下仍能保持高性能的潜力，为视频生成模型的实用化提供了新思路。

Abstract: Diffusion transformers exhibit remarkable video generation capability, yet their prohibitive computational and memory costs hinder practical deployment. Model quantization and attention sparsification are two promising directions for compression, but each alone suffers severe performance degradation under aggressive compression. Combining them promises compounded efficiency gains, but naive integration is ineffective. The sparsity-induced information loss exacerbates quantization noise, leading to amplified attention shifts. To address this, we propose \textbf{QuantSparse}, a unified framework that integrates model quantization with attention sparsification. Specifically, we introduce \textit{Multi-Scale Salient Attention Distillation}, which leverages both global structural guidance and local salient supervision to mitigate quantization-induced bias. In addition, we develop \textit{Second-Order Sparse Attention Reparameterization}, which exploits the temporal stability of second-order residuals to efficiently recover information lost under sparsity. Experiments on HunyuanVideo-13B demonstrate that QuantSparse achieves 20.88 PSNR, substantially outperforming the state-of-the-art quantization baseline Q-VDiT (16.85 PSNR), while simultaneously delivering a \textbf{3.68$\times$} reduction in storage and \textbf{1.88$\times$} acceleration in end-to-end inference. Our code will be released in https://github.com/wlfeng0509/QuantSparse.

[206] HomeSafeBench: A Benchmark for Embodied Vision-Language Models in Free-Exploration Home Safety Inspection

Siyuan Gao,Jiashu Yao,Haoyu Wen,Yuhang Guo,Zeming Liu,Heyan Huang

Main category: cs.CV

TL;DR: 该论文提出了HomeSafeBench基准，用于评估具身视觉语言模型（VLMs）在自由探索家庭安全检查任务中的能力，解决了现有基准的两个主要问题：依赖文本描述和静态视角。

Details Motivation: 现有基准在家庭安全检查任务中存在不足：一是依赖环境文本描述而非视觉信息，二是使用静态视角限制了自由探索。这些问题导致无法准确评估基于VLMs的具身代理能力。

Contribution: 提出了HomeSafeBench基准，包含12,900个数据点，覆盖五种常见家庭安全隐患；提供了动态第一人称视角图像，支持自由探索任务，更全面地评估VLMs的能力。

Method: 通过模拟家庭环境生成动态第一人称视角图像，允许具身代理自由探索房间，从而收集多视角数据。

Result: 实验结果表明显著局限性：即使是表现最佳的VLMs也仅达到10.23%的F1分数，尤其在识别安全隐患和选择探索策略方面表现不佳。

Insight: 当前VLMs在家庭安全检查任务中存在明显不足，未来研究需关注动态多视角环境下的模型优化。

Abstract: Embodied agents can identify and report safety hazards in the home environments. Accurately evaluating their capabilities in home safety inspection tasks is curcial, but existing benchmarks suffer from two key limitations. First, they oversimplify safety inspection tasks by using textual descriptions of the environment instead of direct visual information, which hinders the accurate evaluation of embodied agents based on Vision-Language Models (VLMs). Second, they use a single, static viewpoint for environmental observation, which restricts the agents’ free exploration and cause the omission of certain safety hazards, especially those that are occluded from a fixed viewpoint. To alleviate these issues, we propose HomeSafeBench, a benchmark with 12,900 data points covering five common home safety hazards: fire, electric shock, falling object, trips, and child safety. HomeSafeBench provides dynamic first-person perspective images from simulated home environments, enabling the evaluation of VLM capabilities for home safety inspection. By allowing the embodied agents to freely explore the room, HomeSafeBench provides multiple dynamic perspectives in complex environments for a more thorough inspection. Our comprehensive evaluation of mainstream VLMs on HomeSafeBench reveals that even the best-performing model achieves an F1-score of only 10.23%, demonstrating significant limitations in current VLMs. The models particularly struggle with identifying safety hazards and selecting effective exploration strategies. We hope HomeSafeBench will provide valuable reference and support for future research related to home security inspections. Our dataset and code will be publicly available soon.

[207] Confidence Aware SSD Ensemble with Weighted Boxes Fusion for Weapon Detection

Atharva Jadhav,Arush Karekar,Manas Divekar,Shachi Natu

Main category: cs.CV

TL;DR: 该论文提出了一种基于SSD模型的集成方法，结合多种特征提取骨干网络和加权框融合（WBF）技术，显著提升了武器检测的鲁棒性和准确性。

Details Motivation: 公共安全需求迫切，而现有单一模型在复杂场景（如遮挡、光线变化等）中的检测效果不佳，亟需提升武器检测的鲁棒性。

Contribution: 1. 提出了一种多骨干SSD模型集成框架；2. 采用加权框融合（WBF）技术优化边界框精度；3. 证明了置信度感知融合对集成模型性能的关键作用。

Method: 1. 使用VGG16、ResNet50、EfficientNet和MobileNetV3作为SSD的骨干网络训练多个模型；2. 通过WBF方法融合预测结果，并采用’max’置信度评分策略。

Result: 集成模型的mAP达到0.838，相对最佳单一模型提升了2.948%。

Insight: 融合策略（如WBF）与模型多样性同等重要，置信度感知融合是提升集成模型精度的关键机制。

Abstract: The safety and security of public spaces is of vital importance, driving the need for sophisticated surveillance systems capable of accurately detecting weapons, which are often hampered by issues like partial occlusion, varying lighting, and cluttered backgrounds. While single-model detectors are advanced, they often lack robustness in these challenging conditions. This paper presents the hypothesis that ensemble of Single Shot Multibox Detector (SSD) models with diverse feature extraction backbones can significantly enhance detection robustness. To leverage diverse feature representations, individual SSD models were trained using a selection of backbone networks: VGG16, ResNet50, EfficientNet, and MobileNetV3. The study is conducted on a dataset consisting of images of three distinct weapon classes: guns, heavy weapons and knives. The predictions from these models are combined using the Weighted Boxes Fusion (WBF) method, an ensemble technique designed to optimize bounding box accuracy. Our key finding is that the fusion strategy is as critical as the ensemble’s diversity, a WBF approach using a ‘max’ confidence scoring strategy achieved a mean Average Precision (mAP) of 0.838. This represents a 2.948% relative improvement over the best-performing single model and consistently outperforms other fusion heuristics. This research offers a robust approach to enhancing real-time weapon detection capabilities in surveillance applications by demonstrating that confidence-aware fusion is a key mechanism for improving accuracy metrics of ensembles.

[208] DiffPCN: Latent Diffusion Model Based on Multi-view Depth Images for Point Cloud Completion

Zijun Li,Hongyu Yan,Shijie Li,Kunming Luo,Li Lu,Xulei Yang,Weisi Lin

Main category: cs.CV

TL;DR: DiffPCN是一个基于潜在扩散模型的点云补全框架，通过两阶段（粗生成和细化）方法实现高精度和高完整性的点云补全。

Details Motivation: 点云的非结构化和不规则特性限制了潜在扩散模型（LDMs）在点云补全中的应用。为了充分利用LDMs的强大生成能力，DiffPCN提出了一种新颖的基于多视角深度图像的生成和细化方法。

Contribution: 1. 提出了DiffPCN框架，首次将潜在扩散模型引入点云补全任务；2. 设计了DepthLDM用于生成高质量的多视角深度图像；3. 提出了点去噪网络和关联感知点上采样器，优化粗点云质量。

Method: 1. 通过DepthLDM将部分点云投影为结构化深度图像，生成粗点云；2. 使用点去噪网络移除粗点云中的异常点；3. 通过关联感知点上采样器提升点云密度和保真度。

Result: 实验结果显示，DiffPCN在几何精度和形状完整性上达到最先进水平，显著提升了点云补全的鲁棒性和一致性。

Insight: DiffPCN通过结合潜在扩散模型的生成能力和点云的局部关联特征，为非结构化数据的生成任务提供了新的解决方案。

Abstract: Latent diffusion models (LDMs) have demonstrated remarkable generative capabilities across various low-level vision tasks. However, their potential for point cloud completion remains underexplored due to the unstructured and irregular nature of point clouds. In this work, we propose DiffPCN, a novel diffusion-based coarse-to-fine framework for point cloud completion. Our approach comprises two stages: an initial stage for generating coarse point clouds, and a refinement stage that improves their quality through point denoising and upsampling. Specifically, we first project the unordered and irregular partial point cloud into structured depth images, which serve as conditions for a well-designed DepthLDM to synthesize completed multi-view depth images that are used to form coarse point clouds. In this way, our DiffPCN can yield high-quality and high-completeness coarse point clouds by leveraging LDM’ s powerful generation and comprehension capabilities. Then, since LDMs inevitably introduce outliers into the generated depth maps, we design a Point Denoising Network to remove artifacts from the coarse point cloud by predicting a per-point distance score. Finally, we devise an Association-Aware Point Upsampler, which guides the upsampling process by leveraging local association features between the input point cloud and the corresponding coarse points, further yielding a dense and high-fidelity output. Experimental results demonstrate that our DiffPCN achieves state-of-the-art performance in geometric accuracy and shape completeness, significantly improving the robustness and consistency of point cloud completion.

[209] Video Panels for Long Video Understanding

Lars Doorenbos,Federico Spurio,Juergen Gall

Main category: cs.CV

TL;DR: 该论文提出了一种用于长视频理解的视觉提示策略，通过将多帧图像合并为一个面板，以空间细节换取时间分辨率，从而提升现有模型的性能。

Details Motivation: 现有的视频-语言模型在长视频理解任务上表现不佳，而改进模型复杂度或引入新模块需要大量调整和训练时间。因此，作者提出了一种无需训练、无需参数的通用方法。

Contribution: 1. 提出了一种新颖的视觉提示策略，通过合并多帧图像为面板来解决长视频理解问题。2. 方法是训练无关、参数无关的，可兼容多种模型架构。3. 在多个基准测试中验证了方法的有效性，显著提升了长视频问答任务的准确率。

Method: 1. 设计了一种视觉提示策略，将多帧图像合并为单一面板，以空间分辨率为代价换取更高时间分辨率。2. 无需额外训练或调整模型参数，可直接应用于现有视频-语言模型。

Result: 1. 在TimeScope（Long）数据集上，视频问答任务的准确率提升了高达19.4%。2. 在五种基准测试中，对不同模型架构和规模均表现出一致性改进。

Insight: 1. 通过简单的视觉提示策略，可以在不修改模型结构的情况下提升长视频理解性能。2. 空间与时间分辨率的权衡是长视频理解的关键挑战之一。

Abstract: Recent Video-Language Models (VLMs) achieve promising results on long-video understanding, but their performance still lags behind that achieved on tasks involving images or short videos. This has led to great interest in improving the long context modeling of VLMs by introducing novel modules and additional complexity. % additional training time. In this paper, we take a different approach: rather than fine-tuning VLMs with the limited data available, we attempt to maximize the performance of existing models. To this end, we propose a novel visual prompting strategy specifically designed for long-video understanding. By combining multiple frames as panels into one image, we effectively trade off spatial details for temporal resolution. Our approach is training-free, parameter-free, and model-agnostic, and can be seamlessly integrated into existing VLMs. Extensive experiments on five established benchmarks across a wide range of model architectures, sizes, and context windows confirm the consistency of our approach. For the TimeScope (Long) dataset, which has the longest videos, the accuracy for video question answering is improved by up to 19.4%. Overall, our method raises the bar for long video understanding models. We will make our code available upon acceptance.

[210] M3DLayout: A Multi-Source Dataset of 3D Indoor Layouts and Structured Descriptions for 3D Generation

Yiheng Zhang,Zhuojiang Cai,Mingdao Wang,Meitong Guo,Tianxiao Li,Li Lin,Yuwang Wang

Main category: cs.CV

TL;DR: 论文提出了M3DLayout数据集，一个用于3D室内布局生成的大规模多源数据集，旨在解决现有数据集规模小、多样性和标注质量不足的问题。

Details Motivation: 当前3D室内布局生成模型的能力受限于数据集的规模、多样性和标注质量，限制了模型的性能和应用。

Contribution: 1. 提出了M3DLayout数据集，包含15,080个布局和258k+对象实例，整合了真实扫描、CAD设计和程序生成场景三种来源；2. 每个布局配以结构化文本描述，增强语义和空间关系的学习；3. 通过基准测试验证了数据集的有效性。

Method: 1. 整合多源数据；2. 提供结构化文本描述；3. 基于扩散模型建立文本驱动的布局生成基准。

Result: 实验表明M3DLayout能够支持复杂和详细的场景生成，尤其是Inf3DLayout子集提升了小物体的生成能力。

Insight: 多源和高质量标注的数据集是提升3D场景生成模型性能的关键。

Abstract: In text-driven 3D scene generation, object layout serves as a crucial intermediate representation that bridges high-level language instructions with detailed geometric output. It not only provides a structural blueprint for ensuring physical plausibility but also supports semantic controllability and interactive editing. However, the learning capabilities of current 3D indoor layout generation models are constrained by the limited scale, diversity, and annotation quality of existing datasets. To address this, we introduce M3DLayout, a large-scale, multi-source dataset for 3D indoor layout generation. M3DLayout comprises 15,080 layouts and over 258k object instances, integrating three distinct sources: real-world scans, professional CAD designs, and procedurally generated scenes. Each layout is paired with detailed structured text describing global scene summaries, relational placements of large furniture, and fine-grained arrangements of smaller items. This diverse and richly annotated resource enables models to learn complex spatial and semantic patterns across a wide variety of indoor environments. To assess the potential of M3DLayout, we establish a benchmark using a text-conditioned diffusion model. Experimental results demonstrate that our dataset provides a solid foundation for training layout generation models. Its multi-source composition enhances diversity, notably through the Inf3DLayout subset which provides rich small-object information, enabling the generation of more complex and detailed scenes. We hope that M3DLayout can serve as a valuable resource for advancing research in text-driven 3D scene synthesis.

[211] LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models

Shubhang Bhatnagar,Andy Xu,Kar-Han Tan,Narendra Ahuja

Main category: cs.CV

TL;DR: 本文研究了多模态大语言模型（MLLMs）的超低比特（<4-bit）量化问题，提出了基于层间差异性分析的新方法LUQ（Layerwise Ultra-Low Bit Quantization），通过选择性量化显著减少了内存占用，同时性能下降控制在10%以内。

Details Motivation: 多模态大语言模型（MLLMs）在视觉语言任务中表现优异，但其部署需要巨大的内存和计算资源。现有的后训练量化（PTQ）方法在单模态语言模型中已成功压缩至1-bit，但对MLLMs的效果尚未充分研究。

Contribution: 1. 首次探索MLLMs的超低比特量化问题；2. 揭示了多模态token与文本token的统计特性差异；3. 提出了LUQ方法，基于层间熵差异选择性量化；4. 验证了混合模态token在PTQ中的有效性。

Method: 1. 分析MLLMs中不同层的激活分布差异；2. 选择熵较低的层进行超低比特量化；3. 在PTQ中使用混合模态（图像和文本）提升性能。

Result: 在LLaVA-1.5和Qwen-2.5-VL上评估，LUQ模型的内存占用分别比4-bit模型减少40%和31%，在MME基准上的性能下降<10%。

Insight: 1. MLLMs的层间熵差异提供了量化优化的空间；2. 混合模态token可提升量化性能；3. 选择性量化是降低MLLMs内存占用的有效策略。

Abstract: Large Language Models (LLMs) with multimodal capabilities have revolutionized vision-language tasks, but their deployment often requires huge memory and computational resources. While post-training quantization (PTQ) has successfully compressed language models to as low as 1-bit precision without significant performance loss, its effectiveness for multimodal LLMs (MLLMs) remains relatively unexplored. In this paper, we present the first study on ultra-low bit (<4-bit) quantization for multimodal LLMs. Our analysis reveals that multimodal tokens and intermediate layer activations produced by them exhibit significantly higher statistical variance and entropy compared to text tokens, making them less tolerant to ultra-low bit quantization. However, the activation distributions of multimodal tokens varies significantly over different layers, with some layers having lower entropy activation distributions. We empirically show that such layers in these models can better tolerate ultra-low bit quantization. Building on these insights, we propose a novel strategy for MLLM quantization, LUQ: Layerwise Ultra-Low Bit Quantization, which selectively applies ultra-low bit quantization to layers that are more resilient to it. Additionally, we also show that using a mix of multimodal tokens (image and text) for PTQ boosts VQA performance in the ultra-low bit regime. We evaluate our method on LLaVA-1.5 and Qwen-2.5-VL across 9 popular VQA benchmarks. The resulting LUQ models use 40% and 31% less memory than their 4-bit counterparts, respectively, while exhibiting a performance degradation of less than 10% on the MME benchmark.

[212] FastViDAR: Real-Time Omnidirectional Depth Estimation via Alternative Hierarchical Attention

Hangtian Zhao,Xiang Chen,Yizhe Li,Qianhao Wang,Haibo Lu,Fei Gao

Main category: cs.CV

TL;DR: FastViDAR提出了一种实时多目鱼眼摄像头深度估计框架，通过新型的替代分层注意力（AHA）机制和多视角ERP融合方法，实现了高效的特征融合和深度估计，并在嵌入硬件上达到20 FPS的实时性能。

Details Motivation: 实时全景深度估计在自动驾驶、虚拟现实等领域需求迫切，但现有方法在计算效率和特征融合方面存在瓶颈。

Contribution: 1. 提出替代分层注意力（AHA）机制，高效融合多视角特征；2. 提出ERP融合方法，将多视角深度投影到统一坐标系；3. 利用HM3D和2D3D-S数据集生成真实场景评估数据，实现零样本性能。

Method: 1. 使用AHA机制分层次融合帧内和帧间特征；2. 通过ERP投影实现多视角深度融合；3. 在嵌入硬件上优化实现高效推理。

Result: 模型在真实数据集上表现优异，零样本性能竞争力强，嵌入硬件推理速度达20 FPS。

Insight: AHA机制在减少计算开销的同时实现了有效的跨视角特征融合，ERP投影为多视角深度统一提供了高效解决方案。

Abstract: In this paper we propose FastViDAR, a novel framework that takes four fisheye camera inputs and produces a full $360^\circ$ depth map along with per-camera depth, fusion depth, and confidence estimates. Our main contributions are: (1) We introduce Alternative Hierarchical Attention (AHA) mechanism that efficiently fuses features across views through separate intra-frame and inter-frame windowed self-attention, achieving cross-view feature mixing with reduced overhead. (2) We propose a novel ERP fusion approach that projects multi-view depth estimates to a shared equirectangular coordinate system to obtain the final fusion depth. (3) We generate ERP image-depth pairs using HM3D and 2D3D-S datasets for comprehensive evaluation, demonstrating competitive zero-shot performance on real datasets while achieving up to 20 FPS on NVIDIA Orin NX embedded hardware. Project page: \href{https://3f7dfc.github.io/FastVidar/}{https://3f7dfc.github.io/FastVidar/}

[213] HieraTok: Multi-Scale Visual Tokenizer Improves Image Reconstruction and Generation

Cong Chen,Ziyuan Huang,Cheng Zou,Muzhi Zhu,Kaixiang Ji,Jiajia Liu,Jingdong Chen,Hao Chen,Chunhua Shen

Main category: cs.CV

TL;DR: HieraTok是一种基于多尺度Vision Transformer（ViT）的Tokenizer，通过多尺度下采样和尺度因果注意力机制显著提升了图像重建和生成任务的性能，并在ViT Tokenizer中实现了最优结果。

Details Motivation: 现有的单尺度ViT Tokenizer在建模图像表征时存在局限性。本文提出多尺度Tokenizer设计，以更好地捕捉从全局语义到局部细节的视觉信息。

Contribution: 1. 提出多尺度ViT Tokenizer（HieraTok），首次在多尺度ViT框架下实现Tokenizer设计；
2. 引入尺度因果注意力机制，支持信息从低分辨率到高分辨率的渐进传递；
3. 在图像重建和生成任务中显著优于单尺度Tokenizer，并实现了ViT Tokenizer中的最佳效果。

Method: 1. 多尺度下采样：将Tokenizer编码器生成的Token图下采样为多尺度序列；
2. 尺度因果注意力机制：通过attention机制实现跨尺度信息流动，从全局特征逐步细化到局部细节。

Result: 1. 图像重建任务中rFID提升27.2%（1.47→1.07）；
2. 下游生成任务中，收敛速度提高1.38倍，gFID提升18.9%（16.4→13.3）；
3. 训练扩展后，rFID达到0.45，gFID达到1.82，均为ViT Tokenizer的最佳性能。

Insight: 多尺度表征和渐进信息传递是提升ViT Tokenizer性能的关键。HieraTok的平滑且均匀分布的潜在空间可能是下游任务加速收敛的核心原因。

Abstract: In this work, we present HieraTok, a novel multi-scale Vision Transformer (ViT)-based tokenizer that overcomes the inherent limitation of modeling single-scale representations. This is realized through two key designs: (1) multi-scale downsampling applied to the token map generated by the tokenizer encoder, producing a sequence of multi-scale tokens, and (2) a scale-causal attention mechanism that enables the progressive flow of information from low-resolution global semantic features to high-resolution structural details. Coupling these designs, HieraTok achieves significant improvements in both image reconstruction and generation tasks. Under identical settings, the multi-scale visual tokenizer outperforms its single-scale counterpart by a 27.2% improvement in rFID ($1.47 \rightarrow 1.07$). When integrated into downstream generation frameworks, it achieves a $1.38\times$ faster convergence rate and an 18.9% boost in gFID ($16.4 \rightarrow 13.3$), which may be attributed to the smoother and more uniformly distributed latent space. Furthermore, by scaling up the tokenizer’s training, we demonstrate its potential by a sota rFID of 0.45 and a gFID of 1.82 among ViT tokenizers. To the best of our knowledge, we are the first to introduce multi-scale ViT-based tokenizer in image reconstruction and image generation. We hope our findings and designs advance the ViT-based tokenizers in visual generation tasks.

[214] GRS-SLAM3R: Real-Time Dense SLAM with Gated Recurrent State

Guole Shen,Tianchen Deng,Yanbo Wang,Yongtao Chen,Yilin Shen,Jiuming Liu,Jingchuan Wang

Main category: cs.CV

TL;DR: GRS-SLAM3R提出了一种基于GRU（门控循环单元）的端到端SLAM框架，支持实时稠密场景重建和姿态估计，通过空间记忆和全局一致性优化提升了DUSt3R方法的性能。

Details Motivation: 现有DUSt3R方法仅通过图像对估计点云，忽略了空间记忆和全局一致性，限制了稠密SLAM的性能。

Contribution: 1. 提出GRS-SLAM3R框架，支持序列化输入并全局增量重建；2. 设计了基于Transformer的门控更新模块；3. 提出子图划分与局部对齐策略，提升全局一致性。

Method: 1. 使用潜在状态存储空间记忆；2. 门控更新模块动态聚合3D信息；3. 子图分割与局部对齐，最终注册到全局坐标系。

Result: 实验表明，GRS-SLAM3R在重建精度和实时性上均优于现有方法。

Insight: 空间记忆和全局一致性是稠密SLAM的关键，门控机制和子图策略能有效平衡实时性与精度。

Abstract: DUSt3R-based end-to-end scene reconstruction has recently shown promising results in dense visual SLAM. However, most existing methods only use image pairs to estimate pointmaps, overlooking spatial memory and global consistency.To this end, we introduce GRS-SLAM3R, an end-to-end SLAM framework for dense scene reconstruction and pose estimation from RGB images without any prior knowledge of the scene or camera parameters. Unlike existing DUSt3R-based frameworks, which operate on all image pairs and predict per-pair point maps in local coordinate frames, our method supports sequentialized input and incrementally estimates metric-scale point clouds in the global coordinate. In order to improve consistent spatial correlation, we use a latent state for spatial memory and design a transformer-based gated update module to reset and update the spatial memory that continuously aggregates and tracks relevant 3D information across frames. Furthermore, we partition the scene into submaps, apply local alignment within each submap, and register all submaps into a common world frame using relative constraints, producing a globally consistent map. Experiments on various datasets show that our framework achieves superior reconstruction accuracy while maintaining real-time performance.

[215] ResAD++: Towards Class Agnostic Anomaly Detection via Residual Feature Learning

Xincheng Yao,Chao Shi,Muming Zhao,Guangtao Zhai,Chongyang Zhang

Main category: cs.CV

TL;DR: ResAD++提出了一种类无关的异常检测方法，通过学习残差特征分布而非初始特征分布，实现了特征解相关，并通过特征超球约束和新的损失函数进一步提升性能。

Details Motivation: 当前的单类和多类异常检测方法在新类别上的表现不佳，主要原因是特征学习仍与类别相关。

Contribution: 1. 提出残差特征学习框架ResAD；2. 引入特征超球约束解决尺度相关性；3. 设计了新颖的损失函数和特征分布匹配模块。

Method: 通过学习残差特征分布并约束其特征尺度一致性，结合logbarrier双向收缩OCC损失和特征分布匹配模块进行优化。

Result: 在8个真实异常检测数据集上，ResAD++在新类别中显著优于现有方法，且超越了原始ResAD。

Insight: 残差特征分布的一致性对新类别的泛化能力至关重要，特征解相关和尺度约束是实现这一目标的有效手段。

Abstract: This paper explores the problem of class-agnostic anomaly detection (AD), where the objective is to train one class-agnostic AD model that can generalize to detect anomalies in diverse new classes from different domains without any retraining or fine-tuning on the target data. When applied for new classes, the performance of current single- and multi-class AD methods is still unsatisfactory. One fundamental reason is that representation learning in existing methods is still class-related, namely, feature correlation. To address this issue, we propose residual features and construct a simple but effective framework, termed ResAD. Our core insight is to learn the residual feature distribution rather than the initial feature distribution. Residual features are formed by matching and then subtracting normal reference features. In this way, we can effectively realize feature decorrelation. Even in new classes, the distribution of normal residual features would not remarkably shift from the learned distribution. In addition, we think that residual features still have one issue: scale correlation. To this end, we propose a feature hypersphere constraining approach, which learns to constrain initial normal residual features into a spatial hypersphere for enabling the feature scales of different classes as consistent as possible. Furthermore, we propose a novel logbarrier bidirectional contraction OCC loss and vector quantization based feature distribution matching module to enhance ResAD, leading to the improved version of ResAD (ResAD++). Comprehensive experiments on eight real-world AD datasets demonstrate that our ResAD++ can achieve remarkable AD results when directly used in new classes, outperforming state-of-the-art competing methods and also surpassing ResAD. The code is available at https://github.com/xcyao00/ResAD.

[216] Poivre: Self-Refining Visual Pointing with Reinforcement Learning

Wenjie Yang,Zengfeng Huang

Main category: cs.CV

TL;DR: 论文提出了一种名为Poivre的自微调视觉指向方法，通过强化学习（RL）迭代优化坐标预测，显著提升了视觉语言模型（VLM）在视觉指向任务上的性能。

Details Motivation: 当前视觉语言模型在单一步骤完成视觉指向任务时表现远不及人类，主要原因是模型无法像人类一样通过观察自身动作进行迭代调整。

Contribution: 提出Poivre自微调框架，首次将RL应用于视觉指向任务，设计了简洁有效的奖励机制，并在公开评测中超越了现有最优模型。

Method: 采用Point、Visualize、Refine三步骤迭代优化坐标预测，通过RL训练激励模型的自我微调能力，设计了基于实践效果的理论奖励机制。

Result: Poivre-7B在Point-Bench评测中优于Gemini-2.5-Pro和Molmo-72B等大型模型，性能提升超过3%。

Insight: 通过模仿人类的迭代调整行为，结合RL训练，可以显著提升模型在视觉指向任务上的表现，为未来研究提供了新的思路和工具。

Abstract: Visual pointing, which aims to localize a target by predicting its coordinates on an image, has emerged as an important problem in the realm of vision-language models (VLMs). Despite its broad applicability, recent benchmarks show that current VLMs still fall far behind human performance on this task. A key limitation is that VLMs are typically required to complete the pointing task in a single step, akin to asking humans to point at an object without seeing their own fingers. To address this issue, we propose a simple yet effective self-refining procedure: Point, Visualize, then Refine (Poivre). This procedure enables a VLM to first mark its estimated point, then iteratively refine the coordinates if necessary. Inspired by advances of reasoning models in the natural language domain, we employ reinforcement learning (RL) to incentivize this self-refining ability. For the RL training, we design a neat process reward that is not only empirically effective but also grounded in appealing properties. Our trained model, Poivre-7B, sets a new state of the art on Point-Bench, outperforming both proprietary models such as Gemini-2.5-Pro and large open-source models such as Molmo-72B by over 3%. To support future research, we release our training and inference code, dataset, and the Poivre-7B checkpoint.

[217] PVTAdpNet: Polyp Segmentation using Pyramid vision transformer with a novel Adapter block

Arshia Yousefi Nezhad,Helia Aghaei,Hedieh Sajedi

Main category: cs.CV

TL;DR: PVTAdpNet是一个基于Pyramid Vision Transformer的新型息肉分割模型，结合U-Net结构和适配器块，实现了高精度的实时分割，适用于临床。

Details Motivation: 结直肠癌是常见且致命的癌症，传统结肠镜检查因息肉变异性漏检率高。PVTAdpNet旨在通过改进的特征提取和分割技术提升检测效果。

Contribution: 1. 提出PVTAdpNet，结合Pyramid Vision Transformer和适配器块；2. 设计了新型残差块和适配器跳跃连接；3. 引入Squeeze-and-Excitation注意力机制优化通道特征。

Method: 1. 使用U-Net风格的编码器-解码器结构；2. Pyramid Vision Transformer作为主干网络；3. 新增适配器块和残差块优化梯度流和特征提取；4. 集成注意力机制细化特征。

Result: 在息肉数据集上，PVTAdpNet实现了mDice 0.8851和mIoU 0.8167的高分，展现了优异的实时分割性能。

Insight: 结合Transformer和CNN的优势，并引入适配器和注意力机制，能够在复杂医学图像任务中实现高效分割。

Abstract: Colorectal cancer ranks among the most common and deadly cancers, emphasizing the need for effective early detection and treatment. To address the limitations of traditional colonoscopy, including high miss rates due to polyp variability, we introduce the Pyramid Vision Transformer Adapter Residual Network (PVTAdpNet). This model integrates a U-Net-style encoder-decoder structure with a Pyramid Vision Transformer backbone, novel residual blocks, and adapter-based skip connections. The design enhances feature extraction, dense prediction, and gradient flow, supported by squeeze-and-excitation attention for improved channel-wise feature refinement. PVTAdpNet achieves real-time, accurate polyp segmentation, demonstrating superior performance on benchmark datasets with high mDice and mIoU scores, making it highly suitable for clinical applications. PVTAdpNet obtains a high Dice coefficient of 0.8851 and a mean Intersection over Union (mIoU) of 0.8167 on out-of-distribution polyp datasets. Evaluation of the PolypGen dataset demonstrates PVTAdpNet’s capability for real-time, accurate performance within familiar distributions. The source code of our network is available at https://github.com/ayousefinejad/PVTAdpNet.git

[218] UniAlignment: Semantic Alignment for Unified Image Generation, Understanding, Manipulation and Perception

Xinyang Song,Libin Wang,Weining Wang,Shaozhen Liu,Dandan Zheng,Jingdong Chen,Qi Li,Zhenan Sun

Main category: cs.CV

TL;DR: UniAlignment提出了一种基于扩散变换器的统一多模态生成框架，通过双流扩散训练策略增强跨模态一致性，且在复杂文本指令下表现优异。

Details Motivation: 现有方法依赖视觉语言模型或模块化设计，导致架构碎片化和计算低效。UniAlignment旨在解决这些问题，提升跨模态语义理解的统一性和效率。

Contribution: 1. 提出UniAlignment框架，实现统一的图像生成、理解、操作和感知。2. 设计双流扩散训练策略，增强跨模态一致性。3. 推出SemGen-Bench基准评测跨模态语义一致性。

Method: 采用单扩散变换器框架，结合内在模态和跨模态语义对齐的双流扩散训练，提升模型对复杂指令的鲁棒性。

Result: 在多个任务和基准测试中表现优异，证明了扩散模型在多模态统一生成中的潜力。

Insight: 通过统一的生成框架和语义对齐策略，UniAlignment为多模态任务提供了一种高效且一致的方法。

Abstract: The remarkable success of diffusion models in text-to-image generation has sparked growing interest in expanding their capabilities to a variety of multi-modal tasks, including image understanding, manipulation, and perception. These tasks require advanced semantic comprehension across both visual and textual modalities, especially in scenarios involving complex semantic instructions. However, existing approaches often rely heavily on vision-language models (VLMs) or modular designs for semantic guidance, leading to fragmented architectures and computational inefficiency. To address these challenges, we propose UniAlignment, a unified multimodal generation framework within a single diffusion transformer. UniAlignment introduces a dual-stream diffusion training strategy that incorporates both intrinsic-modal semantic alignment and cross-modal semantic alignment, thereby enhancing the model’s cross-modal consistency and instruction-following robustness. Additionally, we present SemGen-Bench, a new benchmark specifically designed to evaluate multimodal semantic consistency under complex textual instructions. Extensive experiments across multiple tasks and benchmarks demonstrate that UniAlignment outperforms existing baselines, underscoring the significant potential of diffusion models in unified multimodal generation.

[219] GenView++: Unifying Adaptive View Generation and Quality-Driven Supervision for Contrastive Representation Learning

Xiaojie Li,Bei Wang,Jianlong Wu,Yue Yu,Liqiang Nie,Min Zhang

Main category: cs.CV

TL;DR: GenView++ 是一个统一的对比学习框架，通过自适应视图生成和质量驱动的监督机制，解决了现有方法在视图多样性和语义一致性上的不足，显著提升了视觉和视觉-语言任务的性能。

Details Motivation: 当前对比学习方法在高质量正对生成和监督机制上存在不足：视图多样化有限且可能破坏语义，同时缺乏对正对质量的评估导致训练效率低下。

Contribution: 提出 GenView++ 框架，包含自适应视图生成机制和质量驱动的对比学习机制，分别解决视图多样化、语义一致性和监督效率问题。

Method: 1. 多源自适应视图生成：动态调制生成参数，结合图像、文本及图像-文本条件策略生成多样且语义一致的视图；2. 质量驱动的对比学习：根据正对的语义对齐和多样性动态加权训练贡献。

Result: 在视觉任务中，ImageNet 线性分类提升 +2.5%；在视觉-语言任务中，零样本分类平均准确率提升 +12.31%（相比 CLIP）和 +5.31%（相比 SLIP），Flickr30k 文本检索 R@5 提升 +3.2%。

Insight: 通过动态生成多样化视图并结合质量评估的监督机制，能够显著提升对比学习的性能，尤其在跨模态任务中表现突出。

Abstract: The success of contrastive learning depends on the construction and utilization of high-quality positive pairs. However, current methods face critical limitations on two fronts: on the construction side, both handcrafted and generative augmentations often suffer from limited diversity and risk semantic corruption; on the learning side, the absence of a quality assessment mechanism leads to suboptimal supervision where all pairs are treated equally. To tackle these challenges, we propose GenView++, a unified framework that addresses both fronts by introducing two synergistic innovations. To improve pair construction, GenView++ introduces a multi-source adaptive view generation mechanism to synthesize diverse yet semantically coherent views by dynamically modulating generative parameters across image-conditioned, text-conditioned, and image-text-conditioned strategies. Second, a quality-driven contrastive learning mechanism assesses each pair’s semantic alignment and diversity to dynamically reweight their training contribution, prioritizing high-quality pairs while suppressing redundant or misaligned pairs. Extensive experiments demonstrate the effectiveness of GenView++ across both vision and vision-language tasks. For vision representation learning, it improves MoCov2 by +2.5% on ImageNet linear classification. For vision-language learning, it raises the average zero-shot classification accuracy by +12.31% over CLIP and +5.31% over SLIP across ten datasets, and further improves Flickr30k text retrieval R@5 by +3.2%. The code is available at https://github.com/xiaojieli0903/GenViewPlusPlus.

[220] A Modality-Tailored Graph Modeling Framework for Urban Region Representation via Contrastive Learning

Yaya Zhao,Kaiqi Zhao,Zixuan Tang,Zhiyuan Liu,Xiaoling Lu,Yalei Du

Main category: cs.CV

TL;DR: MTGRR是一个针对多模态城市数据的图建模框架，通过定制化和对比学习优化区域表示。

Details Motivation: 现有方法在多模态城市数据处理中，采用统一图神经网络架构，忽略了模态特异性；且在融合阶段忽视空间异质性，导致次优表示。

Contribution: 提出了MTGRR框架，划分模态为聚合级和点级，分别采用MoE图和双级GNN处理；设计了空间感知的多模态融合机制。

Method: 1. 模态分类与定制化处理（MoE图与双级GNN）；2. 动态推断区域特异性融合权重；3. 联合对比学习优化表示。

Result: 在两个真实数据集上，MTGRR在六种模态和三项任务中均优于现有方法。

Insight: 模态特异性与空间异质性是多模态城市数据建模的关键，定制化处理和动态融合机制显著提升表示效果。

Abstract: Graph-based models have emerged as a powerful paradigm for modeling multimodal urban data and learning region representations for various downstream tasks. However, existing approaches face two major limitations. (1) They typically employ identical graph neural network architectures across all modalities, failing to capture modality-specific structures and characteristics. (2) During the fusion stage, they often neglect spatial heterogeneity by assuming that the aggregation weights of different modalities remain invariant across regions, resulting in suboptimal representations. To address these issues, we propose MTGRR, a modality-tailored graph modeling framework for urban region representation, built upon a multimodal dataset comprising point of interest (POI), taxi mobility, land use, road element, remote sensing, and street view images. (1) MTGRR categorizes modalities into two groups based on spatial density and data characteristics: aggregated-level and point-level modalities. For aggregated-level modalities, MTGRR employs a mixture-of-experts (MoE) graph architecture, where each modality is processed by a dedicated expert GNN to capture distinct modality-specific characteristics. For the point-level modality, a dual-level GNN is constructed to extract fine-grained visual semantic features. (2) To obtain effective region representations under spatial heterogeneity, a spatially-aware multimodal fusion mechanism is designed to dynamically infer region-specific modality fusion weights. Building on this graph modeling framework, MTGRR further employs a joint contrastive learning strategy that integrates region aggregated-level, point-level, and fusion-level objectives to optimize region representations. Experiments on two real-world datasets across six modalities and three tasks demonstrate that MTGRR consistently outperforms state-of-the-art baselines, validating its effectiveness.

[221] Texture Vector-Quantization and Reconstruction Aware Prediction for Generative Super-Resolution

Qifan Li,Jiale Zou,Jinhua Zhang,Wei Long,Xinyu Zhou,Shuhang Gu

Main category: cs.CV

TL;DR: 本文提出了一种纹理向量量化和重建感知预测策略（TVQ&RAP），旨在解决生成超分辨率任务中向量量化（VQ）方法的量化误差大和预测器训练不优化的问题。通过任务特性优化量化策略，并利用直通估计器直接训练预测器，实现了计算成本低且逼真的超分辨率效果。

Details Motivation: 现有基于向量量化的方法在编码视觉特征时存在较大的量化误差，且预测器训练仅依赖代码级监督，忽略了最终重建误差，导致先验建模精度不足。本文旨在解决这两个问题。

Contribution: 1. 提出了纹理向量量化策略，针对超分辨率任务特性优化先验建模；2. 提出了重建感知预测策略，利用直通估计器直接以图像级监督训练预测器。

Method: 1. 纹理向量量化仅对缺失纹理的先验建模，减少量化误差；2. 重建感知预测通过直通估计器实现图像级监督训练预测器。

Result: TVQ&RAP模型能够以较小的计算成本生成逼真的超分辨率结果。

Insight: 1. 任务导向的量化策略可以优化先验建模；2. 图像级监督训练预测器能有效降低重建误差。

Abstract: Vector-quantized based models have recently demonstrated strong potential for visual prior modeling. However, existing VQ-based methods simply encode visual features with nearest codebook items and train index predictor with code-level supervision. Due to the richness of visual signal, VQ encoding often leads to large quantization error. Furthermore, training predictor with code-level supervision can not take the final reconstruction errors into consideration, result in sub-optimal prior modeling accuracy. In this paper we address the above two issues and propose a Texture Vector-Quantization and a Reconstruction Aware Prediction strategy. The texture vector-quantization strategy leverages the task character of super-resolution and only introduce codebook to model the prior of missing textures. While the reconstruction aware prediction strategy makes use of the straight-through estimator to directly train index predictor with image-level supervision. Our proposed generative SR model (TVQ&RAP) is able to deliver photo-realistic SR results with small computational cost.

[222] GroupCoOp: Group-robust Fine-tuning via Group Prompt Learning

Nayeong Kim,Seong Joon Oh,Suha Kwak

Main category: cs.CV

TL;DR: GroupCoOp通过组特定提示学习增强视觉语言模型的组鲁棒性，解决了微调数据集中的子组不平衡问题，提升了少数群的辨识能力。

Details Motivation: 视觉语言模型（VLM）的微调易受数据集子组不平衡导致的虚假相关性影响，需要一种高效的方法增强模型的组鲁棒性。

Contribution: 提出了GroupCoOp，一种参数高效微调方法，通过组特定文本提示作为分类器，有效解决了组不平衡问题。

Method: 采用组特定文本提示作为组代表，利用VLM文本编码器的语义知识，即使在少数群样本较少时也能发现有效提示。

Result: 在五种CLIP架构的五个基准测试中表现最佳，偶尔优于全网络微调方法，仅训练了0.016%的参数。

Insight: 组特定提示能够有效捕捉少数群特征，缓解了嵌入空间的分布分散问题，提升了模型的鲁棒性。

Abstract: Parameter-efficient fine-tuning (PEFT) of vision-language models (VLMs) excels in various vision tasks thanks to the rich knowledge and generalization ability of VLMs. However, recent studies revealed that such fine-tuned VLMs are vulnerable to spurious correlations stemming from the subgroup imbalance in the fine-tuning datasets. To resolve this issue, we propose Group Context Optimization (GroupCoOp), a simple and effective debiased fine-tuning algorithm that enhances the group robustness of fine-tuned VLMs. Its key idea is to employ group-specific text prompts as group representatives serving as multiple classifiers for their target class. The rich semantic knowledge of the text encoder of VLM enables the discovery of effective group prompts even for groups with a small number of training samples. Leveraging the group prompts for each class addresses the issues caused by the group-imbalanced training set, such as the neglect of minority groups and the scattered distribution of each class in the embedding space. GroupCoOp achieved the best results on five benchmarks across five CLIP architectures and occasionally outperformed prior methods that fine-tune the entire network, despite training only 0.016% of the network’s parameters.

[223] A Multi-Camera Vision-Based Approach for Fine-Grained Assembly Quality Control

Ali Nazeri,Shashank Mishra,Achim Wagner,Martin Ruskowski,Didier Stricker,Jason Rambach

Main category: cs.CV

TL;DR: 这篇论文提出了一种基于多摄像头的细粒度装配质量控制方法，通过多视角图像融合解决单视角检测的局限性,显著提高了检测精度。

Details Motivation: 现有质量控制方法通常依赖单视角成像或人工检查，易受遮挡、视角限制或光照不一致的影响，导致错误率较高。增加检测站点不仅成本高,还可能中断生产线。

Contribution: 1) 开发了多摄像头视觉系统，结合多视角图像融合方法；2) 构建了一个包含多样化场景的标注数据集；3) 显著提升了小零件装配质量检查的精确率和召回率。

Method: 采用三摄像头系统覆盖装配过程的多角度视图，设计了一种专用的图像融合方法整合多视角检测结果，消除模糊性。

Result: 实验表明，该方法在检测未正确紧固的小零件（如螺丝）时，明显优于单视角方法，且在高精度和召回率上表现突出。

Insight: 多视角融合能有效解决单视角检测的固有缺陷，为工业自动化提供了一种可扩展且高精度的质量控制方案。

Abstract: Quality control is a critical aspect of manufacturing, particularly in ensuring the proper assembly of small components in production lines. Existing solutions often rely on single-view imaging or manual inspection, which are prone to errors due to occlusions, restricted perspectives, or lighting inconsistencies. These limitations require the installation of additional inspection stations, which could disrupt the assembly line and lead to increased downtime and costs. This paper introduces a novel multi-view quality control module designed to address these challenges, integrating a multi-camera imaging system with advanced object detection algorithms. By capturing images from three camera views, the system provides comprehensive visual coverage of components of an assembly process. A tailored image fusion methodology combines results from multiple views, effectively resolving ambiguities and enhancing detection reliability. To support this system, we developed a unique dataset comprising annotated images across diverse scenarios, including varied lighting conditions, occlusions, and angles, to enhance applicability in real-world manufacturing environments. Experimental results show that our approach significantly outperforms single-view methods, achieving high precision and recall rates in the identification of improperly fastened small assembly parts such as screws. This work contributes to industrial automation by overcoming single-view limitations, and providing a scalable, cost-effective, and accurate quality control mechanism that ensures the reliability and safety of the assembly line. The dataset used in this study is publicly available to facilitate further research in this domain.

[224] Assessing Visual Privacy Risks in Multimodal AI: A Novel Taxonomy-Grounded Evaluation of Vision-Language Models

Efthymios Tsaprazlis,Tiantian Feng,Anil Ramakrishna,Rahul Gupta,Shrikanth Narayanan

Main category: cs.CV

TL;DR: 这篇论文通过引入一个全面的视觉隐私分类法，评估了现有视觉语言模型在理解上下文隐私方面的能力，揭示了其不一致性，并呼吁需要更具隐私意识的AI系统。

Details Motivation: 随着大型语言模型（LLMs）和视觉语言模型（VLMs）的发展，AI在理解和整合视觉与语言方面的能力显著提升。然而，这些模型在隐私原则理解和执行方面存在显著不足，缺乏相关的评估资源。本文旨在填补这一空白。

Contribution: 1. 提出了一个全面、多层次的视觉隐私分类法；2. 评估了现有VLMs在隐私理解方面的不一致性；3. 为未来研究提供了基础分类和基准测试。

Method: 1. 设计了一个可扩展的视觉隐私分类法；2. 对多种先进的VLMs进行了隐私理解能力的评估；3. 结合法律框架分析模型的隐私能力。

Result: 研究发现VLMs在上下文隐私理解上存在显著不一致性，揭示了现有系统的局限性。

Insight: AI系统亟需更强大的隐私意识能力，未来的研究应侧重于隐私原则的整合和评估。

Abstract: Artificial Intelligence have profoundly transformed the technological landscape in recent years. Large Language Models (LLMs) have demonstrated impressive abilities in reasoning, text comprehension, contextual pattern recognition, and integrating language with visual understanding. While these advances offer significant benefits, they also reveal critical limitations in the models’ ability to grasp the notion of privacy. There is hence substantial interest in determining if and how these models can understand and enforce privacy principles, particularly given the lack of supporting resources to test such a task. In this work, we address these challenges by examining how legal frameworks can inform the capabilities of these emerging technologies. To this end, we introduce a comprehensive, multi-level Visual Privacy Taxonomy that captures a wide range of privacy issues, designed to be scalable and adaptable to existing and future research needs. Furthermore, we evaluate the capabilities of several state-of-the-art Vision-Language Models (VLMs), revealing significant inconsistencies in their understanding of contextual privacy. Our work contributes both a foundational taxonomy for future research and a critical benchmark of current model limitations, demonstrating the urgent need for more robust, privacy-aware AI systems.

[225] Uni4D-LLM: A Unified SpatioTemporal-Aware VLM for 4D Understanding and Generation

Hanyu Zhou,Gim Hee Lee

Main category: cs.CV

TL;DR: Uni4D-LLM是一个统一的时空感知视觉语言模型，首次实现了4D场景理解和生成任务的联合处理，通过共享表征和架构设计，取得了与SOTA模型相媲美的性能。

Details Motivation: 现有的3D和4D方法通常在理解任务中使用自回归模型，在生成任务中使用扩散模型，导致任务间的范式差异，难以统一处理动态4D场景中的时空建模需求。

Contribution: 提出了首个统一的时空感知VLM框架Uni4D-LLM，能够联合处理4D场景理解和生成任务，并在多个基准测试中表现出色。

Method: 通过共享表征（语义特征和噪声注入的外观特征）和共享架构（基于Transformer的LLM），结合自适应交叉注意力融合4D几何线索，实现了任务统一。

Result: 实验表明，Uni4D-LLM在多个任务上达到了与SOTA模型相当或更优的效果，首次真正统一了4D场景的理解和生成。

Insight: 统一的表征和架构设计是实现多任务联合处理的关键，时空建模的动态性是4D场景理解与生成的核心挑战。

Abstract: Vision-language models (VLMs) have demonstrated strong performance in 2D scene understanding and generation, but extending this unification to the physical world remains an open challenge. Existing 3D and 4D approaches typically embed scene geometry into autoregressive model for semantic understanding and diffusion model for content generation. This paradigm gap prevents a single model from jointly handling both tasks, especially in dynamic 4D settings where spatiotemporal modeling is critical. We propose Uni4D-LLM, the first unified VLM framework with spatiotemporal awareness for 4D scene understanding and generation. Our design is guided by two key insights: 1) Unification requires a shared representation. We extract semantic features for understanding and noisy-injected appearance features for generation, incorporate 4D geometric cues, and fuse them into a spatiotemporal-aware visual representation through adaptive cross-attention. 2) Unification requires a shared architecture. Both autoregression and diffusion are built on Transformer backbones, and this enables integration into a single LLM with task-specific heads. By aligning visual and linguistic representations, our Uni4D-LLM produces predictions for both understanding and generation within one Transformer-based framework. We further apply instruction fine-tuning on diverse 4D vision-language datasets to improve generalization across tasks. Extensive experiments on multiple benchmarks demonstrate that Uni4D-LLM achieves competitive or superior results compared to state-of-the-art models and offers the first true unification of 4D scene understanding and generation.

[226] 2nd Place Report of MOSEv2 Challenge 2025: Concept Guided Video Object Segmentation via SeC

Zhixiong Zhang,Shuangrui Ding,Xiaoyi Dong,Yuhang Zang,Yuhang Cao,Jiaqi Wang

Main category: cs.CV

TL;DR: 本文介绍了在MOSEv2 Challenge 2025中获得第二名的SeC方法，通过引入大型视觉语言模型（LVLM）提升视频目标分割的语义理解能力，显著提高了对复杂场景的鲁棒性。

Details Motivation: 传统半监督视频目标分割方法过度依赖外观匹配，难以应对剧烈视觉变化、遮挡和场景切换等挑战。需要一种基于高层语义理解的方法来提高分割的持久性。

Contribution: 提出了Segment Concept（SeC）框架，利用大型视觉语言模型（LVLM）对目标进行深层语义理解，显著提升了分割性能，并在MOSEv2数据集上实现了零样本迁移，获得第二名。

Method: 采用大型视觉语言模型（LVLM）来建立目标的语义表示，结合视频分割任务实现概念引导的分割。该方法无需微调训练数据，直接迁移至MOSEv2数据集。

Result: 在MOSEv2测试集上取得了39.7的JFn分数，位列Complex VOS赛道第二名，验证了其零样本性能和鲁棒性。

Insight: 深层语义理解可以显著提升视频目标分割对复杂场景的适应性，大型视觉语言模型在零样本迁移任务中表现出色。

Abstract: Semi-supervised Video Object Segmentation aims to segment a specified target throughout a video sequence, initialized by a first-frame mask. Previous methods rely heavily on appearance-based pattern matching and thus exhibit limited robustness against challenges such as drastic visual changes, occlusions, and scene shifts. This failure is often attributed to a lack of high-level conceptual understanding of the target. The recently proposed Segment Concept (SeC) framework mitigated this limitation by using a Large Vision-Language Model (LVLM) to establish a deep semantic understanding of the object for more persistent segmentation. In this work, we evaluate its zero-shot performance on the challenging coMplex video Object SEgmentation v2 (MOSEv2) dataset. Without any fine-tuning on the training set, SeC achieved 39.7 \JFn on the test set and ranked 2nd place in the Complex VOS track of the 7th Large-scale Video Object Segmentation Challenge.

[227] CE-FAM: Concept-Based Explanation via Fusion of Activation Maps

Michihiro Kuroki,Toshihiko Yamasaki

Main category: cs.CV

TL;DR: CE-FAM是一种基于概念的解释方法，通过融合激活图揭示图像分类器的学习概念、相关区域及其贡献，无需标注数据集。

Details Motivation: 现有显著性图仅能高亮重要区域，而基于概念的解释需要揭示概念区域及其贡献。CE-FAM旨在填补这一空白。

Contribution: 1. 提出CE-FAM框架，结合VLM知识识别概念区域及其贡献；2. 引入新的评估指标验证概念区域的准确性。

Method: 使用分支网络共享图像分类器的激活图，模拟VLM嵌入，通过梯度加权融合激活图表示概念区域，量化其对预测的贡献。

Result: CE-FAM在定性和定量评估中优于现有方法，且在零样本推理中表现优异。

Insight: CE-FAM通过融合VLM知识实现了无需标注数据的概念解释，为模型可解释性提供了新思路。

Abstract: Although saliency maps can highlight important regions to explain the reasoning behind image classification in artificial intelligence (AI), the meaning of these regions is left to the user’s interpretation. In contrast, conceptbased explanations decompose AI predictions into humanunderstandable concepts, clarifying their contributions. However, few methods can simultaneously reveal what concepts an image classifier learns, which regions are associated with them, and how they contribute to predictions. We propose a novel concept-based explanation method, Concept-based Explanation via Fusion of Activation Maps (CE-FAM). It employs a branched network that shares activation maps with an image classifier and learns to mimic the embeddings of a Vision and Language Model (VLM). The branch network predicts concepts in an image, and their corresponding regions are represented by a weighted sum of activation maps, with weights given by the gradients of the concept prediction scores. Their contributions are quantified based on their impact on the image classification score. Our method provides a general framework for identifying the concept regions and their contributions while leveraging VLM knowledge to handle arbitrary concepts without requiring an annotated dataset. Furthermore, we introduce a novel evaluation metric to assess the accuracy of the concept regions. Our qualitative and quantitative evaluations demonstrate our method outperforms existing approaches and excels in zero-shot inference for unseen concepts.

[228] FairViT-GAN: A Hybrid Vision Transformer with Adversarial Debiasing for Fair and Explainable Facial Beauty Prediction

Djamel Eddine Boukhari

Main category: cs.CV

TL;DR: FairViT-GAN结合CNN和ViT的优势，提出了一种混合架构，通过对抗性去偏技术减少算法偏见，并在SCUT-FBP5500基准测试中实现了高准确性和公平性。

Details Motivation: 现有面部美感预测模型面临架构限制、人口统计偏见和缺乏透明性问题，需要一种更公平、透明的解决方案。

Contribution: 提出了FairViT-GAN框架，结合CNN和ViT的混合架构，并引入对抗性去偏机制以提升公平性和透明性。

Method: 使用CNN分支提取局部特征，ViT分支建模全局上下文，并通过对抗训练去偏保护属性。

Result: 在SCUT-FBP5500上，Pearson相关性达0.9230，RMSE降至0.2650，性能差距减少82.9%。

Insight: 混合架构和对抗去偏可显著提升模型的公平性和解释性，适用于主观视觉任务的负责任AI开发。

Abstract: Facial Beauty Prediction (FBP) has made significant strides with the application of deep learning, yet state-of-the-art models often exhibit critical limitations, including architectural constraints, inherent demographic biases, and a lack of transparency. Existing methods, primarily based on Convolutional Neural Networks (CNNs), excel at capturing local texture but struggle with global facial harmony, while Vision Transformers (ViTs) effectively model long-range dependencies but can miss fine-grained details. Furthermore, models trained on benchmark datasets can inadvertently learn and perpetuate societal biases related to protected attributes like ethnicity. To address these interconnected challenges, we propose \textbf{FairViT-GAN}, a novel hybrid framework that synergistically integrates a CNN branch for local feature extraction and a ViT branch for global context modeling. More significantly, we introduce an adversarial debiasing mechanism where the feature extractor is explicitly trained to produce representations that are invariant to protected attributes, thereby actively mitigating algorithmic bias. Our framework’s transparency is enhanced by visualizing the distinct focus of each architectural branch. Extensive experiments on the SCUT-FBP5500 benchmark demonstrate that FairViT-GAN not only sets a new state-of-the-art in predictive accuracy, achieving a Pearson Correlation of \textbf{0.9230} and reducing RMSE to \textbf{0.2650}, but also excels in fairness. Our analysis reveals a remarkable \textbf{82.9% reduction in the performance gap} between ethnic subgroups, with the adversary’s classification accuracy dropping to near-random chance (52.1%). We believe FairViT-GAN provides a robust, transparent, and significantly fairer blueprint for developing responsible AI systems for subjective visual assessment.

[229] Sim-DETR: Unlock DETR for Temporal Sentence Grounding

Jiajin Tang,Zhengxuan Wei,Yuchen Zhu,Cheng Shi,Guanbin Li,Liang Lin,Sibei Yang

Main category: cs.CV

TL;DR: Sim-DETR通过改进DETR的解码器层，解决了在时序语句定位任务中DETR性能不佳的问题，提出了语义与位置重叠的自注意力约束和查询-帧对齐机制。

Details Motivation: 研究发现，现有的DETR增强策略不仅未能在时序语句定位任务中提升性能，反而可能降低表现。这源于相似目标时刻查询之间的冲突，以及全局语义与局部定位的内部冲突。

Contribution: Sim-DETR提出两种改进方法：基于语义和位置重叠的自注意力约束，以及查询-帧对齐机制，显著提升了DETR在时序语句定位任务中的性能。

Method: Sim-DETR扩展了标准DETR的解码器层，引入自注意力约束机制以减少查询冲突，并通过查询-帧对齐增强全局与局部上下文的桥接。

Result: 实验表明，Sim-DETR充分发挥了DETR的潜力，为时序语句定位任务提供了强有力的基线模型。

Insight: 时序语句定位任务中，DETR的性能瓶颈主要源于查询间的冲突和上下文整合问题，Sim-DETR通过针对性的改进成功解决了这些问题。

Abstract: Temporal sentence grounding aims to identify exact moments in a video that correspond to a given textual query, typically addressed with detection transformer (DETR) solutions. However, we find that typical strategies designed to enhance DETR do not improve, and may even degrade, its performance in this task. We systematically analyze and identify the root causes of this abnormal behavior: (1) conflicts between queries from similar target moments and (2) internal query conflicts due to the tension between global semantics and local localization. Building on these insights, we propose a simple yet powerful baseline, Sim-DETR, which extends the standard DETR with two minor modifications in the decoder layers: (1) constraining self-attention between queries based on their semantic and positional overlap and (2) adding query-to-frame alignment to bridge the global and local contexts. Experiments demonstrate that Sim-DETR unlocks the full potential of DETR for temporal sentence grounding, offering a strong baseline for future research.

[230] Not All Tokens are Guided Equal: Improving Guidance in Visual Autoregressive Models

Ky Dan Nguyen,Hoang Lam Tran,Anh-Dung Dinh,Daochang Liu,Weidong Cai,Xiuying Wang,Chang Xu

Main category: cs.CV

TL;DR: 论文提出了一种名为信息接地引导（IGG）的新机制，用于解决自回归模型中因渐进分辨率缩放导致的信息不一致问题，从而生成更清晰、更连贯的图像。

Details Motivation: 自回归模型在图像生成中表现出色，但渐进分辨率缩放会导致跨时间步长的信息不一致，进而分散引导信号，影响生成质量。

Contribution: 引入了信息接地引导（IGG），通过注意力机制将引导信号锚定到语义重要区域，显著提升了生成图像的清晰度和语义一致性。

Method: IGG利用注意力机制自适应地强化信息丰富的图像块，确保引导信号与生成内容紧密对齐。

Result: 在类条件生成和文本到图像生成任务中，IGG显著提升了图像质量，为自回归模型设定了新的基准。

Insight: 通过锚定引导信号到关键区域，可以有效缓解信息不一致问题，提升自回归模型的生成质量。

Abstract: Autoregressive (AR) models based on next-scale prediction are rapidly emerging as a powerful tool for image generation, but they face a critical weakness: information inconsistencies between patches across timesteps introduced by progressive resolution scaling. These inconsistencies scatter guidance signals, causing them to drift away from conditioning information and leaving behind ambiguous, unfaithful features. We tackle this challenge with Information-Grounding Guidance (IGG), a novel mechanism that anchors guidance to semantically important regions through attention. By adaptively reinforcing informative patches during sampling, IGG ensures that guidance and content remain tightly aligned. Across both class-conditioned and text-to-image generation tasks, IGG delivers sharper, more coherent, and semantically grounded images, setting a new benchmark for AR-based methods.

[231] PCRI: Measuring Context Robustness in Multimodal Models for Enterprise Applications

Hitesh Laxmichand Patel,Amit Agarwal,Srikant Panda,Hansa Meghwani,Karan Dua,Paul Li,Tao Sheng,Sujith Ravi,Dan Roth

Main category: cs.CV

TL;DR: 该论文提出了Patch Context Robustness Index (PCRI)，一种系统性且可解释的评分方法，用于量化多模态大语言模型（MLLMs）对视觉上下文变化的稳健性，填补了现有评估指标的空白。

Details Motivation: 现实场景中，多模态模型的可靠性常因对无关或干扰性视觉上下文的敏感性而受到影响，而现有评估指标未涵盖这一方面。

Contribution: 提出了PCRI方法，首次系统性地衡量MLLMs在不同视觉上下文粒度下的表现变化，并对19个先进模型进行了评估。

Method: 通过对比局部图像块和完整图像输入的模型性能变化，计算PCRI分数，量化模型对视觉上下文的稳健性。

Result: 研究发现大多数顶尖模型对背景噪声仍表现脆弱，仅少数模型（如InternVL2-26B和Qwen2VL-72B）展现出跨任务的稳健性。

Insight: PCRI不仅提供了模型比较的工具，还揭示了不同模型架构对视觉上下文的处理方式，为未来模型设计和训练策略提供了指导。

Abstract: The reliability of Multimodal Large Language Models (MLLMs) in real-world settings is often undermined by sensitivity to irrelevant or distracting visual context, an aspect not captured by existing evaluation metrics. We introduce the \textbf{Patch Context Robustness Index (PCRI)}, the first systematic and interpretable score for quantifying MLLM robustness to variations in visual context granularity, measuring performance changes between localized image patches and full-image input. Applying PCRI to 19 state-of-the-art MLLMs across 15 vision-language benchmarks, we find that most leading models remain brittle to background noise, with only a few, such as InternVL2-26B and Qwen2VL-72B, demonstrating consistent robustness across tasks. PCRI analysis also highlights how different model architectures handle and integrate visual context, offering actionable diagnostic insight for both researchers and practitioners. PCRI enables rigorous comparison of context robustness, supporting principled model selection and guiding the development of future architectures and training strategies for robust, real-world deployment.

[232] Learning Adaptive Pseudo-Label Selection for Semi-Supervised 3D Object Detection

Taehun Kong,Tae-Kyun Kim

Main category: cs.CV

TL;DR: 该论文提出了一种可学习的伪标签选择模块，用于半监督3D目标检测，通过自适应阈值和评分融合提高伪标签质量，显著提升了性能。

Details Motivation: 半监督3D目标检测需要减少昂贵的3D标注依赖，现有方法依赖手动阈值或动态阈值选择伪标签，忽略了上下文信息，导致伪标签质量不高。

Contribution: 1. 提出可学习的伪标签选择模块；2. 引入评分融合和上下文自适应阈值；3. 设计了软监督策略以抗噪声。

Method: 1. 在教师网络输出层引入两个网络，用于评分融合和自适应阈值选择；2. 通过伪标签与真实边界框的对齐监督阈值学习；3. 软监督策略优先选择高质量伪标签。

Result: 在KITTI和Waymo数据集上表现出色，伪标签选择精度高且覆盖范围广，显著提升了现有方法的性能。

Insight: 自适应阈值和评分融合能有效结合上下文信息提升伪标签质量，软监督策略对噪声鲁棒性强。

Abstract: Semi-supervised 3D object detection (SS3DOD) aims to reduce costly 3D annotations utilizing unlabeled data. Recent studies adopt pseudo-label-based teacher-student frameworks and demonstrate impressive performance. The main challenge of these frameworks is in selecting high-quality pseudo-labels from the teacher’s predictions. Most previous methods, however, select pseudo-labels by comparing confidence scores over thresholds manually set. The latest works tackle the challenge either by dynamic thresholding or refining the quality of pseudo-labels. Such methods still overlook contextual information e.g. object distances, classes, and learning states, and inadequately assess the pseudo-label quality using partial information available from the networks. In this work, we propose a novel SS3DOD framework featuring a learnable pseudo-labeling module designed to automatically and adaptively select high-quality pseudo-labels. Our approach introduces two networks at the teacher output level. These networks reliably assess the quality of pseudo-labels by the score fusion and determine context-adaptive thresholds, which are supervised by the alignment of pseudo-labels over GT bounding boxes. Additionally, we introduce a soft supervision strategy that can learn robustly under pseudo-label noises. This helps the student network prioritize cleaner labels over noisy ones in semi-supervised learning. Extensive experiments on the KITTI and Waymo datasets demonstrate the effectiveness of our method. The proposed method selects high-precision pseudo-labels while maintaining a wider coverage of contexts and a higher recall rate, significantly improving relevant SS3DOD methods.

[233] Tunable-Generalization Diffusion Powered by Self-Supervised Contextual Sub-Data for Low-Dose CT Reconstruction

Guoquan Wei,Zekun Zhou,Liu Shi,Wenzhe Shan,Qiegen Liu

Main category: cs.CV

TL;DR: 论文提出一种名为SuperDiff的新方法，结合自监督上下文子数据和可调泛化扩散模型，用于低剂量CT重建，显著提升了重建性能和泛化能力。

Details Motivation: 当前基于深度学习的低剂量CT去噪模型严重依赖配对数据且泛化能力差，扩散模型也需要学习干净数据的分布，难以满足临床应用需求。自监督方法在剂量扩展时面临模型泛化能力下降的问题。

Contribution: 提出SuperDiff方法，结合上下文子数据相似性自适应感知策略和潜在扩散模型，通过知识蒸馏和像素级自校正融合技术提升重建性能和泛化能力。

Method: 设计双域策略级联：投影域的上下文子数据相似性自适应感知策略提供初始先验，结合潜在扩散模型优化图像细节，并使用像素级自校正融合技术增强图像保真度。

Result: 在数据集和真实数据上的实验表明，SuperDiff在重建和泛化性能上均优于现有最先进方法。

Insight: 通过自监督学习和灵活的双域策略，SuperDiff能够在仅需低剂量CT投影域数据的情况下实现高性能重建，同时支持不同剂量甚至未见剂量的泛化。

Abstract: Current models based on deep learning for low-dose CT denoising rely heavily on paired data and generalize poorly. Even the more concerned diffusion models need to learn the distribution of clean data for reconstruction, which is difficult to satisfy in medical clinical applications. At the same time, self-supervised-based methods face the challenge of significant degradation of generalizability of models pre-trained for the current dose to expand to other doses. To address these issues, this paper proposes a novel method of tunable-generalization diffusion powered by self-supervised contextual sub-data for low-dose CT reconstruction, named SuperDiff. Firstly, a contextual subdata similarity adaptive sensing strategy is designed for denoising centered on the LDCT projection domain, which provides an initial prior for the subsequent progress. Subsequently, the initial prior is used to combine knowledge distillation with a deep combination of latent diffusion models for optimizing image details. The pre-trained model is used for inference reconstruction, and the pixel-level self-correcting fusion technique is proposed for fine-grained reconstruction of the image domain to enhance the image fidelity, using the initial prior and the LDCT image as a guide. In addition, the technique is flexibly applied to the generalization of upper and lower doses or even unseen doses. Dual-domain strategy cascade for self-supervised LDCT denoising, SuperDiff requires only LDCT projection domain data for training and testing. Full qualitative and quantitative evaluations on both datasets and real data show that SuperDiff consistently outperforms existing state-of-the-art methods in terms of reconstruction and generalization performance.

[234] AssemblyHands-X: Modeling 3D Hand-Body Coordination for Understanding Bimanual Human Activities

Tatsuro Banno,Takehiko Ohkawa,Ruicong Liu,Ryosuke Furuta,Yoichi Sato

Main category: cs.CV

TL;DR: AssemblyHands-X 是首个无标记的3D手-身体基准数据集，专注于双手活动中的手-身体协调性对动作识别的影响。通过多视角三角测量和 SMPL-X 网格拟合进行3D姿态标注，实验表明基于姿态的动作推断优于视频基线，且手-身体联合建模能提升识别性能。

Details Motivation: 现有数据集缺乏对双手活动中手-身体协调性的系统评估，且标记运动捕捉技术限制了模型的泛化能力。

Contribution: 提出了首个无标记的3D手-身体基准数据集 AssemblyHands-X，并通过实验验证了手-身体联合建模的重要性。

Method: 采用多视角三角测量与 SMPL-X 网格拟合相结合的方法进行3D姿态标注，并比较了不同输入表示对动作识别的影响。

Result: 基于姿态的动作推断比视频基线更高效和准确，手-身体联合建模显著提升了动作识别性能。

Insight: 手-身体的动态协调性是全面理解双手活动的关键因素。

Abstract: Bimanual human activities inherently involve coordinated movements of both hands and body. However, the impact of this coordination in activity understanding has not been systematically evaluated due to the lack of suitable datasets. Such evaluation demands kinematic-level annotations (e.g., 3D pose) for the hands and body, yet existing 3D activity datasets typically annotate either hand or body pose. Another line of work employs marker-based motion capture to provide full-body pose, but the physical markers introduce visual artifacts, thereby limiting models’ generalization to natural, markerless videos. To address these limitations, we present AssemblyHands-X, the first markerless 3D hand-body benchmark for bimanual activities, designed to study the effect of hand-body coordination for action recognition. We begin by constructing a pipeline for 3D pose annotation from synchronized multi-view videos. Our approach combines multi-view triangulation with SMPL-X mesh fitting, yielding reliable 3D registration of hands and upper body. We then validate different input representations (e.g., video, hand pose, body pose, or hand-body pose) across recent action recognition models based on graph convolution or spatio-temporal attention. Our extensive experiments show that pose-based action inference is more efficient and accurate than video baselines. Moreover, joint modeling of hand and body cues improves action recognition over using hands or upper body alone, highlighting the importance of modeling interdependent hand-body dynamics for a holistic understanding of bimanual activities.

[235] Preserving Cross-Modal Stability for Visual Unlearning in Multimodal Scenarios

Jinghan Xu Yuyang Zhang Qixuan Cai Jiancheng Chen Keqiu Li

Main category: cs.CV

TL;DR: 该论文提出了一种跨模态对比遗忘（CCU）框架，解决了视觉遗忘中跨模态知识损失和类内结构不稳定的问题，通过选择性视觉遗忘、跨模态知识保留和双集对比分离，显著提升了模型性能和时间效率。

Details Motivation: 在自动驾驶等跨模态应用中，视觉模态是最易泄露隐私的。传统机器学习遗忘方法在视觉遗忘时会破坏跨模态知识和类内结构稳定性，导致性能下降。

Contribution: 提出了CCU框架，包含选择性视觉遗忘、跨模态知识保留和双集对比分离三个核心组件，解决了跨模态稳定性和类内结构保持的问题。

Method: 采用逆对比学习分离视觉表征和语义，通过语义一致性保留其他模态的判别性，利用双集对比分离隔离未学习和保留集的结构扰动。

Result: 在三个数据集上的实验证明，CCU方法比基线提升了7.12%的准确率，且仅需基线7%的遗忘时间。

Insight: 在视觉遗忘任务中，保持跨模态知识和类内结构稳定性是关键，CCU通过对比学习和语义一致性实现了高效遗忘与性能稳定。

Abstract: Visual modality is the most vulnerable to privacy leakage in real-world multimodal applications like autonomous driving with visual and radar data; Machine unlearning removes specific training data from pre-trained models to address privacy leakage, however, existing methods fail to preserve cross-modal knowledge and maintain intra-class structural stability of retain data, leading to reduced overall and other modalities’ performance during visual unlearning; to address these challenges, we propose a Cross-modal Contrastive Unlearning (CCU) framework, which integrates three key components: (a) selective visual unlearning: employing inverse contrastive learning to dissociate visual representations from their original semantics, (b) cross-modal knowledge retention: preserving other modalities’ discriminability through semantic consistency, and (c) dual-set contrastive separation: preserving the model performance via isolation of structural perturbations between the unlearn set and retain set; extensive experiments on three datasets demonstrate the superiority of CCU, and our method achieves a 7.12% accuracy improvement with only 7% of the unlearning time compared to the top-accuracy baseline.

[236] Q-FSRU: Quantum-Augmented Frequency-Spectral For Medical Visual Question Answering

Rakesh Thakur,Yusra Tariq,Rakesh Chandra Joshi

Main category: cs.CV

TL;DR: Q-FSRU是一种结合频率谱表示融合（FSRU）和量子检索增强生成（Quantum RAG）的新模型，用于医学视觉问答（VQA）。该方法通过FFT将医学图像和文本特征转换到频域，提升信息处理效率，并通过量子检索技术从外部知识源获取相关医学事实，增强了模型的推理能力和可解释性。在VQA-RAD数据集上的实验表明，Q-FSRU超越了现有模型，尤其在复杂问题上表现突出。

Details Motivation: 医学VQA任务需要同时理解图像和文本信息，这在医疗AI中仍是一个重大挑战。传统方法在处理噪声和复杂推理时表现不佳，因此需要一种更高效且可解释的方法。

Contribution: 1. 提出Q-FSRU模型，结合频率谱表示和量子检索增强技术。2. 通过FFT将数据转换到频域，过滤噪声并聚焦关键信息。3. 引入量子检索系统，增强对复杂医学问题的推理能力。4. 在VQA-RAD数据集上验证了模型的优越性和可解释性。

Method: 1. 使用FSRU模块对医学图像和文本进行频率谱表示，利用FFT提取频域特征。2. 结合Quantum RAG技术，通过量子相似性检索外部医学知识。3. 将检索到的知识与频域特征融合，提升模型推理能力。

Result: Q-FSRU在VQA-RAD数据集上表现优于现有方法，尤其是在需要复杂图像文本推理的任务中。频率和量子信息的结合不仅提升了性能，还增强了模型的可解释性。

Insight: 1. 频域处理可以有效过滤噪声并突出关键信息。2. 量子检索技术能够高效关联外部知识，增强模型的推理能力。3. 融合频域和量子信息的方法为医疗AI的可解释性和性能提供了新思路。

Abstract: Solving tough clinical questions that require both image and text understanding is still a major challenge in healthcare AI. In this work, we propose Q-FSRU, a new model that combines Frequency Spectrum Representation and Fusion (FSRU) with a method called Quantum Retrieval-Augmented Generation (Quantum RAG) for medical Visual Question Answering (VQA). The model takes in features from medical images and related text, then shifts them into the frequency domain using Fast Fourier Transform (FFT). This helps it focus on more meaningful data and filter out noise or less useful information. To improve accuracy and ensure that answers are based on real knowledge, we add a quantum inspired retrieval system. It fetches useful medical facts from external sources using quantum-based similarity techniques. These details are then merged with the frequency-based features for stronger reasoning. We evaluated our model using the VQA-RAD dataset, which includes real radiology images and questions. The results showed that Q-FSRU outperforms earlier models, especially on complex cases needing image text reasoning. The mix of frequency and quantum information improves both performance and explainability. Overall, this approach offers a promising way to build smart, clear, and helpful AI tools for doctors.

[237] LifeCLEF Plant Identification Task 2014

Herve Goeau,Alexis Joly,Pierre Bonnet,Souheil Selmi,Jean-Francois Molino,Daniel Barthelemy,Nozha Boujemaa

Main category: cs.CV

TL;DR: LifeCLEF 2014植物识别任务提供了一个系统化的评估平台，专注于500种树木和草本植物的识别，涵盖了7种图像类型。任务数据通过公民科学项目收集，强调了现实世界的应用性。

Details Motivation: 任务是评估植物识别系统在多样化、非约束条件下的性能，同时推动生物多样性和植物学研究。

Contribution: 提供了多样化的植物图像数据集，促进了多模态和多视角植物识别方法的发展，并展示了社区的兴趣和挑战。

Method: 参与者采用了多种图像检索方法，具体方法在论文中未详细描述，但强调了数据的多样性和未约束条件下的挑战。

Result: 共27个提交结果，来自10个小组，显示了方法的多样性和任务的实际应用潜力。

Insight: 公民科学项目为数据收集提供了新途径，强调了现实世界应用的复杂性，为未来研究提供了挑战和方向。

Abstract: The LifeCLEFs plant identification task provides a testbed for a system-oriented evaluation of plant identification about 500 species trees and herbaceous plants. Seven types of image content are considered: scan and scan-like pictures of leaf, and 6 kinds of detailed views with unconstrained conditions, directly photographed on the plant: flower, fruit, stem & bark, branch, leaf and entire view. The main originality of this data is that it was specifically built through a citizen sciences initiative conducted by Tela Botanica, a French social network of amateur and expert botanists. This makes the task closer to the conditions of a real-world application. This overview presents more precisely the resources and assessments of task, summarizes the retrieval approaches employed by the participating groups, and provides an analysis of the main evaluation results. With a total of ten groups from six countries and with a total of twenty seven submitted runs, involving distinct and original methods, this fourth year task confirms Image & Multimedia Retrieval community interest for biodiversity and botany, and highlights further challenging studies in plant identification.

[238] EWC-Guided Diffusion Replay for Exemplar-Free Continual Learning in Medical Imaging

Anoushka Harit,William Prew,Zhongtian Sun,Florian Markowetz

Main category: cs.CV

TL;DR: 该论文提出了一种基于EWC（弹性权重巩固）和类条件扩散重放的持续学习方法，用于医学影像领域，避免了存储患者样本的需求，同时在多个任务上取得了优异的性能。

Details Motivation: 医学影像基础模型需要随时间适应新任务，但由于隐私和成本限制，无法进行完全重新训练。因此，需要一种高效且保护隐私的持续学习方法。

Contribution: 1. 提出了一种结合EWC和类条件扩散重放的持续学习框架；2. 展示了其在MedMNIST v2和CheXpert任务上的高效性和隐私保护能力；3. 分析了遗忘与重放保真度和Fisher加权参数漂移的关系。

Method: 使用紧凑的Vision Transformer作为主干网络，结合EWC和类条件扩散重放技术，避免存储旧任务样本，同时减少遗忘。

Result: 在CheXpert上达到0.851 AUROC，相比DER++减少30%的遗忘，接近联合训练的0.869 AUROC，同时保持高效和隐私保护。

Insight: 遗忘现象与重放保真度和Fisher加权参数漂移密切相关，这表明扩散重放和突触稳定性在持续学习中具有互补作用。

Abstract: Medical imaging foundation models must adapt over time, yet full retraining is often blocked by privacy constraints and cost. We present a continual learning framework that avoids storing patient exemplars by pairing class conditional diffusion replay with Elastic Weight Consolidation. Using a compact Vision Transformer backbone, we evaluate across eight MedMNIST v2 tasks and CheXpert. On CheXpert our approach attains 0.851 AUROC, reduces forgetting by more than 30% relative to DER\texttt{++}, and approaches joint training at 0.869 AUROC, while remaining efficient and privacy preserving. Analyses connect forgetting to two measurable factors: fidelity of replay and Fisher weighted parameter drift, highlighting the complementary roles of replay diffusion and synaptic stability. The results indicate a practical route for scalable, privacy aware continual adaptation of clinical imaging models.

[239] Adversarial Versus Federated: An Adversarial Learning based Multi-Modality Cross-Domain Federated Medical Segmentation

You Zhou,Lijiang Chen,Shuchang Lyu,Guangxia Cui,Wenpei Bai,Zheng Zhou,Meng Li,Guangliang Cheng,Huiyu Zhou,Qi Zhao

Main category: cs.CV

TL;DR: 提出了一种基于对抗学习的多模态跨领域联邦医疗分割框架FedDA，通过特征级对抗训练对齐不同客户端的特征，解决了联邦学习中多模态医疗数据分割的挑战。

Details Motivation: 联邦学习在医疗领域因数据隐私保护而流行，但不同客户端的医疗图像模态差异导致跨领域分割困难，需要一种高效方法来缓解领域偏移。

Contribution: 1) 提出FedDA框架，通过特征级对抗学习对齐不同客户端的特征；2) 增强了单模态客户端对多模态数据的处理能力；3) 在三个医疗数据集上验证了其鲁棒性。

Method: 在联邦学习中嵌入对抗训练机制，通过对齐客户端的特征图实现跨域特征优化，提升模型在多域的泛化能力。

Result: FedDA在多模态医疗数据集上表现优于现有联邦聚合算法，且通过主客观评估验证了其性能。

Insight: 特征级对抗训练能够有效缓解领域偏移问题，联邦学习结合对抗学习是多模态医疗数据分割的有效解决方案。

Abstract: Federated learning enables collaborative training of machine learning models among different clients while ensuring data privacy, emerging as the mainstream for breaking data silos in the healthcare domain. However, the imbalance of medical resources, data corruption or improper data preservation may lead to a situation where different clients possess medical images of different modality. This heterogeneity poses a significant challenge for cross-domain medical image segmentation within the federated learning framework. To address this challenge, we propose a new Federated Domain Adaptation (FedDA) segmentation training framework. Specifically, we propose a feature-level adversarial learning among clients by aligning feature maps across clients through embedding an adversarial training mechanism. This design can enhance the model’s generalization on multiple domains and alleviate the negative impact from domain-shift. Comprehensive experiments on three medical image datasets demonstrate that our proposed FedDA substantially achieves cross-domain federated aggregation, endowing single modality client with cross-modality processing capabilities, and consistently delivers robust performance compared to state-of-the-art federated aggregation algorithms in objective and subjective assessment. Our code are available at https://github.com/GGbond-study/FedDA.

[240] EditScore: Unlocking Online RL for Image Editing via High-Fidelity Reward Modeling

Xin Luo,Jiahao Wang,Chenyuan Wu,Shitao Xiao,Xiyan Jiang,Defu Lian,Jiajun Zhang,Dong Liu,Zheng liu

Main category: cs.CV

TL;DR: 该论文提出了一种名为EditScore的高保真奖励模型，用于指令引导的图像编辑任务。通过建立EditReward-Bench基准测试，开发了一系列奖励模型（7B-72B），并结合自集成策略，成功解锁了在线强化学习在图像编辑中的应用。

Details Motivation: 当前指令引导的图像编辑模型在处理复杂指令时表现不足，且通常需要多次尝试才能得到理想结果。强化学习虽然潜力巨大，但因缺乏高效、高保真的奖励信号而难以应用。

Contribution: 论文的主要贡献包括：1）建立EditReward-Bench基准测试；2）开发EditScore系列奖励模型；3）提出自集成策略以提升模型性能；4）展示高保真奖励模型如何解锁在线强化学习在图像编辑中的应用。

Method: 论文方法包括：1）通过数据筛选和精心策划训练EditScore模型；2）结合自集成策略优化生成性能；3）利用EditScore推动在线强化学习的策略优化。

Result: EditScore的最大变体在基准测试中甚至超过了GPT-5，并在OmniGen2基模型上实现了显著的性能提升。

Insight: 高保真、领域专用的奖励模型是解锁强化学习在图像编辑中潜力的关键，这为从基准测试到奖励建模再到强化学习训练提供了系统性路径。

Abstract: Instruction-guided image editing has achieved remarkable progress, yet current models still face challenges with complex instructions and often require multiple samples to produce a desired result. Reinforcement Learning (RL) offers a promising solution, but its adoption in image editing has been severely hindered by the lack of a high-fidelity, efficient reward signal. In this work, we present a comprehensive methodology to overcome this barrier, centered on the development of a state-of-the-art, specialized reward model. We first introduce EditReward-Bench, a comprehensive benchmark to systematically evaluate reward models on editing quality. Building on this benchmark, we develop EditScore, a series of reward models (7B-72B) for evaluating the quality of instruction-guided image editing. Through meticulous data curation and filtering, EditScore effectively matches the performance of learning proprietary VLMs. Furthermore, coupled with an effective self-ensemble strategy tailored for the generative nature of EditScore, our largest variant even surpasses GPT-5 in the benchmark. We then demonstrate that a high-fidelity reward model is the key to unlocking online RL for image editing. Our experiments show that, while even the largest open-source VLMs fail to provide an effective learning signal, EditScore enables efficient and robust policy optimization. Applying our framework to a strong base model, OmniGen2, results in a final model that shows a substantial and consistent performance uplift. Overall, this work provides the first systematic path from benchmarking to reward modeling to RL training in image editing, showing that a high-fidelity, domain-specialized reward model is the key to unlocking the full potential of RL in this domain.

[241] MoReact: Generating Reactive Motion from Textual Descriptions

Xiyan Xu,Sirui Xu,Yu-Xiong Wang,Liang-Yan Gui

Main category: cs.CV

TL;DR: MoReact是一种基于扩散模型的文本驱动方法，专注于生成与文本描述相符的人类反应动作序列，通过全局轨迹与局部动作的解耦生成，提升反应的适应性和真实性。

Details Motivation: 现有方法在生成人类反应动作时，通常将多人视为单一实体或仅依赖单方动作，忽略了交互语义信息的整合，导致适应性不足。MoReact旨在填补这一空白，通过文本驱动生成更符合动态交互场景的反应动作。

Contribution: 1. 提出MoReact，一种基于扩散模型的文本驱动方法，专注于生成语义匹配的响应动作；2. 采用全局轨迹与局部动作解耦生成策略，提升动作的适应性和一致性；3. 引入新颖的交互损失函数，增强近距离交互的真实性。

Method: MoReact采用扩散模型，通过两步生成策略：首先生成全局轨迹以指导动作方向，再生成局部动作细节。模型中整合了交互损失函数，优化生成的近距离交互动作的真实性。

Result: 实验表明，MoReact能生成多样、可控且符合文本描述的反应动作，与对手动作高度匹配，同时保持了语义一致性。

Insight: 全局轨迹的生成对局部动作的引导至关重要，而文本驱动的方法能够有效捕捉交互语义信息，为动态场景的反应生成提供了新思路。

Abstract: Modeling and generating human reactions poses a significant challenge with broad applications for computer vision and human-computer interaction. Existing methods either treat multiple individuals as a single entity, directly generating interactions, or rely solely on one person’s motion to generate the other’s reaction, failing to integrate the rich semantic information that underpins human interactions. Yet, these methods often fall short in adaptive responsiveness, i.e., the ability to accurately respond to diverse and dynamic interaction scenarios. Recognizing this gap, our work introduces an approach tailored to address the limitations of existing models by focusing on text-driven human reaction generation. Our model specifically generates realistic motion sequences for individuals that responding to the other’s actions based on a descriptive text of the interaction scenario. The goal is to produce motion sequences that not only complement the opponent’s movements but also semantically fit the described interactions. To achieve this, we present MoReact, a diffusion-based method designed to disentangle the generation of global trajectories and local motions sequentially. This approach stems from the observation that generating global trajectories first is crucial for guiding local motion, ensuring better alignment with given action and text. Furthermore, we introduce a novel interaction loss to enhance the realism of generated close interactions. Our experiments, utilizing data adapted from a two-person motion dataset, demonstrate the efficacy of our approach for this novel task, which is capable of producing realistic, diverse, and controllable reactions that not only closely match the movements of the counterpart but also adhere to the textual guidance. Please find our webpage at https://xiyan-xu.github.io/MoReactWebPage.

[242] Revisit the Imbalance Optimization in Multi-task Learning: An Experimental Analysis

Yihang Guo,Tianyuan Yu,Liang Bai,Yanming Guo,Yirun Ruan,William Li,Weishi Zheng

Main category: cs.CV

TL;DR: 论文系统性分析了多任务学习（MTL）中的不平衡优化问题，发现现有方法的性能在不同数据集上表现不一致，且高级架构仍依赖昂贵的网格搜索。研究表明白初始化视觉基础模型（VFMs）不能解决优化不平衡，而任务特定梯度的范数与优化不平衡强相关。

Details Motivation: 多任务学习中任务干扰导致性能下降（不平衡优化），阻碍其潜力。论文旨在通过实验分析揭示问题的根源并探索解决方案。

Contribution: 1. 系统性分析了MTL优化的不一致性；2. 发现梯度范数与优化不平衡的强相关性；3. 提出一种简单的梯度范数缩放方法，性能媲美昂贵的网格搜索。

Method: 通过实验分析现有优化方法和高级架构的性能差异，发现梯度范数与不平衡优化的关系，并提出梯度范数缩放策略。

Result: 提出的梯度范数缩放方法能有效解决不平衡优化问题，性能接近网格搜索，计算成本更低。

Insight: 理解和控制梯度动态是稳定MTL的直接路径，而非依赖复杂方法或更多数据。

Abstract: Multi-task learning (MTL) aims to build general-purpose vision systems by training a single network to perform multiple tasks jointly. While promising, its potential is often hindered by “unbalanced optimization”, where task interference leads to subpar performance compared to single-task models. To facilitate research in MTL, this paper presents a systematic experimental analysis to dissect the factors contributing to this persistent problem. Our investigation confirms that the performance of existing optimization methods varies inconsistently across datasets, and advanced architectures still rely on costly grid-searched loss weights. Furthermore, we show that while powerful Vision Foundation Models (VFMs) provide strong initialization, they do not inherently resolve the optimization imbalance, and merely increasing data quantity offers limited benefits. A crucial finding emerges from our analysis: a strong correlation exists between the optimization imbalance and the norm of task-specific gradients. We demonstrate that this insight is directly applicable, showing that a straightforward strategy of scaling task losses according to their gradient norms can achieve performance comparable to that of an extensive and computationally expensive grid search. Our comprehensive analysis suggests that understanding and controlling gradient dynamics is a more direct path to stable MTL than developing increasingly complex methods.

[243] Bridging the Task Gap: Multi-Task Adversarial Transferability in CLIP and Its Derivatives

Kuanrong Liu,Siyuan Liang,Cheng Qian,Ming Zhang,Xiaochun Cao

Main category: cs.CV

TL;DR: 这篇论文研究了CLIP及其衍生模型在多任务对抗扰动下的跨任务迁移行为，发现细粒度任务生成的对抗样本比粗粒度任务更具攻击性。作者提出了一种新框架MT-AdvCLIP，显著提高了对抗样本在多任务间的迁移成功率。

Details Motivation: CLIP虽然在图像文本对齐任务中表现优异，但在细粒度任务（如目标检测和语义分割）中表现不佳，且其对抗鲁棒性尚未充分研究。理解对抗样本在多任务间的迁移行为对评估CLIP的泛化能力和安全风险至关重要。

Contribution: 1. 系统分析了CLIP模型在多任务对抗扰动下的迁移行为；2. 提出MT-AdvCLIP框架，通过任务感知特征聚合损失和增强的对抗扰动设计，显著提升对抗样本迁移成功率。

Method: MT-AdvCLIP结合任务感知特征聚合损失，生成具有更强跨任务泛化能力的对抗扰动。该方法无需增加扰动预算，即可显著提升攻击效果。

Result: 实验表明，MT-AdvCLIP在多任务对抗攻击中的平均成功率提升了39%以上，显著优于基线方法。

Insight: 细粒度任务生成的对抗样本在多任务迁移中更具优势，这为多任务鲁棒性评估和对抗样本设计提供了新视角。

Abstract: As a general-purpose vision-language pretraining model, CLIP demonstrates strong generalization ability in image-text alignment tasks and has been widely adopted in downstream applications such as image classification and image-text retrieval. However, it struggles with fine-grained tasks such as object detection and semantic segmentation. While many variants aim to improve CLIP on these tasks, its robustness to adversarial perturbations remains underexplored. Understanding how adversarial examples transfer across tasks is key to assessing CLIP’s generalization limits and security risks. In this work, we conduct a systematic empirical analysis of the cross-task transfer behavior of CLIP-based models on image-text retrieval, object detection, and semantic segmentation under adversarial perturbations. We find that adversarial examples generated from fine-grained tasks (e.g., object detection and semantic segmentation) often exhibit stronger transfer potential than those from coarse-grained tasks, enabling more effective attacks against the original CLIP model. Motivated by this observation, we propose a novel framework, Multi-Task Adversarial CLIP (MT-AdvCLIP), which introduces a task-aware feature aggregation loss and generates perturbations with enhanced cross-task generalization capability. This design strengthens the attack effectiveness of fine-grained task models on the shared CLIP backbone. Experimental results on multiple public datasets show that MT-AdvCLIP significantly improves the adversarial transfer success rate (The average attack success rate across multiple tasks is improved by over 39%.) against various CLIP-derived models, without increasing the perturbation budget. This study reveals the transfer mechanism of adversarial examples in multi-task CLIP models, offering new insights into multi-task robustness evaluation and adversarial example design.

[244] DriveE2E: Closed-Loop Benchmark for End-to-End Autonomous Driving through Real-to-Simulation

Haibao Yu,Wenxian Yang,Ruiyang Hao,Chuanye Wang,Jiaru Zhong,Ping Luo,Zaiqing Nie

Main category: cs.CV

TL;DR: DriveE2E提出了一个闭环基准测试框架，将真实世界的驾驶场景整合到CARLA模拟器中，通过构建数字孪生资产和动态交通场景，提升模拟的真实性。

Details Motivation: 当前基于CARLA模拟器的闭环评测依赖于手动配置的交通场景，与真实世界条件存在差异，限制了评测结果的真实性。因此，DriveE2E旨在提供一个更贴近现实的评测框架。

Contribution: 1. 提出了一种闭环评测框架，结合基础设施传感器数据构建了800个动态交通场景和15个交叉路口的数字孪生。2. 在CARLA模拟器中实现了更真实的模拟环境。3. 提供了一个全面的端到端自动驾驶模型评测基准。

Method: 1. 从100小时的基础设施传感器视频数据中提取800个动态交通场景。2. 构建15个交叉路口的静态数字孪生资产，确保视觉一致性。3. 将这些场景和资产整合到CARLA模拟器中，提供闭环评测。

Result: 提出的评测框架能够更真实地反映复杂城市交叉路口的多样性驾驶行为和环境条件。

Insight: 1. 真实世界数据的整合能够显著提升模拟评测的真实性。2. 基础设施传感器的应用为自动驾驶评测提供了新的数据来源。

Abstract: Closed-loop evaluation is increasingly critical for end-to-end autonomous driving. Current closed-loop benchmarks using the CARLA simulator rely on manually configured traffic scenarios, which can diverge from real-world conditions, limiting their ability to reflect actual driving performance. To address these limitations, we introduce a simple yet challenging closed-loop evaluation framework that closely integrates real-world driving scenarios into the CARLA simulator with infrastructure cooperation. Our approach involves extracting 800 dynamic traffic scenarios selected from a comprehensive 100-hour video dataset captured by high-mounted infrastructure sensors, and creating static digital twin assets for 15 real-world intersections with consistent visual appearance. These digital twins accurately replicate the traffic and environmental characteristics of their real-world counterparts, enabling more realistic simulations in CARLA. This evaluation is challenging due to the diversity of driving behaviors, locations, weather conditions, and times of day at complex urban intersections. In addition, we provide a comprehensive closed-loop benchmark for evaluating end-to-end autonomous driving models. Project URL: \href{https://github.com/AIR-THU/DriveE2E}{https://github.com/AIR-THU/DriveE2E}.

[245] Learning Encoding-Decoding Direction Pairs to Unveil Concepts of Influence in Deep Vision Networks

Alexandros Doumanoglou,Kurt Driessens,Dimitrios Zarpalas

Main category: cs.CV

TL;DR: 本文提出了一种新方法，通过学习编码-解码方向对来揭示深度视觉网络中概念的影响，该方法优于传统的矩阵分解或自编码器方法，并在解释性和干预应用中表现出色。

Details Motivation: 深度视觉网络的潜在空间中的概念表现为方向（概念嵌入），但目前缺乏直接访问这些方向的方法，限制了网络的可解释性和调试能力。

Contribution: 1. 提出了一种新的方法来学习和解释编码-解码方向对；2. 引入不确定性区域对齐技术，利用网络权重揭示可解释方向；3. 展示了该方法在合成数据和真实数据上的有效性。

Method: 1. 通过方向聚类识别解码方向；2. 基于概率视角估计编码方向；3. 利用不确定性区域对齐技术优化方向的解释性。

Result: 1. 在合成数据中恢复真实方向对；2. 在真实数据中解码方向映射到单义可解释概念，且优于无监督基线；3. 信号向量可靠估计编码方向。

Insight: 编码-解码方向对不仅提升了模型的解释性，还支持模型的干预和错误修正，为深度网络的可解释性研究提供了新思路。

Abstract: Empirical evidence shows that deep vision networks represent concepts as directions in latent space, vectors we call concept embeddings. Each concept has a latent factor-a scalar-indicating its presence in an input patch. For a given patch, multiple latent factors are encoded into a compact representation by linearly combining concept embeddings, with the factors as coefficients. Since these embeddings enable such encoding, we call them encoding directions. A latent factor can be recovered via the inner product with a filter, a vector we call a decoding direction. These encoding-decoding direction pairs are not directly accessible, but recovering them helps open the black box of deep networks, enabling understanding, debugging, and improving models. Decoder directions attribute meaning to latent codes, while encoding directions assess concept influence on predictions, with both enabling model correction by unlearning irrelevant concepts. Unlike prior matrix decomposition, autoencoder, or dictionary learning methods that rely on feature reconstruction, we propose a new perspective: decoding directions are identified via directional clustering of activations, and encoding directions are estimated with signal vectors under a probabilistic view. We further leverage network weights through a novel technique, Uncertainty Region Alignment, which reveals interpretable directions affecting predictions. Our analysis shows that (a) on synthetic data, our method recovers ground-truth direction pairs; (b) on real data, decoding directions map to monosemantic, interpretable concepts and outperform unsupervised baselines; and (c) signal vectors faithfully estimate encoding directions, validated via activation maximization. Finally, we demonstrate applications in understanding global model behavior, explaining individual predictions, and intervening to produce counterfactuals or correct errors.

[246] SAR-KnowLIP: Towards Multimodal Foundation Models for Remote Sensing

Yi Yang,Xiaokun Zhang,Qingchen Fang,Ziqi Ye,Rui Li,Li Liu,Haipeng Wang

Main category: cs.CV

TL;DR: 本文提出了SAR-KnowLIP，首个通用的合成孔径雷达（SAR）多模态基础模型，填补了SAR图像建模的空白。通过构建大规模数据集SAR-GEOVL-1M、分层认知链生成对齐文本、自一致迭代优化机制以及统一评估基准，模型在多项下游任务中表现优异。

Details Motivation: 现有方法主要针对RGB图像，SAR图像的建模存在显著空白。SAR因其全天候成像能力在遥感场景理解中不可或缺，亟需多模态基础模型的支持。

Contribution: 1) 引入地理信息属性，构建首个大规模SAR数据集SAR-GEOVL-1M；2) 通过分层认知链生成结构化文本注释；3) 设计自一致迭代优化机制提升跨模态对齐；4) 建立统一评估基准。

Method: 1) 数据集构建；2) 分层认知链生成文本注释；3) 自一致迭代优化机制结合对比、匹配和重建学习的自监督闭环；4) 统一评估基准设计。

Result: SAR-KnowLIP在11项下游任务中表现领先，尤其在目标计数和土地覆盖分类任务中。

Insight: 地理信息的引入和多模态数据对齐对SAR图像理解至关重要；自一致迭代优化机制有效提升了跨模态模型的性能。

Abstract: Cross-modal artificial intelligence has garnered widespread attention in recent years, achieving significant progress in the study of natural images. However, existing methods are mostly designed for RGB imagery, leaving a significant gap in modeling synthetic aperture radar (SAR) imagery. SAR, with its all-day, all-weather imaging capabilities, plays an irreplaceable role in remote sensing scene understanding. To address this gap, this paper proposes SAR-KnowLIP, the first universal SAR multimodal foundational model, along with reusable data and evaluation baselines. Specifically: (1) This work introduces the critical yet long-overlooked attribute of geographic information into remote sensing research, constructing SAR-GEOVL-1M (the first large-scale SAR dataset with complete geographic projection properties), covering multiple satellite platforms, 120,000 images, and 135 cities. (2) Aligned structured text is generated through a hierarchical cognitive chain-of-thought (HCoT), providing more than one million multi-dimensional semantic annotations of landforms, regional functions, target attributes, and spatial relationships. (3) We design a Self-Consistent Iterative Optimization mechanism that continuously enhances cross-modal alignment through a self-supervised closed loop of contrastive, matching, and reconstruction learning on a transferable multimodal encoder. (4) A unified evaluation benchmark is established across 11 representative downstream vision and vision-language tasks, with comparisons against 14 leading foundation models, where SAR-KnowLIP demonstrates leading performance, particularly in object counting and land-cover classification. We expect that SAR-KnowLIP’s large-scale multimodal data, transferable model architecture, and comprehensive experimental benchmark will significantly advance the development of SAR multimodal baseline models.

[247] AutoPrune: Each Complexity Deserves a Pruning Policy

Hanshi Wang,Yuhao Xu,Zekun Xu,Jin Gao,Yufan Liu,Weiming Hu,Ke Wang,Zhipeng Zhang

Main category: cs.CV

TL;DR: AutoPrune提出了复杂性自适应的剪枝策略，通过量化视觉与文本token间的互信息，动态调整剪枝策略以适应不同任务和样本的复杂性，显著降低了计算开销并保持了高精度。

Details Motivation: 现有视觉-语言模型的剪枝策略通常是固定的，未能根据输入样本和任务的复杂性动态调整，无法与模型的整体推理轨迹对齐。

Contribution: 提出了Complexity-Adaptive Pruning (AutoPrune)，一种无需训练的动态剪枝框架，根据任务复杂性生成独特的剪枝策略。

Method: 通过量化视觉与文本token的互信息，将其映射到预算约束的逻辑保留曲线，动态调整剪枝策略。

Result: 在LLaVA-1.5-7B模型上，剪枝89%的视觉token，FLOPs减少76.8%，精度仅下降3.3%。

Insight: 剪枝策略应与输入和任务的复杂性动态对齐，类似于人类视觉处理的渐进式聚焦过程。

Abstract: The established redundancy in visual tokens within large vision-language models allows pruning to effectively reduce their substantial computational demands. Previous methods typically employ heuristic layer-specific pruning strategies where, although the number of tokens removed may differ across decoder layers, the overall pruning schedule is fixed and applied uniformly to all input samples and tasks, failing to align token elimination with the model’s holistic reasoning trajectory. Cognitive science indicates that human visual processing often begins with broad exploration to accumulate evidence before narrowing focus as the target becomes distinct. Our experiments reveal an analogous pattern in these models. This observation suggests that neither a fixed pruning schedule nor a heuristic layer-wise strategy can optimally accommodate the diverse complexities inherent in different inputs. To overcome this limitation, we introduce Complexity-Adaptive Pruning (AutoPrune), a training-free, plug-and-play framework that tailors pruning policies to varying sample and task complexities. Specifically, AutoPrune quantifies the mutual information between visual and textual tokens, then projects this signal to a budget-constrained logistic retention curve. Each such logistic curve, defined by its unique shape, corresponds to the specific complexity of different tasks and can guarantee adherence to predefined computational constraints. We evaluate AutoPrune on standard vision-language tasks and on Vision-Language-Action models for autonomous driving. Notably, when applied to LLaVA-1.5-7B, our method prunes 89% of visual tokens and reduces inference FLOPs by 76.8% while retaining 96.7% of the original accuracy averaged over all tasks. This corresponds to a 9.1% improvement over the recent work PDrop, demonstrating the effectiveness. Code is available at https://github.com/AutoLab-SAI-SJTU/AutoPrune.

[248] CrashSplat: 2D to 3D Vehicle Damage Segmentation in Gaussian Splatting

Dragoş-Andrei Chileban,Andrei-Ştefan Bulzan,Cosmin Cernǎzanu-Glǎvan

Main category: cs.CV

TL;DR: 论文提出了一种名为CrashSplat的自动车辆损伤检测方法，通过将2D分割掩码升维到3D高斯泼溅表示中，实现了基于单视图的3D损伤分割。

Details Motivation: 车辆损伤检测对汽车保险行业至关重要，但现有方法多为2D图像分析，缺乏几何精确性。3D高斯泼溅的新颖视图合成技术为解决这一挑战提供了潜力。

Contribution: 1）提出了一个自动车辆损伤检测流水线，实现了3D损伤分割；2）提出了一种无需学习的单视图3D高斯泼溅分割方法。

Method: 通过结构从运动（SfM）获取相机参数，将高斯泼溅投影到图像平面，并利用Z缓冲和深度/不透明度的正态分布模型进行过滤。

Result: 实验表明，该方法在单视图难以捕获目标（如划痕、小凹痕）的场景中表现优异，超越了多视图一致性方法的限制。

Insight: 无需学习的单视图分割方法在几何复杂或目标仅出现在单一视图的场景中具有显著优势，为3D损伤分析提供了新思路。

Abstract: Automatic car damage detection has been a topic of significant interest for the auto insurance industry as it promises faster, accurate, and cost-effective damage assessments. However, few works have gone beyond 2D image analysis to leverage 3D reconstruction methods, which have the potential to provide a more comprehensive and geometrically accurate representation of the damage. Moreover, recent methods employing 3D representations for novel view synthesis, particularly 3D Gaussian Splatting (3D-GS), have demonstrated the ability to generate accurate and coherent 3D reconstructions from a limited number of views. In this work we introduce an automatic car damage detection pipeline that performs 3D damage segmentation by up-lifting 2D masks. Additionally, we propose a simple yet effective learning-free approach for single-view 3D-GS segmentation. Specifically, Gaussians are projected onto the image plane using camera parameters obtained via Structure from Motion (SfM). They are then filtered through an algorithm that utilizes Z-buffering along with a normal distribution model of depth and opacities. Through experiments we found that this method is particularly effective for challenging scenarios like car damage detection, where target objects (e.g., scratches, small dents) may only be clearly visible in a single view, making multi-view consistency approaches impractical or impossible. The code is publicly available at: https://github.com/DragosChileban/CrashSplat.

[249] HunyuanImage 3.0 Technical Report

Siyu Cao,Hangting Chen,Peng Chen,Yiji Cheng,Yutao Cui,Xinchi Deng,Ying Dong,Kipper Gong,Tianpeng Gu,Xiusen Gu,Tiankai Hang,Duojun Huang,Jie Jiang,Zhengkai Jiang,Weijie Kong,Changlin Li,Donghao Li,Junzhe Li,Xin Li,Yang Li,Zhenxi Li,Zhimin Li,Jiaxin Lin,Linus,Lucaz Liu,Shu Liu,Songtao Liu,Yu Liu,Yuhong Liu,Yanxin Long,Fanbin Lu,Qinglin Lu,Yuyang Peng,Yuanbo Peng,Xiangwei Shen,Yixuan Shi,Jiale Tao,Yangyu Tao,Qi Tian,Pengfei Wan,Chunyu Wang,Kai Wang,Lei Wang,Linqing Wang,Lucas Wang,Qixun Wang,Weiyan Wang,Hao Wen,Bing Wu,Jianbing Wu,Yue Wu,Senhao Xie,Fang Yang,Miles Yang,Xiaofeng Yang,Xuan Yang,Zhantao Yang,Jingmiao Yu,Zheng Yuan,Chao Zhang,Jian-Wei Zhang,Peizhen Zhang,Shi-Xue Zhang,Tao Zhang,Weigang Zhang,Yepeng Zhang,Yingfang Zhang,Zihao Zhang,Zijian Zhang,Penghao Zhao,Zhiyuan Zhao,Xuefei Zhe,Jianchen Zhu,Zhao Zhong

Main category: cs.CV

TL;DR: HunyuanImage 3.0是一个统一多模态理解与生成的自回归模型，其图像生成模块已开源。通过精细数据、架构设计、Chain-of-Thoughts、渐进预训练和后训练等技术，训练了包含800亿参数的MoE模型，是目前最大最强的开源图像生成模型。

Details Motivation: 目标是提供一个强大的开源多模态基础模型，推动社区在多模态领域的研究和应用。

Contribution: 1. 开源HunyuanImage 3.0模型；2. 提出统一多模态理解与生成的框架；3. 结合MoE架构与Chain-of-Thoughts技术；4. 高效的基础设施支持大规模训练和推理。

Method: 1. 数据精细筛选；2. 自回归框架设计；3. Chain-of-Thoughts模式；4. 渐进预训练和激进后训练；5. MoE架构实现高效推理。

Result: 自动与人工评估显示，HunyuanImage 3.0在文本-图像对齐和视觉质量上媲美SOTA模型。

Insight: 结合MoE与Chain-of-Thoughts可以显著提升多模态模型的性能和效率。

Abstract: We present HunyuanImage 3.0, a native multimodal model that unifies multimodal understanding and generation within an autoregressive framework, with its image generation module publicly available. The achievement of HunyuanImage 3.0 relies on several key components, including meticulous data curation, advanced architecture design, a native Chain-of-Thoughts schema, progressive model pre-training, aggressive model post-training, and an efficient infrastructure that enables large-scale training and inference. With these advancements, we successfully trained a Mixture-of-Experts (MoE) model comprising over 80 billion parameters in total, with 13 billion parameters activated per token during inference, making it the largest and most powerful open-source image generative model to date. We conducted extensive experiments and the results of automatic and human evaluation of text-image alignment and visual quality demonstrate that HunyuanImage 3.0 rivals previous state-of-the-art models. By releasing the code and weights of HunyuanImage 3.0, we aim to enable the community to explore new ideas with a state-of-the-art foundation model, fostering a dynamic and vibrant multimodal ecosystem. All open source assets are publicly available at https://github.com/Tencent-Hunyuan/HunyuanImage-3.0

[250] ColLab: A Collaborative Spatial Progressive Data Engine for Referring Expression Comprehension and Generation

Shilan Zhang,Jirui Huang,Ruilin Yao,Cong Wang,Yaxiong Chen,Peng Xu,Shengwu Xiong

Main category: cs.CV

TL;DR: ColLab提出了一种无需人工监督的自动化REC和REG数据生成方法，通过协作式多模态模型交互（CMMI）和空间渐进增强（SPA），显著提升了数据的质量和多样性。

Details Motivation: 当前的REC和REG任务依赖人工标注，成本高且难以扩展。为解决这一问题，需开发自动化数据生成方法。

Contribution: 1. 提出ColLab框架，实现完全自动化的REC和REG数据生成。2. 引入CMMI策略和SPA模块，提升语义理解和空间表达能力。

Method: 1. CMMI结合MLLMs和LLMs的语义理解生成描述。2. SPA模块通过空间渐进增强提高重复实例的表达能力。

Result: ColLab显著提升了数据生成效率和质量，并被ICCV 2025 MARS2挑战赛部分采用。

Insight: 自动化数据生成可解决人工标注的瓶颈，多模态协作和空间增强是提升任务性能的关键。

Abstract: Referring Expression Comprehension (REC) and Referring Expression Generation (REG) are fundamental tasks in multimodal understanding, supporting precise object localization through natural language. However, existing REC and REG datasets rely heavily on manual annotation, which is labor-intensive and difficult to scale. In this paper, we propose ColLab, a collaborative spatial progressive data engine that enables fully automated REC and REG data generation without human supervision. Specifically, our method introduces a Collaborative Multimodal Model Interaction (CMMI) strategy, which leverages the semantic understanding of multimodal large language models (MLLMs) and large language models (LLMs) to generate descriptions. Furthermore, we design a module termed Spatial Progressive Augmentation (SPA) to enhance spatial expressiveness among duplicate instances. Experiments demonstrate that ColLab significantly accelerates the annotation process of REC and REG while improving the quality and discriminability of the generated expressions. In addition to the core methodological contribution, our framework was partially adopted in the data generation pipeline of the ICCV 2025 MARS2 Challenge on Multimodal Reasoning, enriching the dataset with diverse and challenging samples that better reflect real-world reasoning demands.

[251] Reinforcement Learning with Inverse Rewards for World Model Post-training

Yang Ye,Tianyu He,Shuo Yang,Jiang Bian

Main category: cs.CV

TL;DR: 论文提出了一种通过逆向奖励强化学习（RLIR）的后训练框架，用于提升视频世界模型的动作跟随能力，避免了大规模偏好标注的高成本和不可行的规则视频验证。

Details Motivation: 当前视频世界模型在视觉质量和时间一致性方面取得了进步，但其在准确建模人类指定动作方面的能力仍有待探索。直接使用强化学习优化模型动作跟随能力需要合适的奖励函数，但其实现成本高昂且不可行。

Contribution: 提出了RLIR框架，通过逆向动力学模型从生成视频中恢复输入动作，为优化提供可验证的奖励信号，解决了奖励函数定义的难题。

Method: 使用逆向动力学模型将高维视频模态映射到低维动作空间，通过Group Relative Policy Optimization优化模型动作跟随能力。

Result: 实验表明，RLIR在多任务中实现了5-10%的动作跟随能力提升，视觉质量提升达10%，并获得更高的人类偏好评分。

Insight: RLIR为视频世界模型的后训练提供了一种无需大规模标注的奖励信号定义方法，显著提升了模型的实用性和可扩展性。

Abstract: World models simulate dynamic environments, enabling agents to interact with diverse input modalities. Although recent advances have improved the visual quality and temporal consistency of video world models, their ability of accurately modeling human-specified actions remains under-explored. Reinforcement learning presents a promising approach for directly improving the suboptimal action-following capability of pre-trained models, assuming that an appropriate reward function can be defined. However, transferring reinforcement learning post-training methods to world model is impractical due to the prohibitive cost of large-scale preference annotations and the infeasibility of constructing rule-based video verifiers. To address this gap, we propose Reinforcement Learning with Inverse Rewards (RLIR), a post-training framework that derives verifiable reward signals by recovering input actions from generated videos using an Inverse Dynamics Model. By mapping high-dimensional video modality to a low-dimensional action space, RLIR provides an objective and verifiable reward for optimization via Group Relative Policy Optimization. Experiments across autoregressive and diffusion paradigms demonstrate 5-10% gains in action-following, up to 10% improvements in visual quality, and higher human preference scores, establishing RLIR as the first post-training method specifically designed to enhance action-following in video world models.

[252] A Novel Hybrid Deep Learning and Chaotic Dynamics Approach for Thyroid Cancer Classification

Nada Bouchekout,Abdelkrim Boukabou,Morad Grimes,Yassine Habchi,Yassine Himeur,Hamzah Ali Alkhazaleh,Shadi Atalla,Wathiq Mansoor

Main category: cs.CV

TL;DR: 提出了一种结合自适应CNN和CDF9/7小波的甲状腺癌分类方法，通过混沌系统增强特征，实现了98.17%的准确率和优异的计算效率。

Details Motivation: 甲状腺癌诊断的时效性和准确性对治疗至关重要，现有方法在特征提取和泛化能力上有待提升。

Contribution: 1) 结合自适应CNN与小波变换及混沌系统，提升特征判别力；2) 在多个数据集上验证了方法的优越性和泛化性；3) 提供可解释性分析以支持临床决策。

Method: 使用CDF9/7小波的细节系数，并通过n-scroll混沌系统调制，结合自适应CNN进行分类。通过交叉验证和消融实验验证设计有效性。

Result: 在DDTI数据集上达到98.17%准确率，优于EfficientNetV2-S等SOTA方法；计算高效（28.7 ms/图像）。

Insight: 混沌系统能有效增强小波特征的判别力，结合CNN在小数据医学图像分类中表现优越且具临床实用性。

Abstract: Timely and accurate diagnosis is crucial in addressing the global rise in thyroid cancer, ensuring effective treatment strategies and improved patient outcomes. We present an intelligent classification method that couples an Adaptive Convolutional Neural Network (CNN) with Cohen-Daubechies-Feauveau (CDF9/7) wavelets whose detail coefficients are modulated by an n-scroll chaotic system to enrich discriminative features. We evaluate on the public DDTI thyroid ultrasound dataset (n = 1,638 images; 819 malignant / 819 benign) using 5-fold cross-validation, where the proposed method attains 98.17% accuracy, 98.76% sensitivity, 97.58% specificity, 97.55% F1-score, and an AUC of 0.9912. A controlled ablation shows that adding chaotic modulation to CDF9/7 improves accuracy by +8.79 percentage points over a CDF9/7-only CNN (from 89.38% to 98.17%). To objectively position our approach, we trained state-of-the-art backbones on the same data and splits: EfficientNetV2-S (96.58% accuracy; AUC 0.987), Swin-T (96.41%; 0.986), ViT-B/16 (95.72%; 0.983), and ConvNeXt-T (96.94%; 0.987). Our method outperforms the best of these by +1.23 points in accuracy and +0.0042 in AUC, while remaining computationally efficient (28.7 ms per image; 1,125 MB peak VRAM). Robustness is further supported by cross-dataset testing on TCIA (accuracy 95.82%) and transfer to an ISIC skin-lesion subset (n = 28 unique images, augmented to 2,048; accuracy 97.31%). Explainability analyses (Grad-CAM, SHAP, LIME) highlight clinically relevant regions. Altogether, the wavelet-chaos-CNN pipeline delivers state-of-the-art thyroid ultrasound classification with strong generalization and practical runtime characteristics suitable for clinical integration.

[253] VFSI: Validity First Spatial Intelligence for Constraint-Guided Traffic Diffusion

Kargi Chauhan,Leilani H. Gilpin

Main category: cs.CV

TL;DR: VFSI提出了一种基于能量引导的方法，在扩散采样中显式强制执行物理约束，显著提高了交通模拟的真实性和物理有效性，而无需重新训练模型。

Details Motivation: 现代扩散模型生成的交通模拟虽然逼真，但普遍违反物理约束，如碰撞、偏离道路等，突显了现有方法将物理有效性视为辅助属性而非设计要求的局限性。

Contribution: 1. 提出了VFSI框架，通过能量引导在扩散采样中强制执行物理约束；2. 在不重新训练模型的情况下，显著提升了交通模拟的有效性和真实性。

Method: 1. 设计能量函数编码碰撞避免和运动学约束；2. 在扩散去噪过程中引入能量引导，确保生成的轨迹满足物理有效性。

Result: 在Waymo数据集上的实验显示，VFSI将碰撞率降低67%，总体有效性提升87%，同时改善了轨迹精度（ADE降低）。

Insight: 显式约束强制执行在推断阶段是必要且充分的，为提升生成模型的物理有效性提供了通用解决方案。

Abstract: Modern diffusion models generate realistic traffic simulations but systematically violate physical constraints. In a large-scale evaluation of SceneDiffuser++, a state-of-the-art traffic simulator, we find that 50% of generated trajectories violate basic physical laws - vehicles collide, drive off roads, and spawn inside buildings. This reveals a fundamental limitation: current models treat physical validity as an emergent property rather than an architectural requirement. We propose Validity-First Spatial Intelligence (VFSI), which enforces constraints through energy-based guidance during diffusion sampling, without model retraining. By incorporating collision avoidance and kinematic constraints as energy functions, we guide the denoising process toward physically valid trajectories. Across 200 urban scenarios from the Waymo Open Motion Dataset, VFSI reduces collision rates by 67% (24.6% to 8.1%) and improves overall validity by 87% (50.3% to 94.2%), while simultaneously improving realism metrics (ADE: 1.34m to 1.21m). Our model-agnostic approach demonstrates that explicit constraint enforcement during inference is both necessary and sufficient for physically valid traffic simulation.

[254] Towards Redundancy Reduction in Diffusion Models for Efficient Video Super-Resolution

Jinpei Guo,Yifei Ji,Zheng Chen,Yufei Wang,Sizhuo Ma,Yong Guo,Yulun Zhang,Jian Wang

Main category: cs.CV

TL;DR: 该论文提出了OASIS，一种高效的一步扩散模型，用于视频超分辨率（VSR），通过注意力专业化路由减少冗余并提升性能。

Details Motivation: 视频超分辨率任务中，直接使用扩散模型会引入冗余，增加计算负担和学习难度，因为低质量视频已包含大量内容信息。

Contribution: 1. 提出OASIS模型，引入注意力专业化路由以减少冗余；2. 提出渐进训练策略，提升复杂退化情况下的学习能力。

Method: 采用一步扩散模型，结合注意力专业化路由动态分配注意力头，并设计渐进训练策略从一致到不一致退化场景逐步学习。

Result: OASIS在合成和真实数据集上表现最优，推理速度比基线SeedVR2快6.2倍。

Insight: 通过注意力路由和渐进训练，可以有效优化扩散模型在VSR任务中的效率与性能。

Abstract: Diffusion models have recently shown promising results for video super-resolution (VSR). However, directly adapting generative diffusion models to VSR can result in redundancy, since low-quality videos already preserve substantial content information. Such redundancy leads to increased computational overhead and learning burden, as the model performs superfluous operations and must learn to filter out irrelevant information. To address this problem, we propose OASIS, an efficient $\textbf{o}$ne-step diffusion model with $\textbf{a}$ttention $\textbf{s}$pecialization for real-world v$\textbf{i}$deo $\textbf{s}$uper-resolution. OASIS incorporates an attention specialization routing that assigns attention heads to different patterns according to their intrinsic behaviors. This routing mitigates redundancy while effectively preserving pretrained knowledge, allowing diffusion models to better adapt to VSR and achieve stronger performance. Moreover, we propose a simple yet effective progressive training strategy, which starts with temporally consistent degradations and then shifts to inconsistent settings. This strategy facilitates learning under complex degradations. Extensive experiments demonstrate that OASIS achieves state-of-the-art performance on both synthetic and real-world datasets. OASIS also provides superior inference speed, offering a $\textbf{6.2$\times$}$ speedup over one-step diffusion baselines such as SeedVR2. The code will be available at \href{https://github.com/jp-guo/OASIS}{https://github.com/jp-guo/OASIS}.

[255] Advancing Multi-agent Traffic Simulation via R1-Style Reinforcement Fine-Tuning

Muleilan Pei,Shaoshuai Shi,Shaojie Shen

Main category: cs.CV

TL;DR: 论文提出SMART-R1，一种基于R1风格强化微调的范式，用于改进多智能体交通模拟，通过在监督微调和强化微调间交替训练，提升模型泛化能力和仿真真实性。

Details Motivation: 现有数据驱动的交通模拟器主要依赖监督学习，难以解决训练与测试间的分布偏移问题，限制了模型在未见环境中的泛化能力。

Contribution: 1) 提出SMART-R1框架，结合R1风格强化微调；2) 设计面向指标的策略优化算法；3) 提出迭代的SFT-RFT-SFT训练策略。

Method: 通过交替进行监督微调（SFT）和强化微调（RFT），结合指标驱动的策略优化算法，提升模型表现。

Result: 在Waymo Open Sim Agents Challenge上取得SOTA性能，真实性元得分0.7858，排名第一。

Insight: 交替使用监督学习和强化学习可以有效缓解分布偏移问题，提升仿真模型的泛化能力和真实性。

Abstract: Scalable and realistic simulation of multi-agent traffic behavior is critical for advancing autonomous driving technologies. Although existing data-driven simulators have made significant strides in this domain, they predominantly rely on supervised learning to align simulated distributions with real-world driving scenarios. A persistent challenge, however, lies in the distributional shift that arises between training and testing, which often undermines model generalization in unseen environments. To address this limitation, we propose SMART-R1, a novel R1-style reinforcement fine-tuning paradigm tailored for next-token prediction models to better align agent behavior with human preferences and evaluation metrics. Our approach introduces a metric-oriented policy optimization algorithm to improve distribution alignment and an iterative “SFT-RFT-SFT” training strategy that alternates between Supervised Fine-Tuning (SFT) and Reinforcement Fine-Tuning (RFT) to maximize performance gains. Extensive experiments on the large-scale Waymo Open Motion Dataset (WOMD) validate the effectiveness of this simple yet powerful R1-style training framework in enhancing foundation models. The results on the Waymo Open Sim Agents Challenge (WOSAC) showcase that SMART-R1 achieves state-of-the-art performance with an overall realism meta score of 0.7858, ranking first on the leaderboard at the time of submission.

[256] TREAT-Net: Tabular-Referenced Echocardiography Analysis for Acute Coronary Syndrome Treatment Prediction

Diane Kim,Minh Nguyen Nhat To,Sherif Abdalla,Teresa S. M. Tsang,Purang Abolmaesumi,and Christina Luong

Main category: cs.CV

TL;DR: TREAT-Net是一个多模态深度学习框架，结合超声心动图视频和结构化临床记录，非侵入性地预测急性冠状动脉综合征（ACS）的治疗方案。通过表格指导的交叉注意力增强视频解释，并利用晚期融合机制对齐多模态预测。

Details Motivation: 冠状动脉造影是ACS诊断的金标准，但其资源密集和侵入性可能导致患者风险和诊断延迟。TREAT-Net旨在提供一种非侵入性替代方案，以加速治疗决策。

Contribution: 提出了TREAT-Net，首次将表格引导的交叉注意力和多模态晚期融合用于ACS治疗预测，显著提升预测性能。

Method: 结合超声心动图视频和结构化临床记录，使用表格引导的交叉注意力增强视频特征，并通过晚期融合对齐多模态预测结果。

Result: 在9000多例ACS数据上，TREAT-Net平衡准确率达67.6%，AUROC为71.1%，干预预测的一致性达88.6%。

Insight: 多模态数据融合（尤其是视频和表格数据的协同）能显著提升ACS预测准确性，为非侵入性诊断工具的开发提供了新思路。

Abstract: Coronary angiography remains the gold standard for diagnosing Acute Coronary Syndrome (ACS). However, its resource-intensive and invasive nature can expose patients to procedural risks and diagnostic delays, leading to postponed treatment initiation. In this work, we introduce TREAT-Net, a multimodal deep learning framework for ACS treatment prediction that leverages non-invasive modalities, including echocardiography videos and structured clinical records. TREAT-Net integrates tabular-guided cross-attention to enhance video interpretation, along with a late fusion mechanism to align predictions across modalities. Trained on a dataset of over 9000 ACS cases, the model outperforms unimodal and non-fused baselines, achieving a balanced accuracy of 67.6% and an AUROC of 71.1%. Cross-modality agreement analysis demonstrates 88.6% accuracy for intervention prediction. These findings highlight the potential of TREAT-Net as a non-invasive tool for timely and accurate patient triage, particularly in underserved populations with limited access to coronary angiography.

[257] FrameMind: Frame-Interleaved Chain-of-Thought for Video Reasoning via Reinforcement Learning

Haonan Ge,Yiwei Wang,Kai-Wei Chang,Hang Wu,Yujun Cai

Main category: cs.CV

TL;DR: FrameMind提出了一种新的视频推理框架，通过强化学习动态选择帧进行推理，解决了固定采样策略的局限性。

Details Motivation: 当前视频理解模型采用固定帧采样策略，无法根据问题的具体需求动态调整视觉信息，限制了性能。

Contribution: 引入了FrameMind框架，结合Frame-Interleaved Chain-of-Thought（FiCOT）和强化学习，实现动态视觉信息请求。

Method: 提出了Dynamic Resolution Frame Sampling（DRFS）和DRFS-GRPO算法，学习多样化的时空权衡策略。

Result: 在MLVU和VideoMME等基准测试中，表现出显著的性能提升，超越了现有模型。

Insight: 动态帧选择和强化学习的结合可以显著提高视频理解的灵活性和效率。

Abstract: Current video understanding models rely on fixed frame sampling strategies, processing predetermined visual inputs regardless of the specific reasoning requirements of each question. This static approach limits their ability to adaptively gather visual evidence, leading to suboptimal performance on tasks that require either broad temporal coverage or fine-grained spatial detail. In this paper, we introduce FrameMind, an end-to-end framework trained with reinforcement learning that enables models to dynamically request visual information during reasoning through Frame-Interleaved Chain-of-Thought (FiCOT). Unlike traditional approaches, FrameMind operates in multiple turns where the model alternates between textual reasoning and active visual perception, using tools to extract targeted frames or video clips based on identified knowledge gaps. To train effective dynamic sampling policies, we propose Dynamic Resolution Frame Sampling (DRFS), which exposes models to diverse temporal-spatial trade-offs during learning, and DRFS-GRPO, a group-relative policy optimization algorithm that learns from outcome-based rewards without requiring frame-level annotations. Extensive experiments on challenging benchmarks like MLVU and VideoMME demonstrate that our method significantly outperforms existing models, advancing the state of the art in flexible and efficient video understanding.

[258] Generalized Category Discovery in Hyperspectral Images via Prototype Subspace Modeling

Xianlu Li,Nicolas Nadisic,Shaoguang Huang,Aleksandra Pizurica

Main category: cs.CV

TL;DR: 本文提出首个针对高光谱图像(GCD)的原型子空间建模框架，通过多基向量表示类别，显著提升高维特征空间的表现力和判别能力。

Details Motivation: 现有GCD方法主要针对RGB图像，其假设和建模策略难以适应高光谱图像的高维复杂光谱结构，因此需要专门设计。

Contribution: 1. 提出首个高光谱图像的GCD框架；2. 引入原型子空间建模，用多基向量表示类别；3. 设计基向量正交性和重构约束以优化学习。

Method: 使用原型子空间模型表示类别，分为两步：(1) 基向量正交约束增强类间可分性；(2) 重构约束确保基向量能有效重构类样本。

Result: 实验表明，该方法在高光谱数据集上显著优于现有GCD方法。

Insight: 高光谱数据的复杂结构需子空间建模，多基向量比单原型更具表达力，约束设计对提升性能至关重要。

Abstract: Generalized category discovery(GCD) seeks to jointly identify both known and novel categories in unlabeled data. While prior works have mainly focused on RGB images, their assumptions and modeling strategies do not generalize well to hyperspectral images(HSI), which are inherently high-dimensional and exhibit complex spectral structures. In this paper, we propose the first GCD framework tailored for HSI, introducing a prototype subspace modeling model to better capture class structure. Instead of learning a single prototype vector for each category as in existing methods such as SimGCD, we model each category using a set of basis vectors, forming a subspace representation that enables greater expressiveness and discrimination in a high-dimensional feature space. To guide the learning of such bases, we enforce two key constraints: (1) a basis orthogonality constraint that promotes inter-class separability, and (2) a reconstruction constraint that ensures each prototype basis can effectively reconstruct its corresponding class samples. Experimental results on real-world HSI demonstrate that our method significantly outperforms state-of-the-art GCD methods, establishing a strong foundation for generalized category discovery in hyperspectral settings.

[259] $\mathbf{R}^3$: Reconstruction, Raw, and Rain: Deraining Directly in the Bayer Domain

Nate Rothschild,Moshe Kimhi,Avi Mendelson,Chaim Baskin

Main category: cs.CV

TL;DR: 该论文提出了一种直接在Bayer域去雨的方法，避免了传统ISP处理导致的颜色混合、动态范围裁剪和细节模糊问题，并通过实验验证了其优越性。

Details Motivation: 传统去雨方法通常在ISP处理后的sRGB图像上进行，但ISP过程会损失原始Bayer数据的信息。作者希望通过直接在Bayer域学习，保留更多原始信息，提升重建质量。

Contribution: 1) 比较了ISP后和Bayer域重建的性能；2) 发布了首个公开的Raw-Rain数据集；3) 提出了与人类感知更一致的颜色不变性指标ICS。

Method: 直接在Bayer域训练去雨模型，避免ISP处理的信息损失。实验对比了Bayer和sRGB域模型的性能。

Result: Bayer域模型在PSNR和ICS指标上分别提升了+0.99 dB和+1.2%，同时计算效率更高（GFLOPs减半）。

Insight: ISP后处理可能导致信息不可逆损失，直接在Bayer域学习有望为低层视觉任务和端到端相机流水线开辟新方向。

Abstract: Image reconstruction from corrupted images is crucial across many domains. Most reconstruction networks are trained on post-ISP sRGB images, even though the image-signal-processing pipeline irreversibly mixes colors, clips dynamic range, and blurs fine detail. This paper uses the rain degradation problem as a use case to show that these losses are avoidable, and demonstrates that learning directly on raw Bayer mosaics yields superior reconstructions. To substantiate the claim, we (i) evaluate post-ISP and Bayer reconstruction pipelines, (ii) curate Raw-Rain, the first public benchmark of real rainy scenes captured in both 12-bit Bayer and bit-depth-matched sRGB, and (iii) introduce Information Conservation Score (ICS), a color-invariant metric that aligns more closely with human opinion than PSNR or SSIM. On the test split, our raw-domain model improves sRGB results by up to +0.99 dB PSNR and +1.2% ICS, while running faster with half of the GFLOPs. The results advocate an ISP-last paradigm for low-level vision and open the door to end-to-end learnable camera pipelines.

[260] Joint Superpixel and Self-Representation Learning for Scalable Hyperspectral Image Clustering

Xianlu Li,Nicolas Nadisic,Shaoguang Huang,Aleksandra Pizurica

Main category: cs.CV

TL;DR: 该论文提出了一种联合超像素分割和自表示学习的端到端框架，用于高效的HSI聚类。

Details Motivation: HSI子空间聚类通常计算和内存成本高，而现有的超像素分割方法独立于聚类任务，导致分割结果与聚类目标不一致。

Contribution: 1. 提出一个联合优化超像素分割和子空间聚类的端到端框架；2. 引入模型驱动的反馈机制（基于ADMM的自表示网络）指导超像素分割；3. 超像素网络为每个超像素学习独立的紧凑性参数，实现更灵活的分割。

Method: 采用端到端框架：1. 超像素模块与基于ADMM的自表示网络联合优化；2. 通过反馈机制使分割结果具有聚类意识；3. 为每个超像素学习紧凑性参数。

Result: 在多个HSI基准数据集上，该方法优于现有的最先进聚类方法。

Insight: 联合优化超像素分割和子空间聚类可以显著提升HSI聚类的效率和准确性，同时灵活的紧凑性参数设计增强了分割的适应性。

Abstract: Subspace clustering is a powerful unsupervised approach for hyperspectral image (HSI) analysis, but its high computational and memory costs limit scalability. Superpixel segmentation can improve efficiency by reducing the number of data points to process. However, existing superpixel-based methods usually perform segmentation independently of the clustering task, often producing partitions that do not align with the subsequent clustering objective. To address this, we propose a unified end-to-end framework that jointly optimizes superpixel segmentation and subspace clustering. Its core is a feedback mechanism: a self-representation network based on unfolded Alternating Direction Method of Multipliers (ADMM) provides a model-driven signal to guide a differentiable superpixel module. This joint optimization yields clustering-aware partitions that preserve both spectral and spatial structure. Furthermore, our superpixel network learns a unique compactness parameter for each superpixel, enabling more flexible and adaptive segmentation. Extensive experiments on benchmark HSI datasets demonstrate that our method consistently achieves superior accuracy compared with state-of-the-art clustering approaches.

[261] A Second-Order Perspective on Pruning at Initialization and Knowledge Transfer

Leonardo Iurada,Beatrice Occhiena,Tatiana Tommasi

Main category: cs.CV

TL;DR: 本文研究发现，预训练视觉模型在初始化阶段的剪枝不仅能保持零样本性能，还能通过微调恢复未见任务的性能，这得益于大规模预训练带来的有利损失景观。

Details Motivation: 预训练视觉模型的部署受限于其计算和存储成本，而剪枝是压缩模型的有效方法。然而传统方法需要任务特定数据，这在任务未知时成为挑战。

Contribution: 揭示了预训练模型在初始化剪枝后零样本性能的保持能力及微调恢复未见任务性能的现象，为任务未知时的剪枝提供了新思路。

Method: 研究了数据对预训练模型剪枝的影响，通过剪枝和微调实验验证了模型性能的变化，并分析了大规模预训练对损失景观的作用。

Result: 剪枝后的模型在未见任务上仍保持零样本性能，微调后性能还能恢复，尤其是在大规模预训练的数据集上表现更优。

Insight: 大规模预训练不仅提升了模型的泛化能力，还为其剪枝后的适应性和性能恢复提供了有利条件。

Abstract: The widespread availability of pre-trained vision models has enabled numerous deep learning applications through their transferable representations. However, their computational and storage costs often limit practical deployment. Pruning-at-Initialization has emerged as a promising approach to compress models before training, enabling efficient task-specific adaptation. While conventional wisdom suggests that effective pruning requires task-specific data, this creates a challenge when downstream tasks are unknown in advance. In this paper, we investigate how data influences the pruning of pre-trained vision models. Surprisingly, pruning on one task retains the model’s zero-shot performance also on unseen tasks. Furthermore, fine-tuning these pruned models not only improves performance on original seen tasks but can recover held-out tasks’ performance. We attribute this phenomenon to the favorable loss landscapes induced by extensive pre-training on large-scale datasets.

[262] Uncovering Grounding IDs: How External Cues Shape Multi-Modal Binding

Hosein Hasani,Amirmohammad Izadi,Fatemeh Askari,Mobin Bagherian,Sadegh Mohammadian,Mohammad Izadi,Mahdieh Soleymani Baghshah

Main category: cs.CV

TL;DR: 该论文研究了大型视觉语言模型中外部线索如何通过生成潜在标识符（Grounding IDs）来增强多模态绑定，从而提高结构化推理和精确接地的能力。

Details Motivation: 大型视觉语言模型在多模态任务中表现优异，但在结构化推理和精确接地上仍有限制。研究旨在揭示外部视觉结构（如分区和注释）如何通过潜在标识符提升模型性能。

Contribution: 提出了Grounding IDs的概念，揭示了这些标识符通过模态对齐和减少模态间隙，增强多模态绑定的机制。

Method: 通过表示分析和因果干预实验，验证了Grounding IDs在嵌入空间中对齐和绑定对象与外部线索的作用。

Result: 研究表明Grounding IDs能够增强注意力机制，减少幻觉现象，提高跨模态接地的鲁棒性。

Insight: Grounding IDs为理解外部线索如何提升多模态模型性能提供了符号化的解释和实际改进方向。

Abstract: Large vision-language models (LVLMs) show strong performance across multimodal benchmarks but remain limited in structured reasoning and precise grounding. Recent work has demonstrated that adding simple visual structures, such as partitions and annotations, improves accuracy, yet the internal mechanisms underlying these gains remain unclear. We investigate this phenomenon and propose the concept of Grounding IDs, latent identifiers induced by external cues that bind objects to their designated partitions across modalities. Through representation analysis, we find that these identifiers emerge as robust within-partition alignment in embedding space and reduce the modality gap between image and text. Causal interventions further confirm that these identifiers mediate binding between objects and symbolic cues. We show that Grounding IDs strengthen attention between related components, which in turn improves cross-modal grounding and reduces hallucinations. Taken together, our results identify Grounding IDs as a key symbolic mechanism explaining how external cues enhance multimodal binding, offering both interpretability and practical improvements in robustness.

[263] Autoregressive Video Generation beyond Next Frames Prediction

Sucheng Ren,Chen Chen,Zhenbang Wang,Liangchen Song,Xiangxin Zhu,Alan Yuille,Yinfei Yang,Jiasen Lu

Main category: cs.CV

TL;DR: 本文提出VideoAR框架，突破了传统自回归视频生成逐帧预测的限制，支持多种预测单元（如完整帧、关键细节帧、多尺度细化、时空立方体），并通过实验证明时空立方体作为预测单元在质量、速度和时序一致性上的优越性。

Details Motivation: 传统自回归视频生成模型通常逐帧预测，但其假设帧是视频的自然原子单元可能不合理。本文探讨是否存在更优的预测单元，以提升模型性能和效率。

Contribution: 1. 提出VideoAR框架，支持多种预测单元；2. 证明时空立方体作为预测单元的优越性；3. 在质量和效率上超越现有方法。

Method: 通过设计支持多预测单元的框架（如时空立方体），让自回归模型同时在时空维度操作，摆脱逐帧生成的限制。

Result: VideoAR在VBench评测中表现优于现有方法，推理速度更快，并能生成长达分钟级的视频序列。

Insight: 视频生成的预测单元不应局限于帧级，时空立方体等其他单元可能更高效且质量更高。

Abstract: Autoregressive models for video generation typically operate frame-by-frame, extending next-token prediction from language to video’s temporal dimension. We question that unlike word as token is universally agreed in language if frame is a appropriate prediction unit? To address this, we present VideoAR, a unified framework that supports a spectrum of prediction units including full frames, key-detail frames, multiscale refinements, and spatiotemporal cubes. Among these designs, we find model video generation using \textit{spatiotemporal} cubes as prediction units, which allows autoregressive models to operate across both spatial and temporal dimensions simultaneously. This approach eliminates the assumption that frames are the natural atomic units for video autoregression. We evaluate VideoAR across diverse prediction strategies, finding that cube-based prediction consistently delivers superior quality, speed, and temporal coherence. By removing the frame-by-frame constraint, our video generator surpasses state-of-the-art baselines on VBench while achieving faster inference and enabling seamless scaling to minute-long sequences. We hope this work will motivate rethinking sequence decomposition in video and other spatiotemporal domains.

[264] Unified Multi-Modal Interactive & Reactive 3D Motion Generation via Rectified Flow

Prerit Gupta,Shourya Verma,Ananth Grama,Aniket Bera

Main category: cs.CV

TL;DR: DualFlow是一个统一且高效的多模态双人运动生成框架，通过优化流模型实现快速、高质量的3D运动合成，并引入检索增强生成（RAG）模块和多模态对齐策略。

Details Motivation: 现有方法在多模态条件（如文本、音乐）下生成双人运动时存在推断速度慢、误差累积和语义对齐不足的问题，DualFlow旨在解决这些挑战。

Contribution: 1. 提出基于Rectified Flow的高效确定性采样路径；2. 设计RAG模块增强语义对齐；3. 引入对比目标和同步损失优化多模态协调。

Method: 1. 使用Rectified Flow替代扩散模型以减少推断时间和误差；2. RAG模块通过音乐特征和LLM文本分解检索运动示例；3. 结合对比损失和同步损失提升协调性。

Result: 在文本驱动、音乐驱动和多模态交互任务中，DualFlow在运动质量、响应速度和效率上均优于现有方法。

Insight: Rectified Flow的高效采样路径和多模态对齐策略对提升运动生成的实时性和语义一致性至关重要。

Abstract: Generating realistic, context-aware two-person motion conditioned on diverse modalities remains a central challenge in computer graphics, animation, and human-computer interaction. We introduce DualFlow, a unified and efficient framework for multi-modal two-person motion generation. DualFlow conditions 3D motion synthesis on diverse inputs, including text, music, and prior motion sequences. Leveraging rectified flow, it achieves deterministic straight-line sampling paths between noise and data, reducing inference time and mitigating error accumulation common in diffusion-based models. To enhance semantic grounding, DualFlow employs a Retrieval-Augmented Generation (RAG) module that retrieves motion exemplars using music features and LLM-based text decompositions of spatial relations, body movements, and rhythmic patterns. We use contrastive objective that further strengthens alignment with conditioning signals and introduce synchronization loss that improves inter-person coordination. Extensive evaluations across text-to-motion, music-to-motion, and multi-modal interactive benchmarks show consistent gains in motion quality, responsiveness, and efficiency. DualFlow produces temporally coherent and rhythmically synchronized motions, setting state-of-the-art in multi-modal human motion generation.

[265] SVAC: Scaling Is All You Need For Referring Video Object Segmentation

Li Zhang,Haoxiang Gao,Zhihao Zhang,Luoxiao Huang,Tao Zhang

Main category: cs.CV

TL;DR: SVAC提出了一种通过扩展输入帧和分割标记来提升视频语言交互和分割精度的统一模型，解决了RVOS中的计算和动态对象行为问题，取得了SOTA性能。

Details Motivation: 尽管多模态大语言模型（MLLMs）提升了RVOS性能，但仍存在对MLLMs先验知识利用不足、长视频计算成本高以及复杂时序动态处理不足等问题。

Contribution: 1. 提出SVAC模型，通过扩展输入帧和分割标记提升性能；2. 引入ASTC模块压缩视觉标记；3. 设计CSA策略处理动态对象行为。

Method: 1. 扩展输入帧和分割标记；2. 使用ASTC压缩视觉标记；3. 引入CSA策略动态分配资源。

Result: SVAC在多个RVOS基准测试中达到SOTA性能，且计算效率高。

Insight: 通过扩展输入规模和优化计算模块可以显著提升RVOS性能，动态分配策略是处理复杂时序的关键。

Abstract: Referring Video Object Segmentation (RVOS) aims to segment target objects in video sequences based on natural language descriptions. While recent advances in Multi-modal Large Language Models (MLLMs) have improved RVOS performance through enhanced text-video understanding, several challenges remain, including insufficient exploitation of MLLMs’ prior knowledge, prohibitive computational and memory costs for long-duration videos, and inadequate handling of complex temporal dynamics. In this work, we propose SVAC, a unified model that improves RVOS by scaling up input frames and segmentation tokens to enhance video-language interaction and segmentation precision. To address the resulting computational challenges, SVAC incorporates the Anchor-Based Spatio-Temporal Compression (ASTC) module to compress visual tokens while preserving essential spatio-temporal structure. Moreover, the Clip-Specific Allocation (CSA) strategy is introduced to better handle dynamic object behaviors across video clips. Experimental results demonstrate that SVAC achieves state-of-the-art performance on multiple RVOS benchmarks with competitive efficiency. Our code is available at https://github.com/lizhang1998/SVAC.

[266] Generalist Scanner Meets Specialist Locator: A Synergistic Coarse-to-Fine Framework for Robust GUI Grounding

Zhecheng Li,Guoxian Song,Yiwei Wang,Zhen Xiong,Junsong Yuan,Yujun Cai

Main category: cs.CV

TL;DR: 该论文提出了一种名为GMS的框架，将通用的视觉-语言模型（Scanner）与小型的任务专用模型（Locator）结合，通过分层协同实现GUI中的自然语言查询定位，性能显著提升。

Details Motivation: GUI中的自然语言查询定位需要同时理解多样化的UI元素并精准预测空间坐标，现有单一模型难以兼顾泛化能力和精确性。

Contribution: 提出了GMS框架，通过协同通用模型与专用模型的分工（Scanner扫描区域，Locator精确定位），实现了10倍性能提升。

Method: 框架分为五个阶段，结合分层搜索与跨模态通信：通用VLM扫描潜在区域，专用模型在区域内输出精确坐标。

Result: 在ScreenSpot-Pro数据集上，GMS框架的准确率达35.7%，显著优于单一模型（Scanner 2.0%，Locator 3.7%）及其他基线。

Insight: 模拟人类‘扫描-聚焦’行为的分层设计是提升GUI定位性能的关键，泛化与专用的互补性值得关注。

Abstract: Grounding natural language queries in graphical user interfaces (GUIs) presents a challenging task that requires models to comprehend diverse UI elements across various applications and systems, while also accurately predicting the spatial coordinates for the intended operation. To tackle this problem, we propose GMS: Generalist Scanner Meets Specialist Locator, a synergistic coarse-to-fine framework that effectively improves GUI grounding performance. GMS leverages the complementary strengths of general vision-language models (VLMs) and small, task-specific GUI grounding models by assigning them distinct roles within the framework. Specifically, the general VLM acts as a ‘Scanner’ to identify potential regions of interest, while the fine-tuned grounding model serves as a ‘Locator’ that outputs precise coordinates within these regions. This design is inspired by how humans perform GUI grounding, where the eyes scan the interface and the brain focuses on interpretation and localization. Our whole framework consists of five stages and incorporates hierarchical search with cross-modal communication to achieve promising prediction results. Experimental results on the ScreenSpot-Pro dataset show that while the ‘Scanner’ and ‘Locator’ models achieve only $2.0%$ and $3.7%$ accuracy respectively when used independently, their integration within GMS framework yields an overall accuracy of $35.7%$, representing a $10 \times$ improvement. Additionally, GMS significantly outperforms other strong baselines under various settings, demonstrating its robustness and potential for general-purpose GUI grounding.

[267] EYE-DEX: Eye Disease Detection and EXplanation System

Youssef Sabiri,Walid Houmaidi,Amine Abouaomar

Main category: cs.CV

TL;DR: EYE-DEX是一个自动化框架，用于分类10种视网膜疾病，使用21,577张视网膜图像数据集。采用微调的VGG16模型，达到92.36%的测试准确率，并通过Grad-CAM提供视觉解释以增强透明性。

Details Motivation: 全球超过22亿人受视力障碍影响，传统手动诊断耗时且主观。深度学习可自动化分析视网膜图像，提升诊断效率和质量。

Contribution: 1. 提出EYE-DEX框架，支持10种视网膜疾病分类；2. 微调VGG16模型，达到92.36%的SOTA准确率；3. 集成Grad-CAM技术，提供可视化解释，增强临床信任。

Method: 1. 使用VGG16、VGG19和ResNet50三种预训练CNN模型；2. 微调VGG16模型；3. 结合Grad-CAM生成疾病特异区域的视觉热图。

Result: 微调VGG16模型在视网膜疾病分类任务中取得92.36%的测试准确率，优于其他基准模型。

Insight: 1. 深度学习在医学图像分析中表现出色；2. 模型解释性（如Grad-CAM）对提升临床信任至关重要；3. 大规模数据集是提高模型性能的关键。

Abstract: Retinal disease diagnosis is critical in preventing vision loss and reducing socioeconomic burdens. Globally, over 2.2 billion people are affected by some form of vision impairment, resulting in annual productivity losses estimated at $411 billion. Traditional manual grading of retinal fundus images by ophthalmologists is time-consuming and subjective. In contrast, deep learning has revolutionized medical diagnostics by automating retinal image analysis and achieving expert-level performance. In this study, we present EYE-DEX, an automated framework for classifying 10 retinal conditions using the large-scale Retinal Disease Dataset comprising 21,577 eye fundus images. We benchmark three pre-trained Convolutional Neural Network (CNN) models–VGG16, VGG19, and ResNet50–with our finetuned VGG16 achieving a state-of-the-art global benchmark test accuracy of 92.36%. To enhance transparency and explainability, we integrate the Gradient-weighted Class Activation Mapping (Grad-CAM) technique to generate visual explanations highlighting disease-specific regions, thereby fostering clinician trust and reliability in AI-assisted diagnostics.

[268] Asymmetric VAE for One-Step Video Super-Resolution Acceleration

Jianze Li,Yong Guo,Yulun Zhang,Xiaokang Yang

Main category: cs.CV

TL;DR: FastVSR通过高压缩率VAE（f16）和创新的训练策略，显著提升了视频超分辨率的推理效率，速度和性能均优于现有方法。

Details Motivation: 现有基于扩散模型的视频超分辨率方法虽减少了采样步数，但在推理效率上仍有优化空间，FastVSR旨在进一步提升效率。

Contribution: 1) 提出高压缩率VAE（f16）；2) 设计稳定的训练框架；3) 引入基于下界的训练策略；4) 实现显著的推理加速。

Method: 1) 采用f16 VAE结构实现空间压缩；2) 结合像素洗牌和通道复制进行上采样；3) 提出下界引导的训练策略优化收敛。

Result: FastVSR比多步模型快111.9倍，比现有一步模型快3.92倍，同时保持高性能。

Insight: 高压缩率VAE和简化训练目标是提升推理效率的关键，同时稳定训练过程。

Abstract: Diffusion models have significant advantages in the field of real-world video super-resolution and have demonstrated strong performance in past research. In recent diffusion-based video super-resolution (VSR) models, the number of sampling steps has been reduced to just one, yet there remains significant room for further optimization in inference efficiency. In this paper, we propose FastVSR, which achieves substantial reductions in computational cost by implementing a high compression VAE (spatial compression ratio of 16, denoted as f16). We design the structure of the f16 VAE and introduce a stable training framework. We employ pixel shuffle and channel replication to achieve additional upsampling. Furthermore, we propose a lower-bound-guided training strategy, which introduces a simpler training objective as a lower bound for the VAE’s performance. It makes the training process more stable and easier to converge. Experimental results show that FastVSR achieves speedups of 111.9 times compared to multi-step models and 3.92 times compared to existing one-step models. We will release code and models at https://github.com/JianzeLi-114/FastVSR.

[269] High-Order Progressive Trajectory Matching for Medical Image Dataset Distillation

Le Dong,Jinghao Bian,Jingyang Hou,Jingliang Hu,Yilei Shi,Weisheng Dong,Xiao Xiang Zhu,Lichao Mou

Main category: cs.CV

TL;DR: 这篇论文提出了一种高阶渐进轨迹匹配方法（HoP-TM），用于医学图像数据集蒸馏，解决了现有方法忽视中间优化状态信息的问题，提升了蒸馏性能并保护隐私。

Details Motivation: 医学图像分析中，数据共享因隐私法规和复杂机构协议受限。数据集蒸馏通过合成紧凑数据集来捕获真实大型数据集的关键信息，提供了一种解决方案。

Contribution: 1. 提出了一种形状势（shape-wise potential）来捕获参数轨迹的几何结构；2. 设计了易到难的渐进匹配策略，根据参数复杂度逐步处理。

Method: 结合形状势和渐进匹配策略的高阶轨迹匹配方法（HoP-TM），首次关注中间优化状态的信息。

Result: 实验表明，该方法在医学图像分类任务中提升了蒸馏性能，同时保持了与原始数据集训练相当的模型准确性。

Insight: 1. 中间优化状态的信息对数据集蒸馏至关重要；2. 渐进策略和几何结构捕获能够有效提升蒸馏质量。

Abstract: Medical image analysis faces significant challenges in data sharing due to privacy regulations and complex institutional protocols. Dataset distillation offers a solution to address these challenges by synthesizing compact datasets that capture essential information from real, large medical datasets. Trajectory matching has emerged as a promising methodology for dataset distillation; however, existing methods primarily focus on terminal states, overlooking crucial information in intermediate optimization states. We address this limitation by proposing a shape-wise potential that captures the geometric structure of parameter trajectories, and an easy-to-complex matching strategy that progressively addresses parameters based on their complexity. Experiments on medical image classification tasks demonstrate that our method improves distillation performance while preserving privacy and maintaining model accuracy comparable to training on the original datasets. Our code is available at https://github.com/Bian-jh/HoP-TM.

[270] Combining Discrepancy-Confusion Uncertainty and Calibration Diversity for Active Fine-Grained Image Classification

Yinghao Jin,Xi Yang

Main category: cs.CV

TL;DR: 论文提出了一种结合差异困惑不确定性和校准多样性的新方法DECERN，用于主动细粒度图像分类，通过多角度的信息度量提升了样本选择的有效性。

Details Motivation: 在细粒度图像分类中，由于类间差异微小，传统主动学习方法难以准确评估样本的信息量。DECERN旨在解决这一问题，更有效地选择信息丰富的样本。

Contribution: DECERN提出了一种结合差异困惑不确定性和校准多样性的多角度信息度量方法，显著提升了细粒度图像分类中的样本选择效果。

Method: 方法通过量化差异困惑不确定性（捕捉类别方向性和结构稳定性），并结合不确定性加权聚类实现样本多样性校准。

Result: 在7个细粒度图像数据集和26种实验设置中，DECERN优于现有方法，展示了其优越性能。

Insight: 通过融合局部特征和全局多样性，DECERN为细粒度图像分类中的主动学习提供了新思路。

Abstract: Active learning (AL) aims to build high-quality labeled datasets by iteratively selecting the most informative samples from an unlabeled pool under limited annotation budgets. However, in fine-grained image classification, assessing this informativeness is especially challenging due to subtle inter-class differences. In this paper, we introduce a novel method, combining discrepancy-confusion uncertainty and calibration diversity for active fine-grained image classification (DECERN), to effectively perceive the distinctiveness between fine-grained images and evaluate the sample value. DECERN introduces a multifaceted informativeness measure that combines discrepancy-confusion uncertainty and calibration diversity. The discrepancy-confusion uncertainty quantifies the category directionality and structural stability of fine-grained unlabeled data during local feature fusion. Subsequently, uncertainty-weighted clustering is performed to diversify the uncertainty samples. Then we calibrate the diversity to maximize the global diversity of the selected sample while maintaining its local representativeness. Extensive experiments conducted on 7 fine-grained image datasets across 26 distinct experimental settings demonstrate that our method achieves superior performance compared to state-of-the-art methods.

[271] Simulating Post-Neoadjuvant Chemotherapy Breast Cancer MRI via Diffusion Model with Prompt Tuning

Jonghun Kim,Hyunjin Park

Main category: cs.CV

TL;DR: 本文提出了一种基于扩散模型和提示调制的生成方法，用于从治疗前的DCE-MRI图像预测乳腺癌患者在接受新辅助化疗（NAC）后的MRI图像。该方法在图像质量和对肿瘤变化的反映上优于其他生成模型。

Details Motivation: 新辅助化疗（NAC）后的反应监测对乳腺癌治疗计划至关重要，但目前缺乏准确的预测方法。本文旨在通过生成治疗后MRI图像，为精准医疗提供支持。

Contribution: 主要贡献包括：（1）将扩散模型应用于治疗后MRI图像的生成；（2）引入提示调制技术以整合临床因素；（3）在图像质量和对肿瘤变化的反映上优于现有生成模型。

Method: 方法包括：从治疗前的DCE-MRI图像提取最大强度投影图像，利用扩散模型生成治疗后图像，并通过提示调制技术结合临床因素（如治疗时间和反应类型）优化生成结果。

Result: 结果显示，该方法在图像质量指标上优于其他生成模型，尤其是能更准确地反映pCR（病理完全缓解）相关的肿瘤大小变化。消融实验验证了提示调制的必要性。

Insight: 通过结合临床因素的提示调制，扩散模型在医学图像生成任务中表现出更强的适应性和准确性。该方法为个性化治疗提供了潜在的工具。

Abstract: Neoadjuvant chemotherapy (NAC) is a common therapy option before the main surgery for breast cancer. Response to NAC is monitored using follow-up dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI). Accurate prediction of NAC response helps with treatment planning. Here, we adopt maximum intensity projection images from DCE-MRI to generate post-treatment images (i.e., 3 or 12 weeks after NAC) from pre-treatment images leveraging the emerging diffusion model. We introduce prompt tuning to account for the known clinical factors affecting response to NAC. Our model performed better than other generative models in image quality metrics. Our model was better at generating images that reflected changes in tumor size according to pCR compared to other models. Ablation study confirmed the design choices of our method. Our study has the potential to help with precision medicine.

[272] Talk in Pieces, See in Whole: Disentangling and Hierarchical Aggregating Representations for Language-based Object Detection

Sojung An,Kwanyong Park,Yong Jae Lee,Donghyun Kim

Main category: cs.CV

TL;DR: 论文提出TaSe框架，通过解耦和分层聚合语言表示改进基于语言的物体检测能力，在复杂查询中表现显著优于现有方法。

Details Motivation: 现有视觉语言模型（VLMs）在复杂语言查询（如包含描述性属性和关系从句）中表现不佳，主要源于文本编码器未能区分目标对象与其属性、关系。

Contribution: 1) 创建分层的合成描述数据集；2) 提出Talk in Pieces模块解耦文本嵌入；3) 设计See in Whole模块分层聚合解耦后的表示。

Method: 通过解耦损失函数分离对象、属性和关系，再结合分层目标学习分层结构化嵌入。

Result: 在OmniLabel基准测试中性能提升24%。

Insight: 语言的结构性对多模态表示至关重要，解耦和分层聚合能显著提升复杂查询的感知能力。

Abstract: While vision-language models (VLMs) have made significant progress in multimodal perception (e.g., open-vocabulary object detection) with simple language queries, state-of-the-art VLMs still show limited ability to perceive complex queries involving descriptive attributes and relational clauses. Our in-depth analysis shows that these limitations mainly stem from text encoders in VLMs. Such text encoders behave like bags-of-words and fail to separate target objects from their descriptive attributes and relations in complex queries, resulting in frequent false positives. To address this, we propose restructuring linguistic representations according to the hierarchical relations within sentences for language-based object detection. A key insight is the necessity of disentangling textual tokens into core components-objects, attributes, and relations (“talk in pieces”)-and subsequently aggregating them into hierarchically structured sentence-level representations (“see in whole”). Building on this principle, we introduce the TaSe framework with three main contributions: (1) a hierarchical synthetic captioning dataset spanning three tiers from category names to descriptive sentences; (2) Talk in Pieces, the three-component disentanglement module guided by a novel disentanglement loss function, transforms text embeddings into subspace compositions; and (3) See in Whole, which learns to aggregate disentangled components into hierarchically structured embeddings with the guide of proposed hierarchical objectives. The proposed TaSe framework strengthens the inductive bias of hierarchical linguistic structures, resulting in fine-grained multimodal representations for language-based object detection. Experimental results under the OmniLabel benchmark show a 24% performance improvement, demonstrating the importance of linguistic compositionality.

[273] UniVid: The Open-Source Unified Video Model

Jiabin Luo,Junhui Lin,Zeyu Zhang,Biao Wu,Meng Fang,Ling Chen,Hao Tang

Main category: cs.CV

TL;DR: UniVid是一个统一视频模型，通过轻量级适配器将多模态大语言模型（MLLM）与扩散解码器结合，实现视频理解和生成。通过引入温度模态对齐和金字塔反射技术，提升了提示遵从性和高效时间推理。在多个基准测试中表现优异。

Details Motivation: 现有统一视频模型在流式生成时面临语义忠实度不足和跨模态注意力效率低的问题，同时扩展图像中心的MLLM到视频领域代价高昂。UniVid旨在解决这些问题。

Contribution: 1. 提出UniVid，结合MLLM与扩散解码器的统一架构；2. 引入温度模态对齐和金字塔反射技术，提升生成和理解能力；3. 在多个基准测试中实现SOTA性能。

Method: 1. 使用轻量级适配器耦合MLLM与扩散解码器；2. 温度模态对齐优化提示遵从性；3. 金字塔反射通过动态关键帧选择实现高效时间推理。

Result: 在VBench-Long上比EasyAnimateV5.1提升2.2%，在MSVD-QA和ActivityNet-QA上分别提升1.0%和3.3%的准确率。

Insight: 轻量级适配器和动态技术（如金字塔反射）可高效扩展MLLM到视频领域，同时保持生成和理解的性能。

Abstract: Unified video modeling that combines generation and understanding capabilities is increasingly important but faces two key challenges: maintaining semantic faithfulness during flow-based generation due to text-visual token imbalance and the limitations of uniform cross-modal attention across the flow trajectory, and efficiently extending image-centric MLLMs to video without costly retraining. We present UniVid, a unified architecture that couples an MLLM with a diffusion decoder through a lightweight adapter, enabling both video understanding and generation. We introduce Temperature Modality Alignment to improve prompt adherence and Pyramid Reflection for efficient temporal reasoning via dynamic keyframe selection. Extensive experiments on standard benchmarks demonstrate state-of-the-art performance, achieving a 2.2% improvement on VBench-Long total score compared to EasyAnimateV5.1, and 1.0% and 3.3% accuracy gains on MSVD-QA and ActivityNet-QA, respectively, compared with the best prior 7B baselines.

[274] BALR-SAM: Boundary-Aware Low-Rank Adaptation of SAM for Resource-Efficient Medical Image Segmentation

Zelin Liu,Sicheng Dong,Bocheng Li,Yixuan Yang,Jiacheng Ruan,Chenxu Zhou,Suncheng Xiang

Main category: cs.CV

TL;DR: BALR-SAM是一种边界感知低秩适配框架，旨在高效地将SAM模型适配到医学图像分割任务中，通过减少参数需求和资源消耗，同时保持高性能。

Details Motivation: 预训练的大规模视觉基础模型（如SAM）在医学图像分割中表现不佳，因其缺乏领域特异性。如何在临床实践中高效地微调这些模型，以满足资源限制并保持性能，是一个关键挑战。

Contribution: 1. 提出BALR-SAM框架，结合边界感知和低秩适配技术；2. 引入Complementary Detail Enhancement Network（CDEN）捕捉边界特征；3. 在SAM的ViT块中集成低秩适配器，显著减少参数量；4. 提出低秩张量注意力机制，减少内存占用并加速推理。

Method: 1. CDEN利用深度可分离卷积和多尺度融合捕捉边界特征；2. 在SAM的ViT块中引入低秩适配器；3. 在掩码解码器中采用低秩张量注意力机制。

Result: 在标准医学分割数据集上，BALR-SAM无需提示即可超越多个SOTA方法（包括完全微调的MedSAM），仅更新1.8%的参数（11.7M）。

Insight: 边界感知和低秩适配技术结合能显著提升医学图像分割的性能和效率，为资源受限场景提供了一种高效解决方案。

Abstract: Vision foundation models like the Segment Anything Model (SAM), pretrained on large-scale natural image datasets, often struggle in medical image segmentation due to a lack of domain-specific adaptation. In clinical practice, fine-tuning such models efficiently for medical downstream tasks with minimal resource demands, while maintaining strong performance, is challenging. To address these issues, we propose BALR-SAM, a boundary-aware low-rank adaptation framework that enhances SAM for medical imaging. It combines three tailored components: (1) a Complementary Detail Enhancement Network (CDEN) using depthwise separable convolutions and multi-scale fusion to capture boundary-sensitive features essential for accurate segmentation; (2) low-rank adapters integrated into SAM’s Vision Transformer blocks to optimize feature representation and attention for medical contexts, while simultaneously significantly reducing the parameter space; and (3) a low-rank tensor attention mechanism in the mask decoder, cutting memory usage by 75% and boosting inference speed. Experiments on standard medical segmentation datasets show that BALR-SAM, without requiring prompts, outperforms several state-of-the-art (SOTA) methods, including fully fine-tuned MedSAM, while updating just 1.8% (11.7M) of its parameters.

[275] Forge4D: Feed-Forward 4D Human Reconstruction and Interpolation from Uncalibrated Sparse-view Videos

Yingdong Hu,Yisheng He,Jinnan Chen,Weihao Yuan,Kejie Qiu,Zehong Lin,Siyu Zhu,Zilong Dong,Jun Zhang

Main category: cs.CV

TL;DR: Forge4D提出了一种高效的4D人体重建和插值模型，通过结合流式3D高斯重建和稠密运动预测，从未校准的稀疏视角视频中重建动态3D人体，并支持新视角和新时间的合成。

Details Motivation: 现有方法在重建速度或生成新时间表征方面存在局限，影响了动态3D人体重建的应用。

Contribution: 1. 提出了Forge4D模型，解决了从未校准稀疏视角视频中高效重建4D人体的需求。2. 设计了流式3D高斯重建方法和稠密运动预测模块，支持新时间合成。3. 提出了自监督的重定向损失和遮挡感知的光流损失，优化运动预测。

Method: 1. 流式3D高斯重建：从未校准稀疏视角图像重建静态3D高斯，并通过可学习状态令牌保持时间一致性。2. 稠密运动预测：预测相邻帧间的稠密运动，结合遮挡感知的高斯融合插值任意时间点。3. 自监督损失：包括重定向损失和光流损失，确保运动一致性。

Result: 实验验证了模型在领域内外数据集上的有效性。

Insight: 通过联合任务设计和自监督损失，Forge4D在高效性和生成能力上取得了平衡，为动态4D重建提供了新思路。

Abstract: Instant reconstruction of dynamic 3D humans from uncalibrated sparse-view videos is critical for numerous downstream applications. Existing methods, however, are either limited by the slow reconstruction speeds or incapable of generating novel-time representations. To address these challenges, we propose Forge4D, a feed-forward 4D human reconstruction and interpolation model that efficiently reconstructs temporally aligned representations from uncalibrated sparse-view videos, enabling both novel view and novel time synthesis. Our model simplifies the 4D reconstruction and interpolation problem as a joint task of streaming 3D Gaussian reconstruction and dense motion prediction. For the task of streaming 3D Gaussian reconstruction, we first reconstruct static 3D Gaussians from uncalibrated sparse-view images and then introduce learnable state tokens to enforce temporal consistency in a memory-friendly manner by interactively updating shared information across different timestamps. For novel time synthesis, we design a novel motion prediction module to predict dense motions for each 3D Gaussian between two adjacent frames, coupled with an occlusion-aware Gaussian fusion process to interpolate 3D Gaussians at arbitrary timestamps. To overcome the lack of the ground truth for dense motion supervision, we formulate dense motion prediction as a dense point matching task and introduce a self-supervised retargeting loss to optimize this module. An additional occlusion-aware optical flow loss is introduced to ensure motion consistency with plausible human movement, providing stronger regularization. Extensive experiments demonstrate the effectiveness of our model on both in-domain and out-of-domain datasets. Project page and code at: https://zhenliuzju.github.io/huyingdong/Forge4D.

[276] Scalable Audio-Visual Masked Autoencoders for Efficient Affective Video Facial Analysis

Xuecheng Wu,Junxiao Xue,Xinyi Yin,Yunyun Shi,Liangyu Fu,Danlei Huang,Yifan Wang,Jia Zhang,Jiayu Nie,Jun Wang

Main category: cs.CV

TL;DR: 论文提出了一种名为AVF-MAE++的自监督学习框架，专注于情感视频面部分析（AVFA），通过双模态掩码策略和改进的跨模态相关性学习模块，解决了数据稀缺和模态相关性建模的挑战。

Details Motivation: 情感视频面部分析（AVFA）领域面临数据稀缺和多模态相关性建模的挑战，本文旨在通过可扩展的自监督学习方法解决这些问题。

Contribution: 1. 提出AVF-MAE++框架，设计双模态掩码策略和改进的跨模态相关性学习模块；2. 引入渐进式语义注入策略，优化训练过程；3. 在17个数据集上验证了方法的有效性。

Method: 1. 使用双模态掩码策略对音频和视觉模态进行掩码；2. 提出迭代音频-视觉相关性学习模块；3. 采用渐进式语义注入策略分阶段训练模型。

Result: 在17个数据集上的实验表明，AVF-MAE++在多个基准任务中取得了最优性能。

Insight: 1. 可扩展性是提升AVFA性能的关键；2. 跨模态相关性建模对自监督学习至关重要。

Abstract: Affective video facial analysis (AVFA) has emerged as a key research field for building emotion-aware intelligent systems, yet this field continues to suffer from limited data availability. In recent years, the self-supervised learning (SSL) technique of Masked Autoencoders (MAE) has gained momentum, with growing adaptations in its audio-visual contexts. While scaling has proven essential for breakthroughs in general multi-modal learning domains, its specific impact on AVFA remains largely unexplored. Another core challenge in this field is capturing both intra- and inter-modal correlations through scalable audio-visual representations. To tackle these issues, we propose AVF-MAE++, a family of audio-visual MAE models designed to efficiently investigate the scaling properties in AVFA while enhancing cross-modal correlation modeling. Our framework introduces a novel dual masking strategy across audio and visual modalities and strengthens modality encoders with a more holistic design to better support scalable pre-training. Additionally, we present the Iterative Audio-Visual Correlation Learning Module, which improves correlation learning within the SSL paradigm, bridging the limitations of previous methods. To support smooth adaptation and reduce overfitting risks, we further introduce a progressive semantic injection strategy, organizing the model training into three structured stages. Extensive experiments conducted on 17 datasets, covering three major AVFA tasks, demonstrate that AVF-MAE++ achieves consistent state-of-the-art performance across multiple benchmarks. Comprehensive ablation studies further highlight the importance of each proposed component and provide deeper insights into the design choices driving these improvements. Our code and models have been publicly released at Github.

[277] EVLF-FM: Explainable Vision Language Foundation Model for Medicine

Yang Bai,Haoran Cheng,Yang Zhou,Jun Zhou,Arun Thirunavukarasu,Yuhe Ke,Jie Yao,Kanae Fukutsu,Chrystie Wan Ning Quek,Ashley Hong,Laura Gutierrez,Zhen Ling Teo,Darren Shu Jeng Ting,Brian T. Soetikno,Christopher S. Nielsen,Tobias Elze,Zengxiang Li,Linh Le Dinh,Hiok Hong Chan,Victor Koh,Marcus Tan,Kelvin Z. Li,Leonard Yip,Ching Yu Cheng,Yih Chung Tham,Gavin Siew Wei Tan,Leopold Schmetterer,Marcus Ang,Rahat Hussain,Jod Mehta,Tin Aung,Lionel Tim-Ee Cheng,Tran Nguyen Tuan Anh,Chee Leong Cheng,Tien Yin Wong,Nan Liu,Iain Beehuat Tan,Soon Thye Lim,Eyal Klang,Tony Kiat Hon Lim,Rick Siow Mong Goh,Yong Liu,Daniel Shu Wei Ting

Main category: cs.CV

TL;DR: EVLF-FM是一种多模态视觉语言基础模型，旨在统统一广泛的诊断能力与细粒度的可解释性，在多个临床专业中表现优异。

Details Motivation: 目前的医学AI基础模型存在局限性，如模态单一和缺乏透明的推理过程，阻碍了临床应用。EVLF-FM旨在解决这些问题。

Contribution: 开发了一种具有可解释性和推理能力的多模态视觉语言基础模型，支持多种疾病诊断和视觉问答，并在性能和可解释性上超越现有模型。

Method: 采用混合训练策略，结合监督学习和视觉强化微调，实现像素级视觉定位和逐步推理能力。

Result: 在内部验证中，EVLF-FM的准确率和F1分数最高；在外部验证中也表现出强大的零样本和小样本性能。

Insight: 可解释性和多模态能力是提升医学AI临床信任的关键，EVLF-FM的成功展示了基础模型在该领域的潜力。

Abstract: Despite the promise of foundation models in medical AI, current systems remain limited - they are modality-specific and lack transparent reasoning processes, hindering clinical adoption. To address this gap, we present EVLF-FM, a multimodal vision-language foundation model (VLM) designed to unify broad diagnostic capability with fine-grain explainability. The development and testing of EVLF-FM encompassed over 1.3 million total samples from 23 global datasets across eleven imaging modalities related to six clinical specialties: dermatology, hepatology, ophthalmology, pathology, pulmonology, and radiology. External validation employed 8,884 independent test samples from 10 additional datasets across five imaging modalities. Technically, EVLF-FM is developed to assist with multiple disease diagnosis and visual question answering with pixel-level visual grounding and reasoning capabilities. In internal validation for disease diagnostics, EVLF-FM achieved the highest average accuracy (0.858) and F1-score (0.797), outperforming leading generalist and specialist models. In medical visual grounding, EVLF-FM also achieved stellar performance across nine modalities with average mIOU of 0.743 and Acc@0.5 of 0.837. External validations further confirmed strong zero-shot and few-shot performance, with competitive F1-scores despite a smaller model size. Through a hybrid training strategy combining supervised and visual reinforcement fine-tuning, EVLF-FM not only achieves state-of-the-art accuracy but also exhibits step-by-step reasoning, aligning outputs with visual evidence. EVLF-FM is an early multi-disease VLM model with explainability and reasoning capabilities that could advance adoption of and trust in foundation models for real-world clinical deployment.

[278] FreeAction: Training-Free Techniques for Enhanced Fidelity of Trajectory-to-Video Generation

Seungwook Kim,Seunghyeon Lee,Minsu Cho

Main category: cs.CV

TL;DR: 论文介绍了两项无需训练的推理技术，提升基于扩散模型的轨迹到视频生成的真实性和可控性。

Details Motivation: 生成逼真的机器人视频对于构建有效的世界模型和机器人基础模型至关重要。现有的方法往往将被动作向量视为被动条件信号，未能充分利用其指导作用。

Contribution: 提出两项无需训练的推理技术：动作缩放的无分类器引导和动作缩放的噪声截断，主动利用动作参数提升视频生成的质量和可控性。

Method: 1. 动作缩放的无分类器引导：根据动作幅度动态调整引导强度；2. 动作缩放的噪声截断：调整初始噪声分布以匹配目标运动动态。

Result: 在真实机器人操作数据集上的实验表明，这些技术显著提升了动作一致性和视觉质量。

Insight: 主动利用动作参数的动态调节能力，可以显著提升扩散模型中轨迹到视频生成的性能，尤其是在多场景机器人环境下。

Abstract: Generating realistic robot videos from explicit action trajectories is a critical step toward building effective world models and robotics foundation models. We introduce two training-free, inference-time techniques that fully exploit explicit action parameters in diffusion-based robot video generation. Instead of treating action vectors as passive conditioning signals, our methods actively incorporate them to guide both the classifier-free guidance process and the initialization of Gaussian latents. First, action-scaled classifier-free guidance dynamically modulates guidance strength in proportion to action magnitude, enhancing controllability over motion intensity. Second, action-scaled noise truncation adjusts the distribution of initially sampled noise to better align with the desired motion dynamics. Experiments on real robot manipulation datasets demonstrate that these techniques significantly improve action coherence and visual quality across diverse robot environments.

[279] Latent Visual Reasoning

Bangzheng Li,Ximeng Sun,Jiang Liu,Ze Wang,Jialian Wu,Xiaodong Yu,Hao Chen,Emad Barsoum,Muhao Chen,Zicheng Liu

Main category: cs.CV

TL;DR: 论文提出了一种名为Latent Visual Reasoning (LVR)的新方法，通过在视觉嵌入空间中进行自回归推理，显著提升了多模态大语言模型在视觉问题解答任务中的表现。

Details Motivation: 现有的多模态大语言模型虽然通过思维链（CoT）推理提高了任务表现，但其推理过程仍局限于语言空间，视觉信息被视为静态前提条件。这限制了模型在细粒度视觉理解任务中的潜力。

Contribution: 1. 提出了LVR框架，直接在视觉嵌入空间进行推理；2. 结合GRPO算法优化潜在推理与文本生成；3. 在MMVP等视觉任务中取得了显著效果提升。

Method: 1. 使用视觉编码器将图像投影到与语言模型共享的语义空间中；2. 训练语言模型生成潜在状态以重构关键视觉信息；3. 结合GRPO算法进行强化学习。

Result: LVR在MMVP任务中达到了71.67%的准确率，显著优于Qwen2.5-VL的66.67%。

Insight: 直接在视觉空间进行推理能更好地捕捉细粒度视觉信息，提升多模态任务的性能。

Abstract: Multimodal Large Language Models (MLLMs) have achieved notable gains in various tasks by incorporating Chain-of-Thought (CoT) reasoning in language spaces. Recent work extends this direction by leveraging external tools for visual editing, thereby enhancing the visual signal along the reasoning trajectories. Nevertheless, these approaches remain fundamentally constrained: reasoning is still confined to the language space, with visual information treated as static preconditions. We introduce Latent Visual Reasoning (LVR), a new paradigm that enables autoregressive reasoning directly in the visual embedding space. A visual encoder first projects images into visual tokens within a joint semantic space shared with the language model. The language model is then trained to generate latent states that reconstruct key visual tokens critical for answering the query, constituting the process of latent visual reasoning. By interleaving LVR with standard text generation, our model achieves substantial gains on perception-intensive visual question answering tasks. In addition, we adapt the GRPO algorithm to conduct reinforcement learning on latent reasoning, further balancing LVR and textual generation. We show that LVR substantially improves fine-grained visual understanding and perception, achieving 71.67% on MMVP compared to 66.67% with Qwen2.5-VL. Code base and model weights will be released later.

[280] When MLLMs Meet Compression Distortion: A Coding Paradigm Tailored to MLLMs

Jinming Liu,Zhaoyang Jia,Jiahao Li,Bin Li,Xin Jin,Wenjun Zeng,Yan Lu

Main category: cs.CV

TL;DR: 该论文分析了压缩失真对多模态大语言模型（MLLMs）的影响，并提出了一种名为CoTAM的图像编码方法，旨在自适应保护多级特征以满足下游任务需求。实验表明，该方法能节省35.99%的比特率，同时保持任务性能。

Details Motivation: MLLMs通常部署在云端，需要高效的压缩技术以减少边缘设备传输带宽。传统图像编解码器以人类视觉系统为目标优化，不适合MLLMs的多任务需求。

Contribution: 1. 系统分析了压缩失真对MLLMs的影响；2. 提出了一种针对MLLMs的图像编码方法（CoTAM），能自适应保护多级特征；3. 实验证明其比特率节省显著优于现有方法。

Method: 1. 利用CLIP浅层注意力生成重要性图进行比特分配；2. 解码器集成了轻量级适配器和多级损失函数，确保低级细节和高级语义的忠实重构。

Result: CoTAM在MLLMs任务上保持性能的同时，节省了35.99%的比特率，超过了现有神经编解码器。

Insight: 压缩失真对不同层次特征的影响不均匀，针对MLLMs的设计需兼顾多级特征保护和任务需求。

Abstract: The increasing deployment of powerful Multimodal Large Language Models (MLLMs), typically hosted on cloud platforms, urgently requires effective compression techniques to efficiently transmit signal inputs (e.g., images, videos) from edge devices with minimal bandwidth usage. However, conventional image codecs are optimized for fidelity to serve the Human Visual System (HVS) and ill-suited for MLLMs, in which diverse downstream tasks are jointly considered. In this paper, we first systematically analyze the impact of compression artifacts on several mainstream MLLMs. We find that: Compression distortion unevenly impacts different-level image features, leading to varying effects on MLLMs’ downstream tasks depending on their feature-level reliance. Motivated by this discovery, we propose an image Codec TAilored to MLLMs (CoTAM) designed to adaptively protect multi-level features and suit different demands of downstream tasks. The encoder leverages CLIP’s shallow-layer attention to generate an importance map for bit allocation, preserving critical semantic regions. Concurrently, the decoder integrates a lightweight adapter with a multi-level loss function to ensure the faithful reconstruction both of low-level details and high-level semantic context for robust synthesis of cross-level features. Extensive experiments validate that our method achieves up to 35.99% bitrate saving while maintaining the same performance on the MLLM tasks, outperforming previous SOTA neural codecs.

[281] S$^2$NN: Sub-bit Spiking Neural Networks

Wenjie Wei,Malu Zhang,Jieyuan Zhang,Ammar Belatreche,Shuai Wang,Yimeng Shan,Hanwen Liu,Honglin Cao,Guoqing Wang,Yang Yang,Haizhou Li

Main category: cs.CV

TL;DR: 论文提出了一种称为S²NN的亚比特脉冲神经网络，通过将权重表示为小于一比特，进一步探索SNNs的压缩和加速潜力，解决了大规模网络存储和计算的需求问题。

Details Motivation: 尽管二元脉冲神经网络（SNNs）在能效方面表现优异，但其在大规模网络中的存储和计算需求仍然较高。为了进一步压缩和加速SNNs，论文提出了亚比特权重的表示方法。

Contribution: 论文的主要贡献包括：1）提出了S²NN基线，利用核的聚类模式；2）设计了OS-Quant方法，解决异常值引起的编码选择偏差；3）提出了MPFD方法，通过教师模型提升压缩网络的性能。

Method: 论文采用了三种方法：1）建立S²NN基线；2）提出OS-Quant方法，优化异常值的编码选择；3）提出MPFD方法，利用膜电位特征蒸馏提升性能。

Result: 实验结果显示，S²NN在视觉和非视觉任务中均优于现有的量化SNNs，且在性能和效率上表现优异，适用于边缘计算。

Insight: 研究表明，亚比特权重的表示方法可以显著降低SNNs的存储和计算需求，同时通过异常值自适应和特征蒸馏提升性能。

Abstract: Spiking Neural Networks (SNNs) offer an energy-efficient paradigm for machine intelligence, but their continued scaling poses challenges for resource-limited deployment. Despite recent advances in binary SNNs, the storage and computational demands remain substantial for large-scale networks. To further explore the compression and acceleration potential of SNNs, we propose Sub-bit Spiking Neural Networks (S$^2$NNs) that represent weights with less than one bit. Specifically, we first establish an S$^2$NN baseline by leveraging the clustering patterns of kernels in well-trained binary SNNs. This baseline is highly efficient but suffers from \textit{outlier-induced codeword selection bias} during training. To mitigate this issue, we propose an \textit{outlier-aware sub-bit weight quantization} (OS-Quant) method, which optimizes codeword selection by identifying and adaptively scaling outliers. Furthermore, we propose a \textit{membrane potential-based feature distillation} (MPFD) method, improving the performance of highly compressed S$^2$NN via more precise guidance from a teacher model. Extensive results on vision and non-vision tasks reveal that S$^2$NN outperforms existing quantized SNNs in both performance and efficiency, making it promising for edge computing applications.

[282] Skeleton-based Robust Registration Framework for Corrupted 3D Point Clouds

Yongqiang Wang,Weigang Li,Wenping Liu,Zhiqiang Tian,Jinling Li

Main category: cs.CV

TL;DR: 论文提出了一种基于骨架的鲁棒点云配准框架（SRRF），通过引入抗噪的骨架表示，结合点云和骨架的变换，显著提升了在噪声、密度失真和几何变形等条件下的配准性能。

Details Motivation: 由于点云在现实场景中常受到传感器限制、环境噪声和预处理错误的干扰，现有配准方法依赖直接点匹配或表面特征提取，容易受到这些干扰影响，导致配准精度下降。

Contribution: 提出了一个基于骨架的鲁棒配准框架（SRRF），引入了一种抗噪的骨架表示方法，并结合了点云和骨架的变换来提高配准鲁棒性和精度。

Method: 框架将骨架结构整合到配准过程中，结合了点云对齐和骨架对齐的变换结果，并使用一种分布距离损失函数来确保源和目标骨架的一致性。

Result: 在多种噪声场景下的实验表明，SRRF在配准精度上显著优于现有方法，尤其是在密度失真、噪声污染和几何变形等条件下表现突出。

Insight: 通过同时考虑局部几何特征和骨架结构的全局稳定性，SRRF能够在复杂的真实场景中实现鲁棒且准确的点云配准。

Abstract: Point cloud registration is fundamental in 3D vision applications, including autonomous driving, robotics, and medical imaging, where precise alignment of multiple point clouds is essential for accurate environment reconstruction. However, real-world point clouds are often affected by sensor limitations, environmental noise, and preprocessing errors, making registration challenging due to density distortions, noise contamination, and geometric deformations. Existing registration methods rely on direct point matching or surface feature extraction, which are highly susceptible to these corruptions and lead to reduced alignment accuracy. To address these challenges, a skeleton-based robust registration framework is presented, which introduces a corruption-resilient skeletal representation to improve registration robustness and accuracy. The framework integrates skeletal structures into the registration process and combines the transformations obtained from both the corrupted point cloud alignment and its skeleton alignment to achieve optimal registration. In addition, a distribution distance loss function is designed to enforce the consistency between the source and target skeletons, which significantly improves the registration performance. This framework ensures that the alignment considers both the original local geometric features and the global stability of the skeleton structure, resulting in robust and accurate registration results. Experimental evaluations on diverse corrupted datasets demonstrate that SRRF consistently outperforms state-of-the-art registration methods across various corruption scenarios, including density distortions, noise contamination, and geometric deformations. The results confirm the robustness of SRRF in handling corrupted point clouds, making it a potential approach for 3D perception tasks in real-world scenarios.

[283] Robust Partial 3D Point Cloud Registration via Confidence Estimation under Global Context

Yongqiang Wang,Weigang Li,Wenping Liu,Zhe Xu,Zhiqiang Tian

Main category: cs.CV

TL;DR: 本文提出CEGC框架，通过全局上下文下的置信度估计，解决了部分点云配准中的结构模糊性和噪声问题。其结合语义描述符和几何相似性的混合重叠置信度模块，提高了配准的准确性和鲁棒性。

Details Motivation: 部分点云配准在自主感知和3D场景理解中至关重要，但由于结构模糊性、部分可见性和噪声等问题，现有方法效果有限。

Contribution: 提出CEGC框架，首次联合建模重叠置信度和对应关系可靠性，在全局上下文中自适应地加权不确定区域，提升配准性能。

Method: 1）混合重叠置信度估计模块；2）基于全局注意力的上下文感知匹配策略；3）可微分加权奇异值分解求解器。

Result: 在ModelNet40、ScanObjectNN和7Scenes数据集上，CEGC在准确性、鲁棒性和泛化性上优于现有方法。

Insight: 通过全局上下文建模自适应加权策略，CEGC显著提升了部分点云配准的性能，为解决复杂场景中的配准问题提供了新思路。

Abstract: Partial point cloud registration is essential for autonomous perception and 3D scene understanding, yet it remains challenging owing to structural ambiguity, partial visibility, and noise. We address these issues by proposing Confidence Estimation under Global Context (CEGC), a unified, confidence-driven framework for robust partial 3D registration. CEGC enables accurate alignment in complex scenes by jointly modeling overlap confidence and correspondence reliability within a shared global context. Specifically, the hybrid overlap confidence estimation module integrates semantic descriptors and geometric similarity to detect overlapping regions and suppress outliers early. The context-aware matching strategy smitigates ambiguity by employing global attention to assign soft confidence scores to correspondences, improving robustness. These scores guide a differentiable weighted singular value decomposition solver to compute precise transformations. This tightly coupled pipeline adaptively down-weights uncertain regions and emphasizes contextually reliable matches. Experiments on ModelNet40, ScanObjectNN, and 7Scenes 3D vision datasets demonstrate that CEGC outperforms state-of-the-art methods in accuracy, robustness, and generalization. Overall, CEGC offers an interpretable and scalable solution to partial point cloud registration under challenging conditions.

[284] SVGThinker: Instruction-Aligned and Reasoning-Driven Text-to-SVG Generation

Hanqi Chen,Zhongyin Zhao,Ye Chen,Zhujin Liang,Bingbing Ni

Main category: cs.CV

TL;DR: SVGThinker是一个基于大型语言模型（LLM）的文本到SVG生成框架，通过推理驱动和指令对齐的方法，解决了传统方法的泛化能力弱和指令遵循性差的问题。

Details Motivation: 传统文本到SVG生成方法存在泛化能力不足和指令遵循性差的问题，限制了SVG的实用性和编辑性。

Contribution: SVGThinker提出了一种推理驱动的框架，支持完整的SVG原语，生成稳定、高质量且可编辑的SVG。

Method: 框架分为两步：首先通过多模态模型标注图像和代码，然后训练一个LLM，暴露其链式推理过程以减少错误。

Result: 实验表明，SVGThinker生成的SVG在稳定性、编辑性和质量上均优于现有方法，保留了矢量图形的结构优势。

Insight: SVGThinker不仅提升了生成质量，还支持精确的分层编辑，为设计和自动化图形生成提供了新方向。

Abstract: Scalable Vector Graphics (SVG) is a code-based representation for 2D visuals. Leveraging recent advances in large language models (LLMs), we study text-to-SVG generation and address two persistent gaps: weak generalization and poor adherence to input instructions. We present SVGThinker, a reasoning-driven framework that aligns the production of SVG code with the visualization process and supports the full set of SVG primitives. Our pipeline first renders each primitive in sequence and uses a multimodal model to annotate the image and code; we then build stepwise updates that mirror the incremental addition of primitives. On this data, we train an LLM with supervised fine-tuning that exposes its chain-of-thought as intermediate reasoning, improving robustness and reducing errors and hallucinations. Experiments against state-of-the-art baselines show that SVGThinker produces more stable, editable, and higher-quality SVGs while preserving the structural advantages of vector graphics. Unlike image-based methods, our outputs enable precise and hierarchical editing, opening new directions for design, content creation, and automated graphics generation.

[285] FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame Spotlighting

Zefeng He,Xiaoye Qu,Yafu Li,Siyuan Huang,Daizong Liu,Yu Cheng

Main category: cs.CV

TL;DR: FrameThinker是一个新型框架，通过多轮帧聚焦技术使大视觉语言模型（LVLMs）能够高效处理长视频理解和推理任务，显著减少帧处理数量并提升性能。

Details Motivation: 当前LVLMs在长视频推理中存在均匀帧采样和静态文本推理的效率低下问题，无法处理视觉密集型任务，因此提出FrameThinker以解决这些挑战。

Contribution: 1. 提出FrameThinker框架，支持LVLMs迭代式视频内容查询；2. 设计两阶段训练策略（SFT和RL），优化动作选择和奖励函数；3. 在多个基准测试中显著提升性能，并减少帧处理数量。

Method: 1. 使用监督微调（SFT）训练基础动作能力；2. 通过强化学习（RL）优化决策策略，深入探索奖励设计；3. 引入多轮帧聚焦技术。

Result: FrameThinker在LongVideo-Reason上达到76.1%准确率（仅用20.6帧），远超基线模型（+10.4%），且帧处理数量大幅减少。

Insight: 迭代式帧聚焦和多阶段训练策略（SFT+RL）可显著提升LVLMs的长视频处理效率和性能。

Abstract: While Large Vision-Language Models (LVLMs) have achieved substantial progress in video understanding, their application to long video reasoning is hindered by uniform frame sampling and static textual reasoning, which are inefficient and struggle to handle visually intensive video tasks. To overcome these challenges, in this paper, we introduce the concept of thinking with long videos and propose a novel framework FrameThinker. Within this framework, LVLMs are able to iteratively interrogate video content. Developing such video reasoning capabilities in LVLMs presents notable challenges, particularly in adapting the model to new video actions (e.g. select frame), and designing reward functions to guide LVLMs to adopt the newly introduced action. To solve these challenges, we propose a two-phase training strategy, first employing Supervised Fine-Tuning (SFT) to instill fundamental action capabilities, followed by Reinforcement Learning (RL) to optimize a strategic decision-making policy.Notably, in this RL phase, we conduct an in-depth and comprehensive exploration of the reward design for each action and format reward. Extensive experiments on reasoning benchmarks like Video-Holmes, LongVideo-Reason, and long-video understanding benchmarks such as LongVideoBench, MLVU, VideoMME, and LVBench, demonstrate that FrameThinker achieves a significant average improvement of +10.4% over baselines while drastically reducing the number of processed frames. Most notably, our 7B model, FrameThinker establishes a new state-of-the-art on LongVideo-Reason, achieving 76.1% accuracy using an average of only 20.6 frames. This not only outperforms the competitive LongVILA-R1 (72.0%) but does so with over 20x fewer frames (vs. 512), demonstrating unparalleled efficiency and effectiveness.

[286] OMeGa: Joint Optimization of Explicit Meshes and Gaussian Splats for Robust Scene-Level Surface Reconstruction

Yuhang Cao,Haojun Yan,Danya Yao

Main category: cs.CV

TL;DR: OMeGa是一个联合优化显式三角网格和2D高斯溅射的端到端框架，通过灵活的绑定策略和网格约束，显著提升了室内纹理缺失区域的重建精度。

Details Motivation: 现有方法在纹理缺失的室内区域重建几何不准确，且网格提取与优化过程解耦，无法利用网格几何指导溅射优化。OMeGa旨在解决这些问题。

Contribution: 1）提出联合优化显式网格和高斯溅射的端到端框架；2）结合网格约束和单目法线监督，提升几何学习；3）提出启发式迭代网格细化策略，优化重建细节。

Method: 通过绑定策略将高斯溅射的空间属性表达在网格框架中，保留纹理属性于溅射；集成网格约束和法线监督；采用迭代网格细化策略分割高误差面并修剪不可靠面。

Result: 在室内重建基准上实现SOTA，Chamfer-$L_1$误差较基线降低47.3%，同时保持新颖视角渲染质量。

Insight: 联合优化显式网格和高斯溅射能有效解决纹理缺失区域的几何重建问题，网格几何可引导溅射优化，提升整体重建质量和鲁棒性。

Abstract: Neural rendering with Gaussian splatting has advanced novel view synthesis, and most methods reconstruct surfaces via post-hoc mesh extraction. However, existing methods suffer from two limitations: (i) inaccurate geometry in texture-less indoor regions, and (ii) the decoupling of mesh extraction from optimization, thereby missing the opportunity to leverage mesh geometry to guide splat optimization. In this paper, we present OMeGa, an end-to-end framework that jointly optimizes an explicit triangle mesh and 2D Gaussian splats via a flexible binding strategy, where spatial attributes of Gaussian Splats are expressed in the mesh frame and texture attributes are retained on splats. To further improve reconstruction accuracy, we integrate mesh constraints and monocular normal supervision into the optimization, thereby regularizing geometry learning. In addition, we propose a heuristic, iterative mesh-refinement strategy that splits high-error faces and prunes unreliable ones to further improve the detail and accuracy of the reconstructed mesh. OMeGa achieves state-of-the-art performance on challenging indoor reconstruction benchmarks, reducing Chamfer-$L_1$ by 47.3% over the 2DGS baseline while maintaining competitive novel-view rendering quality. The experimental results demonstrate that OMeGa effectively addresses prior limitations in indoor texture-less reconstruction.

[287] Towards Foundation Models for Cryo-ET Subtomogram Analysis

Runmin Jiang,Wanyue Feng,Yuntian Yang,Shriya Pingulkar,Hong Wang,Xi Xiao,Xiaoyu Cao,Genpei Zhang,Xiao Wang,Xiaolong Wu,Tianyang Wang,Yang Liu,Xingjian Li,Min Xu

Main category: cs.CV

TL;DR: 该论文致力于为冷冻电子断层扫描（cryo-ET）的子断层图分析开发基础模型，通过生成大规模合成数据、设计自适应相位标记化的视觉Transformer，以及提出抗噪声对比学习策略，显著提升了子断层图的分类、对齐和平均任务的性能。

Details Motivation: 冷冻电子断层扫描的子断层图分析面临标注稀缺、噪声严重和泛化能力差的问题，限制了结构的有效解析。

Contribution: 1. 引入CryoEngine合成数据生成器；2. 设计了APT-ViT模型，增强几何和语义变化的鲁棒性；3. 提出NRCL策略，提升噪声条件下的表示学习稳定性。

Method: 1. 使用CryoEngine生成大规模合成数据；2. 采用APT-ViT模型，结合自适应相位标记化；3. 应用NRCL策略进行噪声条件下的对比学习。

Result: 在24个合成和真实数据集上验证了在子断层图三大任务上的SOTA性能，并对未见数据集表现出强泛化能力。

Insight: 通过合成数据预训练和噪声鲁棒性设计，可以有效解决冷冻电子断层扫描中低信噪比和标注稀缺的挑战。

Abstract: Cryo-electron tomography (cryo-ET) enables in situ visualization of macromolecular structures, where subtomogram analysis tasks such as classification, alignment, and averaging are critical for structural determination. However, effective analysis is hindered by scarce annotations, severe noise, and poor generalization. To address these challenges, we take the first step towards foundation models for cryo-ET subtomograms. First, we introduce CryoEngine, a large-scale synthetic data generator that produces over 904k subtomograms from 452 particle classes for pretraining. Second, we design an Adaptive Phase Tokenization-enhanced Vision Transformer (APT-ViT), which incorporates adaptive phase tokenization as an equivariance-enhancing module that improves robustness to both geometric and semantic variations. Third, we introduce a Noise-Resilient Contrastive Learning (NRCL) strategy to stabilize representation learning under severe noise conditions. Evaluations across 24 synthetic and real datasets demonstrate state-of-the-art (SOTA) performance on all three major subtomogram tasks and strong generalization to unseen datasets, advancing scalable and robust subtomogram analysis in cryo-ET.

[288] Similarity-Aware Selective State-Space Modeling for Semantic Correspondence

Seungwook Kim,Minsu Cho

Main category: cs.CV

TL;DR: MambaMatcher提出了一种基于选择性状态空间模型（SSM）的高效方法，用于语义对应任务，通过相似性感知的选择性扫描机制优化4D相关映射，显著提升了性能。

Details Motivation: 传统的特征度量方法可能忽略复杂的相关关系，而相关性度量方法因高计算成本受限。MambaMatcher旨在高效建模高维相关性。

Contribution: 1. 提出MambaMatcher方法，利用选择性SSM建模高维相关性；2. 引入相似性感知的选择性扫描机制，优化4D相关映射。

Method: 使用选择性状态空间模型（SSM）和Mamba的线性复杂度算法，高效建模并优化4D相关映射。

Result: 在标准语义对应基准测试中取得了最先进的性能。

Insight: 选择性SSM和相似性感知机制的结合，为高维相关性建模提供了高效解决方案。

Abstract: Establishing semantic correspondences between images is a fundamental yet challenging task in computer vision. Traditional feature-metric methods enhance visual features but may miss complex inter-correlation relationships, while recent correlation-metric approaches are hindered by high computational costs due to processing 4D correlation maps. We introduce MambaMatcher, a novel method that overcomes these limitations by efficiently modeling high-dimensional correlations using selective state-space models (SSMs). By implementing a similarity-aware selective scan mechanism adapted from Mamba’s linear-complexity algorithm, MambaMatcher refines the 4D correlation map effectively without compromising feature map resolution or receptive field. Experiments on standard semantic correspondence benchmarks demonstrate that MambaMatcher achieves state-of-the-art performance.

[289] TP-MVCC: Tri-plane Multi-view Fusion Model for Silkie Chicken Counting

Sirui Chen,Yuhong Feng,Yifeng Wang,Jianghai Liao,Qi Zhang

Main category: cs.CV

TL;DR: 论文提出了一种基于三平面多视图融合的鸡只计数模型TP-MVCC，通过几何投影和三平面融合集成多摄像头特征，提升了密集遮挡场景下的计数准确性，并在真实农场环境中验证其高效性。

Details Motivation: 精确的动物计数对智慧农业至关重要，但密集遮挡和多摄像头视角有限的问题使得计数任务困难重重。

Contribution: 提出了TP-MVCC模型，利用三平面融合和几何投影统一多视图特征；构建了首个真实农场环境下的多视角鸡只数据集。

Method: 通过单视图特征提取、空间变换对齐和多视图融合解码生成场景级密度图。

Result: 实验结果显示TP-MVCC在密集遮挡场景中达到95.1%的准确率，显著优于单视图和传统融合方法。

Insight: 多视图融合能有效解决密集遮挡问题，三平面特征投影为多视角数据整合提供了新思路。

Abstract: Accurate animal counting is essential for smart farming but remains difficult in crowded scenes due to occlusions and limited camera views. To address this, we propose a tri-plane-based multi-view chicken counting model (TP-MVCC), which leverages geometric projection and tri-plane fusion to integrate features from multiple cameras onto a unified ground plane. The framework extracts single-view features, aligns them via spatial transformation, and decodes a scene-level density map for precise chicken counting. In addition, we construct the first multi-view dataset of silkie chickens under real farming conditions. Experiments show that TP-MVCC significantly outperforms single-view and conventional fusion comparisons, achieving 95.1% accuracy and strong robustness in dense, occluded scenarios, demonstrating its practical potential for intelligent agriculture.

[290] Dynamic Orchestration of Multi-Agent System for Real-World Multi-Image Agricultural VQA

Yan Ke,Xin Yu,Heming Du,Scott Chapman,Helen Huang

Main category: cs.CV

TL;DR: 本文提出了一种动态编排的多智能体框架，用于解决农业视觉问答（VQA）中多图像输入和上下文不足的问题，通过四个角色（检索器、反思器、回答器和改进器）协作实现上下文丰富和迭代优化。

Details Motivation: 现有农业VQA系统主要针对单图像或文本查询设计，无法应对多图像输入和动态变化的农业场景，且缺乏系统化的质量控制机制。

Contribution: 提出了一个自反思和自改进的多智能体框架，解决了多图像输入和上下文不足的挑战，并通过实验验证了其性能。

Method: 框架包含四个协作角色：Retriever（检索外部信息）、Reflector（评估和触发重新检索）、Answerer（并行生成候选答案以减少偏差）和Improver（迭代优化答案）。

Result: 在AgMMU基准测试中，该框架在多图像农业问答任务中表现出色。

Insight: 通过动态编排多智能体实现上下文丰富和迭代优化，可以显著提升农业VQA系统在复杂真实场景中的适应性和准确性。

Abstract: Agricultural visual question answering is essential for providing farmers and researchers with accurate and timely knowledge. However, many existing approaches are predominantly developed for evidence-constrained settings such as text-only queries or single-image cases. This design prevents them from coping with real-world agricultural scenarios that often require multi-image inputs with complementary views across spatial scales, and growth stages. Moreover, limited access to up-to-date external agricultural context makes these systems struggle to adapt when evidence is incomplete. In addition, rigid pipelines often lack systematic quality control. To address this gap, we propose a self-reflective and self-improving multi-agent framework that integrates four roles, the Retriever, the Reflector, the Answerer, and the Improver. They collaborate to enable context enrichment, reflective reasoning, answer drafting, and iterative improvement. A Retriever formulates queries and gathers external information, while a Reflector assesses adequacy and triggers sequential reformulation and renewed retrieval. Two Answerers draft candidate responses in parallel to reduce bias. The Improver refines them through iterative checks while ensuring that information from multiple images is effectively aligned and utilized. Experiments on the AgMMU benchmark show that our framework achieves competitive performance on multi-image agricultural QA.

[291] NeRV-Diffusion: Diffuse Implicit Neural Representations for Video Synthesis

Yixuan Ren,Hanyu Wang,Hao Chen,Bo He,Abhinav Shrivastava

Main category: cs.CV

TL;DR: NeRV-Diffusion通过生成神经网络的权重来合成视频，利用隐式神经表示（INR）和扩散模型实现高效的视频生成。

Details Motivation: 视频合成通常依赖于逐帧特征编码和跨帧注意力机制，这不仅计算量大，还可能影响生成质量。NeRV-Diffusion提出以一种整体的方式编码和解码视频，从而避免这些限制。

Contribution: 1) 提出一个超网络编码器，将视频从像素空间转换到神经参数空间；2) 设计了一个隐式扩散变换器，直接在INR权重上进行去噪；3) 通过重用瓶颈潜变量和改进权重分配等技术，提升了INR的表达能力。

Method: 1) 使用超网络将视频编码为INR权重；2) 在INR权重空间应用扩散变换器进行去噪；3) 结合SNR自适应损失权重和计划采样优化训练过程。

Result: 在UCF-101和Kinetics-600等基准测试中，NeRV-Diffusion优于其他基于INR的模型，与非隐式的最先进模型性能相当，同时能实现帧或视频间的平滑插值。

Insight: 通过整体生成视频的神经表示，NeRV-Diffusion避免了逐帧处理的复杂性，同时在权重空间中实现了高质量的降噪和插值能力。

Abstract: We present NeRV-Diffusion, an implicit latent video diffusion model that synthesizes videos via generating neural network weights. The generated weights can be rearranged as the parameters of a convolutional neural network, which forms an implicit neural representation (INR), and decodes into videos with frame indices as the input. Our framework consists of two stages: 1) A hypernetworkbased tokenizer that encodes raw videos from pixel space to neural parameter space, where the bottleneck latent serves as INR weights to decode. 2) An implicit diffusion transformer that denoises on the latent INR weights. In contrast to traditional video tokenizers that encode videos into frame-wise feature maps, NeRV-Diffusion compresses and generates a video holistically as a unified neural network. This enables efficient and high-quality video synthesis via obviating temporal cross-frame attentions in the denoiser and decoding video latent with dedicated decoders. To achieve Gaussian-distributed INR weights with high expressiveness, we reuse the bottleneck latent across all NeRV layers, as well as reform its weight assignment, upsampling connection and input coordinates. We also introduce SNR-adaptive loss weighting and scheduled sampling for effective training of the implicit diffusion model. NeRV-Diffusion reaches superior video generation quality over previous INR-based models and comparable performance to most recent state-of-the-art non-implicit models on real-world video benchmarks including UCF-101 and Kinetics-600. It also brings a smooth INR weight space that facilitates seamless interpolations between frames or videos.

[292] An Enhanced Pyramid Feature Network Based on Long-Range Dependencies for Multi-Organ Medical Image Segmentation

Dayu Tan,Cheng Kong,Yansen Su,Hai Chen,Dongliang Yang,Junfeng Xia,Chunhou Zheng

Main category: cs.CV

TL;DR: LamFormer是一种新型U形网络，通过Linear Attention Mamba（LAM）增强金字塔编码器，解决多器官医学图像分割中的长范围依赖和局部细节提取问题，性能优于现有方法且模型复杂度平衡。

Details Motivation: 当前的多器官医学图像分割方法依赖Transformer，但存在计算成本高和局部细节提取不足的问题。

Contribution: 提出LamFormer网络，结合LAM和PHFA模块，改进长范围依赖建模和局部细节提取；设计RT模块优化全局特征建模。

Method: LamFormer采用U形结构，利用LAM捕获多尺度长范围依赖；PHFA模块聚合不同层特征；RT模块优化全局特征建模。

Result: 在七个复杂多样的数据集上优于现有分割方法，性能优异且模型复杂度平衡。

Insight: 通过结合线性注意力和改进的特征聚合，LamFormer在性能和计算效率之间取得了平衡，为医学图像分割提供了新思路。

Abstract: In the field of multi-organ medical image segmentation, recent methods frequently employ Transformers to capture long-range dependencies from image features. However, these methods overlook the high computational cost of Transformers and their deficiencies in extracting local detailed information. To address high computational costs and inadequate local detail information, we reassess the design of feature extraction modules and propose a new deep-learning network called LamFormer for fine-grained segmentation tasks across multiple organs. LamFormer is a novel U-shaped network that employs Linear Attention Mamba (LAM) in an enhanced pyramid encoder to capture multi-scale long-range dependencies. We construct the Parallel Hierarchical Feature Aggregation (PHFA) module to aggregate features from different layers of the encoder, narrowing the semantic gap among features while filtering information. Finally, we design the Reduced Transformer (RT), which utilizes a distinct computational approach to globally model up-sampled features. RRT enhances the extraction of detailed local information and improves the network’s capability to capture long-range dependencies. LamFormer outperforms existing segmentation methods on seven complex and diverse datasets, demonstrating exceptional performance. Moreover, the proposed network achieves a balance between model performance and model complexity.

[293] DRIFT: Divergent Response in Filtered Transformations for Robust Adversarial Defense

Amira Guesmi,Muhammad Shafique

Main category: cs.CV

TL;DR: DRIFT是一种通过打破梯度共识来提升对抗防御能力的方法，通过随机变换和梯度不一致性训练轻量级滤波器，显著提升了模型对抗攻击的鲁棒性。

Details Motivation: 现有对抗防御方法大多依赖梯度掩码，容易被攻击者利用梯度共识绕过。DRIFT旨在通过主动破坏梯度共识来提升防御能力。

Contribution: 形式化梯度共识并提出理论分析；提出一种结合预测一致性、Jacobian分离和对抗鲁棒性的训练策略；在ImageNet上显著提升鲁棒性。

Method: DRIFT通过训练随机滤波器集合最大化梯度不一致性，同时保持自然预测能力，结合Jacobian和logit空间分离的损失函数。

Result: DRIFT在自适应白盒、迁移和无梯度攻击下均优于现有方法，且计算开销可忽略。

Insight: 梯度不一致性是提升对抗防御的有效原则，轻量级的滤波器设计可以广泛适用于不同架构。

Abstract: Deep neural networks remain highly vulnerable to adversarial examples, and most defenses collapse once gradients can be reliably estimated. We identify \emph{gradient consensus} – the tendency of randomized transformations to yield aligned gradients – as a key driver of adversarial transferability. Attackers exploit this consensus to construct perturbations that remain effective across transformations. We introduce \textbf{DRIFT} (Divergent Response in Filtered Transformations), a stochastic ensemble of lightweight, learnable filters trained to actively disrupt gradient consensus. Unlike prior randomized defenses that rely on gradient masking, DRIFT enforces \emph{gradient dissonance} by maximizing divergence in Jacobian- and logit-space responses while preserving natural predictions. Our contributions are threefold: (i) we formalize gradient consensus and provide a theoretical analysis linking consensus to transferability; (ii) we propose a consensus-divergence training strategy combining prediction consistency, Jacobian separation, logit-space separation, and adversarial robustness; and (iii) we show that DRIFT achieves substantial robustness gains on ImageNet across CNNs and Vision Transformers, outperforming state-of-the-art preprocessing, adversarial training, and diffusion-based defenses under adaptive white-box, transfer-based, and gradient-free attacks. DRIFT delivers these improvements with negligible runtime and memory cost, establishing gradient divergence as a practical and generalizable principle for adversarial defense.

[294] UI-UG: A Unified MLLM for UI Understanding and Generation

Hao Yang,Weijie Qiu,Ru Zhang,Zhou Fang,Ruichao Mao,Xiaoyu Lin,Maji Huang,Zhaosong Huang,Teng Guo,Shuoyang Liu,Hai Rao

Main category: cs.CV

TL;DR: UI-UG是一个统一的多模态大语言模型（MLLM），专注于用户界面（UI）的理解和生成任务。通过结合监督微调（SFT）、组相对策略优化（GRPO）和直接偏好优化（DPO），提升了模型的性能和生成质量。

Details Motivation: 现有MLLMs在领域特定任务（如UI的理解和生成）中表现不佳，难以满足现代复杂UI的需求。

Contribution: 1. 提出UI-UG模型，统一了UI理解和生成能力。2. 引入GRPO和DPO优化方法。3. 设计了高效的工业级工作流程。

Method: 1. 使用SFT和GRPO优化UI理解任务。2. 采用DPO提升UI生成质量。3. 设计LLM友好的领域特定语言（DSL）和工作流程。

Result: UI-UG在UI理解任务上达到SOTA性能，生成任务上与更大规模MLLMs相当，计算成本更低。

Insight: UI理解和生成任务的整合可以相互提升性能和生成质量。

Abstract: Although Multimodal Large Language Models (MLLMs) have been widely applied across domains, they are still facing challenges in domain-specific tasks, such as User Interface (UI) understanding accuracy and UI generation quality. In this paper, we introduce UI-UG (a unified MLLM for UI Understanding and Generation), integrating both capabilities. For understanding tasks, we employ Supervised Fine-tuning (SFT) combined with Group Relative Policy Optimization (GRPO) to enhance fine-grained understanding on the modern complex UI data. For generation tasks, we further use Direct Preference Optimization (DPO) to make our model generate human-preferred UIs. In addition, we propose an industrially effective workflow, including the design of an LLM-friendly domain-specific language (DSL), training strategies, rendering processes, and evaluation metrics. In experiments, our model achieves state-of-the-art (SOTA) performance on understanding tasks, outperforming both larger general-purpose MLLMs and similarly-sized UI-specialized models. Our model is also on par with these larger MLLMs in UI generation performance at a fraction of the computational cost. We also demonstrate that integrating understanding and generation tasks can improve accuracy and quality for both tasks.

[295] Uni-X: Mitigating Modality Conflict with a Two-End-Separated Architecture for Unified Multimodal Models

Jitai Hao,Hao Liu,Xinyan Xiao,Qiang Huang,Jun Yu

Main category: cs.CV

TL;DR: Uni-X是一种两端分离、中间共享的统一多模态模型架构，通过避免浅层和深层的模态冲突，显著提升了训练效率和性能。

Details Motivation: 现有的共享自回归变压器的统一多模态模型存在模态间梯度冲突问题，尤其是在浅层和深层，这源于图像与文本的低层次统计特性差异。

Contribution: 提出了Uni-X架构，两端为模态特异性处理层，中间为语义共享层，有效消除了梯度冲突并提升了模型效率和性能。

Method: 设计了X形架构，两端分离（模态特异性）与中间共享（语义融合），通过调整层分配避免了冲突。

Result: Uni-X在3B参数规模下，性能匹配或超越7B参数的基线模型，生成评分达82，同时在多种任务中表现优异。

Insight: 中间层更适合多模态语义融合，而两端为模态特异性处理可以显著减少冲突，未来多模态模型设计可借鉴此思路。

Abstract: Unified Multimodal Models (UMMs) built on shared autoregressive (AR) transformers are attractive for their architectural simplicity. However, we identify a critical limitation: when trained on multimodal inputs, modality-shared transformers suffer from severe gradient conflicts between vision and text, particularly in shallow and deep layers. We trace this issue to the fundamentally different low-level statistical properties of images and text, while noting that conflicts diminish in middle layers where representations become more abstract and semantically aligned. To overcome this challenge, we propose Uni-X, a two-end-separated, middle-shared architecture. Uni-X dedicates its initial and final layers to modality-specific processing, while maintaining shared parameters in the middle layers for high-level semantic fusion. This X-shaped design not only eliminates gradient conflicts at both ends but also further alleviates residual conflicts in the shared layers. Extensive experiments validate the effectiveness of Uni-X. Under identical training conditions, Uni-X achieves superior training efficiency compared to strong baselines. When scaled to 3B parameters with larger training data, Uni-X matches or surpasses 7B AR-based UMMs, achieving a GenEval score of 82 for image generation alongside strong performance in text and vision understanding tasks. These results establish Uni-X as a parameter-efficient and scalable foundation for future unified multimodal modeling. Our code is available at https://github.com/CURRENTF/Uni-X

[296] DINOReg: Strong Point Cloud Registration with Vision Foundation Model

Congjia Chen,Yufu Qu

Main category: cs.CV

TL;DR: DINOReg提出了一种新的点云配准方法，结合视觉基础模型DINOv2提取的图像特征和几何特征，实现了显著的性能提升。

Details Motivation: 现有的点云配准方法主要依赖几何信息，忽略了图像的纹理和语义信息，且特征融合方式不够高效。DINOReg旨在充分利用视觉和几何信息以提升配准性能。

Contribution: 提出DINOReg网络，结合DINOv2的视觉特征和几何特征；设计混合位置编码以融合图像和点云空间的位置信息。

Method: 使用DINOv2提取图像特征，并在patch级别融合视觉与几何特征；引入混合位置编码增强空间关系感知能力。

Result: 在RGBD-3DMatch和RGBD-3DLoMatch数据集上，显著优于现有方法，配准召回率提高15.7%。

Insight: 视觉基础模型可为点云配准提供丰富的纹理和语义信息；多模态特征的高效融合是提升性能的关键。

Abstract: Point cloud registration is a fundamental task in 3D computer vision. Most existing methods rely solely on geometric information for feature extraction and matching. Recently, several studies have incorporated color information from RGB-D data into feature extraction. Although these methods achieve remarkable improvements, they have not fully exploited the abundant texture and semantic information in images, and the feature fusion is performed in an image-lossy manner, which limit their performance. In this paper, we propose DINOReg, a registration network that sufficiently utilizes both visual and geometric information to solve the point cloud registration problem. Inspired by advances in vision foundation models, we employ DINOv2 to extract informative visual features from images, and fuse visual and geometric features at the patch level. This design effectively combines the rich texture and global semantic information extracted by DINOv2 with the detailed geometric structure information captured by the geometric backbone. Additionally, a mixed positional embedding is proposed to encode positional information from both image space and point cloud space, which enhances the model’s ability to perceive spatial relationships between patches. Extensive experiments on the RGBD-3DMatch and RGBD-3DLoMatch datasets demonstrate that our method achieves significant improvements over state-of-the-art geometry-only and multi-modal registration methods, with a 14.2% increase in patch inlier ratio and a 15.7% increase in registration recall. The code is publicly available at https://github.com/ccjccjccj/DINOReg.

[297] REALIGN: Regularized Procedure Alignment with Matching Video Embeddings via Partial Gromov-Wasserstein Optimal Transport

Soumyadeep Chandra,Kaushik Roy

Main category: cs.CV

TL;DR: REALIGN提出了一种基于正则化融合部分Gromov-Wasserstein最优传输的自监督框架，用于处理教学视频中的非单调步骤顺序、重复动作和无关帧问题，显著提升了性能。

Details Motivation: 教学视频中存在背景片段、重复动作和非单调步骤顺序等问题，现有方法基于强单调性假设难以处理，且缺乏对高阶时间结构的建模。

Contribution: 提出了REALIGN框架，利用R-FPGWOT联合建模视觉对应性和时间关系，通过部分对齐方案和对比学习提升鲁棒性。

Method: 结合了正则化融合部分Gromov-Wasserstein最优传输和序列间对比学习，避免了多正则化和退化解。

Result: 在多个基准测试中，平均F1分数提升了18.9%，时间IoU提升了30%以上，且生成了更可解释的传输映射。

Insight: 通过联合建模视觉和时间关系，可以更有效地处理教学视频中的复杂动态，同时提升模型的可解释性。

Abstract: Learning from procedural videos remains a core challenge in self-supervised representation learning, as real-world instructional data often contains background segments, repeated actions, and steps presented out of order. Such variability violates the strong monotonicity assumptions underlying many alignment methods. Prior state-of-the-art approaches, such as OPEL, leverage Kantorovich Optimal Transport (KOT) to build frame-to-frame correspondences, but rely solely on feature similarity and fail to capture the higher-order temporal structure of a task. In this paper, we introduce REALIGN, a self-supervised framework for procedure learning based on Regularized Fused Partial Gromov-Wasserstein Optimal Transport (R-FPGWOT). In contrast to KOT, our formulation jointly models visual correspondences and temporal relations under a partial alignment scheme, enabling robust handling of irrelevant frames, repeated actions, and non-monotonic step orders common in instructional videos. To stabilize training, we integrate FPGWOT distances with inter-sequence contrastive learning, avoiding the need for multiple regularizers and preventing collapse to degenerate solutions. Across egocentric (EgoProceL) and third-person (ProceL, CrossTask) benchmarks, REALIGN achieves up to 18.9% average F1-score improvements and over 30% temporal IoU gains, while producing more interpretable transport maps that preserve key-step orderings and filter out noise.

[298] Vid-LLM: A Compact Video-based 3D Multimodal LLM with Reconstruction-Reasoning Synergy

Haijier Chen,Bo Xu,Shoujian Zhang,Haoze Liu,Jiaxuan Lin,Jingrong Wang

Main category: cs.CV

TL;DR: Vid-LLM 是一种紧凑的基于视频的3D多模态大语言模型，通过重建-推理协同无需外部3D数据输入，提升了3D场景理解能力。

Details Motivation: 现有的3D多模态大语言模型依赖3D数据输入，限制了其可扩展性和泛化能力。本文旨在通过视频输入直接实现3D场景理解。

Contribution: 提出了Vid-LLM模型，通过Cross-Task Adapter模块和Metric Depth Model整合几何先验，并通过两阶段蒸馏优化实现高效训练。

Method: 设计了Cross-Task Adapter对齐几何先验与视觉语言表征，引入Metric Depth Model恢复真实尺度几何，采用两阶段蒸馏优化策略。

Result: 在3D问答、3D密集标注和3D视觉定位等多个任务上验证了方法的有效性。

Insight: 视频输入可以作为替代3D数据的有效形式，几何先验的整合和多任务优化策略显著提升了3D场景理解能力。

Abstract: Recent developments in Multimodal Large Language Models (MLLMs) have significantly improved Vision-Language (VL) reasoning in 2D domains. However, extending these capabilities to 3D scene understanding remains a major challenge. Existing 3D Multimodal Large Language Models (3D-MLLMs) often depend on 3D data inputs, which limits scalability and generalization. To address this limitation, we propose Vid-LLM, a video-based 3D-MLLM that directly processes video inputs without requiring external 3D data, making it practical for real-world deployment. In our method, the geometric prior are directly used to improve the performance of the sceen perception. To integrate the geometric cues into the MLLM compactly, we design a Cross-Task Adapter (CTA) module to align the 3D geometric priors with the vision-language representations. To ensure geometric consistency and integrity, we introduce a Metric Depth Model that recovers real-scale geometry from the reconstruction outputs. Finally, the model is fine-tuned with a two-stage distillation optimization strategy, realizing fast convergence and stabilizes training. Extensive experiments across diverse benchmarks verified the effectiveness of our method on 3D Question Answering, 3D Dense Captioning and 3D Visual Grounding tasks, demonstrating the superior multi-task capabilities.

[299] PCICF: A Pedestrian Crossing Identification and Classification Framework

Junyi Gu,Beatriz Cabrero-Daniel,Ali Nouri,Lydia Armini,Christian Berger

Main category: cs.CV

TL;DR: PCICF是一个行人穿越识别和分类框架，旨在支持自动驾驶车辆在城市环境中识别和分类复杂的行人穿越场景。通过扩展合成数据集并利用空间填充曲线（SFCs）进行特征匹配，PCICF在实际数据集PIE上表现出色。

Details Motivation: 自动驾驶车辆（如robotaxi）在城市环境中需要可靠地检测和应对行人等脆弱道路使用者（VRUs）。现有的多模态传感器数据处理系统缺乏高质量数据支持，尤其是在复杂行人穿越场景中。

Contribution: 1. 提出PCICF框架，支持系统化的VRU场景识别和分类；2. 扩展合成数据集SMIRK为MoreSMIRK，覆盖多行人穿越场景；3. 利用SFCs高效匹配多维特征；4. 在实际数据集PIE上验证效果。

Method: 1. 扩展SMIRK数据集为MoreSMIRK；2. 使用SFCs将多维特征转化为特征模式并与MoreSMIRK匹配；3. 在实际数据集PIE上评估。

Result: PCICF能成功识别和分类复杂的行人穿越场景（如行人合并或分散），并在计算效率上具备车载应用的潜力。

Insight: 1. SFCs在多维特征匹配中表现高效；2. 合成数据集的扩展支持了复杂场景的模拟；3. PCICF展示了自动驾驶中端到端AI的潜力。

Abstract: We have recently observed the commercial roll-out of robotaxis in various countries. They are deployed within an operational design domain (ODD) on specific routes and environmental conditions, and are subject to continuous monitoring to regain control in safety-critical situations. Since ODDs typically cover urban areas, robotaxis must reliably detect vulnerable road users (VRUs) such as pedestrians, bicyclists, or e-scooter riders. To better handle such varied traffic situations, end-to-end AI, which directly compute vehicle control actions from multi-modal sensor data instead of only for perception, is on the rise. High quality data is needed for systematically training and evaluating such systems within their OOD. In this work, we propose PCICF, a framework to systematically identify and classify VRU situations to support ODD’s incident analysis. We base our work on the existing synthetic dataset SMIRK, and enhance it by extending its single-pedestrian-only design into the MoreSMIRK dataset, a structured dictionary of multi-pedestrian crossing situations constructed systematically. We then use space-filling curves (SFCs) to transform multi-dimensional features of scenarios into characteristic patterns, which we match with corresponding entries in MoreSMIRK. We evaluate PCICF with the large real-world dataset PIE, which contains more than 150 manually annotated pedestrian crossing videos. We show that PCICF can successfully identify and classify complex pedestrian crossings, even when groups of pedestrians merge or split. By leveraging computationally efficient components like SFCs, PCICF has even potential to be used onboard of robotaxis for OOD detection for example. We share an open-source replication package for PCICF containing its algorithms, the complete MoreSMIRK dataset and dictionary, as well as our experiment results presented in: https://github.com/Claud1234/PCICF

[300] CLQ: Cross-Layer Guided Orthogonal-based Quantization for Diffusion Transformers

Kai Liu,Shaoqiu Zhang,Linghe Kong,Yulun Zhang

Main category: cs.CV

TL;DR: CLQ是一种针对扩散变压器（DiTs）的跨层引导正交量化方法，通过跨块校准、正交平滑和跨层参数搜索，实现了W4A4量化下视觉质量几乎无损的高效压缩。

Details Motivation: 扩散变压器（DiTs）在大规模模型和复杂性的推动下提升了视觉生成质量，但其边缘设备部署受到限制，亟需高效量化方法以减少内存消耗并加速推理。

Contribution: 1) 提出跨块校准（CBC）以获得准确的校准数据；2) 设计正交平滑（OBS）处理异常值；3) 提出跨层参数搜索（CLPS）优化量化参数。

Method: 结合CBC、OBS和CLPS，CLQ实现了对DiTs的高效量化，显著降低了内存占用并加速推理。

Result: CLQ成功压缩模型至W4A4，视觉质量和指标几乎无损，内存节省3.98倍，速度提升3.95倍。

Insight: 校准数据的准确性对量化至关重要，跨层引导和正交平滑是处理DiTs异常值的有效方法。

Abstract: Visual generation quality has been greatly promoted with the rapid advances in diffusion transformers (DiTs), which is attributed to the scaling of model size and complexity. However, these attributions also hinder the practical deployment of DiTs on edge devices, limiting their development and application. Serve as an efficient model compression technique, model post-training quantization (PTQ) can reduce the memory consumption and speed up the inference, with inevitable performance degradation. To alleviate the degradation, we propose CLQ, a cross-layer guided orthogonal-based quantization method for DiTs. To be specific, CLQ consists of three key designs. First, we observe that the calibration data used by most of the PTQ methods can not honestly represent the distribution of the activations. Therefore, we propose cross-block calibration (CBC) to obtain accurate calibration data, with which the quantization can be better guided. Second, we propose orthogonal-based smoothing (OBS), which quantifies the outlier score of each channel and leverages block Hadamard matrix to smooth the outliers with negligible overhead. Third, we propose cross-layer parameter searching (CLPS) to search. We evaluate CLQ with both image generation and video generation models and successfully compress the model into W4A4 with negligible degradation in visual quality and metrics. CLQ achieves 3.98x memory saving and 3.95x speedup. Our code is available at \hyperlink{https://github.com/Kai-Liu001/CLQ}{https://github.com/Kai-Liu001/CLQ}.

[301] A Data-Centric Perspective on the Influence of Image Data Quality in Machine Learning Models

Pei-Han Chen,Szu-Chi Chung

Main category: cs.CV

TL;DR: 本文探讨了图像数据集质量对机器学习模型性能的影响，提出了一种整合CleanVision和Fastdup工具的管道，并引入了自动阈值选择等方法。实验表明，不同质量问题对模型的影响各异，CNN对某些失真具有鲁棒性，但对模糊和降质尤其敏感。改进后的方法显著提升了低质量图像检测的F1分数。

Details Motivation: 传统机器学习研究侧重于模型开发，而忽略了训练数据质量的影响。随着模型架构成熟，数据质量逐渐成为关键因素。本文旨在填补图像领域数据集质量评估的系统性研究空白。

Contribution: 1. 系统性评估图像数据集质量及其对模型性能的影响；2.开发了一个整合CleanVision和Fastdup工具的管道，并引入自动阈值选择等改进；3.量化了不同质量问题的具体影响。

Method: 1.使用CIFAKE数据集识别常见质量问题；2.整合CleanVision和Fastdup工具并改进其功能；3.将低质量图像检测建模为二分类任务，以F1分数为评估指标。

Result: 自动阈值方法将单扰动和双扰动下的F1分数分别从0.6794提升至0.9468和从0.7447提升至0.8557；去重策略将F1分数从0.4576提升至0.7928。结果表明，改进后的方法显著提升了检测效果。

Insight: 1.并非所有质量问题对模型的影响相同，CNN对模糊和降质特别敏感；2.自动阈值选择和工具整合可有效提升数据质量评估的效果；3.数据质量应成为机器学习研究的重要方向。

Abstract: In machine learning, research has traditionally focused on model development, with relatively less attention paid to training data. As model architectures have matured and marginal gains from further refinements diminish, data quality has emerged as a critical factor. However, systematic studies on evaluating and ensuring dataset quality in the image domain remain limited. This study investigates methods for systematically assessing image dataset quality and examines how various image quality factors influence model performance. Using the publicly available and relatively clean CIFAKE dataset, we identify common quality issues and quantify their impact on training. Building on these findings, we develop a pipeline that integrates two community-developed tools, CleanVision and Fastdup. We analyze their underlying mechanisms and introduce several enhancements, including automatic threshold selection to detect problematic images without manual tuning. Experimental results demonstrate that not all quality issues exert the same level of impact. While convolutional neural networks show resilience to certain distortions, they are particularly vulnerable to degradations that obscure critical visual features, such as blurring and severe downscaling. To assess the performance of existing tools and the effectiveness of our proposed enhancements, we formulate the detection of low-quality images as a binary classification task and use the F1 score as the evaluation metric. Our automatic thresholding method improves the F1 score from 0.6794 to 0.9468 under single perturbations and from 0.7447 to 0.8557 under dual perturbations. For near-duplicate detection, our deduplication strategy increases the F1 score from 0.4576 to 0.7928. These results underscore the effectiveness of our workflow and provide a foundation for advancing data quality assessment in image-based machine learning.

[302] Rethinking Unsupervised Cross-modal Flow Estimation: Learning from Decoupled Optimization and Consistency Constraint

Runmin Zhang,Jialiang Wang,Si-Yuan Cao,Zhu Yu,Junchen Yu,Guangyi Zhang,Hui-Liang Shen

Main category: cs.CV

TL;DR: DCFlow是一种新型的无监督跨模态光流估计框架，通过解耦优化策略和跨模态一致性约束，显著提升了光流预测的准确性。

Details Motivation: 传统方法仅通过外观相似性隐式学习光流估计，难以解决模态差异和几何不对齐问题。本文旨在通过解耦优化和一致性约束解决这些挑战。

Contribution: 1. 提出解耦优化策略，分别处理模态差异和几何不对齐问题；2. 设计几何感知数据合成流程和离群点鲁棒损失，实现无真实光流的可靠监督；3. 引入跨模态一致性约束，联合优化模态转移网络和光流估计网络。

Method: 1. 解耦优化策略：分别训练模态转移网络和光流估计网络；2. 几何感知数据合成与离群点鲁棒损失；3. 跨模态一致性约束联合优化。

Result: DCFlow在无监督方法中表现优异，并与多种光流估计网络兼容。

Insight: 通过显式处理模态差异和几何不对齐问题，结合无监督学习策略，可以显著提升跨模态光流估计的性能。

Abstract: This work presents DCFlow, a novel unsupervised cross-modal flow estimation framework that integrates a decoupled optimization strategy and a cross-modal consistency constraint. Unlike previous approaches that implicitly learn flow estimation solely from appearance similarity, we introduce a decoupled optimization strategy with task-specific supervision to address modality discrepancy and geometric misalignment distinctly. This is achieved by collaboratively training a modality transfer network and a flow estimation network. To enable reliable motion supervision without ground-truth flow, we propose a geometry-aware data synthesis pipeline combined with an outlier-robust loss. Additionally, we introduce a cross-modal consistency constraint to jointly optimize both networks, significantly improving flow prediction accuracy. For evaluation, we construct a comprehensive cross-modal flow benchmark by repurposing public datasets. Experimental results demonstrate that DCFlow can be integrated with various flow estimation networks and achieves state-of-the-art performance among unsupervised approaches.

[303] UI2V-Bench: An Understanding-based Image-to-video Generation Benchmark

Ailing Zhang,Lina Lei,Dehong Kong,Zhixin Wang,Jiaqi Xu,Fenglong Song,Chun-Le Guo,Chang Liu,Fan Li,Jie Chen

Main category: cs.CV

TL;DR: 论文提出了UI2V-Bench，一个专注于语义理解和推理的图像到视频（I2V）生成评估基准，弥补了现有基准在语义和常识性评估上的不足。

Details Motivation: 现有的I2V评估基准主要关注视频质量和时间一致性，而忽略了模型对输入图像语义的理解以及生成视频是否符合物理规律和人类常识的能力。

Contribution: 提出了UI2V-Bench，引入了四个评估维度（空间理解、属性绑定、类别理解和推理），并设计了基于MLLM的两种评估方法。

Method: 设计了实例级语义理解管道和反馈式推理管道，基于MLLM进行评估，并结合了人工评估。

Result: UI2V-Bench评估了多个开源和闭源I2V模型，结果显示提出的MLLM指标与人工评估高度一致。

Insight: 强调语义理解和推理能力是I2V评估的重要方向，UI2V-Bench为未来研究提供了框架和数据支持。

Abstract: Generative diffusion models are developing rapidly and attracting increasing attention due to their wide range of applications. Image-to-Video (I2V) generation has become a major focus in the field of video synthesis. However, existing evaluation benchmarks primarily focus on aspects such as video quality and temporal consistency, while largely overlooking the model’s ability to understand the semantics of specific subjects in the input image or to ensure that the generated video aligns with physical laws and human commonsense. To address this gap, we propose UI2V-Bench, a novel benchmark for evaluating I2V models with a focus on semantic understanding and reasoning. It introduces four primary evaluation dimensions: spatial understanding, attribute binding, category understanding, and reasoning. To assess these dimensions, we design two evaluation methods based on Multimodal Large Language Models (MLLMs): an instance-level pipeline for fine-grained semantic understanding, and a feedback-based reasoning pipeline that enables step-by-step causal assessment for more accurate evaluation. UI2V-Bench includes approximately 500 carefully constructed text-image pairs and evaluates a range of both open source and closed-source I2V models across all defined dimensions. We further incorporate human evaluations, which show strong alignment with the proposed MLLM-based metrics. Overall, UI2V-Bench fills a critical gap in I2V evaluation by emphasizing semantic comprehension and reasoning ability, offering a robust framework and dataset to support future research and model development in the field.

[304] Beyond Isolated Facts: Synthesizing Narrative and Grounded Supervision for VideoQA

Jianxin Liang,Tan Yue,Yuxuan Wang,Yueqian Wang,Zhihan Yin,Huishuai Zhang,Dongyan Zhao

Main category: cs.CV

TL;DR: 本文提出了一种改进视频问答（VideoQA）模型的监督信号合成框架，通过两种策略（QBP和QBC）生成更丰富的叙事和视觉依据，显著提升了模型性能和泛化能力。

Details Motivation: 当前VideoQA模型的监督信号（孤立的事实问答对）限制了其对视频内容的理解能力，缺乏对事件叙事和因果结构的捕捉。

Contribution: 1）提出QBP策略，从现有问答对生成叙事段落；2）提出QBC策略，为每个问题生成细粒度视觉依据；3）通过合成数据训练模型，显著提升性能。

Method: 1）QBP将视频问答对转化为连贯叙事；2）QBC生成视觉依据以支持答案；3）利用生成模型合成数据，基于下一个标记预测目标训练VideoQA模型。

Result: 在STAR和NExT-QA数据集上取得了新的SOTA性能，例如在STAR上将3B模型提升至72.5%（+4.9%）。QBP还加速了模型收敛2.5倍以上。

Insight: 从孤立事实转向叙事连贯性和视觉依据的合成数据，能够提升模型的准确性、效率和泛化能力。

Abstract: The performance of Video Question Answering (VideoQA) models is fundamentally constrained by the nature of their supervision, which typically consists of isolated, factual question-answer pairs. This “bag-of-facts” approach fails to capture the underlying narrative and causal structure of events, limiting models to a shallow understanding of video content. To move beyond this paradigm, we introduce a framework to synthesize richer supervisory signals. We propose two complementary strategies: Question-Based Paraphrasing (QBP), which synthesizes the diverse inquiries (what, how, why) from a video’s existing set of question-answer pairs into a holistic narrative paragraph that reconstructs the video’s event structure; and Question-Based Captioning (QBC), which generates fine-grained visual rationales, grounding the answer to each question in specific, relevant evidence. Leveraging powerful generative models, we use this synthetic data to train VideoQA models under a unified next-token prediction objective. Extensive experiments on STAR and NExT-QA validate our approach, demonstrating significant accuracy gains and establishing new state-of-the-art results, such as improving a 3B model to 72.5% on STAR (+4.9%) and a 7B model to 80.8% on NExT-QA. Beyond accuracy, our analysis reveals that both QBP and QBC substantially enhance cross-dataset generalization, with QBP additionally accelerating model convergence by over 2.5x. These results demonstrate that shifting data synthesis from isolated facts to narrative coherence and grounded rationales yields a more accurate, efficient, and generalizable training paradigm.

[305] Euclid’s Gift: Enhancing Spatial Perception and Reasoning in Vision-Language Models via Geometric Surrogate Tasks

Shijie Lian,Changti Wu,Laurence Tianruo Yang,Hang Yuan,Bin Yu,Lei Zhang,Kai Chen

Main category: cs.CV

TL;DR: 该论文提出通过几何代理任务增强视觉语言模型的空间感知与推理能力，构建了一个包含3万道几何问题的数据集Euclid30K，并用GRPO方法微调模型。实验表明，该方法在多个空间推理基准上显著提升了零样本性能。

Details Motivation: 当前多模态大语言模型在空间智能（如形状变换、对象旋转、位置关系判断等）方面表现不足。论文旨在通过几何问题解决作为代理任务，提升模型的空间推理能力。

Contribution: 主要贡献包括：(1) 构建了一个高质量的几何问题数据集Euclid30K；(2) 提出使用GRPO方法微调模型，使其掌握欧几里得几何原理；(3) 首次系统性地展示了几何微调对空间技能的广泛迁移能力。

Method: 采用Euclid30K数据集和GRPO（Group Relative Policy Optimization）方法微调Qwen2.5VL和RoboBrain2.0系列模型，使其学会形状识别、计数、关系推理和多步演绎推理。

Result: 微调后的模型在多个空间推理基准（如Super-CLEVR、VSI-Bench等）上实现了显著的零样本性能提升。RoboBrain2.0-Euclid-7B以49.6%的准确率超越之前最优模型。

Insight: 几何问题可以作为有效的代理任务，显著提升视觉语言模型的空间推理能力，且这种能力能广泛迁移到其他空间任务中。

Abstract: Spatial intelligence spans a rich suite of abilities, including visualising and transforming shapes, mentally rotating objects, judging relational positions and containment, and estimating numerosity. However, it still remains a critical unresolved challenge for Multimodal Large Language Models (MLLMs).To fill this gap, we propose to treat Euclidean geometry problem-solving as a surrogate task. Specifically, we meticulously constructed a curated multimodal dataset, called Euclid30K, comprising approximately 30K plane and solid geometry problems. To enable the model to acquire and apply Euclidean principles from these geometry problems, we employed Group Relative Policy Optimization (GRPO) to finetune the Qwen2.5VL family and RoboBrain2.0 family, inspiring the models to identify shapes, count, and relate entities, and perform multi-step deductive reasoning using Euclidean principles. Our experiments demonstrate that the resulting models achieve substantial zero-shot gains across four spatial reasoning benchmarks (Super-CLEVR, Omni3DBench, VSI-Bench, and MindCube) without any task-specific adaptations. Notably, after training on the Euclid30K, the mean VSI-Bench accuracy of all evaluated models rose from 34.5% to 40.5%, improving by 5.5 percentage points. Among them, RoboBrain2.0-Euclid-7B achieves 49.6% accuracy, surpassing the previous state-of-the-art model, Spatial-MLLM.To our knowledge, this is the first systematic study showing that geometry-centric fine-tuning can confer vision-language models with broadly transferable spatial skills. Code and Euclid30K dataset can be found in https://zgca-ai4edu.github.io/Euclids_Gift.

[306] Generalist Multi-Class Anomaly Detection via Distillation to Two Heterogeneous Student Networks

Hangil Park,Yongmin Seo,Tae-Kyun Kim

Main category: cs.CV

TL;DR: 论文提出了一种基于知识蒸馏的双模型集成方法，用于多类别异常检测，结合了针对工业检测的小缺陷和语义异常检测的两种异质性学生模型，实现了跨领域的泛化性能。

Details Motivation: 现有的异常检测方法往往偏向工业检测或语义异常检测单一任务，难以在多类别和跨领域任务中表现一致，缺乏通用性。本文旨在通过双模型集成的知识蒸馏方法解决这一问题。

Contribution: 1. 提出了基于知识蒸馏的双模型集成框架，结合工业检测（Encoder-Decoder）和语义检测（Encoder-Encoder）两种异质性学生模型；2. 利用共享预训练编码器（DINOv2）提取高质量特征；3. 设计了Noisy-OR目标函数联合训练双模型。

Method: 1. 使用DINOv2作为共享编码器；2. 设计Encoder-Decoder模型（工业检测）和Encoder-Encoder模型（语义检测）；3. 通过Noisy-OR目标联合训练；4. 结合局部和语义异常分数计算最终异常得分。

Result: 在8个公开基准测试（包括MVTec-AD、CIFAR-10等）上实现了SOTA性能，MVTec-AD图像级AUROC达99.7%，CIFAR-10达97.8%，优于现有通用模型和部分专用模型。

Insight: 1. 双模型集成能够有效结合不同任务的专长；2. 共享预训练编码器有助于跨领域特征的泛化；3. Noisy-OR目标能够平衡两种任务的优化。

Abstract: Anomaly detection (AD) plays an important role in various real-world applications. Recent advancements in AD, however, are often biased towards industrial inspection, struggle to generalize to broader tasks like semantic anomaly detection and vice versa. Although recent methods have attempted to address general anomaly detection, their performance remains sensitive to dataset-specific settings and single-class tasks. In this paper, we propose a novel dual-model ensemble approach based on knowledge distillation (KD) to bridge this gap. Our framework consists of a teacher and two student models: an Encoder-Decoder model, specialized in detecting patch-level minor defects for industrial AD and an Encoder-Encoder model, optimized for semantic AD. Both models leverage a shared pre-trained encoder (DINOv2) to extract high-quality feature representations. The dual models are jointly learned using the Noisy-OR objective, and the final anomaly score is obtained using the joint probability via local and semantic anomaly scores derived from the respective models. We evaluate our method on eight public benchmarks under both single-class and multi-class settings: MVTec-AD, MVTec-LOCO, VisA and Real-IAD for industrial inspection and CIFAR-10/100, FMNIST and View for semantic anomaly detection. The proposed method achieved state-of-the-art accuracies in both domains, in multi-class as well as single-class settings, demonstrating generalization across multiple domains of anomaly detection. Our model achieved an image-level AUROC of 99.7% on MVTec-AD and 97.8% on CIFAR-10, which is significantly better than the prior general AD models in multi-class settings and even higher than the best specialist models on individual benchmarks.

[307] LaMoGen: Laban Movement-Guided Diffusion for Text-to-Motion Generation

Heechang Kim,Gwanghyun Kim,Se Young Chun

Main category: cs.CV

TL;DR: LaMoGen提出了一种基于拉班运动分析的文本到动作生成方法，通过结合Laban Effort和Shape组件，实现了细粒度和可解释的动作控制。

Details Motivation: 现有的文本到动作生成模型缺乏对动作风格的细粒度控制，且难以通过自然语言表达定量特征。拉班运动分析被舞蹈专家广泛用于描述动作细节，因此作者希望将其融入生成模型中以提高控制性。

Contribution: 提出了一种零样本推理优化方法，通过在预训练扩散模型的采样步骤中更新文本嵌入，实现了对Laban Effort和Shape组件的细粒度控制，无需额外动作数据。

Method: 在预训练的扩散模型中，通过在采样步骤动态优化文本嵌入，引导生成的动作符合目标Laban标签，同时保持动作身份不变。

Result: 实验表明，LaMoGen能够生成多样化的表达性动作，并根据目标Laban标签成功操纵动作属性。

Insight: 将舞蹈领域的量化方法（如拉班分析）融入生成模型，能够显著提高动作生成的细粒度控制和可解释性。

Abstract: Diverse human motion generation is an increasingly important task, having various applications in computer vision, human-computer interaction and animation. While text-to-motion synthesis using diffusion models has shown success in generating high-quality motions, achieving fine-grained expressive motion control remains a significant challenge. This is due to the lack of motion style diversity in datasets and the difficulty of expressing quantitative characteristics in natural language. Laban movement analysis has been widely used by dance experts to express the details of motion including motion quality as consistent as possible. Inspired by that, this work aims for interpretable and expressive control of human motion generation by seamlessly integrating the quantification methods of Laban Effort and Shape components into the text-guided motion generation models. Our proposed zero-shot, inference-time optimization method guides the motion generation model to have desired Laban Effort and Shape components without any additional motion data by updating the text embedding of pretrained diffusion models during the sampling step. We demonstrate that our approach yields diverse expressive motion qualities while preserving motion identity by successfully manipulating motion attributes according to target Laban tags.

[308] NeMo: Needle in a Montage for Video-Language Understanding

Zi-Yuan Hu,Shuo Liang,Duo Zheng,Yanyang Li,Yeyao Tao,Shijia Huang,Wei Feng,Jia Qin,Jianguang Yu,Jing Huang,Meng Fang,Yin Li,Liwei Wang

Main category: cs.CV

TL;DR: 论文提出了NeMo任务和NeMoBench基准，旨在评估视频大语言模型（VideoLLMs）的长上下文回忆和时间定位能力，并通过自动化数据生成管道生成高质量QA对。

Details Motivation: 现有视频语言模型在复杂时序推理能力上的评估缺乏有效协议和基准，亟需新方法填补这一空白。

Contribution: 1. 提出NeMo任务，专注于长上下文回忆和时间定位；2. 开发自动化数据生成管道；3. 构建NeMoBench基准，包含大量多样化视频和QA对。

Method: 1. 通过自动化管道模拟‘大海捞针’测试；2. 生成多时长视频的QA对；3. 评估20种顶尖模型的性能。

Result: 实验证明自动化管道可靠生成数据，NeMoBench有效揭示模型能力与局限。

Insight: 长视频处理与时序理解仍是VideoLLMs的重要挑战。

Abstract: Recent advances in video large language models (VideoLLMs) call for new evaluation protocols and benchmarks for complex temporal reasoning in video-language understanding. Inspired by the needle in a haystack test widely used by LLMs, we introduce a novel task of Needle in a Montage (NeMo), designed to assess VideoLLMs’ critical reasoning capabilities, including long-context recall and temporal grounding. To generate video question answering data for our task, we develop a scalable automated data generation pipeline that facilitates high-quality data synthesis. Built upon the proposed pipeline, we present NeMoBench, a video-language benchmark centered on our task. Specifically, our full set of NeMoBench features 31,378 automatically generated question-answer (QA) pairs from 13,486 videos with various durations ranging from seconds to hours. Experiments demonstrate that our pipeline can reliably and automatically generate high-quality evaluation data, enabling NeMoBench to be continuously updated with the latest videos. We evaluate 20 state-of-the-art models on our benchmark, providing extensive results and key insights into their capabilities and limitations. Our project page is available at: https://lavi-lab.github.io/NeMoBench.

[309] Performance-Efficiency Trade-off for Fashion Image Retrieval

Julio Hurtado,Haoran Ni,Duygu Sap,Connor Mattinson,Martin Lotz

Main category: cs.CV

TL;DR: 本文提出了一种选择性表示框架，用于时尚图像检索的性能与效率权衡，通过聚类和核心集选择方法缩小数据库规模，同时保持检索准确性。

Details Motivation: 时尚产业对环境的影响引发了对二手市场的兴趣，本文旨在通过机器学习方法优化时尚图像检索的可扩展性。

Contribution: 提出了选择性表示框架，结合聚类、核心集选择和离群点剔除方法，显著减少数据库规模而不损失检索精度。

Method: 采用聚类和核心集选择提取代表性样本，引入基于邻域一致性评分的离群点剔除方法。

Result: 在三个公开数据集上验证，数据库规模缩减至10%，同时保持检索准确性。离群点剔除进一步提升性能。

Insight: 性能与效率的权衡可以通过策略性样本选择和离群点剔除实现，为大规模时尚图像检索提供了实用解决方案。

Abstract: The fashion industry has been identified as a major contributor to waste and emissions, leading to an increased interest in promoting the second-hand market. Machine learning methods play an important role in facilitating the creation and expansion of second-hand marketplaces by enabling the large-scale valuation of used garments. We contribute to this line of work by addressing the scalability of second-hand image retrieval from databases. By introducing a selective representation framework, we can shrink databases to 10% of their original size without sacrificing retrieval accuracy. We first explore clustering and coreset selection methods to identify representative samples that capture the key features of each garment and its internal variability. Then, we introduce an efficient outlier removal method, based on a neighbour-homogeneity consistency score measure, that filters out uncharacteristic samples prior to selection. We evaluate our approach on three public datasets: DeepFashion Attribute, DeepFashion Con2Shop, and DeepFashion2. The results demonstrate a clear performance-efficiency trade-off by strategically pruning and selecting representative vectors of images. The retrieval system maintains near-optimal accuracy, while greatly reducing computational costs by reducing the images added to the vector database. Furthermore, applying our outlier removal method to clustering techniques yields even higher retrieval performance by removing non-discriminative samples before the selection.

[310] Mitigating Visual Hallucinations via Semantic Curriculum Preference Optimization in MLLMs

Yuanshuai Li,Yuping Yan,Junfeng Tang,Yunxuan Li,Zeqi Zheng,Yaochu Jin

Main category: cs.CV

TL;DR: 论文提出了一种名为SCPO的新框架，通过语义课程偏好优化来减少多模态大语言模型中的视觉幻觉问题，显著提高了模型性能和事实性。

Details Motivation: 多模态大语言模型（MLLMs）在生成响应时容易出现视觉幻觉，即生成的文本与视觉证据不符。现有方法如DPO难以捕捉细粒度语义差异且容易鼓励捷径学习。

Contribution: 提出了SCPO框架，首次将语义、对称性和课程学习统一用于MLLMs对齐，有效减少了视觉幻觉问题。

Method: SCPO采用基于语义课程偏好对的渐进式学习策略，结合动态参考模型和双向对称目标，同时学习文本和视觉偏好。

Result: 实验证明SCPO在LLaVA模型上显著降低了幻觉率（最高62.9%），并在广泛基准测试中保持了通用能力。

Insight: 通过结合语义课程和对称优化，SCPO不仅解决了视觉幻觉问题，还提升了模型的事实性和稳定性。

Abstract: Multimodal Large Language Models (MLLMs) have significantly improved the performance of various tasks, but continue to suffer from visual hallucinations, a critical issue where generated responses contradict visual evidence. While Direct Preference Optimization(DPO) is widely used for alignment, its application to MLLMs often fails to capture fine-grained semantic differences and encourages shortcut learning. To address these challenges, we propose Semantic Curriculum Preference Optimization (SCPO), a novel framework for MLLM alignment. SCPO employs a progressive, easy-to-hard curriculum built upon our Semantic Curriculum Preference Pairs dataset, which provides fine-grained semantic contrasts sorted by difficulty. This curriculum is trained with a dynamic reference model and a novel symmetric, bidirectional objective to facilitate simultaneous learning from both textual and visual preferences. To our knowledge, SCPO is the first framework to unify semantics, symmetry, and curriculum for MLLMs alignment, effectively mitigating visual hallucinations. Extensive experiments on LLaVA models across various scales and versions validate that SCPO demonstrates superior performance compared to baseline models on multiple hallucination benchmarks, reducing the hallucination rate by up to 62.9%. Moreover, evaluations on generalized benchmarks show that SCPO improves factuality while preserving general capabilities, with its performance remaining stable across general vision-language benchmarks.

[311] Robust Multimodal Semantic Segmentation with Balanced Modality Contributions

Jiaqi Tan,Xu Zheng,Fangyu Li,Yang Liu

Main category: cs.CV

TL;DR: 本文提出了EQUISeg，一种通过均衡模态贡献来解决多模态语义分割中模态不平衡问题的框架，采用交叉模态Transformer块和自引导模块实现高效融合和鲁棒性提升。

Details Motivation: 现有的多模态语义分割方法在主导模态性能下降时整体表现显著退化，亟需解决模态不平衡问题以提高实用性和鲁棒性。

Contribution: 1) 提出EQUISeg框架，通过均衡模态编码实现多模态平衡；2) 设计四阶段交叉模态Transformer块(CMTB)实现高效多模态融合；3) 引入自引导模块(SGM)动态调整模态贡献。

Method: 1) 使用CMTB实现层次化模态选择和融合；2) 通过SGM引入互引导机制动态优化模态权重；3) 在多模态任务中实现均衡性能。

Result: 在多个数据集上的实验表明，EQUISeg显著提升了性能，并有效缓解了模态不平衡对分割任务的负面影响。

Insight: 通过模态均衡和动态自适应机制，多模态模型的鲁棒性可以显著提升，尤其在主导模态退化时仍能保持稳定表现。

Abstract: Multimodal semantic segmentation enhances model robustness by exploiting cross-modal complementarities. However, existing methods often suffer from imbalanced modal dependencies, where overall performance degrades significantly once a dominant modality deteriorates in real-world scenarios. Thus, modality balance has become acritical challenge for practical multimodal segmentation. To address this issue, we propose EQUISeg, a multimodal segmentation framework that balances modality contributions through equal encoding of modalities. Built upon a four-stage Cross-modal Transformer Block(CMTB), EQUISeg enables efficient multimodal fusion and hierarchical selection. Furthermore, we design a Self-guided Module(SGM) that mitigates modality imbalance by introducing a mutual guidance mechanism, enabling each modality to adaptively adjust its contribution and enhance robustness under degraded conditions. Extensive experiments on multiple datasets demonstrate that EQUISeg achieves significant performance gains and effectively alleviates the adverse effects of modality imbalance in segmentation tasks.

[312] CMT: Mid-Training for Efficient Learning of Consistency, Mean Flow, and Flow Map Models

Zheyuan Hu,Chieh-Hsin Lai,Yuki Mitsufuji,Stefano Ermon

Main category: cs.CV

TL;DR: 该论文提出了一种称为“中期训练”（CMT）的新方法，用于高效训练一致性模型和均值流模型，解决了传统训练中的不稳定性和高成本问题。

Details Motivation: 传统的一致性模型（CM）和均值流（MF）在训练中存在不稳定性和对超参数敏感的问题，且计算成本高。尽管从预训练扩散模型初始化有助于缓解部分问题，但仍需解决从小步长到大步长的转换不稳定性。

Contribution: 提出了中期训练（CMT），作为预训练和后训练之间的轻量级中间阶段，能够生成稳定且轨迹一致的初始化模型，从而简化后续流映射学习并显著提升效率。

Method: CMT通过在预训练和后训练之间插入一个轻量级阶段，训练模型直接从预训练样本映射到解算器生成的干净样本，实现稳定初始化。

Result: CMT在多个数据集上实现了最先进的生成效果，如CIFAR-10上两步FID为1.97，ImageNet 512x512上为1.84，同时减少了高达98%的训练数据和GPU时间。

Insight: CMT的核心在于通过轨迹一致性初始化避免了不稳定性和高成本问题，为高效训练流映射模型提供了一种通用且可靠的方法。

Abstract: Flow map models such as Consistency Models (CM) and Mean Flow (MF) enable few-step generation by learning the long jump of the ODE solution of diffusion models, yet training remains unstable, sensitive to hyperparameters, and costly. Initializing from a pre-trained diffusion model helps, but still requires converting infinitesimal steps into a long-jump map, leaving instability unresolved. We introduce mid-training, the first concept and practical method that inserts a lightweight intermediate stage between the (diffusion) pre-training and the final flow map training (i.e., post-training) for vision generation. Concretely, Consistency Mid-Training (CMT) is a compact and principled stage that trains a model to map points along a solver trajectory from a pre-trained model, starting from a prior sample, directly to the solver-generated clean sample. It yields a trajectory-consistent and stable initialization. This initializer outperforms random and diffusion-based baselines and enables fast, robust convergence without heuristics. Initializing post-training with CMT weights further simplifies flow map learning. Empirically, CMT achieves state of the art two step FIDs: 1.97 on CIFAR-10, 1.32 on ImageNet 64x64, and 1.84 on ImageNet 512x512, while using up to 98% less training data and GPU time, compared to CMs. On ImageNet 256x256, CMT reaches 1-step FID 3.34 while cutting total training time by about 50% compared to MF from scratch (FID 3.43). This establishes CMT as a principled, efficient, and general framework for training flow map models.

[313] MMRQA: Signal-Enhanced Multimodal Large Language Models for MRI Quality Assessment

Fankai Jia,Daisong Gan,Zhe Zhang,Zhaochi Wen,Chenchen Dan,Dong Liang,Haifeng Wang

Main category: cs.CV

TL;DR: MMRQA框架结合多模态大语言模型（MLLMs）和信号处理技术，通过模拟伪影、QA对转换和LoRA参数高效融合，实现了MRI质量评估的高性能与可解释性。

Details Motivation: 传统MRI质量评估方法在定量指标和语义理解之间存在权衡，亟需一种既能提供准确结果又具可解释性的解决方案。

Contribution: 1）提出MMRQA框架，整合MLLMs和信号处理；2）通过模拟伪影增强指标提取；3）将指标转化为QA对；4）使用LoRA高效融合参数。

Method: 1）基于MRQy提取指标并模拟伪影；2）用Qwen将指标转为QA对；3）通过LoRA融合LLaVA-OneVision的多模态特征。

Result: 在MR-ART、FastMRI和MyConnectome基准测试中取得SOTA性能，表现强大的零样本泛化能力。

Insight: MMRQA通过量化分析与语义推理的结合，为动态医疗环境中的质量评估提供了可解释且高效的解决方案。

Abstract: Magnetic resonance imaging (MRI) quality assessment is crucial for clinical decision-making, yet remains challenging due to data scarcity and protocol variability. Traditional approaches face fundamental trade-offs: signal-based methods like MRIQC provide quantitative metrics but lack semantic understanding, while deep learning approaches achieve high accuracy but sacrifice interpretability. To address these limitations, we introduce the Multimodal MRI Quality Assessment (MMRQA) framework, pioneering the integration of multimodal large language models (MLLMs) with acquisition-aware signal processing. MMRQA combines three key innovations: robust metric extraction via MRQy augmented with simulated artifacts, structured transformation of metrics into question-answer pairs using Qwen, and parameter-efficient fusion through Low-Rank Adaptation (LoRA) of LLaVA-OneVision. Evaluated on MR-ART, FastMRI, and MyConnectome benchmarks, MMRQA achieves state-of-the-art performance with strong zero-shot generalization, as validated by comprehensive ablation studies. By bridging quantitative analysis with semantic reasoning, our framework generates clinically interpretable outputs that enhance quality control in dynamic medical settings.

[314] CORE-3D: Context-aware Open-vocabulary Retrieval by Embeddings in 3D

Mohamad Amin Mirzaei,Pantea Amoie,Ali Ekhterachian,Matin Mirzababaei

Main category: cs.CV

TL;DR: CORE-3D提出了一种上下文感知的开放词汇3D检索方法，通过改进语义分割和CLIP编码策略，显著提升了3D场景理解的性能。

Details Motivation: 现有方法在进行3D语义分割和检索时，由于依赖2D类无关掩码的直接投影，导致分割碎片化和语义分配不准确。这限制了在复杂环境中的应用效果。

Contribution: 1. 引入SemanticSAM和渐进式粒度细化，生成更精准的对象级掩码；2. 提出上下文感知的CLIP编码策略，融合多视角信息增强语义上下文；3. 在多个3D场景理解任务上验证了方法的有效性。

Method: 1. 使用SemanticSAM生成对象级掩码，避免过分割问题；2. 借助渐进式粒度细化提升掩码质量；3. 结合多视角加权CLIP编码丰富语义上下文。

Result: 在多个基准数据集上的实验表明，该方法在3D语义分割和语言查询对象检索任务中显著优于现有方法。

Insight: 通过改进掩码生成和语义编码策略，可以有效提升3D场景理解的开放词汇检索能力，尤其在复杂环境中表现突出。

Abstract: 3D scene understanding is fundamental for embodied AI and robotics, supporting reliable perception for interaction and navigation. Recent approaches achieve zero-shot, open-vocabulary 3D semantic mapping by assigning embedding vectors to 2D class-agnostic masks generated via vision-language models (VLMs) and projecting these into 3D. However, these methods often produce fragmented masks and inaccurate semantic assignments due to the direct use of raw masks, limiting their effectiveness in complex environments. To address this, we leverage SemanticSAM with progressive granularity refinement to generate more accurate and numerous object-level masks, mitigating the over-segmentation commonly observed in mask generation models such as vanilla SAM, and improving downstream 3D semantic segmentation. To further enhance semantic context, we employ a context-aware CLIP encoding strategy that integrates multiple contextual views of each mask using empirically determined weighting, providing much richer visual context. We evaluate our approach on multiple 3D scene understanding tasks, including 3D semantic segmentation and object retrieval from language queries, across several benchmark datasets. Experimental results demonstrate significant improvements over existing methods, highlighting the effectiveness of our approach.

[315] Diffusion Bridge or Flow Matching? A Unifying Framework and Comparative Analysis

Kaizhen Zhu,Mokai Pan,Zhechuan Yu,Jingya Wang,Jingyi Yu,Ye Shi

Main category: cs.CV

TL;DR: 论文通过随机最优控制和最优传输的统一理论框架，首次对Diffusion Bridge与Flow Matching进行了理论分析与实验验证，证明前者成本函数更低且轨迹更稳定，而后者在数据量减少时效率下降。实验结果表明两者的优劣并存。

Details Motivation: 目前对Diffusion Bridge和Flow Matching两种方法的优劣缺乏统一的解释和理论支持，导致在实际选择时存在模糊性。

Contribution: 1. 提出了一个统一的理论框架（基于随机最优控制和最优传输）；2. 证明了Diffusion Bridge的成本函数更低，轨迹更优；3. 指出了Flow Matching在数据量减少时的表现缺陷；4. 设计了一种基于Transformer的新架构进行实验验证。

Method: 1. 通过随机最优控制理论重新建模Diffusion Bridge和Flow Matching；2. 推导成本函数和性能的理论关系；3. 提出基于Transformer的统一架构进行公平实验对比；4. 在多个任务（如图像修复、超分辨率等）中系统测试。

Result: 实验结果表明Diffusion Bridge在稳定性上表现更好，而Flow Matching在数据量减少时表现下降，验证了理论预测。

Insight: Diffusion Bridge更适合需要稳定轨迹的任务，而Flow Matching的效率依赖于充足的数据。两种方法在不同场景下有各自的优势。

Abstract: Diffusion Bridge and Flow Matching have both demonstrated compelling empirical performance in transformation between arbitrary distributions. However, there remains confusion about which approach is generally preferable, and the substantial discrepancies in their modeling assumptions and practical implementations have hindered a unified theoretical account of their relative merits. We have, for the first time, provided a unified theoretical and experimental validation of these two models. We recast their frameworks through the lens of Stochastic Optimal Control and prove that the cost function of the Diffusion Bridge is lower, guiding the system toward more stable and natural trajectories. Simultaneously, from the perspective of Optimal Transport, interpolation coefficients $t$ and $1-t$ of Flow Matching become increasingly ineffective when the training data size is reduced. To corroborate these theoretical claims, we propose a novel, powerful architecture for Diffusion Bridge built on a latent Transformer, and implement a Flow Matching model with the same structure to enable a fair performance comparison in various experiments. Comprehensive experiments are conducted across Image Inpainting, Super-Resolution, Deblurring, Denoising, Translation, and Style Transfer tasks, systematically varying both the distributional discrepancy (different difficulty) and the training data size. Extensive empirical results align perfectly with our theoretical predictions and allow us to delineate the respective advantages and disadvantages of these two models. Our code is available at https://anonymous.4open.science/r/DBFM-3E8E/.

[316] TokenSwap: Backdoor Attack on the Compositional Understanding of Large Vision-Language Models

Zhifang Zhang,Qiqi Tao,Jiaqi Lv,Na Zhao,Lei Feng,Joey Tianyi Zhou

Main category: cs.CV

TL;DR: 本文提出了TokenSwap，一种针对大型视觉语言模型（LVLM）组合理解能力的隐秘后门攻击方法。与传统固定目标模式的攻击不同，TokenSwap通过干扰对象关系的理解实现攻击，使其难以被检测。

Details Motivation: 现有LVLM后门攻击通常强制模型生成固定目标模式，易被检测。TokenSwap旨在解决这一问题，通过更隐蔽的方式攻击模型的组合理解能力。

Contribution: 提出TokenSwap，一种针对LVLM组合理解能力的隐秘后门攻击方法；引入自适应token加权损失，强化模型对扰动的学习。

Method: 在训练时注入视觉触发器并交换文本答案中关键token的语法角色；采用自适应token加权损失强调交换token的学习。

Result: TokenSwap在多个基准和LVLM架构上实现了高攻击成功率，同时保持了良好的隐蔽性和难以检测性。

Insight: 针对模型组合理解能力的攻击更具隐秘性；自适应损失函数能有效强化攻击效果。

Abstract: Large vision-language models (LVLMs) have achieved impressive performance across a wide range of vision-language tasks, while they remain vulnerable to backdoor attacks. Existing backdoor attacks on LVLMs aim to force the victim model to generate a predefined target pattern, which is either inserted into or replaces the original content. We find that these fixed-pattern attacks are relatively easy to detect, because the attacked LVLM tends to memorize such frequent patterns in the training dataset, thereby exhibiting overconfidence on these targets given poisoned inputs. To address these limitations, we introduce TokenSwap, a more evasive and stealthy backdoor attack that focuses on the compositional understanding capabilities of LVLMs. Instead of enforcing a fixed targeted content, TokenSwap subtly disrupts the understanding of object relationships in text. Specifically, it causes the backdoored model to generate outputs that mention the correct objects in the image but misrepresent their relationships (i.e., bags-of-words behavior). During training, TokenSwap injects a visual trigger into selected samples and simultaneously swaps the grammatical roles of key tokens in the corresponding textual answers. However, the poisoned samples exhibit only subtle differences from the original ones, making it challenging for the model to learn the backdoor behavior. To address this, TokenSwap employs an adaptive token-weighted loss that explicitly emphasizes the learning of swapped tokens, such that the visual triggers and bags-of-words behavior are associated. Extensive experiments demonstrate that TokenSwap achieves high attack success rates while maintaining superior evasiveness and stealthiness across multiple benchmarks and various LVLM architectures.

[317] SCOPE: Semantic Conditioning for Sim2Real Category-Level Object Pose Estimation in Robotics

Peter Hönig,Stefan Thalhammer,Jean-Baptiste Weibel,Matthias Hirschmanner,Markus Vincze

Main category: cs.CV

TL;DR: SCOPE利用扩散模型和DINOv2特征作为连续语义先验，解决了Sim2Real环境中类别级物体姿态估计的问题，性能优于现有方法31.9%，并展示了在未知类别物体上的泛化能力。

Details Motivation: 机器人在开放环境中需要处理未知物体的姿态估计，现有方法依赖离散类别标签，限制了泛化能力。SCOPE旨在通过连续的语义先验和扩散模型解决这一问题。

Contribution: 1. 提出SCOPE，一种基于扩散模型的类别级姿态估计方法；2. 利用DINOv2特征作为连续语义先验，避免依赖离散标签；3. 在Sim2Real环境下显著提升性能，并展示对未知类别物体的泛化能力。

Method: 1. 使用DINOv2特征作为连续语义先验；2. 通过交叉注意力注入语义信息；3. 结合逼真训练数据和点法线噪声模型，减少Sim2Real差距。

Result: 1. 在类别级姿态估计上，相比SOTA提升了31.9%（5°5cm指标）；2. 在未知类别物体上的抓取成功率高达100%。

Insight: 连续语义先验和扩散模型的结合可以显著提升类别级姿态估计的泛化能力，尤其在Sim2Real环境中。DINOv2特征的多模态表示能力是关键。

Abstract: Object manipulation requires accurate object pose estimation. In open environments, robots encounter unknown objects, which requires semantic understanding in order to generalize both to known categories and beyond. To resolve this challenge, we present SCOPE, a diffusion-based category-level object pose estimation model that eliminates the need for discrete category labels by leveraging DINOv2 features as continuous semantic priors. By combining these DINOv2 features with photorealistic training data and a noise model for point normals, we reduce the Sim2Real gap in category-level object pose estimation. Furthermore, injecting the continuous semantic priors via cross-attention enables SCOPE to learn canonicalized object coordinate systems across object instances beyond the distribution of known categories. SCOPE outperforms the current state of the art in synthetically trained category-level object pose estimation, achieving a relative improvement of 31.9% on the 5$^\circ$5cm metric. Additional experiments on two instance-level datasets demonstrate generalization beyond known object categories, enabling grasping of unseen objects from unknown categories with a success rate of up to 100%. Code available: https://github.com/hoenigpeter/scope.

[318] BFSM: 3D Bidirectional Face-Skull Morphable Model

Zidu Wang,Meng Xu,Miao Xu,Hengyuan Ma,Jiankuo Zhao,Xutao Li,Xiangyu Zhu,Zhen Lei

Main category: cs.CV

TL;DR: BFSM提出了一种联合面部-颅骨的3D可变形模型，解决了数据稀缺、配准精度不足等问题，支持面部与颅骨的形状推断和临床应用。

Details Motivation: 构建面部-颅骨联合模型在远程诊断、手术规划等方面具有潜力，但数据稀缺、配准精度低等问题限制了其发展。

Contribution: 1)构建了包含200多个样本的数据集；2)提出了一种新的密集光线匹配配准方法；3)开发了双向的面部-颅骨可变形模型，支持形状推断和组织厚度建模。

Method: 采用密集光线匹配配准方法确保拓扑一致性，并通过共享系数空间实现面部与颅骨的形状推断。

Result: 实验验证了方法的鲁棒性和准确性，展示了BFSM在3D重建和手术规划中的潜力。

Insight: BFSM首次将面部与颅骨的形状推断纳入统一模型，同时考虑了组织厚度的变化，提升了模型的包容性和实用性。

Abstract: Building a joint face-skull morphable model holds great potential for applications such as remote diagnostics, surgical planning, medical education, and physically based facial simulation. However, realizing this vision is constrained by the scarcity of paired face-skull data, insufficient registration accuracy, and limited exploration of reconstruction and clinical applications. Moreover, individuals with craniofacial deformities are often overlooked, resulting in underrepresentation and limited inclusivity. To address these challenges, we first construct a dataset comprising over 200 samples, including both normal cases and rare craniofacial conditions. Each case contains a CT-based skull, a CT-based face, and a high-fidelity textured face scan. Secondly, we propose a novel dense ray matching registration method that ensures topological consistency across face, skull, and their tissue correspondences. Based on this, we introduce the 3D Bidirectional Face-Skull Morphable Model (BFSM), which enables shape inference between the face and skull through a shared coefficient space, while also modeling tissue thickness variation to support one-to-many facial reconstructions from the same skull, reflecting individual changes such as fat over time. Finally, we demonstrate the potential of BFSM in medical applications, including 3D face-skull reconstruction from a single image and surgical planning prediction. Extensive experiments confirm the robustness and accuracy of our method. BFSM is available at https://github.com/wang-zidu/BFSM

[319] Biomechanical-phase based Temporal Segmentation in Sports Videos: a Demonstration on Javelin-Throw

Bikash Kumar Badatya,Vipul Baghel,Jyotirmoy Amin,Ravi Hegde

Main category: cs.CV

TL;DR: 论文提出了一种无监督框架，结合结构化最优传输（SOT）和注意力时空图卷积网络（ASTGCN），用于精英标枪投掷视频的时空分割，显著优于现有方法。

Details Motivation: 传统运动分析方法依赖人工标注或实验室设备，耗时且不可扩展，亟需自动化解决方案。

Contribution: 提出了一种基于SOT的无监督框架，无需人工标注即可识别运动阶段转换，并发布了一个新的标枪投掷视频数据集。

Method: 结合结构化最优传输（SOT）与注意力时空图卷积网络（ASTGCN），实现运动阶段的上下文感知分割。

Result: 在测试数据上达到71.02%的mAP和74.61%的F1分数，显著优于基线方法。

Insight: 结构化最优传输（SOT）可以有效增强时空模型的运动分割能力，结合无监督方法可减少对标注数据的依赖。

Abstract: Precise analysis of athletic motion is central to sports analytics, particularly in disciplines where nuanced biomechanical phases directly impact performance outcomes. Traditional analytics techniques rely on manual annotation or laboratory-based instrumentation, which are time-consuming, costly, and lack scalability. Automatic extraction of relevant kinetic variables requires a robust and contextually appropriate temporal segmentation. Considering the specific case of elite javelin-throw, we present a novel unsupervised framework for such a contextually aware segmentation, which applies the structured optimal transport (SOT) concept to augment the well-known Attention-based Spatio-Temporal Graph Convolutional Network (ASTGCN). This enables the identification of motion phase transitions without requiring expensive manual labeling. Extensive experiments demonstrate that our approach outperforms state-of-the-art unsupervised methods, achieving 71.02% mean average precision (mAP) and 74.61% F1-score on test data, substantially higher than competing baselines. We also release a new dataset of 211 manually annotated professional javelin-throw videos with frame-level annotations, covering key biomechanical phases: approach steps, drive, throw, and recovery.

[320] FreeRet: MLLMs as Training-Free Retrievers

Yuhan Zhu,Xiangyu Zeng,Chenting Wang,Xinhao Li,Yicheng Xu,Ziang Yan,Yi Wang,Limin Wang

Main category: cs.CV

TL;DR: FreeRet提出了一种无需训练的通用检索框架，直接将现有多模态大语言模型（MLLMs）转化为两阶段检索器，通过语义嵌入和推理能力实现高效检索与重排。

Details Motivation: 现有MLLMs作为检索器通常需要额外训练，而FreeRet探索了直接利用预训练MLLMs的潜力，无需训练即可实现高效混合模态检索。

Contribution: 1. 绕过词对齐层获取语义嵌入；2. 通过显式先验条件生成表征；3. 通过中性选择框架减轻重排偏差。

Method: FreeRet采用两阶段框架：1. 从MLLMs直接提取语义嵌入进行快速候选搜索；2. 利用模型推理能力进行精确重排。

Result: 在MMEB和MMEB-V2基准（46个数据集）上，FreeRet显著优于需要数百万对训练的模型。

Insight: 预训练的MLLMs在无需额外训练的条件下，通过合理设计可以成为强大的检索引擎，填补其作为通用模型的空白。

Abstract: Multimodal large language models (MLLMs) are emerging as versatile foundations for mixed-modality retrieval. Yet, they often require heavy post-hoc training to convert them into contrastive encoders for retrieval. This work asks: Can off-the-shelf MLLMs serve as powerful retrievers without additional training? We present FreeRet, a plug-and-play framework that turns any MLLM into a two-stage retriever. FreeRet first derives semantically grounded embeddings directly from the model for fast candidate search, and then exploits its reasoning ability for precise reranking. The framework contributes three advances: bypassing lexical alignment layers to obtain semantically faithful embeddings, conditioning representation generation with explicit priors, and mitigating framing effect in reranking via neutral choice framing. On the MMEB and MMEB-V2 benchmarks spanning 46 datasets, FreeRet substantially outperforms models trained on millions of pairs. Beyond benchmarks, FreeRet is model-agnostic and scales seamlessly across MLLM families and sizes, preserves their generative abilities, supports arbitrary modality combinations, and unifies retrieval, reranking, and generation into end-to-end RAG within a single model. Our findings demonstrate that pretrained MLLMs, when carefully harnessed, can serve as strong retrieval engines without training, closing a critical gap in their role as generalists.

[321] Can you SPLICE it together? A Human Curated Benchmark for Probing Visual Reasoning in VLMs

Mohamad Ballout,Okajevo Wilfred,Seyedalireza Yaghoubi,Nohayr Muhammad Abdelmoneim,Julius Mayer,Elia Bruni

Main category: cs.CV

TL;DR: SPLICE是一个基于COIN数据集的人类标注基准，旨在评估视觉语言模型（VLMs）在多维度事件推理（如时间、因果、空间、上下文和常识）上的表现。结果显示，VLMs在重排事件片段任务中表现远逊于人类，且更依赖语言先验而非视觉理解。

Details Motivation: 现有VLMs在视觉推理任务中的表现与人类存在显著差距，尤其是多维度事件推理能力尚未被充分评估。SPLICE通过人类标注数据集填补了这一空白。

Contribution: 1. 提出了SPLICE基准，包含3,381个视频和11,423个事件片段，覆盖12个类别和180个子类别；2. 评估了VLMs在多维度事件推理上的表现，揭示了其局限性。

Method: 使用COIN数据集构建SPLICE基准，人类标注事件片段并重排为连贯序列；对比人类与VLMs的表现，分析模型在不同推理维度上的差异。

Result: VLMs在重排任务中表现显著低于人类，尤其在依赖上下文和空间推理的任务中；人类标注的文本描述仅提升模型性能，但未影响人类表现。

Insight: VLMs更依赖语言先验而非视觉理解；在时间因果推理任务中表现较好，但在复杂推理任务中仍有局限性。

Abstract: In this work, we introduce SPLICE, a human-curated benchmark derived from the COIN instructional video dataset, designed to probe event-based reasoning across multiple dimensions: temporal, causal, spatial, contextual, and general knowledge. SPLICE includes 3,381 human-filtered videos spanning 12 categories and 180 sub-categories, such as sports, engineering, and housework. These videos are segmented into a total of 11,423 event clips. We evaluate both human participants and state-of-the-art vision-language models (VLMs) on the task of rearranging these clips into coherent event sequences to assess visual reasoning capabilities. Results reveal a significant gap: VLMs struggle to match human performance. While human-annotated textual descriptions improve model accuracy, they do not affect human performance, suggesting that models rely more on language priors than on visual understanding. Even with annotations, VLMs fall short of human-level reasoning, underscoring persistent challenges in visual reasoning. A deeper analysis across sub-categories shows that VLMs perform relatively better on videos where temporal and causal reasoning are dominant, compared to those where contextual and spatial reasoning are dominant. They also perform better on everyday tasks than on specialized ones.

[322] RIFLE: Removal of Image Flicker-Banding via Latent Diffusion Enhancement

Zhu,Libo,Zhou,Zihan,Liu,Xiaoyang,Zhang,Weihang,Shi,Keyu,Fu,Yifan,Zhang,Yulun

Main category: cs.CV

TL;DR: 本文提出RIFLE，一种基于扩散模型的框架，用于消除图像中的闪烁带（FB）并保留细节，同时提出模拟FB的数据生成方法和首个真实世界FB数据集。

Details Motivation: 当前拍摄屏幕时，闪烁带（FB）现象严重影响图像质量和可读性，但相关研究较少，亟需解决这一问题的理论与方法。

Contribution: 1.提出RIFLE框架，首次系统研究FB去除问题；2.设计闪烁带先验估计器（FPE）和掩码损失（ML）；3.提供FB模拟管道和真实世界数据集。

Method: 1.使用扩散模型去除FB；2.FPE预测带属性并注入网络；3.ML专注带区域监督；4.提出合成FB的数据生成方法。

Result: 在真实世界数据集上，RIFLE表现优于现有图像重建基线，尤其在严重FB情况下。

Insight: 1.FB去除需结合专门的先验和局部监督；2.数据模拟对解决稀缺数据问题至关重要；3.扩散模型适合此类细节保持任务。

Abstract: Capturing screens is now routine in our everyday lives. But the photographs of emissive displays are often influenced by the flicker-banding (FB), which is alternating bright%u2013dark stripes that arise from temporal aliasing between a camera’s rolling-shutter readout and the display’s brightness modulation. Unlike moire degradation, which has been extensively studied, the FB remains underexplored despite its frequent and severe impact on readability and perceived quality. We formulate FB removal as a dedicated restoration task and introduce Removal of Image Flicker-Banding via Latent Diffusion Enhancement, RIFLE, a diffusion-based framework designed to remove FB while preserving fine details. We propose the flicker-banding prior estimator (FPE) that predicts key banding attributes and injects it into the restoration network. Additionally, Masked Loss (ML) is proposed to concentrate supervision on banded regions without sacrificing global fidelity. To overcome data scarcity, we provide a simulation pipeline that synthesizes FB in the luminance domain with stochastic jitter in banding angle, banding spacing, and banding width. Feathered boundaries and sensor noise are also applied for a more realistic simulation. For evaluation, we collect a paired real-world FB dataset with pixel-aligned banding-free references captured via long exposure. Across quantitative metrics and visual comparisons on our real-world dataset, RIFLE consistently outperforms recent image reconstruction baselines from mild to severe flicker-banding. To the best of our knowledge, it is the first work to research the simulation and removal of FB. Our work establishes a great foundation for subsequent research in both the dataset construction and the removal model design. Our dataset and code will be released soon.

[323] Learning Object-Centric Representations Based on Slots in Real World Scenarios

Adil Kaan Akan

Main category: cs.CV

TL;DR: 该论文提出了一种基于插槽的对象中心表示学习方法，通过改进预训练扩散模型实现对图像和视频中对象的精细化控制与生成。

Details Motivation: 当前扩散模型通常将图像视为整体，依赖文本条件，难以实现对象级别的编辑。该研究旨在填补这一空白，平衡全局场景一致性与对象的解耦控制。

Contribution: 1. 提出SlotAdapt框架，将轻量级的插槽条件引入预训练扩散模型，实现对象级别的合成与编辑；2. 扩展至视频领域，通过ISA和时序聚合器保持对象的身份与动态一致性；3. 在对象发现、分割、编辑等任务中达到SOTA水平。

Method: 1. 图像领域：SlotAdapt利用注册令牌表示背景/风格，插槽条件模块表示对象；2. 视频领域：结合ISA分离对象身份与姿态，使用Transformer时序聚合器保持帧间一致性。

Result: 在对象发现、分割、组合编辑等任务中表现优异，同时支持无监督的视频对象分割与重构，实现高级编辑任务如对象移除、替换和插入。

Insight: 通过插槽条件保留了预训练模型的生成能力，同时实现对象级别的可控性，为结构化生成工具的设计提供了新思路。

Abstract: A central goal in AI is to represent scenes as compositions of discrete objects, enabling fine-grained, controllable image and video generation. Yet leading diffusion models treat images holistically and rely on text conditioning, creating a mismatch for object-level editing. This thesis introduces a framework that adapts powerful pretrained diffusion models for object-centric synthesis while retaining their generative capacity. We identify a core challenge: balancing global scene coherence with disentangled object control. Our method integrates lightweight, slot-based conditioning into pretrained models, preserving their visual priors while providing object-specific manipulation. For images, SlotAdapt augments diffusion models with a register token for background/style and slot-conditioned modules for objects, reducing text-conditioning bias and achieving state-of-the-art results in object discovery, segmentation, compositional editing, and controllable image generation. We further extend the framework to video. Using Invariant Slot Attention (ISA) to separate object identity from pose and a Transformer-based temporal aggregator, our approach maintains consistent object representations and dynamics across frames. This yields new benchmarks in unsupervised video object segmentation and reconstruction, and supports advanced editing tasks such as object removal, replacement, and insertion without explicit supervision. Overall, this work establishes a general and scalable approach to object-centric generative modeling for images and videos. By bridging human object-based perception and machine learning, it expands the design space for interactive, structured, and user-driven generative tools in creative, scientific, and practical domains.

[324] TemMed-Bench: Evaluating Temporal Medical Image Reasoning in Vision-Language Models

Junyi Zhang,Jia-Chen Gu,Wenbo Hu,Yu Zhou,Robinson Piramuthu,Nanyun Peng

Main category: cs.CV

TL;DR: TemMed-Bench是首个针对临床实践中患者随时间变化的医学图像推理评估基准，揭示了现有大型视觉语言模型（LVLMs）在时间维度上的推理能力不足，并提出了多模态检索增强的潜在改进方向。

Details Motivation: 现有医学推理基准仅关注单次就诊的图像分析，与实际临床实践中医生参考患者历史记录的需求不符，因此需要新的基准来评估LVLMs在时间维度上的表现。

Contribution: 提出了首个专注于时间医学图像推理的基准TemMed-Bench，包含三个任务（VQA、报告生成、图像对选择）和17,000余例知识库，并评估了12种LVLMs的性能。

Method: 通过TemMed-Bench测试集评估LVLMs的表现，并探索了多模态检索增强（视觉和文本）对模型性能的提升效果。

Result: 大多数LVLMs在时间医学图像推理上表现欠佳，部分模型性能接近随机猜测；多模态检索增强显著提升了VQA任务的性能（平均提升2.59%）。

Insight: 多模态检索增强是提升LVLMs在时间医学图像推理任务中的潜在方向，而现有模型的局限性表明这一领域仍需进一步研究。

Abstract: Existing medical reasoning benchmarks for vision-language models primarily focus on analyzing a patient’s condition based on an image from a single visit. However, this setting deviates significantly from real-world clinical practice, where doctors typically refer to a patient’s historical conditions to provide a comprehensive assessment by tracking their changes over time. In this paper, we introduce TemMed-Bench, the first benchmark designed for analyzing changes in patients’ conditions between different clinical visits, which challenges large vision-language models (LVLMs) to reason over temporal medical images. TemMed-Bench consists of a test set comprising three tasks - visual question-answering (VQA), report generation, and image-pair selection - and a supplementary knowledge corpus of over 17,000 instances. With TemMed-Bench, we conduct an evaluation of six proprietary and six open-source LVLMs. Our results show that most LVLMs lack the ability to analyze patients’ condition changes over temporal medical images, and a large proportion perform only at a random-guessing level in the closed-book setting. In contrast, GPT o3, o4-mini and Claude 3.5 Sonnet demonstrate comparatively decent performance, though they have yet to reach the desired level. Furthermore, we explore augmenting the input with both retrieved visual and textual modalities in the medical domain. We also show that multi-modal retrieval augmentation yields notably higher performance gains than no retrieval and textual retrieval alone across most models on our benchmark, with the VQA task showing an average improvement of 2.59%. Overall, we compose a benchmark grounded on real-world clinical practice, and it reveals LVLMs’ limitations in temporal medical image reasoning, as well as highlighting the use of multi-modal retrieval augmentation as a potentially promising direction worth exploring to address this challenge.

[325] GSM8K-V: Can Vision Language Models Solve Grade School Math Word Problems in Visual Contexts

Fan Yuan,Yuchen Yan,Yifan Jiang,Haoran Zhao,Tao Feng,Jinyan Chen,Yanwei Lou,Wenqi Zhang,Yongliang Shen,Weiming Lu,Jun Xiao,Yueting Zhuang

Main category: cs.CV

TL;DR: GSM8K-V是一个多图像视觉数学推理基准测试，通过将文本数学问题转换为视觉形式，揭示了当前视觉语言模型在解决视觉数学问题上的不足。

Details Motivation: 现有的视觉数学推理基准测试主要集中于几何问题，缺乏对多图像环境下数学文字问题的评估。GSM8K-V填补了这一空白。

Contribution: 提出了GSM8K-V基准测试，通过系统地将文本数学问题转换为视觉形式，并评估了多种模型的性能。

Method: 设计了自动化图像生成流程并结合人工标注，将GSM8K文本数据集转换为视觉形式。

Result: 现有模型在GSM8K-V上的表现显著低于文本版本的GSM8K，例如Gemini-2.5-Pro的准确率仅为46.93%。

Insight: 视觉数学推理对模型的视觉理解和上下文推理能力提出了更高要求，未来需开发更强大和通用的视觉语言模型。

Abstract: Vision language models (VLMs) achieve unified modeling of images and text, enabling them to accomplish complex real-world tasks through perception, planning, and reasoning. Among these tasks, reasoning is particularly representative, with mathematical reasoning serving as a prominent example. It highlights the high-level capability of VLMs to comprehend mathematical information in images and to perform sophisticated reasoning. Recently, numerous visual mathematical reasoning benchmarks have been proposed, but they are often restricted to geometry, lack coverage of math word problems, and rarely assess reasoning across multiple images. To address these gaps, we introduce GSM8K-V, a purely visual multi-image mathematical reasoning benchmark. GSM8K-V is built by systematically mapping each sample from the widely used text-based GSM8K into visual form. Through a carefully designed automated image-generation pipeline combined with meticulous human annotation, we curate 1,319 high-quality samples. We evaluate a wide range of open-source and closed-source models on GSM8K-V. Results show that although existing VLMs have nearly saturated performance on text-based GSM8K, there remains substantial room for improvement on GSM8K-V. For example, the best-performing model, Gemini-2.5-Pro, achieves 95.22% accuracy on GSM8K but only 46.93% on GSM8K-V. We conduct a comprehensive analysis of GSM8K-V, examining the limitations of current models as well as potential directions for improvement. GSM8K-V offers a new perspective on visual mathematical reasoning and establishes a benchmark to guide the development of more robust and generalizable VLMs.

[326] SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer

Junsong Chen,Yuyang Zhao,Jincheng Yu,Ruihang Chu,Junyu Chen,Shuai Yang,Xianbang Wang,Yicheng Pan,Daquan Zhou,Huan Ling,Haozhe Liu,Hongwei Yi,Hao Zhang,Muyang Li,Yukang Chen,Han Cai,Sanja Fidler,Ping Luo,Song Han,Enze Xie

Main category: cs.CV

TL;DR: SANA-Video是一种高效的小型扩散模型，能够生成720x1280分辨率、长达一分钟的视频，具有快速的文本-视频对齐能力和低训练成本。

Details Motivation: 当前的视频生成模型通常计算成本高且效率低下，特别是在处理高分辨率长视频时。SANA-Video旨在通过优化注意力和内存管理，实现低成本高效的视频生成。

Contribution: 1. 提出Linear DiT，利用线性注意力提高计算效率；2. 设计基于块线性注意力的恒定内存KV缓存，支持长视频生成；3. 探索高效的数据过滤和训练策略，显著降低训练成本。

Method: 1. 采用线性注意力（Linear DiT）减少计算开销；2. 提出块线性注意力的恒定内存KV缓存机制，避免传统KV缓存的高内存消耗；3. 结合高效数据预处理和训练策略，优化模型训练。

Result: SANA-Video在RTX 5090 GPU上实现高效推理（2.4倍加速），生成5秒720p视频仅需29秒，训练成本仅为同类模型的1%。

Insight: 线性注意力和块状内存管理的结合是高效长视频生成的关键，同时低训练成本使其更具实用性。

Abstract: We introduce SANA-Video, a small diffusion model that can efficiently generate videos up to 720x1280 resolution and minute-length duration. SANA-Video synthesizes high-resolution, high-quality and long videos with strong text-video alignment at a remarkably fast speed, deployable on RTX 5090 GPU. Two core designs ensure our efficient, effective and long video generation: (1) Linear DiT: We leverage linear attention as the core operation, which is more efficient than vanilla attention given the large number of tokens processed in video generation. (2) Constant-Memory KV cache for Block Linear Attention: we design block-wise autoregressive approach for long video generation by employing a constant-memory state, derived from the cumulative properties of linear attention. This KV cache provides the Linear DiT with global context at a fixed memory cost, eliminating the need for a traditional KV cache and enabling efficient, minute-long video generation. In addition, we explore effective data filters and model training strategies, narrowing the training cost to 12 days on 64 H100 GPUs, which is only 1% of the cost of MovieGen. Given its low cost, SANA-Video achieves competitive performance compared to modern state-of-the-art small diffusion models (e.g., Wan 2.1-1.3B and SkyReel-V2-1.3B) while being 16x faster in measured latency. Moreover, SANA-Video can be deployed on RTX 5090 GPUs with NVFP4 precision, accelerating the inference speed of generating a 5-second 720p video from 71s to 29s (2.4x speedup). In summary, SANA-Video enables low-cost, high-quality video generation.

[327] Enhancing Physical Plausibility in Video Generation by Reasoning the Implausibility

Yutong Hao,Chen Chen,Ajmal Saeed Mian,Chang Xu,Daochang Liu

Main category: cs.CV

TL;DR: 本文提出了一种无需训练的框架，通过显式推理物理不合理性并引导生成过程，提升了视频生成的物理合理性。

Details Motivation: 现有扩散模型依赖大规模文本-视频数据集隐式学习物理推理，成本高且难以扩展，仍可能产生违背物理规律的不合理运动。

Contribution: 1. 引入轻量级物理感知推理管道构建反事实提示；2. 提出同步解耦引导（SDG）策略，通过同步方向归一化和轨迹解耦去噪抑制不合理内容。

Method: 1. 构建反事实提示；2. 使用SDG策略，结合同步方向归一化和轨迹解耦去噪。

Result: 实验表明，该方法显著提升了物理合理性且不影响真实感，无需额外训练。消融研究验证了物理感知推理和SDG的有效性。

Insight: 显式推理物理不合理性并通过SDG策略引导生成是一种高效且可插拔的物理感知视频生成范式。

Abstract: Diffusion models can generate realistic videos, but existing methods rely on implicitly learning physical reasoning from large-scale text-video datasets, which is costly, difficult to scale, and still prone to producing implausible motions that violate fundamental physical laws. We introduce a training-free framework that improves physical plausibility at inference time by explicitly reasoning about implausibility and guiding the generation away from it. Specifically, we employ a lightweight physics-aware reasoning pipeline to construct counterfactual prompts that deliberately encode physics-violating behaviors. Then, we propose a novel Synchronized Decoupled Guidance (SDG) strategy, which leverages these prompts through synchronized directional normalization to counteract lagged suppression and trajectory-decoupled denoising to mitigate cumulative trajectory bias, ensuring that implausible content is suppressed immediately and consistently throughout denoising. Experiments across different physical domains show that our approach substantially enhances physical fidelity while maintaining photorealism, despite requiring no additional training. Ablation studies confirm the complementary effectiveness of both the physics-aware reasoning component and SDG. In particular, the aforementioned two designs of SDG are also individually validated to contribute critically to the suppression of implausible content and the overall gains in physical plausibility. This establishes a new and plug-and-play physics-aware paradigm for video generation.

[328] IWR-Bench: Can LVLMs reconstruct interactive webpage from a user interaction video?

Yang Chen,Minghao Liu,Yufan Shen,Yunwen Li,Tianyuan Huang,Xinyu Fang,Tianyu Zheng,Wenxuan Huang,Cheng Yang,Daocheng Fu,Jianbiao Mei,Rong Wu,Licheng Wen,Xuemeng Yang,Song Mao,Qunshu Lin,Zhi Yu,Yongliang Shen,Yu Qiao,Botian Shi

Main category: cs.CV

TL;DR: IWR-Bench提出了一种新的基准测试，用于评估大型视觉语言模型（LVLM）从用户交互视频中重建交互式网页的能力，揭示了当前模型在推理时间动态和事件驱动逻辑方面的不足。

Details Motivation: 现有的基准测试主要关注静态截图到代码的任务，忽视了真实网页应用中的动态交互，因此本文提出了IWR-Bench以填补这一空白。

Contribution: IWR-Bench是一个新基准测试，包含113个任务，涵盖多样化交互复杂性、视觉风格和领域，用于评估LVLM在交互网页重建中的表现。

Method: 基准测试包括用户交互视频和静态资源，评估模型在多模态推理和代码生成两方面的能力，并使用代理评测框架自动评估生成网页的功能正确性和视觉保真度。

Result: 在28个LVLM上的实验表明，最佳模型的总体得分仅为36.35%，功能正确性（24.39%）远低于视觉保真度（64.25%）。

Insight: 当前模型在推理时间动态和事件驱动逻辑方面存在显著不足，IWR-Bench为视觉语言研究提供了一个挑战性前沿。

Abstract: The webpage-to-code task requires models to understand visual representations of webpages and generate corresponding code. However, existing benchmarks primarily focus on static screenshot-to-code tasks, thereby overlooking the dynamic interactions fundamental to real-world web applications. To address this limitation, this paper introduces IWR-Bench, a novel benchmark for evaluating the capabilities of Large Vision-Language Models (LVLMs) in interactive webpage reconstruction from video. IWR-Bench comprises 113 meticulously curated tasks from 100 real-world websites, with 1,001 actions and featuring diverse interaction complexities (e.g., web games), visual styles, and domains. Aligning with standard web development practices, each task includes not only user interaction videos but also all crawled static assets (e.g., images, videos). This benchmark evaluates models on two fundamental challenges: comprehensive multi-modal reasoning to infer interaction logic from video and assets, and advanced code generation to translate this logic into functional code. An agent-as-a-judge framework with a comprehensive metric system automatically assesses the functional correctness and visual fidelity of generated webpages. Extensive experiments on 28 LVLMs reveal a significant challenge: the best model achieves an overall score of only 36.35%, as functional correctness (24.39% IFS) lags significantly behind visual fidelity (64.25% VFS). These results highlight critical limitations in current models’ ability to reason about temporal dynamics and synthesize event-driven logic, establishing IWR-Bench as a challenging frontier for vision-language research. The benchmark and evaluation code will be made publicly available. Code is available at https://github.com/L-O-I/IWR-Bench.

[329] Toward a Vision-Language Foundation Model for Medical Data: Multimodal Dataset and Benchmarks for Vietnamese PET/CT Report Generation

Huu Tien Nguyen,Dac Thai Nguyen,The Minh Duc Nguyen,Trung Thanh Nguyen,Thao Nguyen Truong,Huy Hieu Pham,Johan Barthelemy,Minh Quan Tran,Thanh Tam Nguyen,Quoc Viet Hung Nguyen,Quynh Anh Chau,Hong Son Mai,Thanh Trung Nguyen,Phi Le Nguyen

Main category: cs.CV

TL;DR: 这篇论文提出了一个新的越南语多模态医学数据集，包含大量CT-PET图像与临床报告配对数据，旨在解决医学视觉语言模型在功能成像任务和低资源语言（如越南语）中的局限性。通过引入数据增强和专家验证测试集，论文展示了该数据集显著提升了现有模型的性能。

Details Motivation: 现有的视觉语言基础模型在医学成像领域的应用受到限制，主要因为缺乏多样化的成像模态和多语言临床数据，尤其是在低资源语言（如越南语）中的数据集稀缺。

Contribution: 论文的主要贡献包括：（1）发布了首个越南语的PET/CT多模态医学数据集；（2）提出了一个训练框架，包括数据增强和专家验证；（3）通过在报告生成和视觉问答任务上的实验验证了该数据集的优越性。

Method: 论文提出了一种多模态数据集构建方法，包含大量CT-PET图像与临床报告的配对数据，并采用数据增强技术和专家验证测试集来提升模型的训练效果。

Result: 实验结果表明，该数据集显著提升了现有视觉语言基础模型在医学报告生成和视觉问答任务上的性能。

Insight: 这项研究表明，针对特定语言和医学任务的定制数据集对于提升视觉语言模型的临床实用性至关重要，尤其是在低资源语言环境中。

Abstract: Vision-Language Foundation Models (VLMs), trained on large-scale multimodal datasets, have driven significant advances in Artificial Intelligence by enabling rich cross-modal reasoning. Despite their success in general domains, applying these models to medical imaging remains challenging due to the limited availability of diverse imaging modalities and multilingual clinical data. Most existing medical VLMs are trained on a subset of imaging modalities and focus primarily on high-resource languages, thus limiting their generalizability and clinical utility. To address these limitations, we introduce a novel Vietnamese-language multimodal medical dataset comprising 1,567,062 paired CT-PET images and corresponding 2,757 full-length clinical reports. This dataset is designed to fill two pressing gaps in medical AI development: (1) the lack of PET/CT imaging data in existing VLMs training corpora, which hinders the development of models capable of handling functional imaging tasks; and (2) the underrepresentation of low-resource languages, particularly the Vietnamese language, in medical vision-language research. To the best of our knowledge, this is the first dataset to provide comprehensive PET/CT-report pairs in Vietnamese. We further introduce a training framework to enhance VLMs’ learning, including data augmentation and expert-validated test sets. We conduct comprehensive experiments benchmarking state-of-the-art VLMs on downstream tasks, including medical report generation and visual question answering. The experimental results show that incorporating our dataset significantly improves the performance of existing VLMs. We believe this dataset and benchmark will serve as a pivotal step in advancing the development of more robust VLMs for medical imaging, particularly in low-resource languages, and improving their clinical relevance in Vietnamese healthcare.

[330] Collaborating Vision, Depth, and Thermal Signals for Multi-Modal Tracking: Dataset and Algorithm

Xue-Feng Zhu,Tianyang Xu,Yifan Pan,Jinjie Gu,Xi Li,Jiwen Lu,Xiao-Jun Wu,Josef Kittler

Main category: cs.CV

TL;DR: 论文提出了一种新的多模态跟踪任务，利用了可见光RGB、深度D和热红外TIR三种模态，增强了复杂场景下的鲁棒性。作者构建了RGBDT500数据集，并提出了RDTTrack跟踪器，通过正交投影约束和提示学习技术融合三模态信息，实验结果显著优于现有双模态方法。

Details Motivation: 现有多模态跟踪方法主要集中于双模态（如RGB-D或RGB-T），在复杂场景中表现受限。本文旨在通过引入第三种互补模态（TIR）提升跟踪的鲁棒性。

Contribution: 1. 提出了首个基于RGB-D-TIR三模态的跟踪任务；2. 构建了RGBDT500数据集；3. 设计了RDTTrack跟踪器，通过正交投影约束和提示学习融合三模态信号。

Method: RDTTrack利用预训练的RGB跟踪模型，通过正交投影约束融合深度和热红外模态，并将它们作为提示集成到RGB信号中。

Result: 实验表明，RDTTrack在复杂场景中的跟踪精度和鲁棒性显著优于现有双模态方法。

Insight: 三模态信号（RGB-D-TIR）的互补性可以显著提升跟踪性能，尤其是在复杂环境中。

Abstract: Existing multi-modal object tracking approaches primarily focus on dual-modal paradigms, such as RGB-Depth or RGB-Thermal, yet remain challenged in complex scenarios due to limited input modalities. To address this gap, this work introduces a novel multi-modal tracking task that leverages three complementary modalities, including visible RGB, Depth (D), and Thermal Infrared (TIR), aiming to enhance robustness in complex scenarios. To support this task, we construct a new multi-modal tracking dataset, coined RGBDT500, which consists of 500 videos with synchronised frames across the three modalities. Each frame provides spatially aligned RGB, depth, and thermal infrared images with precise object bounding box annotations. Furthermore, we propose a novel multi-modal tracker, dubbed RDTTrack. RDTTrack integrates tri-modal information for robust tracking by leveraging a pretrained RGB-only tracking model and prompt learning techniques. In specific, RDTTrack fuses thermal infrared and depth modalities under a proposed orthogonal projection constraint, then integrates them with RGB signals as prompts for the pre-trained foundation tracking model, effectively harmonising tri-modal complementary cues. The experimental results demonstrate the effectiveness and advantages of the proposed method, showing significant improvements over existing dual-modal approaches in terms of tracking accuracy and robustness in complex scenarios.

[331] VTPerception-R1: Enhancing Multimodal Reasoning via Explicit Visual and Textual Perceptual Grounding

Yizhuo Ding,Mingkang Chen,Zhibang Feng,Tong Xiao,Wanying Qu,Wenqi Shao,Yanwei Fu

Main category: cs.CV

TL;DR: VTPerception-R1通过显式视觉和文本感知提升多模态推理能力，提出两阶段框架：感知增强微调和感知感知强化学习，显著改善推理准确性和鲁棒性。

Details Motivation: 多模态大语言模型（MLLMs）在推理时缺乏对感知证据的显式利用，限制了其性能。

Contribution: 1. 系统研究感知策略；2. 提出VTPerception-R1框架；3. 实验验证显著提升效果。

Method: 1. 第一阶段：感知增强微调；2. 第二阶段：带视觉、文本及一致性奖励的感知感知强化学习。

Result: VTPerception-R1在多个任务中显著提升推理准确性和鲁棒性。

Insight: 显式感知（尤其是文本线索）对小模型效果提升最显著。

Abstract: Multimodal large language models (MLLMs) often struggle to ground reasoning in perceptual evidence. We present a systematic study of perception strategies-explicit, implicit, visual, and textual-across four multimodal benchmarks and two MLLMs. Our findings show that explicit perception, especially when paired with textual cues, consistently yields the best improvements, particularly for smaller models. Based on this insight, we propose VTPerception-R1, a unified two-stage framework that decouples perception from reasoning. Stage 1 introduces perception-augmented fine-tuning, and Stage 2 applies perception-aware reinforcement learning with novel visual, textual, and consistency rewards. Experiments demonstrate that VTPerception-R1 significantly improves reasoning accuracy and robustness across diverse tasks, offering a scalable and auditable solution for perception-grounded multimodal reasoning. Our code is available at: https://github.com/yizhuoDi/VTPerceprion-R1.

[332] LOVE-R1: Advancing Long Video Understanding with an Adaptive Zoom-in Mechanism via Multi-Step Reasoning

Shenghao Fu,Qize Yang,Yuan-Ming Li,Xihan Wei,Xiaohua Xie,Wei-Shi Zheng

Main category: cs.CV

TL;DR: LOVE-R1提出了一种自适应缩放的多步推理机制，通过动态调整视频帧的分辨率和采样密度，解决了长视频理解中时空感知的冲突，显著提升了性能。

Details Motivation: 长视频理解在当前的大型视频语言模型（LVLMs）中仍具挑战性，因为统一的帧采样机制无法兼顾时间线索和空间细节。LOVE-R1旨在通过自适应缩放机制平衡这两者。

Contribution: 1）提出自适应缩放的多步推理机制，动态调整帧分辨率和采样密度；2）通过38k高质量的CoT数据和解耦强化微调训练模型；3）在长视频理解基准上显著超越基线模型。

Method: 1）采用慢快自适应帧采样机制；2）基于多步推理动态缩放视频片段；3）使用CoT数据和解耦强化微调（decoupled reinforcement finetuning）优化模型。

Result: 在4个常见长视频理解基准上，LOVE-R1平均比基线Qwen2.5-VL高出3.1%。

Insight: 1）动态分辨率调整是解决时空感知冲突的有效方法；2）多步推理和解耦强化微调显著提升了模型的细粒度理解能力。

Abstract: Long video understanding is still challenging for recent Large Video-Language Models (LVLMs) due to the conflict between long-form temporal understanding and detailed spatial perception. LVLMs with a uniform frame sampling mechanism, which samples frames with an equal frame size and fixed sampling rate, inevitably sacrifice either temporal clues or spatial details, resulting in suboptimal solutions. To mitigate this dilemma, we propose LOVE-R1, a model that can adaptively zoom in on a video clip. The model is first provided with densely sampled frames but in a small resolution. If some spatial details are needed, the model can zoom in on a clip of interest with a large frame resolution based on its reasoning until key visual information is obtained. The whole process is implemented as a multi-step reasoning process. To train the reasoning ability, we first finetune the model on our collected 38k high-quality CoT data and enhance it with decoupled reinforcement finetuning. As outcome rewards can not provide fine-grained process supervision, we decouple multi-step reasoning into multiple single-step reasoning and optimize the internal zoom-in ability explicitly. Experiments on long video understanding benchmarks show that our model with the slow-fast adaptive frame sampling mechanism achieves a great trade-off between sampling density and frame resolutions, and LOVE-R1 outperforms our baseline Qwen2.5-VL by an average of 3.1% points across 4 common long video understanding benchmarks.

[333] Vision Function Layer in Multimodal LLMs

Cheng Shi,Yizhou Yu,Sibei Yang

Main category: cs.CV

TL;DR: 研究发现多模态大语言模型（MLLMs）中的视觉功能解码分布在不同的解码器层中，表现为视觉功能层（VFL）。通过视觉标记交换框架揭示了这些层的顺序与人类行为一致，并提出了VFL-LoRA和VFL-select方法，显著提升了模型效率和数据选择性能。

Details Motivation: 探索MLLMs中视觉功能解码的分布规律及其与人类行为的关联，以提高模型效率和下游任务表现。

Contribution: 1.发现视觉功能层（VFL）的存在及其分布规律；2.提出视觉标记交换分析框架；3.开发VFL-LoRA和VFL-select方法，显著提升模型训练和数据选择效率。

Method: 1.视觉标记交换（Visual Token Swapping）框架分析层功能；2.VFL-LoRA：选择性微调VFL；3.VFL-select：基于VFL功能的数据自动分类与选择。

Result: 1.VFL-LoRA优于全LoRA训练；2.VFL-select以20%数据达到98%全数据性能；3.揭示VFL顺序与人类行为一致。

Insight: MLLMs的视觉功能分层且有序，与人类行为一致；选择性微调和数据分类可显著提升模型效率和性能。

Abstract: This study identifies that visual-related functional decoding is distributed across different decoder layers in Multimodal Large Language Models (MLLMs). Typically, each function, such as counting, grounding, or OCR recognition, narrows down to two or three layers, which we define as Vision Function Layers (VFL). Additionally, the depth and its order of different VFLs exhibits a consistent pattern across different MLLMs, which is well-aligned with human behaviors (e.g., recognition occurs first, followed by counting, and then grounding). These findings are derived from Visual Token Swapping, our novel analytical framework that modifies targeted KV cache entries to precisely elucidate layer-specific functions during decoding. Furthermore, these insights offer substantial utility in tailoring MLLMs for real-world downstream applications. For instance, when LoRA training is selectively applied to VFLs whose functions align with the training data, VFL-LoRA not only outperform full-LoRA but also prevent out-of-domain function forgetting. Moreover, by analyzing the performance differential on training data when particular VFLs are ablated, VFL-select automatically classifies data by function, enabling highly efficient data selection to directly bolster corresponding capabilities. Consequently, VFL-select surpasses human experts in data selection, and achieves 98% of full-data performance with only 20% of the original dataset. This study delivers deeper comprehension of MLLM visual processing, fostering the creation of more efficient, interpretable, and robust models.

[334] TACO-Net: Topological Signatures Triumph in 3D Object Classification

Anirban Ghosh,Ayan Dutta

Main category: cs.CV

TL;DR: TACO-Net提出了一种结合拓扑数据分析与图像滤波技术的3D物体分类方法，通过提取点云的拓扑特征并使用轻量级1D CNN进行分类，在合成和真实数据集上实现了SOTA性能。

Details Motivation: 3D物体分类在计算机视觉、机器人和自动驾驶等领域具有重要应用，但由于点云的无序性、不规则性和噪声，高精度分类仍然具有挑战性。

Contribution: 1. 提出了一种新颖的3D物体分类框架TACO-Net，结合拓扑数据分析和图像滤波技术；2. 提出将点云转换为体素化二值3D图像以提取拓扑特征；3. 使用轻量级1D CNN进行分类，在多个数据集上实现了SOTA性能。

Method: 1. 将点云转换为体素化二值3D图像；2. 利用拓扑数据分析提取区分性特征；3. 使用1D CNN对提取的特征进行分类。

Result: 在ModelNet40和ModelNet10上分别达到99.05%和99.52%的准确率，并在大规模真实数据集OmniObject3D和多种噪声输入下表现出强鲁棒性。

Insight: 拓扑特征在3D物体分类中具有显著优势，尤其是在处理无序和噪声点云时，TACO-Net的轻量化设计也为实际应用提供了可能性。

Abstract: 3D object classification is a crucial problem due to its significant practical relevance in many fields, including computer vision, robotics, and autonomous driving. Although deep learning methods applied to point clouds sampled on CAD models of the objects and/or captured by LiDAR or RGBD cameras have achieved remarkable success in recent years, achieving high classification accuracy remains a challenging problem due to the unordered point clouds and their irregularity and noise. To this end, we propose a novel state-of-the-art (SOTA) 3D object classification technique that combines topological data analysis with various image filtration techniques to classify objects when they are represented using point clouds. We transform every point cloud into a voxelized binary 3D image to extract distinguishing topological features. Next, we train a lightweight one-dimensional Convolutional Neural Network (1D CNN) using the extracted feature set from the training dataset. Our framework, TACO-Net, sets a new state-of-the-art by achieving $99.05%$ and $99.52%$ accuracy on the widely used synthetic benchmarks ModelNet40 and ModelNet10, and further demonstrates its robustness on the large-scale real-world OmniObject3D dataset. When tested with ten different kinds of corrupted ModelNet40 inputs, the proposed TACO-Net demonstrates strong resiliency overall.

[335] Training-Free Token Pruning via Zeroth-Order Gradient Estimation in Vision-Language Models

Youngeun Kim,Youjia Zhang,Huiling Liu,Aecheon Jung,Sunwoo Lee,Sungeun Hong

Main category: cs.CV

TL;DR: 本文提出了一种无需训练的令牌剪枝框架，通过零阶梯度估计选择对模型输出敏感的视觉令牌，显著提升了视觉语言模型的推理效率。

Details Motivation: 大型视觉语言模型（VLMs）的多模态推理能力强大，但其冗余的视觉令牌导致推理成本高昂。现有的令牌剪枝方法要么依赖不稳定的注意力分数，要么可能丢弃关键区域。为解决这些问题，作者提出了一种更高效的方法。

Contribution: 提出了一种无需训练的令牌剪枝框架，通过零阶扰动估计令牌敏感性，选择互补的视觉令牌而非重叠信息，显著提升了推理效率。

Method: 利用零阶梯度估计器在投影层的浅层扰动，通过轻量级前向传播（无需反向传播）近似令牌对输出的影响。

Result: 在多个VLMs和基准测试中，该方法能剪枝高达94.4%的令牌，同时保持准确性，并将端到端推理速度提升至基线方法的2.30倍。

Insight: 令牌敏感性估计可通过轻量级扰动实现，而无需依赖复杂的训练过程或注意力机制，为高效模型优化提供了新思路。

Abstract: Large Vision-Language Models (VLMs) enable strong multimodal reasoning but incur heavy inference costs from redundant visual tokens. Token pruning alleviates this issue, yet existing approaches face limitations. Attention-based methods rely on raw attention scores, which are often unstable across layers and heads and can lead to redundant selections. Diversity-based methods improve robustness by selecting tokens far apart in feature space but risk dropping regions needed for accurate prediction. We propose \ours, a training-free framework built on a simple intuition: tokens with higher sensitivity are more likely to influence the model’s output, and they should also capture complementary visual cues rather than overlapping information. To achieve this, we estimate token sensitivity using zeroth-order perturbations at the projection layer, a shallow and computationally light component of the model. This approach measures how small random perturbations affect the projection outputs, allowing us to approximate each token’s influence through lightweight forward passes without backpropagation. Extensive experiments across multiple VLMs and benchmarks show that \ours consistently outperforms prior methods, pruning up to 94.4% of tokens while maintaining accuracy and significantly improving efficiency, achieving up to 2.30x faster end-to-end inference over the baseline.

[336] ELPG-DTFS: Prior-Guided Adaptive Time-Frequency Graph Neural Network for EEG Depression Diagnosis

Jingru Qiu,Jiale Liang,Xuanhan Fan,Mingda Zhang,Zhenli He

Main category: cs.CV

TL;DR: ELPG-DTFS是一种基于先验知识的自适应时频图神经网络，用于EEG抑郁症诊断，通过引入通道-频带注意力、动态功能链接和神经科学先验知识，显著提升了诊断性能。

Details Motivation: 抑郁症诊断依赖主观量表，EEG提供了低成本生物标志物，但现有深度模型忽略了动态功能链接和先验知识，限制了准确性和解释性。

Contribution: 提出ELPG-DTFS，包含通道-频带注意力、动态功能链接的可学习邻接矩阵和神经科学先验知识注入。

Method: 通道-频带注意力捕捉跨频带信息、动态邻接矩阵建模功能链接、残差知识图通路注入先验知识。

Result: 在MODMA数据集上达到97.63%准确率和97.33% F1分数，超越现有方法ACM-GNN。

Insight: 动态功能链接和先验知识的结合显著提升了EEG抑郁症诊断的性能和可解释性。

Abstract: Timely and objective screening of major depressive disorder (MDD) is vital, yet diagnosis still relies on subjective scales. Electroencephalography (EEG) provides a low-cost biomarker, but existing deep models treat spectra as static images, fix inter-channel graphs, and ignore prior knowledge, limiting accuracy and interpretability. We propose ELPG-DTFS, a prior-guided adaptive time-frequency graph neural network that introduces: (1) channel-band attention with cross-band mutual information, (2) a learnable adjacency matrix for dynamic functional links, and (3) a residual knowledge-graph pathway injecting neuroscience priors. On the 128-channel MODMA dataset (53 subjects), ELPG-DTFS achieves 97.63% accuracy and 97.33% F1, surpassing the 2025 state-of-the-art ACM-GNN. Ablation shows that removing any module lowers F1 by up to 4.35, confirming their complementary value. ELPG-DTFS thus offers a robust and interpretable framework for next-generation EEG-based MDD diagnostics.

[337] Vision At Night: Exploring Biologically Inspired Preprocessing For Improved Robustness Via Color And Contrast Transformations

Lorena Stracke,Lia Nimmermann,Shashank Agnihotri,Margret Keuper,Volker Blanz

Main category: cs.CV

TL;DR: 论文提出一种基于生物视觉机制的输入预处理方法，通过DoG滤波增强对比度和颜色对立通道，提升语义分割模型在夜间、雾、雪等恶劣条件下的鲁棒性，同时不影响原始性能。

Details Motivation: 受人类视觉系统中对比度增强和颜色对立机制的启发，研究旨在改善语义分割模型在恶劣条件下的鲁棒性，无需修改模型架构或训练过程。

Contribution: 1）提出一种生物启发的预处理方法，通过DoG滤波增强输入图像的对比度和颜色对立通道；2）验证该方法在Cityscapes、ACDC和Dark Zurich数据集上提升模型鲁棒性，同时保持原始性能。

Method: 对RGB、灰度及颜色对立通道应用Difference-of-Gaussians (DoG)滤波，增强局部对比度。该方法轻量且模型无关。

Result: 预处理方法在不修改模型的情况下，提升了语义分割模型在夜间、雾、雪等恶劣条件下的鲁棒性，同时不影响标准数据集上的性能。

Insight: 1）生物视觉机制可为计算机视觉任务提供有效启发；2）轻量预处理可显著提升模型鲁棒性，无需复杂架构调整。

Abstract: Inspired by the human visual system’s mechanisms for contrast enhancement and color-opponency, we explore biologically motivated input preprocessing for robust semantic segmentation. By applying Difference-of-Gaussians (DoG) filtering to RGB, grayscale, and opponent-color channels, we enhance local contrast without modifying model architecture or training. Evaluations on Cityscapes, ACDC, and Dark Zurich show that such preprocessing maintains in-distribution performance while improving robustness to adverse conditions like night, fog, and snow. As this processing is model-agnostic and lightweight, it holds potential for integration into imaging pipelines, enabling imaging systems to deliver task-ready, robust inputs for downstream vision models in safety-critical environments.

[338] StreamForest: Efficient Online Video Understanding with Persistent Event Memory

Xiangyu Zeng,Kefan Qiu,Qingyu Zhang,Xinhao Li,Jing Wang,Jiaxin Li,Ziang Yan,Kun Tian,Meng Tian,Xinhai Zhao,Yi Wang,Limin Wang

Main category: cs.CV

TL;DR: StreamForest是一个专为实时视频理解设计的架构，通过持久事件记忆森林和细粒度时空窗口提升实时感知能力，并在多个基准测试中取得最先进性能。

Details Motivation: 现有的多模态大语言模型(MLLMs)在实时流视频场景中的表现受限，主要由于历史视觉特征的存储限制和实时时空推理不足。

Contribution: 1. 提出StreamForest架构，包含持久事件记忆森林和细粒度时空窗口；2. 发布了专门用于流视频任务的指令调优数据集OnlineIT；3. 提出新的自动驾驶场景基准测试ODV-Bench。

Method: 1. 持久事件记忆森林通过基于时间距离、内容相似性和合并频率的惩罚函数组织视频帧；2. 细粒度时空窗口捕捉短期视觉线索；3. 使用OnlineIT数据集进行指令调优。

Result: StreamForest在StreamingBench、OVBench和OVO-Bench上的准确率分别为77.3%、60.5%和55.6%，即使在极端的视觉令牌压缩下（1024令牌），仍能保持96.8%的平均准确率。

Insight: StreamForest的设计展示了在有限计算资源下高效保留长期记忆的潜力，同时强调了实时感知与长期推理的结合对视频理解的重要性。

Abstract: Multimodal Large Language Models (MLLMs) have recently achieved remarkable progress in video understanding. However, their effectiveness in real-time streaming scenarios remains limited due to storage constraints of historical visual features and insufficient real-time spatiotemporal reasoning. To address these challenges, we propose StreamForest, a novel architecture specifically designed for streaming video understanding. Central to StreamForest is the Persistent Event Memory Forest, a memory mechanism that adaptively organizes video frames into multiple event-level tree structures. This process is guided by penalty functions based on temporal distance, content similarity, and merge frequency, enabling efficient long-term memory retention under limited computational resources. To enhance real-time perception, we introduce a Fine-grained Spatiotemporal Window, which captures detailed short-term visual cues to improve current scene perception. Additionally, we present OnlineIT, an instruction-tuning dataset tailored for streaming video tasks. OnlineIT significantly boosts MLLM performance in both real-time perception and future prediction. To evaluate generalization in practical applications, we introduce ODV-Bench, a new benchmark focused on real-time streaming video understanding in autonomous driving scenarios. Experimental results demonstrate that StreamForest achieves the state-of-the-art performance, with accuracies of 77.3% on StreamingBench, 60.5% on OVBench, and 55.6% on OVO-Bench. In particular, even under extreme visual token compression (limited to 1024 tokens), the model retains 96.8% of its average accuracy in eight benchmarks relative to the default setting. These results underscore the robustness, efficiency, and generalizability of StreamForest for streaming video understanding.

[339] Environment-Aware Satellite Image Generation with Diffusion Models

Nikos Kostagiolas,Pantelis Georgiades,Yannis Panagakis,Mihalis A. Nicolaou

Main category: cs.CV

TL;DR: 该论文提出了一个基于扩散模型的环境感知卫星图像生成方法，通过结合文本、元数据和视觉数据三种控制信号，解决了现有方法在环境上下文不足、数据缺失或损坏以及用户意图反映不充分等问题。

Details Motivation: 现有扩散模型在遥感图像生成中存在环境上下文利用不足、数据缺失问题处理不佳以及用户意图难以准确反映的局限。

Contribution: 1) 首次将动态环境条件作为卫星图像生成的控制信号之一；2) 提出了一种元数据融合策略，处理部分损坏或缺失的观测数据；3) 发布了一个结合文本、元数据和视觉数据的多模态公开数据集。

Method: 通过扩散模型，结合文本、元数据和视觉数据三种控制信号，动态生成卫星图像；采用元数据融合策略建模属性嵌入交互，提升对缺失或损坏数据的鲁棒性。

Result: 该方法在单图像和时间序列生成中均优于现有方法，定性（对缺失元数据的鲁棒性和控制输入的响应性）和定量（更高保真度、准确性和生成质量）评估指标均有提升。

Insight: 环境上下文的引入显著提升了卫星图像生成的性能，为下游任务提供了潜在的高质量数据生成工具。

Abstract: Diffusion-based foundation models have recently garnered much attention in the field of generative modeling due to their ability to generate images of high quality and fidelity. Although not straightforward, their recent application to the field of remote sensing signaled the first successful trials towards harnessing the large volume of publicly available datasets containing multimodal information. Despite their success, existing methods face considerable limitations: they rely on limited environmental context, struggle with missing or corrupted data, and often fail to reliably reflect user intentions in generated outputs. In this work, we propose a novel diffusion model conditioned on environmental context, that is able to generate satellite images by conditioning from any combination of three different control signals: a) text, b) metadata, and c) visual data. In contrast to previous works, the proposed method is i) to our knowledge, the first of its kind to condition satellite image generation on dynamic environmental conditions as part of its control signals, and ii) incorporating a metadata fusion strategy that models attribute embedding interactions to account for partially corrupt and/or missing observations. Our method outperforms previous methods both qualitatively (robustness to missing metadata, higher responsiveness to control inputs) and quantitatively (higher fidelity, accuracy, and quality of generations measured using 6 different metrics) in the trials of single-image and temporal generation. The reported results support our hypothesis that conditioning on environmental context can improve the performance of foundation models for satellite imagery, and render our model a promising candidate for usage in downstream tasks. The collected 3-modal dataset is to our knowledge, the first publicly-available dataset to combine data from these three different mediums.

[340] ThermalGen: Style-Disentangled Flow-Based Generative Models for RGB-to-Thermal Image Translation

Jiuhong Xiao,Roshan Nayak,Ning Zhang,Daniel Tortei,Giuseppe Loianno

Main category: cs.CV

TL;DR: ThermalGen是一个基于流的生成模型，用于RGB到热图像的转换，具有风格解耦机制和多数据集支持，性能优于现有方法。

Details Motivation: 由于同步校准的RGB-热图像对稀缺，阻碍了多模态任务的发展，ThermalGen旨在通过生成合成热图像解决这一问题。

Contribution: 1. 提出了ThermalGen，首个能处理多样化视角、传感器特性和环境的RGB-T图像转换模型；2. 收集并发布了多个大规模RGB-T数据集。

Method: 采用了基于流的生成模型，结合RGB图像条件架构和风格解耦机制，支持多数据集的训练。

Result: 在多个RGB-T基准测试中表现优异，优于现有的GAN和扩散模型方法。

Insight: ThermalGen的成功表明，基于流的生成模型在多模态图像转换任务中具有潜力，尤其在处理多样化数据时。

Abstract: Paired RGB-thermal data is crucial for visual-thermal sensor fusion and cross-modality tasks, including important applications such as multi-modal image alignment and retrieval. However, the scarcity of synchronized and calibrated RGB-thermal image pairs presents a major obstacle to progress in these areas. To overcome this challenge, RGB-to-Thermal (RGB-T) image translation has emerged as a promising solution, enabling the synthesis of thermal images from abundant RGB datasets for training purposes. In this study, we propose ThermalGen, an adaptive flow-based generative model for RGB-T image translation, incorporating an RGB image conditioning architecture and a style-disentangled mechanism. To support large-scale training, we curated eight public satellite-aerial, aerial, and ground RGB-T paired datasets, and introduced three new large-scale satellite-aerial RGB-T datasets–DJI-day, Bosonplus-day, and Bosonplus-night–captured across diverse times, sensor types, and geographic regions. Extensive evaluations across multiple RGB-T benchmarks demonstrate that ThermalGen achieves comparable or superior translation performance compared to existing GAN-based and diffusion-based methods. To our knowledge, ThermalGen is the first RGB-T image translation model capable of synthesizing thermal images that reflect significant variations in viewpoints, sensor characteristics, and environmental conditions. Project page: http://xjh19971.github.io/ThermalGen

[341] VAGUEGAN: Stealthy Poisoning and Backdoor Attacks on Image Generative Pipelines

Mostafa Mohaimen Akand Faisal,Rabeya Amin Jhuma

Main category: cs.CV

TL;DR: VagueGAN 是一种针对图像生成管道的攻击方法，通过结合 PoisonerNet 和生成器-判别器对，生成隐蔽的触发器，导致生成图像的定向变化。攻击在保留甚至提升视觉质量的同时，暴露了像素级防御的盲点。

Details Motivation: 生成模型（如 GANs 和扩散模型）广泛用于合成逼真图像和支持下游任务。然而，针对生成管道的攻击研究较少，尤其是那些输入微小扰动会导致输出可控变化的隐蔽攻击。

Contribution: 1) 提出了 VagueGAN 攻击框架；2) 展示了潜伏空间中毒可以保留或提升视觉质量；3) 揭示了像素级防御的局限性。

Method: 结合 PoisonerNet（模块化扰动网络）和生成器-判别器对，生成隐蔽触发器。通过感知和频域分析评估隐蔽性，并在 ControlNet 中验证方法的迁移性。

Result: 实验结果显示，中毒输出的视觉质量甚至可能高于干净样本，且扰动在生成器中能保持一致性。

Insight: 潜伏空间中毒可能比像素级扰动更隐蔽且高效，挑战了传统防御假设，引发对生成管道安全性的新思考。

Abstract: Generative models such as GANs and diffusion models are widely used to synthesize photorealistic images and to support downstream creative and editing tasks. While adversarial attacks on discriminative models are well studied, attacks targeting generative pipelines where small, stealthy perturbations in inputs lead to controlled changes in outputs are less explored. This study introduces VagueGAN, an attack pipeline combining a modular perturbation network PoisonerNet with a Generator Discriminator pair to craft stealthy triggers that cause targeted changes in generated images. Attack efficacy is evaluated using a custom proxy metric, while stealth is analyzed through perceptual and frequency domain measures. The transferability of the method to a modern diffusion based pipeline is further examined through ControlNet guided editing. Interestingly, the experiments show that poisoned outputs can display higher visual quality compared to clean counterparts, challenging the assumption that poisoning necessarily reduces fidelity. Unlike conventional pixel level perturbations, latent space poisoning in GANs and diffusion pipelines can retain or even enhance output aesthetics, exposing a blind spot in pixel level defenses. Moreover, carefully optimized perturbations can produce consistent, stealthy effects on generator outputs while remaining visually inconspicuous, raising concerns for the integrity of image generation pipelines.

[342] DAM: Dual Active Learning with Multimodal Foundation Model for Source-Free Domain Adaptation

Xi Chen,Hongxun Yao,Zhaopan Xu,Kui Jiang

Main category: cs.CV

TL;DR: 该论文提出了一种名为DAM的新框架，通过结合多模态视觉-语言（ViL）模型的监督信号与稀疏人工标注，提升了无源域适应（SFADA）的性能。

Details Motivation: 当前的无源域适应方法通常将视觉-语言模型和数据监督视为独立来源，缺乏有效的融合机制，限制了性能提升。

Contribution: 1. 提出DAM框架，整合多模态监督信号与人工标注；2. 采用双向蒸馏机制，促进目标模型与双监督信号之间的知识交换；3. 在多类SFADA基准测试中取得最先进性能。

Method: 1. 利用ViL模型初始化稳定的目标；2. 通过双向蒸馏机制融合稀疏人工标注与ViL监督信号；3. 在迭代适应过程中优化模型性能。

Result: DAM在多个SFADA基准测试中超越了现有方法，并提升了性能表现。

Insight: 多模态监督信号可以有效补充稀疏标注数据，双向蒸馏机制是提升域适应性能的关键。

Abstract: Source-free active domain adaptation (SFADA) enhances knowledge transfer from a source model to an unlabeled target domain using limited manual labels selected via active learning. While recent domain adaptation studies have introduced Vision-and-Language (ViL) models to improve pseudo-label quality or feature alignment, they often treat ViL-based and data supervision as separate sources, lacking effective fusion. To overcome this limitation, we propose Dual Active learning with Multimodal (DAM) foundation model, a novel framework that integrates multimodal supervision from a ViL model to complement sparse human annotations, thereby forming a dual supervisory signal. DAM initializes stable ViL-guided targets and employs a bidirectional distillation mechanism to foster mutual knowledge exchange between the target model and the dual supervisions during iterative adaptation. Extensive experiments demonstrate that DAM consistently outperforms existing methods and sets a new state-of-the-art across multiple SFADA benchmarks and active learning strategies.

[343] Attention Surgery: An Efficient Recipe to Linearize Your Video Diffusion Transformer

Mohsen Ghafoorian,Denis Korzhenkov,Amirhossein Habibian

Main category: cs.CV

TL;DR: 通过引入Attention Surgery框架，无需从头训练即可线性化或混合预训练视频扩散模型中的注意力机制，显著降低计算成本并保持生成质量。

Details Motivation: 当前基于Transformer的视频扩散模型（VDMs）在生成高质量视频方面表现出色，但其自注意力机制的二次计算成本限制了长序列和高分辨率的应用。虽然线性注意力能降低计算复杂度，但先前方法难以在不进行昂贵重新训练的情况下匹配softmax注意力的表达能力。

Contribution: 1. 提出Attention Surgery框架，线性化或混合预训练VDMs中的注意力机制；2. 引入混合注意力机制（结合softmax和线性token）+轻量级蒸馏微调流程；3. 设计成本感知的块速率策略，平衡各层的表达性和效率。

Method: 1. 混合注意力机制：结合softmax和线性token；2. 轻量级蒸馏和微调流程，仅需少量GPU-days；3. 成本感知的块速率策略，动态调整各层的注意力类型。

Result: 在Wan2.1 1.3B（SOTA DiT-based VDM）上实现首个具有竞争力的次二次注意力VDMs，减少40%的FLOPs成本，同时在VBench和VBench-2.0基准上保持生成质量。

Insight: 1. 无需从头训练即可优化预训练模型的效率；2. 混合注意力机制是平衡计算成本和模型性能的有效方法；3. 轻量级适配策略对实际应用具有重要意义。

Abstract: Transformer-based video diffusion models (VDMs) deliver state-of-the-art video generation quality but are constrained by the quadratic cost of self-attention, making long sequences and high resolutions computationally expensive. While linear attention offers sub-quadratic complexity, prior attempts fail to match the expressiveness of softmax attention without costly retraining. We introduce \textit{Attention Surgery}, an efficient framework for \textit{linearizing} or \textit{hybridizing} attention in pretrained VDMs without training from scratch. Inspired by recent advances in language models, our method combines a novel hybrid attention mechanism-mixing softmax and linear tokens-with a lightweight distillation and fine-tuning pipeline requiring only a few GPU-days. Additionally, we incorporate a cost-aware block-rate strategy to balance expressiveness and efficiency across layers. Applied to Wan2.1 1.3B, a state-of-the-art DiT-based VDM, Attention Surgery achieves the first competitive sub-quadratic attention video diffusion models, reducing attention cost by up to 40% in terms of FLOPs, while maintaining generation quality as measured on the standard VBench and VBench-2.0 benchmarks.

[344] OpenGPT-4o-Image: A Comprehensive Dataset for Advanced Image Generation and Editing

Zhihong Chen,Xuehai Bai,Yang Shi,Chaoyou Fu,Huanyu Zhang,Haotian Wang,Xiaoyan Sun,Zhang Zhang,Liang Wang,Yuanxing Zhang,Pengfei Wan,Yi-Fan Zhang

Main category: cs.CV

TL;DR: OpenGPT-4o-Image是一个大规模数据集，通过结合分层任务分类法和自动化数据生成方法构建，提升了图像生成与编辑任务的性能。

Details Motivation: 现有数据集缺乏系统性和挑战性场景，限制了多模态模型的性能。

Contribution: 提出了OpenGPT-4o-Image数据集，覆盖11大领域和51个子任务，自动化生成80k高质量指令-图像对。

Method: 采用分层任务分类法和自动化数据生成流水线（结合结构化资源池和GPT-4o）。

Result: 在多个基准测试中显著提升性能（编辑任务提升18%，生成任务提升13%）。

Insight: 系统化数据构建是提升多模态AI能力的关键。

Abstract: The performance of unified multimodal models for image generation and editing is fundamentally constrained by the quality and comprehensiveness of their training data. While existing datasets have covered basic tasks like style transfer and simple object manipulation, they often lack the systematic structure and challenging scenarios required for real-world applications. To address this bottleneck, we introduce OpenGPT-4o-Image, a large-scale dataset constructed using a novel methodology that combines hierarchical task taxonomy with automated data generation. Our taxonomy not only includes fundamental capabilities such as text rendering and style control but also introduces highly practical yet challenging categories like scientific imagery for chemistry illustrations and complex instruction editing requiring simultaneous execution of multiple operations. Through an automated pipeline leveraging structured resource pools and GPT-4o, we generate 80k high-quality instruction-image pairs with controlled diversity, covering 11 major domains and 51 subtasks. Extensive experiments show that fine-tuning leading models on our dataset achieves significant performance gains across multiple benchmarks, with improvements of up to 18% on editing tasks (UniWorld-V1 on ImgEdit-Bench) and 13% on generation tasks (Harmon on GenEval). Our work demonstrates that systematic data construction is key to advancing multimodal AI capabilities.

[345] Segmentor-Guided Counterfactual Fine-Tuning for Image Synthesis

Tian Xia,Matthew Sinclair,Andreas Schuh,Fabio De Sousa Ribeiro,Raghav Mehta,Rajat Rasal,Esther Puyol-Antón,Samuel Gerber,Kersten Petersen,Michiel Schaap,Ben Glocker

Main category: cs.CV

TL;DR: 本文提出了一种名为Seg-CFT的方法，通过分割器引导的反事实微调生成局部一致且有效的反事实图像，避免了传统方法因依赖外部分类器导致的全局干扰问题。

Details Motivation: 传统方法依赖外部分类器或回归器进行反事实图像生成，导致结构特异性干预时可能引发全局干扰，而像素级标签映射需要繁琐的人工标注。

Contribution: 提出了Seg-CFT方法，利用分割器引导的反事实微调，简化了结构特异性变量的干预流程，同时生成了局部一致的反事实图像。

Method: Seg-CFT结合分割器指导，通过对图像进行微调，确保干预仅限于目标结构，避免了全局干扰。

Result: 在胸部X光片生成和冠状动脉疾病建模中表现出色，生成了真实且一致性强的图像。

Insight: 分割器的引入显著提升了反事实图像生成的质量和局部一致性，为医学图像分析提供了新工具。

Abstract: Counterfactual image generation is a powerful tool for augmenting training data, de-biasing datasets, and modeling disease. Current approaches rely on external classifiers or regressors to increase the effectiveness of subject-level interventions (e.g., changing the patient’s age). For structure-specific interventions (e.g., changing the area of the left lung in a chest radiograph), we show that this is insufficient, and can result in undesirable global effects across the image domain. Previous work used pixel-level label maps as guidance, requiring a user to provide hypothetical segmentations which are tedious and difficult to obtain. We propose Segmentor-guided Counterfactual Fine-Tuning (Seg-CFT), which preserves the simplicity of intervening on scalar-valued, structure-specific variables while producing locally coherent and effective counterfactuals. We demonstrate the capability of generating realistic chest radiographs, and we show promising results for modeling coronary artery disease. Code: https://github.com/biomedia-mira/seg-cft.

[346] Scalable GANs with Transformers

Sangeek Hyun,MinKyu Lee,Jae-Pil Heo

Main category: cs.CV

TL;DR: 本文研究了GAN的可扩展性，通过潜在空间训练和纯Transformer架构实现了高效生成，并解决了GAN扩展中的问题，提出了轻量级中间监督和宽度感知学习率调整方法，最终在ImageNet-256上取得了SOTA效果。

Details Motivation: 生成模型的可扩展性推动了近期进展，但GAN的可扩展性原则尚未充分探索。本文旨在通过潜在空间训练和Transformer架构改进GAN的可扩展性。

Contribution: 1) 提出在紧凑VAE潜在空间中训练GAN；2) 采用纯Transformer架构生成器和判别器；3) 分析GAN扩展中的问题并提出解决方案。

Method: 在VAE潜在空间中训练，使用Transformer架构，并引入轻量级中间监督和宽度感知学习率调整。

Result: GAT-XL/2在ImageNet-256上以40轮训练达到FID 2.96的SOTA性能，训练轮数仅为基线方法的1/6。

Insight: GAN的可扩展性可通过潜在空间训练和Transformer架构实现，但需注意早期层利用不足和优化不稳定问题。

Abstract: Scalability has driven recent advances in generative modeling, yet its principles remain underexplored for adversarial learning. We investigate the scalability of Generative Adversarial Networks (GANs) through two design choices that have proven to be effective in other types of generative models: training in a compact Variational Autoencoder latent space and adopting purely transformer-based generators and discriminators. Training in latent space enables efficient computation while preserving perceptual fidelity, and this efficiency pairs naturally with plain transformers, whose performance scales with computational budget. Building on these choices, we analyze failure modes that emerge when naively scaling GANs. Specifically, we find issues as underutilization of early layers in the generator and optimization instability as the network scales. Accordingly, we provide simple and scale-friendly solutions as lightweight intermediate supervision and width-aware learning-rate adjustment. Our experiments show that GAT, a purely transformer-based and latent-space GANs, can be easily trained reliably across a wide range of capacities (S through XL). Moreover, GAT-XL/2 achieves state-of-the-art single-step, class-conditional generation performance (FID of 2.96) on ImageNet-256 in just 40 epochs, 6x fewer epochs than strong baselines.

[347] Perceive, Reflect and Understand Long Video: Progressive Multi-Granular Clue Exploration with Interactive Agents

Jiahua Li,Kun Wei,Zhe Xu,Zibo Su,Xu Yang,Cheng Deng

Main category: cs.CV

TL;DR: 论文提出CogniGPT框架，通过多粒度感知代理（MGPA）和验证增强反思代理（VERA）的交互循环，实现对长视频的高效可靠理解。实验表明，CogniGPT在多个数据集上表现优异，尤其在EgoSchema上仅用11.2帧即超越现有无训练方法。

Details Motivation: 长视频具有时间复杂性和任务相关信息稀疏的特点，现有基于大语言模型的方法在完整性和高效性上仍存在不足。受人类渐进视觉认知启发，提出一种交互式框架以更高效地捕捉任务关键信息。

Contribution: 1. 提出CogniGPT框架，结合MGPA和VERA的交互循环，实现高效且可靠的长视频理解；2. MGPA模拟人类视觉注意力机制，VERA验证关键线索以减少幻觉；3. 在多个数据集上验证了方法的优越性，尤其是在EgoSchema上表现突出。

Method: 1. MGPA通过多粒度感知捕捉任务相关信息，模仿人类视觉的分散和聚焦注意力；2. VERA验证关键线索并优化后续感知策略；3. 通过交互式过程探索最小化但可靠的任务相关线索集。

Result: 在EgoSchema、Video-MME、NExT-QA和MovieChat数据集上，CogniGPT在准确性和效率上均表现优异。在EgoSchema上，仅用11.2帧即超越现有无训练方法，性能接近Gemini 1.5-Pro。

Insight: 1. 交互式多代理结构能有效解决长视频理解中的信息稀疏问题；2. 人类视觉认知机制的可借鉴性；3. 验证反思对减少幻觉和提高效率的重要性。

Abstract: Long videos, characterized by temporal complexity and sparse task-relevant information, pose significant reasoning challenges for AI systems. Although various Large Language Model (LLM)-based approaches have advanced long video understanding, they still struggle to achieve both completeness and efficiency in capturing task-critical information. Inspired by human progressive visual cognition, we propose CogniGPT, a framework that leverages an interactive loop between Multi-Granular Perception Agent (MGPA) and Verification-Enhanced Reflection Agent (VERA) for efficient and reliable long video understanding. Specifically, MGPA mimics human visual divergent and focused attention to capture task-related information, while VERA verifies perceived key clues to mitigate hallucination and optimize subsequent perception strategies. Through this interactive process, CogniGPT explores a minimal set of informative and reliable task-related clues. Extensive experiments on EgoSchema, Video-MME, NExT-QA, and MovieChat datasets demonstrate CogniGPT’s superiority in both accuracy and efficiency. Notably, on EgoSchema, it surpasses existing training-free methods using only 11.2 frames and achieves performance comparable to Gemini 1.5-Pro.

[348] Social 3D Scene Graphs: Modeling Human Actions and Relations for Interactive Service Robots

Ermanno Bartoli,Dennis Rotondi,Buwei He,Patric Jensfelt,Kai O. Arras,Iolanda Leite

Main category: cs.CV

TL;DR: 本文提出了Social 3D Scene Graphs，一种增强的3D场景图表示，用于捕捉人类及其与环境的关系，并引入了一个新基准，实验表明该方法提升了人类活动预测和关系推理能力。

Details Motivation: 现有3D场景图方法忽视了人类在场景中的角色，且缺乏对人类与环境关系的标注数据，限制了机器人对社会交互的理解能力。

Contribution: 1. 提出Social 3D Scene Graphs，扩展3D场景图以包含人类及其关系；2. 引入包含全面标注关系的合成场景基准；3. 展示了该方法在人类活动预测和关系推理上的优势。

Method: 利用开放词汇框架构建增强的3D场景图，捕捉人类属性、活动及本地与远程关系。

Result: 实验证明该方法显著提升了人类活动预测的准确性，并增强了机器人对人类与环境关系的推理能力。

Insight: 通过结合人类行为建模与环境语义表示，可以更全面地为机器人提供社交智能支持。

Abstract: Understanding how people interact with their surroundings and each other is essential for enabling robots to act in socially compliant and context-aware ways. While 3D Scene Graphs have emerged as a powerful semantic representation for scene understanding, existing approaches largely ignore humans in the scene, also due to the lack of annotated human-environment relationships. Moreover, existing methods typically capture only open-vocabulary relations from single image frames, which limits their ability to model long-range interactions beyond the observed content. We introduce Social 3D Scene Graphs, an augmented 3D Scene Graph representation that captures humans, their attributes, activities and relationships in the environment, both local and remote, using an open-vocabulary framework. Furthermore, we introduce a new benchmark consisting of synthetic environments with comprehensive human-scene relationship annotations and diverse types of queries for evaluating social scene understanding in 3D. The experiments demonstrate that our representation improves human activity prediction and reasoning about human-environment relations, paving the way toward socially intelligent robots.

[349] On-the-Fly Data Augmentation for Brain Tumor Segmentation

Ishika Jain,Siri Willems,Steven Latre,Tom De Schepper

Main category: cs.CV

TL;DR: 该论文提出了一种动态数据增强策略，利用预训练的生成对抗网络（GliGANs）在训练过程中实时插入合成的脑瘤数据，以提高脑瘤分割模型的鲁棒性和泛化能力。

Details Motivation: 脑瘤分割模型需要在治疗前后扫描中表现出鲁棒性，但对高质量标注数据的需求限制了模型的泛化能力。传统数据增强方法存储大量3D数据计算成本高，因此需要一种更高效的方法。

Contribution: 提出了动态数据增强策略（on-the-fly augmentation），结合预训练的GliGANs实时生成合成肿瘤数据，避免了大量存储需求，提升了模型性能。

Method: 在nnU-Net框架基础上，使用预训练的GliGANs动态生成肿瘤数据，并评估了三种模型及它们的集成效果。

Result: 模型在BraTS 2025验证平台上的分割表现优异（如Dice分数为ET: 0.79, WT: 0.88），并赢得了BraTS Lighthouse Challenge 2025 Task 1第一名。

Insight: 动态数据增强策略可以有效解决数据稀缺和存储成本问题，同时提升模型的泛化能力和分割精度。

Abstract: Robust segmentation across both pre-treatment and post-treatment glioma scans can be helpful for consistent tumor monitoring and treatment planning. BraTS 2025 Task 1 addresses this by challenging models to generalize across varying tumor appearances throughout the treatment timeline. However, training such generalized models requires access to diverse, high-quality annotated data, which is often limited. While data augmentation can alleviate this, storing large volumes of augmented 3D data is computationally expensive. To address these challenges, we propose an on-the-fly augmentation strategy that dynamically inserts synthetic tumors using pretrained generative adversarial networks (GliGANs) during training. We evaluate three nnU-Net-based models and their ensembles: (1) a baseline without external augmentation, (2) a regular on-the-fly augmented model, and (3) a model with customized on-the-fly augmentation. Built upon the nnU-Net framework, our pipeline leverages pretrained GliGAN weights and tumor insertion methods from prior challenge-winning solutions. An ensemble of the three models achieves lesion-wise Dice scores of 0.79 (ET), 0.749 (NETC), 0.872 (RC), 0.825 (SNFH), 0.79 (TC), and 0.88 (WT) on the online BraTS 2025 validation platform. This work ranked first in the BraTS Lighthouse Challenge 2025 Task 1- Adult Glioma Segmentation.

[350] Wan-Alpha: High-Quality Text-to-Video Generation with Alpha Channel

Haotian Dong,Wenjing Wang,Chen Li,Di Lin

Main category: cs.CV

TL;DR: 该论文提出了一种名为Wan-Alpha的新框架，用于生成带有Alpha通道的高质量透明视频。通过联合学习RGB和Alpha通道，并结合变分自编码器（VAE）和扩散变换器，显著提升了视觉质量、运动真实性和透明度渲染。

Details Motivation: 现有方法在RGBA视频生成中常忽视视觉质量，限制了实际应用。因此，作者提出了一个能够同时生成高质量RGB和Alpha通道的解决方案。

Contribution: 1. 提出Wan-Alpha框架，联合学习RGB和Alpha通道；2. 设计了一种有效的VAE，将Alpha通道编码到RGB潜在空间；3. 构建了高质量、多样化的RGBA视频数据集。

Method: 结合变分自编码器（VAE）和扩散变换器，利用联合学习的方式生成RGBA视频，并通过高多样性数据集支持训练。

Result: 与现有方法相比，Wan-Alpha在视觉质量、运动真实性和透明度渲染方面表现出色，并能生成半透明物体、发光效果和细粒度细节。

Insight: Alpha通道的有效编码和联合学习对RGBA视频生成至关重要，高质量的数据集是训练扩散模型的关键。

Abstract: RGBA video generation, which includes an alpha channel to represent transparency, is gaining increasing attention across a wide range of applications. However, existing methods often neglect visual quality, limiting their practical usability. In this paper, we propose \textit{Wan-Alpha}, a new framework that generates transparent videos by learning both RGB and alpha channels jointly. We design an effective variational autoencoder (VAE) that encodes the alpha channel into the RGB latent space. Then, to support the training of our diffusion transformer, we construct a high-quality and diverse RGBA video dataset. Compared with state-of-the-art methods, our model demonstrates superior performance in visual quality, motion realism, and transparency rendering. Notably, our model can generate a wide variety of semi-transparent objects, glowing effects, and fine-grained details such as hair strands. The released model is available on our website: \href{https://donghaotian123.github.io/Wan-Alpha/}{https://donghaotian123.github.io/Wan-Alpha/}.

[351] SDPose: Exploiting Diffusion Priors for Out-of-Domain and Robust Pose Estimation

Shuang Liang,Jing He,Chuanmeizhi Wang,Lejun Liao,Guo Zhang,Yingcong Chen,Yuan Yuan

Main category: cs.CV

TL;DR: SDPose利用预训练的扩散模型（如Stable Diffusion）的潜特征进行人体姿态估计，通过轻量级卷积头和辅助RGB重建分支提高跨域鲁棒性，并在COCO-OOD等跨域基准上取得SOTA性能。

Details Motivation: 扩散模型的多尺度潜特征对密集预测任务（如姿态估计）具有潜力，但此前工作主要关注生成任务，其在结构化输出（如姿态估计）中的应用尚待探索。

Contribution: 1. 提出SDPose框架，直接在Stable Diffusion的潜空间中预测关键点热图；2. 通过轻量级卷积头和辅助RGB分支防止过拟合并提升跨域鲁棒性；3. 构建COCO-OOD基准验证方法的鲁棒性。

Method: 1. 利用Stable Diffusion的U-Net潜空间预测关键点热图；2. 采用轻量级卷积头映射潜特征；3. 引入辅助RGB重建分支保留生成语义。

Result: 在COCO验证集上与Sapiens-1B/2B性能相当，在HumanArt和COCO-OOD上达到SOTA，并展示了其在零样本标注和控制生成任务中的潜力。

Insight: 扩散模型的潜特征可用于结构化输出任务，轻量级设计和辅助任务能有效提升跨域性能。

Abstract: Pre-trained diffusion models provide rich multi-scale latent features and are emerging as powerful vision backbones. While recent works such as Marigold\citep{ke2024repurposing} and Lotus\citep{he2024lotus} adapt diffusion priors for dense prediction with strong cross-domain generalization, their potential for structured outputs (e.g., human pose estimation) remains underexplored. In this paper, we propose \textbf{SDPose}, a fine-tuning framework built upon Stable Diffusion to fully exploit pre-trained diffusion priors for human pose estimation. First, rather than modifying cross-attention modules or introducing learnable embeddings, we directly predict keypoint heatmaps in the SD U-Net’s image latent space to preserve the original generative priors. Second, we map these latent features into keypoint heatmaps through a lightweight convolutional pose head, which avoids disrupting the pre-trained backbone. Finally, to prevent overfitting and enhance out-of-distribution robustness, we incorporate an auxiliary RGB reconstruction branch that preserves domain-transferable generative semantics. To evaluate robustness under domain shift, we further construct \textbf{COCO-OOD}, a style-transferred variant of COCO with preserved annotations. With just one-fifth of the training schedule used by Sapiens on COCO, SDPose attains parity with Sapiens-1B/2B on the COCO validation set and establishes a new state of the art on the cross-domain benchmarks HumanArt and COCO-OOD. Furthermore, we showcase SDPose as a zero-shot pose annotator for downstream controllable generation tasks, including ControlNet-based image synthesis and video generation, where it delivers qualitatively superior pose guidance.

[352] PanoWorld-X: Generating Explorable Panoramic Worlds via Sphere-Aware Video Diffusion

Yuyang Yin,HaoXiang Guo,Fangfu Liu,Mengyu Wang,Hanwen Liang,Eric Li,Yikai Wang,Xiaojie Jin,Yao Zhao,Yunchao Wei

Main category: cs.CV

TL;DR: PanoWorld-X是一个用于生成高质量、可控全景视频的新框架，通过球面感知扩散Transformer和大规模数据集解决了现有方法的视场限制和相机控制不足问题。

Details Motivation: 现有方法在生成全景世界时面临视场狭窄或相机控制不足的问题，限制了场景的连续性和自由探索性。

Contribution: 提出了PanoWorld-X框架，结合球面感知扩散Transformer和大规模全景视频数据集，提升了全景视频的生成质量和可控性。

Method: 1. 通过Unreal Engine构建大规模全景视频-探索路径数据集；2. 引入球面感知扩散Transformer，重新投影特征到球面以实现几何一致性。

Result: PanoWorld-X在运动范围、控制精度和视觉质量方面表现优异，展示了实际应用潜力。

Insight: 球面几何建模对全景数据生成至关重要，传统视频扩散的归纳偏置可能与球面数据不对齐，需专门设计架构。

Abstract: Generating a complete and explorable 360-degree visual world enables a wide range of downstream applications. While prior works have advanced the field, they remain constrained by either narrow field-of-view limitations, which hinder the synthesis of continuous and holistic scenes, or insufficient camera controllability that restricts free exploration by users or autonomous agents. To address this, we propose PanoWorld-X, a novel framework for high-fidelity and controllable panoramic video generation with diverse camera trajectories. Specifically, we first construct a large-scale dataset of panoramic video-exploration route pairs by simulating camera trajectories in virtual 3D environments via Unreal Engine. As the spherical geometry of panoramic data misaligns with the inductive priors from conventional video diffusion, we then introduce a Sphere-Aware Diffusion Transformer architecture that reprojects equirectangular features onto the spherical surface to model geometric adjacency in latent space, significantly enhancing visual fidelity and spatiotemporal continuity. Extensive experiments demonstrate that our PanoWorld-X achieves superior performance in various aspects, including motion range, control precision, and visual quality, underscoring its potential for real-world applications.

[353] LVT: Large-Scale Scene Reconstruction via Local View Transformers

Tooba Imtiaz,Lucy Chai,Kathryn Heal,Xuan Luo,Jungyeon Park,Jennifer Dy,John Flynn

Main category: cs.CV

TL;DR: LVT提出了一种基于局部视图Transformer的大规模场景重建方法，避免了传统Transformer的二次复杂度问题，通过局部邻域处理和几何位置编码实现高效的大场景重建和新视角合成。

Details Motivation: 传统Transformer在3D视觉和新视角合成中存在二次复杂度问题，难以扩展到大规模场景，因此需要一种高效的替代方法。

Contribution: 提出了Local View Transformer（LVT），一种通过局部邻域处理和高斯Splat表示实现的大规模场景重建方法，避免了二次复杂度问题。

Method: 利用局部邻域处理和几何变换位置编码，将视图信息转换为3D高斯Splat表示，支持颜色和不透明度的视角依赖性。

Result: LVT支持单次前向传播完成任意大规模高分辨率场景的重建，效果显著。

Insight: 局部邻域信息比全局信息更能有效捕捉场景结构，几何位置编码在提升模型性能中发挥了关键作用。

Abstract: Large transformer models are proving to be a powerful tool for 3D vision and novel view synthesis. However, the standard Transformer’s well-known quadratic complexity makes it difficult to scale these methods to large scenes. To address this challenge, we propose the Local View Transformer (LVT), a large-scale scene reconstruction and novel view synthesis architecture that circumvents the need for the quadratic attention operation. Motivated by the insight that spatially nearby views provide more useful signal about the local scene composition than distant views, our model processes all information in a local neighborhood around each view. To attend to tokens in nearby views, we leverage a novel positional encoding that conditions on the relative geometric transformation between the query and nearby views. We decode the output of our model into a 3D Gaussian Splat scene representation that includes both color and opacity view-dependence. Taken together, the Local View Transformer enables reconstruction of arbitrarily large, high-resolution scenes in a single forward pass. See our project page for results and interactive demos https://toobaimt.github.io/lvt/.

[354] GeoVLM-R1: Reinforcement Fine-Tuning for Improved Remote Sensing Reasoning

Mustansar Fiaz,Hiyam Debary,Paolo Fraccaro,Danda Paudel,Luc Van Gool,Fahad Khan,Salman Khan

Main category: cs.CV

TL;DR: GeoVLM-R1提出了一种基于强化学习的后训练框架，通过任务感知奖励提升遥感图像推理能力，并在多个EO基准测试中表现优异。

Details Motivation: 现有强化学习方法在自然图像领域表现优异，但在遥感图像（EO）任务中应用不足。EO任务的多样性和复杂性需要任务感知的推理能力。

Contribution: 提出了一个新颖的后训练框架，结合任务感知奖励，提升遥感图像的推理能力、优化稳定性和鲁棒性。

Method: 采用强化学习进行微调，设计任务感知奖励机制，适配多样化的EO任务。

Result: 在多个EO基准测试中，性能超越现有通用和专用视觉语言模型。

Insight: 任务感知奖励机制是提升遥感图像推理能力的关键，强化学习可有效适配复杂EO任务。

Abstract: Recent advances in reinforcement learning (RL) have delivered strong reasoning capabilities in natural image domains, yet their potential for Earth Observation (EO) remains largely unexplored. EO tasks introduce unique challenges, spanning referred object detection, image or region captioning, change detection, grounding, and temporal analysis, that demand task aware reasoning. We propose a novel post training framework that incorporates task aware rewards to enable effective adaptation of reasoning based RL models to diverse EO tasks. This training strategy enhances reasoning capabilities for remote sensing images, stabilizes optimization, and improves robustness. Extensive experiments across multiple EO benchmarks show consistent performance gains over state of the art generic and specialized vision language models. Code and models will be released publicly at https://mustansarfiaz.github.io/GeoVLM-R1/ .

[355] STAGE: Stable and Generalizable GRPO for Autoregressive Image Generation

Xiaoxiao Ma,Haibo Qiu,Guohui Zhang,Zhixiong Zeng,Siqi Yang,Lin Ma,Feng Zhao

Main category: cs.CV

TL;DR: 该论文提出了STAGE，一种稳定且通用的框架，用于解决在自回归图像生成中应用GRPO时的不稳定性和泛化性问题。通过两种关键方法（优势/KL重加权和熵奖励），STAGE提升了生成质量、训练稳定性及跨任务泛化能力。

Details Motivation: 现有的GRPO算法在自回归图像生成中存在训练不稳定和泛化性差的问题，容易破坏预训练模型能力。为了解决这些挑战，论文提出了STAGE框架。

Contribution: 1. 识别了自回归图像生成中GRPO的两大问题（矛盾梯度和不稳定的策略熵动态）；2. 提出了STAGE框架，包含优势/KL重加权和熵奖励两种方法；3. 实验证明STAGE在生成质量、训练稳定性和跨任务泛化性上优于基线。

Method: 1. 优势/KL重加权：通过相似性感知的重加权减少矛盾梯度；2. 熵奖励：引入基于熵的奖励以稳定训练过程。这两种方法共同减轻了对预训练分布的破坏和奖励攻击。

Result: 在多基准测试中，STAGE显著提升了视觉质量、训练稳定性及跨任务泛化能力，优于基线GRPO。

Insight: 1. 处理矛盾梯度和策略熵动态是改进GRPO的关键；2. 熵奖励是一种有效的稳定训练工具；3. STAGE的方法可以推广到其他需要稳定训练的强化学习场景。

Abstract: Reinforcement learning has recently been explored to improve text-to-image generation, yet applying existing GRPO algorithms to autoregressive (AR) image models remains challenging. The instability of the training process easily disrupts the pretrained model capability during long runs, resulting in marginal gains, degraded image quality, and poor generalization. In this work, we revisit GRPO for AR image generation and identify two key issues: contradictory gradients from unnecessary tokens and unstable policy entropy dynamics. To address these, we introduce STAGE, a stable and generalizable framework that leverages two targeted solutions: 1) Advantage/KL reweighting. Similarity-aware reweighting to alleviate conflicting updates; and 2) Entropy reward. An entropy-based reward corresponding to reference model to stabilize learning. With the help of alleviating conflicts between tokens and an entropy reward for stabilizing training, we reduce disruption of the pretrained distribution and mitigate reward hacking, which in turn improves generalization and transfer better to other benchmarks. Experiments across multiple benchmarks show that STAGE consistently improves visual quality, stability, and cross-task generalization compared to baseline GRPO.

[356] VT-FSL: Bridging Vision and Text with LLMs for Few-Shot Learning

Wenhao Li,Qiangchang Wang,Xianjing Meng,Zhibin Wu,Yilong Yin

Main category: cs.CV

TL;DR: VT-FSL是一种新颖的少样本学习框架，通过结合大型语言模型（LLMs）和视觉支持图像生成精确的跨模态提示，解决了现有方法中语义幻觉的问题，并在多个基准测试中取得了最先进的性能。

Details Motivation: 现有的少样本学习方法虽然通过引入额外语义信息或设计复杂语义融合模块来增强支持特征，但仍存在语义幻觉的问题，导致噪声指导和昂贵的修正成本。VT-FSL旨在通过结合视觉和文本信息，克服这些问题。

Contribution: 1. 提出了一种新颖的跨模态提示生成方法（CIP），利用LLMs和支持图像生成精确的类别描述。2. 设计了跨模态几何对齐（CGA），通过非线性关系对齐多模态表示。3. 在多个少样本学习基准上取得了最先进的性能。

Method: VT-FSL主要由两部分组成：跨模态迭代提示（CIP）和跨模态几何对齐（CGA）。CIP通过LLMs生成精确的类别描述，并合成语义一致的图像作为补充提示；CGA通过最小化3维平行多面体的核化体积对齐多模态表示。

Result: VT-FSL在标准、跨域和细粒度少样本学习场景的十个基准测试中均取得了最先进的性能。

Insight: 结合LLMs和视觉支持图像生成跨模态提示可以显著减少语义噪声，提升少样本学习的性能；几何对齐方法能够有效捕捉多模态表示之间的全局非线性关系。

Abstract: Few-shot learning (FSL) aims to recognize novel concepts from only a few labeled support samples. Recent studies enhance support features by incorporating additional semantic information or designing complex semantic fusion modules. However, they still suffer from hallucinating semantics that contradict the visual evidence due to the lack of grounding in actual instances, resulting in noisy guidance and costly corrections. To address these issues, we propose a novel framework, bridging Vision and Text with LLMs for Few-Shot Learning (VT-FSL), which constructs precise cross-modal prompts conditioned on Large Language Models (LLMs) and support images, seamlessly integrating them through a geometry-aware alignment. It mainly consists of Cross-modal Iterative Prompting (CIP) and Cross-modal Geometric Alignment (CGA). Specifically, the CIP conditions an LLM on both class names and support images to generate precise class descriptions iteratively in a single structured reasoning pass. These descriptions not only enrich the semantic understanding of novel classes but also enable the zero-shot synthesis of semantically consistent images. The descriptions and synthetic images act respectively as complementary textual and visual prompts, providing high-level class semantics and low-level intra-class diversity to compensate for limited support data. Furthermore, the CGA jointly aligns the fused textual, support, and synthetic visual representations by minimizing the kernelized volume of the 3-dimensional parallelotope they span. It captures global and nonlinear relationships among all representations, enabling structured and consistent multimodal integration. The proposed VT-FSL method establishes new state-of-the-art performance across ten diverse benchmarks, including standard, cross-domain, and fine-grained few-shot learning scenarios. Code is available at https://github.com/peacelwh/VT-FSL.

[357] A Scalable Distributed Framework for Multimodal GigaVoxel Image Registration

Rohit Jena,Vedant Zope,Pratik Chaudhari,James C. Gee

Main category: cs.CV

TL;DR: FFDP是一种分布式框架，通过优化非GEMM瓶颈和卷积感知的张量分片，实现了前所未有规模的多模态图像配准，显著提升了性能和效率。

Details Motivation: 生物医学和生命科学中的图像配准算法未能与图像采集能力的提升同步扩展，需要一种高效的分布式框架来处理大规模问题。

Contribution: FFDP框架通过非GEMM融合内核和分布式架构，实现了前所未有的多模态图像配准规模，提升了性能和内存效率。

Method: FFDP采用IO感知的非GEMM融合内核和卷积感知的张量分片技术，结合分布式框架，优化了大规模图像配准的计算瓶颈。

Result: 在8个A6000 GPU上，FFDP能够在约1分钟内完成570倍于标准临床数据规模的多模态配准，加速现有方法6-7倍，并减少峰值内存消耗20-59%。

Insight: FFDP展示了通过分布式优化非GEMM瓶颈和内存管理，可以显著扩展图像配准的规模，同时保持高效性，适用于未来的大规模生物医学数据处理。

Abstract: In this work, we propose FFDP, a set of IO-aware non-GEMM fused kernels supplemented with a distributed framework for image registration at unprecedented scales. Image registration is an inverse problem fundamental to biomedical and life sciences, but algorithms have not scaled in tandem with image acquisition capabilities. Our framework complements existing model parallelism techniques proposed for large-scale transformer training by optimizing non-GEMM bottlenecks and enabling convolution-aware tensor sharding. We demonstrate unprecedented capabilities by performing multimodal registration of a 100 micron ex-vivo human brain MRI volume at native resolution - an inverse problem more than 570x larger than a standard clinical datum in about a minute using only 8 A6000 GPUs. FFDP accelerates existing state-of-the-art optimization and deep learning registration pipelines by upto 6 - 7x while reducing peak memory consumption by 20 - 59%. Comparative analysis on a 250 micron dataset shows that FFDP can fit upto 64x larger problems than existing SOTA on a single GPU, and highlights both the performance and efficiency gains of FFDP compared to SOTA image registration methods.

[358] BRIDGE – Building Reinforcement-Learning Depth-to-Image Data Generation Engine for Monocular Depth Estimation

Dingning Liu,Haoyu Guo,Jingyi Zhou,Tong He

Main category: cs.CV

TL;DR: BRIDGE提出了一种基于强化学习的深度到图像（D2I）生成框架，大规模生成了高质量的RGB图像及其对应的深度数据，用于单目深度估计（MDE）任务的训练。

Details Motivation: 传统单目深度估计方法受限于数据稀缺和质量问题，影响了模型的鲁棒性。BRIDGE旨在通过生成大规模、多样化的合成数据来弥补这一不足。

Contribution: BRIDGE的主要贡献是提出了一种RL优化的深度到图像生成框架，生成了20M+的高质量RGB图像及其深度数据，并设计了混合监督策略以提升训练效果。

Method: 方法包括：（1）利用RL优化的D2I框架生成合成数据；（2）采用混合监督策略，结合教师伪标签和真实深度数据进行训练。

Result: BRIDGE在规模和领域多样性上取得了突破，在定量评估和复杂场景细节捕捉上超越了现有SOTA方法。

Insight: 通过强化学习优化数据生成过程，可以有效提升合成数据的质量，进而改善单目深度估计模型的泛化能力和鲁棒性。

Abstract: Monocular Depth Estimation (MDE) is a foundational task for computer vision. Traditional methods are limited by data scarcity and quality, hindering their robustness. To overcome this, we propose BRIDGE, an RL-optimized depth-to-image (D2I) generation framework that synthesizes over 20M realistic and geometrically accurate RGB images, each intrinsically paired with its ground truth depth, from diverse source depth maps. Then we train our depth estimation model on this dataset, employing a hybrid supervision strategy that integrates teacher pseudo-labels with ground truth depth for comprehensive and robust training. This innovative data generation and training paradigm enables BRIDGE to achieve breakthroughs in scale and domain diversity, consistently outperforming existing state-of-the-art approaches quantitatively and in complex scene detail capture, thereby fostering general and robust depth features. Code and models are available at https://dingning-liu.github.io/bridge.github.io/.

[359] MANI-Pure: Magnitude-Adaptive Noise Injection for Adversarial Purification

Xiaoyi Huang,Junwei Wu,Kejia Zhang,Carl Yang,Zhiming Luo

Main category: cs.CV

TL;DR: MANI-Pure提出了一种基于幅值自适应噪声注入的对抗性净化框架，通过有针对性地在不同频率区域注入噪声，有效抑制高频率、低幅值区域的对抗扰动，同时保留重要语义内容。

Details Motivation: 现有的对抗性净化方法通常采用均匀噪声注入，这会不加区分地扰动所有频率，破坏语义结构并削弱鲁棒性。本文通过实证研究发现对抗扰动并非均匀分布，而是主要集中在高频率区域，且幅度分布因攻击类型而异。

Contribution: MANI-Pure的主要贡献是提出了一种幅值自适应净化框架，利用输入的幅值谱指导净化过程，有针对性地注入噪声，显著提升鲁棒性。

Method: MANI-Pure的核心方法是自适应应用异质、频率定向的噪声注入，重点抑制高频率、低幅值区域的对抗扰动，同时保留低频关键语义内容。

Result: 在CIFAR-10和ImageNet-1K上的实验表明，MANI-Pure将原始分类器的干净精度差距缩小至0.59以内，鲁棒精度提升2.15，并在RobustBench排行榜上取得最高鲁棒精度，超越此前最优方法。

Insight: 对抗扰动主要集中在高频率、低幅值区域，因此通过针对性噪声注入可以有效净化对抗样本，而不破坏低频语义结构。

Abstract: Adversarial purification with diffusion models has emerged as a promising defense strategy, but existing methods typically rely on uniform noise injection, which indiscriminately perturbs all frequencies, corrupting semantic structures and undermining robustness. Our empirical study reveals that adversarial perturbations are not uniformly distributed: they are predominantly concentrated in high-frequency regions, with heterogeneous magnitude intensity patterns that vary across frequencies and attack types. Motivated by this observation, we introduce MANI-Pure, a magnitude-adaptive purification framework that leverages the magnitude spectrum of inputs to guide the purification process. Instead of injecting homogeneous noise, MANI-Pure adaptively applies heterogeneous, frequency-targeted noise, effectively suppressing adversarial perturbations in fragile high-frequency, low-magnitude bands while preserving semantically critical low-frequency content. Extensive experiments on CIFAR-10 and ImageNet-1K validate the effectiveness of MANI-Pure. It narrows the clean accuracy gap to within 0.59 of the original classifier, while boosting robust accuracy by 2.15, and achieves the top-1 robust accuracy on the RobustBench leaderboard, surpassing the previous state-of-the-art method.

[360] Score Distillation of Flow Matching Models

Mingyuan Zhou,Yi Gu,Huangjie Zheng,Liangchen Song,Guande He,Yizhe Zhang,Wenze Hu,Yinfei Yang

Main category: cs.CV

TL;DR: 该论文提出了一个基于贝叶斯规则和条件期望的统一框架，将高斯扩散和流匹配联系起来，并成功将分数蒸馏（SiD）技术扩展到文本到图像的流匹配模型中，展示了其广泛的适用性和稳定性。

Details Motivation: 扩散模型在高质量图像生成方面表现优异，但其迭代采样速度较慢。蒸馏方法可以缓解这一问题，但流匹配模型与扩散模型的等价性引发了分数蒸馏技术是否可以直接迁移的问题。

Contribution: 论文的主要贡献是提出了一个简单的统一框架，将高斯扩散和流匹配联系起来，并将分数蒸馏技术扩展到多种预训练的文本到图像流匹配模型中，无需微调或架构修改。

Method: 通过贝叶斯规则和条件期望的统一视角，扩展分数蒸馏（SiD）技术，并将其应用于以DiT为骨干的多种流匹配模型（如SANA、SD3-Medium等）。

Result: 实验表明，SiD技术能够直接应用于这些流匹配模型，且在数据无关和数据辅助的设置中均表现稳定，无需额外的微调或架构调整。

Insight: 论文证明了分数蒸馏技术可以广泛适用于文本到图像的流匹配模型，消除了之前对稳定性和合理性的担忧，并统一了扩散模型和流匹配模型的加速技术。

Abstract: Diffusion models achieve high-quality image generation but are limited by slow iterative sampling. Distillation methods alleviate this by enabling one- or few-step generation. Flow matching, originally introduced as a distinct framework, has since been shown to be theoretically equivalent to diffusion under Gaussian assumptions, raising the question of whether distillation techniques such as score distillation transfer directly. We provide a simple derivation – based on Bayes’ rule and conditional expectations – that unifies Gaussian diffusion and flow matching without relying on ODE/SDE formulations. Building on this view, we extend Score identity Distillation (SiD) to pretrained text-to-image flow-matching models, including SANA, SD3-Medium, SD3.5-Medium/Large, and FLUX.1-dev, all with DiT backbones. Experiments show that, with only modest flow-matching- and DiT-specific adjustments, SiD works out of the box across these models, in both data-free and data-aided settings, without requiring teacher finetuning or architectural changes. This provides the first systematic evidence that score distillation applies broadly to text-to-image flow matching models, resolving prior concerns about stability and soundness and unifying acceleration techniques across diffusion- and flow-based generators. We will make the PyTorch implementation publicly available.

[361] Fast Feature Field ($\text{F}^3$): A Predictive Representation of Events

Richeek Das,Kostas Daniilidis,Pratik Chaudhari

Main category: cs.CV

TL;DR: 该论文提出了一种称为快速特征场（F3）的表示方法，用于处理事件相机的数据。F3通过学习预测未来事件来保留场景结构和运动信息，具有高效性和鲁棒性。

Details Motivation: 事件相机数据具有稀疏性、噪声多样性和事件速率变化的特点，传统方法难以高效处理。研究者希望通过一种新的表示方法，解决这些问题并支持多种下游任务。

Contribution: 提出快速特征场（F3），一种高效、鲁棒的表示方法，能够预测未来事件并保留关键信息。F3支持多种任务，如光流估计、语义分割和深度估计。

Method: 结合多分辨率哈希编码和深度集合（deep sets）技术，F3将事件数据表示为多通道图像，并通过预测未来事件优化表示。

Result: 在多项任务中（光流估计、语义分割、深度估计）取得最优性能，支持多种分辨率和事件速率，计算效率高（120 Hz HD，440 Hz VGA）。

Insight: F3的成功表明，通过预测未来事件学习表示可以显著提升事件相机数据的处理效率和下游任务的性能。

Abstract: This paper develops a mathematical argument and algorithms for building representations of data from event-based cameras, that we call Fast Feature Field ($\text{F}^3$). We learn this representation by predicting future events from past events and show that it preserves scene structure and motion information. $\text{F}^3$ exploits the sparsity of event data and is robust to noise and variations in event rates. It can be computed efficiently using ideas from multi-resolution hash encoding and deep sets - achieving 120 Hz at HD and 440 Hz at VGA resolutions. $\text{F}^3$ represents events within a contiguous spatiotemporal volume as a multi-channel image, enabling a range of downstream tasks. We obtain state-of-the-art performance on optical flow estimation, semantic segmentation, and monocular metric depth estimation, on data from three robotic platforms (a car, a quadruped robot and a flying platform), across different lighting conditions (daytime, nighttime), environments (indoors, outdoors, urban, as well as off-road) and dynamic vision sensors (resolutions and event rates). Our implementations can predict these tasks at 25-75 Hz at HD resolution.

[362] VideoAnchor: Reinforcing Subspace-Structured Visual Cues for Coherent Visual-Spatial Reasoning

Zhaozhi Wang,Tong Zhang,Mingyue Guo,Yaowei Wang,Qixiang Ye

Main category: cs.CV

TL;DR: VideoAnchor通过在Transformer注意力机制中引入稀疏子空间聚类的自表达性，强化跨帧视觉线索，显著提升了多模态大语言模型（MLLMs）在视觉空间推理任务中的表现。

Details Motivation: 当前的多模态大语言模型在视觉语言对齐方面表现优秀，但在视觉空间推理任务中存在不足。研究发现这是由于注意力机制中视觉标记被语言标记掩盖，导致模型难以跨帧识别同一视觉线索。

Contribution: 提出了VideoAnchor模块，利用子空间相似性在无需重新训练的情况下强化跨帧视觉线索，显著提升了模型在视觉空间推理任务中的性能。

Method: 将稀疏子空间聚类的自表达性与Transformer注意力机制结合，设计了一个即插即用的VideoAnchor模块，通过子空间亲和力锚定共享视觉结构。

Result: 在多个基准测试和主干模型上取得了显著性能提升，如VSI-Bench和Video-MME任务中分别提高3.2%和4.6%。

Insight: 揭示了注意力机制中视觉线索被忽视的问题，并通过子空间聚类的方法提出了一种高效且无需重新训练的解决方案。

Abstract: Multimodal Large Language Models (MLLMs) have achieved impressive progress in vision-language alignment, yet they remain limited in visual-spatial reasoning. We first identify that this limitation arises from the attention mechanism: visual tokens are overshadowed by language tokens, preventing the model from consistently recognizing the same visual cues across frames. To address this challenge, we draw a novel connection between the self-expressiveness property in sparse subspace clustering and the attention mechanism in Transformers. Building on this insight, we propose VideoAnchor, a plug-and-play module that leverages subspace affinities to reinforce visual cues across frames without retraining, effectively anchoring attention to shared visual structures. Extensive experiments across benchmarks and backbone models show consistent performance gains – $e.g.$, 3.2% and 4.6% improvements on VSI-Bench and Video-MME (spatial-related tasks) with InternVL2-8B and Qwen2.5VL-72B – while qualitative analyses demonstrate more coherent subspace partitions and stronger visual grounding. Our codes will be made public available at https://github.com/feufhd/VideoAnchor.

[363] Rolling Forcing: Autoregressive Long Video Diffusion in Real Time

Kunhao Liu,Wenbo Hu,Jiale Xu,Ying Shan,Shijian Lu

Main category: cs.CV

TL;DR: Rolling Forcing是一种实时生成长视频的新技术，通过联合去噪、注意力汇机制和高效训练算法，显著减少误差累积。

Details Motivation: 现有视频流生成技术在长时序中容易累积误差，导致生成质量下降。Rolling Forcing旨在解决这一问题，实现高质量、低延迟的长视频流生成。

Contribution: 1. 联合去噪方案；2. 注意力汇机制；3. 高效训练算法。

Method: 1. 同时去噪多帧，缓解严格因果关系；2. 使用注意力汇机制保持全局一致性；3. 非重叠窗口训练减少暴露偏差。

Result: 实验表明，Rolling Forcing能在单GPU上实时生成长达数分钟的视频，显著减少误差累积。

Insight: 联合去噪和注意力汇机制是提升长视频生成一致性的关键，而高效训练算法是实际应用的重要保障。

Abstract: Streaming video generation, as one fundamental component in interactive world models and neural game engines, aims to generate high-quality, low-latency, and temporally coherent long video streams. However, most existing work suffers from severe error accumulation that often significantly degrades the generated stream videos over long horizons. We design Rolling Forcing, a novel video generation technique that enables streaming long videos with minimal error accumulation. Rolling Forcing comes with three novel designs. First, instead of iteratively sampling individual frames, which accelerates error propagation, we design a joint denoising scheme that simultaneously denoises multiple frames with progressively increasing noise levels. This design relaxes the strict causality across adjacent frames, effectively suppressing error growth. Second, we introduce the attention sink mechanism into the long-horizon stream video generation task, which allows the model to keep key value states of initial frames as a global context anchor and thereby enhances long-term global consistency. Third, we design an efficient training algorithm that enables few-step distillation over largely extended denoising windows. This algorithm operates on non-overlapping windows and mitigates exposure bias conditioned on self-generated histories. Extensive experiments show that Rolling Forcing enables real-time streaming generation of multi-minute videos on a single GPU, with substantially reduced error accumulation.

[364] YOLO26: Key Architectural Enhancements and Performance Benchmarking for Real-Time Object Detection

Ranjan Sapkota,Rahul Harsha Cheppally,Ajay Sharda,Manoj Karkee

Main category: cs.CV

TL;DR: YOLO26是YOLO系列的最新成员，专注于边缘设备的实时目标检测，通过架构创新（如无NMS推理、STAL标签分配等）提升了效率和精度。

Details Motivation: 为解决边缘设备上实时目标检测的效率和精度问题，YOLO26引入了多项技术创新。

Contribution: 1. 提出无NMS的端到端推理；2. 移除DFL简化导出；3. 引入ProgLoss和STAL提升小目标检测；4. 采用MuSGD优化器。

Method: 结合无NMS推理、改进的损失函数（ProgLoss）、STAL标签分配和MuSGD优化器，优化模型性能。

Result: 在NVIDIA Orin Jetson平台上，YOLO26在效率和精度上优于YOLOv8、YOLO11等版本。

Insight: YOLO26通过简化流程和优化组件，展示了边缘设备上实时目标检测的潜力。

Abstract: This study presents Key Architectural Enhancements and Performance Benchmarking of Ultralytics YOLO26 for real-time edge object detection, providing a comprehensive overview of the design principles of YOLO26, technological advances, and deployment readiness. YOLO26, released in September 2025 by Ultralytics, represents the newest and most cutting-edge member of the You Only Look Once (YOLO) family, engineered to push the boundaries of efficiency and accuracy on edge and low-power devices. This paper highlights architectural innovations in YOLO26, including end-to-end NMS-free inference, removal of Distribution Focal Loss (DFL) for streamlined exports, introduction of ProgLoss and Small-Target-Aware Label Assignment (STAL) for improved stability and small-object detection, and the adoption of the MuSGD optimizer inspired by large language model training. In addition, we report performance benchmarks for YOLO26 across edge devices, specifically NVIDIA Orin Jetson platforms, and compare results against YOLOv8 and YOLO11 (previous Ultralytics releases) as well as YOLOv12 and YOLOv13, which bridged the lineage between YOLO11 and YOLO26. Our comparative analysis highlights superior efficiency of YOLO26, accuracy, and deployment versatility, establishing it as a pivotal milestone in the YOLO evolution.

[365] Personalized Vision via Visual In-Context Learning

Yuxin Jiang,Yuchao Gu,Yiren Song,Ivor Tsang,Mike Zheng Shou

Main category: cs.CV

TL;DR: 论文提出了一种名为PICO的视觉上下文学习框架，通过扩散变换器实现个性化视觉任务，无需重新训练即可适应新的用户定义任务。

Details Motivation: 现有视觉模型在大型标注数据集上训练表现良好，但无法灵活适应个性化任务（如用户自定义对象或目标）。传统方法依赖高成本微调或合成数据，缺乏灵活性。

Contribution: 提出了PICO框架，利用扩散变换器实现视觉上下文学习；构建了VisRel数据集，强调任务多样性而非规模；设计了注意力引导的种子评分器提升推理可靠性。

Method: PICO通过四面板框架和扩散变换器实现视觉上下文学习，仅需单个标注样本即可推断变换并应用于新输入。VisRel数据集和种子评分器是关键支撑。

Result: PICO在实验中超越微调和合成数据基线，灵活适应新任务，并在识别和生成任务中表现优异。

Insight: 任务多样性比数据集规模更能驱动模型的泛化能力，视觉上下文学习为个性化视觉任务提供了高效解决方案。

Abstract: Modern vision models, trained on large-scale annotated datasets, excel at predefined tasks but struggle with personalized vision – tasks defined at test time by users with customized objects or novel objectives. Existing personalization approaches rely on costly fine-tuning or synthetic data pipelines, which are inflexible and restricted to fixed task formats. Visual in-context learning (ICL) offers a promising alternative, yet prior methods confine to narrow, in-domain tasks and fail to generalize to open-ended personalization. We introduce Personalized In-Context Operator (PICO), a simple four-panel framework that repurposes diffusion transformers as visual in-context learners. Given a single annotated exemplar, PICO infers the underlying transformation and applies it to new inputs without retraining. To enable this, we construct VisRel, a compact yet diverse tuning dataset, showing that task diversity, rather than scale, drives robust generalization. We further propose an attention-guided seed scorer that improves reliability via efficient inference scaling. Extensive experiments demonstrate that PICO (i) surpasses fine-tuning and synthetic-data baselines, (ii) flexibly adapts to novel user-defined tasks, and (iii) generalizes across both recognition and generation.

[366] Mitigating Hallucination in Multimodal LLMs with Layer Contrastive Decoding

Bingkui Tong,Jiaer Xia,Kaiyang Zhou

Main category: cs.CV

TL;DR: 论文提出了一种名为Layer Contrastive Decoding（LayerCD）的方法，旨在减少多模态大语言模型（MLLMs）中的幻觉现象，通过对比视觉编码器中不同层次的特征输出分布来过滤不一致的生成内容。

Details Motivation: 多模态大语言模型虽然在感知和推理上表现优异，但常常产生与输入图像上下文不符的幻觉输出，包括对象、属性和关系的不准确描述。

Contribution: 主要的贡献是提出了LayerCD方法，通过对比浅层和深层视觉特征的输出分布，有效减少幻觉现象。

Method: 方法的核心是对比视觉编码器中浅层和深层特征生成的输出分布，浅层特征容易导致幻觉，而深层特征更具高层次的推理能力。

Result: 在多个幻觉基准测试中，LayerCD显著优于当前最先进的方法。

Insight: 研究表明，浅层视觉特征容易引发幻觉，而深层特征更适合用于高层次的推理任务。

Abstract: Multimodal Large Language Models (MLLMs) have shown impressive perception and reasoning capabilities, yet they often suffer from hallucinations – generating outputs that are linguistically coherent but inconsistent with the context of the input image, including inaccuracies in objects, attributes, and relations. To address this challenge, we propose a simple approach called Layer Contrastive Decoding (LayerCD). Our design is motivated by the observation that shallow visual features are much more likely than deep visual features to cause an MLLM to hallucinate as they only capture biased, low-level information that is insufficient for high-level reasoning. Therefore, LayerCD aims to filter out hallucinations by contrasting the output distributions generated from visual features of different levels, specifically those from the shallow and deep layers of the vision encoder, respectively. We conduct extensive experiments on two hallucination benchmarks and show that LayerCD significantly outperforms current state-of-the-art. The code for LayerCD is available at https://github.com/maifoundations/LayerCD .

[367] GHOST: Hallucination-Inducing Image Generation for Multimodal LLMs

Aryan Yazdan Parast,Parsa Hosseini,Hesam Asadollahzadeh,Arshia Soltani Moakhar,Basim Azam,Soheil Feizi,Naveed Akhtar

Main category: cs.CV

TL;DR: GHOST是一种通过生成诱导幻觉的图像来压力测试多模态大语言模型（MLLMs）的自动方法，成功率高且无需人工干预。

Details Motivation: MLLMs存在物体幻觉问题（感知到图像中不存在的物体），但现有研究依赖于静态基准测试，无法全面揭示模型的幻觉漏洞。

Contribution: 提出了GHOST方法，通过优化图像嵌入空间和利用扩散模型生成高质量且无目标物体的图像，成功诱导MLLMs产生幻觉。

Method: GHOST在图像嵌入空间中优化误导模型的特征，同时保持目标物体缺失，然后利用扩散模型生成看起来自然的图像。

Result: 在多个模型（如GLM-4.1V-Thinking）上实现了超过28%的幻觉成功率，远超之前1%的数据驱动方法。

Insight: GHOST不仅揭示了MLLMs的幻觉漏洞，还可用于微调以提高模型可靠性，展示了诊断和纠正的双重价值。

Abstract: Object hallucination in Multimodal Large Language Models (MLLMs) is a persistent failure mode that causes the model to perceive objects absent in the image. This weakness of MLLMs is currently studied using static benchmarks with fixed visual scenarios, which preempts the possibility of uncovering model-specific or unanticipated hallucination vulnerabilities. We introduce GHOST (Generating Hallucinations via Optimizing Stealth Tokens), a method designed to stress-test MLLMs by actively generating images that induce hallucination. GHOST is fully automatic and requires no human supervision or prior knowledge. It operates by optimizing in the image embedding space to mislead the model while keeping the target object absent, and then guiding a diffusion model conditioned on the embedding to generate natural-looking images. The resulting images remain visually natural and close to the original input, yet introduce subtle misleading cues that cause the model to hallucinate. We evaluate our method across a range of models, including reasoning models like GLM-4.1V-Thinking, and achieve a hallucination success rate exceeding 28%, compared to around 1% in prior data-driven discovery methods. We confirm that the generated images are both high-quality and object-free through quantitative metrics and human evaluation. Also, GHOST uncovers transferable vulnerabilities: images optimized for Qwen2.5-VL induce hallucinations in GPT-4o at a 66.5% rate. Finally, we show that fine-tuning on our images mitigates hallucination, positioning GHOST as both a diagnostic and corrective tool for building more reliable multimodal systems.

[368] DC-VideoGen: Efficient Video Generation with Deep Compression Video Autoencoder

Junyu Chen,Wenkun He,Yuchao Gu,Yuyang Zhao,Jincheng Yu,Junsong Chen,Dongyun Zou,Yujun Lin,Zhekai Zhang,Muyang Li,Haocheng Xi,Ligeng Zhu,Enze Xie,Song Han,Han Cai

Main category: cs.CV

TL;DR: DC-VideoGen是一个高效的视频生成加速框架，通过深度压缩视频自动编码器和轻量级微调来提升任何预训练视频扩散模型的效率，显著降低推理延迟。

Details Motivation: 现有的视频生成模型在效率和计算资源上存在瓶颈，尤其是在高分辨率视频生成时。DC-VideoGen旨在通过深度压缩和高效适配策略解决这一问题。

Contribution: 1) 提出了一种新颖的深度压缩视频自动编码器（DC-VAE），支持高压缩比的同时保持重建质量；2) 设计了AE-Adapt-V策略，实现预训练模型到新潜在空间的快速稳定迁移。

Method: 1) DC-VAE采用分块因果时间设计，实现32x/64x空间和4x时间压缩；2) AE-Adapt-V通过轻量级微调，仅需10 GPU天即可完成适配。

Result: 将Wan-2.1-14B模型适配DC-VideoGen后，推理延迟降低14.8倍，且无需牺牲质量；单GPU支持2160x3840分辨率视频生成。

Insight: 通过深度压缩和高效适配策略，DC-VideoGen在降低计算成本的同时保持了生成质量，为高分辨率视频生成的实用化提供了可能。

Abstract: We introduce DC-VideoGen, a post-training acceleration framework for efficient video generation. DC-VideoGen can be applied to any pre-trained video diffusion model, improving efficiency by adapting it to a deep compression latent space with lightweight fine-tuning. The framework builds on two key innovations: (i) a Deep Compression Video Autoencoder with a novel chunk-causal temporal design that achieves 32x/64x spatial and 4x temporal compression while preserving reconstruction quality and generalization to longer videos; and (ii) AE-Adapt-V, a robust adaptation strategy that enables rapid and stable transfer of pre-trained models into the new latent space. Adapting the pre-trained Wan-2.1-14B model with DC-VideoGen requires only 10 GPU days on the NVIDIA H100 GPU. The accelerated models achieve up to 14.8x lower inference latency than their base counterparts without compromising quality, and further enable 2160x3840 video generation on a single GPU. Code: https://github.com/dc-ai-projects/DC-VideoGen.

[369] PAD3R: Pose-Aware Dynamic 3D Reconstruction from Casual Videos

Ting-Hsuan Liao,Haowen Liu,Yiran Xu,Songwei Ge,Gengshan Yang,Jia-Bin Huang

Main category: cs.CV

TL;DR: PAD3R提出了一种从随意拍摄的单目视频中重建可变形3D对象的方法，通过结合生成先验和可微分渲染，能够处理长视频序列中的大尺度变形和相机运动。

Details Motivation: 现有方法在处理长视频中的大尺度变形、相机运动和有限视角覆盖时表现不佳，亟需一种更鲁棒且通用的解决方案。

Contribution: PAD3R的核心贡献是提出了一种个性化的对象中心位姿估计器，结合了生成先验和可微分渲染，从而实现了高保真的类别无关3D重建。

Method: 方法包括训练一个由预训练图像到3D模型监督的位姿估计器，并优化可变形3D高斯表示，同时利用长期2D点跟踪进行正则化。

Result: 实验表明PAD3R在多种挑战性场景下均表现出色，具有优异的鲁棒性和泛化能力。

Insight: PAD3R展示了生成先验与可微分渲染结合在动态场景理解和3D内容创作中的潜力，为解决复杂变形问题提供了新思路。

Abstract: We present PAD3R, a method for reconstructing deformable 3D objects from casually captured, unposed monocular videos. Unlike existing approaches, PAD3R handles long video sequences featuring substantial object deformation, large-scale camera movement, and limited view coverage that typically challenge conventional systems. At its core, our approach trains a personalized, object-centric pose estimator, supervised by a pre-trained image-to-3D model. This guides the optimization of deformable 3D Gaussian representation. The optimization is further regularized by long-term 2D point tracking over the entire input video. By combining generative priors and differentiable rendering, PAD3R reconstructs high-fidelity, articulated 3D representations of objects in a category-agnostic way. Extensive qualitative and quantitative results show that PAD3R is robust and generalizes well across challenging scenarios, highlighting its potential for dynamic scene understanding and 3D content creation.

[370] PixelCraft: A Multi-Agent System for High-Fidelity Visual Reasoning on Structured Images

Shuoshuo Zhang,Zijian Li,Yizhen Zhang,Jingjing Fu,Lei Song,Jiang Bian,Jun Zhang,Yujiu Yang,Rui Wang

Main category: cs.CV

TL;DR: PixelCraft提出了一个多智能体系统，用于结构化图像的高保真处理和灵活视觉推理，通过动态三阶段工作流程解决了现有方法的局限。

Details Motivation: 现有方法在处理结构化图像时存在低保真图像处理和线性推理模式的限制，影响了视觉推理的有效性。

Contribution: 提出了PixelCraft系统，结合了高保真图像处理和动态视觉推理能力，通过多智能体协作显著提升了复杂任务的性能。

Method: 系统由多个模块组成，包括调度器、规划器、推理器、批评器和视觉工具智能体，采用三阶段工作流程（工具选择、智能体讨论、自我批评）进行动态推理。

Result: 在图表和几何基准测试中表现优异，显著提升了MLLM的视觉推理能力。

Insight: 通过多智能体协作和高保真图像处理，可以显著提升结构化图像的视觉推理性能。

Abstract: Structured images (e.g., charts and geometric diagrams) remain challenging for multimodal large language models (MLLMs), as perceptual slips can cascade into erroneous conclusions. Intermediate visual cues can steer reasoning; however, existing cue-based methods are constrained with low-fidelity image processing and linear, rigid reasoning patterns, limiting their effectiveness on complex structured-image tasks. In this paper, we propose PixelCraft, a novel multi-agent system for high-fidelity image processing and flexible visual reasoning on structured images. The system comprises a dispatcher, a planner, a reasoner, critics, and a set of visual tool agents. To achieve high-fidelity processing, we construct a high-quality corpus and fine-tune an MLLM into a grounding model, whose pixel-level localizations are integrated with traditional computer vision (CV) algorithms in tool agents. Building on this foundation, PixelCraft facilitates flexible visual reasoning through a dynamic three-stage workflow of tool selection, agent discussion, and self-criticism. Moreover, unlike prior linear reasoning patterns that simply append historical images, PixelCraft maintains an image memory to allow the planner to adaptively revisit earlier visual steps, explore alternative reasoning branches, and dynamically adjust the reasoning trajectory during discussion. Extensive experiments on challenging chart and geometry benchmarks demonstrate that PixelCraft significantly improves visual reasoning performance for advanced MLLMs, setting a new standard for structured image reasoning. Our code will be available at https://github.com/microsoft/PixelCraft.

[371] FlashI2V: Fourier-Guided Latent Shifting Prevents Conditional Image Leakage in Image-to-Video Generation

Yunyang Ge,Xinhua Cheng,Chengshu Zhao,Xianyi He,Shenghai Yuan,Bin Lin,Bin Zhu,Li Yuan

Main category: cs.CV

TL;DR: FlashI2V通过傅里叶引导的潜在偏移防止图像到视频生成中的条件图像泄漏问题，提升了跨域数据的性能。

Details Motivation: 现有的图像到视频（I2V）方法在生成视频时容易因条件图像泄漏导致性能下降，如运动缓慢和颜色不一致问题。

Contribution: 提出FlashI2V方法，结合潜在偏移和傅里叶引导，有效防止条件图像泄漏并提升跨域性能。

Method: 1. 潜在偏移：通过移除噪声潜在中的条件图像信息，隐式结合条件；2. 傅里叶引导：利用高频特征加速收敛并调整生成视频的细节。

Result: FlashI2V在Vbench-I2V上动态度得分53.01，超越CogVideoX1.5-5B-I2V和Wan2.1-I2V-14B-480P。

Insight: 防止条件图像泄漏可显著提升I2V模型的跨域性能，且较小的模型也能实现更好的效果。

Abstract: In Image-to-Video (I2V) generation, a video is created using an input image as the first-frame condition. Existing I2V methods concatenate the full information of the conditional image with noisy latents to achieve high fidelity. However, the denoisers in these methods tend to shortcut the conditional image, which is known as conditional image leakage, leading to performance degradation issues such as slow motion and color inconsistency. In this work, we further clarify that conditional image leakage leads to overfitting to in-domain data and decreases the performance in out-of-domain scenarios. Moreover, we introduce Fourier-Guided Latent Shifting I2V, named FlashI2V, to prevent conditional image leakage. Concretely, FlashI2V consists of: (1) Latent Shifting. We modify the source and target distributions of flow matching by subtracting the conditional image information from the noisy latents, thereby incorporating the condition implicitly. (2) Fourier Guidance. We use high-frequency magnitude features obtained by the Fourier Transform to accelerate convergence and enable the adjustment of detail levels in the generated video. Experimental results show that our method effectively overcomes conditional image leakage and achieves the best generalization and performance on out-of-domain data among various I2V paradigms. With only 1.3B parameters, FlashI2V achieves a dynamic degree score of 53.01 on Vbench-I2V, surpassing CogVideoX1.5-5B-I2V and Wan2.1-I2V-14B-480P. Github page: https://pku-yuangroup.github.io/FlashI2V/

[372] Visual Jigsaw Post-Training Improves MLLMs

Penghao Wu,Yushan Zhang,Haiwen Diao,Bo Li,Lewei Lu,Ziwei Liu

Main category: cs.CV

TL;DR: 该论文提出了Visual Jigsaw，一种自监督的后训练框架，旨在通过视觉拼图任务增强多模态大语言模型（MLLMs）的视觉理解能力。

Details Motivation: 当前MLLMs的后训练范式主要以文本为中心，视觉输入仅用于提取稀疏线索。然而，视觉理解是MLLMs的核心能力，需专注于视觉而非依赖文本中介。

Contribution: 提出了Visual Jigsaw，一种无需额外视觉生成组件的自监督后训练框架，通过视觉排序任务增强MLLMs的视觉理解能力。

Method: 将视觉输入分割、打乱，要求模型通过自然语言生成正确排列顺序。该方法自然地与可验证奖励的强化学习（RLVR）对齐，无需标注。

Result: 在图像、视频和3D数据等多种视觉模态上实验表明，显著提升了细粒度感知、时序推理和3D空间理解能力。

Insight: 证明了视觉为中心的自监督任务在后训练MLLMs中的潜力，为未来视觉导向的设计提供了新思路。

Abstract: Reinforcement learning based post-training has recently emerged as a powerful paradigm for enhancing the alignment and reasoning capabilities of multimodal large language models (MLLMs). While vision-centric post-training is crucial for enhancing MLLMs’ intrinsic understanding of visual signals, current post-training paradigms are predominantly text-centric, where dense visual inputs are only leveraged to extract sparse cues for text-based reasoning. There exist a few approaches in this direction, however, they often still rely on text as an intermediate mediator or introduce additional visual generative designs. In this work, we introduce Visual Jigsaw, a generic self-supervised post-training framework designed to strengthen visual understanding in MLLMs. Visual Jigsaw is formulated as a general ordering task: visual inputs are partitioned, shuffled, and the model must reconstruct the visual information by producing the correct permutation in natural language. This naturally aligns with reinforcement learning from verifiable rewards (RLVR), requires no additional visual generative components, and derives its supervisory signal automatically without any annotations. We instantiate Visual Jigsaw across three visual modalities, including images, videos, and 3D data. Extensive experiments demonstrate substantial improvements in fine-grained perception, temporal reasoning, and 3D spatial understanding. Our findings highlight the potential of self-supervised vision-centric tasks in post-training MLLMs and aim to inspire further research on vision-centric pretext designs. Project Page: https://penghao-wu.github.io/visual_jigsaw/

cs.HC [Back]

[373] Bridging the behavior-neural gap: A multimodal AI reveals the brain’s geometry of emotion more accurately than human self-reports

Changde Du,Yizhuo Lu,Zhongyu Huang,Yi Sun,Zisen Zhou,Shaozheng Qin,Huiguang He

Main category: cs.HC

TL;DR: 该论文提出了一种多模态AI方法，通过大规模相似性判断捕捉情绪的神经几何，优于人类自我报告的数据。

Details Motivation: 情绪表征对人类认知和社会互动至关重要，但传统评分量表无法准确预测大脑活动，存在'行为-神经鸿沟'。

Contribution: 文章的主要贡献在于展示了多模态大语言模型（MLLM）能够生成更贴近神经活动的情感表征，并超越了人类行为评分和单语言模型（LLM）。

Method: 使用AI模型（MLLM和LLM）对2180个情感视频进行数百万次三元组’不同类’判断，生成30维嵌入表征，并与神经活动对齐。

Result: MLLM的情感表征在预测人类情绪处理网络的神经活动时精度最高，且优于LLM和人类行为评分。

Insight: 研究表明，多模态学习（尤其是视觉数据）对构建神经对齐的情感框架至关重要，AI模型能自主发展出丰富且神经对齐的情感表征。

Abstract: The ability to represent emotion plays a significant role in human cognition and social interaction, yet the high-dimensional geometry of this affective space and its neural underpinnings remain debated. A key challenge, the behavior-neural gap,' is the limited ability of human self-reports to predict brain activity. Here we test the hypothesis that this gap arises from the constraints of traditional rating scales and that large-scale similarity judgments can more faithfully capture the brain's affective geometry. Using AI models as cognitive agents,’ we collected millions of triplet odd-one-out judgments from a multimodal large language model (MLLM) and a language-only model (LLM) in response to 2,180 emotionally evocative videos. We found that the emergent 30-dimensional embeddings from these models are highly interpretable and organize emotion primarily along categorical lines, yet in a blended fashion that incorporates dimensional properties. Most remarkably, the MLLM’s representation predicted neural activity in human emotion-processing networks with the highest accuracy, outperforming not only the LLM but also, counterintuitively, representations derived directly from human behavioral ratings. This result supports our primary hypothesis and suggests that sensory grounding–learning from rich visual data–is critical for developing a truly neurally-aligned conceptual framework for emotion. Our findings provide compelling evidence that MLLMs can autonomously develop rich, neurally-aligned affective representations, offering a powerful paradigm to bridge the gap between subjective experience and its neural substrates. Project page: https://reedonepeck.github.io/ai-emotion.github.io/.

cs.LG [Back]

[374] CAOTE: KV Cache Selection for LLMs via Attention Output Error-Based Token Eviction

Raghavv Goel,Junyoung Park,Mukul Gagrani,Dalton Jones,Matthew Morse,Harper Langston,Mingu Lee,Chris Lott

Main category: cs.LG

TL;DR: 提出了一种基于注意力输出误差的KV缓存令牌逐出方法CAOTE，结合注意力分数和值向量优化逐出误差，显著提升了下游任务准确性。

Details Motivation: 长上下文支持的LLM面临内存和计算瓶颈，现有基于注意力分数的令牌逐出方法忽略了令牌对注意力输出的贡献信息。

Contribution: 首次提出利用值向量信息的令牌逐出标准CAOTE，以封闭形式结合注意力分数和值向量，优化逐出误差。

Method: CAOTE通过计算令牌对注意力输出的贡献作为逐出标准，并与现有基于注意力的逐出方法灵活结合。

Result: 实验表明，CAOTE与最先进的基于注意力分数的方法结合时，能持续提升下游任务的准确性。

Insight: 令牌逐出过程中利用值向量信息至关重要，CAOTE为优化KV缓存提供了新的启发式方法。

Abstract: While long context support of large language models has extended their abilities, it also incurs challenges in memory and compute which becomes crucial bottlenecks in resource-restricted devices. Token eviction, a widely adopted post-training methodology designed to alleviate the bottlenecks by evicting less important tokens from the cache, typically uses attention scores as proxy metrics for token importance. However, one major limitation of attention score as a token-wise importance metrics is that it lacks the information about contribution of tokens to the attention output. In this paper, we propose a simple eviction criterion based on the contribution of cached tokens to attention outputs. Our method, CAOTE, optimizes for eviction error due to token eviction, by seamlessly integrating attention scores and value vectors. This is the first method which uses value tokens on top of attention-based eviction scores in closed-form. Additionally, CAOTE can act as a meta-heuristic method with flexible usage with any token eviction method. We show that CAOTE, when combined with the state-of-the-art attention score-based methods, always improves accuracies on the downstream task, indicating the importance of leveraging information from values during token eviction process.

[375] Adaptive Margin RLHF via Preference over Preferences

Yaswanth Chittepu,Prasann Singhal,Greg Durrett,Scott Niekum

Main category: cs.LG

TL;DR: 该论文提出了一种基于偏好强度的自适应边际优化方法DPO-PoP，通过偏好对偏好的标注来推断动态边际，从而提升RLHF中的泛化性和生成质量。

Details Motivation: 现有RLHF方法在奖励模型学习中通常使用固定或简单的边际，无法反映偏好强度的差异性，且依赖噪声较大的评分信息。

Contribution: 提出了DPO-PoP方法，利用偏好对偏好的标注动态调整边际，显著提升了分类和生成性能。

Method: 扩展了直接偏好优化(DPO)，通过偏好对偏好的标注信号推断每个数据点的动态边际，并提出两种采样策略平衡分类和生成性能。

Result: 在UltraFeedback数据集上，DPO-PoP优于原始DPO、固定边际DPO及真实边际DPO。

Insight: 偏好强度建模能提升泛化性；分类和生成性能存在权衡，需根据目标选择合适的采样策略。

Abstract: Margin-based optimization is fundamental to improving generalization and robustness in classification tasks. In the context of reward model learning from preferences within Reinforcement Learning from Human Feedback (RLHF), existing methods typically rely on no margins, fixed margins, or margins that are simplistic functions of preference ratings. However, such formulations often fail to account for the varying strengths of different preferences, for example some preferences are associated with larger margins between responses, or they rely on noisy margin information derived from ratings. We argue that modeling the strength of preferences can lead to better generalization and more faithful alignment. Furthermore, many existing methods that use adaptive margins assume access to accurate preference scores, which can be difficult for humans to provide reliably. We propose an approach that leverages preferences over preferences, that is annotations indicating which of two preferences reflects a stronger distinction. We use this ordinal signal to infer adaptive margins on a per-datapoint basis. We introduce an extension to Direct Preference Optimization (DPO), DPO-PoP, that incorporates adaptive margins from preference-over-preference supervision, enabling improved discriminative and generative performance. Empirically, our method outperforms vanilla DPO, DPO with fixed margins, and DPO with ground-truth margins on the UltraFeedback dataset. Additionally, we show that there is a tradeoff between discriminative and generative performance: improving test classification accuracy, particularly by correctly labeling weaker preferences at the expense of stronger ones, can lead to a decline in generative quality. To navigate this tradeoff, we propose two sampling strategies to gather preference-over-preference labels: one favoring discriminative performance and one favoring generative performance.

[376] Causally-Enhanced Reinforcement Policy Optimization

Xiangqi Wang,Yue Huang,Yujun Zhou,Xiaonan Luo,Kehan Guo,Xiangliang Zhang

Main category: cs.LG

TL;DR: 论文提出了CE-PO（因果增强策略优化），一种通过估计因果一致性的可微代理来增强强化学习策略优化的方法，以减少LLM（大语言模型）中的奖励利用和不忠实推理。

Details Motivation: 现有的LLM在强化目标训练中常通过捷径策略生成表面正确的答案，但其推理过程可能依赖虚假或不忠实的原因，且在因果扰动下表现不佳。

Contribution: 引入CE-PO框架，通过Jacobian灵敏度估计模型内部影响，并结合因果一致的奖励信号与任务准确性反馈，以提升模型的鲁棒性和推理忠实性。

Method: 利用Jacobian基础灵敏度估计模型内部影响，采用反事实信号增强因果一致性，并通过Minkowski组合器将一致性分数与任务准确性反馈融合，适用于PPO/GRPO优化框架。

Result: 在4个数据集上，CE-PO平均提升准确性5.49%（最高9.58%），并显著提升了对因果相关性翻转和轻微反事实编辑的鲁棒性。

Insight: 通过显式建模因果一致性并融合到奖励设计中，可以有效减少LLM的不忠实推理行为，提升其在复杂任务中的鲁棒性。

Abstract: Large language models (LLMs) trained with reinforcement objectives often achieve superficially correct answers via shortcut strategies, pairing correct outputs with spurious or unfaithful reasoning and degrading under small causal perturbations. We introduce Causally-Enhanced Policy Optimization (CE-PO), a drop-in reward-shaping framework that augments policy optimization with a differentiable proxy for causal coherence along the generation pathway from prompt (Z) to rationale (X) to answer (Y). CE-PO estimates model-internal influence with Jacobian-based sensitivities, counterfactually hardens these signals to suppress nuisance cues, and fuses the resulting coherence score with task-accuracy feedback via a Minkowski (power-mean) combiner, exposing a single tunable between accuracy and coherence trade-off. The unified reward integrates with PPO/GRPO without architectural changes. Across reasoning benchmarks and causal stress tests, CE-PO reduces reward hacking and unfaithful chain-of-thought while improving robustness to correlation-causation flips and light counterfactual edits, all at near-parity accuracy. Experimental results across 4 datasets show that CE-PO improves accuracy over baselines by 5.49% on average (up to 9.58%), while improving robustness to correlation-causation flips and light counterfactual edits.

[377] RHYTHM: Reasoning with Hierarchical Temporal Tokenization for Human Mobility

Haoyu He,Haozheng Luo,Yan Chen,Qi R. Wang

Main category: cs.LG

TL;DR: RHYTHM利用分层时间标记化和预训练LLM的冻结主干，显著提升了人类移动性预测的性能和效率。

Details Motivation: 人类移动性预测中的长距离依赖和多尺度周期性行为增加了建模的复杂性。

Contribution: 提出了RHYTHM框架，结合LLM的分层注意力机制和预计算提示嵌入，优化了轨迹编码和预测。

Method: 采用时间标记化分割轨迹为每日段，并通过分层注意力和预计算嵌入提升LLM的表现。

Result: 在三个真实数据集上，RHYTHM提升了2.4%的整体精度，周末预测提升5.0%，训练时间减少24.6%。

Insight: 冻结LLM主干和分层注意力设计有效地平衡了计算成本和模型性能。

Abstract: Predicting human mobility is inherently challenging due to complex long-range dependencies and multi-scale periodic behaviors. To address this, we introduce RHYTHM (Reasoning with Hierarchical Temporal Tokenization for Human Mobility), a unified framework that leverages large language models (LLMs) as general-purpose spatio-temporal predictors and trajectory reasoners. Methodologically, RHYTHM employs temporal tokenization to partition each trajectory into daily segments and encode them as discrete tokens with hierarchical attention that captures both daily and weekly dependencies, thereby significantly reducing the sequence length while preserving cyclical information. Additionally, we enrich token representations by adding pre-computed prompt embeddings for trajectory segments and prediction targets via a frozen LLM, and feeding these combined embeddings back into the LLM backbone to capture complex interdependencies. Computationally, RHYTHM freezes the pretrained LLM’s backbone to reduce attention complexity and memory cost. We evaluate our model against state-of-the-art methods using three real-world datasets. Notably, RHYTHM achieves a 2.4% improvement in overall accuracy, a 5.0% increase on weekends, and a 24.6% reduction in training time. Code is publicly available at https://github.com/he-h/rhythm.

[378] C$^2$GSPG: Confidence-calibrated Group Sequence Policy Gradient towards Self-aware Reasoning

Haotian Liu,Shuo Wang,Hongteng Xu

Main category: cs.LG

TL;DR: 论文提出了一种名为C²GSPG的置信度校准分组序列策略梯度方法，旨在解决强化学习中的过度自信问题，提升模型的推理能力和自我认知能力。

Details Motivation: 现有的强化学习方法（如GRPO及其变体）在推理任务中存在过度自信的问题，限制了模型的自我认知能力。因此，提出一种既能提升推理性能又能抑制过度自信的方法是必要的。

Contribution: 1. 提出了GSPG框架，消除了GRPO及其变体中常见的token级别偏差；
2. 定义了序列级别的置信度计算方法，并通过交叉熵正则化进行校准；
3. 针对非二元奖励，设计了非线性归一化和自适应正则化裁剪策略。

Method: 1. 提出GSPG框架，利用序列级别的概率计算模型置信度；
2. 使用交叉熵正则化校准置信度与奖励的关系；
3. 对非二元奖励进行非线性归一化和自适应正则化裁剪，避免目标冲突。

Result: 在逻辑和数学推理任务中，C²GSPG在推理准确性和置信度校准方面均优于现有方法。

Insight: 置信校准正则化与GSPG框架协同工作，在二元奖励任务中完全一致，非二元任务中通过技术手段缓解冲突，证明了方法的有效性和普适性。

Abstract: Reinforcement Learning (RL) methods, exemplified by Group Relative Policy Optimization (GRPO) and its variants, play a central role in developing reasoning models. However, these methods often suffer from a critical overconfidence issue, which prevents them from achieving self-aware reasoning models. In this study, we propose a simple yet effective confidence-calibration group sequence policy gradient method, called C$^2$GSPG, which simultaneously enhances reasoning performance while suppressing overconfidence. In principle, we propose a Group Sequence Policy Gradient (GSPG) framework for learning reasoning models, which eliminates the token-level bias commonly appearing in GRPO and its variants. In this framework, we define the model confidence for each reasoning problem using the normalized sequence-level probability, and then apply a cross-entropy regularizer to calibrate the model confidence to the sequence’s reward. We demonstrate that the confidence calibration regularizer and GSPG are collaborative for binary rewards, as their objectives always share the same gradient direction. For non-binary rewards, we apply nonlinear reward normalization and adaptive regularizer clipping, mitigating the potential conflict between the two objectives. Applying C$^2$GSPG to post-train large language models in logical and mathematical reasoning tasks, we show its superiority over state-of-the-art methods in both reasoning accuracy and confidence calibration. The code of C$^2$GSPG is available at https://github.com/HaotianLiu123/CCGSPG.

[379] SPEC-RL: Accelerating On-Policy Reinforcement Learning via Speculative Rollouts

Bingshuai Liu,Ante Wang,Zijun Min,Liang Yao,Haibo Zhang,Yang Liu,Anxiang Zeng,Jinsong Su

Main category: cs.LG

TL;DR: SPEC-RL提出了一种通过推测性解码优化强化学习训练中计算密集型rollout阶段的方法，显著减少计算时间，同时保持策略质量。

Details Motivation: 现有的强化学习训练方法在rollout阶段存在计算冗余问题，连续训练周期的轨迹片段经常重叠，浪费计算资源。

Contribution: SPEC-RL将推测性解码引入RL rollouts，重用历史轨迹片段并通过草案-验证机制扩展，减少冗余计算。

Method: 该方法利用历史轨迹作为推测前缀，结合草案-验证机制生成新轨迹，提升计算效率。

Result: 实验表明，SPEC-RL在多个数学推理和泛化基准上将rollout时间减少2-3倍，且不影响策略质量。

Insight: 推测性解码不仅能用于语言模型生成，也可优化强化学习的rollout阶段，为RLVR的可扩展性提供新思路。

Abstract: Large Language Models (LLMs) increasingly rely on reinforcement learning with verifiable rewards (RLVR) to elicit reliable chain-of-thought reasoning. However, the training process remains bottlenecked by the computationally expensive rollout stage. Existing acceleration methods-such as parallelization, objective- and data-driven modifications, and replay buffers-either incur diminishing returns, introduce bias, or overlook redundancy across iterations. We identify that rollouts from consecutive training epochs frequently share a large portion of overlapping segments, wasting computation. To address this, we propose SPEC-RL, a novel framework that integrates SPECulative decoding with the RL rollout process. SPEC-RL reuses prior trajectory segments as speculative prefixes and extends them via a draft-and-verify mechanism, avoiding redundant generation while ensuring policy consistency. Experiments on diverse math reasoning and generalization benchmarks, including GSM8K, MATH-500, OlympiadBench, MMLU-STEM, and others, demonstrate that SPEC-RL reduces rollout time by 2-3x without compromising policy quality. As a purely rollout-stage enhancement, SPEC-RL integrates seamlessly with mainstream algorithms (e.g., PPO, GRPO, DAPO), offering a general and practical path to scale RLVR for large reasoning models. Our code is available at https://github.com/ShopeeLLM/Spec-RL

[380] Temporal Generalization: A Reality Check

Divyam Madaan,Sumit Chopra,Kyunghyun Cho

Main category: cs.LG

TL;DR: 该论文探讨了机器学习模型在仅依赖过去数据时能否实现时间泛化性，并通过参数插值和外推等方法在多个任务上进行了实验，结果显示简单基线方法（使用最新模型参数）在所有场景中表现最优，凸显了时间泛化的困难性。

Details Motivation: 机器学习模型在分布偏移时性能下降，论文旨在研究仅依赖过去数据能否实现时间泛化性，以验证模型对未来数据的预测能力。

Contribution: 论文的主要贡献是通过实验比较了参数插值和外推等方法，验证了简单基线方法的有效性，并指出时间泛化的固有挑战。

Method: 论文采用了参数插值（凸组合过去模型参数）和参数外推（超越凸包的外推）两种主要方法，并在多个时间序列任务上进行了评测。

Result: 实验结果表明，所有评估方法均未在所有任务中超越简单基线方法（使用最新模型参数），强调了时间泛化的困难性。

Insight: 研究揭示了在没有未来数据或强假设时，时间泛化具有固有挑战性，提醒研究人员对泛化性声明需持谨慎态度。

Abstract: Machine learning (ML) models often struggle to maintain performance under distribution shifts, leading to inaccurate predictions on unseen future data. In this work, we investigate whether and under what conditions models can achieve such a generalization when relying solely on past data. We explore two primary approaches: convex combinations of past model parameters (\emph{parameter interpolation}) and explicit extrapolation beyond the convex hull of past parameters (\emph{parameter extrapolation}). We benchmark several methods within these categories on a diverse set of temporal tasks, including language modeling, news summarization, news tag prediction, academic paper categorization, satellite image-based land use classification over time, and historical yearbook photo gender prediction. Our empirical findings show that none of the evaluated methods consistently outperforms the simple baseline of using the latest available model parameters in all scenarios. In the absence of access to future data or robust assumptions about the underlying data-generating process, these results underscore the inherent difficulties of generalizing and extrapolating to future data and warrant caution when evaluating claims of such generalization.

[381] Anchored Supervised Fine-Tuning

He Zhu,Junyou Su,Peng Lai,Ren Ma,Wenjia Zhang,Linyi Yang,Guanhua Chen

Main category: cs.LG

TL;DR: 论文提出了一种名为锚定监督微调（ASFT）的方法，通过在动态微调（DFT）中引入轻量级的KL正则化，解决了DFT缺乏分布锚定导致的训练不稳定问题，显著提升了在数学推理、医学知识基础和代码生成等任务上的表现。

Details Motivation: 传统的监督微调（SFT）容易过拟合，而强化学习（RL）虽然泛化能力强但计算成本高。动态微调（DFT）作为中间方法虽然在某些领域表现优异，但存在不稳定问题。作者希望通过理论分析改进DFT，提出更稳定的方法。

Contribution: 1. 通过奖励加权回归（RWR）框架分析了DFT，揭示了其缺乏分布锚定的问题；2. 提出ASFT方法，引入KL正则化解决训练不稳定问题；3. 在多个任务上验证ASFT的优越性，计算开销低。

Method: ASFT在DFT的基础上增加了轻量级的KL正则化，保持DFT的优势同时解决其不稳定问题。RWR框架为分析提供了理论基础。

Result: ASFT在数学推理、医学知识基础和代码生成任务中均优于SFT和DFT，计算成本低且稳定性高。

Insight: 理论分析可以帮助设计更高效且稳定的微调方法，KL正则化是一种简单有效的稳定化手段。

Abstract: Post-training of large language models involves a fundamental trade-off between supervised fine-tuning (SFT), which efficiently mimics demonstrations but tends to memorize, and reinforcement learning (RL), which achieves better generalization at higher computational cost. Dynamic Fine-Tuning (DFT) recently emerged as a promising middle ground, reweighting SFT objectives with token probabilities and achieving improvements in certain reasoning domains, though it exhibits instability in other tasks. We provide a analysis of DFT through the reward-weighted regression (RWR) framework, revealing that it corresponds to a specific auxiliary distribution choice that yields provably tighter RL bounds than standard SFT. However, our analysis also uncovers a critical limitation: this construction lacks distributional anchoring, leading to progressive drift that undermines training stability. To address this, we propose Anchored Supervised Fine-Tuning (ASFT), which augments DFT’s reweighting with lightweight KL regularization to preserve tightness while ensuring stability. Empirically, ASFT consistently outperforms both SFT and DFT across mathematical reasoning, medical knowledge grounding, and code generation, achieving substantial improvements with minimal computational overhead. Our RWR framework provides a systematic lens for understanding post-training methods and demonstrates that principled theoretical analysis leads to both stronger guarantees and practical gains.

[382] Knowledge Homophily in Large Language Models

Utkarsh Sahu,Zhisheng Qi,Mahantesh Halappanavar,Nedim Lipka,Ryan A. Rossi,Franck Dernoncourt,Yu Zhang,Yao Ma,Yu Wang

Main category: cs.LG

TL;DR: 该论文研究了大型语言模型（LLMs）中的知识同质性现象，通过图表示和GNN模型预测知识覆盖，优化知识注入和多跳推理的效率。

Details Motivation: LLMs作为知识库的应用日益广泛，但其知识结构尚未充分研究。受认知神经科学中语义聚类和启动效应的启发，探索LLMs中类似的知识同质性模式。

Contribution: 1.提出LLMs知识同质性的概念；2.设计了一种基于图的GNN回归模型预测知识覆盖；3.优化了知识注入和多跳推理的效率。

Method: 1.将LLMs知识映射为图表示；2.分析实体及其邻居的知识性关系；3.用GNN回归模型预测知识性分数，优先检查知识覆盖率低的节点。

Result: GNN模型有效预测知识性分数，优化了知识注入和多跳推理的效率，提高了标签预算下的知识覆盖率。

Insight: LLMs的知识呈现同质性分布，利用图结构可以高效识别和填补知识空白，为知识增强任务提供新方向。

Abstract: Large Language Models (LLMs) have been increasingly studied as neural knowledge bases for supporting knowledge-intensive applications such as question answering and fact checking. However, the structural organization of their knowledge remains unexplored. Inspired by cognitive neuroscience findings, such as semantic clustering and priming, where knowing one fact increases the likelihood of recalling related facts, we investigate an analogous knowledge homophily pattern in LLMs. To this end, we map LLM knowledge into a graph representation through knowledge checking at both the triplet and entity levels. After that, we analyze the knowledgeability relationship between an entity and its neighbors, discovering that LLMs tend to possess a similar level of knowledge about entities positioned closer in the graph. Motivated by this homophily principle, we propose a Graph Neural Network (GNN) regression model to estimate entity-level knowledgeability scores for triplets by leveraging their neighborhood scores. The predicted knowledgeability enables us to prioritize checking less well-known triplets, thereby maximizing knowledge coverage under the same labeling budget. This not only improves the efficiency of active labeling for fine-tuning to inject knowledge into LLMs but also enhances multi-hop path retrieval in reasoning-intensive question answering.

[383] Beyond the Exploration-Exploitation Trade-off: A Hidden State Approach for LLM Reasoning in RLVR

Fanding Huang,Guanbo Huang,Xiao Fan,Yi He,Xiao Liang,Xiao Chen,Qinting Jiang,Faisal Nadeem Khan,Jingyan Jiang,Zhi Wang

Main category: cs.LG

TL;DR: 这篇论文提出了VERL方法，通过隐藏状态空间的ER及其衍生指标ERV/ERA，解耦探索与利用的动态，实现了两者的协同增强，显著提升了LLM推理能力。

Details Motivation: 传统RLVR将探索与利用视为权衡关系，但作者认为这可能仅是测量层面的表象。论文通过隐藏状态空间的分析，揭示了两者可以解耦的潜在机会。

Contribution: 1. 提出ER及其衍生指标ERV/ERA，量化隐藏状态空间的探索与利用动态；2. 开发VERL方法，首次通过直接调整RL优势函数实现两者的协同增强。

Method: 采用Effective Rank (ER) 量化探索动态，并提出其衍生的ERV和ERA指标捕捉利用动态。VERL方法利用ERA作为预测元控制器，设计双通道奖励结构，协同增强探索与利用。

Result: 在多样化LLM和推理基准测试中，VERL带来了一致的性能提升，特别是在Gaokao 2024数据集上实现了21.4%的绝对准确率提升。

Insight: 探索与利用并非必须权衡的关系，通过隐藏状态空间的分析和方法设计，可以同时增强两者的能力。

Abstract: A prevailing view in Reinforcement Learning for Verifiable Rewards (RLVR) interprets recent progress through the lens of an exploration-exploitation trade-off, a perspective largely shaped by token-level metrics. We re-examine this perspective, proposing that this perceived trade-off may not be a fundamental constraint but rather an artifact of the measurement level. To investigate this, we shift the analysis to the semantically rich hidden-state space, adopting Effective Rank (ER) to quantify exploration and proposing its novel first- and second-order derivatives, named Effective Rank Velocity (ERV) and Effective Rank Acceleration (ERA), to capture exploitation dynamics. Our analysis reveals that at the hidden-state level, exploration and exploitation could be decoupled (Sec. 4). This finding reveals an opportunity to enhance both capacities simultaneously. This insight motivates our method, Velocity-Exploiting Rank-Learning (VERL), the first to operationalize the principle of synergistic exploration-exploitation enhancement by directly shaping the RL advantage function. The key innovation is leveraging the theoretically stable ERA as a predictive meta-controller to create a synergistic, dual-channel incentive structure. Instead of forcing a trade-off, VERL prospectively amplifies rewards for exploration to preempt overconfidence and reinforces exploitative gains to consolidate reasoning. Experiments across diverse LLMs and reasoning benchmarks show consistent gains, including up to 21.4% absolute accuracy improvement on the challenging Gaokao 2024 dataset.

[384] Explore-Execute Chain: Towards an Efficient Structured Reasoning Paradigm

Kaisen Yang,Lixuan He,Rushi Shah,Kaicheng Yang,Qinwei Ma,Dianbo Liu,Alex Lamb

Main category: cs.LG

TL;DR: 《Explore-Execute Chain》提出了一种结构化的推理框架$E^2C$，通过将推理分解为探索阶段和执行阶段，解决了现有方法（如CoT）在高层次规划与低层次执行耦合时的问题，提高了计算效率、路径探索能力和可解释性。

Details Motivation: 现有方法（如Chain-of-Thought）的推理过程在高层次规划和低层次执行上存在耦合，导致计算效率低、路径探索有限且可解释性差。

Contribution: 提出$E^2C$框架，将推理分解为探索（随机生成高层次计划）和执行（确定性执行）两个阶段；设计了结合监督微调和强化学习的训练方法；展示了高效的测试时扩展和跨领域适应性。

Method: 采用两阶段推理（探索+执行）和两阶段训练（SFT+RL）。SFT阶段通过数据生成算法强化计划的严格遵循，RL阶段利用了探索的信息性。

Result: 在AIME’2024上达到58.1%的准确率，仅使用Forest-of-Thought方法10%的解码token；在跨域适应中，EF-SFT仅用3.5%的token，实现了比标准SFT高14.5%的准确率。

Insight: 分离规划与执行不仅能提升效率和性能，还能增强模型的泛化能力和可解释性。

Abstract: Chain-of-Thought (CoT) and its variants have markedly advanced the reasoning abilities of Large Language Models (LLMs), yet their monolithic and auto-regressive architecture inherently conflates high-level strategic planning with low-level step-by-step execution, leading to computational inefficiency, limited exploration of reasoning paths, and reduced interpretability. To overcome these issues, we propose the Explore-Execute Chain ($E^2C$), a structured reasoning framework that decouples reasoning into two distinct phases: an exploratory phase that stochastically generates succinct high-level plans, followed by an execution phase that deterministically carries out the chosen plan. Our approach incorporates a two-stage training methodology, which combines Supervised Fine-Tuning (SFT) - augmented by a novel data generation algorithm enforcing strict plan adherence - with a subsequent Reinforcement Learning (RL) stage that capitalizes on the informativeness of exploration and reinforces the determinism of execution.This decomposition enables an efficient test-time scaling strategy: on AIME’2024, $E^2C$ Test Time Scaling reaches 58.1% accuracy using <10% of the decoding tokens required by comparable methods (e.g., Forest-of-Thought), sharply cutting self-consistency overhead. For cross-domain adaptation, our Exploration-Focused SFT (EF-SFT) fine-tunes with only 3.5% of the tokens used by standard SFT yet yields up to 14.5% higher accuracy than standard SFT on medical benchmarks, delivering state-of-the-art performance, strong generalization, and greater interpretability by separating planning from execution. The code and pre-trained models for the project are available at: https://github.com/yks23/Explore-Execute-Chain.git

[385] Group-Relative REINFORCE Is Secretly an Off-Policy Algorithm: Demystifying Some Myths About GRPO and Its Friends

Chaorui Yao,Yanxi Chen,Yuchang Sun,Yushuo Chen,Wenhao Zhang,Xuchen Pan,Yaliang Li,Bolin Ding

Main category: cs.LG

TL;DR: 这篇论文揭示了Group-Relative REINFORCE（GRPO）及其相关算法实际上是一种离策略算法，打破了其传统上被认为是严格的on-policy算法的误区。通过理论推导和实验验证，论文提出了将REINFORCE适应离策略环境的通用原则。

Details Motivation: 长期以来，REINFORCE及其变体（如GRPO）被认为只能在严格on-policy环境下工作，但在实际应用中（如大型语言模型的强化学习），离策略数据的需求日益增长。论文旨在重新审视这些算法的本质，并解决其在离策略环境中的适用性问题。

Contribution: 论文的主要贡献包括：1）首次从基本原理推导出Group-Relative REINFORCE的离策略解释；2）提出并验证了两种将REINFORCE适应离策略环境的通用原则；3）重新解读了OPMD和AsymRE算法，揭示了它们与REINFORCE的联系；4）为看似启发式的数据加权策略提供了理论支持。

Method: 论文通过理论推导，展示了Group-Relative REINFORCE可以不依赖特定数据分布假设而自然地支持离策略学习。方法核心包括：1）正则化策略更新；2）主动调整数据分布。此外，论文还通过实验验证了这些原则的有效性。

Result: 理论和实验结果表明，Group-Relative REINFORCE可以有效应用于离策略环境。提出的方法不仅统一了OPMD和AsymRE等算法，还为这些算法的性能提供了理论解释。

Insight: 论文揭示了REINFORCE系列算法在离策略环境中的潜力，打破了传统观念。此外，论文提出的正则化和数据分布调整原则为未来算法设计提供了新的方向。

Abstract: Off-policy reinforcement learning (RL) for large language models (LLMs) is attracting growing interest, driven by practical constraints in real-world applications, the complexity of LLM-RL infrastructure, and the need for further innovations of RL methodologies. While classic REINFORCE and its modern variants like Group Relative Policy Optimization (GRPO) are typically regarded as on-policy algorithms with limited tolerance of off-policyness, we present in this work a first-principles derivation for group-relative REINFORCE without assuming a specific training data distribution, showing that it admits a native off-policy interpretation. This perspective yields two general principles for adapting REINFORCE to off-policy settings: regularizing policy updates, and actively shaping the data distribution. Our analysis demystifies some myths about the roles of importance sampling and clipping in GRPO, unifies and reinterprets two recent algorithms – Online Policy Mirror Descent (OPMD) and Asymmetric REINFORCE (AsymRE) – as regularized forms of the REINFORCE loss, and offers theoretical justification for seemingly heuristic data-weighting strategies. Our findings lead to actionable insights that are validated with extensive empirical studies, and open up new opportunities for principled algorithm design in off-policy RL for LLMs. Source code for this work is available at https://github.com/modelscope/Trinity-RFT/tree/main/examples/rec_gsm8k.

[386] When Greedy Wins: Emergent Exploitation Bias in Meta-Bandit LLM Training

Sanxing Chen,Xiaoyin Chen,Yukun Huang,Roy Xie,Bhuwan Dhingra

Main category: cs.LG

TL;DR: 该论文研究了通过监督微调（SFT）和强化学习（RL）训练大型语言模型（LLM）在序列决策任务（如多臂老虎机问题）中的表现，发现这些方法虽然提高了性能，但也可能导致模型过早放弃探索，表现出更强的贪婪性。

Details Motivation: LLM在序列决策中的探索能力不足，现有方法（如SFT和RL）的效果和泛化能力尚不明确。

Contribution: 比较了SFT和RL的优劣，揭示了训练后的LLM倾向于更贪婪的利用策略，并提出了改进奖励设计和评估的建议。

Method: 通过SFT训练专家轨迹和RL训练（使用定制奖励信号，如遗憾形奖励和算法奖励）来提升LLM的性能。

Result: 训练后的LLM性能接近UCB和Thompson Sampling，但泛化时容易过早放弃探索。RL/SFT训练的模型比预训练模型更易出现早期灾难性失败。

Insight: 训练方法可能导致模型偏向贪婪策略，需要设计更全面的奖励和评估指标以提升探索鲁棒性。

Abstract: While Large Language Models (LLMs) hold promise to become autonomous agents, they often explore suboptimally in sequential decision-making. Recent work has sought to enhance this capability via supervised fine-tuning (SFT) or reinforcement learning (RL), improving regret on the classic multi-armed bandit task. However, it remains unclear how these learning methods shape exploration strategies and how well they generalize. We investigate both paradigms by training LLMs with SFT on expert trajectories and RL with a range of tailored reward signals including a strategic, regret-shaped reward to reduce variance, and an algorithmic reward that enables oracle imitation. The resulting agents outperform pre-trained models and achieve performance comparable to Upper Confidence Bound (UCB) and Thompson Sampling, with robust generalization to 6x longer horizons and across bandit families. Behavioral analysis reveals that gains often stem from more sophisticated but greedier exploitation: RL/SFT agents are more prone to early catastrophic failure than pre-trained models, prematurely abandoning exploration. Furthermore, agents trained to imitate UCB learn to outperform their teacher by adopting more exploitative variants. Our findings clarify when each training paradigm is preferable and advocate tailored reward design and evaluation beyond average regret to promote robust exploratory behavior.

[387] ORPO-Distill: Mixed-Policy Preference Optimization for Cross-Architecture LLM Distillation

Aasheesh Singh,Vishal Vaddina,Dagnachew Birru

Main category: cs.LG

TL;DR: ORPO-Distill提出了一个通用的跨架构LLM蒸馏方法，通过偏好优化任务实现知识转移，优于传统方法。

Details Motivation: 传统CoT蒸馏方法在跨架构LLM蒸馏中存在局限性，需要更有效的方式转移多样化推理路径的知识。

Contribution: 提出ORPO-Distill方法，采用Odds-Ratio偏好优化目标，结合混合策略利用学生生成输出，提升蒸馏效果。

Method: 基于Odds-Ratio偏好优化对比教师和学生推理路径，采用混合策略整合学生输出。

Result: 在五个数据集和多个学生模型上实验，一致优于传统黑盒KD基线方法。

Insight: 偏好优化目标和混合策略的结合能够更有效地实现跨架构知识蒸馏。

Abstract: We introduce ORPO-Distill, a general-purpose method for cross-architecture LLM distillation that formulates the problem as a preference optimization task. Unlike standard CoT distillation, the approach transfers knowledge through diverse reasoning traces. It employs an Odds-Ratio Preference Optimization objective that contrasts teacher and student traces for more effective learning, and adopts a mixed-policy strategy for utilizing student-generated outputs, outperforming both off- and on-policy alternatives. Experiments on five datasets and multiple student models show consistent improvements over conventional black-box KD baselines.

[388] Rethinking Entropy Regularization in Large Reasoning Models

Yuxian Jiang,Yafu Li,Guanxu Chen,Dongrui Liu,Yu Cheng,Jing Shao

Main category: cs.LG

TL;DR: 论文分析了熵正则化在大型推理模型（LRMs）中的失效问题，并提出了一种名为SIREN的方法，通过选择性熵正则化来解决熵爆炸和过早收敛的问题。

Details Motivation: 传统的熵正则化方法在大型推理模型的强化学习中表现不佳，导致熵爆炸和过早收敛。需要一种新方法来限制探索范围，提升训练稳定性。

Contribution: 提出了SIREN方法，通过两步熵掩码机制（top-p掩码和peak-entropy掩码）及自锚定的正则化形式，有效解决熵爆炸问题，提升模型性能。

Method: SIREN结合了top-p掩码和peak-entropy掩码，将熵正则化限制在关键动作和状态子集内，并通过自锚定形式稳定训练。

Result: 在五个数学基准上，SIREN显著优于其他熵相关方法，如在AIME24/25上提升了6.6 maj@k，同时保持了适当的熵值和回答多样性。

Insight: 选择性熵正则化是关键，它能避免全局熵爆炸，同时保留验证性能，解决了大型推理模型中常见的过早收敛问题。

Abstract: Reinforcement learning with verifiable rewards (RLVR) has shown great promise in enhancing the reasoning abilities of large reasoning models (LRMs). However, it suffers from a critical issue: entropy collapse and premature convergence. Naive entropy regularization, a common approach for encouraging exploration in the traditional RL literature, fails to address this problem in the context of LRM. Our analysis reveals that this failure stems from the vast action space and long trajectories in LRMs, which easily trigger a global entropy explosion as the model indiscriminately explores all possible actions and states. To address this, we propose SIREN (SelectIve entRopy rEgularizatioN), a method that confines exploration to a meaningful subset of actions and states. SIREN achieves this through a two-step entropy masking mechanism, consisting of a top-p mask and a peak-entropy mask. In addition, regularization is transformed into a self-anchored form to stabilize training. Across five mathematical benchmarks, SIREN attains superior average performance over previous entropy-related RLVR approaches, exemplified by a +6.6 maj@k improvement on AIME24/25 with Qwen2.5-Math-7B. Further analysis confirms that SIREN promotes greater response diversity and maintains entropy at an appropriate level, which helps to preserve the validation pass@k throughout training. This effectively mitigates the premature convergence problem common in RLVR for LRM.

[389] SIRI: Scaling Iterative Reinforcement Learning with Interleaved Compression

Haoming Wen,Yushi Bai,Juanzi Li,Jie Tang

Main category: cs.LG

TL;DR: SIRI提出了一种交替压缩与扩展推理预算的训练方法，通过动态调整最大推理长度，在性能与效率间取得平衡。

Details Motivation: 现有大型推理模型存在重复思维模式，缩减推理长度常以性能下降为代价，SIRI旨在解决这一权衡问题。

Contribution: 1. 提出SIRI框架，通过交替压缩与扩展推理预算实现高效推理；2. 在压缩阶段减少冗余推理，扩展阶段支持长程规划；3. 实验结果显著提升性能并降低token使用量。

Method: 训练中迭代交替压缩（缩短推理长度）和扩展（放宽长度限制），动态调整模型推理能力。

Result: SIRI-low在AIME24上性能提升43.2%，token使用减少46.9%；SIRI-high达到最高精度。

Insight: 动态调整推理长度能平衡探索与效率，逐步逼近性能-效率的Pareto前沿。

Abstract: We introduce SIRI, Scaling Iterative Reinforcement Learning with Interleaved Compression, a simple yet effective RL approach for Large Reasoning Models (LRMs) that enables more efficient and accurate reasoning. Existing studies have observed repetitive thinking patterns in LRMs, and attempts to reduce them often come at the cost of performance. In this paper, we show that this trade-off can be overcome through a training regime that iteratively alternates between compressing and expanding the reasoning budget, by dynamically adjusting the maximum rollout length during training. The compression phase cuts the rollout length, forcing the model to make precise and valuable decisions within a limited context, which effectively reduces redundant tokens and increases reasoning density. The expansion phase then relaxes the length limit, providing space for the model to explore and plan in long-horizon settings. Remarkably, we find that after each compression-expansion cycle, the model’s performance improves even as its output length decreases, steadily pushing it closer to the Pareto frontier in the performance-efficiency trade-off. Training on DeepSeek-R1-Distill-Qwen-1.5B, SIRI-low improves performance on AIME24 by 43.2% while reducing token usage by 46.9% after three iterations, and SIRI-high achieves the highest accuracy compared to all other methods (Figure 1). Our findings shed light on the potential of periodically oscillating the LRM’s output truncation length during training to dynamically balance exploration and efficiency in reasoning, converging towards an optimal “sweet spot” between the two. Our models are publicly available.

[390] Localizing Adversarial Attacks To Produces More Imperceptible Noise

Pavan Reddy,Aditya Sanjay Gujral

Main category: cs.LG

TL;DR: 该论文通过引入二元掩码技术，系统地评估了局部对抗攻击的有效性、不易察觉性和计算效率。相比全局攻击，局部攻击在像素扰动、PSNR和SSIM指标上表现更优，但需付出更高的计算代价和轻微的攻击成功率下降。

Details Motivation: 传统对抗攻击通常采用全局扰动，而局部对抗噪声的效果和潜力尚未被充分探索。论文旨在填补这一空白。

Contribution: 1. 提出局部对抗攻击方法并量化其效果；2. 比较多种攻击方法在局部条件下的表现；3. 揭示PGD和C&W等方法在局部约束下的鲁棒性。

Method: 引入二元掩码将噪声限制在特定区域，使用FGSM、PGD和C&W等方法进行实验评估。

Result: 局部攻击在像素扰动、PSNR和SSIM上优于全局攻击，但攻击成功率和计算效率略有下降。PGD和C&W在局部约束下表现更佳。

Insight: 局部对抗攻击在提升噪声不易察觉性方面具有潜力，但需权衡计算成本和攻击效果，迭代方法更适合局部攻击场景。

Abstract: Adversarial attacks in machine learning traditionally focus on global perturbations to input data, yet the potential of localized adversarial noise remains underexplored. This study systematically evaluates localized adversarial attacks across widely-used methods, including FGSM, PGD, and C&W, to quantify their effectiveness, imperceptibility, and computational efficiency. By introducing a binary mask to constrain noise to specific regions, localized attacks achieve significantly lower mean pixel perturbations, higher Peak Signal-to-Noise Ratios (PSNR), and improved Structural Similarity Index (SSIM) compared to global attacks. However, these benefits come at the cost of increased computational effort and a modest reduction in Attack Success Rate (ASR). Our results highlight that iterative methods, such as PGD and C&W, are more robust to localization constraints than single-step methods like FGSM, maintaining higher ASR and imperceptibility metrics. This work provides a comprehensive analysis of localized adversarial attacks, offering practical insights for advancing attack strategies and designing robust defensive systems.

[391] Graph Your Own Prompt

Xi Ding,Lei Wang,Piotr Koniusz,Yongsheng Gao

Main category: cs.LG

TL;DR: 论文提出了一种名为图一致性正则化（GCR）的新框架，通过将模型预测生成的关系图结构引入学习过程，以增强类别感知和语义特征表示。

Details Motivation: 深度网络学习到的特征表示通常包含噪声类间相似性，与模型预测语义不符，GCR旨在通过图结构对齐解决这一问题。

Contribution: 主要贡献是提出了参数无关的图一致性层（GCL）和多层跨空间图对齐机制，通过自适应加权学习层重要性。

Method: GCL构建批次级特征相似图，并与全局类别感知掩码预测图对齐；采用自适应加权机制学习层重要性。

Result: 实验表明，GCR能提升特征结构清晰度、增强类内凝聚力并改善泛化能力。

Insight: GCR通过自提示机制，利用模型自身输出来优化内部结构，为学习预测结构提供了新视角。

Abstract: We propose Graph Consistency Regularization (GCR), a novel framework that injects relational graph structures, derived from model predictions, into the learning process to promote class-aware, semantically meaningful feature representations. Functioning as a form of self-prompting, GCR enables the model to refine its internal structure using its own outputs. While deep networks learn rich representations, these often capture noisy inter-class similarities that contradict the model’s predicted semantics. GCR addresses this issue by introducing parameter-free Graph Consistency Layers (GCLs) at arbitrary depths. Each GCL builds a batch-level feature similarity graph and aligns it with a global, class-aware masked prediction graph, derived by modulating softmax prediction similarities with intra-class indicators. This alignment enforces that feature-level relationships reflect class-consistent prediction behavior, acting as a semantic regularizer throughout the network. Unlike prior work, GCR introduces a multi-layer, cross-space graph alignment mechanism with adaptive weighting, where layer importance is learned from graph discrepancy magnitudes. This allows the model to prioritize semantically reliable layers and suppress noisy ones, enhancing feature quality without modifying the architecture or training procedure. GCR is model-agnostic, lightweight, and improves semantic structure across various networks and datasets. Experiments show that GCR promotes cleaner feature structure, stronger intra-class cohesion, and improved generalization, offering a new perspective on learning from prediction structure. Project website Code

[392] GBSK: Skeleton Clustering via Granular-ball Computing and Multi-Sampling for Large-Scale Data

Yewang Chen,Junfeng Li,Shuyin Xia,Qinghong Lai,Xinbo Gao,Guoyin Wang,Dongdong Cheng,Yi Liu,Yi Wang

Main category: cs.LG

TL;DR: GBSK是一种基于粒度球计算和多采样的新型骨架聚类算法，用于高效处理大规模数据集的聚类任务。其通过多粒度球构造逐步揭示数据的统计“骨架”，显著降低了计算开销并保持高聚类精度。

Details Motivation: 大规模数据集的聚类任务通常面临计算复杂度高、效率低的问题。传统方法难以在保持精度的同时高效处理此类数据，因此需要一种新的方法来平衡计算效率和聚类质量。

Contribution: 提出了GBSK算法，利用粒度球计算和多采样技术，构建数据的统计骨架以近似原始数据的结构和分布；同时还提出自适应版本AGBSK，简化参数设置，增强实用性。

Method: 通过多采样数据集和构造多粒度球，逐步抽象出数据的骨架；采用自适应策略进一步优化参数设置。

Result: 在标准硬件上进行的实验表明，GBSK在包含1亿实例的高维数据集上表现高效且聚类性能强。

Insight: 利用数据的多粒度抽象可以显著提升大规模聚类任务的效率，同时自适应参数设计有助于算法的实际部署。

Abstract: To effectively handle clustering task for large-scale datasets, we propose a novel scalable skeleton clustering algorithm, namely GBSK, which leverages the granular-ball technique to capture the underlying structure of data. By multi-sampling the dataset and constructing multi-grained granular-balls, GBSK progressively uncovers a statistical “skeleton” – a spatial abstraction that approximates the essential structure and distribution of the original data. This strategy enables GBSK to dramatically reduce computational overhead while maintaining high clustering accuracy. In addition, we introduce an adaptive version, AGBSK, with simplified parameter settings to enhance usability and facilitate deployment in real-world scenarios. Extensive experiments conducted on standard computing hardware demonstrate that GBSK achieves high efficiency and strong clustering performance on large-scale datasets, including one with up to 100 million instances across 256 dimensions. Our implementation and experimental results are available at: https://github.com/XFastDataLab/GBSK/.

[393] Efficient Multi-turn RL for GUI Agents via Decoupled Training and Adaptive Data Curation

Pengxiang Li,Zechen Hu,Zirui Shang,Jingrong Wu,Yang Liu,Hui Liu,Zhi Gao,Chenrui Shi,Bofei Zhang,Zihao Zhang,Xiaochuan Shi,Zedong YU,Yuwei Wu,Xinxiao Wu,Yunde Jia,Liuyu Xiang,Zhaofeng He,Qing Li

Main category: cs.LG

TL;DR: 论文提出了DART框架，通过解耦训练和自适应数据管理，解决了GUI代理在多轮RL训练中的效率问题，显著提升了任务成功率。

Details Motivation: GUI代理在多轮RL训练中面临交互速度慢和高质量交互数据不足的问题，亟需高效的学习框架。

Contribution: 提出了DART框架，通过解耦设计和自适应数据管理，显著提升训练效率和任务成功率。

Method: DART分为环境集群、rollout服务、数据管理和训练器四个异步模块，并引入自适应数据管理策略。

Result: 在OSWorld基准测试中，DART-GUI-7B任务成功率为42.13%，比基线模型高出14.61%。

Insight: 解耦设计和自适应数据管理能显著提升RL训练的效率和效果，尤其在GUI代理任务中。

Abstract: Vision-language model (VLM) based GUI agents show promise for automating complex desktop and mobile tasks, but face significant challenges in applying reinforcement learning (RL): (1) slow multi-turn interactions with GUI environments for policy rollout, and (2) insufficient high-quality agent-environment interactions for policy learning. To address these challenges, we propose DART, a Decoupled Agentic RL Training framework for GUI agents, which coordinates heterogeneous modules in a highly decoupled manner. DART separates the training system into four asynchronous modules: environment cluster, rollout service, data manager, and trainer. This design enables non-blocking communication, asynchronous training, rollout-wise trajectory sampling, and per-worker model synchronization, significantly improving the system efficiency: 1.6GPU utilization for rollout, 1.9 training throughput, and 5.5* environment utilization. To facilitate effective learning from abundant samples, we introduce an adaptive data curation scheme: (1) pre-collecting successful trajectories for challenging tasks to supplement sparse success in online sampling; (2) dynamically adjusting rollout numbers and trajectory lengths based on task difficulty; (3) training selectively on high-entropy steps to prioritize critical decisions; (4) stabilizing learning via truncated importance sampling for policy mismatch between policy rollout and updating. On the OSWorld benchmark, DART-GUI-7B achieves a 42.13% task success rate, a 14.61% absolute gain over the base model, and 7.34% higher than open-source SOTA. We will fully open-source our training framework, data, and model checkpoints via computer-use-agents.github.io/dart-gui, which we believe is a timely contribution to the open-source community of agentic RL training.

[394] SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse-Linear Attention

Jintao Zhang,Haoxu Wang,Kai Jiang,Shuo Yang,Kaiwen Zheng,Haocheng Xi,Ziteng Wang,Hongzhou Zhu,Min Zhao,Ion Stoica,Joseph E. Gonzalez,Jun Zhu,Jianfei Chen

Main category: cs.LG

TL;DR: SLA是一种可微调的稀疏-线性注意力机制，通过分类注意力权重为关键、边缘和可忽略三部分，显著减少DiT模型的计算复杂度，实现20倍注意力计算减少和2.2倍端到端视频生成加速，不损失生成质量。

Details Motivation: 在Diffusion Transformer（DiT）模型中，尤其是视频生成时，注意力机制的延迟是主要瓶颈，因其序列长且计算复杂度为二次方。

Contribution: 提出SLA（Sparse-Linear Attention），一种融合稀疏和线性注意力的可训练方法，将注意力权重分类为关键、边缘和可忽略三类，分别采用不同的计算方式，显著减少计算量。

Method: SLA通过分析注意力权重的高/低秩特性，将关键权重用O(N^2)计算，边缘权重用O(N)线性计算，跳过可忽略权重，并将这些计算合并到一个GPU核中，支持前向和反向传播。

Result: 实验显示，SLA减少了95%的注意力计算，DiT模型的注意力计算速度提升13.7倍，视频生成端到端速度提升2.2倍，且生成质量不下降。

Insight: 注意力权重具有天然的稀疏性和低秩特性，利用这一特性可以显著优化计算效率。

Abstract: In Diffusion Transformer (DiT) models, particularly for video generation, attention latency is a major bottleneck due to the long sequence length and the quadratic complexity. We find that attention weights can be separated into two parts: a small fraction of large weights with high rank and the remaining weights with very low rank. This naturally suggests applying sparse acceleration to the first part and low-rank acceleration to the second. Based on this finding, we propose SLA (Sparse-Linear Attention), a trainable attention method that fuses sparse and linear attention to accelerate diffusion models. SLA classifies attention weights into critical, marginal, and negligible categories, applying O(N^2) attention to critical weights, O(N) attention to marginal weights, and skipping negligible ones. SLA combines these computations into a single GPU kernel and supports both forward and backward passes. With only a few fine-tuning steps using SLA, DiT models achieve a 20x reduction in attention computation, resulting in significant acceleration without loss of generation quality. Experiments show that SLA reduces attention computation by 95% without degrading end-to-end generation quality, outperforming baseline methods. In addition, we implement an efficient GPU kernel for SLA, which yields a 13.7x speedup in attention computation and a 2.2x end-to-end speedup in video generation on Wan2.1-1.3B.

[395] GPS-MTM: Capturing Pattern of Normalcy in GPS-Trajectories with self-supervised learning

Umang Garg,Bowen Zhang,Anantanjit Subrahmanya,Chandrakanth Gudavalli,BS Manjunath

Main category: cs.LG

TL;DR: GPS-MTM是一种基于自监督学习的轨迹建模基础模型，通过分解移动数据的模态（状态和动作）并利用双向Transformer，在无需手动标注的情况下学习语义关联，显著提升了轨迹填充和下一站预测等下游任务的表现。

Details Motivation: 现有轨迹建模方法多将轨迹扁平化为坐标流，难以捕捉人类移动的语义模式。GPS-MTM旨在通过分解移动数据的模态并利用自监督学习，构建更强大的轨迹基础模型。

Contribution: 提出了GPS-MTM模型，首次将轨迹分解为状态和动作两种模态，并通过双向Transformer和掩码建模目标实现自监督学习，显著提升了轨迹分析的性能。

Method: 将轨迹分解为状态（兴趣点类别）和动作（移动过渡）两种模态，采用双向Transformer和掩码建模目标进行自监督学习，重建缺失的轨迹片段。

Result: 在Numosim-LA、Urban Anomalies和Geolife等基准数据集上，GPS-MTM在轨迹填充和下一站预测等任务中表现优异，尤其在依赖上下文推理的动态任务中优势显著。

Insight: GPS-MTM表明，分解轨迹为语义模态并结合自监督学习，能够有效捕捉人类移动的正常模式，为大规模轨迹分析提供了新思路。

Abstract: Foundation models have driven remarkable progress in text, vision, and video understanding, and are now poised to unlock similar breakthroughs in trajectory modeling. We introduce the GPSMasked Trajectory Transformer (GPS-MTM), a foundation model for large-scale mobility data that captures patterns of normalcy in human movement. Unlike prior approaches that flatten trajectories into coordinate streams, GPS-MTM decomposes mobility into two complementary modalities: states (point-of-interest categories) and actions (agent transitions). Leveraging a bi-directional Transformer with a self-supervised masked modeling objective, the model reconstructs missing segments across modalities, enabling it to learn rich semantic correlations without manual labels. Across benchmark datasets, including Numosim-LA, Urban Anomalies, and Geolife, GPS-MTM consistently outperforms on downstream tasks such as trajectory infilling and next-stop prediction. Its advantages are most pronounced in dynamic tasks (inverse and forward dynamics), where contextual reasoning is critical. These results establish GPS-MTM as a robust foundation model for trajectory analytics, positioning mobility data as a first-class modality for large-scale representation learning. Code is released for further reference.

[396] AQUAIR: A High-Resolution Indoor Environmental Quality Dataset for Smart Aquaculture Monitoring

Youssef Sabiri,Walid Houmaidi,Ouail El Maadi,Yousra Chtouki

Main category: cs.LG

TL;DR: AQUAIR 是一个高分辨率的室内环境质量数据集，专注于智能水产养殖监测，填补了当前公共数据集的空白，为预测和异常检测工具的开发提供了支持。

Details Motivation: 当前智能水产养殖系统缺乏描述养殖池周围空气环境的公共数据集，限制了预测和异常检测工具的研发，从而影响了鱼类福利和能源优化。

Contribution: AQUAIR 提供了一个开源、高质量的公共数据集，记录了室内养殖场的六项环境指标，并提供了完整的质量控制和处理流程。

Method: 数据集通过 Awair HOME 监测器每五分钟采样一次，记录了空气温度、湿度、二氧化碳等指标。数据处理流程包括时间戳标准化、插补缺失值和导出分析就绪的数据表格。

Result: 数据集包含超过23,000条时间戳观测数据，展示了稳定的环境条件和明显的喂食时间峰值，适用于短时预测、事件检测和传感器漂移研究。

Insight: AQUAIR 填补了智能水产养殖信息学中的数据空白，为机器学习和环境传感研究提供了可重复的基准，特别是在循环水产系统中头部空间动态的研究。

Abstract: Smart aquaculture systems depend on rich environmental data streams to protect fish welfare, optimize feeding, and reduce energy use. Yet public datasets that describe the air surrounding indoor tanks remain scarce, limiting the development of forecasting and anomaly-detection tools that couple head-space conditions with water-quality dynamics. We therefore introduce AQUAIR, an open-access public dataset that logs six Indoor Environmental Quality (IEQ) variables–air temperature, relative humidity, carbon dioxide, total volatile organic compounds, PM2.5 and PM10–inside a fish aquaculture facility in Amghass, Azrou, Morocco. A single Awair HOME monitor sampled every five minutes from 14 October 2024 to 9 January 2025, producing more than 23,000 time-stamped observations that are fully quality-controlled and publicly archived on Figshare. We describe the sensor placement, ISO-compliant mounting height, calibration checks against reference instruments, and an open-source processing pipeline that normalizes timestamps, interpolates short gaps, and exports analysis-ready tables. Exploratory statistics show stable conditions (median CO2 = 758 ppm; PM2.5 = 12 micrograms/m3) with pronounced feeding-time peaks, offering rich structure for short-horizon forecasting, event detection, and sensor drift studies. AQUAIR thus fills a critical gap in smart aquaculture informatics and provides a reproducible benchmark for data-centric machine learning curricula and environmental sensing research focused on head-space dynamics in recirculating aquaculture systems.

[397] Clebsch-Gordan Transformer: Fast and Global Equivariant Attention

Owen Lewis Howell,Linfeng Zhao,Xupeng Zhu,Yaoyao Qian,Haojie Huang,Lingfeng Sun,Wil Thomason,Robert Platt,Robin Walters

Main category: cs.LG

TL;DR: 论文提出Clebsch-Gordan Transformer，通过基于SO(3)不可约表示的Clebsch-Gordan卷积实现高效的全局注意力，支持所有阶数的等变性特征，计算复杂度为O(N log N)，并在多个任务中验证了其性能。

Details Motivation: 现有等变性Transformer仅支持低阶等变特征和局部上下文窗口，限制了表达能力和性能，亟需一种高效支持全局注意力和高阶等变性的方法。

Contribution: 1. 提出Clebsch-Gordan Transformer，通过Clebsch-Gordan卷积实现高效的全局注意力；2. 支持所有阶数的等变性特征，计算复杂度为O(N log N)；3. 在高阶不可约特征上扩展性良好；4. 可选地支持标记置换等变性。

Method: 1. 基于SO(3)不可约表示设计Clebsch-Gordan卷积；2. 利用Clebsch-Gordan矩阵的稀疏性优化高阶特征的计算；3. 通过权重共享或数据增广实现标记置换等变性。

Result: 在n-body模拟、QM9、ModelNet点云分类和机器人抓取任务上，方法在显存占用、速度和精度上均优于现有等变性Transformer。

Insight: 通过几何结构的等变性建模可显著提升Transformer在物理和视觉任务中的性能，同时高效实现全局注意力是关键。

Abstract: The global attention mechanism is one of the keys to the success of transformer architecture, but it incurs quadratic computational costs in relation to the number of tokens. On the other hand, equivariant models, which leverage the underlying geometric structures of problem instance, often achieve superior accuracy in physical, biochemical, computer vision, and robotic tasks, at the cost of additional compute requirements. As a result, existing equivariant transformers only support low-order equivariant features and local context windows, limiting their expressiveness and performance. This work proposes Clebsch-Gordan Transformer, achieving efficient global attention by a novel Clebsch-Gordon Convolution on $\SO(3)$ irreducible representations. Our method enables equivariant modeling of features at all orders while achieving ${O}(N \log N)$ input token complexity. Additionally, the proposed method scales well with high-order irreducible features, by exploiting the sparsity of the Clebsch-Gordon matrix. Lastly, we also incorporate optional token permutation equivariance through either weight sharing or data augmentation. We benchmark our method on a diverse set of benchmarks including n-body simulation, QM9, ModelNet point cloud classification and a robotic grasping dataset, showing clear gains over existing equivariant transformers in GPU memory size, speed, and accuracy.

[398] Semantic Editing with Coupled Stochastic Differential Equations

Jianxin Zhang,Clayton Scott

Main category: cs.LG

TL;DR: 论文提出了一种利用耦合随机微分方程（coupled SDEs）的方法，用于改进预训练文本到图像模型的内容编辑能力，实现了高语义保真度和图像细节的一致性。

Details Motivation: 现有的预训练文本到图像模型在内容编辑时容易破坏细节或引入不希望的伪影，需要一种无需重新训练或辅助网络的解决方案。

Contribution: 提出了利用耦合SDEs的方法，通过共享相关噪声驱动源图像和编辑图像，实现了高语义保真度和视觉一致性的图像编辑。

Method: 利用耦合SDEs引导预训练生成模型的采样过程，包括扩散模型和整流流模型。通过共享噪声，保证编辑图像与源图像的视觉一致性。

Result: 该方法无需额外训练或辅助网络，实现了高语义保真度和像素级一致性的图像编辑。

Insight: 耦合SDEs为生成式AI提供了一种简单且强大的控制工具，能够有效平衡语义编辑和视觉一致性之间的需求。

Abstract: Editing the content of an image with a pretrained text-to-image model remains challenging. Existing methods often distort fine details or introduce unintended artifacts. We propose using coupled stochastic differential equations (coupled SDEs) to guide the sampling process of any pre-trained generative model that can be sampled by solving an SDE, including diffusion and rectified flow models. By driving both the source image and the edited image with the same correlated noise, our approach steers new samples toward the desired semantics while preserving visual similarity to the source. The method works out-of-the-box-without retraining or auxiliary networks-and achieves high prompt fidelity along with near-pixel-level consistency. These results position coupled SDEs as a simple yet powerful tool for controlled generative AI.

[399] Rethinking JEPA: Compute-Efficient Video SSL with Frozen Teachers

Xianhang Li,Chen Huang,Chun-Liang Li,Eran Malach,Josh Susskind,Vimal Thilak,Etai Littwin

Main category: cs.LG

TL;DR: 论文提出了SALT方法，通过冻结教师模型的权重，分两阶段训练学生模型，实现了计算高效的视频自监督学习。

Details Motivation: 传统的V-JEPA方法通过EMA更新教师模型防止表征坍塌，但限制了模型选择和架构灵活性。重新思考该方法，发现冻结教师模型同样可行。

Contribution: 提出了SALT方法，分像素重建（教师）和掩码潜在预测（学生）两阶段训练，实现了透明、高效且可扩展的视频表征学习。

Method: 1. 用像素重建目标训练教师模型；2. 冻结教师模型，训练学生模型预测掩码潜在空间。去除了EMA的复杂性。

Result: 学生模型在冻结评估中优于V-JEPA 2，计算效率更高，且对不同质量的教师模型表现出鲁棒性。

Insight: 学生模型的性能对教师模型质量依赖较低，计算资源应主要分配给学生模型而非教师模型。

Abstract: Video Joint Embedding Predictive Architectures (V-JEPA) learn generalizable off-the-shelf video representation by predicting masked regions in latent space with an exponential moving average (EMA)-updated teacher. While EMA prevents representation collapse, it complicates scalable model selection and couples teacher and student architectures. We revisit masked-latent prediction and show that a frozen teacher suffices. Concretely, we (i) train a target encoder with a simple pixel-reconstruction objective under V-JEPA masking, then (ii) freeze it and train a student to predict the teacher’s latents on masked regions. This leads to a two-stage, unregularized scheme that we refer to as SALT (Static-teacher Asymmetric Latent Training). SALT decouples optimization into pixel reconstruction (teacher) and masked latent prediction (student), increasing transparency, efficiency, and scalability while preserving the ability of representation to generalize under frozen evaluation. Empirically, our student models outperform recently proposed V-JEPA 2 encoders under frozen backbone evaluation across diverse benchmarks. They are also more compute-optimal: at matched pretraining FLOPs, our method achieves higher probing accuracy, and its scaling curves dominate V-JEPA’s accuracy-FLOPs Pareto frontier. Finally, we find that student quality is remarkably robust to teacher quality: high-performing students emerge even with small, sub-optimal teachers. This points to a compute budget allocation that should overwhelmingly favor the student. These results position SALT as a simple, scalable, and compute-efficient alternative to EMA-based self-distillation for video representation learning.

[400] A TRIANGLE Enables Multimodal Alignment Beyond Cosine Similarity

Giordano Cicchetti,Eleonora Grassucci,Danilo Comminiello

Main category: cs.LG

TL;DR: 论文提出TRIANGLE，一种新的多模态对齐度量方法，通过三角形面积相似性直接在高维空间计算，显著提升多模态模型的性能。

Details Motivation: 当前多模态学习模型在模态对齐上存在局限性，部分模态未被有效对齐，导致模型在下游任务中表现不佳。

Contribution: 提出TRIANGLE度量方法，通过三角形面积相似性替代余弦相似性，直接在高维空间对齐三模态。

Method: 采用三角形面积相似性计算模态嵌入之间的相似度，避免额外的融合层或成对相似性计算。

Result: 在多模态任务（如视频-文本检索、音频-文本检索等）中，TRIANGLE在Recall@1上比基于余弦的方法提升高达9个百分点。

Insight: TRIANGLE提供了可解释的对齐依据，同时显著提升了多模态模型的性能。

Abstract: Multimodal learning plays a pivotal role in advancing artificial intelligence systems by incorporating information from multiple modalities to build a more comprehensive representation. Despite its importance, current state-of-the-art models still suffer from severe limitations that prevent the successful development of a fully multimodal model. Such methods may not provide indicators that all the involved modalities are effectively aligned. As a result, some modalities may not be aligned, undermining the effectiveness of the model in downstream tasks where multiple modalities should provide additional information that the model fails to exploit. In this paper, we present TRIANGLE: TRI-modAl Neural Geometric LEarning, the novel proposed similarity measure that is directly computed in the higher-dimensional space spanned by the modality embeddings. TRIANGLE improves the joint alignment of three modalities via a triangle-area similarity, avoiding additional fusion layers or pairwise similarities. When incorporated in contrastive losses replacing cosine similarity, TRIANGLE significantly boosts the performance of multimodal modeling, while yielding interpretable alignment rationales. Extensive evaluation in three-modal tasks such as video-text and audio-text retrieval or audio-video classification, demonstrates that TRIANGLE achieves state-of-the-art results across different datasets improving the performance of cosine-based methods up to 9 points of Recall@1.

[401] Score-based Membership Inference on Diffusion Models

Mingxing Rao,Bowen Qu,Daniel Moyer

Main category: cs.LG

TL;DR: 这篇论文研究了基于评分的成员推理攻击（MIA）在扩散模型中的应用，提出了SimA这一高效单查询攻击方法，并发现潜在扩散模型（LDM）比像素空间模型更抗攻击。

Details Motivation: 扩散模型在生成任务中表现出色，但其隐私风险尚未充分研究。成员推理攻击可判断样本是否属于训练集，可能导致隐私泄露。

Contribution: 1.理论分析了扩散模型中评分函数的局部均值特性及其与训练集的距离关系；2.提出了SimA攻击方法；3.发现LDM因VAE的潜在瓶颈而更抗攻击。

Method: 通过分析去噪器输出的范数揭示样本与训练集的接近程度，提出SimA攻击方法，并在DDPM和LDM上验证。

Result: SimA在多种扩散模型上表现优异，LDM因潜在空间的信息瓶颈更抗攻击。

Insight: 潜在扩散模型的隐私保护优势源于VAE的强信息瓶颈，未来需要更深入理解VAE的反演问题。

Abstract: Membership inference attacks (MIAs) against diffusion models have emerged as a pressing privacy concern, as these models may inadvertently reveal whether a given sample was part of their training set. We present a theoretical and empirical study of score-based MIAs, focusing on the predicted noise vectors that diffusion models learn to approximate. We show that the expected denoiser output points toward a kernel-weighted local mean of nearby training samples, such that its norm encodes proximity to the training set and thereby reveals membership. Building on this observation, we propose SimA, a single-query attack that provides a principled, efficient alternative to existing multi-query methods. SimA achieves consistently strong performance across variants of DDPM, Latent Diffusion Model (LDM). Notably, we find that Latent Diffusion Models are surprisingly less vulnerable than pixel-space models, due to the strong information bottleneck imposed by their latent auto-encoder. We further investigate this by differing the regularization hyperparameters ($\beta$ in $\beta$-VAE) in latent channel and suggest a strategy to make LDM training more robust to MIA. Our results solidify the theory of score-based MIAs, while highlighting that Latent Diffusion class of methods requires better understanding of inversion for VAE, and not simply inversion of the Diffusion process

astro-ph.IM [Back]

[402] Interpreting deep learning-based stellar mass estimation via causal analysis and mutual information decomposition

Wei Zhang,Qiufan Lin,Yuan-Sen Ting,Shupei Chen,Hengxin Ruan,Song Li,Yifan Wang

Main category: astro-ph.IM

TL;DR: 论文通过因果分析和互信息分解技术，解释了基于深度学习的星系质量估计模型，揭示了输入数据的多成分贡献及其物理意义。

Details Motivation: 由于端到端深度学习模型的不可解释性和关联性，难以理解除积分测光之外的信息（如形态学）如何影响星系质量的估计。为此，研究旨在通过因果分析和互信息分解技术提升模型的可解释性。

Contribution: 主要贡献是通过因果分析和互信息分解技术，量化了输入数据对星系质量估计的冗余、独特和协同贡献，并提供了物理解释。

Method: 结合SDSS和WISE数据，使用因果分析揭示变量间的因果路径，通过互信息分解量化输入数据的多成分贡献。

Result: 研究获得了有意义的结果，为基于图像的模型提供了物理解释，并展示了深度学习与可解释性技术结合的潜力。

Insight: 研究揭示了深度学习模型在星系质量估计中的信息利用机制，为天体物理学参数估计和复杂多变量物理过程的研究提供了新方法。

Abstract: End-to-end deep learning models fed with multi-band galaxy images are powerful data-driven tools used to estimate galaxy physical properties in the absence of spectroscopy. However, due to a lack of interpretability and the associational nature of such models, it is difficult to understand how the information additional to integrated photometry (e.g., morphology) contributes to the estimation task. Improving our understanding in this field would enable further advances into unraveling the physical connections among galaxy properties and optimizing data exploitation. Therefore, our work is aimed at interpreting the deep learning-based estimation of stellar mass via two interpretability techniques: causal analysis and mutual information decomposition. The former reveals the causal paths between multiple variables beyond nondirectional statistical associations, while the latter quantifies the multicomponent contributions (i.e., redundant, unique, and synergistic) of different input data to the stellar mass estimation. Using data from the Sloan Digital Sky Survey (SDSS) and the Wide-field Infrared Survey Explorer (WISE), we obtained meaningful results that provide physical interpretations for image-based models. Our work demonstrates the gains from combining deep learning with interpretability techniques, and holds promise in promoting more data-driven astrophysical research (e.g., astrophysical parameter estimations and investigations on complex multivariate physical processes).

eess.SP [Back]

[403] YOLO-based Bearing Fault Diagnosis With Continuous Wavelet Transform

Po-Heng Chou,Wei-Lung Mao,Ru-Ping Lin

Main category: eess.SP

TL;DR: 本文提出了一种基于YOLO的轴承故障诊断框架，通过连续小波变换（CWT）生成的时频谱图进行分类，显著提高了准确性和泛化能力。

Details Motivation: 传统轴承故障诊断方法在处理瞬态故障特征时存在局限性，且泛化能力较差。本文旨在利用YOLO模型的空间检测能力，结合CWT提供的时频信息，实现更高效的故障诊断。

Contribution: 1. 提出了CWT-YOLO框架，首次将YOLO应用于轴承故障诊断；2. 通过Morlet小波将振动信号转换为时频谱图，捕捉瞬态故障特征；3. 实验证明YOLOv11在多个数据集上达到接近99%的mAP，优于基线方法MCNN-LSTM。

Method: 1. 使用Morlet小波的CWT将一维振动信号转换为时频谱图；2. 采用YOLOv9、v10和v11模型对谱图进行分类和故障区域检测；3. 在CWRU、PU和IMS数据集上进行评估。

Result: YOLOv11在CWRU、PU和IMS数据集上的mAP分别为99.4%、97.8%和99.5%，显著优于MCNN-LSTM基线方法。

Insight: 1. YOLO的空间检测能力结合CWT时频信息，能有效捕捉轴承故障的瞬态特征；2. 通过可视化故障区域，为旋转机械状态监测提供了实用解决方案。

Abstract: This letter proposes a YOLO-based framework for spatial bearing fault diagnosis using time-frequency spectrograms derived from continuous wavelet transform (CWT). One-dimensional vibration signals are first transformed into time-frequency spectrograms using Morlet wavelets to capture transient fault signatures. These spectrograms are then processed by YOLOv9, v10, and v11 models to classify fault types. Evaluated on three benchmark datasets, including Case Western Reserve University (CWRU), Paderborn University (PU), and Intelligent Maintenance System (IMS), the proposed CWT-YOLO pipeline achieves significantly higher accuracy and generalizability than the baseline MCNN-LSTM model. Notably, YOLOv11 reaches mAP scores of 99.4% (CWRU), 97.8% (PU), and 99.5% (IMS). In addition, its region-aware detection mechanism enables direct visualization of fault locations in spectrograms, offering a practical solution for condition monitoring in rotating machinery.

[404] Introducing Multimodal Paradigm for Learning Sleep Staging PSG via General-Purpose Model

Jianheng Zhou,Chenyu Liu,Jinan Zhou,Yi Ding,Yang Liu,Haoran Luo,Ziyu Jia,Xinliang Zhou

Main category: eess.SP

TL;DR: 提出一种新的多模态范式，利用通用多模态大模型从PSG信号生成的二维图像中学习睡眠分期，效果优于现有方法。

Details Motivation: 现有睡眠分期的自动方法依赖复杂信号和领域专用模型，缺乏直观性且需要大量数据。本文旨在通过临床诊断实践启发的方法克服这些限制。

Contribution: 1. 将一维PSG信号转换为二维波形图像；2. 通过多模态通用模型学习睡眠分期；3. 在多个公开数据集上验证方法的有效性。

Method: 1. 将PSG信号转为直观的二维图像；2. 微调多模态大模型学习图像表征；3. 模型模拟临床医生视觉诊断流程。

Result: 在ISRUC、MASS和SHHS数据集上，方法在准确性和鲁棒性上超越现有基准。解释性分析显示模型模拟了人类专家的视觉诊断逻辑。

Insight: 1. 通用模型可通过多模态学习适应新领域；2. 图像化信号能有效保留关键信息；3. 方法无需预训练即可快速适应新任务。

Abstract: Sleep staging is essential for diagnosing sleep disorders and assessing neurological health. Existing automatic methods typically extract features from complex polysomnography (PSG) signals and train domain-specific models, which often lack intuitiveness and require large, specialized datasets. To overcome these limitations, we introduce a new paradigm for sleep staging that leverages large multimodal general-purpose models to emulate clinical diagnostic practices. Specifically, we convert raw one-dimensional PSG time-series into intuitive two-dimensional waveform images and then fine-tune a multimodal large model to learn from these representations. Experiments on three public datasets (ISRUC, MASS, SHHS) demonstrate that our approach enables general-purpose models, without prior exposure to sleep data, to acquire robust staging capabilities. Moreover, explanation analysis reveals our model learned to mimic the visual diagnostic workflow of human experts for sleep staging by PSG images. The proposed method consistently outperforms state-of-the-art baselines in accuracy and robustness, highlighting its efficiency and practical value for medical applications. The code for the signal-to-image pipeline and the PSG image dataset will be released.

cs.NE [Back]

[405] Accuracy-Robustness Trade Off via Spiking Neural Network Gradient Sparsity Trail

Nhan T. Luu

Main category: cs.NE

TL;DR: SNNs在特定架构下表现出自然的梯度稀疏性，无需显式正则化即可达到最佳对抗防御性能，但稀疏梯度在提升鲁棒性的同时会损害模型的泛化能力。

Details Motivation: SNNs因其能效高和内存占用小受到关注，但其对抗鲁棒性仍未被充分探索。

Contribution: 揭示了SNNs在特定配置下具有自然的梯度稀疏性，无需正则化即可实现高性能对抗防御。

Method: 通过分析SNNs的梯度稀疏性及其对鲁棒性和泛化能力的影响。

Result: 展示了SNNs在对抗防御中的优异性能，但揭示了鲁棒性与泛化能力的权衡。

Insight: 梯度稀疏性是SNNs对抗防御的关键，但需要在鲁棒性和泛化能力之间找到平衡。

Abstract: Spiking Neural Networks (SNNs) have attracted growing interest in both computational neuroscience and artificial intelligence, primarily due to their inherent energy efficiency and compact memory footprint. However, achieving adversarial robustness in SNNs, particularly for vision-related tasks, remains a nascent and underexplored challenge. Recent studies have proposed leveraging sparse gradients as a form of regularization to enhance robustness against adversarial perturbations. In this work, we present a surprising finding: under specific architectural configurations, SNNs exhibit natural gradient sparsity and can achieve state-of-the-art adversarial defense performance without the need for any explicit regularization. Further analysis reveals a trade-off between robustness and generalization: while sparse gradients contribute to improved adversarial resilience, they can impair the model’s ability to generalize; conversely, denser gradients support better generalization but increase vulnerability to attacks.

cs.SE [Back]

[406] Metamorphic Testing for Audio Content Moderation Software

Wenxuan Wang,Yongjiang Wu,Junyuan Zhang,Shuqing Li,Yun Peng,Wenting Chen,Shuai Wang,Michael R. Lyu

Main category: cs.SE

TL;DR: 该论文提出了一种名为MTAM的变形测试框架，用于评估音频内容审核软件对对抗性音频的检测能力，结果显示该方法在商业和学术模型中均能发现显著错误。

Details Motivation: 随着音频平台的普及，有害音频内容的传播问题日益严重，而现有的审核工具容易被对抗性修改绕过，因此需要一种更有效的测试方法。

Contribution: 提出了MTAM框架，定义了14种变形关系（Metamorphic Relations），用于生成对抗性测试用例，显著提高了对音频内容审核软件的测试覆盖率。

Method: 基于音频特征和启发式的两类扰动，MTAM生成对抗性音频内容，并通过测试商业和学术审核模型验证其有效性。

Result: MTAM在测试五种商业软件和一种学术模型时，错误发现率（EFR）最高达51.1%和45.7%，证明了其有效性。

Insight: 对抗性扰动（如修改音高或插入噪声）可绕过现有审核工具，表明当前音频审核技术仍存在明显漏洞。

Abstract: The rapid growth of audio-centric platforms and applications such as WhatsApp and Twitter has transformed the way people communicate and share audio content in modern society. However, these platforms are increasingly misused to disseminate harmful audio content, such as hate speech, deceptive advertisements, and explicit material, which can have significant negative consequences (e.g., detrimental effects on mental health). In response, researchers and practitioners have been actively developing and deploying audio content moderation tools to tackle this issue. Despite these efforts, malicious actors can bypass moderation systems by making subtle alterations to audio content, such as modifying pitch or inserting noise. Moreover, the effectiveness of modern audio moderation tools against such adversarial inputs remains insufficiently studied. To address these challenges, we propose MTAM, a Metamorphic Testing framework for Audio content Moderation software. Specifically, we conduct a pilot study on 2000 audio clips and define 14 metamorphic relations across two perturbation categories: Audio Features-Based and Heuristic perturbations. MTAM applies these metamorphic relations to toxic audio content to generate test cases that remain harmful while being more likely to evade detection. In our evaluation, we employ MTAM to test five commercial textual content moderation software and an academic model against three kinds of toxic content. The results show that MTAM achieves up to 38.6%, 18.3%, 35.1%, 16.7%, and 51.1% error finding rates (EFR) when testing commercial moderation software provided by Gladia, Assembly AI, Baidu, Nextdata, and Tencent, respectively, and it obtains up to 45.7% EFR when testing the state-of-the-art algorithms from the academy.

cs.AI [Back]

[407] Benefits and Pitfalls of Reinforcement Learning for Language Model Planning: A Theoretical Perspective

Siwei Wang,Yifei Shen,Haoran Sun,Shi Feng,Shang-Hua Teng,Li Dong,Yaru Hao,Wei Chen

Main category: cs.AI

TL;DR: 这篇论文从理论角度分析了强化学习（RL）在语言模型规划中的优势和局限，揭示了探索的重要性以及不同RL方法的差异。

Details Motivation: 尽管强化学习显著提升了大型语言模型（LLMs）的规划能力，但其理论依据尚不明确。作者旨在通过理论分析揭示RL的优势和潜在问题。

Contribution: 论文的主要贡献包括：1）揭示了监督微调（SFT）可能引入虚假解，而RL通过探索实现正确规划；2）分析了策略梯度（PG）的多样性崩溃问题与Q学习的优势；3）提出了防止Q学习中奖励作弊的奖励设计方法。

Method: 作者采用了一种基于图的抽象分析方法，专注于策略梯度（PG）和Q学习方法，并通过理论推导和实验验证其结论。

Result: 在Blocksworld基准测试中，论文验证了理论分析的结果，展示了PG的多样性崩溃和Q学习的优势。

Insight: 探索是RL有效规划的关键，而Q学习在保持多样性和离策略学习方面优于PG；此外，奖励设计对RL方法的成功至关重要。

Abstract: Recent reinforcement learning (RL) methods have substantially enhanced the planning capabilities of Large Language Models (LLMs), yet the theoretical basis for their effectiveness remains elusive. In this work, we investigate RL’s benefits and limitations through a tractable graph-based abstraction, focusing on policy gradient (PG) and Q-learning methods. Our theoretical analyses reveal that supervised fine-tuning (SFT) may introduce co-occurrence-based spurious solutions, whereas RL achieves correct planning primarily through exploration, underscoring exploration’s role in enabling better generalization. However, we also show that PG suffers from diversity collapse, where output diversity decreases during training and persists even after perfect accuracy is attained. By contrast, Q-learning provides two key advantages: off-policy learning and diversity preservation at convergence. We further demonstrate that careful reward design is necessary to prevent reward hacking in Q-learning. Finally, applying our framework to the real-world planning benchmark Blocksworld, we confirm that these behaviors manifest in practice.

[408] Kimi-Dev: Agentless Training as Skill Prior for SWE-Agents

Zonghan Yang,Shengjie Wang,Kelin Fu,Wenyang He,Weimin Xiong,Yibo Liu,Yibo Miao,Bofei Gao,Yejie Wang,Yingwei Ma,Yanhao Li,Yue Liu,Zhenxing Hu,Kaitai Zhang,Shuyi Wang,Huarong Chen,Flood Sung,Yang Liu,Yang Gao,Zhilin Yang,Tianyu Liu

Main category: cs.AI

TL;DR: Kimi-Dev通过Agentless训练生成技能先验，结合SWE-Agent框架，实现了高效的软件工程任务解决，性能与Claude 3.5 Sonnet相当。

Details Motivation: 当前LLM在软件工程中的解决方案分为多轮交互的SWE-Agent和无代理的单步验证方法，作者认为两者可结合，通过Agentless训练生成的技能先验提升SWE-Agent的效率和效果。

Contribution: 提出Kimi-Dev开源工具，首次展示Agentless训练生成的技能先验（如定位、代码编辑和自我反思）可高效适配SWE-Agent，性能达到SWE-bench Verified的60.4%。

Method: 通过Agentless训练生成技能先验，并利用5k公开轨迹进行SFT适配，将Kimi-Dev应用于SWE-Agent框架。

Result: Kimi-Dev在SWE-bench Verified上达到60.4%的成绩，适配SWE-Agent后实现48.6%的pass@1，与Claude 3.5 Sonnet相当。

Insight: 结构化技能先验可弥合工作流和代理框架的鸿沟，为可迁移的编码代理提供新思路。

Abstract: Large Language Models (LLMs) are increasingly applied to software engineering (SWE), with SWE-bench as a key benchmark. Solutions are split into SWE-Agent frameworks with multi-turn interactions and workflow-based Agentless methods with single-turn verifiable steps. We argue these paradigms are not mutually exclusive: reasoning-intensive Agentless training induces skill priors, including localization, code edit, and self-reflection that enable efficient and effective SWE-Agent adaptation. In this work, we first curate the Agentless training recipe and present Kimi-Dev, an open-source SWE LLM achieving 60.4% on SWE-bench Verified, the best among workflow approaches. With additional SFT adaptation on 5k publicly-available trajectories, Kimi-Dev powers SWE-Agents to 48.6% pass@1, on par with that of Claude 3.5 Sonnet (241022 version). These results show that structured skill priors from Agentless training can bridge workflow and agentic frameworks for transferable coding agents.

[409] Multiplayer Nash Preference Optimization

Fang Wu,Xu Huang,Weihao Xuan,Zhiwei Zhang,Yijia Xiao,Guancheng Wan,Xiaomin Li,Bing Hu,Peng Xia,Jure Leskovec,Yejin Choi

Main category: cs.AI

TL;DR: 论文提出了多玩家纳什偏好优化（MNPO），将纳什学习从博弈扩展到多玩家场景，以更好地捕捉现实世界中非传递性和多样化的偏好结构。

Details Motivation: 现有的基于Bradley-Terry假设的奖励方法无法捕捉复杂的偏好结构，而两玩家纳什学习（NLHF）方法虽然有效，但仍局限于单一对手的偏见。

Contribution: MNPO将NLHF扩展到多玩家场景，定义了多玩家纳什均衡，并引入了对偶间隙来衡量近似质量。

Method: MNPO将对齐问题建模为n玩家博弈，每个策略与对手群体竞争，同时向参考模型正则化。

Result: 实验显示MNPO在指令跟随基准测试中优于现有NLHF基线，尤其在异构标注条件和混合策略评估场景下表现更优。

Insight: MNPO不仅继承了双玩家方法的理论保证，还通过多玩家竞争动态丰富了偏好结构的覆盖能力。

Abstract: Reinforcement learning from human feedback (RLHF) has emerged as the standard paradigm for aligning large language models (LLMs) with human preferences. However, reward-based methods built on the Bradley-Terry assumption struggle to capture the non-transitive and heterogeneous nature of real-world preferences. To address this, recent studies have reframed alignment as a two-player Nash game, giving rise to Nash learning from human feedback (NLHF). While this perspective has inspired algorithms such as INPO, ONPO, and EGPO with strong theoretical and empirical guarantees, they remain fundamentally restricted to two-player interactions, creating a single-opponent bias that fails to capture the full complexity of realistic preference structures. In this work, we introduce Multiplayer Nash Preference Optimization (MNPO), a novel framework that generalizes NLHF to the multiplayer regime. It formulates alignment as an $n$-player game, where each policy competes against a population of opponents while being regularized toward a reference model. Our framework establishes well-defined Nash equilibria in multiplayer settings and extends the concept of duality gap to quantify approximation quality. We demonstrate that MNPO inherits the equilibrium guarantees of two-player methods while enabling richer competitive dynamics and improved coverage of diverse preference structures. Through comprehensive empirical evaluation, we show that MNPO consistently outperforms existing NLHF baselines on instruction-following benchmarks, achieving superior alignment quality under heterogeneous annotator conditions and mixed-policy evaluation scenarios. Together, these results establish MNPO as a principled and scalable framework for aligning LLMs with complex, non-transitive human preferences. Code is available at https://github.com/smiles724/MNPO.

[410] $p$-less Sampling: A Robust Hyperparameter-Free Approach for LLM Decoding

Runyan Tan,Shuang Wu,Phillip Howard

Main category: cs.AI

TL;DR: 该论文提出了一种新的无超参数采样方法$p$-less sampling，用于大语言模型（LLM）的解码过程。该方法基于信息论动态设置截断阈值，避免了现有方法对超参数的依赖，并在高温度下仍能保持高质量的生成结果。通过实验验证，$p$-less sampling在数学、逻辑推理和创意写作任务中表现优于现有方法。

Details Motivation: 当前基于采样的解码方法对超参数的选择敏感，且性能会因任务和温度配置的不同而变化。因此，需要一种稳健的无超参数方法，能在不同条件下保持高质量的生成结果。

Contribution: 1. 提出了一种无超参数的信息论采样方法$p$-less sampling；2. 动态设置截断阈值，减少对超参数的依赖；3. 在高温度下仍能保持高质量生成；4. 提升了推理效率。

Method: $p$-less sampling基于信息论，在每个解码步骤中根据整个概率分布动态设置截断阈值。该方法无需超参数，且适用于不同任务和温度配置。

Result: 实验表明，$p$-less sampling在数学、逻辑推理和创意写作任务中均优于现有采样方法，且在高温度下生成质量下降较少。此外，该方法还提高了推理效率。

Insight: 1. 信息论方法可以有效替代超参数调优；2. 动态阈值设置可以提升生成质量和效率；3. 无超参数方法在实际应用中更具普适性。

Abstract: Obtaining high-quality outputs from Large Language Models (LLMs) often depends upon the choice of a sampling-based decoding strategy to probabilistically choose the next token at each generation step. While a variety of such sampling methods have been proposed, their performance can be sensitive to the selection of hyperparameters which may require different settings depending upon the generation task and temperature configuration. In this work, we introduce $p$-less sampling: an information-theoretic approach to sampling which dynamically sets a truncation threshold at each decoding step based on the entire token probability distribution. Unlike existing methods, $p$-less sampling has no hyperparameters and consistently produces high-quality outputs as temperature increases. We provide theoretical perspectives on $p$-less sampling to ground our proposed method and conduct experiments to empirically validate its effectiveness across a range of math, logical reasoning, and creative writing tasks. Our results demonstrate how $p$-less sampling consistently outperforms existing sampling approaches while exhibiting much less degradation in text quality at higher temperature values. We further show how $p$-less achieves greater inference-time efficiency than alternative methods through lower average token sampling times and shorter generation lengths, without sacrificing accuracy. Finally, we provide analyses to highlight the benefits of $p$-less through qualitative examples, case studies, and diversity assessments.

[411] Learning How to Use Tools, Not Just When: Pattern-Aware Tool-Integrated Reasoning

Ningning Xu,Yuxuan Jiang,Shubhashis Roy Dipta

Main category: cs.AI

TL;DR: 这篇论文提出了一个模式感知的工具集成推理框架，通过区分计算器模式和算法模式，并统一模式选择与教师偏好，显著提升了代码使用率和准确性。

Details Motivation: 现有的工具集成推理方法主要关注何时调用工具，而忽略了如何正确应用工具。作者发现工具应用的两种常见模式（计算器模式和算法模式）选择不当可能导致推理失败，即使逻辑本身是正确的。

Contribution: 提出了一个两阶段的模式感知框架，首先从两种模式中构建代码能力，然后将模式选择与教师偏好对齐，显著提升了工具集成推理的效果。

Method: 1. 识别工具应用的两种常见模式：计算器模式和算法模式。2. 设计两阶段框架：第一阶段训练模型掌握两种模式的代码能力，第二阶段将模式选择与教师偏好对齐。

Result: 在多个数学数据集（如MATH500和AIME24）上，该方法显著提升了代码使用率和准确性（例如，MATH500上的Code@1从64.0%提升到70.5%，AIME24从26.7%提升到50.0%）。

Insight: 工具的正确应用模式选择对推理性能至关重要，模式感知的方法能够有效提升工具集成推理的效果。

Abstract: Tool-integrated reasoning (TIR) has become a key approach for improving large reasoning models (LRMs) on complex problems. Prior work has mainly studied when to invoke tools, while overlooking how tools are applied. We identify two common patterns: a calculator pattern that uses code for direct computation, and an algorithmic pattern that encodes problems as programs. Misaligned choices often cause failures even when reasoning is sound. We propose a two-stage framework that first builds code competence from both patterns and then aligns pattern selection with teacher preferences. Across challenging math datasets, our pattern-aware method substantially improves both code usage and accuracy, for instance raising Code@1 on MATH500 from 64.0% to 70.5% and on AIME24 from 26.7% to 50.0%. These gains highlight the effectiveness of a pattern-aware approach for tool-integrated reasoning.

[412] Your Models Have Thought Enough: Training Large Reasoning Models to Stop Overthinking

Jinyi Han,Ying Huang,Ying Liao,Zishang Jiang,Xikun Lu,Haiquan Zhao,Xinyi Wang,Guanghao Zhou,Sihang Jiang,Jiaqing Liang,Weikang Zhou,Zeye Sun,Fei Yu,Yanghua Xiao

Main category: cs.AI

TL;DR: 论文提出了一种名为JET的方法，通过训练大规模推理模型（LRMs）主动终止不必要的推理步骤，从而在保持精度的同时显著提高推理效率。

Details Motivation: 尽管大规模推理模型在复杂任务上表现出色，但其深层推理通常伴随高昂计算成本。现有强化学习方法在构建短推理路径时仍存在困难。

Contribution: 提出Just-Enough Thinking（JET）方法，通过轨迹截断和质量控制长度奖励，实现高效推理而不牺牲精度。

Method: JET在推理阶段截断轨迹以暴露模型于短的、分布一致的推理路径，并引入质量控制的长度奖励鼓励简洁推理。

Result: 实验表明，JET显著提升效率且不降低精度，如在Olympiad基准上，DeepSeek-Distill-Qwen-1.5B模型输出长度减少46.3%的同时准确率提升4.6%。

Insight: LRM在推理早期已积累足够信息，后续步骤冗余；通过主动终止推理可兼顾效率与效果。

Abstract: Large Reasoning Models (LRMs) have achieved impressive performance on challenging tasks, yet their deep reasoning often incurs substantial computational costs. To achieve efficient reasoning, existing reinforcement learning methods still struggle to construct short reasoning path during the rollout stage, limiting effective learning. Inspired by Evidence Accumulation Models, we find that LRMs have accumulated sufficient information early in reasoning, making further reasoning steps redundant. Based on this insight, we propose Just-Enough Thinking (JET), which trains models to proactively terminate unnecessary reasoning. JET performs trajectory truncation during rollout to expose the model to short, distributionally consistent reasoning paths. Besides, it uses a quality-controlled length reward to better encourage concise reasoning while maintaining correctness. Extensive experiments demonstrate that JET significantly improves reasoning efficiency without sacrificing accuracy. Especially, DeepSeek-Distill-Qwen-1.5B achieves a 4.6% accuracy gain while reducing output length by 46.3% on the Olympiad benchmark. Our code is available in the GitHub.

[413] Mapping Overlaps in Benchmarks through Perplexity in the Wild

Siyang Wu,Honglin Bao,Sida Li,Ari Holtzman,James A. Evans

Main category: cs.AI

TL;DR: 该论文通过困惑度分析大型语言模型（LLM）基准测试的共性与重叠，提出了一种基于自然语料的基准签名方法，揭示了不同任务间的能力关联与差异。

Details Motivation: 现有LLM基准测试中，任务间的重叠性和有效性缺乏系统性分析。作者希望通过困惑度揭示LLM在不同任务中的能力分布及其潜在联系。

Contribution: 1.提出基准签名方法，通过困惑度量化LLM能力覆盖；2.大规模评估揭示了知识、推理等任务的共性，以及编程任务的独特性；3.发现性能重叠与语义重叠的分离现象。

Method: 1.从自然语料中提取显著词元；2.利用线性回归和逐步选择方法分析32个LLM在88个基准上的表现；3.通过困惑度和性能相关性定义基准签名。

Result: 知识、推理任务重叠度高，多语言和文化任务差异显著；编程任务与其他任务重叠最少；基准签名对性能无关因素稳健。

Insight: 困惑度能有效揭示LLM能力的底层关联，帮助理解基准的有效性及LLM能力的真实分布。

Abstract: We develop signatures of capacity familiarity to characterize large language model (LLM) benchmarks and their meaningful overlaps. Benchmark signatures probe the capacity required for benchmark performance. We formally define them as a set of salient tokens drawn from in-the-wild, naturally authored corpora, where LLM token perplexity, reflecting more or less pre-training exposure, becomes highly predictive of LLM benchmark performance. Through a large-scale meta-evaluation, we extract benchmark signatures via stepwise forward selection with linear regressions across 32 LLMs and 88 benchmarks spanning diverse knowledge, coding, logic, instruction following, math, language, reasoning, and world modeling. Our analysis situates signatures in relation to both the semantic similarity of benchmark questions and the correlation of model performance. While performance overlaps are universally high and semantic overlaps remain confined to a narrow mid-range, benchmark signatures prove highly informative in capturing variation, overlap, and divergence. We observe overlap in knowledge and reasoning subtasks, whereas multilingual and cultural benchmarks exhibit less similarity, even compared to cross-task overlap. Notably, performance-level results are strongly influenced by benchmark-orthogonal factors such as question format, highlighting limitations in LLM generalization, the conflation of performance with ability, and issues inherent in current mainstream benchmark agreement studies. Benchmark signatures, however, remain robust to such effects. Ultimately, we identify cross-functional overlaps across logic, math, language, instruction following, and world modeling, with coding emerging as the least overlapping domain. Together, these findings provide mechanistic insights into benchmark validity and LLM sensitivities, and sketch the underlying landscape of interconnected LLM capabilities.

[414] From Reasoning to Answer: Empirical, Attention-Based and Mechanistic Insights into Distilled DeepSeek R1 Models

Jue Zhang,Qingwei Lin,Saravan Rajmohan,Dongmei Zhang

Main category: cs.AI

TL;DR: 这篇论文通过实证评估、注意力分析和机制干预，研究了蒸馏DeepSeek R1模型中推理过程对答案生成的影响，揭示了推理标记对答案生成的定向功能流。

Details Motivation: 大型推理模型（LRMs）在生成答案时会同时生成显式的推理轨迹，但这些推理轨迹对答案生成的具体影响尚不明确。论文旨在探究推理与答案生成之间的关系。

Contribution: 1）实证表明显式推理能提高答案质量；2）注意力分析发现答案标记显著关注推理标记；3）通过机制干预验证了推理标记对答案生成的直接影响。

Method: 采用三阶段研究方法：1）实证评估；2）注意力分析（识别Reasoning-Focus Heads）；3）激活修补（activation patching）的机制干预。

Result: 研究发现关键推理标记的扰动会可靠地改变最终答案，证实了从推理到答案的信息流是定向且功能性的。

Insight: 中间推理过程在模型输出中具有功能性作用，揭示了LRMs如何利用推理标记改善答案生成。

Abstract: Large Reasoning Models (LRMs) generate explicit reasoning traces alongside final answers, yet the extent to which these traces influence answer generation remains unclear. In this work, we conduct a three-stage investigation into the interplay between reasoning and answer generation in three distilled DeepSeek R1 models. First, through empirical evaluation, we demonstrate that including explicit reasoning consistently improves answer quality across diverse domains. Second, attention analysis reveals that answer tokens attend substantially to reasoning tokens, with certain mid-layer Reasoning-Focus Heads (RFHs) closely tracking the reasoning trajectory, including self-reflective cues. Third, we apply mechanistic interventions using activation patching to assess the dependence of answer tokens on reasoning activations. Our results show that perturbations to key reasoning tokens can reliably alter the final answers, confirming a directional and functional flow of information from reasoning to answer. These findings deepen our understanding of how LRMs leverage reasoning tokens for answer generation, highlighting the functional role of intermediate reasoning in shaping model outputs. Our data and code are publicly available at \href{https://aka.ms/R2A-code}{this URL}.

[415] SafeSearch: Automated Red-Teaming for the Safety of LLM-Based Search Agents

Jianshuo Dong,Sheng Guo,Hao Wang,Zhuotao Liu,Tianwei Zhang,Ke Xu,Minlie Huang,Han Qiu

Main category: cs.AI

TL;DR: 这篇论文提出了一个名为SafeSearch的自动红队框架，用于评估基于LLM的搜索代理的安全性，揭示了其在不可靠搜索结果下的脆弱性，并展示了常用防御措施的局限性。

Details Motivation: LLM搜索代理接入互联网虽然扩展了信息来源，但也引入了安全隐患，如搜索结果不可靠可能导致误导行为。研究旨在系统性评估和提升搜索代理的安全性。

Contribution: 1) 提出自动红队框架SafeSearch，系统化、可扩展且低成本；2) 构建包含300个测试用例的SafeSearch基准，覆盖五类风险；3) 评估多种搜索代理和LLM的脆弱性。

Method: 通过两项真实实验验证搜索结果的不可靠性及其影响，开发SafeSearch框架，包括基准测试和评估方法。

Result: 实验显示，LLM搜索代理在不可靠网站下的最高攻击成功率（ASR）达90.5%，且常见防御措施效果有限。

Insight: 突出了当前LLM搜索代理的安全挑战，强调了系统性评估的必要性，为未来安全开发提供了工具和方向。

Abstract: Search agents connect LLMs to the Internet, enabling access to broader and more up-to-date information. However, unreliable search results may also pose safety threats to end users, establishing a new threat surface. In this work, we conduct two in-the-wild experiments to demonstrate both the prevalence of low-quality search results and their potential to misguide agent behaviors. To counter this threat, we introduce an automated red-teaming framework that is systematic, scalable, and cost-efficient, enabling lightweight and harmless safety assessments of search agents. Building on this framework, we construct the SafeSearch benchmark, which includes 300 test cases covering five categories of risks (e.g., misinformation and indirect prompt injection). Using this benchmark, we evaluate three representative search agent scaffolds, covering search workflow, tool-calling, and deep research, across 7 proprietary and 8 open-source backend LLMs. Our results reveal substantial vulnerabilities of LLM-based search agents: when exposed to unreliable websites, the highest ASR reached 90.5% for GPT-4.1-mini under a search workflow setting. Moreover, our analysis highlights the limited effectiveness of common defense practices, such as reminder prompting. This emphasizes the value of our framework in promoting transparency for safer agent development. Our codebase and test cases are publicly available: https://github.com/jianshuod/SafeSearch.

[416] From What to Why: A Multi-Agent System for Evidence-based Chemical Reaction Condition Reasoning

Cheng Yang,Jiaxuan Lu,Haiyuan Wan,Junchi Yu,Feiwei Qin

Main category: cs.AI

TL;DR: ChemMAS是一个多智能体系统，将化学反应条件预测任务转化为基于证据的推理过程，提供了可解释的依据。

Details Motivation: 现有的大型语言模型（LLMs）在化学反应条件推荐中表现良好，但缺乏解释推荐依据的能力，限制了其在科学工作中的应用。

Contribution: 提出了ChemMAS，将条件预测任务分解为多个可解释的子任务，并提供基于化学知识和检索先例的理性依据。

Method: 采用多智能体系统，分解任务为机制推理、多通道检索、约束感知的代理辩论和理性聚合。

Result: ChemMAS在Top-1准确率上优于领域基线20-35%，并领先通用LLMs 10-15%，同时提供可验证的解释。

Insight: 通过多智能体系统分解任务并提供证据支持，可以显著提升模型的解释性和性能，适用于高风险的科学研究场景。

Abstract: The chemical reaction recommendation is to select proper reaction condition parameters for chemical reactions, which is pivotal to accelerating chemical science. With the rapid development of large language models (LLMs), there is growing interest in leveraging their reasoning and planning capabilities for reaction condition recommendation. Despite their success, existing methods rarely explain the rationale behind the recommended reaction conditions, limiting their utility in high-stakes scientific workflows. In this work, we propose ChemMAS, a multi-agent system that reframes condition prediction as an evidence-based reasoning task. ChemMAS decomposes the task into mechanistic grounding, multi-channel recall, constraint-aware agentic debate, and rationale aggregation. Each decision is backed by interpretable justifications grounded in chemical knowledge and retrieved precedents. Experiments show that ChemMAS achieves 20-35% gains over domain-specific baselines and outperforms general-purpose LLMs by 10-15% in Top-1 accuracy, while offering falsifiable, human-trustable rationales, which establishes a new paradigm for explainable AI in scientific discovery.

[417] Conditional Advantage Estimation for Reinforcement Learning in Large Reasoning Models

Guanxu Chen,Yafu Li,Yuxian Jiang,Chen Qian,Qihan Ren,Jingyi Yang,Yu Cheng,Dongrui Liu,Jing Shao

Main category: cs.AI

TL;DR: CANON是一种新的强化学习优势估计方法，通过无方向性先验来优化大型语言模型的推理能力，在数学推理和高复杂度逻辑任务中表现优于现有方法。

Details Motivation: 现有基于RLVR的方法在优化LLM推理能力时依赖于方向性先验（如奖励或优势调整），但缺乏超参数调优容易导致偏差和失败。CANON旨在消除这种方向性假设，更灵活地利用目标指标。

Contribution: 提出CANON方法，通过分组比较和组内优势估计，放大目标指标的影响，无需预设其方向。该方法在数学推理和高复杂度逻辑任务中显著优于现有方法。

Method: CANON将采样响应按目标指标（如熵或响应长度）分组（高值和低值），通过组间比较确定趋势贡献，并在组内识别最优响应，动态调整优势估计。

Result: 在三种LLM上，基于熵的CANON在数学推理和高复杂度逻辑任务中表现最佳；应用于响应长度时，CANON进一步提高了token效率，优化了性能-成本的帕累托边界。

Insight: 移除方向性先验可以更灵活地利用目标指标，避免人为偏差；分组比较和动态优势估计是优化RLVR性能的有效策略。

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) for large language models (LLMs) has achieved remarkable progress in enhancing LLMs’ reasoning capabilities on tasks with clear correctness criteria, such as mathematical reasoning tasks. Several training metrics, such as entropy or response length, have been observed to correlate with different reasoning behaviors in reinforcement learning. Prior approaches incorporate such priors through reward or advantage shaping, which often relies on hand-crafted penalties and preferences (e.g., higher-is-better or lower-is-better). However, without careful hyperparameter tuning, these directional priors can be overly biased and may lead to failure. To this end, we introduce Conditional advANtage estimatiON (CANON), amplifying the impact of the target metric without presuming its direction. Specifically, CANON regroups the sampled responses into two groups based on the higher or lower value of a target metric, measures which metric trend contributes to better performance through inter-group comparison, and identifies the better response within the same group. In summary, CANON based on entropy consistently outperforms prior methods across three LLMs on both math reasoning and high-complexity logic tasks. When applied to response length, CANON further improves token efficiency, yielding a more favorable Pareto frontier in the performance-cost trade-off.

[418] Reasoning or Retrieval? A Study of Answer Attribution on Large Reasoning Models

Yuhui Wang,Changjiang Li,Guangke Chen,Jiacheng Liang,Ting Wang

Main category: cs.AI

TL;DR: 该论文研究了大型推理模型（LRMs）在回答问题时，是通过真正的推理（Chain-of-Thought, CoT）还是记忆检索机制生成答案，并通过实验验证了两种机制的同时存在及其影响因素。研究发现现有模型的训练范式存在漏洞，并提出了一种新框架FARL，以抑制记忆检索的捷径，提升模型的真实推理能力。

Details Motivation: 大型推理模型在解决问题时展现出强大的推理能力，但其答案与推理过程不一致的现象引发了研究者的关注。论文通过实验探究了这种不一致的根源，即模型可能同时依赖推理和记忆检索两种机制。

Contribution: 1.验证了大型推理模型中推理和记忆检索两种机制并存的现象；2.揭示了模型的训练范式（如强化学习）可能助长记忆检索的捷径；3.提出了FARL框架，通过抑制检索机制提升模型的真实推理能力。

Method: 论文设计了控制实验，通过在推理过程中引入误导性线索或在检索阶段提供错误答案，验证了两种机制的影响因素。基于发现，提出了FARL框架，结合记忆遗忘和强化学习，抑制检索捷径。

Result: 实验证实推理和检索机制在不同领域、模型规模和训练方法下表现不同。FARL框架显著提升了模型的推理主导行为，增强了泛化推理能力。

Insight: 1.当前模型的推理能力可能被高估，部分答案依赖记忆检索；2.训练方法的缺陷可能导致模型避开真实推理；3.FARL为改进模型的推理能力提供了新方向。

Abstract: Large reasoning models (LRMs) exhibit unprecedented capabilities in solving complex problems through Chain-of-Thought (CoT) reasoning. However, recent studies reveal that their final answers often contradict their own reasoning traces. We hypothesize that this inconsistency stems from two competing mechanisms for generating answers: CoT reasoning and memory retrieval. To test this hypothesis, we conduct controlled experiments that challenge LRMs with misleading cues during reasoning and/or corrupted answers during retrieval. Our results across models and datasets confirm that both mechanisms operate simultaneously, with their relative dominance influenced by multiple factors: problem domains, model scales, and fine-tuning approaches (e.g., reinforcement learning vs. distillation). The findings reveal a critical limitation in current reasoning fine-tuning paradigms: models can exploit the retrieval mechanism as a shortcut, effectively “hacking” the reward signal and undermining genuine reasoning development. To address this challenge, we introduce FARL, a novel fine-tuning framework that integrates memory unlearning with reinforcement learning. By carefully suppressing retrieval shortcuts during the fine-tuning process, FARL promotes reasoning-dominant behavior and enhances generalizable reasoning capabilities.

[419] Learning to Ponder: Adaptive Reasoning in Latent Space

Yixin He,Lumingyuan Tang

Main category: cs.AI

TL;DR: FR-Ponder是一个无需修改主干模型权重、通过潜在空间自适应分配计算资源的推理框架，利用小于1M参数的控制器动态调整推理深度，提高计算效率与任务准确率。

Details Motivation: 现有方法（如Best-of-N）对复杂和简单查询使用统一计算深度，浪费资源或推理不足。FR-Ponder旨在实现自适应计算分配以提高效率与准确性。

Contribution: 1. 提出FR-Ponder框架，动态调整推理深度；2. 潜在空间导航机制（latent steering）；3. 采用GRPO优化计算与性能平衡。

Method: 1. 控制器观察隐藏状态，通过添加预计算导航向量决定终止或继续推理；2. GRPO优化计算分配；3. 课程学习与奖励调优。

Result: 在GSM8K和MATH500上优于基线，降低FLOPs的同时保持或提升准确率。

Insight: 1. 潜在导航向量与问题复杂性相关；2. 自适应计算分配可通过轻量控制器实现；3. GRPO有效平衡计算成本与性能。

Abstract: Test-time compute has emerged as a key paradigm for enhancing LLM reasoning, yet prevailing approaches like Best-of-N and majority voting apply uniform depth across inputs, wasting computation on simple queries while potentially under-thinking complex ones. We present FR-Ponder, a single-graph, backbone-training-free framework that allocates instance-adaptive reasoning compute via latent steering. A less than 1M-param controller observes hidden states and decides to halt or apply a small ponder step by adding a pre-computed steering vector to frozen representations. Our method extracts the latent steering vector associated with deeper reasoning outputs and direct IO from LLM and re-applies it through a tunable scaling factor, allowing the model to adapt its reasoning depth to the complexity of each input. To balance performance and computational cost, we employ Group Relative Policy Optimization (GRPO) as a reward signal to adaptively regulate reasoning depth, achieving task accuracy while mitigating overreasoning. Through curriculum learning and careful reward engineering, FR-Ponder learns calibrated compute allocation correlated with problem difficulty. On GSM8K and MATH500, FR-Ponder improves the compute-accuracy frontier, delivering lower FLOPs with better matched accuracy and comparing favorably to early-exit baselines, without modifying backbone weights. Analyses visualize interpretable steering directions and show learned compute allocation correlates with problem difficulty.

[420] SpecExit: Accelerating Large Reasoning Model via Speculative Exit

Rubing Yang,Huajun Bai,Song Liu,Guanghua Yu,Runzhi Fan,Yanbin Dang,Jiejing Zhang,Kai Liu,Jianchen Zhu,Peng Chen

Main category: cs.AI

TL;DR: SpecExit是一种新颖的框架，通过直接从轻量级草稿模型中预测未来令牌和提前退出信号，解决了大型推理模型（LRMs）的过思考问题，显著减少了生成长度和端到端延迟。

Details Motivation: 大型推理模型在完成任务时经常因为过思考而产生不必要的长输出，增加端到端延迟，影响了实际部署的效率。现有的提前退出机制虽然有效，但依赖探测机制，引入了额外的开销，限制了速度提升和通用性。

Contribution: 提出了SpecExit框架，利用轻量级草稿模型的隐藏状态预测未来令牌和提前退出信号，避免了探测开销，显著提高了效率。

Method: 通过草稿模型的隐藏状态直接预测未来令牌和提前退出信号，避免了传统探测机制的额外开销，提升了推理效率。

Result: 平均生成长度减少了66%，端到端延迟提升了2.5倍，同时保持了准确性。

Insight: 隐藏状态不仅可用于推测解码，还能高效地提供提前退出信号，展示了隐藏状态在高效推理中的广泛应用潜力。

Abstract: Despite their strong performance on reasoning tasks, large reasoning models (LRMs) often suffer from overthinking, producing unnecessarily long outputs and incurring high end-to-end latency, a significant limitation to their real-world deployment. To address overthinking, early-exit mechanisms have been proposed to terminate reasoning before typical completion, showing that this approach can effectively shorten generation length with minimal impact on accuracy. However, their reliance on probing mechanisms introduces a detection overhead that limits their end-to-end latency gains and compromises their generalizability across diverse problems. Inspired by the use of hidden states in speculative decoding, we propose SpecExit, a novel framework that predicts both future tokens and an early-exit signal directly from a lightweight draft model without probing overhead. Our method offers significant improvements, reducing average generation length by 66% and achieving a 2.5x speedup in end-to-end latency compared to the speculative decoding baseline, without compromising accuracy. Our method leverages the inherent signals from hidden states to provide effective early-exit signals, suggesting broader use of hidden states for efficient reasoning. Our code is available at https://github.com/Tencent/AngelSlim.

[421] AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety Alignment of Large Reasoning Models

Zihao Zhu,Xinyu Wu,Gehan Hu,Siwei Lyu,Ke Xu,Baoyuan Wu

Main category: cs.AI

TL;DR: AdvChain是一个针对大型推理模型的安全对齐范式，通过对抗性CoT调整，解决了CoT推理中安全挑战的滚雪球效应。

Details Motivation: CoT推理的多步骤特性带来了新的安全挑战，传统对齐方法无法解决滚雪球效应，导致有害顺从或过度拒绝。

Contribution: 提出AdvChain，通过构建包含诱惑-纠正和犹豫-纠正样本的数据集，使模型学会动态自校正。

Method: 使用对抗性CoT调整方法，训练模型从有害推理偏差和不必要谨慎中恢复。

Result: AdvChain显著提升了对抗越狱攻击和CoT劫持的鲁棒性，同时减少对良性提示的过度拒绝。

Insight: 通过自校正能力实现安全性与实用性的平衡，为构建更可靠的推理模型提供了新方向。

Abstract: Large Reasoning Models (LRMs) have demonstrated remarkable capabilities in complex problem-solving through Chain-of-Thought (CoT) reasoning. However, the multi-step nature of CoT introduces new safety challenges that extend beyond conventional language model alignment. We identify a failure mode in current safety CoT tuning methods: the \textit{snowball effect}, where minor reasoning deviations progressively amplify throughout the thought process, leading to either harmful compliance or excessive refusal. This effect stems from models being trained to imitate perfect reasoning scripts without learning to self-correct. To address this limitation, we propose AdvChain, an alignment paradigm that teaches models dynamic self-correction through adversarial CoT tuning. Our method involves constructing a dataset containing Temptation-Correction and Hesitation-Correction samples, where models learn to recover from harmful reasoning drifts and unnecessary cautions. Extensive experiments show that AdvChain significantly enhances robustness against jailbreak attacks and CoT hijacking while substantially reducing over-refusal on benign prompts, achieving a superior safety-utility balance without compromising reasoning capabilities. Our work establishes a new direction for building more robust and reliable reasoning models.

[422] SCI-Verifier: Scientific Verifier with Thinking

Shenghe Zheng,Chenyu Huang,Fangchen Yu,Junchi Yao,Jingqi Ye,Tao Chen,Yun Luo,Ning Ding,LEI BAI,Ganqu Cui,Peng Ye

Main category: cs.AI

TL;DR: 本文提出SCI-Verifier和SCI-VerifyBench，分别从数据和模型层面解决科学领域答案验证的挑战，强调推理能力的重要性。

Details Motivation: 大语言模型在科学推理中的应用日益广泛，但答案格式复杂且等价表达多样，使得答案验证成为关键但困难的任务。现有验证方法存在系统性评估标准缺失和对繁琐规则的依赖等问题。

Contribution: 1. 构建跨学科的SCI-VerifyBench基准，覆盖数学、物理等多领域，增强领域特异性等价变换；2. 提出SCI-Verifier，通过增强推理能力的统一验证器，提升逻辑推理和等价判断能力。

Method: 1. 数据层面：从真实LLM响应构建SCI-VerifyBench，结合领域特异性变换生成高质量数据；2. 模型层面：通过后训练增强SCI-Verifier的推理能力。

Result: SCI-Verifier展示了强大的逻辑推理和等价判断能力，输出简洁稳定；SCI-VerifyBench提供了系统性评估框架。

Insight: 推理能力对科学验证至关重要；跨学科基准和统一验证器的结合可显著提升LLM在科学领域的可靠性和适用性。

Abstract: As large language models (LLMs) are increasingly applied to scientific reasoning, the complexity of answer formats and the diversity of equivalent expressions make answer verification a critical yet challenging task. Existing verification studies in scientific domains suffer from two major limitations: (a) the absence of systematic evaluation standards and insufficient disciplinary coverage, which hinders their comprehensive assessment; and (b) heavy reliance on cumbersome rule design or prompt engineering, which reduces their effectiveness in complex reasoning scenarios or limits their cross-disciplinary generalization. To address these challenges, we propose solutions at both the data and model levels. On the data side, we construct SCI-VerifyBench, a cross-disciplinary benchmark covering mathematics, physics, biology, chemistry, and general scientific QA. The benchmark is built from real LLM responses and enhanced with domain-specific equivalence transformations that generate challenging and realistic data. Model-based and expert annotations ensure both quality and diversity, enabling rigorous evaluation of verification ability. On the model side, we emphasize the importance of reasoning for verification and introduce SCI-Verifier, a unified reasoning-augmented verifier for scientific domains. Through post-training, SCI-Verifier demonstrates strong logical reasoning and equivalence judgment capabilities while maintaining concise and stable outputs. Together, SCI-VerifyBench and SCI-Verifier provide a principled framework for scientific verification, offering both systematic evaluation and practical pathways to enhance the reliability and applicability of LLMs in scientific domains.

[423] Towards Safe Reasoning in Large Reasoning Models via Corrective Intervention

Yichi Zhang,Yue Ding,Jingwen Yang,Tianwei Luo,Dongbai Li,Ranjie Duan,Qiang Liu,Hang Su,Yinpeng Dong,Jun Zhu

Main category: cs.AI

TL;DR: 论文提出了一种名为Intervened Preference Optimization (IPO)的方法，通过干预性监督确保大型推理模型（LRMs）的安全性，显著减少了有害内容和推理中的安全隐患。

Details Motivation: 现有大型推理模型在复杂问题解决中表现优异，但其链式推理（CoT）中常包含有害内容，且现有方法忽略了推理本身的安全性，潜在风险高。

Contribution: 提出了Intervened Preference Optimization (IPO)方法，通过替代合规步骤为安全触发器，构建偏好学习对，显著提升推理和响应的安全性。

Method: IPO方法分析安全推理的关键特征（如安全触发器、合规线索），并通过干预性监督修正不安全推理轨迹。

Result: 在对抗性安全基准测试中，IPO相对减少30%以上的有害性，同时保持多样推理任务的高性能。

Insight: 安全推理的关键在于少数关键步骤的安全性触发器，合规线索与不安全推理相关，干预性措施可有效引导推理安全化。

Abstract: Although Large Reasoning Models (LRMs) have progressed in solving complex problems, their chain-of-thought (CoT) reasoning often contains harmful content that can persist even when the final responses appear safe. We show that this issue still remains in existing methods which overlook the unique significance of safe reasoning, undermining their trustworthiness and posing potential risks in applications if unsafe reasoning is accessible for and exploited by malicious users. We therefore shift our focus to aligning the safety of reasoning itself in this paper and explore process supervision as the solution. However, simply rewarding safe reasoning proves inadequate due to low rollout diversity and limited training signals. To tackle this challenge, we first delve into the characteristics of safe reasoning and uncover several critical insights that 1) safe reasoning is often consolidated by a few critical steps of safety triggers; 2) compliance cues strongly correlate with unsafe continuations; and 3) corrective interventions reliably steer unsafe trajectories towards safer traces. Motivated by these, we propose Intervened Preference Optimization (IPO), an alignment method that enforces safe reasoning by substituting compliance steps with safety triggers and constructing pairs for preference learning with strong signals. Experiments on jailbreak and adversarial safety benchmarks demonstrate that IPO remarkably improves overall safety regarding both reasoning and responses, outperforming SFT-based and RL-based baselines with a relative reduction of over 30% in harmfulness, while preserving excellent performance across diverse reasoning tasks. The results highlight the importance of explicit alignment for reasoning and provide a practical path to safer LRMs.

[424] On the Self-awareness of Large Reasoning Models’ Capability Boundaries

Qingjie Zhang,Yujia Fu,Yang Wang,Liu Yan,Tao Wei,Ke Xu,Minlie Huang,Han Qiu

Main category: cs.AI

TL;DR: 研究了大型推理模型(LRMs)是否具备对自身能力边界的自我认知能力，并提出两种优化策略以提高其推理效率和可靠性。

Details Motivation: LRMs在处理复杂推理任务时表现出色，但在面对困难问题时往往陷入无效推理，浪费计算资源并给出错误答案。这凸显了现有推理范式忽视了问题与模型能力边界的关系。

Contribution: 1) 发现LRMs可以通过推理表达或隐藏状态识别自身能力边界；2) 提出了基于推理表达监控和隐藏状态监控的优化策略，显著提高了模型的效率和可靠性。

Method: 通过分析黑盒模型的推理表达和白盒模型的隐藏状态，识别模型对问题可解性的判断信号。基于此提出两种监控策略：推理表达监控（黑盒）和隐藏状态监控（白盒）。

Result: 实验表明，提出的策略在不牺牲准确性的情况下，避免了无效推理，显著减少token使用量（62.7%-93.6%），提高了模型的效率和可靠性。

Insight: 模型的推理表达和隐藏状态中已隐含了对问题可解性的判断信息，通过监控这些信号，可以提前终止无效推理，优化资源分配。

Abstract: Large Reasoning Models (LRMs) have shown impressive performance on complex reasoning tasks such as mathematics, yet they also display misbehaviors that expose their limitations. In particular, when faced with hard questions, LRMs often engage in unproductive reasoning until context limit, producing wrong answers while wasting substantial computation. This phenomenon reflects a fundamental issue: current answering paradigms overlook the relationship between questions and LRMs’ capability boundaries. In this paper, we investigate whether LRMs possess self-awareness of capability boundaries. We begin by an observation that LRMs may know what they cannot solve through expressed reasoning confidence. For black-box models, we find that reasoning expressions reveal boundary signals, with accelerated growing confidence trajectory for solvable problems but convergent uncertainty trajectory for unsolvable ones. For white-box models, we show that hidden states of the last input token encode boundary information, with solvable and unsolvable problems linearly separable even before reasoning begins. Building on these findings, we propose two simple yet effective optimization strategies: reasoning expression monitoring and hidden states monitoring. Experiments demonstrate that these boundary-aware strategies enable LRMs to avoid unproductive reasoning without sacrificing accuracy, significantly improving reliability and efficiency by cutting token usage up to 62.7 - 93.6%.

[425] Pushing LLMs to Their Logical Reasoning Bound: The Role of Data Reasoning Intensity

Zhen Bi,Zhenlin Hu,Jinnan Yang,Mingyang Chen,Cheng Deng,Yida Xue,Zeyu Yang,Qing Shen,Zhenfang Liu,Kang Zhao,Ningyu Zhang,Jungang Lou

Main category: cs.AI

TL;DR: 论文提出了数据推理强度（DRI）这一新指标，量化训练数据的逻辑推理复杂性，并通过数据重新优化策略提升LLM的逻辑推理能力，优于传统以数据量为中心的优化方法。

Details Motivation: 现有研究多关注数据格式转换，忽视了训练样本内部的推理复杂性，限制了LLM在逻辑推理上的潜力发挥。论文旨在通过量化数据推理复杂性并优化对齐模型推理边界，释放LLM的认知潜力。

Contribution: 1. 提出DRI指标，量化训练数据的逻辑推理复杂性。2. 设计了一种数据重新优化策略，通过提升数据推理强度而非增加数据量来优化LLM性能。3. 在强化学习框架下验证了方法的有效性。

Method: 1. 分解并聚合样本的逻辑结构，定义DRI指标。2. 设计数据重新优化策略，通过提升数据推理强度对齐模型推理边界。3. 在实验中对比传统数据优化方法验证效果。

Result: 实验表明，该方法显著提升了LLM的逻辑推理性能和泛化能力，尤其在强化学习框架下验证了其有效性。

Insight: 优化数据的逻辑推理复杂性比单纯增加数据量或变换数据格式更能有效提升LLM的认知潜力，揭示了未来数据优化的新方向。

Abstract: Recent advances in large language models (LLMs) highlight the importance of training data structure and quality in shaping reasoning behavior. However, most existing approaches focus on transforming data formats while neglecting the internal reasoning complexity of training samples, leaving the reasoning potential of data under-explored and underutilized. In this work, we posit that LLM logical reasoning performance is jointly constrained by the potential of the training data and the cognitive capacity of the model. To make this relationship measurable, we introduce Data Reasoning Intensity (DRI), a novel metric that quantifies the latent logical reasoning complexity of samples by decomposing and aggregating their logical structures. This allows us to analyze how well current LLMs utilize logical reasoning signals and identify performance gaps relative to data potential. Based on this insight, we introduce a re-cognizing optimization strategy that systematically enhances the logical reasoning intensity of training data.Rather than increasing data volume, our method re-optimizes existing samples to better align with the LLM’s logical reasoning boundary. Extensive experiments show that our approach significantly improves performance and generalization over data-centric strategies. We further validate our method under a reinforcement learning framework. Our results indicate that prioritizing reasoning complexity in data rather than sheer scale or superficial form is essential to realizing LLMs’ full cognitive potential.

Huihao Jing,Wenbin Hu,Hongyu Luo,Jianhui Yang,Wei Fan,Haoran Li,Yangqiu Song

Main category: cs.AI

TL;DR: MASLegalBench是针对多智能体系统（MAS）在法律推理任务中的评估基准，专注于演绎推理和智能体协作，填补了现有法律基准的空白。

Details Motivation: 多智能体系统（MAS）结合大语言模型（LLM）在复杂任务中展现出潜力，但法律领域缺乏针对MAS特性的评估方法。

Contribution: 提出了MASLegalBench，首个针对MAS设计的法律推理基准，专注于演绎推理和智能体协作。

Method: 以GDPR为应用场景，设计背景知识和复杂推理过程，手动构建多角色MAS架构并进行实验。

Result: 实验结果揭示了现有LLM模型和MAS架构的优势、局限性和改进空间。

Insight: MASLegalBench为法律领域的MAS研究提供了标准化的评估工具，促进了智能体协作在法律任务中的应用。

Abstract: Multi-agent systems (MAS), leveraging the remarkable capabilities of Large Language Models (LLMs), show great potential in addressing complex tasks. In this context, integrating MAS with legal tasks is a crucial step. While previous studies have developed legal benchmarks for LLM agents, none are specifically designed to consider the unique advantages of MAS, such as task decomposition, agent specialization, and flexible training. In fact, the lack of evaluation methods limits the potential of MAS in the legal domain. To address this gap, we propose MASLegalBench, a legal benchmark tailored for MAS and designed with a deductive reasoning approach. Our benchmark uses GDPR as the application scenario, encompassing extensive background knowledge and covering complex reasoning processes that effectively reflect the intricacies of real-world legal situations. Furthermore, we manually design various role-based MAS and conduct extensive experiments using different state-of-the-art LLMs. Our results highlight the strengths, limitations, and potential areas for improvement of existing models and MAS architectures.

[427] From $f(x)$ and $g(x)$ to $f(g(x))$: LLMs Learn New Skills in RL by Composing Old Ones

Lifan Yuan,Weize Chen,Yuchen Zhang,Ganqu Cui,Hanbin Wang,Ziming You,Ning Ding,Zhiyuan Liu,Maosong Sun,Hao Peng

Main category: cs.AI

TL;DR: 论文通过合成框架证明LLMs可以通过强化学习（RL）组合已有技能学习全新技能，类似人类认知学习方式，且具有迁移性和泛化能力，而仅靠下一个标记训练无法实现。

Details Motivation: 探讨RL是否真的能让LLMs学习全新技能，而非仅重新加权已有能力，以澄清RL在LLM后训练中的作用。

Contribution: 提供实证表明LLMs可通过RL组合已有技能学习新技能；开发合成研究框架；揭示RL对模型推理行为的根本改变。

Method: 定义字符串转换任务作为技能，在RL中训练LLMs组合已知函数f和g为未见函数h(x)=g(f(x))，并验证迁移和泛化能力。

Result: RL使LLMs学会未见函数组合，且能泛化到更复杂任务；组合能力可迁移到不相关任务；RL显著改变模型推理行为。

Insight: 基础模型先掌握基本技能，再用RL激励组合和高级技能，为解决复杂问题提供新思路；RL在LLM学习中具有独特价值。

Abstract: Does RL teach LLMs genuinely new skills, or does it merely activate existing ones? This question lies at the core of ongoing debates about the role of RL in LLM post-training. On one side, strong empirical results can be achieved with RL even without preceding supervised finetuning; on the other, critics argue that RL contributes little beyond reweighting existing reasoning strategies. This work provides concrete evidence that LLMs can acquire genuinely new skills during RL by composing existing ones, mirroring one of the central mechanisms by which humans acquire new cognitive skills. To mitigate data contamination and other confounding factors, and to allow precise control over task complexity, we develop a synthetic framework for our investigation. Specifically, we define a skill as the ability to infer the output of a string transformation function f(x) given x. When an LLM has already learned f and g prior to RL, our experiments reveal that RL enables it to learn unseen compositions of them h(x)=g(f(x)). Further, this compositional ability generalizes to more difficult problems such as compositions of >2 functions unseen during RL training. Surprisingly, our experiments show that compositional skill acquired on a source task transfers to a different target task. This transfer happens even without compositional training on the target, requiring only prior knowledge of the target’s atomic skills. Our qualitative analysis shows that RL fundamentally changes the reasoning behaviors of the models. In contrast, next-token training with the same data yields none of these findings. Our systematic experiments provide fresh insights into LLM learning, suggesting the value of first building base models with basic skills, then using RL to incentivize advanced, generalizable skills for complex problems.

[428] The Era of Real-World Human Interaction: RL from User Conversations

Chuanyang Jin,Jing Xu,Bo Liu,Leitian Tao,Olga Golovneva,Tianmin Shu,Wenting Zhao,Xian Li,Jason Weston

Main category: cs.AI

TL;DR: 论文提出了一种直接从真实用户对话中学习的强化学习范式RLHI，通过用户引导的重写和基于用户的奖励，将用户长期交互历史与即时偏好结合，显著提升了模型的个性化和指令遵循能力。

Details Motivation: 当前对话模型通常依赖专家标注的反馈进行对齐，缺乏直接从用户自然交互中学习的能力。真实用户对话提供了丰富的个性化信息，能够更有效地实现模型的持续改进和多维对齐。

Contribution: 1. 提出了RLHI范式，直接从真实用户对话中学习；2. 开发了两种互补方法：用户引导的重写和基于用户奖励的学习；3. 将用户长期交互历史（如persona）与即时偏好结合，优化模型表现。

Method: 1. RLHI with User-Guided Rewrites：根据用户自然语言反馈重写不满意的输出；2. RLHI with User-Based Rewards：通过基于用户长期交互历史的奖励模型学习；3. 结合persona条件优化，提升个性化表现。

Result: 在WildChat数据集上的实验表明，RLHI方法在个性化和指令遵循任务上优于基线模型，且在推理任务上也表现优异，验证了直接从用户交互中学习的有效性。

Insight: 真实用户对话是一种可扩展且高效的监督信号，能够支持模型的个性化对齐和持续改进，未来可以在更广泛的场景中应用。

Abstract: We posit that to achieve continual model improvement and multifaceted alignment, future models must learn from natural human interaction. Current conversational models are aligned using pre-annotated, expert-generated human feedback. In this work, we introduce Reinforcement Learning from Human Interaction (RLHI), a paradigm that learns directly from in-the-wild user conversations. We develop two complementary methods: (1) RLHI with User-Guided Rewrites, which revises unsatisfactory model outputs based on users’ natural-language follow-up responses, (2) RLHI with User-Based Rewards, which learns via a reward model conditioned on knowledge of the user’s long-term interaction history (termed persona). Together, these methods link long-term user personas to turn-level preferences via persona-conditioned preference optimization. Trained on conversations derived from WildChat, both RLHI variants outperform strong baselines in personalization and instruction-following, and similar feedback enhances performance on reasoning benchmarks. These results suggest organic human interaction offers scalable, effective supervision for personalized alignment.

[429] ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory

Siru Ouyang,Jun Yan,I-Hung Hsu,Yanfei Chen,Ke Jiang,Zifeng Wang,Rujun Han,Long T. Le,Samira Daruki,Xiangru Tang,Vishy Tirumalashetty,George Lee,Mahsan Rofouei,Hangfei Lin,Jiawei Han,Chen-Yu Lee,Tomas Pfister

Main category: cs.AI

TL;DR: ReasoningBank提出了一种新型的记忆框架，用于从代理的成功和失败经验中提炼可泛化的推理策略，并通过MaTTS技术加速学习过程，实现代理的自我进化。

Details Motivation: 当前大型语言模型代理在处理持续任务流时，未能有效利用历史交互经验，导致重复错误和效率低下。

Contribution: 1. 提出了ReasoningBank记忆框架，从代理的经验中提炼推理策略；2. 引入了MaTTS技术，通过计算资源分配加速学习过程。

Method: 1. 通过ReasoningBank存储和自我评估的经验提炼通用推理策略；2. MaTTS通过分配更多计算资源生成多样化经验，提升记忆质量。

Result: 在网页浏览和软件工程任务中，ReasoningBank超越了仅存储原始轨迹或成功任务的现有方法，显著提升了效率和效果。

Insight: 结合记忆和测试时扩展（MaTTS）能够实现代理的自我进化，展示了记忆驱动扩展的潜力。

Abstract: With the growing adoption of large language model agents in persistent real-world roles, they naturally encounter continuous streams of tasks. A key limitation, however, is their failure to learn from the accumulated interaction history, forcing them to discard valuable insights and repeat past errors. We propose ReasoningBank, a novel memory framework that distills generalizable reasoning strategies from an agent’s self-judged successful and failed experiences. At test time, an agent retrieves relevant memories from ReasoningBank to inform its interaction and then integrates new learnings back, enabling it to become more capable over time. Building on this powerful experience learner, we further introduce memory-aware test-time scaling (MaTTS), which accelerates and diversifies this learning process by scaling up the agent’s interaction experience. By allocating more compute to each task, the agent generates abundant, diverse experiences that provide rich contrastive signals for synthesizing higher-quality memory. The better memory in turn guides more effective scaling, establishing a powerful synergy between memory and test-time scaling. Across web browsing and software engineering benchmarks, ReasoningBank consistently outperforms existing memory mechanisms that store raw trajectories or only successful task routines, improving both effectiveness and efficiency; MaTTS further amplifies these gains. These findings establish memory-driven experience scaling as a new scaling dimension, enabling agents to self-evolve with emergent behaviors naturally arise.

[430] Mixture-of-Visual-Thoughts: Exploring Context-Adaptive Reasoning Mode Selection for General Visual Reasoning

Zejun Li,Yingxiu Zhao,Jiwen Zhang,Siyuan Wang,Yang Yao,Runzhou Zhao,Jun Song,Bo Zheng,Zhongyu Wei

Main category: cs.AI

TL;DR: 论文提出了一种自适应视觉推理范式Mixture-of-Visual-Thoughts（MoVT），通过统一不同推理模式并结合上下文自适应选择，提升模型的通用推理能力。

Details Motivation: 当前视觉推理方法主要针对特定推理模式设计，缺乏通用性。本文旨在通过统一多模式推理并结合上下文选择，提升模型的通用推理能力。

Contribution: 提出了MoVT范式，结合AdaVaR框架，统一不同推理模式并通过强化学习引导上下文自适应选择，提升通用视觉推理能力。

Method: 采用两阶段学习框架AdaVaR：监督冷启动阶段统一学习多模式推理，强化学习阶段通过AdaGRPO算法诱导模式选择能力。

Result: 实验表明AdaVaR能有效学习多模式推理并实现自适应选择，在不同场景中表现一致提升。

Insight: MoVT为构建通用视觉推理模型提供了有效解决方案，上下文自适应是多模式推理的关键。

Abstract: Current visual reasoning methods mainly focus on exploring specific reasoning modes. Although improvements can be achieved in particular domains, they struggle to develop general reasoning capabilities. Inspired by this, we propose a novel adaptive reasoning paradigm, Mixture-of-Visual-Thoughts (MoVT), which unifies different reasoning modes within a single model and guides it to select the appropriate mode based on context. To achieve this, we introduce AdaVaR, a two-stage Adaptive Visual Reasoning learning framework: different modes are unified and learned during the supervised cold-start stage, and the mode selection capability is induced via an RL process with a carefully designed AdaGRPO algorithm. Extensive experiments show that AdaVaR effectively guides the model to learn and differentiate multiple modes and perform context-adaptive mode selection, achieving consistent improvement across various scenarios, highlighting MoVT as an effective solution for building general visual reasoning models.

[431] AttAnchor: Guiding Cross-Modal Token Alignment in VLMs with Attention Anchors

Junyang Zhang,Tianyi Zhu,Thierry Tambe

Main category: cs.AI

TL;DR: AttAnchor提出了一种参数无关的框架，通过注意力锚点改进视觉语言模型（VLM）中的跨模态令牌对齐，提升任务性能和减少幻觉现象。

Details Motivation: 现有的视觉语言模型因依赖简单的令牌拼接和模态无关的位置编码，导致跨模态令牌对齐不佳，引发幻觉和性能下降问题。

Contribution: 提出了注意力锚点机制，显式地引导跨模态令牌对齐，无需额外参数，显著提升了模型的性能和抗幻觉能力。

Method: 通过将文本令牌插入到相关的视觉区块附近，创建语义标记，引导模型关注正确的图像区域，从而改善跨模态令牌的对齐。

Result: 在15个指标和基准测试中的13个上取得提升，推理任务最高提升32%，幻觉测试最高15%，轻量化模型性能超越大型模型。

Insight: 跨模态令牌的联合聚类而非单模态内分组，能更有效地实现跨模态对齐，提升视觉语言模型的性能。

Abstract: A fundamental reason for the dominance of attention over RNNs and LSTMs in LLMs is its ability to capture long-range dependencies by modeling direct interactions between all tokens, overcoming the sequential limitations of recurrent architectures. Similarly, a key reason why today’s vision language models (VLMs) hallucinate and underperform pure language models is that they rely on direct concatenation of image and text tokens with a modality-blinded positional encoding, which conveniently adopts the pretrained LLM backbone but forces unnecessary long-distance attention between semantically related tokens across modalities. This underscores the urgent need for mechanisms that efficiently enhance token locality and cross-modal alignment. In response, we propose Attention Anchor, a parameter-free framework that efficiently groups semantically similar tokens across modalities, improving cross-modal locality. By inserting text tokens near relevant visual patches, we create semantic signposts that reveal true content-based cross-modal attention scores, guiding the model to focus on the correct image regions for tasks such as VQA, MMBench and POPE. This improves answer accuracy and reduces hallucinations without disrupting the prompt’s semantic flow. AttAnchor achieves improvements across 13 out of 15 different metrics and benchmarks, including up to 32% gains on reasoning tasks and up to 15% improvements on hallucination benchmarks. AttAnchor enables TinyLLaVA 1B to outperform much larger models like LLaVA 7B and QwenVL 3B on POPE with only 0.1% inference time overhead. To the best of our knowledge, this work is among the first to investigate mixed-modal token grouping, where text and image tokens are clustered jointly into shared groups rather than being grouped within a single modality or merely aligned post-hoc with additional alignment losses.

[432] Training Vision-Language Process Reward Models for Test-Time Scaling in Multimodal Reasoning: Key Insights and Lessons Learned

Brandon Ong,Tej Deep Pala,Vernon Toh,William Chandra Tjhi,Soujanya Poria

Main category: cs.AI

TL;DR: 该论文研究如何通过过程奖励模型（VL-PRMs）提升视觉语言模型的推理能力，提出了一种混合数据合成框架和感知级监督方法，并系统评估了多种测试时扩展策略。

Details Motivation: 现有视觉语言过程奖励模型（VL-PRMs）依赖蒙特卡洛树搜索（MCTS）构建数据，但其生成的监督信号噪声大且泛化能力有限。该研究旨在探索VL-PRMs的设计空间，提升其在多模态推理中的可靠性和扩展性。

Contribution: 1. 提出混合数据合成框架，结合MCTS和强视觉语言模型的判断，生成更准确的步骤级标签；2. 引入感知级监督，使PRM能显式检测视觉基础阶段的错误；3. 系统评估测试时扩展策略，验证VL-PRMs在多模态任务中的有效性。

Method: 1. 使用混合数据合成方法（MCTS+强VLM）生成高质量数据；2. 设计感知级监督机制，专注于视觉基础错误检测；3. 测试时扩展策略的系统评估（如将VL-PRMs用作结果奖励模型）。

Result: 在五个多模态基准测试中，VL-PRMs表现优异：（1）测试时扩展中优于过程步骤选择；（2）小模型能媲美甚至超越大模型的错误检测能力；（3）揭示了强VLM的潜在推理能力；（4）感知级监督显著提升性能；（5）未训练的数学推理任务中表现良好。

Insight: 1. VL-PRMs作为结果奖励模型在测试时扩展中具有优势；2. 模型大小并非错误检测能力的关键；3. 感知级监督是多模态任务的关键；4. 测试时扩展策略需针对不同任务定制。

Abstract: Process Reward Models (PRMs) provide step-level supervision that improves the reliability of reasoning in large language models. While PRMs have been extensively studied in text-based domains, their extension to Vision Language Models (VLMs) remains limited. Existing Vision-Language PRMs (VL-PRMs) rely on Monte Carlo Tree Search (MCTS) for data construction, which can often produce noisy supervision signals and limit generalization across tasks. In this work, we aim to elucidate the design space of VL-PRMs by exploring diverse strategies for dataset construction, training, and test-time scaling. First, we introduce a hybrid data synthesis framework that combines MCTS with judgments from a strong VLM, producing more accurate step-level labels. Second, we propose perception-focused supervision, enabling our PRM to explicitly detect errors at the visual grounding stage of reasoning. Third, we systematically evaluate multiple test-time scaling strategies, showing that our PRMs can reliably guide VLMs toward more accurate solutions. Our experiments covering five diverse multimodal benchmarks (MMMU, PuzzleVQA, AlgoPuzzleVQA, MathVista, and MathVision) reveal several key insights: (i) VL-PRMs when used as Outcome Reward Models (ORMs) during test-time scaling (TTS) can outperform VL-PRM guided process step selection, (ii) smaller VL-PRMs can match or even surpass larger ones in detecting process errors, (iii) VL-PRMs uncover latent reasoning abilities in stronger VLM backbones, (iv) perception-level supervision leads to significant gains in test-time scaling, and (v) TTS performance of different policies improve on advanced math reasoning datasets despite not training VL-PRMs on such datasets. We hope our work will motivate further research and support the advancement of VLMs.

[433] BridgeDrive: Diffusion Bridge Policy for Closed-Loop Trajectory Planning in Autonomous Driving

Shu Liu,Wenlin Chen,Weihao Li,Zheng Wang,Lijin Yang,Jianing Huang,Yipin Zhang,Zhongzhan Huang,Ze Cheng,Hao Yang

Main category: cs.AI

TL;DR: BridgeDrive提出了一种基于扩散桥的闭环轨迹规划方法，通过锚点引导扩散模型，提升自动驾驶在复杂动态场景中的规划能力，实现了Bench2Drive基准上5%的性能提升。

Details Motivation: 扩散模型在自动驾驶中表现出捕捉多模态行为的能力，但在闭环环境中如何有效引导仍是一个挑战。现有方法依赖截断的调度策略，可能影响性能和理论一致性。

Contribution: 1. 引入BridgeDrive，一种基于扩散桥的闭环轨迹规划方法；2. 提出一种理论一致的扩散框架，将锚点转化为细粒度轨迹计划；3. 兼容高效的ODE求解器，适用于实时自动驾驶部署。

Method: 通过锚点引导扩散模型，结合ODE求解器，实现从锚点到细粒度轨迹的转换。该方法避免了截断调度，提升了理论的严谨性和性能。

Result: 在Bench2Drive基准上取得了最先进性能，成功率比现有方法提高了5%。

Insight: 锚点引导的扩散模型在复杂动态场景中表现优异，ODE求解器的兼容性为实时部署提供了重要支持。

Abstract: Diffusion-based planners have shown great promise for autonomous driving due to their ability to capture multi-modal driving behaviors. However, guiding these models effectively in reactive, closed-loop environments remains a significant challenge. Simple conditioning often fails to provide sufficient guidance in complex and dynamic driving scenarios. Recent work attempts to use typical expert driving behaviors (i.e., anchors) to guide diffusion models but relies on a truncated schedule, which introduces theoretical inconsistencies and can compromise performance. To address this, we introduce BridgeDrive, a novel anchor-guided diffusion bridge policy for closed-loop trajectory planning. Our approach provides a principled diffusion framework that effectively translates anchors into fine-grained trajectory plans, appropriately responding to varying traffic conditions. Our planner is compatible with efficient ODE solvers, a critical factor for real-time autonomous driving deployment. We achieve state-of-the-art performance on the Bench2Drive benchmark, improving the success rate by 5% over prior arts.

[434] Transparent Visual Reasoning via Object-Centric Agent Collaboration

Benjamin Teoh,Ben Glocker,Francesca Toni,Avinash Kori

Main category: cs.AI

TL;DR: 论文提出了OCEAN框架，通过对象中心表征和多智能体协作实现透明视觉推理，展示了在性能和解释性上的优势。

Details Motivation: 解决可解释AI中如何生成基于人类可理解概念的视觉解释的挑战。

Contribution: 提出了OCEAN框架，结合对象中心表征和多智能体协作，实现了透明且可解释的视觉推理。

Method: 采用对象中心表征和基于博弈论的多智能体协作，训练端到端的可解释模型。

Result: 在多个数据集上表现与黑盒模型相当，且用户研究表明其解释更直观可信。

Insight: 对象中心表征和多智能体协作是实现可解释AI的有效途径。

Abstract: A central challenge in explainable AI, particularly in the visual domain, is producing explanations grounded in human-understandable concepts. To tackle this, we introduce OCEAN (Object-Centric Explananda via Agent Negotiation), a novel, inherently interpretable framework built on object-centric representations and a transparent multi-agent reasoning process. The game-theoretic reasoning process drives agents to agree on coherent and discriminative evidence, resulting in a faithful and interpretable decision-making process. We train OCEAN end-to-end and benchmark it against standard visual classifiers and popular posthoc explanation tools like GradCAM and LIME across two diagnostic multi-object datasets. Our results demonstrate competitive performance with respect to state-of-the-art black-box models with a faithful reasoning process, which was reflected by our user study, where participants consistently rated OCEAN’s explanations as more intuitive and trustworthy.

[435] Vision-and-Language Navigation with Analogical Textual Descriptions in LLMs

Yue Zhang,Tianyi Ma,Zun Wang,Yanyuan Qiao,Parisa Kordjamshidi

Main category: cs.AI

TL;DR: 论文提出了一种通过引入多视角文本描述和改进类比推理的LLM导航方法，显著提高了Vision-and-Language Navigation（VLN）任务的性能。

Details Motivation: 现有的零样本LLM导航方法要么过度简化视觉细节（通过文本描述编码图像），要么难以捕捉高层语义（直接处理原始图像）。

Contribution: 通过结合多视角文本描述和类比推理，增强了导航代理的全局场景理解和空间推理能力。

Method: 利用LLM处理多视角文本描述，并通过类比推理提升场景理解和导航决策。

Result: 在R2R数据集上验证了方法的有效性，导航性能显著提升。

Insight: 多视角文本描述和类比推理的结合为VLN任务提供了更丰富的语义信息和高层推理能力。

Abstract: Integrating large language models (LLMs) into embodied AI models is becoming increasingly prevalent. However, existing zero-shot LLM-based Vision-and-Language Navigation (VLN) agents either encode images as textual scene descriptions, potentially oversimplifying visual details, or process raw image inputs, which can fail to capture abstract semantics required for high-level reasoning. In this paper, we improve the navigation agent’s contextual understanding by incorporating textual descriptions from multiple perspectives that facilitate analogical reasoning across images. By leveraging text-based analogical reasoning, the agent enhances its global scene understanding and spatial reasoning, leading to more accurate action decisions. We evaluate our approach on the R2R dataset, where our experiments demonstrate significant improvements in navigation performance.

eess.AS [Back]

[436] VSSFlow: Unifying Video-conditioned Sound and Speech Generation via Joint Learning

Xin Cheng,Yuyue Wang,Xihua Wang,Yihan Wu,Kaisi Guan,Yijing Chen,Peng Zhang,Xiaojiang Liu,Meng Cao,Ruihua Song

Main category: eess.AS

TL;DR: VSSFlow提出了一种统一的流匹配框架，将视频到声音（V2S）和视觉文本到语音（VisualTTS）任务整合到一个模型中，通过条件聚合机制处理不同的输入信号，并利用交叉注意力和自注意力的归纳偏置优化生成效果。

Details Motivation: 现有的视频到声音和视觉文本到语音任务通常分开处理，缺乏统一的框架，且尝试统一的任务面临输入条件差异和训练复杂性问题。

Contribution: VSSFlow是首个将V2S和VisualTTS统一到一个流匹配框架的工作，提出了条件聚合机制和注意力的动态使用，同时发现联合训练可以共享音频先验，提升性能。

Method: 采用基于流匹配的生成模型，动态结合交叉注意力（处理视频条件）和自注意力（处理语音文本），并通过端到端训练学习共享的音频先验。

Result: 在V2S和VisualTTS基准测试中超过了领域专用模型，验证了统一生成模型的潜力。

Insight: 联合训练可以学习任务间共享的音频先验，加速收敛并提升生成质量，无需复杂的训练策略设计。

Abstract: Video-conditioned sound and speech generation, encompassing video-to-sound (V2S) and visual text-to-speech (VisualTTS) tasks, are conventionally addressed as separate tasks, with limited exploration to unify them within a signle framework. Recent attempts to unify V2S and VisualTTS face challenges in handling distinct condition types (e.g., heterogeneous video and transcript conditions) and require complex training stages. Unifying these two tasks remains an open problem. To bridge this gap, we present VSSFlow, which seamlessly integrates both V2S and VisualTTS tasks into a unified flow-matching framework. VSSFlow uses a novel condition aggregation mechanism to handle distinct input signals. We find that cross-attention and self-attention layer exhibit different inductive biases in the process of introducing condition. Therefore, VSSFlow leverages these inductive biases to effectively handle different representations: cross-attention for ambiguous video conditions and self-attention for more deterministic speech transcripts. Furthermore, contrary to the prevailing belief that joint training on the two tasks requires complex training strategies and may degrade performance, we find that VSSFlow benefits from the end-to-end joint learning process for sound and speech generation without extra designs on training stages. Detailed analysis attributes it to the learned general audio prior shared between tasks, which accelerates convergence, enhances conditional generation, and stabilizes the classifier-free guidance process. Extensive experiments demonstrate that VSSFlow surpasses the state-of-the-art domain-specific baselines on both V2S and VisualTTS benchmarks, underscoring the critical potential of unified generative models.

[437] AISHELL6-whisper: A Chinese Mandarin Audio-visual Whisper Speech Dataset with Speech Recognition Baselines

Cancan Li,Fei Su,Juan Liu,Hui Bu,Yulong Wan,Hongbin Suo,Ming Li

Main category: eess.AS

TL;DR: 该论文介绍了AISHELL6-Whisper，一个中文普通话视听耳语语音数据集，提供了30小时的耳语和平行正常语音，并提出了一种基于Whisper-Flamingo框架的视听语音识别基线方法。

Details Motivation: 当前中文普通话的视听耳语语音识别缺乏大规模数据集，限制了相关研究的发展。该论文旨在填补这一空白。

Contribution: 1. 发布了首个大规模开源中文普通话视听耳语语音数据集AISHELL6-Whisper；2. 提出了一种基于Whisper-Flamingo框架的视听语音识别基线方法。

Method: 使用Whisper-Flamingo框架，通过并行训练策略对齐不同语音类型的嵌入，并采用投影层适应耳语语音的频谱特性。

Result: 测试集上耳语语音的字符错误率（CER）为4.13%，正常语音为1.11%，并在wTIMIT基准上实现了新的最先进结果。

Insight: 数据集和基线的开放共享将为耳语语音识别研究提供重要资源，促进多模态语音识别技术的发展。

Abstract: Whisper speech recognition is crucial not only for ensuring privacy in sensitive communications but also for providing a critical communication bridge for patients under vocal restraint and enabling discrete interaction in noise-sensitive environments. The development of Chinese mandarin audio-visual whisper speech recognition is hindered by the lack of large-scale datasets. We present AISHELL6-Whisper, a large-scale open-source audio-visual whisper speech dataset, featuring 30 hours each of whisper speech and parallel normal speech, with synchronized frontal facial videos. Moreover, we propose an audio-visual speech recognition (AVSR) baseline based on the Whisper-Flamingo framework, which integrates a parallel training strategy to align embeddings across speech types, and employs a projection layer to adapt to whisper speech’s spectral properties. The model achieves a Character Error Rate (CER) of 4.13% for whisper speech and 1.11% for normal speech in the test set of our dataset, and establishes new state-of-the-art results on the wTIMIT benchmark. The dataset and the AVSR baseline codes are open-sourced at https://zutm.github.io/AISHELL6-Whisper.

cs.CR [Back]

[438] MaskSQL: Safeguarding Privacy for LLM-Based Text-to-SQL via Abstraction

Sepideh Abedini,Shubhankar Mohapatra,D. B. Emerson,Masoumeh Shafieinejad,Jesse C. Cresswell,Xi He

Main category: cs.CR

TL;DR: MaskSQL是一个通过抽象化保护隐私的文本到SQL框架，利用LLM的同时避免敏感数据共享，平衡隐私与性能。

Details Motivation: LLM在文本到SQL等任务中表现优异，但在隐私敏感领域的使用受限，因为需要共享数据。尽管小型语言模型（SLM）可以本地部署保护隐私，但在复杂任务中表现不佳。

Contribution: 提出MaskSQL框架，通过抽象化机制在LLM提示中隐藏敏感信息，既保护隐私又保持任务性能。

Method: 采用抽象化而非完全删除或泛化敏感信息的方法，保留关键信息并丢弃细节，优化隐私与性能的权衡。

Result: 实验显示MaskSQL优于SLM模型，性能接近LLM基准，同时保护隐私。

Insight: 抽象化是一种有效的隐私保护方法，适用于需要平衡隐私与性能的任务。

Abstract: Large language models (LLMs) have shown promising performance on tasks that require reasoning, such as text-to-SQL, code generation, and debugging. However, regulatory frameworks with strict privacy requirements constrain their integration into sensitive systems. State-of-the-art LLMs are also proprietary, costly, and resource-intensive, making local deployment impractical. Consequently, utilizing such LLMs often requires sharing data with third-party providers, raising privacy concerns and risking noncompliance with regulations. Although fine-tuned small language models (SLMs) can outperform LLMs on certain tasks and be deployed locally to mitigate privacy concerns, they underperform on more complex tasks such as text-to-SQL translation. In this work, we introduce MaskSQL, a text-to-SQL framework that utilizes abstraction as a privacy protection mechanism to mask sensitive information in LLM prompts. Unlike redaction, which removes content entirely, or generalization, which broadens tokens, abstraction retains essential information while discarding unnecessary details, striking an effective privacy-utility balance for the text-to-SQL task. Moreover, by providing mechanisms to control the privacy-utility tradeoff, MaskSQL facilitates adoption across a broader range of use cases. Our experimental results show that MaskSQL outperforms leading SLM-based text-to-SQL models and achieves performance approaching state-of-the-art LLM-based models, while preserving privacy.

[439] Responsible Diffusion: A Comprehensive Survey on Safety, Ethics, and Trust in Diffusion Models

Kang Wei,Xin Yuan,Fushuo Huo,Chuan Ma,Long Yuan,Songze Li,Ming Ding,Dacheng Tao

Main category: cs.CR

TL;DR: 这篇论文全面调查了扩散模型在安全性、伦理性和可信度方面的潜在威胁及对策，旨在推动生成式人工智能的技术能力和应用成熟度。

Details Motivation: 扩散模型在各领域展现出强大的数据生成能力，但其潜藏的安全、伦理和信任问题尚未得到充分探讨，亟需系统性的研究以促进其健康发展。

Contribution: 论文系统梳理了扩散模型的威胁类型及应对措施，并提供了具体案例和未来研究方向，填补了该领域的文献空白。

Method: 采用文献综述和系统性分类的方法，对扩散模型的威胁（如数据隐私、恶意生成内容）及对应的防御技术（如内容过滤、模型鲁棒性增强）进行了详细分析。

Result: 研究表明扩散模型的安全性和伦理性问题复杂多样，需要多学科协作解决。论文还总结了当前研究的局限性和未解决的问题。

Insight: 扩散模型的广泛应用需伴随严格的监管和技术保障，未来的研究应关注跨学科合作和新技术的开发，以确保其负责任的使用。

Abstract: Diffusion models (DMs) have been investigated in various domains due to their ability to generate high-quality data, thereby attracting significant attention. However, similar to traditional deep learning systems, there also exist potential threats to DMs. To provide advanced and comprehensive insights into safety, ethics, and trust in DMs, this survey comprehensively elucidates its framework, threats, and countermeasures. Each threat and its countermeasures are systematically examined and categorized to facilitate thorough analysis. Furthermore, we introduce specific examples of how DMs are used, what dangers they might bring, and ways to protect against these dangers. Finally, we discuss key lessons learned, highlight open challenges related to DM security, and outline prospective research directions in this critical field. This work aims to accelerate progress not only in the technical capabilities of generative artificial intelligence but also in the maturity and wisdom of its application.

[440] StolenLoRA: Exploring LoRA Extraction Attacks via Synthetic Data

Yixu Wang,Yan Teng,Yingchun Wang,Xingjun Ma

Main category: cs.CR

TL;DR: 论文提出了LoRA提取攻击的新方法StolenLoRA，利用合成数据和半监督学习策略，实验显示攻击成功率达96.60%，揭示了LoRA调优模型的安全漏洞。

Details Motivation: LoRA等高效微调方法在快速部署定制模型方面表现出色，但其紧凑性带来了新的安全隐患，尤其是模型提取攻击的风险。

Contribution: 1. 首次提出LoRA提取攻击的概念；2. 提出StolenLoRA方法，利用合成数据和DSL策略高效提取LoRA模型；3. 揭示了LoRA调优模型的脆弱性，并探讨初步防御策略。

Method: 结合大型语言模型生成合成数据，并采用基于分歧的半监督学习（DSL）策略，最大化从有限查询中获取的信息。

Result: StolenLoRA在10k查询内达到96.60%的攻击成功率，甚至在不同预训练骨干的跨骨干场景中有效。

Insight: LoRA调优模型对提取攻击特别脆弱，亟需针对PEFT方法的防御机制；多样化LoRA部署可能是潜在防御方案。

Abstract: Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA have transformed vision model adaptation, enabling the rapid deployment of customized models. However, the compactness of LoRA adaptations introduces new safety concerns, particularly their vulnerability to model extraction attacks. This paper introduces a new focus of model extraction attacks named LoRA extraction that extracts LoRA-adaptive models based on a public pre-trained model. We then propose a novel extraction method called StolenLoRA which trains a substitute model to extract the functionality of a LoRA-adapted model using synthetic data. StolenLoRA leverages a Large Language Model to craft effective prompts for data generation, and it incorporates a Disagreement-based Semi-supervised Learning (DSL) strategy to maximize information gain from limited queries. Our experiments demonstrate the effectiveness of StolenLoRA, achieving up to a 96.60% attack success rate with only 10k queries, even in cross-backbone scenarios where the attacker and victim models utilize different pre-trained backbones. These findings reveal the specific vulnerability of LoRA-adapted models to this type of extraction and underscore the urgent need for robust defense mechanisms tailored to PEFT methods. We also explore a preliminary defense strategy based on diversified LoRA deployments, highlighting its potential to mitigate such attacks.

cs.DL [Back]

[441] Overview of SCIDOCA 2025 Shared Task on Citation Prediction, Discovery, and Placement

An Dao,Vu Tran,Le-Minh Nguyen,Yuji Matsumoto

Main category: cs.DL

TL;DR: SCIDOCA 2025共享任务聚焦科学文献中的引用发现与预测，包含三个子任务：引用发现、掩码引用预测和引用句子预测。数据集基于S2ORC构建，包含60,000+标注段落，7个团队参与。

Details Motivation: 为科学文献理解提供新的评估基准，推动引用建模的研究。

Contribution: 发布了大规模标注数据集，定义了三个子任务，分析了系统性能。

Method: 基于S2ORC构建数据集，设计三个子任务：引用发现、掩码引用预测和引用句子预测。

Result: 三个团队提交结果，报告了各子任务的性能指标，分析了系统效果。

Insight: 共享任务为科学文档理解提供了新的研究方向，尤其是引用建模。

Abstract: We present an overview of the SCIDOCA 2025 Shared Task, which focuses on citation discovery and prediction in scientific documents. The task is divided into three subtasks: (1) Citation Discovery, where systems must identify relevant references for a given paragraph; (2) Masked Citation Prediction, which requires selecting the correct citation for masked citation slots; and (3) Citation Sentence Prediction, where systems must determine the correct reference for each cited sentence. We release a large-scale dataset constructed from the Semantic Scholar Open Research Corpus (S2ORC), containing over 60,000 annotated paragraphs and a curated reference set. The test set consists of 1,000 paragraphs from distinct papers, each annotated with ground-truth citations and distractor candidates. A total of seven teams registered, with three submitting results. We report performance metrics across all subtasks and analyze the effectiveness of submitted systems. This shared task provides a new benchmark for evaluating citation modeling and encourages future research in scientific document understanding. The dataset and task materials are publicly available at https://github.com/daotuanan/scidoca2025-shared-task.

q-bio.NC [Back]

[442] Targeted perturbations reveal brain-like local coding axes in robustified, but not standard, ANN-based brain models

Nikolas McNeal,N. Apurva Ratan Murty

Main category: q-bio.NC

TL;DR: 研究表明，通过小规模对抗性扰动测试可评估ANN模型的局部表征几何特性，揭示标准模型的脆弱性，而鲁棒化模型的编码维度更接近人脑特性。

Details Motivation: 当前ANN模型在预测神经反应方面表现相似，需要更强标准评估其对视觉系统的模拟能力。

Contribution: 提出使用对抗性扰动测试局部表征几何特性的新标准，发现鲁棒化模型的编码维度更稳定且语义化。

Method: 通过小规模对抗性扰动测试多种ANN模型的局部表征几何特性，比较其敏感性和编码维度。

Result: 标准模型对扰动极为敏感，编码维度不稳定；鲁棒化模型的编码维度更具泛化性和语义意义。

Insight: 对抗性扰动测试可作为评估神经编码模型的强有力工具，鲁棒化模型更接近人脑的实际编码机制。

Abstract: Artificial neural networks (ANNs) have become the de facto standard for modeling the human visual system, primarily due to their success in predicting neural responses. However, with many models now achieving similar predictive accuracy, we need a stronger criterion. Here, we use small-scale adversarial probes to characterize the local representational geometry of many highly predictive ANN-based brain models. We report four key findings. First, we show that most contemporary ANN-based brain models are unexpectedly fragile. Despite high prediction scores, their response predictions are highly sensitive to small, imperceptible perturbations, revealing unreliable local coding directions. Second, we demonstrate that a model’s sensitivity to adversarial probes can better discriminate between candidate neural encoding models than prediction accuracy alone. Third, we find that standard models rely on distinct local coding directions that do not transfer across model architectures. Finally, we show that adversarial probes from robustified models produce generalizable and semantically meaningful changes, suggesting that they capture the local coding dimensions of the visual system. Together, our work shows that local representational geometry provides a stronger criterion for brain model evaluation. We also provide empirical grounds for favoring robust models, whose more stable coding axes not only align better with neural selectivity but also generate concrete, testable predictions for future experiments.

cs.RO [Back]

[443] ReSeFlow: Rectifying SE(3)-Equivariant Policy Learning Flows

Zhitao Wang,Yanke Wang,Jiangtao Wen,Roberto Horowitz,Yuxing Han

Main category: cs.RO

TL;DR: 论文提出ReSeFlow，一种结合SE(3)等变扩散模型和修正流的方法，用于高效生成机器人操纵任务的轨迹级策略，显著降低推理时间并提升性能。

Details Motivation: 在非结构化环境中，机器人操纵任务需要生成鲁棒且长时程的策略。SE(3)等变扩散模型虽然数据高效，但推理时间成本高。作者受修正流的高效推理启发，提出了ReSeFlow来解决这一问题。

Contribution: 1. 提出ReSeFlow，结合SE(3)等变性和修正流，实现高效的策略生成。2. 通过保留旋转和平移对称性，提升模型在刚性运动下的鲁棒泛化能力。3. 实验证明仅需一步推理即可优于基线方法，显著降低误差。

Method: 1. 将修正流引入SE(3)等变扩散模型，优化推理效率。2. 使用SE(3)等变网络保持对称性。3. 在模拟基准上验证方法的性能。

Result: ReSeFlow仅需一步推理即可超越基线方法（100步推理），在绘画任务中误差降低48.5%，旋转三角形任务中降低21.9%。

Insight: SE(3)等变性与修正流的结合为生成策略学习模型的实际应用提供了高效的数据和推理方案。

Abstract: Robotic manipulation in unstructured environments requires the generation of robust and long-horizon trajectory-level policy with conditions of perceptual observations and benefits from the advantages of SE(3)-equivariant diffusion models that are data-efficient. However, these models suffer from the inference time costs. Inspired by the inference efficiency of rectified flows, we introduce the rectification to the SE(3)-diffusion models and propose the ReSeFlow, i.e., Rectifying SE(3)-Equivariant Policy Learning Flows, providing fast, geodesic-consistent, least-computational policy generation. Crucially, both components employ SE(3)-equivariant networks to preserve rotational and translational symmetry, enabling robust generalization under rigid-body motions. With the verification on the simulated benchmarks, we find that the proposed ReSeFlow with only one inference step can achieve better performance with lower geodesic distance than the baseline methods, achieving up to a 48.5% error reduction on the painting task and a 21.9% reduction on the rotating triangle task compared to the baseline’s 100-step inference. This method takes advantages of both SE(3) equivariance and rectified flow and puts it forward for the real-world application of generative policy learning models with the data and inference efficiency.

[444] Robot Learning from Any Images

Siheng Zhao,Jiageng Mao,Wei Chow,Zeyu Shangguan,Tianheng Shi,Rong Xue,Yuxi Zheng,Yijia Weng,Yang You,Daniel Seita,Leonidas Guibas,Sergey Zakharov,Vitor Guizilini,Yue Wang

Main category: cs.RO

TL;DR: RoLA框架将任意真实图像转换为支持物理交互的机器人环境，无需额外硬件或数字资产，实现了高效、多样化的机器人数据生成与应用。

Details Motivation: 现有机器人学习方法通常依赖专用硬件或预先构建的数字环境，限制了数据的多样性和可扩展性。RoLA的目标是通过单张图像直接生成交互式机器人环境，降低数据生成的门槛。

Contribution: RoLA的主要贡献包括：1) 提出一种单视角物理场景恢复方法；2) 设计高效的视觉融合策略；3) 展示了框架在多样应用中的灵活性，如互联网图像学习与真实-仿真-真实系统。

Method: RoLA通过单视角物理场景恢复方法重建环境，并结合视觉融合策略生成逼真数据。核心步骤包括从图像中提取物理信息并构建交互式环境。

Result: 实验表明，RoLA能够快速生成大量多样化机器人数据，并在多个应用场景中表现出色，例如机械臂和人形机器人的任务学习。

Insight: RoLA突破了传统机器人数据生成的依赖限制，通过单张图像实现了高效、低成本的数据扩展，为机器人学习开辟了新途径。

Abstract: We introduce RoLA, a framework that transforms any in-the-wild image into an interactive, physics-enabled robotic environment. Unlike previous methods, RoLA operates directly on a single image without requiring additional hardware or digital assets. Our framework democratizes robotic data generation by producing massive visuomotor robotic demonstrations within minutes from a wide range of image sources, including camera captures, robotic datasets, and Internet images. At its core, our approach combines a novel method for single-view physical scene recovery with an efficient visual blending strategy for photorealistic data collection. We demonstrate RoLA’s versatility across applications like scalable robotic data generation and augmentation, robot learning from Internet images, and single-image real-to-sim-to-real systems for manipulators and humanoids. Video results are available at https://sihengz02.github.io/RoLA .

[445] Leave No Observation Behind: Real-time Correction for VLA Action Chunks

Kohei Sendai,Maxime Alvarez,Tatsuya Matsushima,Yutaka Matsuo,Yusuke Iwasawa

Main category: cs.RO

TL;DR: 本文提出了一种轻量级的实时动作块矫正模块（A2C2），用于改进VLA模型在高延迟和长时任务中的反应性和连贯性，无需重新训练基础策略。

Details Motivation: VLA模型通过预测动作块提高效率和时序一致性，但动作块化在高延迟和长时任务中会损害反应性。

Contribution: A2C2模块通过结合最新观测、基础动作、位置特征和基础策略特征，实时校正动作块，恢复闭环响应性。

Method: A2C2是一个轻量级的矫正头，每步运行时结合多源信息输出校正值。它是异步执行的，且无需修改基础策略。

Result: 在动态Kinetix任务库和LIBERO Spatial上，A2C2显著提高了成功率（分别比RTC高23%和7%），并增强了长时任务的鲁棒性。

Insight: A2C2是一种即插即用的机制，适用于实时控制中的大容量动作块策略部署。

Abstract: To improve efficiency and temporal coherence, Vision-Language-Action (VLA) models often predict action chunks; however, this action chunking harms reactivity under inference delay and long horizons. We introduce Asynchronous Action Chunk Correction (A2C2), which is a lightweight real-time chunk correction head that runs every control step and adds a time-aware correction to any off-the-shelf VLA’s action chunk. The module combines the latest observation, the predicted action from VLA (base action), a positional feature that encodes the index of the base action within the chunk, and some features from the base policy, then outputs a per-step correction. This preserves the base model’s competence while restoring closed-loop responsiveness. The approach requires no retraining of the base policy and is orthogonal to asynchronous execution schemes such as Real Time Chunking (RTC). On the dynamic Kinetix task suite (12 tasks) and LIBERO Spatial, our method yields consistent success rate improvements across increasing delays and execution horizons (+23% point and +7% point respectively, compared to RTC), and also improves robustness for long horizons even with zero injected delay. Since the correction head is small and fast, there is minimal overhead compared to the inference of large VLA models. These results indicate that A2C2 is an effective, plug-in mechanism for deploying high-capacity chunking policies in real-time control.

[446] RAVEN: Resilient Aerial Navigation via Open-Set Semantic Memory and Behavior Adaptation

Seungchan Kim,Omar Alama,Dmytro Kurdydyk,John Keller,Nikhil Keetha,Wenshan Wang,Yonatan Bisk,Sebastian Scherer

Main category: cs.RO

TL;DR: RAVEN是一个基于3D记忆的行为树框架，用于非结构化户外环境的空中语义导航，通过持久记忆和自适应行为提升导航性能。

Details Motivation: 户外语义导航面临长距离搜索和非结构化环境的挑战，现有方法要么是反应式的短视行为，要么依赖离线预计算的场景图，缺乏适应性。

Contribution: 1) 引入空间一致的语义体素-射线地图作为持久记忆；2) 结合短程体素搜索和长程射线搜索；3) 利用视觉语言模型提供辅助线索缓解目标稀疏问题。

Method: 基于行为树的框架，结合语义体素-射线地图和视觉语言模型，自适应切换行为。

Result: 在仿真环境中性能优于基线85.25%，并通过真实空中机器人验证。

Insight: 持久记忆和多模态线索的结合是提升户外语义导航适应性和效率的关键。

Abstract: Aerial outdoor semantic navigation requires robots to explore large, unstructured environments to locate target objects. Recent advances in semantic navigation have demonstrated open-set object-goal navigation in indoor settings, but these methods remain limited by constrained spatial ranges and structured layouts, making them unsuitable for long-range outdoor search. While outdoor semantic navigation approaches exist, they either rely on reactive policies based on current observations, which tend to produce short-sighted behaviors, or precompute scene graphs offline for navigation, limiting adaptability to online deployment. We present RAVEN, a 3D memory-based, behavior tree framework for aerial semantic navigation in unstructured outdoor environments. It (1) uses a spatially consistent semantic voxel-ray map as persistent memory, enabling long-horizon planning and avoiding purely reactive behaviors, (2) combines short-range voxel search and long-range ray search to scale to large environments, (3) leverages a large vision-language model to suggest auxiliary cues, mitigating sparsity of outdoor targets. These components are coordinated by a behavior tree, which adaptively switches behaviors for robust operation. We evaluate RAVEN in 10 photorealistic outdoor simulation environments over 100 semantic tasks, encompassing single-object search, multi-class, multi-instance navigation and sequential task changes. Results show RAVEN outperforms baselines by 85.25% in simulation and demonstrate its real-world applicability through deployment on an aerial robot in outdoor field tests.

[447] Focusing on What Matters: Object-Agent-centric Tokenization for Vision Language Action models

Rokas Bendikas,Daniel Dijkman,Markus Peschl,Sanjay Haresh,Pietro Mazzaglia

Main category: cs.RO

TL;DR: 论文提出了Oat-VLA方法，通过对象-代理中心的视觉输入分词化方案，显著降低了Vision-Language-Action（VLA）模型的训练计算成本，同时保持了性能。

Details Motivation: 当前Vision-Language-Action（VLA）模型在适应机器人领域时，由于视觉输入的分词化方案问题，导致计算成本过高。作者希望通过改进分词化方法提高效率。

Contribution: 提出了Oat-VLA方法，一种面向对象和代理的分词化方案，显著减少了视觉token数量，提升了训练效率和模型性能。

Method: 基于对象中心表示学习的思路，Oat-VLA引入了对场景对象和代理视觉信息的归纳偏置，从而减少了不必要的token数量。

Result: 实验表明，Oat-VLA在LIBERO测试套件上收敛速度至少快两倍，并在真实世界的拾放任务中优于OpenVLA。

Insight: 对象和代理中心的视觉分词化能够显著提升机器人领域的模型效率，同时不牺牲性能。

Abstract: Vision-Language-Action (VLA) models offer a pivotal approach to learning robotic manipulation at scale by repurposing large pre-trained Vision-Language-Models (VLM) to output robotic actions. However, adapting VLMs for robotic domains comes with an unnecessarily high computational cost, which we attribute to the tokenization scheme of visual inputs. In this work, we aim to enable efficient VLA training by proposing Oat-VLA, an Object-Agent-centric Tokenization for VLAs. Building on the insights of object-centric representation learning, our method introduces an inductive bias towards scene objects and the agent’s own visual information. As a result, we find that Oat-VLA can drastically reduce the number of visual tokens to just a few tokens without sacrificing performance. We reveal that Oat-VLA converges at least twice as fast as OpenVLA on the LIBERO suite, as well as outperform OpenVLA in diverse real-world pick and place tasks.

[448] Mash, Spread, Slice! Learning to Manipulate Object States via Visual Spatial Progress

Priyanka Mandikal,Jiaheng Hu,Shivin Dass,Sagnik Majumder,Roberto Martín-Martín,Kristen Grauman

Main category: cs.RO

TL;DR: SPARTA是一种统一的框架，用于处理物体状态变化的操作任务，如糊状化、铺展或切片。通过空间渐进的视觉分割和强化学习，它能够高效地感知和操控物体状态的变化。

Details Motivation: 大多数机器人操作任务专注于物体的运动状态变化，如抓取、放置或旋转，而忽略了物理和视觉状态逐渐变化的任务（如糊状化、铺展）。SPARTA旨在填补这一空白。

Contribution: SPARTA是首个统一处理物体状态变化任务的框架，其核心贡献包括空间渐进的变化分割、结构化策略观察和稠密奖励机制。

Method: SPARTA通过视觉分割映射捕捉物体状态的变化，生成结构化观察和稠密奖励。方法包括强化学习和贪婪控制两种策略。

Result: 在10种真实物体的3个任务中，SPARTA显著优于稀疏奖励和视觉目标基线，训练时间和精度均有提升。

Insight: 空间渐进的视觉表示是广泛物体状态操作任务的通用基础，能够高效地感知和控制物体状态的变化。

Abstract: Most robot manipulation focuses on changing the kinematic state of objects: picking, placing, opening, or rotating them. However, a wide range of real-world manipulation tasks involve a different class of object state change–such as mashing, spreading, or slicing–where the object’s physical and visual state evolve progressively without necessarily changing its position. We present SPARTA, the first unified framework for the family of object state change manipulation tasks. Our key insight is that these tasks share a common structural pattern: they involve spatially-progressing, object-centric changes that can be represented as regions transitioning from an actionable to a transformed state. Building on this insight, SPARTA integrates spatially progressing object change segmentation maps, a visual skill to perceive actionable vs. transformed regions for specific object state change tasks, to generate a) structured policy observations that strip away appearance variability, and b) dense rewards that capture incremental progress over time. These are leveraged in two SPARTA policy variants: reinforcement learning for fine-grained control without demonstrations or simulation; and greedy control for fast, lightweight deployment. We validate SPARTA on a real robot for three challenging tasks across 10 diverse real-world objects, achieving significant improvements in training time and accuracy over sparse rewards and visual goal-conditioned baselines. Our results highlight progress-aware visual representations as a versatile foundation for the broader family of object state manipulation tasks. Project website: https://vision.cs.utexas.edu/projects/sparta-robot

[449] PROFusion: Robust and Accurate Dense Reconstruction via Camera Pose Regression and Optimization

Siyan Dong,Zijun Wang,Lulu Cai,Yi Ma,Yanchao Yang

Main category: cs.RO

TL;DR: 论文提出了一种结合学习与优化的方法，通过相机位姿回归网络提供初步位姿估计，再通过优化算法精细化深度图像对齐，从而在相机运动不稳定时实现高精度的稠密重建。

Details Motivation: 现有的RGB-D SLAM系统在相机运动不稳定（如大视角变化或快速抖动）时表现不佳：传统优化方法需要良好的初始化但难以处理大运动，而学习型方法鲁棒性较强但精度不足。因此，作者提出了结合两者的方案。

Contribution: 主要贡献包括：1）提出了一种学习与优化结合的稠密重建框架；2）设计了一个相机位姿回归网络以提供可靠的初始位姿；3）引入随机优化算法精细化对齐深度图像。

Method: 方法分为两部分：1）使用相机位姿回归网络预测连续的RGB-D帧之间的相对位姿；2）通过随机优化算法将深度图像进一步与场景几何对齐。

Result: 实验表明，该方法在具有挑战性的基准测试中优于最佳竞争对手，同时在稳定运动序列中保持相当的精度，且能实时运行。

Insight: 论文表明，简单且原理明确的技术（学习初始化+优化细化）的结合，可以同时实现不稳定运动下的鲁棒性和稠密重建的高精度。

Abstract: Real-time dense scene reconstruction during unstable camera motions is crucial for robotics, yet current RGB-D SLAM systems fail when cameras experience large viewpoint changes, fast motions, or sudden shaking. Classical optimization-based methods deliver high accuracy but fail with poor initialization during large motions, while learning-based approaches provide robustness but lack sufficient accuracy for dense reconstruction. We address this challenge through a combination of learning-based initialization with optimization-based refinement. Our method employs a camera pose regression network to predict metric-aware relative poses from consecutive RGB-D frames, which serve as reliable starting points for a randomized optimization algorithm that further aligns depth images with the scene geometry. Extensive experiments demonstrate promising results: our approach outperforms the best competitor on challenging benchmarks, while maintaining comparable accuracy on stable motion sequences. The system operates in real-time, showcasing that combining simple and principled techniques can achieve both robustness for unstable motions and accuracy for dense reconstruction. Project page: https://github.com/siyandong/PROFusion.

[450] DRCP: Diffusion on Reinforced Cooperative Perception for Perceiving Beyond Limits

Lantao Li,Kang Yang,Rui Song,Chen Sun

Main category: cs.RO

TL;DR: DRCP提出了一种实时可部署的框架，通过跨模态合作感知和轻量级扩散优化模块，提升自动驾驶在极端条件下的感知能力。

Details Motivation: 虽然在多智能体融合和感知主干网络方面已取得进展，但在实际部署中仍面临部分检测和噪声累积等问题，限制了检测精度。

Contribution: 1) 提出了Precise-Pyramid-Cross-Modality-Cross-Agent模块，优化跨模态合作感知；2) 引入了Mask-Diffusion-Mask-Aggregation模块，通过轻量级扩散提升特征的鲁棒性。

Method: 结合跨模态注意力融合和自适应卷积利用外部特征，并通过扩散模型优化鸟瞰图特征，使其更接近任务最优流形。

Result: 在移动平台上实现了实时性能，并在极端条件下显著提升了鲁棒性。

Insight: 扩散模型不仅可以用于生成任务，还能有效提升感知任务的鲁棒性和精度。

Abstract: Cooperative perception enabled by Vehicle-to-Everything communication has shown great promise in enhancing situational awareness for autonomous vehicles and other mobile robotic platforms. Despite recent advances in perception backbones and multi-agent fusion, real-world deployments remain challenged by hard detection cases, exemplified by partial detections and noise accumulation which limit downstream detection accuracy. This work presents Diffusion on Reinforced Cooperative Perception (DRCP), a real-time deployable framework designed to address aforementioned issues in dynamic driving environments. DRCP integrates two key components: (1) Precise-Pyramid-Cross-Modality-Cross-Agent, a cross-modal cooperative perception module that leverages camera-intrinsic-aware angular partitioning for attention-based fusion and adaptive convolution to better exploit external features; and (2) Mask-Diffusion-Mask-Aggregation, a novel lightweight diffusion-based refinement module that encourages robustness against feature perturbations and aligns bird’s-eye-view features closer to the task-optimal manifold. The proposed system achieves real-time performance on mobile platforms while significantly improving robustness under challenging conditions. Code will be released in late 2025.

[451] AIRoA MoMa Dataset: A Large-Scale Hierarchical Dataset for Mobile Manipulation

Ryosuke Takanami,Petr Khrapchenkov,Shu Morikuni,Jumpei Arima,Yuta Takaba,Shunsuke Maeda,Takuya Okubo,Genki Sano,Satoshi Sekioka,Aoi Kadoya,Motonari Kambara,Naoya Nishiura,Haruto Suzuki,Takanori Yoshimoto,Koya Sakamoto,Shinnosuke Ono,Hu Yang,Daichi Yashima,Aoi Horo,Tomohiro Motoda,Kensuke Chiyoma,Hiroshi Ito,Koki Fukuda,Akihito Goto,Kazumi Morinaga,Yuya Ikeda,Riko Kawada,Masaki Yoshikawa,Norio Kosuge,Yuki Noguchi,Kei Ota,Tatsuya Matsushima,Yusuke Iwasawa,Yutaka Matsuo,Tetsuya Ogata

Main category: cs.RO

TL;DR: AIRoA MoMa Dataset是一个大规模的多模态数据集，专注于移动操作任务，包含同步多模态数据和分层标注，旨在推动Vision-Language-Action模型的发展。

Details Motivation: 机器人从受控环境转向非结构化人类环境时，需要通用的智能体能够可靠地执行自然语言指令。现有数据集缺乏同步力扭矩感知、分层标注和显式失败案例，限制了研究进展。

Contribution: 1. 提出了AIRoA MoMa Dataset，包含同步的RGB图像、关节状态、六轴腕力扭矩信号等；2. 引入两层标注架构（子目标和原始动作），支持分层学习和错误分析；3. 数据集规模大，包含25,469个任务片段（约94小时）。

Method: 数据集由Human Support Robot (HSR)收集，标准化为LeRobot v2.1格式，整合了移动操作、接触丰富的交互和长时程任务结构。

Result: 数据集首次公开，为Vision-Language-Action模型提供了重要基准。

Insight: 同步的多模态数据和分层标注设计可显著提升模型的鲁棒性和泛化能力。

Abstract: As robots transition from controlled settings to unstructured human environments, building generalist agents that can reliably follow natural language instructions remains a central challenge. Progress in robust mobile manipulation requires large-scale multimodal datasets that capture contact-rich and long-horizon tasks, yet existing resources lack synchronized force-torque sensing, hierarchical annotations, and explicit failure cases. We address this gap with the AIRoA MoMa Dataset, a large-scale real-world multimodal dataset for mobile manipulation. It includes synchronized RGB images, joint states, six-axis wrist force-torque signals, and internal robot states, together with a novel two-layer annotation schema of sub-goals and primitive actions for hierarchical learning and error analysis. The initial dataset comprises 25,469 episodes (approx. 94 hours) collected with the Human Support Robot (HSR) and is fully standardized in the LeRobot v2.1 format. By uniquely integrating mobile manipulation, contact-rich interaction, and long-horizon structure, AIRoA MoMa provides a critical benchmark for advancing the next generation of Vision-Language-Action models. The first version of our dataset is now available at https://huggingface.co/datasets/airoa-org/airoa-moma .

cs.IR [Back]

[452] Retro*: Optimizing LLMs for Reasoning-Intensive Document Retrieval

Junwei Lan,Jianlyu Chen,Zheng Liu,Chaofan Li,Siqi Bao,Defu Lian

Main category: cs.IR

TL;DR: Retro*提出了一种新的文档检索方法，专注于通过细粒度推理评估任务与文档的相关性，结合评分机制和多推理轨迹整合，优化大型语言模型在推理密集型任务中的表现，显著提升了BRIGHT基准上的性能。

Details Motivation: 随着LLM代理和RAG的普及，需要检索与任务间接或隐式相关的文档，这对现有IR技术提出了挑战。Retro*旨在解决现有方法在适用性、可扩展性和效率上的不足。

Contribution: 1. 提出了一种基于评分的相关性评分机制，支持细粒度和可解释的相关性评估；2. 支持通过多推理轨迹整合提高测试时的可靠性；3. 设计了一种针对评分机制的强化学习算法，充分利用训练样本轨迹。

Method: Retro*结合了评分机制和多推理轨迹整合，利用强化学习优化相关性评分，其中使用复合奖励来充分利用训练样本的轨迹。

Result: 在BRIGHT基准上，Retro*显著优于现有文档检索方法，达到了最新的性能水平。

Insight: Retro*的评分机制和多推理轨迹整合为推理密集型文档检索提供了新的思路，强化学习的优化进一步提升了模型的推理能力。

Abstract: With the growing popularity of LLM agents and RAG, it has become increasingly important to retrieve documents that are essential for solving a task, even when their connection to the task is indirect or implicit. Addressing this problem requires fine-grained reasoning to accurately assess the relevance between the task and each candidate document. This capability, however, poses a significant challenge for existing IR techniques. Despite recent progress in reasoning-enhanced IR, existing approaches still face significant challenges in applicability, scalability, and efficiency. In this work, we propose Retro*, a novel approach for reasoning-intensive document retrieval. Our method introduces a rubric-based relevance scoring mechanism, enabling the model to reason about the relationship between a task and a document based on explicitly defined criteria, whereby producing a fine-grained, interpretable relevance score. Retro* also supports test-time scaling by combining multiple reasoning trajectories via score integration, which produces more reliable relevance estimates. To optimize Retro*’s reasoning capabilities, we introduce a novel reinforcement learning algorithm tailored for its relevance scoring mechanism, which employs two composite rewards to fully exploit the trajectories of each training sample. Our experiments show that Retro* outperforms existing document retrieval methods with notable advantages, leading to state-of-the-art performance on the BRIGHT benchmark.

eess.IV [Back]

[453] S$^3$F-Net: A Multi-Modal Approach to Medical Image Classification via Spatial-Spectral Summarizer Fusion Network

Md. Saiful Bari Siddiqui,Mohammed Imamul Hassan Bhuiyan

Main category: eess.IV

TL;DR: S$^3$F-Net提出了一种多模态医学图像分类方法，通过融合空间和频域特征提升分类性能，显著优于仅使用空间特征的基线模型。

Details Motivation: 传统CNN主要关注空间特征，忽略了频域信息的重要性，而这在医学图像分析中可能对捕捉全局模式至关重要。

Contribution: 提出了S$^3$F-Net，一个双分支框架，同时学习空间和频域特征，并通过高效的SpectralFilter层和轻量级Summarizer网络实现频域特征的提取与融合。

Method: 使用空间CNN分支和频域分支SpectraNet，通过SpectralFilter层和Bilinear/Concatenation Fusion方法实现特征融合。

Result: 在多个医学影像数据集中表现出色，最高提升5.13%的准确率，并在BRISC2025数据集上达到98.76%的SOTA水平。

Insight: 双域特征融合能够根据输入病理动态调整对空间和频域分支的依赖，证明了频域信息在医学图像分析中的重要性。

Abstract: Convolutional Neural Networks have become a cornerstone of medical image analysis due to their proficiency in learning hierarchical spatial features. However, this focus on a single domain is inefficient at capturing global, holistic patterns and fails to explicitly model an image’s frequency-domain characteristics. To address these challenges, we propose the Spatial-Spectral Summarizer Fusion Network (S$^3$F-Net), a dual-branch framework that learns from both spatial and spectral representations simultaneously. The S$^3$F-Net performs a fusion of a deep spatial CNN with our proposed shallow spectral encoder, SpectraNet. SpectraNet features the proposed SpectralFilter layer, which leverages the Convolution Theorem by applying a bank of learnable filters directly to an image’s full Fourier spectrum via a computation-efficient element-wise multiplication. This allows the SpectralFilter layer to attain a global receptive field instantaneously, with its output being distilled by a lightweight summarizer network. We evaluate S$^3$F-Net across four medical imaging datasets spanning different modalities to validate its efficacy and generalizability. Our framework consistently and significantly outperforms its strong spatial-only baseline in all cases, with accuracy improvements of up to 5.13%. With a powerful Bilinear Fusion, S$^3$F-Net achieves a SOTA competitive accuracy of 98.76% on the BRISC2025 dataset. Concatenation Fusion performs better on the texture-dominant Chest X-Ray Pneumonia dataset, achieving 93.11% accuracy, surpassing many top-performing, much deeper models. Our explainability analysis also reveals that the S$^3$F-Net learns to dynamically adjust its reliance on each branch based on the input pathology. These results verify that our dual-domain approach is a powerful and generalizable paradigm for medical image analysis.

[454] ReCon-GS: Continuum-Preserved Guassian Streaming for Fast and Compact Reconstruction of Dynamic Scenes

Jiaye Fu,Qiankun Gao,Chengxiang Wen,Yanmin Wu,Siwei Ma,Jiaqi Zhang,Jian Zhang

Main category: eess.IV

TL;DR: ReCon-GS是一种新型的动态场景重建框架，通过动态分配多级锚点高斯模型和层次重配置策略，显著提高了重建效率和存储效率。

Details Motivation: 在线自由视点视频（FVV）重建面临速度慢、运动估计不一致和存储需求高的问题，ReCon-GS旨在解决这些挑战。

Contribution: 提出ReCon-GS框架，其贡献包括动态分配多级锚点高斯模型、设计层次重配置策略和存储感知优化机制。

Method: 通过密度自适应的多级锚点高斯模型捕捉几何变形，利用动态层次重配置保持运动表达能力，并通过存储感知优化调整存储密度。

Result: 实验表明，ReCon-GS在训练效率上提升约15%，内存需求减少50%以上，同时重建质量优于现有方法。

Insight: 层次化的变形表示和存储优化机制是实现高效动态场景重建的关键。

Abstract: Online free-viewpoint video (FVV) reconstruction is challenged by slow per-frame optimization, inconsistent motion estimation, and unsustainable storage demands. To address these challenges, we propose the Reconfigurable Continuum Gaussian Stream, dubbed ReCon-GS, a novel storage-aware framework that enables high fidelity online dynamic scene reconstruction and real-time rendering. Specifically, we dynamically allocate multi-level Anchor Gaussians in a density-adaptive fashion to capture inter-frame geometric deformations, thereby decomposing scene motion into compact coarse-to-fine representations. Then, we design a dynamic hierarchy reconfiguration strategy that preserves localized motion expressiveness through on-demand anchor re-hierarchization, while ensuring temporal consistency through intra-hierarchical deformation inheritance that confines transformation priors to their respective hierarchy levels. Furthermore, we introduce a storage-aware optimization mechanism that flexibly adjusts the density of Anchor Gaussians at different hierarchy levels, enabling a controllable trade-off between reconstruction fidelity and memory usage. Extensive experiments on three widely used datasets demonstrate that, compared to state-of-the-art methods, ReCon-GS improves training efficiency by approximately 15% and achieves superior FVV synthesis quality with enhanced robustness and stability. Moreover, at equivalent rendering quality, ReCon-GS slashes memory requirements by over 50% compared to leading state-of-the-art methods.

[455] Wavelet-Assisted Mamba for Satellite-Derived Sea Surface Temperature Super-Resolution

Wankun Chen,Feng Gao,Yanhai Gan,Jingchao Cao,Junyu Dong,Qian Du

Main category: eess.IV

TL;DR: 该论文提出了一种基于小波辅助Mamba的卫星海表温度超分辨率框架（WMSR），通过结合低频状态空间模块（LFSSM）和高频增强模块（HFEM），显著提升了海表温度数据的超分辨率性能。

Details Motivation: 海表温度（SST）是气候变化的重要指标，但由于物理成像的限制，获取高分辨率SST数据具有挑战性。现有的深度学习方法在建模长距离依赖时存在不足，而Mamba模型基于状态空间模型（SSM），具有线性复杂度潜力，但其在SST超分辨率中的应用尚未充分探索。

Contribution: 提出了WMSR框架，结合低频状态空间模块和高频增强模块，充分利用SSM的全局建模能力和高频纹理校正能力，显著提升了SST数据的超分辨率质量。

Method: 1. LFSSM：使用2D-SSM捕捉输入数据的全局信息，保留低频关键温度信息。2. HFEM：采用像素差异卷积匹配和校正高频特征，生成清晰纹理。

Result: 在三个SST数据集上的实验表明，WMSR优于现有最先进方法，展现了更高的超分辨率性能。

Insight: 将小波分析与Mamba模型结合，能够有效处理SST数据中的全局低频信息和局部高频纹理，为遥感图像超分辨率提供了新思路。

Abstract: Sea surface temperature (SST) is an essential indicator of global climate change and one of the most intuitive factors reflecting ocean conditions. Obtaining high-resolution SST data remains challenging due to limitations in physical imaging, and super-resolution via deep neural networks is a promising solution. Recently, Mamba-based approaches leveraging State Space Models (SSM) have demonstrated significant potential for long-range dependency modeling with linear complexity. However, their application to SST data super-resolution remains largely unexplored. To this end, we propose the Wavelet-assisted Mamba Super-Resolution (WMSR) framework for satellite-derived SST data. The WMSR includes two key components: the Low-Frequency State Space Module (LFSSM) and High-Frequency Enhancement Module (HFEM). The LFSSM uses 2D-SSM to capture global information of the input data, and the robust global modeling capabilities of SSM are exploited to preserve the critical temperature information in the low-frequency component. The HFEM employs the pixel difference convolution to match and correct the high-frequency feature, achieving accurate and clear textures. Through comprehensive experiments on three SST datasets, our WMSR demonstrated superior performance over state-of-the-art methods. Our codes and datasets will be made publicly available at https://github.com/oucailab/WMSR.

[456] A Novel Preprocessing Unit for Effective Deep Learning based Classification and Grading of Diabetic Retinopathy

Pranoti Nage,Sanjay Shitole

Main category: eess.IV

TL;DR: 该论文提出了一种新的预处理单元，结合改进的Mask RCNN和SSA-VGG-16分类器，用于糖尿病视网膜病变（DR）和糖尿病性黄斑水肿（DME）的早期检测和分级。

Details Motivation: 早期检测DR和DME对防止视力损失至关重要，但现有方法在噪声过滤、对比度增强和特征提取方面存在不足。

Contribution: 1. 提出了一种新型自适应可变距离斑点（AVDS）滤波器，结合多种距离计算方法提升对比度；2. 使用改进的Mask RCNN进行分割；3. 提出SSA-VGG-16分类器，结合空间注意力机制提升分类效果。

Method: 1. 预处理阶段：模糊滤波去噪、非线性扩散滤波去除伪影、AVDS滤波器增强对比度；2. 分割阶段：改进的Mask RCNN；3. 分类阶段：SSA-VGG-16提取特征并进行分类。

Result: 在两个数据集（IDRiD和MESSIDOR）上验证了方法的有效性，AVDS滤波器中汉明距离方法在对比度提升方面表现最佳，欧氏距离在PSNR上表现更好。

Insight: 自适应距离方法的选择对预处理效果至关重要，空间注意力机制的引入有助于捕捉图像中的关键区域，提升分类性能。

Abstract: Early detection of diabetic retinopathy (DR) is crucial as it allows for timely intervention, preventing vision loss and enabling effective management of diabetic complications. This research performs detection of DR and DME at an early stage through the proposed framework which includes three stages: preprocessing, segmentation, feature extraction, and classification. In the preprocessing stage, noise filtering is performed by fuzzy filtering, artefact removal is performed by non-linear diffusion filtering, and the contrast improvement is performed by a novel filter called Adaptive Variable Distance Speckle (AVDS) filter. The AVDS filter employs four distance calculation methods such as Euclidean, Bhattacharya, Manhattan, and Hamming. The filter adaptively chooses a distance method which produces the highest contrast value amongst all 3 methods. From the analysis, hamming distance method was found to achieve better results for contrast and Euclidean distance showing less error value with high PSNR. The segmentation stage is performed using Improved Mask-Regional Convolutional Neural Networks (Mask RCNN). In the final stage, feature extraction and classification using novel Self-Spatial Attention infused VGG-16 (SSA-VGG-16), which effectively captures both global contextual relationships and critical spatial regions within retinal images, thereby improving the accuracy and robustness of DR and DME detection and grading. The effectiveness of the proposed method is assessed using two distinct datasets: IDRiD and MESSIDOR.

cs.MA [Back]

[457] MAS$^2$: Self-Generative, Self-Configuring, Self-Rectifying Multi-Agent Systems

Kun Wang,Guibin Zhang,ManKit Ye,Xinyu Deng,Dongxia Wang,Xiaobin Hu,Jinyang Guo,Yang Liu,Yufei Guo

Main category: cs.MA

TL;DR: MAS^2是一个基于递归自生成原则的多智能体系统，能够自主构建适应动态环境的定制化多智能体系统，通过生成-实现-修正三元组实现实时任务需求下的动态组合与自适应调整。

Details Motivation: 现有的多智能体系统多为一次性生成部署模式，无法应对动态和不确定性环境，限制了其在实际应用中的表现。

Contribution: 提出MAS^2框架，通过递归自生成和动态调整机制，解决了现有系统的僵化问题，提升了性能与适应性。

Method: 采用生成-实现-修正三元组策略，结合协作树优化训练，动态组合与调整目标智能体系统。

Result: 在7个基准测试中性能提升高达19.6%，且能有效利用新LLM实现15.1%的性能增益，同时保持高效的成本效益。

Insight: 递归自生成和动态调整是多智能体系统适应复杂环境的关键策略。

Abstract: The past two years have witnessed the meteoric rise of Large Language Model (LLM)-powered multi-agent systems (MAS), which harness collective intelligence and exhibit a remarkable trajectory toward self-evolution. This paradigm has rapidly progressed from manually engineered systems that require bespoke configuration of prompts, tools, roles, and communication protocols toward frameworks capable of automated orchestration. Yet, dominant automatic multi-agent systems, whether generated by external modules or a single LLM agent, largely adhere to a rigid \textit{generate-once-and-deploy}'' paradigm, rendering the resulting systems brittle and ill-prepared for the dynamism and uncertainty of real-world environments. To transcend this limitation, we introduce MAS$^2$, a paradigm predicated on the principle of recursive self-generation: a multi-agent system that autonomously architects bespoke multi-agent systems for diverse problems. Technically, we devise a \textit{generator-implementer-rectifier}’’ tri-agent team capable of dynamically composing and adaptively rectifying a target agent system in response to real-time task demands. Collaborative Tree Optimization is proposed to train and specialize these meta-agents. Extensive evaluation across seven benchmarks reveals that MAS$^2$ achieves performance gains of up to $19.6%$ over state-of-the-art MAS in complex scenarios such as deep research and code generation. Moreover, MAS$^2$ exhibits superior cross-backbone generalization, effectively leveraging previously unseen LLMs to yield improvements of up to $15.1%$. Crucially, these gains are attained without incurring excessive token costs, as MAS$^2$ consistently resides on the Pareto frontier of cost-performance trade-offs. The source codes are available at https://github.com/yeyeyeah2/MAS2.

cs.SD [Back]

[458] DiaMoE-TTS: A Unified IPA-Based Dialect TTS Framework with Mixture-of-Experts and Parameter-Efficient Zero-Shot Adaptation

Ziqi Chen,Gongyu Chen,Yihua Wang,Chaofan Ding,Zihao chen,Wei-Qiang Zhang

Main category: cs.SD

TL;DR: DiaMoE-TTS是一个基于国际音标（IPA）的统一方言TTS框架，通过混合专家（MoE）和参数高效零样本适应技术，解决方言数据稀缺和语音复杂性挑战。

Details Motivation: 方言语音具有丰富文化和语言多样性，但方言TTS系统面临数据稀缺、拼写不一致和语音复杂性等问题，需要高效解决方案。

Contribution: 提出DiaMoE-TTS框架，统一IPA表示，引入方言感知MoE建模语音差异，并通过LoRA和条件适配器实现参数高效零样本适应。

Method: 基于F5-TTS架构，结合MoE和LoRA等技术，标准化语音表示并高效适应新方言。

Result: 实验表明该系统能以少量数据生成自然语音，实现零样本适应未见方言和专业领域（如京剧）。

Insight: IPA标准化和参数高效适应技术可显著提升方言TTS的可扩展性和数据利用率。

Abstract: Dialect speech embodies rich cultural and linguistic diversity, yet building text-to-speech (TTS) systems for dialects remains challenging due to scarce data, inconsistent orthographies, and complex phonetic variation. To address these issues, we present DiaMoE-TTS, a unified IPA-based framework that standardizes phonetic representations and resolves grapheme-to-phoneme ambiguities. Built upon the F5-TTS architecture, the system introduces a dialect-aware Mixture-of-Experts (MoE) to model phonological differences and employs parameter-efficient adaptation with Low-Rank Adaptors (LoRA) and Conditioning Adapters for rapid transfer to new dialects. Unlike approaches dependent on large-scale or proprietary resources, DiaMoE-TTS enables scalable, open-data-driven synthesis. Experiments demonstrate natural and expressive speech generation, achieving zero-shot performance on unseen dialects and specialized domains such as Peking Opera with only a few hours of data.

[459] MGM-Omni: Scaling Omni LLMs to Personalized Long-Horizon Speech

Chengyao Wang,Zhisheng Zhong,Bohao Peng,Senqiao Yang,Yuqi Liu,Haokun Gui,Bin Xia,Jingyao Li,Bei Yu,Jiaya Jia

Main category: cs.SD

TL;DR: MGM-Omni是一种统一的Omni LLM，支持多模态理解和长时程语音生成。其双轨令牌架构分离了多模态推理和实时语音生成，提升了效率和跨模态交互能力。

Details Motivation: 现有语音合成多采用级联管道，缺乏统一的端到端解决方案。MGM-Omni旨在解决这一问题，实现高效的多模态理解和可控的个性化语音生成。

Contribution: 1. 提出双轨令牌架构，解耦多模态推理和语音生成；2. 设计了统一的训练策略和双音频编码器，支持长时程音频感知；3. 提出基于块的并行解码方案，加速推理并支持零样本语音克隆。

Method: 采用双轨令牌架构，包含多模态推理和语音生成两个并行轨道。结合双音频编码器和基于块的并行解码方案，优化训练和推理效率。

Result: 实验表明，MGM-Omni在长时程语音生成、音色稳定性和多模态理解方面优于现有开源模型。

Insight: MGM-Omni为多模态理解和语音生成提供了一种高效统一的范式，有望推动个性化语音技术的发展。

Abstract: We present MGM-Omni, a unified Omni LLM for omni-modal understanding and expressive, long-horizon speech generation. Unlike cascaded pipelines that isolate speech synthesis, MGM-Omni adopts a “brain-mouth” design with a dual-track, token-based architecture that cleanly decouples multimodal reasoning from real-time speech generation. This design enables efficient cross-modal interaction and low-latency, streaming speech generation. For understanding, a unified training strategy coupled with a dual audio encoder design enables long-form audio perception across diverse acoustic conditions. For generation, a chunk-based parallel decoding scheme narrows the text speech token-rate gap, accelerating inference and supporting streaming zero-shot voice cloning with stable timbre over extended durations. Compared to concurrent work, MGM-Omni achieves these capabilities with markedly data-efficient training. Extensive experiments demonstrate that MGM-Omni outperforms existing open source models in preserving timbre identity across extended sequences, producing natural and context-aware speech, and achieving superior long-form audio and omnimodal understanding. MGM-Omni establishes an efficient, end-to-end paradigm for omnimodal understanding and controllable, personalised long-horizon speech generation.

[460] Efficient Audio-Visual Speech Separation with Discrete Lip Semantics and Multi-Scale Global-Local Attention

Kai Li,Kejun Gao,Xiaolin Hu

Main category: cs.SD

TL;DR: 论文提出了一种高效的视听语音分离方法Dolphin，通过轻量化的视觉特征提取和音频分离模块，显著提升了分离质量和效率。

Details Motivation: 现有的视听语音分离方法通常参数量大且计算成本高，难以作为预处理步骤部署在实际应用中。

Contribution: 提出了Dolphin方法，包括轻量化的双路径视频编码器DP-LipCoder和全局-局部注意力分离器，显著减少了参数量和计算成本，同时提升了分离性能。

Method: 1. DP-LipCoder将唇动转化为离散语义令牌；2. 音频分离器采用全局-局部注意力模块捕捉多尺度依赖关系。

Result: 在三个基准数据集上超越SOTA，参数减少50%以上，计算量降低2.4倍，GPU推理速度提升6倍以上。

Insight: 轻量化和高效的注意力机制设计是提升视听语音分离实际应用性的关键。

Abstract: Audio-visual speech separation (AVSS) methods leverage visual cues to extract target speech and have demonstrated strong separation quality in noisy acoustic environments. However, these methods usually involve a large number of parameters and require high computational cost, which is unacceptable in many applications where speech separation serves as only a preprocessing step for further speech processing. To address this issue, we propose an efficient AVSS method, named Dolphin. For visual feature extraction, we develop DP-LipCoder, a dual-path lightweight video encoder that transforms lip-motion into discrete audio-aligned semantic tokens. For audio separation, we construct a lightweight encoder-decoder separator, in which each layer incorporates a global-local attention (GLA) block to efficiently capture multi-scale dependencies. Experiments on three benchmark datasets showed that Dolphin not only surpassed the current state-of-the-art (SOTA) model in separation quality but also achieved remarkable improvements in efficiency: over 50% fewer parameters, more than 2.4x reduction in MACs, and over 6x faster GPU inference speed. These results indicate that Dolphin offers a practical and deployable solution for high-performance AVSS in real-world scenarios. Our code and demo page are publicly available at http://cslikai.cn/Dolphin/.

[461] Discovering “Words” in Music: Unsupervised Learning of Compositional Sparse Code for Symbolic Music

Tianle Wang,Sirui Zhang,Xinyi Tong,Peiyang Yu,Jishang Chen,Liangke Zhao,Xinpu Gao,Yves Zhu,Tiezheng Ge,Bo Zheng,Duo Xu,Yang Liu,Xin Jin,Feng Yu,Songchun Zhu

Main category: cs.SD

TL;DR: 这篇论文提出了一种无监督机器学习算法，从符号音乐数据中发现重复出现的模式（称为音乐”词”），并通过两阶段EM框架优化这些模式的提取。

Details Motivation: 音乐中的重复模式反映了音乐的结构和认知过程，但提取这些模式因语义模糊性而具有挑战性。本文旨在解决这一问题，为AI音乐和音乐学研究提供工具。

Contribution: 提出了一个两阶段EM框架，用于发现音乐”词”并重构数据；验证了最小化编码长度可以减少语义模糊性，揭示了人类编码系统对音乐语义的影响。

Method: 采用两阶段EM算法：1. 开发音乐”词”字典；2. 重构音乐数据。通过统计优化解决语义模糊性问题。

Result: 算法在人类专家标注数据上的IoU得分为0.61，表明其有效性。方法支持AI音乐任务和音乐学分析。

Insight: 最小化编码长度是人类音乐创作的潜在优化原则，为跨风格和作曲家的音乐分析提供了新视角。

Abstract: This paper presents an unsupervised machine learning algorithm that identifies recurring patterns – referred to as music-words'' -- from symbolic music data. These patterns are fundamental to musical structure and reflect the cognitive processes involved in composition. However, extracting these patterns remains challenging because of the inherent semantic ambiguity in musical interpretation. We formulate the task of music-word discovery as a statistical optimization problem and propose a two-stage Expectation-Maximization (EM)-based learning framework: 1. Developing a music-word dictionary; 2. Reconstructing the music data. When evaluated against human expert annotations, the algorithm achieved an Intersection over Union (IoU) score of 0.61. Our findings indicate that minimizing code length effectively addresses semantic ambiguity, suggesting that human optimization of encoding systems shapes musical semantics. This approach enables computers to extract basic building blocks’’ from music data, facilitating structural analysis and sparse encoding. The method has two primary applications. First, in AI music, it supports downstream tasks such as music generation, classification, style transfer, and improvisation. Second, in musicology, it provides a tool for analyzing compositional patterns and offers insights into the principle of minimal encoding across diverse musical styles and composers.

q-bio.QM [Back]

[462] Patient-specific Biomolecular Instruction Tuning

Irsyad Adam,Zekai Chen,David Laub,Shaun Porwal,Arda Pekis,Kevin Brown

Main category: q-bio.QM

TL;DR: 提出了一种名为KRONOS的图-LLM框架，结合分子交互拓扑和蛋白质组学数据，增强了LLM在癌症临床任务中的推理能力，同时发布了首个蛋白质组学指令微调数据集CPTAC-PROTSTRUCT。

Details Motivation: 癌症的个性化治疗需要理解患者特异性蛋白质组学数据，但目前缺乏专门的数据集和模型架构来支持这一目标。

Contribution: 1) 发布首个蛋白质组学指令微调数据集CPTAC-PROTSTRUCT；2) 提出KRONOS框架，结合分子交互图增强LLM的临床推理能力。

Method: KRONOS框架利用分子交互拓扑学和蛋白质组学数据，生成患者特异性图表示，并通过指令微调实现临床任务的推理。

Result: KRONOS在分子分类、时间轨迹建模和肿瘤分期预测等任务中表现优异。

Insight: 通过结合分子交互图和LLM，可以显著提升对患者特异性蛋白质组学数据的理解，推动精准医疗的发展。

Abstract: Proteomics data is essential to pathogenic understanding of a disease phenotype. In cancer, analysis of molecular signatures enables precision medicine through the identification of biological processes that drive individualized tumor progression, therapeutic resistance, and clinical heterogeneity. Recent advances in multimodal large language models (LLMs) have shown remarkable capacity to integrate and reason across heterogeneous data modalities. However, performing multi-modal language modeling for molecular understanding of patient-specific proteomics remains a significant challenge due to two barriers: (1) the lack of instruction-tuning datasets that enable clinical interpretation from proteomics data, and (2) the absence of language modeling architectures designed to capture the rich heterogeneity of molecular data. In this work, we introduce CPTAC-PROTSTRUCT, the first instruction tuning dataset for molecular understanding of oncology, comprising over 400k open-ended examples derived from individualized proteomic profiles curated from the largest national proteomics cancer study (CPTAC). Additionally, we propose KRONOS (Knowledge Representation of patient Omics Networks in Oncology via Structured tuning), a novel graph-LLM framework that leverages molecular interaction topology with proteomics to learn patient-specific graph representations for enhanced clinical reasoning. We show that KRONOS achieves competitive performance across benchmark clinical tasks, including molecular classification, temporal trajectory modeling, and tumor stage prediction from proteomics data. Ultimately, this approach empowers LLMs to understand patient-level pathogenesis, advancing precision medicine through more accurate diagnosis, prognosis, and treatment stratification.

cs.GR [Back]

[463] ZeroScene: A Zero-Shot Framework for 3D Scene Generation from a Single Image and Controllable Texture Editing

Xiang Tang,Ruotong Li,Xiaopeng Fan

Main category: cs.GR

TL;DR: ZeroScene is a zero-shot framework for generating 3D scenes from a single image and enabling controllable texture editing. It leverages large vision models to ensure scene coherence and maintains texture consistency through advanced techniques.

Details Motivation: Existing methods struggle with simultaneous quality of individual assets and scene coherence in 3D reconstruction, as well as maintaining texture consistency during editing.

Contribution: ZeroScene introduces a zero-shot framework for single image-to-3D scene reconstruction and texture editing, utilizing object-level segmentation, depth information, and optimization techniques.

Method: The method extracts 2D segmentation and depth from images to infer spatial relationships, optimizes 3D and 2D projection losses for scene alignment, and uses diffusion models with mask-guided generation for texture editing.

Result: Experiments show the framework achieves geometric and appearance accuracy, faithfully reconstructs scenes, and produces detailed textures aligned with text prompts.

Insight: Leveraging large vision models and combining optimization techniques with diffusion models can significantly enhance the quality and coherence of 3D scene generation and editing.

Abstract: In the field of 3D content generation, single image scene reconstruction methods still struggle to simultaneously ensure the quality of individual assets and the coherence of the overall scene in complex environments, while texture editing techniques often fail to maintain both local continuity and multi-view consistency. In this paper, we propose a novel system ZeroScene, which leverages the prior knowledge of large vision models to accomplish both single image-to-3D scene reconstruction and texture editing in a zero-shot manner. ZeroScene extracts object-level 2D segmentation and depth information from input images to infer spatial relationships within the scene. It then jointly optimizes 3D and 2D projection losses of the point cloud to update object poses for precise scene alignment, ultimately constructing a coherent and complete 3D scene that encompasses both foreground and background. Moreover, ZeroScene supports texture editing of objects in the scene. By imposing constraints on the diffusion model and introducing a mask-guided progressive image generation strategy, we effectively maintain texture consistency across multiple viewpoints and further enhance the realism of rendered results through Physically Based Rendering (PBR) material estimation. Experimental results demonstrate that our framework not only ensures the geometric and appearance accuracy of generated assets, but also faithfully reconstructs scene layouts and produces highly detailed textures that closely align with text prompts.

[464] StrucADT: Generating Structure-controlled 3D Point Clouds with Adjacency Diffusion Transformer

Zhenyu Shu,Jiajun Shen,Zhongui Chen,Xiaoguang Han,Shiqing Xin

Main category: cs.GR

TL;DR: 该论文提出了StrucADT，一种通过结构（部件存在性和相邻关系）控制3D点云生成的方法，解决了现有方法缺乏可控性的问题，实现了高质量、多样化的可控点云生成。

Details Motivation: 现有的3D点云生成方法虽然能生成多样且真实的形状，但缺乏对生成形状的精确控制，限制了其大规模应用。论文提出通过结构（部件及其相邻关系）控制生成，以满足用户特定需求。

Contribution: 1.首次提出通过结构（部件存在性和相邻关系）控制3D点云生成；2.提出StructureGraph表示法；3.设计了StrucADT模型，包含StructureGraphNet、cCNF Prior和Diffusion Transformer模块，实现了结构一致的点云生成。

Method: 1.手动标注点云部件的相邻关系，构建StructureGraph；2.使用StructureGraphNet提取结构感知的潜在特征；3.通过cCNF Prior学习由部件相邻关系控制的特征分布；4.基于潜在特征和部件相邻关系的Diffusion Transformer生成点云。

Result: 实验表明，StrucADT在ShapeNet数据集上实现了高质量的多样化点云生成，支持用户指定形状结构的可控生成，并在可控点云生成领域达到SOTA性能。

Insight: 1.结构控制（部件及其相邻关系）为3D点云生成提供了新的可控维度；2.StructureGraph表示法和StrucADT模型为未来可控生成任务提供了参考。

Abstract: In the field of 3D point cloud generation, numerous 3D generative models have demonstrated the ability to generate diverse and realistic 3D shapes. However, the majority of these approaches struggle to generate controllable 3D point cloud shapes that meet user-specific requirements, hindering the large-scale application of 3D point cloud generation. To address the challenge of lacking control in 3D point cloud generation, we are the first to propose controlling the generation of point clouds by shape structures that comprise part existences and part adjacency relationships. We manually annotate the adjacency relationships between the segmented parts of point cloud shapes, thereby constructing a StructureGraph representation. Based on this StructureGraph representation, we introduce StrucADT, a novel structure-controllable point cloud generation model, which consists of StructureGraphNet module to extract structure-aware latent features, cCNF Prior module to learn the distribution of the latent features controlled by the part adjacency, and Diffusion Transformer module conditioned on the latent features and part adjacency to generate structure-consistent point cloud shapes. Experimental results demonstrate that our structure-controllable 3D point cloud generation method produces high-quality and diverse point cloud shapes, enabling the generation of controllable point clouds based on user-specified shape structures and achieving state-of-the-art performance in controllable point cloud generation on the ShapeNet dataset.

[465] ReLumix: Extending Image Relighting to Video via Video Diffusion Models

Lezhong Wang,Shutong Jin,Ruiqi Cui,Anders Bjorholm Dahl,Jeppe Revall Frisvad,Siavash Bigdeli

Main category: cs.GR

TL;DR: 本文提出了一种名为ReLumix的新方法，通过两阶段处理和视频扩散模型，将图像重光照技术无缝扩展到视频中。

Details Motivation: 视频后期制作中动态光照控制的灵活性不足，现有方法局限于特定重光照模型。ReLumix旨在解决这一问题，提供一种通用的解决方案。

Contribution: 1. 将视频重光照任务解耦为独立的两阶段过程；2. 引入门控交叉注意力机制和时间自举策略，确保时间连贯性；3. 展示了基于合成数据的模型在真实视频中的泛化能力。

Method: 1. 艺术家使用喜欢的图像重光照技术处理参考帧；2. 微调的视频扩散模型（SVD）将目标光照传播到整个序列；3. 提出门控交叉注意力和时间自举策略以优化结果。

Result: ReLumix在视觉保真度上显著提升，能够灵活应用于真实世界视频，展现良好的泛化性能。

Insight: 解耦光照和时序合成任务，结合扩散模型的强大先验，为视频重光照提供了灵活且高质量的解决方案。

Abstract: Controlling illumination during video post-production is a crucial yet elusive goal in computational photography. Existing methods often lack flexibility, restricting users to certain relighting models. This paper introduces ReLumix, a novel framework that decouples the relighting algorithm from temporal synthesis, thereby enabling any image relighting technique to be seamlessly applied to video. Our approach reformulates video relighting into a simple yet effective two-stage process: (1) an artist relights a single reference frame using any preferred image-based technique (e.g., Diffusion Models, physics-based renderers); and (2) a fine-tuned stable video diffusion (SVD) model seamlessly propagates this target illumination throughout the sequence. To ensure temporal coherence and prevent artifacts, we introduce a gated cross-attention mechanism for smooth feature blending and a temporal bootstrapping strategy that harnesses SVD’s powerful motion priors. Although trained on synthetic data, ReLumix shows competitive generalization to real-world videos. The method demonstrates significant improvements in visual fidelity, offering a scalable and versatile solution for dynamic lighting control.

[466] Unsupervised Representation Learning for 3D Mesh Parameterization with Semantic and Visibility Objectives

AmirHossein Zamani,Bruno Roy,Arianna Rampini

Main category: cs.GR

TL;DR: 论文提出了一种无监督学习方法，通过结合语义和可见性目标，自动化3D网格参数化（UV映射）过程，解决了现有方法忽略语义对齐和可见性优化的问题。

Details Motivation: 现有的3D生成模型依赖手动UV映射，耗时且需要专业知识；同时，自动方法忽视了语义对齐和切割缝位置的可见性优化，导致纹理生成效果不佳。

Contribution: 1. 提出了一个无监督可微分框架，结合语义和可见性目标优化UV映射。2. 设计了语义分块和AO加权的切割缝优化方法。3. 公开了代码实现。

Method: 1. 语义分块：将网格划分为语义3D部分。2. 无监督UV参数化：每个部分独立参数化。3. AO加权优化：使用环境光遮蔽（AO）引导切割缝位置。

Result: 实验表明，该方法生成的UV图在纹理生成和减少可见缝方面优于现有基准方法。

Insight: 结合语义和可见性优化显著提升了UV映射的质量，为自动化3D内容生成提供了新思路。

Abstract: Recent 3D generative models produce high-quality textures for 3D mesh objects. However, they commonly rely on the heavy assumption that input 3D meshes are accompanied by manual mesh parameterization (UV mapping), a manual task that requires both technical precision and artistic judgment. Industry surveys show that this process often accounts for a significant share of asset creation, creating a major bottleneck for 3D content creators. Moreover, existing automatic methods often ignore two perceptually important criteria: (1) semantic awareness (UV charts should align semantically similar 3D parts across shapes) and (2) visibility awareness (cutting seams should lie in regions unlikely to be seen). To overcome these shortcomings and to automate the mesh parameterization process, we present an unsupervised differentiable framework that augments standard geometry-preserving UV learning with semantic- and visibility-aware objectives. For semantic-awareness, our pipeline (i) segments the mesh into semantic 3D parts, (ii) applies an unsupervised learned per-part UV-parameterization backbone, and (iii) aggregates per-part charts into a unified UV atlas. For visibility-awareness, we use ambient occlusion (AO) as an exposure proxy and back-propagate a soft differentiable AO-weighted seam objective to steer cutting seams toward occluded regions. By conducting qualitative and quantitative evaluations against state-of-the-art methods, we show that the proposed method produces UV atlases that better support texture generation and reduce perceptible seam artifacts compared to recent baselines. Our implementation code is publicly available at: https://github.com/AHHHZ975/Semantic-Visibility-UV-Param.