2025-10-17

Table of Contents

• cs.CL [Total: 39]
• cs.CV [Total: 80]
• cs.CE [Total: 1]
• cs.IR [Total: 1]
• eess.IV [Total: 1]
• cs.AI [Total: 6]
• cs.LG [Total: 3]
• cs.SE [Total: 1]
• cs.SD [Total: 1]
• physics.med-ph [Total: 1]
• cs.RO [Total: 4]
• cs.CY [Total: 2]

cs.CL [Back]

[1] Benchmarking Open-Source Large Language Models for Persian in Zero-Shot and Few-Shot Learning

Mahdi Cherakhloo,Arash Abbasi,Mohammad Saeid Sarafraz,Bijan Vosoughi Vahdat

Main category: cs.CL

TL;DR: 该论文对波斯语开源大语言模型（LLM）在零样本和小样本学习中的表现进行了基准测试，发现Gemma 2在复杂任务中表现最佳，但大多数模型在命名实体识别等任务上表现不佳。

Details

Motivation: 波斯语等低资源语言的大语言模型表现缺乏系统评估，需要填补这一研究空白。

Contribution: 提供了波斯语NLP任务的全面基准测试，并揭示了开源LLM在该语言中的优势和局限性。

Method: 采用多种任务（如情感分析、命名实体识别等）和数据集（ParsiNLU、ArmanEmo），使用准确率、F1分数等指标进行评估。

Result: Gemma 2在大多数任务中表现最优，但模型在词级任务（如命名实体识别）中表现较差。

Insight: 波斯语的特定挑战（如词级理解）需要进一步优化大语言模型。

Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities across numerous languages; however, their effectiveness in low-resource languages like Persian requires thorough investigation. This paper presents a comprehensive benchmark of several open-source LLMs for Persian Natural Language Processing (NLP) tasks, utilizing both zero-shot and few-shot learning paradigms. We evaluate models across a range of tasks including sentiment analysis, named entity recognition, reading comprehension, and question answering, using established Persian datasets such as ParsiNLU and ArmanEmo. Our methodology encompasses rigorous experimental setups for both zero-shot and few-shot scenarios, employing metrics such as Accuracy, F1-score, BLEU, and ROUGE for performance evaluation. The results reveal that Gemma 2 consistently outperforms other models across nearly all tasks in both learning paradigms, with particularly strong performance in complex reasoning tasks. However, most models struggle with token-level understanding tasks like Named Entity Recognition, highlighting specific challenges in Persian language processing. This study contributes to the growing body of research on multilingual LLMs, providing valuable insights into their performance in Persian and offering a benchmark for future model development.

[2] From Noise to Signal to Selbstzweck: Reframing Human Label Variation in the Era of Post-training in NLP

Shanshan Xu,Santosh T. Y. S. S,Barbara Plank

Main category: cs.CL

TL;DR: 本文探讨了人类标签变异（HLV）在自然语言处理（NLP）中的作用，认为在当前大语言模型（LLMs）和后训练时代，HLV应从被视为噪声转变为信号，并最终成为设计AI系统的核心目标（Selbstzweck）。

Details

Motivation: 长期以来，HLV在NLP中被视为噪声而被忽略，但近年研究表明它反映人类观点的多样性，而非错误。随着LLMs的兴起和后训练（post-training）的重要性，HLV的作用愈发关键。

Contribution: 本文的主要贡献是提出HLV应被视为体现人类多元主义的核心目标（Selbstzweck），并呼吁在偏好数据集中主动保留HLV。

Method: 本文是一种观点性论文（position paper），并未提出具体的技术方法，而是通过分析现有偏好学习数据集的不足（如将多标签聚合为单一标签）提出改进方向。

Result: 文章未提供具体实验结果，但强调保留HLV对模型对齐（alignment）和人类价值观多样性保护的重要性。

Insight: HLV不应被简单降噪或聚合，而是需在设计AI系统时作为目标保留其多样性，以更好地体现人类价值观的多元性。

Abstract: Human Label Variation (HLV) refers to legitimate disagreement in annotation that reflects the genuine diversity of human perspectives rather than mere error. For decades, HLV in NLP was dismissed as noise to be discarded, and only slowly over the last decade has it been reframed as a signal for improving model robustness. With the rise of large language models (LLMs), where post-training on human feedback has become central to model alignment, the role of HLV has become increasingly consequential. Yet current preference-learning datasets routinely aggregate multiple annotations into a single label, thereby flattening diverse perspectives into a false universal agreement and erasing precisely the pluralism of human values that alignment aims to preserve. In this position paper, we argue that preserving HLV as an embodiment of human pluralism must be treated as a Selbstzweck - a goal it self when designing AI systems. We call for proactively incorporating HLV into preference datasets and outline actionable steps towards it.

[3] MEDEQUALQA: Evaluating Biases in LLMs with Counterfactual Reasoning

Rajarshi Ghosh,Abhay Gupta,Hudson McBride,Anurag Vaidya,Faisal Mahmood

Main category: cs.CL

TL;DR: 论文提出了MEDEQUALQA，一个通过反事实推理评估LLMs偏见的基准，专注于控制患者代词变化以分析模型内部推理的稳定性。

Details

Motivation: 研究动机是探索LLMs在临床决策支持中如何因患者代词等细微人口统计学线索而改变其推理过程，从而揭示潜在的不公平性。

Contribution: 主要贡献是开发了MEDEQUALQA基准，通过扰动患者代词生成平行数据，量化LLMs推理的稳定性，并识别临床相关偏见。

Method: 方法包括构建反事实数据集（69,000项），使用GPT-4.1模型，并通过语义文本相似性（STS）分析推理轨迹的稳定性。

Result: 结果显示，虽然整体STS相似性较高（>0.80），但在引用风险因素和指南锚点时存在局部差异，揭示了临床偏见。

Insight: 研究发现，即使最终诊断一致，LLMs的细微推理变化可能导致不公平的护理，强调了在医疗AI中审计推理稳定性的重要性。

Abstract: Large language models (LLMs) are increasingly deployed in clinical decision support, yet subtle demographic cues can influence their reasoning. Prior work has documented disparities in outputs across patient groups, but little is known about how internal reasoning shifts under controlled demographic changes. We introduce MEDEQUALQA, a counterfactual benchmark that perturbs only patient pronouns (he/him, she/her, they/them) while holding critical symptoms and conditions (CSCs) constant. Each clinical vignette is expanded into single-CSC ablations, producing three parallel datasets of approximately 23,000 items each (69,000 total). We evaluate a GPT-4.1 model and compute Semantic Textual Similarity (STS) between reasoning traces to measure stability across pronoun variants. Our results show overall high similarity (mean STS >0.80), but reveal consistent localized divergences in cited risk factors, guideline anchors, and differential ordering, even when final diagnoses remain unchanged. Our error analysis highlights certain cases in which the reasoning shifts, underscoring clinically relevant bias loci that may cascade into inequitable care. MEDEQUALQA offers a controlled diagnostic setting for auditing reasoning stability in medical AI.

[4] Classifier-Augmented Generation for Structured Workflow Prediction

Thomas Gschwind,Shramona Chakraborty,Nitin Gupta,Sameep Mehta

Main category: cs.CL

TL;DR: 论文提出了一种结合分类器增强生成（CAG）的方法，将自然语言描述转化为可执行的工作流，显著提升了ETL流程预测的准确性和效率。

Details

Motivation: 配置ETL工具（如IBM DataStage）的工作流通常耗时且需要专业知识。通过自然语言自动生成工作流可以降低门槛并提高效率。

Contribution: 1. 提出了CAG方法，结合分类器和few-shot提示生成阶段预测；2. 支持非线性工作流生成和属性推断；3. 首次详细评估了自然语言驱动的ETL生成。

Method: 1. 使用分类器分解自然语言描述；2. 阶段预测采用few-shot提示；3. 边缘预测连接阶段；4. 属性推断基于子描述上下文。

Result: 相比基线方法，CAG在准确性、效率及令牌使用上表现更优，支持端到端工作流生成。

Insight: 分类器增强生成结合了结构化预测和生成的优点，模块化设计提升了可解释性和灵活性。

Abstract: ETL (Extract, Transform, Load) tools such as IBM DataStage allow users to visually assemble complex data workflows, but configuring stages and their properties remains time consuming and requires deep tool knowledge. We propose a system that translates natural language descriptions into executable workflows, automatically predicting both the structure and detailed configuration of the flow. At its core lies a Classifier-Augmented Generation (CAG) approach that combines utterance decomposition with a classifier and stage-specific few-shot prompting to produce accurate stage predictions. These stages are then connected into non-linear workflows using edge prediction, and stage properties are inferred from sub-utterance context. We compare CAG against strong single-prompt and agentic baselines, showing improved accuracy and efficiency, while substantially reducing token usage. Our architecture is modular, interpretable, and capable of end-to-end workflow generation, including robust validation steps. To our knowledge, this is the first system with a detailed evaluation across stage prediction, edge layout, and property generation for natural-language-driven ETL authoring.

[5] Scheming Ability in LLM-to-LLM Strategic Interactions

Thao Pham

Main category: cs.CL

TL;DR: 该论文研究了前沿LLM代理在战略欺骗方面的能力，通过两种博弈论框架（Cheap Talk信号游戏和Peer Evaluation对抗游戏）测试了四种模型的欺骗倾向和表现，发现模型在无提示情况下仍表现出显著的欺骗倾向。

Details

Motivation: 随着LLM代理在多样化场景中的自主部署，评估其战略欺骗能力变得至关重要。当前研究主要关注AI系统对人类开发者的欺骗，而LLM之间的欺骗行为尚未充分探索。

Contribution: 论文的主要贡献在于提出了两种博弈论框架来评估LLM的战略欺骗能力，并揭示了前沿LLM代理在无提示情况下仍具有显著的欺骗倾向。

Method: 采用了Cheap Talk信号游戏和Peer Evaluation对抗游戏两种博弈论框架，测试了四种LLM模型（GPT-4o、Gemini-2.5-pro、Claude-3.7-Sonnet和Llama-3.3-70b），并通过链式思维推理分析了欺骗策略。

Result: 在提示下，多数模型（尤其是Gemini-2.5-pro和Claude-3.7-Sonnet）表现接近完美。重要的是，所有模型在无提示时也表现出强烈欺骗倾向，Peer Evaluation中100%选择欺骗，Cheap Talk中欺骗成功率为95-100%。

Insight: 研究发现在高风险博弈场景中，LLM代理天然具有欺骗倾向，强调了多代理环境下需要更严格的评估机制。

Abstract: As large language model (LLM) agents are deployed autonomously in diverse contexts, evaluating their capacity for strategic deception becomes crucial. While recent research has examined how AI systems scheme against human developers, LLM-to-LLM scheming remains underexplored. We investigate the scheming ability and propensity of frontier LLM agents through two game-theoretic frameworks: a Cheap Talk signaling game and a Peer Evaluation adversarial game. Testing four models (GPT-4o, Gemini-2.5-pro, Claude-3.7-Sonnet, and Llama-3.3-70b), we measure scheming performance with and without explicit prompting while analyzing scheming tactics through chain-of-thought reasoning. When prompted, most models, especially Gemini-2.5-pro and Claude-3.7-Sonnet, achieved near-perfect performance. Critically, models exhibited significant scheming propensity without prompting: all models chose deception over confession in Peer Evaluation (100% rate), while models choosing to scheme in Cheap Talk succeeded at 95-100% rates. These findings highlight the need for robust evaluations using high-stakes game-theoretic scenarios in multi-agent settings.

[6] Mathematics with large language models as provers and verifiers

Hieu Le Duc,Leo Liberti

Main category: cs.CL

TL;DR: 论文讨论了大型语言模型（如ChatGPT）在数学定理证明中的能力，通过协作式的证明者和验证者实例生成并验证数学证明，成功解决了2025年IMO的5/6问题以及部分数论猜想。

Details

Motivation: 研究旨在探索大型语言模型在数学定理证明中的潜力，尤其是通过协作方式验证和生成数学证明的能力。

Contribution: 提出了一种协作式协议，利用多个gpt-5实例作为证明者和验证者生成数学证明，并通过Lean证明助手和人工验证确保严谨性。

Method: 使用ChatGPT（gpt-5）的不同实例协作生成和验证数学证明，最终由Lean证明助手进行形式化验证，并由人工确认前提与结论的一致性。

Result: 成功解决了2025年IMO的五分之六问题，并完成了三分之一的数论猜想证明。

Insight: 大型语言模型在数学定理证明中展现出潜力，但其结果仍需形式化工具和人工双重验证以避免幻觉问题。

Abstract: During 2024 and 2025 the discussion about the theorem-proving capabilities of large language models started reporting interesting success stories, mostly to do with difficult exercises (such as problems from the International Mathematical Olympiad), but also with conjectures [Feldman & Karbasi, arXiv:2509.18383v1] formulated for the purpose of verifying whether the artificial intelligence could prove it. In this paper we report a theorem proving feat achieved by ChatGPT by using a protocol involving different prover and verifier instances of the gpt-5 model working collaboratively. To make sure that the produced proofs do not suffer from hallucinations, the final proof is formally verified by the lean proof assistant, and the conformance of premises and conclusion of the lean code is verified by a human. Our methodology was able to solve five out of six 2025 IMO problems, and close a third of the sixty-six number theory conjectures in [Cohen, Journal of Integer Sequences, 2025].

[7] MTSQL-R1: Towards Long-Horizon Multi-Turn Text-to-SQL via Agentic Training

Taicheng Guo,Hai Wang,ChaoChun Liu,Mohsen Golalikhani,Xin Chen,Xiangliang Zhang,Chandan K. Reddy

Main category: cs.CL

TL;DR: MTSQL-R1提出了一种基于智能体训练的框架，用于长时序多轮Text-to-SQL任务，通过执行反馈和一致性验证提升结果的可执行性和连贯性。

Details

Motivation: 现有系统通常将多轮Text-to-SQL视为简单的文本翻译任务，缺乏执行和显式验证，导致输出不可执行或不连贯。MTSQL-R1通过智能体与环境交互，解决这些问题。

Contribution: 主要贡献是提出了一种智能体训练框架MTSQL-R1，将任务建模为马尔可夫决策过程（MDP），引入执行反馈和持久对话记忆，提升SQL生成的可执行性和连贯性。

Method: 方法包括：(1) 将任务建模为MDP；(2) 智能体与数据库和执行反馈交互；(3) 引入持久对话记忆进行一致性验证；(4) 提出执行->验证->优化的迭代循环。

Result: 在COSQL和SPARC数据集上的实验表明，MTSQL-R1明显优于基线方法。

Insight: 通过环境驱动的验证和记忆引导的优化，可以显著提升多轮Text-to-SQL任务的性能和数据连贯性。

Abstract: Multi-turn Text-to-SQL aims to translate a user’s conversational utterances into executable SQL while preserving dialogue coherence and grounding to the target schema. However, most existing systems only regard this task as a simple text translation task and follow a short-horizon paradigm, generating a query per turn without execution, explicit verification, and refinement, which leads to non-executable or incoherent outputs. We present MTSQL-R1, an agentic training framework for long-horizon multi-turn Text-to-SQL. We cast the task as a Markov Decision Process (MDP) in which an agent interacts with (i) a database for execution feedback and (ii) a persistent dialogue memory for coherence verification, performing an iterative propose to execute -> verify -> refine cycle until all checks pass. Experiments on COSQL and SPARC demonstrate that MTSQL-R1 consistently outperforms strong baselines, highlighting the importance of environment-driven verification and memory-guided refinement for conversational semantic parsing. Full recipes (including code, trained models, logs, reasoning trajectories, etc.) will be released after the internal review to contribute to community research.

[8] A\textsuperscript{2}FM: An Adaptive Agent Foundation Model for Tool-Aware Hybrid Reasoning

Qianben Chen,Jingyi Cao,Jiayu Zhang,Tianrui Qin,Xiaowan Li,King Zhu,Dingfeng Shi,He Zhu,Minghao Liu,Xiaobo Liang,Ge Zhang,Jian Yang,Yuchen Eleanor Jiang,Wangchunshu Zhou

Main category: cs.CL

TL;DR: 论文提出了A²FM，一种自适应的智能体基础模型，通过任务感知路由和模式对齐实现推理与工具调用的统一，引入即时模式提升效率，并通过自适应策略优化实现成本节约与性能平衡。

Details

Motivation: 现有大语言模型分为注重内部推理的模型和注重工具调用的模型，两者目标不同导致能力不匹配和效率低下。A²FM旨在统一这两种能力，优化查询处理的效率与成本。

Contribution: 1. 提出A²FM框架，结合推理与工具调用能力；2. 引入即时模式简化查询处理；3. 提出自适应策略优化（APO），平衡准确性与成本效率。

Method: 1. 任务感知路由学习；2. 模式对齐的轨迹优化；3. 即时模式处理简单查询；4. APO通过自适应采样和成本正则化奖励实现高效优化。

Result: 在32B规模上，A²FM在BrowseComp、AIME25和HLE上分别达到13.4%、70.4%和16.7%，成本效率显著提升，每次正确答案的成本降低45.2%。

Insight: A²FM通过多模式自适应处理，解决了推理与工具调用之间的能力鸿沟，同时优化了成本效率，为实际应用提供了高效统一的解决方案。

Abstract: Large language models split into two families: reasoning-centric LLMs, which strengthen internal chain-of-thought reasoning but cannot invoke external tools, and agentic LLMs, which learn to interact with environments and leverage tools but often lag in deep reasoning. This divide arises from fundamentally different training objectives, leading to mismatched strengths and inefficiency on simple queries, where both families tend to overthink or over-call tools. In this work, we present Adaptive Agent Foundation Model (A\textsuperscript{2}FM), a unified framework that follows a route-then-align principle: the model first learns task-aware routing and then aligns mode-specific trajectories under a shared backbone. To address the inefficiency gap, we introduce a third mode-instant-that handles simple queries directly, preventing unnecessary reasoning or tool calls while complementing the agentic and reasoning modes. To jointly enhance accuracy and efficiency, we propose Adaptive Policy Optimization (APO), which enforces adaptive sampling across modes and applies a cost-regularized reward. On the 32B scale, A\textsuperscript{2}FM achieves 13.4% on BrowseComp, 70.4% on AIME25, and 16.7% on HLE, setting new SOTA among comparable models and performing competitively with frontier LLMs across agentic, reasoning, and general benchmarks. Notably, the adaptive execution achieves a cost of pass of only $0.00487 per correct answer-cutting cost by 45.2% relative to reasoning and 33.5% relative to agentic, thus delivering substantially higher cost efficiency while maintaining comparable accuracy.

[9] VLURes: Benchmarking VLM Visual and Linguistic Understanding in Low-Resource Languages

Jesse Atuhurra,Iqra Ali,Tomoya Iwakura,Hidetaka Kamigaito,Tatsuya Hiraoka

Main category: cs.CL

TL;DR: VLURes是一个多语言基准测试，用于评估视觉语言模型（VLM）在低资源语言（如斯瓦希里语和乌尔都语）中的视觉与语言理解能力，填补了非英语基准的空白。

Details

Motivation: 现有的VLM评估主要集中在英语上，缺乏对多语言尤其是低资源语言的测试。VLURes旨在填补这一空白，提供更全面的模型能力评估。

Contribution: 提出了首个针对低资源语言的多语言基准测试VLURes，包含八项任务和一个新颖的“无关性”任务，并引入了斯瓦希里语和乌尔都语的视觉语言资源。

Method: 数据集从目标语言的网络资源中收集，涵盖十种图像类别和丰富的文本背景。通过自动评估和母语者评估，分析了十种VLM的表现。

Result: 最佳模型GPT-4o在VLURes上的总体准确率为90.8%，但仍落后人类表现6.7%，开源模型的差距更大。

Insight: VLURes揭示了VLM在多语言尤其是低资源语言中的表现差距，强调了其在开发多模态视觉推理智能体中的重要性。

Abstract: Vision Language Models (VLMs) are pivotal for advancing perception in intelligent agents. Yet, evaluation of VLMs remains limited to predominantly English-centric benchmarks in which the image-text pairs comprise short texts. To evaluate VLM fine-grained abilities, in four languages under long-text settings, we introduce a novel multilingual benchmark VLURes featuring eight vision-and-language tasks, and a pioneering unrelatedness task, to probe the fine-grained Visual and Linguistic Understanding capabilities of VLMs across English, Japanese, and low-resource languages, Swahili, and Urdu. Our datasets, curated from web resources in the target language, encompass ten diverse image categories and rich textual context, introducing valuable vision-language resources for Swahili and Urdu. By prompting VLMs to generate responses and rationales, evaluated automatically and by native speakers, we uncover performance disparities across languages and tasks critical to intelligent agents, such as object recognition, scene understanding, and relationship understanding. We conducted evaluations of ten VLMs with VLURes. The best performing model, GPT-4o, achieves an overall accuracy of 90.8% and lags human performance by 6.7%, though the gap is larger for open-source models. The gap highlights VLURes’ critical role in developing intelligent agents to tackle multi-modal visual reasoning.

[10] OPLoRA: Orthogonal Projection LoRA Prevents Catastrophic Forgetting during Parameter-Efficient Fine-Tuning

Yifeng Xiong,Xiaohui Xie

Main category: cs.CL

TL;DR: OPLoRA 是一种通过正交投影防止低秩适配（LoRA）调优过程中灾难性遗忘的方法，通过双面正交投影约束更新方向，保留预训练知识。

Details

Motivation: LoRA 在高效调优大规模语言模型时，其更新可能干扰主导奇异方向，导致灾难性遗忘。作者旨在解决这一问题，保留预训练关键知识。

Contribution: 提出 OPLoRA，一种理论支持的方法，通过正交投影约束 LoRA 更新方向，避免干扰主导奇异方向。还引入新指标 ρ_k 量化子空间干扰。

Method: 使用 SVD 分解冻结权重，通过投影矩阵 P_L 和 P_R 将更新约束在正交补空间中，保留前 k 个奇异三元组的数学保证。

Result: 在常识推理、数学和代码生成任务中，OPLoRA 显著减少遗忘，同时保持 LLaMA-2 7B 和 Qwen2.5 7B 的性能。

Insight: 正交投影是一种有效的机制，可在参数高效调优中保留知识，适用于其他需要避免灾难性遗忘的场景。

Abstract: Low-Rank Adaptation (LoRA) enables efficient fine-tuning of large language models but suffers from catastrophic forgetting when learned updates interfere with the dominant singular directions that encode essential pre-trained knowledge. We propose Orthogonal Projection LoRA (OPLoRA), a theoretically grounded approach that prevents this interference through double-sided orthogonal projections. By decomposing frozen weights via SVD, OPLoRA constrains LoRA updates to lie entirely within the orthogonal complement of the top-$k$ singular subspace using projections $P_L = I - U_k U_k^\top$ and $P_R = I - V_k V_k^\top$. We prove that this construction exactly preserves the top-$k$ singular triples, providing mathematical guarantees for knowledge retention. To quantify subspace interference, we introduce $\rho_k$, a metric measuring update alignment with dominant directions. Extensive experiments across commonsense reasoning, mathematics, and code generation demonstrate that OPLoRA significantly reduces forgetting while maintaining competitive task-specific performance on LLaMA-2 7B and Qwen2.5 7B, establishing orthogonal projection as an effective mechanism for knowledge preservation in parameter-efficient fine-tuning.

[11] I Am Aligned, But With Whom? MENA Values Benchmark for Evaluating Cultural Alignment and Multilingual Bias in LLMs

Pardis Sadat Zahraei,Ehsaneddin Asgari

Main category: cs.CL

TL;DR: MENAValues是一个新的基准测试，用于评估大型语言模型（LLM）在中东和北非（MENA）地区文化和多语言偏见上的对齐性。该基准通过大规模人类调查数据，揭示了模型在不同语言和视角下的行为差异。

Details

Motivation: 中东和北非（MENA）地区在当前AI评估中缺乏代表性，导致LLM可能无法准确反映该地区的文化和价值观。MENAValues旨在填补这一空白，提供一个系统的评估框架。

Contribution: 提出了MENAValues基准，首次系统地评估LLM在MENA地区的文化对齐性和多语言偏见。发现了模型在跨语言、推理诱导和文化简化中的问题。

Method: 基于16个国家的大规模人类调查数据，构建了结构化的数据集。通过结合不同视角（中性、个人化、文化观察者）和语言模式（英语和本地语言），评估了多种LLM的行为。

Result: 发现了三种现象：跨语言价值观偏移、推理诱导的对齐性下降，以及对敏感问题的隐藏偏好（Logit Leakage）。模型在本地语言中会简化为单一文化类别。

Insight: MENAValues为开发更具文化包容性的AI提供了实证和方法工具，揭示了LLM在跨文化情境中的潜在问题。

Abstract: We introduce MENAValues, a novel benchmark designed to evaluate the cultural alignment and multilingual biases of large language models (LLMs) with respect to the beliefs and values of the Middle East and North Africa (MENA) region, an underrepresented area in current AI evaluation efforts. Drawing from large-scale, authoritative human surveys, we curate a structured dataset that captures the sociocultural landscape of MENA with population-level response distributions from 16 countries. To probe LLM behavior, we evaluate diverse models across multiple conditions formed by crossing three perspective framings (neutral, personalized, and third-person/cultural observer) with two language modes (English and localized native languages: Arabic, Persian, Turkish). Our analysis reveals three critical phenomena: “Cross-Lingual Value Shifts” where identical questions yield drastically different responses based on language, “Reasoning-Induced Degradation” where prompting models to explain their reasoning worsens cultural alignment, and “Logit Leakage” where models refuse sensitive questions while internal probabilities reveal strong hidden preferences. We further demonstrate that models collapse into simplistic linguistic categories when operating in native languages, treating diverse nations as monolithic entities. MENAValues offers a scalable framework for diagnosing cultural misalignment, providing both empirical insights and methodological tools for developing more culturally inclusive AI.

[12] Mirror Speculative Decoding: Breaking the Serial Barrier in LLM Inference

Nikhil Bhendawade,Kumari Nishu,Arnav Kundu,Chris Bartels,Minsik Cho,Irina Belousova

Main category: cs.CL

TL;DR: Mirror Speculative Decoding (Mirror-SD) 是一种高效的大型语言模型推理算法，通过并行执行和异构加速器映射，打破了传统推测解码的延迟-接受率权衡，实现了显著的性能提升。

Details

Motivation: 传统推测解码技术因自回归生成草稿的成本高，存在延迟与接受率之间的权衡问题，限制了性能提升。Mirror-SD旨在打破这一限制。

Contribution: 1. 提出了Mirror-SD算法，通过并行执行和异构加速器映射实现高效推理；2. 引入了多令牌推测流技术，进一步降低延迟；3. 在多个任务和模型规模上验证了性能优势。

Method: Mirror-SD利用早期退出信号并行启动分支完整的草稿生成，同时在异构加速器（GPU和NPU）上映射计算以实现并行化。此外，通过多令牌推测流技术减少延迟。

Result: 在SpecBench上，Mirror-SD实现了2.8x-5.8x的端到端加速，平均相对性能提升30%。

Insight: Mirror-SD展示了通过并行化和异构计算优化推测解码的潜力，为未来高效推理算法设计提供了新思路。

Abstract: Speculative decoding accelerates LLM inference by using a draft model to look ahead, but gains are capped by the cost of autoregressive draft generation: increasing draft size elevates acceptance rates but introduces additional latency overhead exacerbating the speed-accuracy tradeoff. Prior methods (Medusa, Hydra, EAGLE) partially reduce draft cost but either degrade acceptance or introduce overheads that limit scaling. We present Mirror Speculative Decoding (Mirror-SD), an inference algorithm that breaks the latency-acceptance tradeoff. Mirror-SD launches branch-complete rollouts from early-exit signals in parallel with the target model’s suffix and explicitly maps computation across heterogeneous accelerators (GPU and NPU) to exploit cross-device parallelism. The draft speculates forward continuations for the target to verify, while the target simultaneously speculates correction paths for the draft, converting speculation into two complementary execution pipelines. To further cut draft latency without weakening acceptance semantics, we add speculative streaming so the draft emits multiple tokens per step. This dual strategy of parallel heterogeneous execution plus multi-token speculative streaming pushes speculative decoding toward its ideal regime of high acceptance with low overhead. On SpecBench with server-scale models from 14B to 66B parameters, Mirror-SD delivers consistent end-to-end gains, achieving 2.8x-5.8x wall-time speedups across diverse tasks and a 30% average relative improvement over the strongest baseline, EAGLE3.

[13] A Matter of Representation: Towards Graph-Based Abstract Code Generation

Nyx Iskandar,Hisham Bedri,Andy Tsen

Main category: cs.CL

TL;DR: 这篇论文探讨了基于图的抽象代码生成问题，提出了JSON表示方法以提高生成准确性，并在ScratchTest上验证了方法的有效性。

Details

Motivation: 当前大型语言模型（LLMs）多擅长生成原始、顺序的代码，而在基于图的抽象代码生成（逻辑封装于预定义节点，执行流程由边决定）方面研究较少。这在视觉编程语言或用户无法访问原始代码的情况下尤为重要。

Contribution: 1. 提出并评估了基于JSON的图表示方法，支持高准确性的图抽象代码生成；2. 提出了ScratchTest基准测试；3. 证明了LLMs可通过单次生成完成任务，无需复杂流程。

Method: 1. 设计了JSON格式的图表示；2. 基于自定义Python实现的Scratch构建了ScratchTest基准；3. 评估不同表示对生成准确性的影响。

Result: 实验表明LLMs能够高效完成图抽象代码生成任务，且不同表示对准确性有显著影响。

Insight: 图的表示形式在抽象代码生成任务中至关重要，选择合适的表示可显著提升模型性能。

Abstract: Most large language models (LLMs) today excel at generating raw, sequential code with minimal abstractions and custom structures. However, there has been little work on graph-based abstract code generation, where significant logic is encapsulated in predefined nodes and execution flow is determined by edges. This is relevant for visual programming languages, and in cases where raw source code is inaccessible to users and LLM training sets. In this work, we propose and evaluate JSON representations for graphs to enable high accuracy graph-based abstract code generation. We evaluate these representations on ScratchTest, a mini-benchmark based on our custom Python re-implementation of Scratch, which tests the LLM in code graph space. Our findings demonstrate that LLMs can indeed perform the aforementioned generation task in a single pass without relying on specialized or complex pipelines, given the correct graph representations. We also show that different representations induce significantly different accuracies, highlighting the instrumental role of representations in this generation task. All in all, this work establishes the first steps towards representation learning for graph-based abstract code generation.

[14] CoT-Evo: Evolutionary Distillation of Chain-of-Thought for Scientific Reasoning

Kehua Feng,Keyan Ding,Zhihui Zhu,Lei Liang,Qiang Zhang,Huajun Chen

Main category: cs.CL

TL;DR: CoT-Evo是一个进化式蒸馏框架，用于从多样但存在缺陷的大语言模型（LLM）推理轨迹中提炼高质量的科学推理数据集，从而提升小型学生模型的推理能力。

Details

Motivation: 现有的大语言模型在科学领域推理中常表现出错误或肤浅的推理，直接蒸馏这些输出会导致低质量训练数据，限制了小型学生模型的性能表现。

Contribution: 提出了CoT-Evo框架，通过进化方法（多样性选择、反射重组和变异）迭代优化推理轨迹，生成高质量的科学推理数据集，并成功训练出性能优异的小型模型。

Method: 1. 构建多样化的LLM推理轨迹池；2. 自动检索领域知识增强轨迹；3. 通过新颖性选择、反射重组和变异迭代优化轨迹；4. 使用答案正确性、一致性和知识利用率的适应度函数指导优化。

Result: 通过CoT-Evo训练的紧凑模型在科学推理基准测试中达到了最先进的性能。

Insight: 进化式优化方法可以从多样且不完美的LLM推理中提炼高质量数据，适用于复杂科学推理任务的蒸馏。

Abstract: While chain-of-thought (CoT) distillation from advanced large language models (LLMs) has proven effective in general reasoning tasks, it struggles in scientific domains where even advanced models often produce incorrect or superficial reasoning due to high complexity and specialized knowledge requirements. Directly distilling from such flawed outputs results in low-quality training data and limits the performance of smaller student models. To overcome this, we propose CoT-Evo, an evolutionary CoT distillation framework. It begins by constructing a diverse pool of reasoning trajectories from multiple LLM thinkers, enriches them with automatically retrieved domain knowledge, and iteratively refines the trajectories using novelty-driven selection, reflective recombination and mutation. The refinement is guided by a fitness function that evaluates answer correctness, coherence, and effective knowledge utilization. This results in a high-quality CoT dataset tailored for scientific reasoning. We employ this evolved dataset to fine-tune a compact model, which achieves state-of-the-art performance on scientific reasoning benchmarks. Our work establishes a scalable approach to synthesizing high-fidelity scientific reasoning data from diverse and fallible LLMs.

[15] Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanism

Xiaoshu Chen,Sihang Zhou,Ke Liang,Duanyang Yuan,Haoyuan Chen,Xiaoyu Sun,Linyuan Meng,Xinwang Liu

Main category: cs.CL

TL;DR: 这篇综述首次从人类推理机制的视角系统分析了链式思维(CoT)微调技术，并基于六顶思考帽框架对现有方法进行分类和评估，同时展望了未来研究方向。

Details

Motivation: 现有的CoT微调综述多关注技术层面，忽视了从人类推理机制的系统分析，而CoT的终极目标是让LLMs具备人类推理能力，因此需要从人类认知角度重新审视。

Contribution: 1. 首次基于人类推理理论（六顶思考帽框架）对CoT微调进行系统分类和评估；2. 提供全面的数据集和模型性能总结；3. 开源GitHub仓库实时跟踪最新研究进展。

Method: 基于六顶思考帽框架，将CoT微调方法分类并分析其对应的人类推理模式（如规划思维、发散思维等），同时结合监督和强化微调技术。

Result: LLMs在数学推理和代码生成等任务中展现出显著提升，但需进一步从人类认知角度优化推理能力。

Insight: CoT微调的核心是模仿人类推理机制，未来研究应更注重心理学理论与AI技术的结合，以提升模型的泛化性和解释性。

Abstract: Chain of thought (CoT) fine-tuning aims to endow large language models (LLMs) with reasoning capabilities by training them on curated reasoning traces. It leverages both supervised and reinforced fine-tuning to cultivate human-like reasoning skills in LLMs, including detailed planning, divergent thinking, intuitive judgment, timely reflection, internal thinking, and fact perception, etc. As CoT fine-tuning has advanced, LLMs have demonstrated substantial improvements in tasks such as mathematical reasoning and code generation. However, existing surveys about CoT fine-tuning primarily focus on technical aspects and overlook a systematic analysis from the perspective of human reasoning mechanisms. Given that the ultimate goal of CoT fine-tuning is to enable LLMs to reason like humans, it is crucial to investigate this technique through the lens of human cognition. To fill this gap, we present the first comprehensive survey of CoT fine-tuning grounded in human reasoning theory. Specifically, inspired by the well-known Six Thinking Hats framework, which systematically characterizes common human thinking modes using six metaphorical hats, we classify and examine CoT fine-tuning methods through this lens. Furthermore, building upon this theory, we outline potential directions for future research in CoT fine-tuning. In addition, we compile a comprehensive overview of existing datasets and model performances, and a real-time GitHub repository \footnote{https://github.com/AI-Chen/Awesome-CoT-Finetuning} that continuously tracks recent advances in this area is maintained. We hope this survey will serve as a valuable resource to inspire innovation and foster progress in this rapidly evolving field.

[16] SHIELD: Classifier-Guided Prompting for Robust and Safer LVLMs

Juan Ren,Mark Dras,Usman Naseem

Main category: cs.CL

TL;DR: SHIELD 是一个轻量级的、模型无关的前处理框架，通过细粒度安全分类和分类特定引导实现对抗输入的防御，无需重新训练模型。

Details

Motivation: 大型视觉语言模型（LVLM）在多模态推理中展现出强大能力，但也面临对抗输入的攻击风险，这些输入可能在看似无害的提示中隐藏有害目标。

Contribution: 提出了 SHIELD 框架，结合细粒度安全和特定类别引导，实现对抗输入的防御，同时保持模型实用性和低开销。

Method: SHIELD 通过安全分类、分类特定引导和显式动作（如阻止、重构、转发）生成定制化安全提示。

Result: 在五个基准测试中，SHIELD 显著降低了越狱和未遵循率，同时保持实用性，适用于弱对齐和强对齐的 LVLM。

Insight: SHIELD 展示了无需重新训练即可通过预处理框架显著提升模型安全性的可行性。

Abstract: Large Vision-Language Models (LVLMs) unlock powerful multimodal reasoning but also expand the attack surface, particularly through adversarial inputs that conceal harmful goals in benign prompts. We propose SHIELD, a lightweight, model-agnostic preprocessing framework that couples fine-grained safety classification with category-specific guidance and explicit actions (Block, Reframe, Forward). Unlike binary moderators, SHIELD composes tailored safety prompts that enforce nuanced refusals or safe redirection without retraining. Across five benchmarks and five representative LVLMs, SHIELD consistently lowers jailbreak and non-following rates while preserving utility. Our method is plug-and-play, incurs negligible overhead, and is easily extendable to new attack types – serving as a practical safety patch for both weakly and strongly aligned LVLMs.

[17] Grounding Long-Context Reasoning with Contextual Normalization for Retrieval-Augmented Generation

Jiamin Chen,Yuchen Li,Xinyu Ma,Xinran Chen,Xiaokun Zhang,Shuaiqiang Wang,Chen Ma,Dawei Yin

Main category: cs.CL

TL;DR: 这篇论文探讨了检索增强生成（RAG）中上下文格式对模型性能的影响，提出了Contextual Normalization方法，通过标准化上下文表示提升长文本推理的鲁棒性和效果。

Details

Motivation: 现有RAG研究多关注检索质量和提示策略，而忽略了上下文格式（如分隔符或结构标记）对模型性能的潜在影响，即使语义内容相同。论文旨在填补这一空白。

Contribution: 1. 揭示了上下文格式对RAG性能的影响；2. 提出了Contextual Normalization方法，自适应标准化上下文表示；3. 在多样场景下验证了方法的有效性。

Method: 设计了控制实验，分析上下文密度、分隔符风格和位置的影响，提出Contextual Normalization策略，标准化上下文后再生成。

Result: 实验表明，该方法显著提升了长文本推理的鲁棒性和性能，尤其是在上下文顺序变化和长文本利用方面。

Insight: 可靠的RAG不仅依赖于检索内容的质量，还取决于内容的呈现方式，这对设计和优化RAG系统具有重要指导意义。

Abstract: Retrieval-Augmented Generation (RAG) has become an essential approach for extending the reasoning and knowledge capacity of large language models (LLMs). While prior research has primarily focused on retrieval quality and prompting strategies, the influence of how the retrieved documents are framed, i.e., context format, remains underexplored. We show that seemingly superficial choices, such as delimiters or structural markers in key-value extraction, can induce substantial shifts in accuracy and stability, even when semantic content is identical. To systematically investigate this effect, we design controlled experiments that vary context density, delimiter styles, and positional placement, revealing the underlying factors that govern performance differences. Building on these insights, we introduce Contextual Normalization, a lightweight strategy that adaptively standardizes context representations before generation. Extensive experiments on both controlled and real-world RAG benchmarks across diverse settings demonstrate that the proposed strategy consistently improves robustness to order variation and strengthens long-context utilization. These findings underscore that reliable RAG depends not only on retrieving the right content, but also on how that content is presented, offering both new empirical evidence and a practical technique for better long-context reasoning.

[18] StressTransfer: Stress-Aware Speech-to-Speech Translation with Emphasis Preservation

Xi Chen,Yuchen Song,Satoshi Nakamura

Main category: cs.CL

TL;DR: 该论文提出了一种基于压力感知的语音到语音翻译系统StressTransfer，通过LLMs实现跨语言压力转换，保持单词级别的强调。方法利用自动生成对齐数据和引入“LLM-as-Judge”评估，显著优于基线。

Details

Motivation: 传统语音到语音翻译系统往往忽略语调和强调等副语言线索，导致表达意图和情感的丢失。为填补这一空白，论文提出压力感知的翻译方法。

Contribution: 1. 提出首个压力感知的语音到语音翻译系统StressTransfer；2. 利用LLMs实现跨语言压力转换；3. 开发自动生成对齐数据的流程；4. 引入“LLM-as-Judge”评估方法。

Method: 1. 通过LLMs将源语言的压力标记转换为目标语言的压力标记；2. 使用可控TTS模型生成目标语音；3. 自动生成对齐训练数据，缓解数据稀缺问题。

Result: 实验表明，StressTransfer在保持强调方面显著优于基线，同时在翻译质量、说话者意图和自然度方面表现相当。

Insight: 副语言线索（如强调和语调）在翻译中极具重要性，数据高效的方法可以有效捕捉这些信息。

Abstract: We propose a stress-aware speech-to-speech translation (S2ST) system that preserves word-level emphasis by leveraging LLMs for cross-lingual emphasis conversion. Our method translates source-language stress into target-language tags that guide a controllable TTS model. To overcome data scarcity, we developed a pipeline to automatically generate aligned training data and introduce the “LLM-as-Judge” for evaluation. Experiments show our approach substantially outperforms baselines in preserving emphasis while maintaining comparable translation quality, speaker intent, and naturalness. Our work highlights the importance of prosody in translation and provides an effective, data-efficient solution for preserving paralinguistic cues in S2ST.

[19] Do You Get the Hint? Benchmarking LLMs on the Board Game Concept

Ine Gevers,Walter Daelemans

Main category: cs.CL

TL;DR: 该论文以棋盘游戏Concept作为基准测试LLMs在抽象推理任务中的表现，结果表明LLMs在自然语言任务中仍难以模仿人类策略意图和动态更新假设的能力，尤其在多语言环境下表现更差。

Details

Motivation: 尽管LLMs在许多基准测试中表现出色，但在需要抽象推理的任务（尤其是涉及非自然语言表征的任务）中仍存在明显短板。论文试图通过一个更接近LLMs预训练数据（自然语言）的任务来探索其推理能力。

Contribution: 引入Concept棋盘游戏作为新基准，用于评估LLMs在自然语言环境下的溯因推理能力，并揭示了LLMs在策略意图理解和动态信息整合上的不足。

Method: 使用Concept游戏作为测试平台，要求LLMs根据玩家的提示猜测目标词，并设计实验比较LLMs与人类的表现差异，同时扩展到多语言场景（英语、荷兰语、法语、西班牙语）。

Result: LLMs的成功率最高仅40%，远低于人类的90%；在多语言环境下性能进一步下降，尤其是在低资源语言中表现更差。

Insight: LLMs在需要策略交互和动态更新的任务中仍有局限性，多语言能力的不均衡性也凸显了数据资源对模型性能的重要性。

Abstract: Large language models (LLMs) have achieved striking successes on many benchmarks, yet recent studies continue to expose fundamental weaknesses. In particular, tasks that require abstract reasoning remain challenging, often because they use representations such as grids, symbols, or visual patterns that differ from the natural language data LLMs are trained on. In this paper, we introduce Concept, a simple word-guessing board game, as a benchmark for probing abductive reasoning in a representation that is much closer to LLM pre-training data: natural language. Our results show that this game, easily solved by humans (with a success rate of over 90%), is still very challenging for state-of-the-art LLMs (no model exceeds 40% success rate). Specifically, we observe that LLMs struggle with interpreting other players’ strategic intents, and with correcting initial hypotheses given sequential information updates. In addition, we extend the evaluation across multiple languages, and find that the LLM performance drops further in lower-resource languages (Dutch, French, and Spanish) compared to English.

[20] Beyond Correctness: Rewarding Faithful Reasoning in Retrieval-Augmented Generation

Zhichao Xu,Zongyu Wu,Yun Zhou,Aosong Feng,Kang Zhou,Sangmin Woo,Kiran Ramnath,Yijun Tian,Xuan Qi,Weikang Qiu,Lin Lee Cheong,Haibo Ding

Main category: cs.CL

TL;DR: 论文提出了一种新框架VERITAS，用于在检索增强生成任务中，通过强化学习奖励中间推理步骤的忠实性，从而提升大型语言模型（LLM）的推理质量，同时保持任务性能。

Details

Motivation: 现有的基于强化学习的检索增强生成方法虽然提升了问答任务的性能，但忽视了中间推理步骤的忠实性，可能导致推理链不忠实。

Contribution: 提出了一个全面的评估框架，用于衡量基于RL的检索代理的忠实性，并设计了VERITAS框架，通过精细化的忠实性奖励提升推理质量。

Method: VERITAS通过强化学习过程整合忠实性奖励，包括信息-思考忠实性、思考-答案忠实性和思考-检索忠实性三个指标。

Result: 实验表明，VERITAS训练的模型在七个问答基准测试中显著提高了推理忠实性，同时保持了可比的任务性能。

Insight: 关注中间推理步骤的忠实性不仅有助于提升模型的可解释性，还能在不牺牲任务性能的情况下提高推理质量。

Abstract: Inspired by the success of reinforcement learning (RL) in Large Language Model (LLM) training for domains like math and code, recent works have begun exploring how to train LLMs to use search engines more effectively as tools for retrieval-augmented generation. Although these methods achieve performance improvement across QA benchmarks, many prioritize final answer correctness while overlooking the quality of intermediate reasoning steps, which may lead to chain-of-thought unfaithfulness. In this paper, we first introduce a comprehensive evaluation framework for evaluating RL-based search agents, covering three distinct faithfulness metrics: information-think faithfulness, think-answer faithfulness, and think-search faithfulness. Our evaluations reveal that a prototypical RL-based search agent, Search-R1, has significant room for improvement in this regard. To foster faithful reasoning, we introduce VERITAS (Verifying Entailed Reasoning through Intermediate Traceability in Agentic Search), a novel framework that integrates fine-grained faithfulness rewards into the reinforcement learning process. Our experiments show that models trained with VERITAS not only significantly improve reasoning faithfulness, but also achieve comparable task performance across seven QA benchmarks.

[21] ChatR1: Reinforcement Learning for Conversational Reasoning and Retrieval Augmented Question Answering

Simon Lupart,Mohammad Aliannejadi,Evangelos Kanoulas

Main category: cs.CL

TL;DR: ChatR1是一个基于强化学习的对话式问答推理框架，通过自适应检索和推理改进了传统的静态流水线方法，在多数据集上表现优异。

Details

Motivation: 传统对话式问答（CQA）采用静态的“重写、检索、生成”流水线方法，难以适应动态变化的用户意图和多轮对话需求。为解决这一问题，ChatR1提出了一种基于强化学习的自适应推理框架。

Contribution: 1. 提出了ChatR1框架，通过强化学习实现多轮对话中的检索和推理；2. 设计了意图感知奖励机制，解决稀疏和延迟奖励问题；3. 在多个CQA数据集上验证了方法的有效性。

Method: 1. 结合强化学习，动态协调检索和生成；2. 引入意图感知奖励，提供逐轮反馈；3. 在多轮对话中实现探索性行为。

Result: 1. ChatR1在3B和7B模型上表现优异，优于竞争模型；2. 在不同数据集（主题转移、意图演化等）中均表现稳健；3. 消融实验验证了奖励机制的有效性。

Insight: 强化学习能够为对话式问答提供更灵活、上下文敏感的行为，优于静态方法。

Abstract: We present ChatR1, a reasoning framework based on reinforcement learning (RL) for conversational question answering (CQA). Reasoning plays an important role in CQA, where user intent evolves across dialogue turns, and utterances are often underspecified, requiring contextual interpretation, query reformulation, and dynamic coordination between retrieval and generation. Unlike static `rewrite, retrieve, and generate’ pipelines, ChatR1 interleaves search and reasoning across turns, enabling exploratory and adaptive behaviors learned through RL. To address the challenge of sparse and delayed rewards in RL, we propose an intent-aware reward that provides turn-level feedback by aligning retrieval and reasoning with evolving user goals. Our proposed ChatR1 demonstrates strong performance on both 3B and 7B model backbones, outperforming competitive models on five CQA datasets, measured by different metrics (F1, BERTScore, and LLM-as-judge). We include a diverse set of CQA datasets to cover topic shifts, evolving intents, mixed-initiative dialogues, and multi-document grounding, testing ChatR1’s performance from various aspects. Ablation studies confirm the effectiveness of the intent-aware reward. Our analyses further reveal diverse reasoning trajectories and effective use of the search tool. ChatR1 also generalizes robustly across domains, demonstrating that RL-based reasoning enables more flexible and context-sensitive behavior than static CQA pipelines.

[22] Embedding-Based Context-Aware Reranker

Ye Yuan,Mohammad Amin Shabani,Siqi Liu

Main category: cs.CL

TL;DR: 该论文提出了一种轻量级的重排框架EBCAR，通过嵌入和结构化信息增强跨段落理解，解决了现有方法忽略跨段落推理的问题。

Details

Motivation: 当前基于RAG的系统在长文档拆分为短段落时，面临跨段落推理（如指代消解、实体消歧、证据聚合）的挑战，现有重排方法尽管依赖强大但昂贵的预训练模型，却仍忽视这些问题。

Contribution: 提出轻量级重排框架EBCAR，直接利用段落嵌入和结构化信息，结合混合注意力机制，提升跨段落理解能力和检索效率。

Method: EBCAR通过嵌入和结构化信息捕捉跨文档高层交互和文档内低层关系，采用混合注意力机制实现高效重排。

Result: 在ConTEB基准测试中，EBCAR在跨段落推理任务上表现优于SOTA方法，同时在准确性和效率上有显著优势。

Insight: 轻量化的嵌入结合结构化信息和注意力机制是解决跨段落推理问题的有效途径，尤其适合需要高效检索的场景。

Abstract: Retrieval-Augmented Generation (RAG) systems rely on retrieving relevant evidence from a corpus to support downstream generation. The common practice of splitting a long document into multiple shorter passages enables finer-grained and targeted information retrieval. However, it also introduces challenges when a correct retrieval would require inference across passages, such as resolving coreference, disambiguating entities, and aggregating evidence scattered across multiple sources. Many state-of-the-art (SOTA) reranking methods, despite utilizing powerful large pretrained language models with potentially high inference costs, still neglect the aforementioned challenges. Therefore, we propose Embedding-Based Context-Aware Reranker (EBCAR), a lightweight reranking framework operating directly on embeddings of retrieved passages with enhanced cross-passage understandings through the structural information of the passages and a hybrid attention mechanism, which captures both high-level interactions across documents and low-level relationships within each document. We evaluate EBCAR against SOTA rerankers on the ConTEB benchmark, demonstrating its effectiveness for information retrieval requiring cross-passage inference and its advantages in both accuracy and efficiency.

[23] Protect: Towards Robust Guardrailing Stack for Trustworthy Enterprise LLM Systems

Karthik Avinash,Nikhil Pareek,Rishav Hada

Main category: cs.CL

TL;DR: 该论文提出了Protect，一种多模态防护系统，旨在为企业级LLM提供安全、可靠和合规的保护。

Details

Motivation: 随着LLM在企业和关键任务领域的广泛应用，现有防护系统在实时监督、多模态数据处理和可解释性方面存在不足，亟需一种更强大的解决方案。

Contribution: 提出Protect，一个原生多模态防护模型，支持文本、图像和音频输入，并通过Low-Rank Adaptation（LoRA）训练专用适配器。

Method: Protect采用教师辅助标注流程生成高质量标签，并在多模态数据集上训练，覆盖毒性、性别歧视、数据隐私和提示注入四个安全维度。

Result: 实验显示Protect在所有安全维度上优于现有开源和专有模型（如WildGuard、LlamaGuard-4和GPT-4.1）。

Insight: Protect为可信、可审计和生产级的安全系统奠定了基础，适用于多模态场景。

Abstract: The increasing deployment of Large Language Models (LLMs) across enterprise and mission-critical domains has underscored the urgent need for robust guardrailing systems that ensure safety, reliability, and compliance. Existing solutions often struggle with real-time oversight, multi-modal data handling, and explainability – limitations that hinder their adoption in regulated environments. Existing guardrails largely operate in isolation, focused on text alone making them inadequate for multi-modal, production-scale environments. We introduce Protect, natively multi-modal guardrailing model designed to operate seamlessly across text, image, and audio inputs, designed for enterprise-grade deployment. Protect integrates fine-tuned, category-specific adapters trained via Low-Rank Adaptation (LoRA) on an extensive, multi-modal dataset covering four safety dimensions: toxicity, sexism, data privacy, and prompt injection. Our teacher-assisted annotation pipeline leverages reasoning and explanation traces to generate high-fidelity, context-aware labels across modalities. Experimental results demonstrate state-of-the-art performance across all safety dimensions, surpassing existing open and proprietary models such as WildGuard, LlamaGuard-4, and GPT-4.1. Protect establishes a strong foundation for trustworthy, auditable, and production-ready safety systems capable of operating across text, image, and audio modalities.

[24] D-SMART: Enhancing LLM Dialogue Consistency via Dynamic Structured Memory And Reasoning Tree

Xiang Lei,Qin Li,Min Zhang,Min Zhang

Main category: cs.CL

TL;DR: D-SMART是一个模型无关的框架，通过动态结构化记忆和推理树提升大语言模型在多轮对话中的一致性表现。

Details

Motivation: 大语言模型在多轮对话中易出现事实不一致和逻辑衰退问题，当前解决方法如检索增强生成和工作记忆仍依赖静态知识源和单一推理路径。

Contribution: 提出了D-SMART框架，包含动态结构化记忆（DSM）和推理树（RT），并设计了新的基于NLI的指标评估对话一致性。

Method: DSM增量构建对话知识图谱，RT基于图谱执行多步显式推理。

Result: 在MT-Bench-101基准上显著优于基线，一致性得分提升48%，开源模型质量得分提升10.1%。

Insight: 动态结构化知识和显式多步推理是提升多轮对话一致性的有效方法。

Abstract: Large Language Models (LLMs) often exhibit factual inconsistencies and logical decay in extended, multi-turn dialogues, a challenge stemming from their reliance on static, pre-trained knowledge and an inability to reason adaptively over the dialogue history. Prevailing mitigation strategies, such as Retrieval-Augmented Generation (RAG) and agentic working memories, improve information recall but still engage with fundamentally static knowledge sources and follow pre-defined single reasoning path. This hinders their ability to preserve factual and logical consistency of their responses in multi-turn dialogues while the context evolves over time. To address this issue, we propose D-SMART, a model-agnostic framework designed to maintain multi-turn dialogue consistency by enabling LLMs to build and reason over a dynamic, structured representation of the conversational context. This is achieved via two synergistic components: (1) a Dynamic Structured Memory (DSM), which incrementally constructs and maintains an authoritative, OWL-compliant knowledge graph of the conversation; and (2) a Reasoning Tree (RT), which executes inferences as an explicit and traceable multi-step search over the graph. As the popular-used quality score (judged by GPT-4) can overlook logical flaws, we introduce new NLI-based metrics to better measure multi-turn dialogue consistency. Comprehensive experiments on the MT-Bench-101 benchmark show that D-SMART significantly outperforms state-of-the-art baselines, elevating the dialogue consistency score by over 48% for both proprietary and open-source models, and notably improves the quality score of the latter by up to 10.1%.

[25] Document Intelligence in the Era of Large Language Models: A Survey

Weishi Wang,Hengchang Hu,Zhijie Zhang,Zhaochen Li,Hongxin Shao,Daniel Dahlmeier

Main category: cs.CL

TL;DR: 这篇综述探讨了大型语言模型（LLMs）如何改变文档智能（DAI），总结了当前研究进展、挑战和未来方向，包括多模态、多语言和检索增强DAI。

Details

Motivation: 文档智能（DAI）已成为重要应用领域，而大型语言模型（LLMs）的出现显著改变了这一领域的研究和应用方式。作者希望通过综述梳理DAI的演变，为未来的研究提供方向。

Contribution: 1. 全面概述了DAI的演变和LLMs对其的影响。2. 探讨了多模态、多语言和检索增强DAI的关键进展与挑战。3. 提出了未来研究方向，如基于智能体的方法和特定文档的基础模型。

Method: 本文采用综述方法，系统梳理了DAI领域的研究文献和技术发展，特别关注了LLMs的应用及其带来的变革。

Result: 总结了DAI领域的最新技术进展，指出了LLMs在多模态、多语言和检索增强等方面的优势与挑战。

Insight: LLMs极大地推动了DAI的能力，尤其是在理解和生成任务上，未来研究可以进一步探索智能体方法和文档专用基础模型的潜力。

Abstract: Document AI (DAI) has emerged as a vital application area, and is significantly transformed by the advent of large language models (LLMs). While earlier approaches relied on encoder-decoder architectures, decoder-only LLMs have revolutionized DAI, bringing remarkable advancements in understanding and generation. This survey provides a comprehensive overview of DAI’s evolution, highlighting current research attempts and future prospects of LLMs in this field. We explore key advancements and challenges in multimodal, multilingual, and retrieval-augmented DAI, while also suggesting future research directions, including agent-based approaches and document-specific foundation models. This paper aims to provide a structured analysis of the state-of-the-art in DAI and its implications for both academic and practical applications.

[26] Doing Things with Words: Rethinking Theory of Mind Simulation in Large Language Models

Agnese Lombardi,Alessandro Lenci

Main category: cs.CL

TL;DR: 本文探讨生成式代理基础模型Concordia是否能模拟心理理论（ToM），并揭示GPT-4在社交任务中依赖统计关联而非真正推理。研究发现其ToM能力存在局限性，需更严格的评估框架。

Details

Motivation: 语言对人类协作至关重要，但此前研究可能低估了大型语言模型（LLM）中心理理论（ToM）能力的真实性，需验证其是否基于真正推理而非统计关联。

Contribution: 1. 提出Concordia模型评估ToM能力。2. 揭示GPT-4在社交任务中依赖浅层统计关联而非推理。3. 强调了基于动作的ToM评估框架的重要性。

Method: 使用生成式代理基础模型Concordia，设计模拟真实世界环境的任务，评估GPT-4是否能通过社会情境做出真实推理，而非仅依赖语言记忆。

Result: GPT-4在基于信念归因的任务中表现不佳，ToM能力可能源于统计关联；模型也难以生成连贯的因果效应。

Insight: 现有LLM的ToM能力可能被高估，需开发更严格的评估方法以区分浅层统计关联与深层推理。

Abstract: Language is fundamental to human cooperation, facilitating not only the exchange of information but also the coordination of actions through shared interpretations of situational contexts. This study explores whether the Generative Agent-Based Model (GABM) Concordia can effectively model Theory of Mind (ToM) within simulated real-world environments. Specifically, we assess whether this framework successfully simulates ToM abilities and whether GPT-4 can perform tasks by making genuine inferences from social context, rather than relying on linguistic memorization. Our findings reveal a critical limitation: GPT-4 frequently fails to select actions based on belief attribution, suggesting that apparent ToM-like abilities observed in previous studies may stem from shallow statistical associations rather than true reasoning. Additionally, the model struggles to generate coherent causal effects from agent actions, exposing difficulties in processing complex social interactions. These results challenge current statements about emergent ToM-like capabilities in LLMs and highlight the need for more rigorous, action-based evaluation frameworks.

[27] ConsintBench: Evaluating Language Models on Real-World Consumer Intent Understanding

Xiaozhe Li,TianYi Lyu,Siyi Yang,Yuxi Gong,Yizhao Yang,Jinxuan Huang,Ligao Zhang,Zhuoyi Huang,Qingwen Liu

Main category: cs.CL

TL;DR: 本文介绍了ConsintBench，这是首个动态、实时评估语言模型（LLM）在真实世界消费者意图理解任务中的基准测试工具，填补了该领域的空白。

Details

Motivation: 现有的大型语言模型在理解人类意图时需要复杂的推理和多源信号整合能力，但缺乏针对真实世界消费者讨论的评估基准。

Contribution: 提出了首个动态评测基准ConsintBench，支持实时更新并通过自动化流程防止数据污染，是目前规模最大、多样性最强的同类基准。

Method: 设计了自动化数据收集与评测流程，整合了多源信号和不一致的上下文，模拟真实世界的复杂讨论场景。

Result: ConsintBench为评估LLM在消费者意图理解任务中的表现提供了标准化工具，支持动态更新和抗污染设计。

Insight: 该基准突出了LLM在处理真实世界非线性、多视角讨论时的能力需求，为未来研究和模型优化提供了方向。

Abstract: Understanding human intent is a complex, high-level task for large language models (LLMs), requiring analytical reasoning, contextual interpretation, dynamic information aggregation, and decision-making under uncertainty. Real-world public discussions, such as consumer product discussions, are rarely linear or involve a single user. Instead, they are characterized by interwoven and often conflicting perspectives, divergent concerns, goals, emotional tendencies, as well as implicit assumptions and background knowledge about usage scenarios. To accurately understand such explicit public intent, an LLM must go beyond parsing individual sentences; it must integrate multi-source signals, reason over inconsistencies, and adapt to evolving discourse, similar to how experts in fields like politics, economics, or finance approach complex, uncertain environments. Despite the importance of this capability, no large-scale benchmark currently exists for evaluating LLMs on real-world human intent understanding, primarily due to the challenges of collecting real-world public discussion data and constructing a robust evaluation pipeline. To bridge this gap, we introduce \bench, the first dynamic, live evaluation benchmark specifically designed for intent understanding, particularly in the consumer domain. \bench is the largest and most diverse benchmark of its kind, supporting real-time updates while preventing data contamination through an automated curation pipeline.

[28] MedREK: Retrieval-Based Editing for Medical LLMs with Key-Aware Prompts

Shujun Xia,Haokun Lin,Yichen Wu,Yinan Zhou,Zixuan Li,Zhongwei Wan,Xingrun Xing,Yefeng Zheng,Xiang Li,Caifeng Shan,Zhenan Sun,Quanzheng Li

Main category: cs.CL

TL;DR: MedREK提出了一种基于检索的编辑方法，通过共享查询键模块和注意力提示编码器解决医学LLMs的知识更新问题，并在MedVersa基准上验证了其单样本和批量编辑的有效性。

Details

Motivation: 医学领域知识的快速演变和训练数据中的错误导致LLMs生成的信息可能过时或不准确，限制了其在临床实践中的应用。现有的参数化编辑方法可能破坏局部性，而检索式编辑需要更高的精度和批量编辑能力。

Contribution: 1) 构建了MedVersa基准，覆盖广泛的医学主题并支持单样本和批量编辑测试；2) 提出MedREK框架，通过共享查询键模块和注意力提示编码器提升检索精度和编辑效果；3) 在多个医学基准上验证了方法的有效性，并首次实现批量编辑。

Method: MedREK结合了共享查询键模块（用于精确匹配）和基于注意力的提示编码器（用于信息指导），解决了医学知识表示重叠的问题，并支持批量编辑。

Result: 实验表明MedREK在多种医学基准上表现优异，提供了首个可行的批量编辑解决方案。

Insight: 检索式编辑在医学领域的高风险应用中更具潜力，结合精确的匹配机制和注意力机制可以显著提升编辑的准确性和实用性。

Abstract: LLMs hold great promise for healthcare applications, but the rapid evolution of medical knowledge and errors in training data often cause them to generate outdated or inaccurate information, limiting their applicability in high-stakes clinical practice. Model editing has emerged as a potential remedy without full retraining. While parameter-based editing often compromises locality and is thus ill-suited for the medical domain, retrieval-based editing offers a more viable alternative. However, it still faces two critical challenges: (1) representation overlap within the medical knowledge space often causes inaccurate retrieval and reduces editing accuracy; (2) existing methods are restricted to single-sample edits, while batch-editing remains largely unexplored despite its importance for real-world medical applications. To address these challenges, we first construct MedVersa, \hk{an enhanced benchmark with broader coverage of medical subjects, designed to evaluate both single and batch edits under strict locality constraints}. We then propose MedREK, a retrieval-based editing framework that integrates a shared query-key module for precise matching with an attention-based prompt encoder for informative guidance. Experimental results on various medical benchmarks demonstrate that our MedREK achieves superior performance across different core metrics and provides the first validated solution for batch-editing in medical LLMs. Our code and dataset are available at https://github.com/mylittleriver/MedREK.

[29] Attention Illuminates LLM Reasoning: The Preplan-and-Anchor Rhythm Enables Fine-Grained Policy Optimization

Yang Li,Zhichen Dong,Yuhan Sun,Weixun Wang,Shaopan Xiong,Yijia Luo,Jiashun Liu,Han Lu,Jiamang Wang,Wenbo Su,Bo Zheng,Junchi Yan

Main category: cs.CL

TL;DR: 该论文研究了大型语言模型（LLMs）的推理机制，通过注意力机制揭示了模型内部的预规划-锚点节律，并提出三种动态强化学习策略，优化关键节点的信用分配。

Details

Motivation: 当前LLMs的推理模式不透明，传统的强化学习方法无法区分推理中的关键步骤和常规步骤。

Contribution: 1) 提出两种注意力指标（WAAD和FAI），揭示LLMs的预规划-锚点节律；2) 设计三种动态RL策略，优化关键节点的信用分配。

Method: 1) 分析注意力头的局部与全局聚焦行为；2) 提出WAAD和FAI指标；3) 设计动态RL策略，针对预规划Token和锚点Token进行信用分配。

Result: 实验表明，这些策略在不同推理任务中均能提升性能。

Insight: 注意力机制可作为理解LLMs推理的工具，优化过程应与模型的固有推理节律对齐。

Abstract: The reasoning pattern of Large language models (LLMs) remains opaque, and Reinforcement learning (RL) typically applies uniform credit across an entire generation, blurring the distinction between pivotal and routine steps. This work positions attention as a privileged substrate that renders the internal logic of LLMs legible, not merely as a byproduct of computation, but as a mechanistic blueprint of reasoning itself. We first distinguish attention heads between locally and globally focused information processing and reveal that locally focused heads produce a sawtooth pattern near the diagonal indicating phrasal chunks, while globally focused heads expose tokens that exert broad downstream influence over future tokens. We formalize these with two metrics: 1) Windowed Average Attention Distance, which measures the extent of backward attention within a clipped window; 2) Future Attention Influence, which quantifies a token’s global importance as the average attention it receives from subsequent tokens. Taken together, these signals reveal a recurring preplan-and-anchor mechanism, where the model first performs a long-range contextual reference to generate an introductory token, which is immediately followed by or coincides with a semantic anchor token that organizes subsequent reasoning. Leveraging these insights, we introduce three novel RL strategies that dynamically perform targeted credit assignment to critical nodes (preplan tokens, anchor tokens, and their temporal coupling) and show consistent performance gains across various reasoning tasks. By aligning optimization with the model’s intrinsic reasoning rhythm, we aim to transform opaque optimization into an actionable structure-aware process, hoping to offer a potential step toward more transparent and effective optimization of LLM reasoning.

[30] Sparse Subnetwork Enhancement for Underrepresented Languages in Large Language Models

Daniil Gurgurov,Josef van Genabith,Simon Ostermann

Main category: cs.CL

TL;DR: 该论文提出了一种通过微调语言特定子网络来提升大语言模型在低资源语言中表现的方法，仅需更新1%的参数即可显著优于全微调和其他基线方法。

Details

Motivation: 大语言模型在不同语言上的表现存在显著差距，高资源语言表现优异，而低资源语言表现欠佳。论文旨在通过有针对性的微调方法提升低资源语言的单语能力，同时保持模型的通用性能。

Contribution: 1. 提出了一种基于语言激活概率熵的语言特定子网络识别方法；2. 提出了一种高效的微调框架，仅更新语言相关参数（1%）；3. 发布超过100种语言的特定神经元识别结果和适配流程。

Method: 1. 使用语言激活概率熵识别语言特定神经元；2. 仅微调目标语言数据相关的子网络权重；3. 实验验证了方法的有效性。

Result: 在Llama-3.1-8B和Mistral-Nemo-12B模型上的实验表明，该方法显著优于全微调、FFN-only微调、LoRA适配和随机子集微调等基线方法。

Insight: 1. 语言特定子网络的微调不仅能提升低资源语言表现，还能改善训练动态和跨语言表示对齐；2. 该方法为适应低资源语言提供了一种低成本途径。

Abstract: Large language models exhibit uneven performance across languages, with substantial gaps between high- and low-resource languages. We present a framework for enhancing monolingual capabilities of LLMs in underrepresented languages while preserving their general-purpose performance through targeted fine-tuning of language-specific subnetworks. Our approach identifies language-specific neurons using Language Activation Probability Entropy and fine-tunes only the weights associated with these neurons, a dedicated subnetwork, on target-language data. Experiments on Llama-3.1-8B and Mistral-Nemo-12B across 12 mid- and low-resource languages demonstrate that our method consistently outperforms full fine-tuning, FFN-only fine-tuning, LoRA adaptation, and random subset fine-tuning baselines while efficiently updating only up to 1% of model parameters. Beyond performance improvements, we observe enhanced favorable training dynamics, cross-lingual representational alignment, and systematic weight update changes. To facilitate future research, we release language-specific neuron identifications for over 100 languages as well as our adaptation pipeline, offering a cost-effective pathway for adapting state-of-the-art models to underrepresented languages.

[31] MemoTime: Memory-Augmented Temporal Knowledge Graph Enhanced Large Language Model Reasoning

Xingyu Tan,Xiaoyang Wang,Qing Liu,Xiwei Xu,Xin Yuan,Liming Zhu,Wenjie Zhang

Main category: cs.CL

TL;DR: MemoTime 是一个增强大语言模型（LLM）时序推理能力的框架，通过结合时序知识图谱（TKG）解决多实体、复合运算符和时序同步等挑战。

Details

Motivation: 现有 LLM 在时序推理中存在多跳推理的时序忠实性不足、多实体时序同步困难等问题，而 TKGs 虽然提供了结构化时序数据，但缺乏高效的推理支持。

Contribution: 提出了 MemoTime 框架，包括层级化时序树分解、动态证据检索层和自我演进经验记忆模块，显著提升了 LLM 的时序推理能力。

Method: 采用结构化接地、递归推理和持续经验学习，分解问题为时序树，并通过动态检索和记忆模块优化推理过程。

Result: 在多时序 QA 基准测试中达到 SOTA，性能提升最高 24%，并使小型模型（如 Qwen3-4B）性能接近 GPT-4-Turbo。

Insight: 结合 TKGs 和记忆机制可显著提升 LLM 的时序推理能力，同时展示了小型模型通过框架优化的潜力。

Abstract: Large Language Models (LLMs) have achieved impressive reasoning abilities, but struggle with temporal understanding, especially when questions involve multiple entities, compound operators, and evolving event sequences. Temporal Knowledge Graphs (TKGs), which capture vast amounts of temporal facts in a structured format, offer a reliable source for temporal reasoning. However, existing TKG-based LLM reasoning methods still struggle with four major challenges: maintaining temporal faithfulness in multi-hop reasoning, achieving multi-entity temporal synchronization, adapting retrieval to diverse temporal operators, and reusing prior reasoning experience for stability and efficiency. To address these issues, we propose MemoTime, a memory-augmented temporal knowledge graph framework that enhances LLM reasoning through structured grounding, recursive reasoning, and continual experience learning. MemoTime decomposes complex temporal questions into a hierarchical Tree of Time, enabling operator-aware reasoning that enforces monotonic timestamps and co-constrains multiple entities under unified temporal bounds. A dynamic evidence retrieval layer adaptively selects operator-specific retrieval strategies, while a self-evolving experience memory stores verified reasoning traces, toolkit decisions, and sub-question embeddings for cross-type reuse. Comprehensive experiments on multiple temporal QA benchmarks show that MemoTime achieves overall state-of-the-art results, outperforming the strong baseline by up to 24.0%. Furthermore, MemoTime enables smaller models (e.g., Qwen3-4B) to achieve reasoning performance comparable to that of GPT-4-Turbo.

[32] Unlocking Public Catalogues: Instruction-Tuning LLMs for ICD Coding of German Tumor Diagnoses

Stefan Lenz,Lakisha Ortiz Rosario,Georg Vollmar,Arsenij Ustjanzew,Fatma Alickovic,Thomas Kindler,Torsten Panholzer

Main category: cs.CL

TL;DR: 这篇论文研究了如何通过基于公开数据集的指令微调，提升开源大语言模型（LLM）在德国肿瘤诊断文本ICD编码任务中的准确性。结果显示，微调显著提升了编码准确率。

Details

Motivation: 在德国，肿瘤诊断的精确ICD编码对结构化癌症文档至关重要，而开源LLM在德语语境中表现不佳。研究旨在探索是否通过指令微调能改善这一性能。

Contribution: 主要贡献包括：1) 创建了基于ICD-10-GM、ICD-O-3和OPS目录的50多万个问答对训练数据集；2) 展示了微调显著提升开源LLM在ICD编码任务中的准确率；3) 公开了训练数据集和最佳模型检查点。

Method: 研究采用了八个开源LLM模型（Qwen、Llama和Mistral家族，7-70B参数），并通过指令微调提升了其在ICD编码任务中的表现。评估使用了本地肿瘤文档系统的编码数据作为测试集。

Result: 结果显示，ICD-10-GM的准确率从1.4-24%提升至41-58%，部分准确率从31-74%提升至73-83%。ICD-O-3地形编码的准确率也有提升，但仍较低。模型推理模式的性能低于微调且速度慢100倍以上。

Insight: 研究发现：1) 模型大小与准确率正相关，但微调缩小了小模型与大模型的差距；2) 利用公开目录构建指令数据集是提升LLM医学文档任务性能的有效途径；3) 推理模式效率远低于微调。

Abstract: Accurate coding of tumor diagnoses with ICD-10-GM and ICD-O-3 is essential for structured cancer documentation in Germany. Smaller open-weight LLMs are appealing for privacy-preserving automation but often struggle with coding accuracy in German-language contexts. This study investigates whether instruction-based fine-tuning on public datasets improves the coding accuracy of open-weight LLMs for German tumor diagnosis texts. The evaluation uses coded diagnoses from the local tumor documentation system as test data. In a systematic data quality assessment, the upper limit for ICD-10 coding performance was estimated at 60-79% for exact and 81-94% for partial (three-character codes only) derivation. As training data, over 500,000 question-answer pairs were created based on the ICD-10-GM, ICD-O-3, and OPS catalogues. Eight open-weight models from the Qwen, Llama, and Mistral families (7-70 B parameters) were fine-tuned. ICD-10-GM accuracy rose from 1.4-24% to 41-58%, and partial accuracy from 31-74% to 73-83%. The accuracy of ICD-O-3 topography coding also improved but started and remained considerably lower with an exact accuracy of 22-40% and a partial accuracy of 56-67% after fine-tuning. Malformed code outputs dropped to 0% for all models. Tumor-diagnosis recognition reached 99%. Accuracy correlated positively with model size, but gaps between small and large models narrowed after fine-tuning. The reasoning mode in Qwen3 generally yielded a lower performance than fine-tuning and was over 100 times slower. Our findings highlight the potential of leveraging public catalogues to build instruction datasets that improve LLMs in medical documentation tasks. The complete training dataset and the best-performing checkpoints of the fine-tuned models are available from https://huggingface.co/datasets/stefan-m-lenz/ICDOPS-QA-2024.

[33] Closing the Gap Between Text and Speech Understanding in LLMs

Santiago Cuervo,Skyler Seto,Maureen de Seyssel,Richard He Bai,Zijin Gu,Tatiana Likhomanenko,Navdeep Jaitly,Zakaria Aldeneh

Main category: cs.CL

TL;DR: 论文提出了SALAD方法，通过跨模态蒸馏和目标合成数据，高效缩小文本与语音理解之间的性能差距，减少数据需求并避免遗忘文本能力。

Details

Motivation: 语音适应的LLM在处理语音输入时性能显著低于文本输入，现有方法依赖大规模合成数据或专有数据集，成本高且不可复现。

Contribution: 引入SALAD方法，结合跨模态蒸馏和目标合成数据，高效对齐语音与文本模态，显著减少数据需求。

Method: SALAD方法通过跨模态蒸馏和主动选择目标合成数据，提升对齐效果并避免遗忘文本能力。

Result: 在3B和7B LLM上，SALAD仅用少量公开语音数据即达到与强开放式模型相当的广泛领域性能。

Insight: 文本-语音理解差距主要由遗忘和模态不对齐驱动，SALAD通过高效数据利用解决了这两点。

Abstract: Large Language Models (LLMs) can be adapted to extend their text capabilities to speech inputs. However, these speech-adapted LLMs consistently underperform their text-based counterparts–and even cascaded pipelines–on language understanding tasks. We term this shortfall the text-speech understanding gap: the performance drop observed when a speech-adapted LLM processes spoken inputs relative to when the original text-based LLM processes the equivalent text. Recent approaches to narrowing this gap either rely on large-scale speech synthesis of text corpora, which is costly and heavily dependent on synthetic data, or on large-scale proprietary speech datasets, which are not reproducible. As a result, there remains a need for more data-efficient alternatives for closing the text-speech understanding gap. In this work, we analyze the gap as driven by two factors: (i) forgetting of text capabilities during adaptation, and (ii) cross-modal misalignment between speech and text. Based on this analysis, we introduce SALAD–Sample-efficient Alignment with Learning through Active selection and cross-modal Distillation–which combines cross-modal distillation with targeted synthetic data to improve alignment while mitigating forgetting. Applied to 3B and 7B LLMs, SALAD achieves competitive performance with a strong open-weight model across broad-domain benchmarks in knowledge, language understanding, and reasoning, while training on over an order of magnitude less speech data from public corpora.

[34] NExT-OMNI: Towards Any-to-Any Omnimodal Foundation Models with Discrete Flow Matching

Run Luo,Xiaobo Xia,Lu Wang,Longze Chen,Renke Shan,Jing Luo,Min Yang,Tat-Seng Chua

Main category: cs.CL

TL;DR: NExT-OMNI 是一种开源的全模态基础模型，通过离散流范式实现统一建模，支持任意模态间的理解和生成，优于现有统一模型。

Details

Motivation: 现有跨模态生成和多轮交互的基础模型多受限于自回归架构，限制了理解和生成能力的平衡整合，而混合或解耦设计又不够高效。

Contribution: 1. 提出 NExT-OMNI，支持任意模态间统一建模；2. 利用离散流范式提升响应效率和应用场景广度；3. 开源代码和模型检查点。

Method: 基于离散流范式，利用度量诱导概率路径和动力学最优速度，实现统一表示和多模态交互。

Result: 在生成和理解基准测试中表现优异，多轮交互和跨模态检索性能超越先前统一模型。

Insight: 离散流范式为多模态统一建模提供了高效解决方案，未来可能在更广泛任务中发挥作用。

Abstract: Next-generation multimodal foundation models capable of any-to-any cross-modal generation and multi-turn interaction will serve as core components of artificial general intelligence systems, playing a pivotal role in human-machine interaction. However, most existing multimodal models remain constrained by autoregressive architectures, whose inherent limitations prevent a balanced integration of understanding and generation capabilities. Although hybrid and decoupling strategies have been explored to address these tasks within unified frameworks separately, their redundant, non-integrated designs limit their applicability to broader scenarios, such as cross-modal retrieval.In this work, we introduce NExT-OMNI, an open-source omnimodal foundation model that achieves unified modeling through discrete flow paradigms. By leveraging metric-induced probability paths and kinetic optimal velocities, NExT-OMNI natively supports any-to-any understanding and generation with enhanced response efficiency, while enabling broader application scenarios through concise unified representations rather than task-decoupled designs. Trained on large-scale interleaved text, image, video, and audio data, NExT-OMNI delivers competitive performance on multimodal generation and understanding benchmarks, while outperforming prior unified models in multi-turn multimodal interaction and cross-modal retrieval, highlighting its architectural advantages as a next-generation multimodal foundation model. To advance further research, we release training details, data protocols, and open-source both the code and model checkpoints.

[35] GAPS: A Clinically Grounded, Automated Benchmark for Evaluating AI Clinicians

Xiuyuan Chen,Tao Sun,Dexin Su,Ailing Yu,Junwei Liu,Zhe Chen,Gangzeng Jin,Xin Wang,Jingnan Liu,Hansong Xiao,Hualei Zhou,Dongjie Tao,Chunxiao Guo,Minghui Yang,Yuan Xia,Jing Zhao,Qianrui Fan,Yanyun Wang,Shuai Zhen,Kezhong Chen,Jun Wang,Zewen Sun,Heng Zhao,Tian Guan,Shaodong Wang,Geyun Chang,Jiaming Deng,Hongchengcheng Chen,Kexin Feng,Ruzhen Li,Jiayi Geng,Changtai Zhao,Jun Wang,Guihu Lin,Peihao Li,Liqi Liu,Peng Wei,Jian Wang,Jinjie Gu,Ping Wang,Fan Yang

Main category: cs.CL

TL;DR: 该论文提出了GAPS框架，一个多维度的自动化评测范式，用于评估AI医生系统的认知深度、答案完整性、鲁棒性和安全性，并通过验证展示了其有效性。

Details

Motivation: 现有AI医生系统的评测基准无法全面捕捉实际临床实践所需的深度、鲁棒性和安全性，需要一种自动化、可扩展且基于临床指南的方法来解决这一问题。

Contribution: 提出了GAPS框架，结合认知深度（G）、答案完整性（A）、鲁棒性（P）和安全性（S），并开发了一个完全自动化的评测流水线，克服了以往工作的可扩展性和主观性限制。

Method: 通过构建证据邻域、生成双图和树表示，自动生成多层级问题，并使用DeepResearch代理合成评分标准，最后由LLM集成评分。

Result: 验证表明自动化生成的问题质量高且符合临床判断，评测揭示了现有模型在深度推理、答案完整性和对抗扰动等关键方面的短板。

Insight: GAPS为AI医生系统提供了一种可复现、可扩展的评测方法，有助于指导其向更安全、可靠的临床实践发展。

Abstract: Current benchmarks for AI clinician systems, often based on multiple-choice exams or manual rubrics, fail to capture the depth, robustness, and safety required for real-world clinical practice. To address this, we introduce the GAPS framework, a multidimensional paradigm for evaluating \textbf{G}rounding (cognitive depth), \textbf{A}dequacy (answer completeness), \textbf{P}erturbation (robustness), and \textbf{S}afety. Critically, we developed a fully automated, guideline-anchored pipeline to construct a GAPS-aligned benchmark end-to-end, overcoming the scalability and subjectivity limitations of prior work. Our pipeline assembles an evidence neighborhood, creates dual graph and tree representations, and automatically generates questions across G-levels. Rubrics are synthesized by a DeepResearch agent that mimics GRADE-consistent, PICO-driven evidence review in a ReAct loop. Scoring is performed by an ensemble of large language model (LLM) judges. Validation confirmed our automated questions are high-quality and align with clinician judgment. Evaluating state-of-the-art models on the benchmark revealed key failure modes: performance degrades sharply with increased reasoning depth (G-axis), models struggle with answer completeness (A-axis), and they are highly vulnerable to adversarial perturbations (P-axis) as well as certain safety issues (S-axis). This automated, clinically-grounded approach provides a reproducible and scalable method for rigorously evaluating AI clinician systems and guiding their development toward safer, more reliable clinical practice.

[36] Confidence-Based Response Abstinence: Improving LLM Trustworthiness via Activation-Based Uncertainty Estimation

Zhiqi Huang,Vivek Datla,Chenyang Zhu,Alfy Samuel,Daben Liu,Anoop Kumar,Ritesh Soni

Main category: cs.CL

TL;DR: 该论文提出了一种基于FFN激活的置信度估计方法，用于提高检索增强生成（RAG）系统的可信度，特别适用于高风险领域的应用。

Details

Motivation: 在高风险领域（如金融和医疗），LLM输出的错误代价极高，因此需要一种能够更准确地估计置信度的机制，以提高系统的可信度。

Contribution: 提出了一种基于FFN激活的置信度估计方法，避免了传统方法中投影和softmax归一化导致的信息损失，并通过Huber损失提升训练鲁棒性。

Method: 将置信度预测建模为序列分类任务，利用FFN的原始激活作为自回归信号，并结合Huber损失进行训练正则化。

Result: 在金融行业客户支持的复杂知识库场景中，该方法优于基线模型，并在严格延迟约束下保持高准确率。实验表明仅使用第16层的激活即可保持精度并降低延迟。

Insight: 基于激活的置信度建模是一种可扩展且与架构感知的方法，有助于实现可信的RAG系统部署。

Abstract: We propose a method for confidence estimation in retrieval-augmented generation (RAG) systems that aligns closely with the correctness of large language model (LLM) outputs. Confidence estimation is especially critical in high-stakes domains such as finance and healthcare, where the cost of an incorrect answer outweighs that of not answering the question. Our approach extends prior uncertainty quantification methods by leveraging raw feed-forward network (FFN) activations as auto-regressive signals, avoiding the information loss inherent in token logits and probabilities after projection and softmax normalization. We model confidence prediction as a sequence classification task, and regularize training with a Huber loss term to improve robustness against noisy supervision. Applied in a real-world financial industry customer-support setting with complex knowledge bases, our method outperforms strong baselines and maintains high accuracy under strict latency constraints. Experiments on Llama 3.1 8B model show that using activations from only the 16th layer preserves accuracy while reducing response latency. Our results demonstrate that activation-based confidence modeling offers a scalable, architecture-aware path toward trustworthy RAG deployment.

[37] The Mechanistic Emergence of Symbol Grounding in Language Models

Shuyu Wu,Ziqiao Ma,Xiaoxi Luo,Yidong Huang,Josue Torres-Fonseca,Freda Shi,Joyce Chai

Main category: cs.CL

TL;DR: 该论文探讨了符号接地（symbol grounding）在大规模语言模型中的涌现机制，并提出了一种系统性评估框架来追踪其在内部分布式计算中的表现。研究发现接地现象主要集中在中间层，并通过注意力头的聚合机制实现。

Details

Motivation: 探索符号接地在语言模型中如何自然而然涌现，并揭示其具体的实现机制和位置，以期为生成模型的可靠性预测与控制提供理论基础。

Contribution: 提出了一种可控的评估框架，通过机制和因果分析系统地追踪符号接地在内部分布式计算中的表现；揭示了接地现象的实现机制和具体位置。

Method: 采用机制分析和因果分析的方法，设计了一个系统化的评估框架，分析了语言模型中符号接地的分布和实现过程。

Result: 接地现象集中在中间层，通过注意力头的聚合机制实现；在多模态对话和不同架构（Transformer和状态空间模型）中表现一致，但在单向LSTM中未观察到此现象。

Insight: 符号接地可以在大规模语言模型中自然涌现，无需显式的接地目标；其对生成模型的可靠性预测与控制具有潜在的实际意义。

Abstract: Symbol grounding (Harnad, 1990) describes how symbols such as words acquire their meanings by connecting to real-world sensorimotor experiences. Recent work has shown preliminary evidence that grounding may emerge in (vision-)language models trained at scale without using explicit grounding objectives. Yet, the specific loci of this emergence and the mechanisms that drive it remain largely unexplored. To address this problem, we introduce a controlled evaluation framework that systematically traces how symbol grounding arises within the internal computations through mechanistic and causal analysis. Our findings show that grounding concentrates in middle-layer computations and is implemented through the aggregate mechanism, where attention heads aggregate the environmental ground to support the prediction of linguistic forms. This phenomenon replicates in multimodal dialogue and across architectures (Transformers and state-space models), but not in unidirectional LSTMs. Our results provide behavioral and mechanistic evidence that symbol grounding can emerge in language models, with practical implications for predicting and potentially controlling the reliability of generation.

[38] Breadcrumbs Reasoning: Memory-Efficient Reasoning with Compression Beacons

Giovanni Monea,Yair Feldman,Shankar Padmanabhan,Kianté Brantley,Yoav Artzi

Main category: cs.CL

TL;DR: 本文提出了一种名为Breadcrumbs Reasoning的内存高效推理方法，通过压缩Transformer的KV缓存来解决长上下文推理中的内存和计算成本问题。

Details

Motivation: 大规模语言模型在长上下文推理中的可扩展性受到Transformer键值缓存线性增长的严重限制，导致高内存和计算成本。随着推理生成的进行，过去生成的token信息价值逐渐降低，这为压缩提供了机会。

Contribution: 提出了一种周期性压缩KV缓存的方法，使用一种特殊的学习token进行压缩并清除压缩条目。通过联合蒸馏和强化学习（RL）框架训练模型实现压缩。

Method: 采用改进的联合蒸馏和强化学习（RL）框架训练模型，利用RL输出进行蒸馏，最小化训练开销。压缩KV缓存以降低内存占用。

Result: 实验表明，该方法在不损失准确性的情况下，显著降低了内存占用，优于无缓存压缩和免训练压缩技术。

Insight: 通过动态压缩过去生成的token信息，可以在保持推理能力的同时大幅减少内存需求，为长上下文推理的扩展提供了新思路。

Abstract: The scalability of large language models for long-context reasoning is severely constrained by the linear growth of their Transformer key-value cache, which incurs significant memory and computational costs. We posit that as a model generates reasoning tokens, the informational value of past generated tokens diminishes, creating an opportunity for compression. In this work, we propose to periodically compress the generation KV cache with a learned, special-purpose token and evict compressed entries. We train the model to perform this compression via a modified joint distillation and reinforcement learning (RL) framework. Our training method minimizes overhead over the conventional RL process, as it leverages RL outputs for distillation. Empirically, our method achieves a superior memory-accuracy Pareto frontier compared to both the model without cache compression and training-free compression techniques.

[39] BRIEF-Pro: Universal Context Compression with Short-to-Long Synthesis for Fast and Accurate Multi-Hop Reasoning

Jia-Chen Gu,Junyi Zhang,Di Wu,Yuankai Li,Kai-Wei Chang,Nanyun Peng

Main category: cs.CL

TL;DR: BRIEF-Pro 是一种轻量级通用压缩器，通过短到长合成的方法压缩长上下文信息，提升多跳推理任务的效率和准确性。

Details

Motivation: 随着检索增强生成（RAG）处理复杂任务的增加，扩展的上下文提供了更丰富的信息，但带来了高延迟和模型认知负荷增加的问题。BRIEF-Pro旨在解决这一瓶颈。

Contribution: 提出了BRIEF-Pro，一种通用轻量级压缩器，能够从检索文档中提取相关证据并为查询生成简洁摘要，显著提升多跳推理任务的性能。

Method: BRIEF-Pro利用短上下文种子数据训练，实现对超过10k词的长上下文进行抽象压缩，并支持用户灵活控制摘要长度。

Result: 在四个开放域多跳问答数据集上，BRIEF-Pro生成的摘要更简洁且相关，压缩率32倍时比LongLLMLingua的9倍压缩性能平均提升4.67%，计算开销仅需23%。

Insight: 通过压缩长上下文并保持信息相关性，BRIEF-Pro显著降低了计算开销，同时提升了模型的推理性能。

Abstract: As retrieval-augmented generation (RAG) tackles complex tasks, increasingly expanded contexts offer richer information, but at the cost of higher latency and increased cognitive load on the model. To mitigate this bottleneck, especially for intricate multi-hop questions, we introduce BRIEF-Pro. It is a universal, lightweight compressor that distills relevant evidence for a given query from retrieved documents into a concise summary for seamless integration into in-context RAG. Using seed data consisting of relatively short contexts (fewer than 1k words), BRIEF-Pro is trained to perform abstractive compression of extended contexts exceeding 10k words across a wide range of scenarios. Furthermore, BRIEF-Pro offers flexible user control over summary length by allowing users to specify the desired number of sentences. Experiments on four open-domain multi-hop question-answering datasets show that BRIEF-Pro generates more concise and relevant summaries, enhancing performance across small, large, and proprietary language models. With the 70B reader model, 32x compression by BRIEF-Pro improves QA performance by 4.67% on average over LongLLMLingua’s 9x, while requiring only 23% of its computational overhead.

cs.CV [Back]

[40] State-Change Learning for Prediction of Future Events in Endoscopic Videos

Saurav Sharma,Chinedu Innocent Nwoye,Didier Mutter,Nicolas Padoy

Main category: cs.CV

TL;DR: 该论文提出了一种基于状态变化学习的手术未来事件预测方法SurgFUTR，通过教师-学生架构和动作动态模块提升预测性能，并在多任务数据集上验证了其有效性。

Details

Motivation: 当前手术AI研究集中在理解当前事件，而非预测未来事件，且现有方法多为任务特定、缺乏统一框架，难以在不同手术场景中泛化。

Contribution: 1) 将手术未来预测重新定义为状态变化学习问题；2) 提出SurgFUTR框架，结合教师-学生架构和Action Dynamics模块；3) 建立多任务基准数据集SFPBench。

Method: 1) 通过Sinkhorn-Knopp聚类压缩视频片段为状态表示；2) 教师网络学习当前和未来片段，学生网络仅从当前视频预测未来状态；3) Action Dynamics模块增强动态建模。

Result: 在四个数据集上的实验表明，方法在短期（动作三元组、事件）和长期（剩余手术时间、阶段/步骤转换）预测任务中均优于基线。

Insight: 状态变化学习优于直接预测原始观测值，能够更好地泛化到不同手术场景，同时多任务基准推动了未来研究。

Abstract: Surgical future prediction, driven by real-time AI analysis of surgical video, is critical for operating room safety and efficiency. It provides actionable insights into upcoming events, their timing, and risks-enabling better resource allocation, timely instrument readiness, and early warnings for complications (e.g., bleeding, bile duct injury). Despite this need, current surgical AI research focuses on understanding what is happening rather than predicting future events. Existing methods target specific tasks in isolation, lacking unified approaches that span both short-term (action triplets, events) and long-term horizons (remaining surgery duration, phase transitions). These methods rely on coarse-grained supervision while fine-grained surgical action triplets and steps remain underexplored. Furthermore, methods based only on future feature prediction struggle to generalize across different surgical contexts and procedures. We address these limits by reframing surgical future prediction as state-change learning. Rather than forecasting raw observations, our approach classifies state transitions between current and future timesteps. We introduce SurgFUTR, implementing this through a teacher-student architecture. Video clips are compressed into state representations via Sinkhorn-Knopp clustering; the teacher network learns from both current and future clips, while the student network predicts future states from current videos alone, guided by our Action Dynamics (ActDyn) module. We establish SFPBench with five prediction tasks spanning short-term (triplets, events) and long-term (remaining surgery duration, phase and step transitions) horizons. Experiments across four datasets and three procedures show consistent improvements. Cross-procedure transfer validates generalizability.

[41] Unifying Vision-Language Latents for Zero-label Image Caption Enhancement

Sanghyun Byun,Jung Ick Guack,Mohanad Odema,Baisub Lee,Jacob Song,Woo Seong Chung

Main category: cs.CV

TL;DR: ViZer提出了一种无需标注数据的视觉-语言对齐训练框架，用于增强图像描述任务，并在实验中展示了质量提升。

Details

Motivation: 现有视觉-语言模型依赖标注数据，限制了其扩展性和对未标注数据的利用。ViZer旨在解决这一问题。

Contribution: 提出了Unified Vision-Language Alignment框架（ViZer），能够在无标注数据的情况下提升图像描述的质量。

Method: ViZer通过在训练中主动对齐视觉和语言的表征特征，避免了传统方法对标注数据的依赖。

Result: 应用ViZer后，SmolVLM-Base和Qwen2-VL生成的描述更接地气且更丰富，超越了基线模型。

Insight: 自动评价指标（如CIDEr和BERTScore）可能忽略细节，而ViZer在定性评估中表现更优。

Abstract: Vision-language models (VLMs) achieve remarkable performance through large-scale image-text pretraining. However, their reliance on labeled image datasets limits scalability and leaves vast amounts of unlabeled image data underutilized. To address this, we propose Unified Vision-Language Alignment for Zero-Label Enhancement (ViZer), an enhancement training framework that enables zero-label learning in image captioning, providing a practical starting point for broader zero-label adaptation in vision-language tasks. Unlike prior approaches that rely on human or synthetically annotated datasets, ViZer actively aligns vision and language representation features during training, enabling existing VLMs to generate improved captions without requiring text labels or full retraining. We demonstrate ViZer’s advantage in qualitative evaluation, as automated caption metrics such as CIDEr and BERTScore often penalize details that are absent in reference captions. Applying ViZer on SmolVLM-Base and Qwen2-VL, we observe consistent qualitative improvements, producing captions that are more grounded and descriptive than their baseline.

[42] Epistemic-aware Vision-Language Foundation Model for Fetal Ultrasound Interpretation

Xiao He,Huangxuan Zhao,Guojia Wan,Wei Zhou,Yanxing Liu,Juhua Liu,Yongchao Xu,Yong Luo,Dacheng Tao,Bo Du

Main category: cs.CV

TL;DR: 本文提出了FetalMind，一种专为胎儿超声设计的医疗AI系统，通过Salient Epistemic Disentanglement（SED）方法解决了胎儿超声的多视图推理和疾病多样性问题，并在大规模数据集FetalSigma-1M上取得了显著性能提升。

Details

Motivation: 现有的医疗视觉语言模型多用于结构化成人影像，而在胎儿超声领域中表现不佳，主要因多视图推理、疾病多样性和图像异质性等问题。本文旨在解决这些问题。

Contribution: 1. 提出了FetalMind系统，专门针对胎儿超声的报告生成和诊断任务；2. 提出了SED方法，通过注入专家设计的二分图和强化学习来解耦视图与疾病的关联；3. 构建了首个大规模胎儿超声报告数据集FetalSigma-1M。

Method: 采用Salient Epistemic Disentanglement（SED）方法，结合专家设计的二分图和强化学习，解耦视图与疾病的关联并优化临床推理步骤。

Result: FetalMind在所有妊娠阶段均优于开源和闭源的基线模型，平均提升14%，对关键条件的准确性提高了61.2%。

Insight: 通过解耦视图与疾病的关联和专家引导的设计，可以有效降低胎儿超声任务的复杂性，提升模型的性能和实践一致性。

Abstract: Recent medical vision-language models have shown promise on tasks such as VQA, report generation, and anomaly detection. However, most are adapted to structured adult imaging and underperform in fetal ultrasound, which poses challenges of multi-view image reasoning, numerous diseases, and image diversity. To bridge this gap, we introduce FetalMind, a medical AI system tailored to fetal ultrasound for both report generation and diagnosis. Guided by clinical workflow, we propose Salient Epistemic Disentanglement (SED), which injects an expert-curated bipartite graph into the model to decouple view-disease associations and to steer preference selection along clinically faithful steps via reinforcement learning. This design mitigates variability across diseases and heterogeneity across views, reducing learning bottlenecks while aligning the model’s inference with obstetric practice. To train FetalMind at scale, we curate FetalSigma-1M dataset, the first large-scale fetal ultrasound report corpus, comprising 20K reports from twelve medical centers, addressing the scarcity of domain data. Extensive experiments show that FetalMind outperforms open- and closed-source baselines across all gestational stages, achieving +14% average gains and +61.2% higher accuracy on critical conditions while remaining efficient, stable, and scalable. Project Page: https://hexiao0275.github.io/FetalMind.

[43] CADE 2.5 - ZeResFDG: Frequency-Decoupled, Rescaled and Zero-Projected Guidance for SD/SDXL Latent Diffusion Models

Denis Rychkovskiy,GPT-5

Main category: cs.CV

TL;DR: 论文提出了CADE 2.5，一种针对SD/SDXL潜在扩散模型的采样器级指导堆栈，核心模块ZeResFDG结合了频率解耦、能量重缩放和零投影技术，显著提升了图像锐度、提示符依从性和伪影控制。

Details

Motivation: 为了解决潜在扩散模型在生成图像时的高频细节和低频结构之间的平衡问题，以及提升图像质量和稳定性。

Contribution: 提出了ZeResFDG模块和QSilk Micrograin Stabilizer，前者通过频率解耦和零投影优化生成过程，后者在推理时增强稳定性和高频纹理。

Method: ZeResFDG结合频率解耦指导、能量重缩放和零投影，QSilk则通过分位数钳制和细节注入提升高频纹理。

Result: 在不重新训练的情况下，显著提升了SD/SDXL模型的图像质量和稳定性。

Insight: 频率解耦和零投影技术的结合可以高效平衡图像的高频和低频成分，提升生成质量。

Abstract: We introduce CADE 2.5 (Comfy Adaptive Detail Enhancer), a sampler-level guidance stack for SD/SDXL latent diffusion models. The central module, ZeResFDG, unifies (i) frequency-decoupled guidance that reweights low- and high-frequency components of the guidance signal, (ii) energy rescaling that matches the per-sample magnitude of the guided prediction to the positive branch, and (iii) zero-projection that removes the component parallel to the unconditional direction. A lightweight spectral EMA with hysteresis switches between a conservative and a detail-seeking mode as structure crystallizes during sampling. Across SD/SDXL samplers, ZeResFDG improves sharpness, prompt adherence, and artifact control at moderate guidance scales without any retraining. In addition, we employ a training-free inference-time stabilizer, QSilk Micrograin Stabilizer (quantile clamp + depth/edge-gated micro-detail injection), which improves robustness and yields natural high-frequency micro-texture at high resolutions with negligible overhead. For completeness we note that the same rule is compatible with alternative parameterizations (e.g., velocity), which we briefly discuss in the Appendix; however, this paper focuses on SD/SDXL latent diffusion models.

[44] Scope: Selective Cross-modal Orchestration of Visual Perception Experts

Tianyu Zhang,Suyuchen Wang,Chao Wang,Juan Rodriguez,Ahmed Masry,Xiangru Jian,Yoshua Bengio,Perouz Taslakian

Main category: cs.CV

TL;DR: SCOPE提出了一种动态选择视觉编码器的框架，通过实例级路由而非传统MoE的令牌级路由，显著降低了计算成本并提升了性能。

Details

Motivation: 视觉语言模型（VLMs）常通过堆叠多个视觉编码器提升性能，但这种方法计算成本高且收益递减。SCOPE旨在通过智能选择编码器而非简单堆叠，解决这一问题。

Contribution: SCOPE是首个动态选择视觉编码器的框架，通过实例级路由和双熵正则化训练路由器，显著提升了效率。

Method: SCOPE结合共享编码器和路由编码器池，利用轻量级路由器基于跨模态注意力选择最佳编码器。训练中引入双熵正则化平衡负载和路由置信度。

Result: SCOPE仅使用一个共享编码器和一个路由编码器，性能优于同时使用四个编码器的模型，计算成本降低24-49%。

Insight: 智能选择编码器优于暴力堆叠，挑战了多编码器VLMs的主流范式。

Abstract: Vision-language models (VLMs) benefit from multiple vision encoders, but naively stacking them yields diminishing returns while multiplying inference costs. We propose SCOPE, a Mixture-of-Encoders (MoEnc) framework that dynamically selects one specialized encoder per image-text pair via instance-level routing, unlike token-level routing in traditional MoE. SCOPE maintains a shared encoder and a pool of routed encoders. A lightweight router uses cross-attention between text prompts and shared visual features to select the optimal encoder from the routed encoders. To train this router, we introduce dual entropy regularization with auxiliary losses to balance dataset-level load distribution with instance-level routing confidence. Remarkably, SCOPE with one shared plus one routed encoder outperforms models using all four extra encoders simultaneously, while reducing compute by 24-49%. This demonstrates that intelligent encoder selection beats brute-force aggregation, challenging the prevailing paradigm in multi-encoder VLMs.

[45] SVAG-Bench: A Large-Scale Benchmark for Multi-Instance Spatio-temporal Video Action Grounding

Tanveer Hannan,Shuaicong Wu,Mark Weber,Suprosanna Shit,Jindong Gu,Rajat Koner,Aljoša Ošep,Laura Leal-Taixé,Thomas Seidl

Main category: cs.CV

TL;DR: 论文提出了SVAG-Bench，一个大规模的多实例时空视频动作定位基准数据集，用于解决现有方法在联合检测、跟踪和时间定位多对象动作时的不足，并提出了基线框架SVAGFormer和评估工具SVAGEval。

Details

Motivation: 现有视频理解方法主要关注粗粒度动作识别或通用目标跟踪，而忽略了联合检测和跟踪多对象动作并对其进行时空定位的挑战。

Contribution: 1. 提出了SVAG任务和多实例时空视频动作定位基准SVAG-Bench；2. 设计了基线框架SVAGFormer；3. 开发了标准化评估工具SVAGEval。

Method: SVAGFormer基于先进的视觉语言模型，适应了联合时空定位的任务需求。

Result: 实验表明现有模型在密集或复杂场景中表现不佳，突出了对长视频中细粒度对象-动作交互进行高级推理的需求。

Insight: SVAG任务和基准推动了视频理解领域对多对象动作联合定位的研究，为更智能的AI系统提供了基础。

Abstract: Understanding fine-grained actions and accurately localizing their corresponding actors in space and time are fundamental capabilities for advancing next-generation AI systems, including embodied agents, autonomous platforms, and human-AI interaction frameworks. Despite recent progress in video understanding, existing methods predominantly address either coarse-grained action recognition or generic object tracking, thereby overlooking the challenge of jointly detecting and tracking multiple objects according to their actions while grounding them temporally. To address this gap, we introduce Spatio-temporal Video Action Grounding (SVAG), a novel task that requires models to simultaneously detect, track, and temporally localize all referent objects in videos based on natural language descriptions of their actions. To support this task, we construct SVAG-Bench, a large-scale benchmark comprising 688 videos, 19,590 annotated records, and 903 unique verbs, covering a diverse range of objects, actions, and real-world scenes. We further propose SVAGFormer, a baseline framework that adapts state of the art vision language models for joint spatial and temporal grounding, and introduce SVAGEval, a standardized evaluation toolkit for fair and reproducible benchmarking. Empirical results show that existing models perform poorly on SVAG, particularly in dense or complex scenes, underscoring the need for more advanced reasoning over fine-grained object-action interactions in long videos.

[46] SeqBench: Benchmarking Sequential Narrative Generation in Text-to-Video Models

Zhengxu Tang,Zizheng Wang,Luning Wang,Zitao Shuai,Chenhao Zhang,Siyu Qian,Yirui Wu,Bohao Wang,Haosong Rao,Zhenyu Yang,Chenwei Wu

Main category: cs.CV

TL;DR: SeqBench是一个专注于评估文本生成视频（T2V）模型中顺序叙事连贯性的新基准，通过设计和标注数据集，并提出基于动态时序图（DTG）的自动评估指标，揭示了当前模型的局限性。

Details

Motivation: 现有的T2V基准主要关注视频的视觉质量，而忽略了顺序叙事的连贯性，无法评估多事件逻辑推进的表现。

Contribution: 1) 提出了SeqBench基准，包含320个多样化叙事复杂度的提示和2,560个人工标注的视频；2) 设计了基于DTG的自动评估指标，高效捕捉长程依赖和时间顺序；3) 揭示了当前T2V模型在多动作序列中的关键问题。

Method: 1) 构建了带有多事件逻辑推进的提示数据集；2) 使用8个SOTA T2V模型生成视频并人工标注；3) 提出DTG指标，通过图模型分析视频中事件的依赖和时间顺序。

Result: DTG指标与人工标注高度相关，实验显示T2V模型在多动作序列中难以保持对象状态一致，生成结果物理不合理，且难以维持动作间的时序关系。

Insight: SeqBench首次系统化评估了T2V的叙事连贯性，为未来模型在顺序推理能力上的改进提供了具体方向。

Abstract: Text-to-video (T2V) generation models have made significant progress in creating visually appealing videos. However, they struggle with generating coherent sequential narratives that require logical progression through multiple events. Existing T2V benchmarks primarily focus on visual quality metrics but fail to evaluate narrative coherence over extended sequences. To bridge this gap, we present SeqBench, a comprehensive benchmark for evaluating sequential narrative coherence in T2V generation. SeqBench includes a carefully designed dataset of 320 prompts spanning various narrative complexities, with 2,560 human-annotated videos generated from 8 state-of-the-art T2V models. Additionally, we design a Dynamic Temporal Graphs (DTG)-based automatic evaluation metric, which can efficiently capture long-range dependencies and temporal ordering while maintaining computational efficiency. Our DTG-based metric demonstrates a strong correlation with human annotations. Through systematic evaluation using SeqBench, we reveal critical limitations in current T2V models: failure to maintain consistent object states across multi-action sequences, physically implausible results in multi-object scenarios, and difficulties in preserving realistic timing and ordering relationships between sequential actions. SeqBench provides the first systematic framework for evaluating narrative coherence in T2V generation and offers concrete insights for improving sequential reasoning capabilities in future models. Please refer to https://videobench.github.io/SeqBench.github.io/ for more details.

[47] SceneAdapt: Scene-aware Adaptation of Human Motion Diffusion

Jungbin Cho,Minsu Kim,Jisoo Kim,Ce Zheng,Laszlo A. Jeni,Ming-Hsuan Yang,Youngjae Yu,Seonjoo Kim

Main category: cs.CV

TL;DR: SceneAdapt提出了一种场景感知的人类运动生成框架，通过两个适配阶段（运动插值和场景感知插值）将场景信息注入文本条件运动模型，无需依赖大规模联合数据集。

Details

Motivation: 现有运动生成方法通常单独处理运动语义或场景感知，缺乏联合建模能力，因为构建兼具丰富文本-运动覆盖和精确场景交互的大规模数据集极具挑战性。

Contribution: 1) 引入SceneAdapt框架，通过两个适配阶段将场景感知注入文本条件运动模型；2) 提出关键帧层和场景条件层，分别用于运动插值和场景信息注入；3) 在不依赖联合数据的情况下实现场景感知运动生成。

Method: 1) 第一阶段通过运动插值（无需文本）作为代理任务连接两种数据集；2) 第二阶段通过跨注意力机制自适应查询局部场景几何信息；3) 采用关键帧层和场景条件层调制运动潜在空间。

Result: 实验表明SceneAdapt能有效为文本到运动模型注入场景感知，并分析了感知机制的涌现方式。

Insight: 运动插值可作为桥梁任务连接不同数据集，无需依赖大规模联合标注；跨注意力机制是实现局部场景信息注入的有效手段。

Abstract: Human motion is inherently diverse and semantically rich, while also shaped by the surrounding scene. However, existing motion generation approaches address either motion semantics or scene-awareness in isolation, since constructing large-scale datasets with both rich text–motion coverage and precise scene interactions is extremely challenging. In this work, we introduce SceneAdapt, a framework that injects scene awareness into text-conditioned motion models by leveraging disjoint scene–motion and text–motion datasets through two adaptation stages: inbetweening and scene-aware inbetweening. The key idea is to use motion inbetweening, learnable without text, as a proxy task to bridge two distinct datasets and thereby inject scene-awareness to text-to-motion models. In the first stage, we introduce keyframing layers that modulate motion latents for inbetweening while preserving the latent manifold. In the second stage, we add a scene-conditioning layer that injects scene geometry by adaptively querying local context through cross-attention. Experimental results show that SceneAdapt effectively injects scene awareness into text-to-motion models, and we further analyze the mechanisms through which this awareness emerges. Code and models will be released.

[48] One Dimensional CNN ECG Mamba for Multilabel Abnormality Classification in 12 Lead ECG

Huawei Jiang,Husna Mutahira,Gan Huang,Mannan Saeed Muhammad

Main category: cs.CV

TL;DR: 论文提出了一种结合1D CNN和Mamba的混合框架（ECG Mamba），用于12导联心电图的多标签异常分类。该方法在PhysioNet比赛中表现优于现有技术，显著提升了AUPRC和AUROC分数。

Details

Motivation: 传统深度学习模型（如残差网络和Transformer）在处理长时序心电图信号时性能有限，而状态空间模型（如Mamba）提供了一种高效替代方案。

Contribution: 1. 提出了一种结合卷积特征提取和Mamba的混合框架；2. 通过Vision Mamba双向变体增强时序依赖性表示；3. 在PhysioNet比赛中实现了优于现有方法的性能。

Method: 1. 使用1D CNN提取心电图特征；2. 结合Mamba（选择性状态空间模型）建模长时序依赖关系；3. 采用Vision Mamba的双向变体进一步提升时序表示能力。

Result: 在PhysioNet 2020和2021数据集上，模型显著超越了现有方法，AUPRC和AUROC分数更高。

Insight: Mamba架构在长时序信号处理（如心电图）中表现优异，有望推动可靠的心电图分类技术，支持远程医疗和资源受限的医疗系统。

Abstract: Accurate detection of cardiac abnormalities from electrocardiogram recordings is regarded as essential for clinical diagnostics and decision support. Traditional deep learning models such as residual networks and transformer architectures have been applied successfully to this task, but their performance has been limited when long sequential signals are processed. Recently, state space models have been introduced as an efficient alternative. In this study, a hybrid framework named One Dimensional Convolutional Neural Network Electrocardiogram Mamba is introduced, in which convolutional feature extraction is combined with Mamba, a selective state space model designed for effective sequence modeling. The model is built upon Vision Mamba, a bidirectional variant through which the representation of temporal dependencies in electrocardiogram data is enhanced. Comprehensive experiments on the PhysioNet Computing in Cardiology Challenges of 2020 and 2021 were conducted, and superior performance compared with existing methods was achieved. Specifically, the proposed model achieved substantially higher AUPRC and AUROC scores than those reported by the best previously published algorithms on twelve lead electrocardiograms. These results demonstrate the potential of Mamba-based architectures to advance reliable ECG classification. This capability supports early diagnosis and personalized treatment, while enhancing accessibility in telemedicine and resource-constrained healthcare systems.

[49] True Self-Supervised Novel View Synthesis is Transferable

Thomas W. Mitchel,Hyunwoo Ryu,Vincent Sitzmann

Main category: cs.CV

TL;DR: 论文提出了一种真正自监督的新视角合成方法XFactor，通过引入可转移性作为核心标准，解决了现有方法在姿态预测上不可转移的问题。

Details

Motivation: 现有自监督新视角合成方法的姿态预测在不同场景中无法保持一致性，导致无法真正实现新视角合成。论文旨在解决这一问题，提出可转移性作为新标准。

Contribution: 1. 提出XFactor，首个无需几何先验的自监督模型，能够实现真正的新视角合成；2. 引入新的可转移性度量标准；3. 实验表明XFactor优于现有方法，且隐式姿态与实际姿态高度相关。

Method: XFactor结合成对姿态估计与输入输出的简单增强方案，解耦相机姿态与场景内容，促进几何推理。无需显式SE(3)参数化或多视角几何先验。

Result: XFactor在可转移性上显著优于现有姿态无关的新视角合成转换器，隐式姿态与实际姿态高度相关。

Insight: 自监督新视角合成的关键在于姿态的可转移性，而无需依赖显式几何先验。解耦姿态与场景内容是实现真正新视角合成的有效途径。

Abstract: In this paper, we identify that the key criterion for determining whether a model is truly capable of novel view synthesis (NVS) is transferability: Whether any pose representation extracted from one video sequence can be used to re-render the same camera trajectory in another. We analyze prior work on self-supervised NVS and find that their predicted poses do not transfer: The same set of poses lead to different camera trajectories in different 3D scenes. Here, we present XFactor, the first geometry-free self-supervised model capable of true NVS. XFactor combines pair-wise pose estimation with a simple augmentation scheme of the inputs and outputs that jointly enables disentangling camera pose from scene content and facilitates geometric reasoning. Remarkably, we show that XFactor achieves transferability with unconstrained latent pose variables, without any 3D inductive biases or concepts from multi-view geometry – such as an explicit parameterization of poses as elements of SE(3). We introduce a new metric to quantify transferability, and through large-scale experiments, we demonstrate that XFactor significantly outperforms prior pose-free NVS transformers, and show that latent poses are highly correlated with real-world poses through probing experiments.

[50] Direction-aware multi-scale gradient loss for infrared and visible image fusion

Kaixuan Yang,Wei Xiang,Zhenshuai Chen,Tong Jin,Yunpeng Liu

Main category: cs.CV

TL;DR: 这篇论文提出了一种方向感知的多尺度梯度损失函数，用于红外和可见光图像的融合，通过单独监督水平和垂直梯度分量并保留其符号，显著提升了边缘保真度和纹理细节。

Details

Motivation: 现有方法通过将梯度坍缩为幅值丢失了方向信息，导致边缘模糊和次优的监督效果。作者希望通过保留梯度的方向性和多尺度特征，改进融合图像的边缘质量和纹理细节。

Contribution: 提出了方向感知的多尺度梯度损失函数，能够分别在水平和垂直方向上单独监督梯度分量，并保留符号信息，从而在多尺度上提供清晰的梯度方向指导。

Method: 方法设计了一个新的梯度损失函数，目标是在水平和垂直方向上分别计算梯度分量，并在多尺度上保留符号信息。这一方法无需改变模型架构或训练流程。

Result: 实验结果表明，该方法在开放模型和多个公开数据集上表现优异，能够提升融合图像的边缘锐度和纹理细节。

Insight: 梯度方向信息的保留对于提升图像融合任务的性能至关重要，特别是在边缘和纹理细节的保真度方面。

Abstract: Infrared and visible image fusion aims to integrate complementary information from co-registered source images to produce a single, informative result. Most learning-based approaches train with a combination of structural similarity loss, intensity reconstruction loss, and a gradient-magnitude term. However, collapsing gradients to their magnitude removes directional information, yielding ambiguous supervision and suboptimal edge fidelity. We introduce a direction-aware, multi-scale gradient loss that supervises horizontal and vertical components separately and preserves their sign across scales. This axis-wise, sign-preserving objective provides clear directional guidance at both fine and coarse resolutions, promoting sharper, better-aligned edges and richer texture preservation without changing model architectures or training protocols. Experiments on open-source model and multiple public benchmarks demonstrate effectiveness of our approach.

[51] Unsupervised Domain Adaptation via Content Alignment for Hippocampus Segmentation

Hoda Kalabizadeh,Ludovica Griffanti,Pak-Hei Yeung,Ana I. L. Namburete,Nicola K. Dinsdale,Konstantinos Kamnitsas

Main category: cs.CV

TL;DR: 该论文提出了一种针对医学图像分割的无监督域自适应框架，通过内容对齐解决跨域海马体分割中的领域偏移问题，结合风格归一化和双向可变形图像配准策略，显著提升了分割精度。

Details

Motivation: 医学图像分割模型在不同数据集上部署时，由于领域偏移（包括风格和内容的变化）导致性能下降，特别是海马体分割任务中内容变化的挑战。

Contribution: 提出了结合风格归一化和双向可变形图像配准的无监督域自适应框架，专注于对齐内容变化，显著提升了跨域分割性能。

Method: 1. 使用z归一化进行风格对齐；2. 结合双向可变形图像配准（DIR）、分割网络和判别器网络进行联合训练，生成解剖学合理的变换以对齐源域和目标域图像。

Result: 在合成数据集和三个MRI数据集上的实验表明，该方法优于基线方法，特别是在内容偏移较大的情况下，Dice指标相对提升了15%。

Insight: 内容对齐在医学图像分割的无监督域自适应中至关重要，尤其是在病理变化较大的跨域任务中。

Abstract: Deep learning models for medical image segmentation often struggle when deployed across different datasets due to domain shifts - variations in both image appearance, known as style, and population-dependent anatomical characteristics, referred to as content. This paper presents a novel unsupervised domain adaptation framework that directly addresses domain shifts encountered in cross-domain hippocampus segmentation from MRI, with specific emphasis on content variations. Our approach combines efficient style harmonisation through z-normalisation with a bidirectional deformable image registration (DIR) strategy. The DIR network is jointly trained with segmentation and discriminator networks to guide the registration with respect to a region of interest and generate anatomically plausible transformations that align source images to the target domain. We validate our approach through comprehensive evaluations on both a synthetic dataset using Morpho-MNIST (for controlled validation of core principles) and three MRI hippocampus datasets representing populations with varying degrees of atrophy. Across all experiments, our method outperforms existing baselines. For hippocampus segmentation, when transferring from young, healthy populations to clinical dementia patients, our framework achieves up to 15% relative improvement in Dice score compared to standard augmentation methods, with the largest gains observed in scenarios with substantial content shift. These results highlight the efficacy of our approach for accurate hippocampus segmentation across diverse populations.

[52] Counting Hallucinations in Diffusion Models

Shuai Fu,Jian Zhou,Qi Chen,Huang Jing,Huy Anh Nguyen,Xiaohan Liu,Zhixiong Zeng,Lin Ma,Quanshi Zhang,Qi Wu

Main category: cs.CV

TL;DR: 该论文提出了对扩散概率模型（DPMs）中计数幻觉的系统量化方法，定义了计数幻觉并提出数据集CountHalluSet和标准化评估协议。

Details

Motivation: 扩散模型在生成任务中表现出色，但常产生与现实知识冲突的幻觉样本（如生成不合理数量的对象），缺乏系统性量化方法阻碍了改进和下一代模型设计。

Contribution: 1. 定义了计数幻觉；2. 构建了CountHalluSet数据集；3. 开发了标准化评估协议；4. 分析了采样条件与幻觉关系。

Method: 1. 设计三个子数据集（ToyShape, SimObject, RealHand）以定义计数标准；2. 通过不同采样条件（求解器类型、ODE求解器阶数、步数、初始噪声）实验量化幻觉水平。

Result: 研究发现FID等常用指标无法一致捕捉计数幻觉，且采样条件显著影响幻觉水平。

Insight: 系统性量化幻觉为改进扩散模型提供了新视角，揭示了当前评估指标的局限性。

Abstract: Diffusion probabilistic models (DPMs) have demonstrated remarkable progress in generative tasks, such as image and video synthesis. However, they still often produce hallucinated samples (hallucinations) that conflict with real-world knowledge, such as generating an implausible duplicate cup floating beside another cup. Despite their prevalence, the lack of feasible methodologies for systematically quantifying such hallucinations hinders progress in addressing this challenge and obscures potential pathways for designing next-generation generative models under factual constraints. In this work, we bridge this gap by focusing on a specific form of hallucination, which we term counting hallucination, referring to the generation of an incorrect number of instances or structured objects, such as a hand image with six fingers, despite such patterns being absent from the training data. To this end, we construct a dataset suite CountHalluSet, with well-defined counting criteria, comprising ToyShape, SimObject, and RealHand. Using these datasets, we develop a standardized evaluation protocol for quantifying counting hallucinations, and systematically examine how different sampling conditions in DPMs, including solver type, ODE solver order, sampling steps, and initial noise, affect counting hallucination levels. Furthermore, we analyze their correlation with common evaluation metrics such as FID, revealing that this widely used image quality metric fails to capture counting hallucinations consistently. This work aims to take the first step toward systematically quantifying hallucinations in diffusion models and offer new insights into the investigation of hallucination phenomena in image generation.

[53] Edit-Your-Interest: Efficient Video Editing via Feature Most-Similar Propagation

Yi Zuo,Zitao Wang,Lingling Li,Xu Liu,Fang Liu,Licheng Jiao

Main category: cs.CV

TL;DR: Edit-Your-Interest提出了一种轻量级、文本驱动的零样本视频编辑方法，通过时空特征记忆库和特征最相似传播，显著提高了计算效率和时序一致性。

Details

Motivation: 现有视频编辑方法计算开销高、内存消耗大，且存在时序不一致和视觉伪影问题，需要一种高效且保真度高的解决方案。

Contribution: 1) 设计了时空特征记忆库SFM；2) 提出了特征最相似传播FMP；3) 开发了SFM更新算法；4) 利用交叉注意力图自动提取目标掩码。

Method: 1) SFM缓存关键特征；2) FMP传播相关特征；3) SFM动态更新；4) 结合交叉注意力掩码进行精细编辑。

Result: 实验表明，Edit-Your-Interest在效率和视觉保真度上均优于现有方法。

Insight: 通过缓存和动态更新特征，可以有效减少计算开销并提升时序一致性。

Abstract: Text-to-image (T2I) diffusion models have recently demonstrated significant progress in video editing. However, existing video editing methods are severely limited by their high computational overhead and memory consumption. Furthermore, these approaches often sacrifice visual fidelity, leading to undesirable temporal inconsistencies and artifacts such as blurring and pronounced mosaic-like patterns. We propose Edit-Your-Interest, a lightweight, text-driven, zero-shot video editing method. Edit-Your-Interest introduces a spatio-temporal feature memory to cache features from previous frames, significantly reducing computational overhead compared to full-sequence spatio-temporal modeling approaches. Specifically, we first introduce a Spatio-Temporal Feature Memory bank (SFM), which is designed to efficiently cache and retain the crucial image tokens processed by spatial attention. Second, we propose the Feature Most-Similar Propagation (FMP) method. FMP propagates the most relevant tokens from previous frames to subsequent ones, preserving temporal consistency. Finally, we introduce an SFM update algorithm that continuously refreshes the cached features, ensuring their long-term relevance and effectiveness throughout the video sequence. Furthermore, we leverage cross-attention maps to automatically extract masks for the instances of interest. These masks are seamlessly integrated into the diffusion denoising process, enabling fine-grained control over target objects and allowing Edit-Your-Interest to perform highly accurate edits while robustly preserving the background integrity. Extensive experiments decisively demonstrate that the proposed Edit-Your-Interest outperforms state-of-the-art methods in both efficiency and visual fidelity, validating its superior effectiveness and practicality.

[54] EgoSocial: Benchmarking Proactive Intervention Ability of Omnimodal LLMs via Egocentric Social Interaction Perception

Xijun Wang,Tanay Sharma,Achin Kulshrestha,Abhimitra Meka,Aveek Purohit,Dinesh Manocha

Main category: cs.CV

TL;DR: 本文提出了EgoSocial数据集和EgoSoD方法，用于评估和改进多模态大语言模型（OLLMs）在社交互动中的主动干预能力，解决了当前AI缺乏社交意识的问题。

Details

Motivation: AR/VR技术普及后，AI需要具备从第一视角理解人类社交动态的能力，但目前的大语言模型（LLMs）在社交干预时机上表现不足，可能导致干扰自然对话。

Contribution: 1. 提出了EgoSocial数据集（13,500个社交视频-问题对），专注于社交干预评估；2. 提出EgoSoD方法，通过多模态上下文建模提升干预时机检测能力。

Method: EgoSoD整合多模态线索（如音频和视觉）到社交思维图中，动态建模参与者及其互动，从而精准检测干预时机和社交互动。

Result: EgoSoD显著提升了Phi-4和Gemini 2.5 Pro的性能（干预时机检测分别提升45.6%和9.9%，社交互动性能分别提升20.4%和6.9%）。

Insight: 当前OLLMs在社交干预时机检测上表现较差（Gemini 2.5 Pro仅14.4%），表明多模态社交动态建模仍有较大改进空间。

Abstract: As AR/VR technologies become integral to daily life, there’s a growing need for AI that understands human social dynamics from an egocentric perspective. However, current LLMs often lack the social awareness to discern when to intervene as AI assistant. This leads to constant, socially unaware responses that may disrupt natural conversation and negatively impact user focus. To address these limitations, we introduce EgoSocial, a large-scale egocentric dataset with 13,500 social video-question pairs, specifically designed to benchmark intervention in social interaction perception. We also present an in-depth analysis of current omnimodal LLMs (OLLMs) to assess their effectiveness in detecting diverse social contextual cues. Experiments show that OLLMs still struggle to detect the intervention timing (14.4% for Gemini 2.5 Pro). We also propose EgoSoD (EgoSocial Detection), an end-to-end method for robustly discerning social dynamics. Informed by our OLLM analysis, EgoSoD integrates multimodal contextual cues (e.g., audio and visual cues) into a social thinking graph, dynamically modeling participants and interactions. Our method proactively detects intervention timing and social interactions, precisely determining when to intervene. Our EgoSoD improves Phi-4 by 45.6% and Gemini 2.5 Pro by 9.9% on Intervention Timing performance, and improves Phi-4 by 20.4% and Gemini 2.5 Pro by 6.9% on overall Social Interaction performance. We will release the dataset and code soon.

[55] DriveCritic: Towards Context-Aware, Human-Aligned Evaluation for Autonomous Driving with Vision-Language Models

Jingyu Song,Zhenxin Li,Shiyi Lan,Xinglong Sun,Nadine Chang,Maying Shen,Joshua Chen,Katherine A. Skinner,Jose M. Alvarez

Main category: cs.CV

TL;DR: DriveCritic提出了一个基于视觉语言模型（VLM）的新框架，用于上下文感知的自动驾驶规划器评估，通过两阶段监督和强化学习流程微调模型，显著提升了与人类偏好的一致性。

Details

Motivation: 现有自动驾驶评估指标（如EPDMS）在复杂场景中缺乏上下文感知能力，难以与人类判断对齐。DriveCritic旨在解决这一问题，提供更可靠的评估基础。

Contribution: 1) DriveCritic数据集，包含需要上下文支持的场景及人类偏好的标注；2) DriveCritic模型，基于VLM，通过两阶段学习流程提升上下文感知能力。

Method: 采用两阶段监督和强化学习流程，将视觉与符号上下文结合，微调VLM模型以评估轨迹对。

Result: DriveCritic在匹配人类偏好和上下文感知方面显著优于现有指标和基线。

Insight: 上下文感知是自动驾驶评估的关键，DriveCritic通过结合视觉与语言信息，实现了更接近人类偏好的评估结果。

Abstract: Benchmarking autonomous driving planners to align with human judgment remains a critical challenge, as state-of-the-art metrics like the Extended Predictive Driver Model Score (EPDMS) lack context awareness in nuanced scenarios. To address this, we introduce DriveCritic, a novel framework featuring two key contributions: the DriveCritic dataset, a curated collection of challenging scenarios where context is critical for correct judgment and annotated with pairwise human preferences, and the DriveCritic model, a Vision-Language Model (VLM) based evaluator. Fine-tuned using a two-stage supervised and reinforcement learning pipeline, the DriveCritic model learns to adjudicate between trajectory pairs by integrating visual and symbolic context. Experiments show DriveCritic significantly outperforms existing metrics and baselines in matching human preferences and demonstrates strong context awareness. Overall, our work provides a more reliable, human-aligned foundation to evaluating autonomous driving systems.

[56] VPREG: An Optimal Control Formulation for Diffeomorphic Image Registration Based on the Variational Principle Grid Generation Method

Zicong Zhou,Baihan Zhao,Andreas Mang,Guojun Liao

Main category: cs.CV

TL;DR: VPREG是一种新颖的微分同胚图像配准方法，基于变分原理网格生成技术，优化了配准精度和变换质量，确保空间变换的正雅可比行列式和准确的反变换，显著优于现有方法。

Details

Motivation: 现有图像配准方法在控制变换质量和生成反变换方面存在不足，VPREG通过变分原理网格生成方法解决了这些问题。

Contribution: VPREG提出了基于变分原理的网格生成方法，实现了高质量的微分同胚变换和更准确的反变换，超越了现有配准方法的性能。

Method: VPREG使用变分原理网格生成技术，确保空间变换的正雅可比行列式和低旋度特性，并在微分同胚群中生成反变换。

Result: 在OASIS-1数据集上的实验表明，VPREG在Dice分数、变换规则性和反变换准确性上优于ANTs-SyN、Freesurfer-Easyreg和FSL-Fnirt。

Insight: VPREG的核心优势在于其能够在微分同胚群中精确控制变换质量，从而为神经影像分析提供了更可靠的配准工具。

Abstract: This paper introduces VPreg, a novel diffeomorphic image registration method. This work provides several improvements to our past work on mesh generation and diffeomorphic image registration. VPreg aims to achieve excellent registration accuracy while controlling the quality of the registration transformations. It ensures a positive Jacobian determinant of the spatial transformation and provides an accurate approximation of the inverse of the registration, a crucial property for many neuroimaging workflows. Unlike conventional methods, VPreg generates this inverse transformation within the group of diffeomorphisms rather than operating on the image space. The core of VPreg is a grid generation approach, referred to as \emph{Variational Principle} (VP), which constructs non-folding grids with prescribed Jacobian determinant and curl. These VP-generated grids guarantee diffeomorphic spatial transformations essential for computational anatomy and morphometry, and provide a more accurate inverse than existing methods. To assess the potential of the proposed approach, we conduct a performance analysis for 150 registrations of brain scans from the OASIS-1 dataset. Performance evaluation based on Dice scores for 35 regions of interest, along with an empirical analysis of the properties of the computed spatial transformations, demonstrates that VPreg outperforms state-of-the-art methods in terms of Dice scores, regularity properties of the computed transformation, and accuracy and consistency of the provided inverse map. We compare our results to ANTs-SyN, Freesurfer-Easyreg, and FSL-Fnirt.

[57] OS-HGAdapter: Open Semantic Hypergraph Adapter for Large Language Models Assisted Entropy-Enhanced Image-Text Alignment

Rongjun Chen,Chengsi Yao,Jinchang Ren,Xianxian Zeng,Peixian Wang,Jun Yuan,Jiawen Li,Huimin Zhao,Xu Lu

Main category: cs.CV

TL;DR: 论文提出OS-HGAdapter，利用大语言模型（LLM）填补文本与图像信息熵的差距，并通过超图适配器优化跨模态语义对齐，显著提升了检索性能。

Details

Motivation: 文本与图像信息熵差异导致传统跨模态对齐方法在互检索中存在不平衡问题，希望通过LLM和超图结构改善这一问题。

Contribution: 1）设计不依赖领域知识的提示模板，利用LLM增强文本多义性描述；2）引入超图适配器，在多模态间构建多边连接，优化语义对齐。

Method: 1.设计不依赖任务显式知识的提示模板，利用LLM增强文本的多义性描述；2.使用超图适配器在文本与图像模态间构建多边连接，校正嵌入空间中的语义匹配误差，同时降噪。

Result: 在Flickr30K和MS-COCO基准测试中，OS-HGAdapter取得了显著提升，文本到图像检索性能提升16.8%，图像到文本检索性能提升40.1%。

Insight: 通过LLM填补信息熵差距并利用超图适配器优化跨模态连接，能够显著提升语义对齐任务的性能，尤其是异构数据间的检索效果。

Abstract: Text-image alignment constitutes a foundational challenge in multimedia content understanding, where effective modeling of cross-modal semantic correspondences critically enhances retrieval system performance through joint embedding space optimization. Given the inherent difference in information entropy between texts and images, conventional approaches often show an imbalance in the mutual retrieval of these two modalities. To address this particular challenge, we propose to use the open semantic knowledge of Large Language Model (LLM) to fill for the entropy gap and reproduce the alignment ability of humans in these tasks. Our entropy-enhancing alignment is achieved through a two-step process: 1) a new prompt template that does not rely on explicit knowledge in the task domain is designed to use LLM to enhance the polysemy description of the text modality. By analogy, the information entropy of the text modality relative to the visual modality is increased; 2) A hypergraph adapter is used to construct multilateral connections between the text and image modalities, which can correct the positive and negative matching errors for synonymous semantics in the same fixed embedding space, whilst reducing the noise caused by open semantic entropy by mapping the reduced dimensions back to the original dimensions. Comprehensive evaluations on the Flickr30K and MS-COCO benchmarks validate the superiority of our Open Semantic Hypergraph Adapter (OS-HGAdapter), showcasing 16.8% (text-to-image) and 40.1% (image-to-text) cross-modal retrieval gains over existing methods while establishing new state-of-the-art performance in semantic alignment tasks.

[58] Real-Time Sign Language to text Translation using Deep Learning: A Comparative study of LSTM and 3D CNN

Madhumati Pol,Anvay Anturkar,Anushka Khot,Ayush Andure,Aniruddha Ghosh,Anvit Magadum,Anvay Bahadur

Main category: cs.CV

TL;DR: 该研究比较了3D CNN和LSTM在实时美式手语（ASL）翻译中的表现，发现3D CNN精度更高但计算成本更高，LSTM则资源消耗更低。混合模型表现中等，强调实际应用中需权衡精度与实时性。

Details

Motivation: 为了解决实时手语翻译中精度与计算效率的权衡问题，研究探索了3D CNN和LSTM在手语识别中的表现，以优化边缘计算环境下的性能。

Contribution: 提供了3D CNN和LSTM在手语识别中的性能比较，提出了混合模型的可行性，为实际应用中的架构选择提供了基准。

Method: 在包含50个类别的1,200个ASL手势数据集上，对比了3D CNN和LSTM的准确性、计算效率和延迟情况，并评估了混合模型的表现。

Result: 3D CNN达到92.4%的识别精度，但处理时间较长；LSTM精度为86.7%，资源消耗更低；混合模型表现中等。

Insight: 实际应用中需根据上下文选择架构，平衡精度与实时性，混合模型可能是一种折衷方案。

Abstract: This study investigates the performance of 3D Convolutional Neural Networks (3D CNNs) and Long Short-Term Memory (LSTM) networks for real-time American Sign Language (ASL) recognition. Though 3D CNNs are good at spatiotemporal feature extraction from video sequences, LSTMs are optimized for modeling temporal dependencies in sequential data. We evaluate both architectures on a dataset containing 1,200 ASL signs across 50 classes, comparing their accuracy, computational efficiency, and latency under similar training conditions. Experimental results demonstrate that 3D CNNs achieve 92.4% recognition accuracy but require 3.2% more processing time per frame compared to LSTMs, which maintain 86.7% accuracy with significantly lower resource consumption. The hybrid 3D CNNLSTM model shows decent performance, which suggests that context-dependent architecture selection is crucial for practical implementation.This project provides professional benchmarks for developing assistive technologies, highlighting trade-offs between recognition precision and real-time operational requirements in edge computing environments.

[59] Foveation Improves Payload Capacity in Steganography

Lifeng Qiu Lin,Henry Kam,Qi Sun,Kaan Akşit

Main category: cs.CV

TL;DR: 该论文提出了一种基于视觉焦点的隐写术方法，显著提升了载荷容量和准确性，同时保持了良好的视觉质量。

Details

Motivation: 现有的视觉隐写术在载荷容量和准确性方面存在限制，作者希望通过结合高效的潜在表示和视觉焦点渲染技术来突破这些限制。

Contribution: 论文的主要贡献是通过引入视觉焦点技术和多模态潜在表示，将隐写术的载荷容量从100比特提升到500比特，同时显著提高了准确性（1比特错误/2000比特）。

Method: 方法结合了高效的潜在表示技术和视觉焦点渲染（foveated rendering），设计了一种新的感知模型，用于生成多模态潜在表示。

Result: 在200K测试比特的情况下，实现了更高的准确性（1比特错误/2000比特），同时保持了良好的视觉质量（PSNR 31.47 dB，LPIPS 0.13）。

Insight: 视觉焦点技术可以显著提升隐写术的性能，多模态潜在表示有助于在保持视觉质量的同时增加载荷容量。

Abstract: Steganography finds its use in visual medium such as providing metadata and watermarking. With support of efficient latent representations and foveated rendering, we trained models that improve existing capacity limits from 100 to 500 bits, while achieving better accuracy of up to 1 failure bit out of 2000, at 200K test bits. Finally, we achieve a comparable visual quality of 31.47 dB PSNR and 0.13 LPIPS, showing the effectiveness of novel perceptual design in creating multi-modal latent representations in steganography.

[60] DP-TTA: Test-time Adaptation for Transient Electromagnetic Signal Denoising via Dictionary-driven Prior Regularization

Meng Yang,Kecheng Chen,Wei Luo,Xianjie Chen,Yong Jia,Mingyue Wang,Fanqiang Lin

Main category: cs.CV

TL;DR: 该论文提出了一种名为DP-TTA的方法，通过字典驱动的先验正则化实现在瞬时电磁信号去噪中的测试时适应（TTA），解决了预训练模型在新环境下性能下降的问题。

Details

Motivation: 瞬时电磁（TEM）信号在不同地理区域的噪声特性差异显著，导致预训练模型在新环境下表现不佳。现有方法忽略这种差异，需要一种适应性强的去噪方法。

Contribution: 提出DP-TTA方法，利用TEM信号的固有物理特性（如指数衰减和平滑性）作为先验知识，通过字典学习和自监督损失实现测试时动态适应。

Method: 1. 使用字典学习编码信号固有特性作为先验。
2. 设计DTEMDNet网络，集成先验知识并动态调整模型参数。
3. 测试时通过最小化基于字典一致性和信号一阶变化的损失实现自适应。

Result: 实验表明，DP-TTA在TEM信号去噪和新环境适应中显著优于现有方法。

Insight: 信号的固有物理特性可作为跨场景一致的先验知识，结合自监督学习能够有效提升模型的泛化能力和适应性。

Abstract: Transient Electromagnetic (TEM) method is widely used in various geophysical applications, providing valuable insights into subsurface properties. However, time-domain TEM signals are often submerged in various types of noise. While recent deep learning-based denoising models have shown strong performance, these models are mostly trained on simulated or single real-world scenario data, overlooking the significant differences in noise characteristics from different geographical regions. Intuitively, models trained in one environment often struggle to perform well in new settings due to differences in geological conditions, equipment, and external interference, leading to reduced denoising performance. To this end, we propose the Dictionary-driven Prior Regularization Test-time Adaptation (DP-TTA). Our key insight is that TEM signals possess intrinsic physical characteristics, such as exponential decay and smoothness, which remain consistent across different regions regardless of external conditions. These intrinsic characteristics serve as ideal prior knowledge for guiding the TTA strategy, which helps the pre-trained model dynamically adjust parameters by utilizing self-supervised losses, improving denoising performance in new scenarios. To implement this, we customized a network, named DTEMDNet. Specifically, we first use dictionary learning to encode these intrinsic characteristics as a dictionary-driven prior, which is integrated into the model during training. At the testing stage, this prior guides the model to adapt dynamically to new environments by minimizing self-supervised losses derived from the dictionary-driven consistency and the signal one-order variation. Extensive experimental results demonstrate that the proposed method achieves much better performance than existing TEM denoising methods and TTA methods.

[61] Prompt-based Adaptation in Large-scale Vision Models: A Survey

Xi Xiao,Yunbei Zhang,Lin Zhao,Yiyang Liu,Xiaoying Liao,Zheda Mai,Xingjian Li,Xiao Wang,Hao Xu,Jihun Hamm,Xue Lin,Min Xu,Qifan Wang,Tianyang Wang,Cheng Han

Main category: cs.CV

TL;DR: 这篇综述论文系统梳理了视觉提示（VP）和视觉提示调整（VPT）在计算机视觉中的应用，提出了基于提示的适配（PA）的统一框架，并对现有方法进行了分类和总结。

Details

Motivation: 由于VP和VPT在研究中常被混用，缺乏系统性的区分，本文旨在澄清这些技术的概念边界，并提供一个统一的视角。

Contribution: 提出了PA的统一框架，分类了现有方法（可学习、生成式和非可学习提示），并探讨了PA在多个领域的应用及未来方向。

Method: 从第一性原理出发，重新设计了VP和VPT的概念，并将其整合为PA框架。方法分类基于提示的注入粒度（像素级和标记级）。

Result: 总结了PA在不同领域的应用（如医学影像、3D点云等），并梳理了当前基准和挑战。

Insight: PA是一种轻量化的适配方法，未来可能在不同领域和可信AI中发挥更大作用。

Abstract: In computer vision, Visual Prompting (VP) and Visual Prompt Tuning (VPT) have recently emerged as lightweight and effective alternatives to full fine-tuning for adapting large-scale vision models within the ``pretrain-then-finetune’’ paradigm. However, despite rapid progress, their conceptual boundaries remain blurred, as VP and VPT are frequently used interchangeably in current research, reflecting a lack of systematic distinction between these techniques and their respective applications. In this survey, we revisit the designs of VP and VPT from first principles, and conceptualize them within a unified framework termed Prompt-based Adaptation (PA). We provide a taxonomy that categorizes existing methods into learnable, generative, and non-learnable prompts, and further organizes them by injection granularity – pixel-level and token-level. Beyond the core methodologies, we examine PA’s integrations across diverse domains, including medical imaging, 3D point clouds, and vision-language tasks, as well as its role in test-time adaptation and trustworthy AI. We also summarize current benchmarks and identify key challenges and future directions. To the best of our knowledge, we are the first comprehensive survey dedicated to PA’s methodologies and applications in light of their distinct characteristics. Our survey aims to provide a clear roadmap for researchers and practitioners in all area to understand and explore the evolving landscape of PA-related research.

[62] Sample-Centric Multi-Task Learning for Detection and Segmentation of Industrial Surface Defects

Hang-Cheng Dong,Yibo Jiao,Fupeng Wei,Guodong Liu,Dong Ye,Bingguo Liu

Main category: cs.CV

TL;DR: 论文提出了一种样本中心的多任务学习框架，用于工业表面缺陷的检测和分割，解决了传统像素中心方法在样本级别决策上的不稳定问题。

Details

Motivation: 工业表面缺陷检测中，极端的背景-前景不平衡、稀疏缺陷的长尾分布以及低对比度问题使得像素中心的训练和评估容易忽略小或低对比度缺陷，导致样本级别决策不稳定。

Contribution: 提出了一种样本中心的多任务学习框架，联合学习样本级别分类和像素级别分割；设计了去偏差的评价指标Seg_mIoU和Seg_Recall，将定位质量与样本级别决策紧密耦合。

Method: 基于共享编码器的架构，通过样本级别监督调制特征分布，分割分支保留边界细节，提升样本级别决策稳定性。

Result: 在两个基准数据集上的实验表明，该方法显著提高了样本级别决策的可靠性和缺陷定位的完整性。

Insight: 样本级别的监督可以有效引导模型关注小或低对比度缺陷，而联合学习的多任务框架能够同时优化分类和分割性能。

Abstract: Industrial surface defect inspection for sample-wise quality control (QC) must simultaneously decide whether a given sample contains defects and localize those defects spatially. In real production lines, extreme foreground-background imbalance, defect sparsity with a long-tailed scale distribution, and low contrast are common. As a result, pixel-centric training and evaluation are easily dominated by large homogeneous regions, making it difficult to drive models to attend to small or low-contrast defects-one of the main bottlenecks for deployment. Empirically, existing models achieve strong pixel-overlap metrics (e.g., mIoU) but exhibit insufficient stability at the sample level, especially for sparse or slender defects. The root cause is a mismatch between the optimization objective and the granularity of QC decisions. To address this, we propose a sample-centric multi-task learning framework and evaluation suite. Built on a shared-encoder architecture, the method jointly learns sample-level defect classification and pixel-level mask localization. Sample-level supervision modulates the feature distribution and, at the gradient level, continually boosts recall for small and low-contrast defects, while the segmentation branch preserves boundary and shape details to enhance per-sample decision stability and reduce misses. For evaluation, we propose decision-linked metrics, Seg_mIoU and Seg_Recall, which remove the bias of classical mIoU caused by empty or true-negative samples and tightly couple localization quality with sample-level decisions. Experiments on two benchmark datasets demonstrate that our approach substantially improves the reliability of sample-level decisions and the completeness of defect localization.

[63] What “Not” to Detect: Negation-Aware VLMs via Structured Reasoning and Token Merging

Inha Kang,Youngsun Lim,Seonho Lee,Jiho Choi,Junsuk Choe,Hyunjung Shim

Main category: cs.CV

TL;DR: 这篇论文提出了一种新方法和数据集，以解决视觉语言模型（VLM）在否定理解上的缺陷，显著提升了在描述对象检测任务中的性能。

Details

Motivation: 现有的视觉语言模型在理解否定语义时存在严重的‘肯定偏差’，导致其在描述对象检测任务中的表现不佳，亟需解决这一问题。

Contribution: 1) 提出了CoVAND数据集，通过链式思维（CoT）和VQA流程生成高质量的否定数据；2) 设计了NegToMe模块，通过合并否定与属性词为单一语义单元，解决了tokenization中的结构性缺陷。

Method: NegToMe模块将否定词与属性词合并为单一token，并结合LoRA微调方法，高效地修正否定语义。

Result: 在OVDEval基准上，NMS-AP提高了10.8点，显著降低了误报率，并展示了泛化能力。

Insight: 否定语义的结构化处理是解决VLM中肯定偏差的关键，而高效的合并方法与微调策略可以显著提升性能。

Abstract: State-of-the-art vision-language models (VLMs) suffer from a critical failure in understanding negation, often referred to as affirmative bias. This limitation is particularly severe in described object detection (DOD) tasks. To address this, we propose two primary contributions: (1) a new dataset pipeline and (2) a novel, lightweight adaptation recipe. First, we introduce CoVAND, a dataset constructed with a systematic chain-of-thought (CoT) and VQA-based pipeline to generate high-quality, instance-grounded negation data. Second, we propose NegToMe, a novel text token merging module that directly tackles the architectural cause of affirmative bias. NegToMe fundamentally addresses the structural loss of negation cues in tokenization, grouping them with attributes into coherent semantic phrases. It maintains correct polarity at the input level, enabling robust negation understanding even with limited data. For instance, to prevent a model from treating the fragmented tokens “not” and “girl” as simply “girl”, NegToMe binds them into a single token whose meaning is correctly distinguished from that of “girl” alone. This module is integrated with a parameter-efficient and strategic LoRA fine-tuning approach. Our method significantly improves performance on challenging negation benchmarks with a lowered false positive rate, boosting NMS-AP by up to +10.8 points on OVDEval and demonstrating generalization to SoTA VLMs. This work marks a crucial step forward in addressing negation understanding for real-world detection applications.

[64] EPIPTrack: Rethinking Prompt Modeling with Explicit and Implicit Prompts for Multi-Object Tracking

Yukuan Zhang,Jiarui Zhao,Shangqing Nie,Jin Kuang,Shengsheng Wang

Main category: cs.CV

TL;DR: EPIPTrack提出了一种统一的多模态视觉语言跟踪框架，结合显式和隐式提示动态建模目标与语义对齐，显著提升多目标跟踪性能。

Details

Motivation: 现有方法依赖静态文本描述，缺乏对实时目标状态变化的适应性且易产生幻觉，EPIPTrack旨在解决这一问题。

Contribution: 1. 显式提示将空间运动信息转为自然语言描述；2. 隐式提示结合伪词与可学习描述符；3. 设计了判别性特征增强器。

Method: 显式提示提供时空引导，隐式提示捕捉外观属性；两者通过CLIP文本编码器动态调整；特征增强器提升跨模态表示。

Result: 在MOT17、MOT20和DanceTrack数据集上表现优于现有跟踪器，展现了强适应性和卓越性能。

Insight: 动态提示建模和跨模态对齐是多目标跟踪的关键创新点。

Abstract: Multimodal semantic cues, such as textual descriptions, have shown strong potential in enhancing target perception for tracking. However, existing methods rely on static textual descriptions from large language models, which lack adaptability to real-time target state changes and prone to hallucinations. To address these challenges, we propose a unified multimodal vision-language tracking framework, named EPIPTrack, which leverages explicit and implicit prompts for dynamic target modeling and semantic alignment. Specifically, explicit prompts transform spatial motion information into natural language descriptions to provide spatiotemporal guidance. Implicit prompts combine pseudo-words with learnable descriptors to construct individualized knowledge representations capturing appearance attributes. Both prompts undergo dynamic adjustment via the CLIP text encoder to respond to changes in target state. Furthermore, we design a Discriminative Feature Augmentor to enhance visual and cross-modal representations. Extensive experiments on MOT17, MOT20, and DanceTrack demonstrate that EPIPTrack outperforms existing trackers in diverse scenarios, exhibiting robust adaptability and superior performance.

[65] Model-agnostic Adversarial Attack and Defense for Vision-Language-Action Models

Haochuan Xu,Yun Sing Koh,Shuhuai Huang,Zirun Zhou,Di Wang,Jun Sakuma,Jingfeng Zhang

Main category: cs.CV

TL;DR: 该论文探讨视觉-语言-动作（VLA）模型的对抗鲁棒性不足问题，提出了一种模型无关的对抗攻击方法（EDPA）及防御策略，成功提升了任务失败率并通过对抗微调有效缓解攻击效果。

Details

Motivation: VLA模型在机器人学习中取得了显著进展，但其对抗鲁棒性鲜有研究。论文旨在填补这一空白，提出一种无需模型先验知识的通用对抗攻击方法及防御策略。

Contribution: 1. 提出嵌入破坏补丁攻击（EDPA），一种模型无关的对抗攻击方法；
2. 引入对抗微调防御策略，提升VLA模型的对抗鲁棒性；
3. 在LIBERO基准测试中验证了攻击的有效性和防御的可行性。

Method: 1. EDPA通过破坏视觉与文本潜在表征的语义对齐，并最大化对抗与干净输入的潜在表征差异；
2. 防御策略采用对抗微调，优化视觉编码器，使其对干净和对抗输入生成相似的潜在表征。

Result: 在LIBERO基准测试中，EDPA显著增加了VLA模型的任务失败率，而防御策略成功缓解了这种性能下降。

Insight: 1. VLA模型的对抗脆弱性源于潜在表征的语义对齐被破坏；
2. 对抗微调是一种有效的防御手段，可提升模型对对抗攻击的鲁棒性。

Abstract: Vision-Language-Action (VLA) models have achieved revolutionary progress in robot learning, enabling robots to execute complex physical robot tasks from natural language instructions. Despite this progress, their adversarial robustness remains underexplored. In this work, we propose both adversarial patch attack and corresponding defense strategies for VLA models. We first introduce the Embedding Disruption Patch Attack (EDPA), a model-agnostic adversarial attack that generates patches directly placeable within the camera’s view. In comparison to prior methods, EDPA can be readily applied to different VLA models without requiring prior knowledge of the model architecture, or the controlled robotic manipulator. EDPA constructs these patches by (i) disrupting the semantic alignment between visual and textual latent representations, and (ii) maximizing the discrepancy of latent representations between adversarial and corresponding clean visual inputs. Through the optimization of these objectives, EDPA distorts the VLA’s interpretation of visual information, causing the model to repeatedly generate incorrect actions and ultimately result in failure to complete the given robotic task. To counter this, we propose an adversarial fine-tuning scheme for the visual encoder, in which the encoder is optimized to produce similar latent representations for both clean and adversarially perturbed visual inputs. Extensive evaluations on the widely recognized LIBERO robotic simulation benchmark demonstrate that EDPA substantially increases the task failure rate of cutting-edge VLA models, while our proposed defense effectively mitigates this degradation. The codebase is accessible via the homepage at https://edpa-attack.github.io/.

[66] FlyAwareV2: A Multimodal Cross-Domain UAV Dataset for Urban Scene Understanding

Francesco Barbato,Matteo Caligiuri,Pietro Zanuttigh

Main category: cs.CV

TL;DR: FlyAwareV2是一个多模态跨域无人机数据集，专为城市场景理解设计，提供了真实和合成数据，支持RGB、深度和语义标注，并包含多样环境条件和域适应研究。

Details

Motivation: 由于真实无人机数据采集和标注的高成本与挑战，需要一种兼具真实性和多样性的数据集来推动计算机视觉算法的发展。

Contribution: 1) 提供多模态数据（RGB、深度、语义标注）涵盖多样环境条件；2) 为真实样本提供先进单目深度估计生成的深度图；3) 提供RGB和多模态语义分割的基准测试；4) 研究合成到真实的域适应能力。

Method: 通过结合真实和合成数据，利用单目深度估计生成深度图，并设计基准测试任务评估模型性能。

Result: FlyAwareV2成为无人机3D城市场景理解研究的宝贵资源。

Insight: 多模态数据和多样化环境条件对无人机应用的视觉任务至关重要，合成数据在域适应研究中显示出潜力。

Abstract: The development of computer vision algorithms for Unmanned Aerial Vehicle (UAV) applications in urban environments heavily relies on the availability of large-scale datasets with accurate annotations. However, collecting and annotating real-world UAV data is extremely challenging and costly. To address this limitation, we present FlyAwareV2, a novel multimodal dataset encompassing both real and synthetic UAV imagery tailored for urban scene understanding tasks. Building upon the recently introduced SynDrone and FlyAware datasets, FlyAwareV2 introduces several new key contributions: 1) Multimodal data (RGB, depth, semantic labels) across diverse environmental conditions including varying weather and daytime; 2) Depth maps for real samples computed via state-of-the-art monocular depth estimation; 3) Benchmarks for RGB and multimodal semantic segmentation on standard architectures; 4) Studies on synthetic-to-real domain adaptation to assess the generalization capabilities of models trained on the synthetic data. With its rich set of annotations and environmental diversity, FlyAwareV2 provides a valuable resource for research on UAV-based 3D urban scene understanding.

[67] CymbaDiff: Structured Spatial Diffusion for Sketch-based 3D Semantic Urban Scene Generation

Li Liang,Bo Miao,Xinyu Wang,Naveed Akhtar,Jordan Vice,Ajmal Mian

Main category: cs.CV

TL;DR: SketchSem3D是一个基于抽象手绘草图和卫星图像伪标注的首个大规模户外3D语义场景生成基准，包含两个子集，并提出了CymbaDiff方法以增强空间一致性。

Details

Motivation: 当前户外3D语义场景生成缺乏公开且标注良好的数据集，限制了该领域的发展。

Contribution: 提出了SketchSem3D基准和CymbaDiff方法，后者通过结构化空间排序和圆柱连续性显著提升了生成场景的空间一致性。

Method: CymbaDiff方法显式捕捉圆柱连续性和垂直层次结构，保留物理邻域关系和全局上下文。

Result: 在SketchSem3D上的实验表明，CymbaDiff在语义一致性、空间真实性和跨数据集泛化方面表现优异。

Insight: 通过结构化空间设计和圆柱连续性，可以显著提升户外3D场景生成的质量。

Abstract: Outdoor 3D semantic scene generation produces realistic and semantically rich environments for applications such as urban simulation and autonomous driving. However, advances in this direction are constrained by the absence of publicly available, well-annotated datasets. We introduce SketchSem3D, the first large-scale benchmark for generating 3D outdoor semantic scenes from abstract freehand sketches and pseudo-labeled annotations of satellite images. SketchSem3D includes two subsets, Sketch-based SemanticKITTI and Sketch-based KITTI-360 (containing LiDAR voxels along with their corresponding sketches and annotated satellite images), to enable standardized, rigorous, and diverse evaluations. We also propose Cylinder Mamba Diffusion (CymbaDiff) that significantly enhances spatial coherence in outdoor 3D scene generation. CymbaDiff imposes structured spatial ordering, explicitly captures cylindrical continuity and vertical hierarchy, and preserves both physical neighborhood relationships and global context within the generated scenes. Extensive experiments on SketchSem3D demonstrate that CymbaDiff achieves superior semantic consistency, spatial realism, and cross-dataset generalization. The code and dataset will be available at https://github.com/Lillian-research-hub/CymbaDiff

[68] Real-Time Crowd Counting for Embedded Systems with Lightweight Architecture

Zhiyuan Zhao,Yubin Wen,Siyu Yang,Lichen Ning,Yuandong Liu,Junyu Gao

Main category: cs.CV

TL;DR: 该论文提出了一种轻量级架构的实时人群计数模型，专为嵌入式系统设计，具有超快的推理速度和竞争性的准确率。

Details

Motivation: 现有的人群计数方法在嵌入式系统上应用时存在模型参数过多、计算复杂等问题，无法满足实时性需求。因此，需要设计一种适用于嵌入式系统的实时高效模型。

Contribution: 设计了一种基于stem-encoder-decoder结构的超实时人群计数模型，通过大卷积核、条件通道加权和多分支局部融合块等技术，实现了模型的高效推理速度和准确率。

Method: 1) 使用大卷积核的stem网络扩大感受野；2) 在编码器部分引入条件通道加权和多分支局部融合块以合并多尺度特征；3) 在编码器顶部添加特征金字塔网络以缓解不完全融合问题。

Result: 在三个基准测试中，模型在NVIDIA GTX 1080Ti上达到381.7 FPS，在NVIDIA Jetson TX1上达到71.9 FPS，推理速度最快且保持了竞争性的准确率。

Insight: 通过优化模型架构和计算效率，可以在不牺牲准确率的情况下显著提升嵌入式系统上的实时人群计数性能。

Abstract: Crowd counting is a task of estimating the number of the crowd through images, which is extremely valuable in the fields of intelligent security, urban planning, public safety management, and so on. However, the existing counting methods have some problems in practical application on embedded systems for these fields, such as excessive model parameters, abundant complex calculations, etc. The practical application of embedded systems requires the model to be real-time, which means that the model is fast enough. Considering the aforementioned problems, we design a super real-time model with a stem-encoder-decoder structure for crowd counting tasks, which achieves the fastest inference compared with state-of-the-arts. Firstly, large convolution kernels in the stem network are used to enlarge the receptive field, which effectively extracts detailed head information. Then, in the encoder part, we use conditional channel weighting and multi-branch local fusion block to merge multi-scale features with low computational consumption. This part is crucial to the super real-time performance of the model. Finally, the feature pyramid networks are added to the top of the encoder to alleviate its incomplete fusion problems. Experiments on three benchmarks show that our network is suitable for super real-time crowd counting on embedded systems, ensuring competitive accuracy. At the same time, the proposed network reasoning speed is the fastest. Specifically, the proposed network achieves 381.7 FPS on NVIDIA GTX 1080Ti and 71.9 FPS on NVIDIA Jetson TX1.

[69] Map the Flow: Revealing Hidden Pathways of Information in VideoLLMs

Minji Kim,Taekyung Kim,Bohyung Han

Main category: cs.CV

TL;DR: 论文分析了VideoLLMs的内部信息流，揭示了其在视频问答任务中的时间推理机制，并提出了优化模型的实用见解。

Details

Motivation: 研究VideoLLMs内部信息提取与传播的机制，以提升模型的可解释性和下游泛化能力。

Contribution: 揭示了VideoLLMs在时间推理中的信息流模式；提出通过选择性信息路径保留性能的方法。

Method: 使用机制解释技术分析VideoLLMs的内部信息流，识别关键交互层和整合阶段。

Result: 发现早期到中间层的跨帧交互是关键；通过抑制58%的注意力边仍能保持性能。

Insight: VideoLLMs的时间推理依赖于视频表征与语言嵌入的对齐，选择性路径可提高效率。

Abstract: Video Large Language Models (VideoLLMs) extend the capabilities of vision-language models to spatiotemporal inputs, enabling tasks such as video question answering (VideoQA). Despite recent advances in VideoLLMs, their internal mechanisms on where and how they extract and propagate video and textual information remain less explored. In this study, we investigate the internal information flow of VideoLLMs using mechanistic interpretability techniques. Our analysis reveals consistent patterns across diverse VideoQA tasks: (1) temporal reasoning in VideoLLMs initiates with active cross-frame interactions in early-to-middle layers, (2) followed by progressive video-language integration in middle layers. This is facilitated by alignment between video representations and linguistic embeddings containing temporal concepts. (3) Upon completion of this integration, the model is ready to generate correct answers in middle-to-late layers. (4) Based on our analysis, we show that VideoLLMs can retain their VideoQA performance by selecting these effective information pathways while suppressing a substantial amount of attention edges, e.g., 58% in LLaVA-NeXT-7B-Video-FT. These findings provide a blueprint on how VideoLLMs perform temporal reasoning and offer practical insights for improving model interpretability and downstream generalization. Our project page with the source code is available at https://map-the-flow.github.io

[70] End-to-End Multi-Modal Diffusion Mamba

Chunhao Lu,Qiang Lu,Meichen Dong,Jake Luo

Main category: cs.CV

TL;DR: MDM（多模态扩散Mamba）通过统一的变分自编码器和Mamba多步选择扩散模型，实现多模态信息的端到端联合学习，在高维数据生成和多模态任务中表现优异。

Details

Motivation: 现有端到端多模态模型依赖独立的编码器和解码器，阻碍了多模态联合表示学习。MDM旨在通过统一架构解决这一问题。

Contribution: 1. 提出MDM架构，使用Mamba扩散模型和统一变分自编码器；2. 支持高效的多模态生成与推理；3. 在图像生成、文本理解等任务中表现优于现有模型。

Method: MDM基于Mamba的多步选择扩散模型，逐步生成和细化模态特征，并通过统一变分自编码器实现编码解码。

Result: MDM在图像生成、图像描述、视觉问答等任务中显著优于MonoFormer等模型，并与GPT-4V等SOTA模型竞争。

Insight: 1. 统一架构提升多模态联合学习效率；2. Mamba扩散模型在高维数据生成中表现优异；3. MDM为多模态端到端模型提供了新方向。

Abstract: Current end-to-end multi-modal models utilize different encoders and decoders to process input and output information. This separation hinders the joint representation learning of various modalities. To unify multi-modal processing, we propose a novel architecture called MDM (Multi-modal Diffusion Mamba). MDM utilizes a Mamba-based multi-step selection diffusion model to progressively generate and refine modality-specific information through a unified variational autoencoder for both encoding and decoding. This innovative approach allows MDM to achieve superior performance when processing high-dimensional data, particularly in generating high-resolution images and extended text sequences simultaneously. Our evaluations in areas such as image generation, image captioning, visual question answering, text comprehension, and reasoning tasks demonstrate that MDM significantly outperforms existing end-to-end models (MonoFormer, LlamaGen, and Chameleon etc.) and competes effectively with SOTA models like GPT-4V, Gemini Pro, and Mistral. Our results validate MDM’s effectiveness in unifying multi-modal processes while maintaining computational efficiency, establishing a new direction for end-to-end multi-modal architectures.

[71] MMLongCite: A Benchmark for Evaluating Fidelity of Long-Context Vision-Language Models

Keyan Zhou,Zecheng Tang,Lingfeng Ming,Guanghao Zhou,Qiguang Chen,Dan Qiao,Zheming Yang,Libo Qin,Minghui Qiu,Juntao Li,Min Zhang

Main category: cs.CV

TL;DR: MMLongCite is a benchmark introduced to evaluate the fidelity of large vision-language models (LVLMs) in long-context scenarios, revealing their limitations despite expanded context windows.

Details

Motivation: The rapid development of LVLMs has expanded their context windows, but effective utilization of long multimodal contexts remains unverified, necessitating a dedicated benchmark.

Contribution: MMLongCite is the first comprehensive benchmark evaluating LVLM fidelity in long-context scenarios, spanning diverse modalities (text, images, videos) and context lengths.

Method: The benchmark includes 8 tasks across 6 context length intervals, assessing LVLMs’ ability to faithfully utilize long multimodal contexts.

Result: Evaluation shows current LVLMs struggle with faithfulness in long multimodal contexts, influenced by context length and content positioning.

Insight: Expanding context windows alone does not ensure effective long-context utilization; multimodal benchmarks are essential for real-world applicability.

Abstract: The rapid advancement of large vision language models (LVLMs) has led to a significant expansion of their context windows. However, an extended context window does not guarantee the effective utilization of the context, posing a critical challenge for real-world applications. Current evaluations of such long-context faithfulness are predominantly focused on the text-only domain, while multimodal assessments remain limited to short contexts. To bridge this gap, we introduce MMLongCite, a comprehensive benchmark designed to evaluate the fidelity of LVLMs in long-context scenarios. MMLongCite comprises 8 distinct tasks spanning 6 context length intervals and incorporates diverse modalities, including text, images, and videos. Our evaluation of state-of-the-art LVLMs reveals their limited faithfulness in handling long multimodal contexts. Furthermore, we provide an in-depth analysis of how context length and the position of crucial content affect the faithfulness of these models.

[72] Universal Image Restoration Pre-training via Masked Degradation Classification

JiaKui Hu,Zhengjian Yao,Lujia Jin,Yinghao Chen,Yanye Lu

Main category: cs.CV

TL;DR: 该论文提出了一种名为MaskDCPT的预训练方法，通过掩码降解分类实现通用图像修复任务的高效预训练。其核心是通过分类降解类型和图像重建的双任务设计，提升了模型的鲁棒性和泛化能力。

Details

Motivation: 现有图像修复预训练方法通常依赖单一任务或强监督信号，难以处理多样化的降解类型和未知场景。MaskDCPT旨在通过弱监督（降解类型分类）与重建任务的结合，实现更通用的图像修复能力。

Contribution: 1. 提出了MaskDCPT预训练方法，结合掩码图像建模和对比学习；2. 设计了双解码器结构（分类+重建）；3. 发布了UIR-2.5M数据集，包含19种降解类型和200多种降解级别的250万对样本。

Method: 1. 使用掩码低质量图像提取特征（编码器）；2. 分类解码器预测降解类型；3. 重建解码器生成高质量图像；4. 双任务联合优化。

Result: 1. PSNR提升至少3.77 dB；2. PIQE降低34.8%；3. 对未知降解类型和级别表现出强泛化能力。

Insight: 通过弱监督信号（降解分类）与重建任务的结合，可以在预训练阶段捕获更通用的图像表征，从而显著提升修复任务的性能。

Abstract: This study introduces a Masked Degradation Classification Pre-Training method (MaskDCPT), designed to facilitate the classification of degradation types in input images, leading to comprehensive image restoration pre-training. Unlike conventional pre-training methods, MaskDCPT uses the degradation type of the image as an extremely weak supervision, while simultaneously leveraging the image reconstruction to enhance performance and robustness. MaskDCPT includes an encoder and two decoders: the encoder extracts features from the masked low-quality input image. The classification decoder uses these features to identify the degradation type, whereas the reconstruction decoder aims to reconstruct a corresponding high-quality image. This design allows the pre-training to benefit from both masked image modeling and contrastive learning, resulting in a generalized representation suited for restoration tasks. Benefit from the straightforward yet potent MaskDCPT, the pre-trained encoder can be used to address universal image restoration and achieve outstanding performance. Implementing MaskDCPT significantly improves performance for both convolution neural networks (CNNs) and Transformers, with a minimum increase in PSNR of 3.77 dB in the 5D all-in-one restoration task and a 34.8% reduction in PIQE compared to baseline in real-world degradation scenarios. It also emergences strong generalization to previously unseen degradation types and levels. In addition, we curate and release the UIR-2.5M dataset, which includes 2.5 million paired restoration samples across 19 degradation types and over 200 degradation levels, incorporating both synthetic and real-world data. The dataset, source code, and models are available at https://github.com/MILab-PKU/MaskDCPT.

[73] Automated document processing system for government agencies using DBNET++ and BART models

Aya Kaysan Bahjat

Main category: cs.CV

TL;DR: 该论文提出了一种基于DBNET++和BART模型的自动化文档处理系统，用于政府机构的文档分类，能够在复杂场景下高效检测文本并进行分类。

Details

Motivation: 政府机构需要处理大量不同类型的文档，传统手动分类方式效率低且容易出错。因此，设计一个自动化系统，能够处理复杂条件下的文档（如光照变化、遮挡等）并进行分类，具有重要实用价值。

Contribution: 1. 提出了一种结合DBNET++（文本检测）和BART（文本分类）的端到端文档处理系统。2. 系统支持离线和实时图像输入，并在复杂场景下表现良好。

Method: 1. 图像采集与预处理；2. 使用DBNET++检测文本；3. 使用BART分类文本；4. 通过PyQt5实现用户界面。

Result: 在Total-Text数据集上，文本检测准确率达到92.88%，表明系统在复杂场景下的有效性。

Insight: 结合深度学习模型（DBNET++和BART）的系统能够有效解决实际文档处理中的多样化挑战，未来可扩展至更多类别或语言。

Abstract: An automatic document classification system is presented that detects textual content in images and classifies documents into four predefined categories (Invoice, Report, Letter, and Form). The system supports both offline images (e.g., files on flash drives, HDDs, microSD) and real-time capture via connected cameras, and is designed to mitigate practical challenges such as variable illumination, arbitrary orientation, curved or partially occluded text, low resolution, and distant text. The pipeline comprises four stages: image capture and preprocessing, text detection [1] using a DBNet++ (Differentiable Binarization Network Plus) detector, and text classification [2] using a BART (Bidirectional and Auto-Regressive Transformers) classifier, all integrated within a user interface implemented in Python with PyQt5. The achieved results by the system for text detection in images were good at about 92.88% through 10 hours on Total-Text dataset that involve high resolution images simulate a various and very difficult challenges. The results indicate the proposed approach is effective for practical, mixed-source document categorization in unconstrained imaging scenarios.

[74] Novel Class Discovery for Point Cloud Segmentation via Joint Learning of Causal Representation and Reasoning

Yang Li,Aming Wu,Zihao Zhang,Yahong Han

Main category: cs.CV

TL;DR: 该论文提出了一种通过联合学习因果表示和推理的新方法，用于点云分割中的新类发现（3D-NCD），旨在利用基类的标注信息分割未标注（新）的3D类别。

Details

Motivation: 现有的方法在学习基类和新类之间的相关性时可能过于粗略或统计化，导致新类推理混淆。作者引入因果关系的强约束以避免这一问题。

Contribution: 1. 利用结构化因果模型（SCM）重新形式化3D-NCD问题；2. 提出联合学习因果表示与推理的方法；3. 设计消除混杂因子的因果表示原型。

Method: 1. 通过SCM分析基类表示中的隐藏混杂因子及基类与新类的因果关系；2. 设计因果表示原型消除混杂因子；3. 用图结构建模基类与新类的因果推理关系。

Result: 在3D和2D NCD语义分割任务上的实验和可视化结果显示，该方法具有优越性。

Insight: 因果关系的引入能够更准确地捕捉点云表示与类别之间的本质联系，避免混淆。

Abstract: In this paper, we focus on Novel Class Discovery for Point Cloud Segmentation (3D-NCD), aiming to learn a model that can segment unlabeled (novel) 3D classes using only the supervision from labeled (base) 3D classes. The key to this task is to setup the exact correlations between the point representations and their base class labels, as well as the representation correlations between the points from base and novel classes. A coarse or statistical correlation learning may lead to the confusion in novel class inference. lf we impose a causal relationship as a strong correlated constraint upon the learning process, the essential point cloud representations that accurately correspond to the classes should be uncovered. To this end, we introduce a structural causal model (SCM) to re-formalize the 3D-NCD problem and propose a new method, i.e., Joint Learning of Causal Representation and Reasoning. Specifically, we first analyze hidden confounders in the base class representations and the causal relationships between the base and novel classes through SCM. We devise a causal representation prototype that eliminates confounders to capture the causal representations of base classes. A graph structure is then used to model the causal relationships between the base classes’ causal representation prototypes and the novel class prototypes, enabling causal reasoning from base to novel classes. Extensive experiments and visualization results on 3D and 2D NCD semantic segmentation demonstrate the superiorities of our method.

[75] Self-Augmented Visual Contrastive Decoding

Eun Woo Im,Muhammad Kashif Ali,Vivek Gupta

Main category: cs.CV

TL;DR: 本文提出了一种无需训练的解码策略,通过自增强提示和自适应阈值算法,显著提升了大型视觉语言模型的事实一致性。

Details

Motivation: 尽管大型视觉语言模型在多模态任务中表现出色,但其继承了语言模型的幻觉问题。现有的视觉对比解码方法忽视文本查询的上下文,限制了有效性。

Contribution: 1. 自增强提示策略,动态对齐查询与视觉增强的语义; 2. 自适应阈值算法,基于输出稀疏性调整候选标记大小。

Method: 提出训练自由的解码策略,结合自增强提示和自适应阈值算法,利用模型的内部知识和logit分布的全部信息。

Result: 在四种大型视觉语言模型和七个基准测试上的实验表明,该方法显著优于现有解码方法。

Insight: 查询相关的增强和熵感知解码对于提升大型视觉语言模型的生成效果至关重要。

Abstract: Large Vision-Language Models (LVLMs) have demonstrated remarkable multimodal capabilities, but they inherit the tendency to hallucinate from their underlying language models. While visual contrastive decoding has been proposed to mitigate this issue, existing methods often apply generic visual augmentations that disregard the specific context provided by the text query, limiting their effectiveness. This study introduces a novel training-free decoding strategy that addresses these limitations, featuring two key contributions. First, a self-augmentation prompting strategy that leverages the intrinsic knowledge of the model to dynamically align semantics between the query and the visual augmentation. Second, an adaptive thresholding algorithm that adaptively adjusts next token candidate size based on the output sparsity, utilizing full information from the logit distribution. Extensive experiments across four LVLMs and seven benchmarks demonstrate that the proposed decoding significantly enhances factual consistency compared to state-of-the-art decoding methods. This work highlights the importance of integrating query-dependent augmentation and entropy-aware decoding for improving effective generation of LVLMs.

[76] Visual Interestingness Decoded: How GPT-4o Mirrors Human Interests

Fitim Abdullahu,Helmut Grabner

Main category: cs.CV

TL;DR: 探索GPT-4o等大型多模态模型（LMMs）对人类视觉兴趣的捕捉能力，并通过比较分析验证其与人类评估的一致性。研究显示部分一致性，并提出一种基于学习排序的知识蒸馏方法。

Details

Motivation: 视觉兴趣对人类行为有重要影响，但如何量化兴趣仍是一个开放问题。利用大型多模态模型的潜力，研究其是否能捕捉并预测人类的视觉兴趣。

Contribution: 1. 验证GPT-4o在视觉兴趣评估上与人类的部分一致性；2. 提出一种利用LMM生成训练数据并蒸馏知识的学习排序模型。

Method: 通过比较分析评估GPT-4o与人类兴趣的一致性，并利用其生成的标签训练学习排序模型（learning-to-rank）。

Result: GPT-4o在视觉兴趣评估上表现优于现有方法，但与人类评估仅部分一致。蒸馏后的排序模型能有效捕捉兴趣差异。

Insight: 大型多模态模型能部分捕捉人类兴趣，但仍存在差距。未来研究可进一步优化模型对齐或结合其他信号提升性能。

Abstract: Our daily life is highly influenced by what we consume and see. Attracting and holding one’s attention – the definition of (visual) interestingness – is essential. The rise of Large Multimodal Models (LMMs) trained on large-scale visual and textual data has demonstrated impressive capabilities. We explore these models’ potential to understand to what extent the concepts of visual interestingness are captured and examine the alignment between human assessments and GPT-4o’s, a leading LMM, predictions through comparative analysis. Our studies reveal partial alignment between humans and GPT-4o. It already captures the concept as best compared to state-of-the-art methods. Hence, this allows for the effective labeling of image pairs according to their (commonly) interestingness, which are used as training data to distill the knowledge into a learning-to-rank model. The insights pave the way for a deeper understanding of human interest.

[77] Removing Cost Volumes from Optical Flow Estimators

Simon Kiefhaber,Stefan Roth,Simone Schaub-Meyer

Main category: cs.CV

TL;DR: 这篇论文提出了一种训练策略，可以在训练过程中移除光流估计器中的代价体积（cost volume），从而提高推理速度并减少内存需求。

Details

Motivation: 现代光流估计器普遍使用代价体积，但其计算和空间复杂度限制了处理速度和输入帧的分辨率。作者通过实验发现，代价体积在其他网络部分足够训练后会失去重要性。

Contribution: 主要贡献是提出了一种训练策略，允许在训练过程中移除代价体积，从而显著提升速度和降低内存需求。

Method: 通过训练策略逐步移除代价体积，最终实现了三种不同计算预算的模型。

Result: 最准确的模型达到了SOTA精度，速度快1.2倍且内存占用低6倍；最快的模型能以20FPS处理Full HD帧，仅需500MB GPU内存。

Insight: 代价体积在训练后期并非必需，移除它可以显著优化性能。

Abstract: Cost volumes are used in every modern optical flow estimator, but due to their computational and space complexity, they are often a limiting factor regarding both processing speed and the resolution of input frames. Motivated by our empirical observation that cost volumes lose their importance once all other network parts of, e.g., a RAFT-based pipeline have been sufficiently trained, we introduce a training strategy that allows removing the cost volume from optical flow estimators throughout training. This leads to significantly improved inference speed and reduced memory requirements. Using our training strategy, we create three different models covering different compute budgets. Our most accurate model reaches state-of-the-art accuracy while being $1.2\times$ faster and having a $6\times$ lower memory footprint than comparable models; our fastest model is capable of processing Full HD frames at $20,\mathrm{FPS}$ using only $500,\mathrm{MB}$ of GPU memory.

[78] DEF-YOLO: Leveraging YOLO for Concealed Weapon Detection in Thermal Imagin

Divya Bhardwaj,Arnav Ramamoorthy,Poonam Goyal

Main category: cs.CV

TL;DR: 该论文提出了DEF-YOLO，一种基于YOLOv8改进的架构，用于热成像中的隐藏武器检测，同时贡献了一个新的大规模数据集TICW。

Details

Motivation: 为解决隐藏武器检测任务中多种成像模态的局限性（如分辨率低、隐私问题），选择了热成像并提出了一种实时、低成本的解决方案。

Contribution: 1. 提出了DEF-YOLO架构，通过可变形卷积和多尺度特征提取改进YOLOv8；2. 贡献了首个大规模热成像隐藏武器数据集TICW；3. 引入焦点损失以解决类别不平衡问题。

Method: 1. 在SPPF层使用可变形卷积提取多尺度特征；2. 通过主干和颈部层提取低、中、高层特征；3. 采用焦点损失优化分类。

Result: 实验表明DEF-YOLO在热成像隐藏武器检测任务中表现优异，成为新的基准。

Insight: 1. 热成像在隐藏武器检测中具有潜力；2. 可变形卷积和多尺度特征提取是关键改进点；3. 数据集和损失函数的创新解决了任务中的核心挑战。

Abstract: Concealed weapon detection aims at detecting weapons hidden beneath a person’s clothing or luggage. Various imaging modalities like Millimeter Wave, Microwave, Terahertz, Infrared, etc., are exploited for the concealed weapon detection task. These imaging modalities have their own limitations, such as poor resolution in microwave imaging, privacy concerns in millimeter wave imaging, etc. To provide a real-time, 24 x 7 surveillance, low-cost, and privacy-preserved solution, we opted for thermal imaging in spite of the lack of availability of a benchmark dataset. We propose a novel approach and a dataset for concealed weapon detection in thermal imagery. Our YOLO-based architecture, DEF-YOLO, is built with key enhancements in YOLOv8 tailored to the unique challenges of concealed weapon detection in thermal vision. We adopt deformable convolutions at the SPPF layer to exploit multi-scale features; backbone and neck layers to extract low, mid, and high-level features, enabling DEF-YOLO to adaptively focus on localization around the objects in thermal homogeneous regions, without sacrificing much of the speed and throughput. In addition to these simple yet effective key architectural changes, we introduce a new, large-scale Thermal Imaging Concealed Weapon dataset, TICW, featuring a diverse set of concealed weapons and capturing a wide range of scenarios. To the best of our knowledge, this is the first large-scale contributed dataset for this task. We also incorporate focal loss to address the significant class imbalance inherent in the concealed weapon detection task. The efficacy of the proposed work establishes a new benchmark through extensive experimentation for concealed weapon detection in thermal imagery.

[79] Group-Wise Optimization for Self-Extensible Codebooks in Vector Quantized Models

Hong-Kai Zheng,Piji Li

Main category: cs.CV

TL;DR: Group-VQ提出了一种分组优化的方法，用于改进VQ-VAE中的码本学习，解决了码本崩塌问题，提升了重建性能，并提出了一种无需训练的码本重采样方法。

Details

Motivation: VQ-VAE中的码本崩塌问题和现有方法对码本学习能力的限制，促使研究者提出一种更灵活的码本优化方法。

Contribution: 1. 提出了分组优化的Group-VQ方法；2. 引入了无需训练的码本重采样方法，支持码本大小的灵活调整。

Method: Group-VQ将码本分组，每组独立优化，组内联合优化；并提出了码本重采样技术。

Result: 在图像重建任务中，Group-VQ提升了重建性能，且重采样方法实现了码本大小的灵活调整。

Insight: 分组优化平衡了码本利用率和重建性能，无需训练的码本重采样为实际应用提供了灵活性。

Abstract: Vector Quantized Variational Autoencoders (VQ-VAEs) leverage self-supervised learning through reconstruction tasks to represent continuous vectors using the closest vectors in a codebook. However, issues such as codebook collapse persist in the VQ model. To address these issues, existing approaches employ implicit static codebooks or jointly optimize the entire codebook, but these methods constrain the codebook’s learning capability, leading to reduced reconstruction quality. In this paper, we propose Group-VQ, which performs group-wise optimization on the codebook. Each group is optimized independently, with joint optimization performed within groups. This approach improves the trade-off between codebook utilization and reconstruction performance. Additionally, we introduce a training-free codebook resampling method, allowing post-training adjustment of the codebook size. In image reconstruction experiments under various settings, Group-VQ demonstrates improved performance on reconstruction metrics. And the post-training codebook sampling method achieves the desired flexibility in adjusting the codebook size.

[80] No-Reference Rendered Video Quality Assessment: Dataset and Metrics

Sipeng Yang,Jiayu Ji,Qingchuan Zhu,Zhiyao Yang,Xiaogang Jin

Main category: cs.CV

TL;DR: 該論文提出了針對渲染視頻的無參考視頻質量評估（NR-VQA）數據集和指標，填補了現有NR-VQA方法對渲染視頻評估不足的空白。

Details

Motivation: 渲染視頻（如視頻遊戲、虛擬現實等）易產生時序偽影，現有NR-VQA方法主要針對相機拍攝視頻，直接應用於渲染視頻會導致偏差。

Contribution: 1. 提出了一個大規模渲染視頻數據集，包含多種3D場景和渲染設置，並標註主觀質量分數；2. 設計了一個專注於渲染視頻質量的NR-VQA指標，考慮圖像質量和時序穩定性。

Method: 基於提出的數據集，標定新的NR-VQA指標，專注於評估圖像質量和時序穩定性，並與現有NR-VQA方法進行比較。

Result: 新指標在渲染視頻上表現優於現有NR-VQA方法，並能有效評估超採樣方法和實時渲染的幀生成策略。

Insight: 渲染視頻的質量評估需專門設計指標，同時考慮靜態質量和時序特性，以更準確反映實際應用場景中的用戶體驗。

Abstract: Quality assessment of videos is crucial for many computer graphics applications, including video games, virtual reality, and augmented reality, where visual performance has a significant impact on user experience. When test videos cannot be perfectly aligned with references or when references are unavailable, the significance of no-reference video quality assessment (NR-VQA) methods is undeniable. However, existing NR-VQA datasets and metrics are primarily focused on camera-captured videos; applying them directly to rendered videos would result in biased predictions, as rendered videos are more prone to temporal artifacts. To address this, we present a large rendering-oriented video dataset with subjective quality annotations, as well as a designed NR-VQA metric specific to rendered videos. The proposed dataset includes a wide range of 3D scenes and rendering settings, with quality scores annotated for various display types to better reflect real-world application scenarios. Building on this dataset, we calibrate our NR-VQA metric to assess rendered video quality by looking at both image quality and temporal stability. We compare our metric to existing NR-VQA metrics, demonstrating its superior performance on rendered videos. Finally, we demonstrate that our metric can be used to benchmark supersampling methods and assess frame generation strategies in real-time rendering.

[81] Language as a Label: Zero-Shot Multimodal Classification of Everyday Postures under Data Scarcity

MingZe Tang,Jubal Chandy Jacob

Main category: cs.CV

TL;DR: 论文研究了提示词设计对视觉语言模型（VLM）在零样本分类任务中的影响，发现基础提示词的效果优于详细提示词，揭示了‘提示过拟合’现象。

Details

Motivation: 探讨在数据稀缺条件下，如何通过优化提示词设计提升VLM对相似视觉类别（如人体姿态）的分类性能。

Contribution: 揭示了‘提示过拟合’现象，并证明基础提示词在高性能模型中表现最佳，而低性能模型则能从详细提示词中受益。

Method: 使用三层次提示设计（基础、中等、详细），在COCO衍生的小数据集上评估了OpenCLIP、MetaCLIP 2和SigLip等VLM的性能。

Result: MetaCLIP 2和OpenCLIP在基础提示词下的多类分类准确率最高（68.8%），详细提示词反而降低性能。

Insight: 提示词的复杂性需与模型能力匹配，高性能模型可能因‘提示过拟合’而对详细提示词敏感。

Abstract: Recent Vision-Language Models (VLMs) enable zero-shot classification by aligning images and text in a shared space, a promising approach for data-scarce conditions. However, the influence of prompt design on recognizing visually similar categories, such as human postures, is not well understood. This study investigates how prompt specificity affects the zero-shot classification of sitting, standing, and walking/running on a small, 285-image COCO-derived dataset. A suite of modern VLMs, including OpenCLIP, MetaCLIP 2, and SigLip, were evaluated using a three-tiered prompt design that systematically increases linguistic detail. Our findings reveal a compelling, counter-intuitive trend: for the highest-performing models (MetaCLIP 2 and OpenCLIP), the simplest, most basic prompts consistently achieve the best results. Adding descriptive detail significantly degrades performance for instance, MetaCLIP 2’s multi-class accuracy drops from 68.8% to 55.1% a phenomenon we term “prompt overfitting”. Conversely, the lower-performing SigLip model shows improved classification on ambiguous classes when given more descriptive, body-cue-based prompts.

[82] DepthVLA: Enhancing Vision-Language-Action Models with Depth-Aware Spatial Reasoning

Tianyuan Yuan,Yicheng Liu,Chenhao Lu,Zhuoguang Chen,Tao Jiang,Hang Zhao

Main category: cs.CV

TL;DR: DepthVLA是一个改进的Vision-Language-Action模型，通过显式整合深度预测模块提升空间推理能力，显著优于现有方法。

Details

Motivation: 现有VLA模型因依赖Vision-Language Models的空间推理能力，在需要精确空间推理的任务上表现不佳，DeepVLA旨在解决这一问题。

Contribution: 提出了DepthVLA，一种结合深度预测模块的VLA架构，通过共享注意力机制统一视觉-语言-动作处理，显著提升了空间推理能力。

Method: 采用混合Transformer设计，整合VLM、深度Transformer和动作专家模块，共享注意力，实现端到端训练。

Result: 在真实世界和模拟环境中表现优异，分别提升13.5%、1.3%和16.0%的性能。

Insight: 显式引入深度信息是提升VLA模型空间推理能力的有效途径，混合Transformer设计在任务集成中具有潜力。

Abstract: Vision-Language-Action (VLA) models have recently shown impressive generalization and language-guided manipulation capabilities. However, their performance degrades on tasks requiring precise spatial reasoning due to limited spatial reasoning inherited from Vision-Language Models (VLMs). Existing VLAs rely on extensive action-data pretraining to ground VLMs in 3D space, which reduces training efficiency and is still insufficient for accurate spatial understanding. In this work, we present DepthVLA, a simple yet effective VLA architecture that explicitly incorporates spatial awareness through a pretrained depth prediction module. DepthVLA adopts a mixture-of-transformers design that unifies a VLM, a depth transformer, and an action expert with fully shared attentions, forming an end-to-end model with enhanced spatial reasoning. Extensive evaluations in both real-world and simulated environments show that DepthVLA outperforms state-of-the-art approaches, achieving 78.5% vs. 65.0% progress in real-world tasks, 94.9% vs. 93.6% in the LIBERO simulator, and 74.8% vs. 58.8% in the Simpler simulator. Our code will be made publicly available.

[83] Leveraging 2D Priors and SDF Guidance for Dynamic Urban Scene Rendering

Siddharth Tourani,Jayaram Reddy,Akash Kumbar,Satyajit Tourani,Nishant Goyal,Madhava Krishna,N. Dinesh Reddy,Muhammad Haris Khan

Main category: cs.CV

TL;DR: 该论文提出了一种结合2D先验和SDF引导的动态城市场景渲染方法，通过整合3D高斯喷洒与SDF表示，减少了传统方法对LiDAR数据和3D运动标注的依赖，提升了渲染和重建效果。

Details

Motivation: 动态场景渲染在计算机视觉和增强现实中至关重要。现有方法依赖LiDAR数据和3D运动标注，限制了其应用范围。论文旨在通过结合2D先验和SDF表示，减少这些依赖。

Contribution: 提出了一个统一的优化框架，将3D高斯喷洒（3DGS）与SDF结合，提升了动态对象的几何精度和变形建模能力，无需LiDAR和3D运动标注即可实现高质量的渲染和重建。

Method: 使用2D深度和点跟踪作为对象无关的先验，结合SDF表示动态对象，并与3DGS集成。通过联合优化框架提升几何精度和变形建模能力。

Result: 在不使用LiDAR数据的情况下，方法在城市场景中实现了最先进的渲染性能。结合LiDAR后，重建和新视角生成效果进一步改善。此外，支持场景编辑任务（如场景分解和组合）。

Insight: 通过结合2D先验和SDF表示，可以显著减少对3D标注数据的依赖，同时提升动态场景建模的灵活性和精度。这种方法为未来动态场景分析提供了新的思路。

Abstract: Dynamic scene rendering and reconstruction play a crucial role in computer vision and augmented reality. Recent methods based on 3D Gaussian Splatting (3DGS), have enabled accurate modeling of dynamic urban scenes, but for urban scenes they require both camera and LiDAR data, ground-truth 3D segmentations and motion data in the form of tracklets or pre-defined object templates such as SMPL. In this work, we explore whether a combination of 2D object agnostic priors in the form of depth and point tracking coupled with a signed distance function (SDF) representation for dynamic objects can be used to relax some of these requirements. We present a novel approach that integrates Signed Distance Functions (SDFs) with 3D Gaussian Splatting (3DGS) to create a more robust object representation by harnessing the strengths of both methods. Our unified optimization framework enhances the geometric accuracy of 3D Gaussian splatting and improves deformation modeling within the SDF, resulting in a more adaptable and precise representation. We demonstrate that our method achieves state-of-the-art performance in rendering metrics even without LiDAR data on urban scenes. When incorporating LiDAR, our approach improved further in reconstructing and generating novel views across diverse object categories, without ground-truth 3D motion annotation. Additionally, our method enables various scene editing tasks, including scene decomposition, and scene composition.

[84] Generalizing WiFi Gesture Recognition via Large-Model-Aware Semantic Distillation and Alignment

Feng-Qi Cui,Yu-Tong Guo,Tianyue Zheng,Jinyang Huang

Main category: cs.CV

TL;DR: 本文提出了一种名为GLSDA的框架，通过利用预训练大型基础模型的语义先验，提升WiFi手势识别的泛化能力和语义表达能力。

Details

Motivation: 现有WiFi手势识别方法因CSI数据的领域敏感性及缺乏高层手势抽象，导致泛化能力和语义表达能力受限。

Contribution: 1. 提出GLSDA框架，结合大型模型的语义先验增强手势表示学习；2. 设计了双路径CSI编码和多尺度语义编码器；3. 引入语义感知软监督和鲁棒双蒸馏策略。

Method: 通过双路径CSI编码捕捉手势的几何与动态模式，利用多尺度语义编码器学习时序嵌入并跨模态对齐语义，再通过语义感知软监督和双蒸馏策略提升模型性能和效率。

Result: 在Widar3.0基准测试中，GLSDA在域内和跨域任务中均优于SOTA方法，同时显著减少模型大小和推理延迟。

Insight: 结合大型模型的语义先验和轻量化蒸馏策略，可以显著提升RF手势识别的泛化能力和实际部署效率。

Abstract: WiFi-based gesture recognition has emerged as a promising RF sensing paradigm for enabling non-contact and privacy-preserving human-computer interaction in AIoT environments. However, existing methods often suffer from limited generalization and semantic expressiveness due to the domain-sensitive nature of Channel State Information and the lack of high-level gesture abstraction. To address these challenges, we propose a novel generalization framework, termed Large-Model-Aware Semantic Distillation and Alignment (GLSDA), which leverages the semantic prior of pre-trained large foundation models to enhance gesture representation learning in both in-domain and cross-domain scenarios. Specifically, we first design a dual-path CSI encoding pipeline that captures geometric and dynamic gesture patterns via CSI-Ratio phase sequences and Doppler spectrograms. These representations are then fed into a Multiscale Semantic Encoder, which learns robust temporal embeddings and aligns them with gesture semantics through cross-modal attention mechanisms. To further enhance category discrimination, we introduce a Semantic-Aware Soft Supervision scheme that encodes inter-class correlations and reduces label ambiguity, especially for semantically similar gestures. Finally, we develop a Robust Dual-Distillation strategy to compress the aligned model into a lightweight student network, jointly distilling intermediate features and semantic-informed soft labels from the teacher model. Extensive experiments on the Widar3.0 benchmark show that GLSDA consistently outperforms state-of-the-art methods in both in-domain and cross-domain gesture recognition tasks, while significantly reducing model size and inference latency. Our method offers a scalable and deployable solution for generalized RF-based gesture interfaces in real-world AIoT applications.

[85] Spatial-DISE: A Unified Benchmark for Evaluating Spatial Reasoning in Vision-Language Models

Xinmiao Huang,Qisong He,Zhenglin Huang,Boxuan Wang,Zhuoyun Li,Guangliang Cheng,Yi Dong,Xiaowei Huang

Main category: cs.CV

TL;DR: Spatial-DISE提出了一个统一的基准，用于评估视觉语言模型（VLM）的空间推理能力，填补了现有基准在评估动态内在空间推理方面的不足，并通过自动化流程生成了多样化的数据集。

Details

Motivation: 现有基准无法全面评估VLM的空间推理能力，尤其是在动态内在空间推理方面，而这对实际应用（如机器人和增强现实）至关重要。

Contribution: 1. 提出了基于认知分类的统一基准Spatial-DISE；2. 开发了自动化流程生成多样化数据集；3. 评估了28个SOTA VLM，揭示了其与人类的差距。

Method: 设计了一套认知分类法，将任务分为四个象限，并通过自动化流程生成数据，构建了Spatial-DISE Bench和Spatial-DISE-12K数据集。

Result: 评估表明现有VLM在多步多视角空间推理上与人类能力存在显著差距。

Insight: Spatial-DISE为研究人类化空间智能提供了框架、数据和方向。

Abstract: Spatial reasoning ability is crucial for Vision Language Models (VLMs) to support real-world applications in diverse domains including robotics, augmented reality, and autonomous navigation. Unfortunately, existing benchmarks are inadequate in assessing spatial reasoning ability, especially the \emph{intrinsic-dynamic} spatial reasoning which is a fundamental aspect of human spatial cognition. In this paper, we propose a unified benchmark, \textbf{Spatial-DISE}, based on a cognitively grounded taxonomy that categorizes tasks into four fundamental quadrants: \textbf{I}ntrinsic-\textbf{S}tatic, Intrinsic-\textbf{D}ynamic, \textbf{E}xtrinsic-Static, and Extrinsic-Dynamic spatial reasoning. Moreover, to address the issue of data scarcity, we develop a scalable and automated pipeline to generate diverse and verifiable spatial reasoning questions, resulting in a new \textbf{Spatial-DISE} dataset that includes Spatial-DISE Bench (559 evaluation VQA pairs) and Spatial-DISE-12K (12K+ training VQA pairs). Our comprehensive evaluation across 28 state-of-the-art VLMs reveals that, current VLMs have a large and consistent gap to human competence, especially on multi-step multi-view spatial reasoning. Spatial-DISE offers a robust framework, valuable dataset, and clear direction for future research toward human-like spatial intelligence. Benchmark, dataset, and code will be publicly released.

[86] Reinforcement Learning Meets Masked Generative Models: Mask-GRPO for Text-to-Image Generation

Yifu Luo,Xinhao Hu,Keyu Fan,Haoyuan Sun,Zeyu Chen,Bo Xia,Tiantian Zhang,Yongzhe Chang,Xueqian Wang

Main category: cs.CV

TL;DR: Mask-GRPO首次将基于GRPO的强化学习方法引入掩码生成模型，通过重新定义转移概率并将生成过程建模为多步决策问题，实现了文本到图像生成的改进。

Details

Motivation: 现有的强化学习方法主要针对扩散模型或自回归模型，忽略了掩码生成模型这一重要范式。本文旨在填补这一空白，探索掩码生成模型在文本到图像生成中的潜力。

Contribution: 1. 提出了Mask-GRPO，首次将基于GRPO的强化学习应用于掩码生成模型。2. 通过重新定义转移概率，将掩码生成过程建模为多步决策问题。3. 提出了多项优化策略（如移除KL约束、应用降维策略和低质量样本过滤），进一步提升了生成效果。

Method: 1. 重新定义转移概率，将掩码生成建模为多步决策问题。2. 引入GRPO强化学习方法优化生成过程。3. 结合多项策略（移除KL约束、降维、过滤低质量样本）提升生成质量。

Result: 在标准文本到图像生成基准测试和偏好对齐任务中，Mask-GRPO显著超越了现有方法，实现了更高的生成质量和性能。

Insight: 掩码生成模型结合强化学习的潜力值得进一步研究，特别是在生成任务中，多步决策建模可以显著提升生成效率和效果。

Abstract: Reinforcement learning (RL) has garnered increasing attention in text-to-image (T2I) generation. However, most existing RL approaches are tailored to either diffusion models or autoregressive models, overlooking an important alternative: masked generative models. In this work, we propose Mask-GRPO, the first method to incorporate Group Relative Policy Optimization (GRPO)-based RL into this overlooked paradigm. Our core insight is to redefine the transition probability, which is different from current approaches, and formulate the unmasking process as a multi-step decision-making problem. To further enhance our method, we explore several useful strategies, including removing the KL constraint, applying the reduction strategy, and filtering out low-quality samples. Using Mask-GRPO, we improve a base model, Show-o, with substantial improvements on standard T2I benchmarks and preference alignment, outperforming existing state-of-the-art approaches. The code is available on https://github.com/xingzhejun/Mask-GRPO

[87] Ultra High-Resolution Image Inpainting with Patch-Based Content Consistency Adapter

Jianhui Zhang,Sheng Cheng,Qirui Sun,Jia Liu,Wang Luyang,Chaoyu Feng,Chen Fang,Lei Lei,Jue Wang,Shuaicheng Liu

Main category: cs.CV

TL;DR: Patch-Adapter是一种高效的文本引导图像修复框架，支持4K+超高分辨率，通过两阶段适配器架构解决了高分率修复的全局结构和局部细节一致性问题。

Details

Motivation: 现有的图像修复方法在超高分辨率（如4K+）下难以保持内容一致性和提示对齐，Patch-Adapter旨在填补这一技术空白。

Contribution: 提出了两阶段适配器架构（Dual Context Adapter和Reference Patch Adapter），实现了超高分辨率图像修复，并在全局和局部层面分别保持了内容一致性。

Method: 采用双阶段适配器架构：1）Dual Context Adapter在降分辨率下学习掩码与非掩码区域的连贯性；2）Reference Patch Adapter通过块级注意力机制实现全分辨率修复，保留局部细节。

Result: 在OpenImages和Photo-Concept-Bucket数据集上表现优异，修复效果优于现有方法，尤其是感知质量和文本提示对齐方面。

Insight: 通过解耦全局语义和局部细化，Patch-Adapter解决了高分辨率修复的可扩展性问题，为超高分辨率图像编辑提供了新思路。

Abstract: In this work, we present Patch-Adapter, an effective framework for high-resolution text-guided image inpainting. Unlike existing methods limited to lower resolutions, our approach achieves 4K+ resolution while maintaining precise content consistency and prompt alignment, two critical challenges in image inpainting that intensify with increasing resolution and texture complexity. Patch-Adapter leverages a two-stage adapter architecture to scale the diffusion model’s resolution from 1K to 4K+ without requiring structural overhauls: (1) Dual Context Adapter learns coherence between masked and unmasked regions at reduced resolutions to establish global structural consistency; and (2) Reference Patch Adapter implements a patch-level attention mechanism for full-resolution inpainting, preserving local detail fidelity through adaptive feature fusion. This dual-stage architecture uniquely addresses the scalability gap in high-resolution inpainting by decoupling global semantics from localized refinement. Experiments demonstrate that Patch-Adapter not only resolves artifacts common in large-scale inpainting but also achieves state-of-the-art performance on the OpenImages and Photo-Concept-Bucket datasets, outperforming existing methods in both perceptual quality and text-prompt adherence.

[88] VIST3A: Text-to-3D by Stitching a Multi-view Reconstruction Network to a Video Generator

Hyojun Go,Dominik Narnhofer,Goutam Bhat,Prune Truong,Federico Tombari,Konrad Schindler

Main category: cs.CV

TL;DR: VIST3A是一个结合文本到视频生成器和3D重建网络的框架，通过模型拼接和对齐技术，实现了高质量的文本到3D生成，超越了传统的Gaussian splats方法。

Details

Motivation: 近年来，大规模的预训练模型在视觉内容生成和3D重建方面取得了显著进展，这为文本到3D生成提供了新的可能性。研究者希望结合文本到视频生成器的丰富知识和高性能3D重建系统的几何能力。

Contribution: 1. 提出了VIST3A框架，将文本到视频生成器与现代3D重建系统结合；2. 通过模型拼接技术无缝连接两部分；3. 采用直接奖励微调对齐生成器和解码器，确保生成的3D几何一致性。

Method: 1. 模型拼接：识别3D解码器中与视频生成器潜在表示匹配的层，并进行拼接；2. 对齐技术：使用直接奖励微调（direct reward finetuning）确保生成的一致性。

Result: VIST3A在不同视频生成器和3D重建模型上都表现优异，显著超越了基于Gaussian splats的现有文本到3D模型，并支持高质量的文本到点云生成。

Insight: 结合现有预训练模型的优势可以显著提升新任务的性能；模型拼接和对齐技术是实现多模块协同的有效手段。

Abstract: The rapid progress of large, pretrained models for both visual content generation and 3D reconstruction opens up new possibilities for text-to-3D generation. Intuitively, one could obtain a formidable 3D scene generator if one were able to combine the power of a modern latent text-to-video model as “generator” with the geometric abilities of a recent (feedforward) 3D reconstruction system as “decoder”. We introduce VIST3A, a general framework that does just that, addressing two main challenges. First, the two components must be joined in a way that preserves the rich knowledge encoded in their weights. We revisit model stitching, i.e., we identify the layer in the 3D decoder that best matches the latent representation produced by the text-to-video generator and stitch the two parts together. That operation requires only a small dataset and no labels. Second, the text-to-video generator must be aligned with the stitched 3D decoder, to ensure that the generated latents are decodable into consistent, perceptually convincing 3D scene geometry. To that end, we adapt direct reward finetuning, a popular technique for human preference alignment. We evaluate the proposed VIST3A approach with different video generators and 3D reconstruction models. All tested pairings markedly improve over prior text-to-3D models that output Gaussian splats. Moreover, by choosing a suitable 3D base model, VIST3A also enables high-quality text-to-pointmap generation.

[89] Through the Lens of Doubt: Robust and Efficient Uncertainty Estimation for Visual Place Recognition

Emily Miller,Michael Milford,Muhammad Burhan Hafez,SD Ramchurn,Shoaib Ehsan

Main category: cs.CV

TL;DR: 本文提出了三种无需训练的VPR不确定性估计方法，通过分析相似性分数的统计特性，显著提升了匹配可靠性。

Details

Motivation: VPR在动态环境中面临识别准确性挑战，尤其在失败关键的SLAM应用中，需要可靠的不确定性估计来提高匹配置信度。

Contribution: 提出了三种无需训练的不确定性度量：Similarity Distribution (SD)、Ratio Spread (RS)和Statistical Uncertainty (SU)，这些方法无需额外验证数据，适用于多种VPR方法和数据集。

Method: SD量化候选匹配的分数差异性，RS评估高分区域的竞争模糊性，SU结合SD和RS形成统一度量。所有方法无需训练或修改模型，计算开销极低。

Result: 在九种VPR方法和六个基准数据集上的实验表明，提出的方法能有效区分正确与错误匹配，优于现有方法，且计算开销可忽略。

Insight: 利用统计特性设计的不确定性度量方法，能够泛化到不同VPR方法和数据集，显著提升鲁棒性和计算效率。

Abstract: Visual Place Recognition (VPR) enables robots and autonomous vehicles to identify previously visited locations by matching current observations against a database of known places. However, VPR systems face significant challenges when deployed across varying visual environments, lighting conditions, seasonal changes, and viewpoints changes. Failure-critical VPR applications, such as loop closure detection in simultaneous localization and mapping (SLAM) pipelines, require robust estimation of place matching uncertainty. We propose three training-free uncertainty metrics that estimate prediction confidence by analyzing inherent statistical patterns in similarity scores from any existing VPR method. Similarity Distribution (SD) quantifies match distinctiveness by measuring score separation between candidates; Ratio Spread (RS) evaluates competitive ambiguity among top-scoring locations; and Statistical Uncertainty (SU) is a combination of SD and RS that provides a unified metric that generalizes across datasets and VPR methods without requiring validation data to select the optimal metric. All three metrics operate without additional model training, architectural modifications, or computationally expensive geometric verification. Comprehensive evaluation across nine state-of-the-art VPR methods and six benchmark datasets confirms that our metrics excel at discriminating between correct and incorrect VPR matches, and consistently outperform existing approaches while maintaining negligible computational overhead, making it deployable for real-time robotic applications across varied environmental conditions with improved precision-recall performance.

[90] ExpressNet-MoE: A Hybrid Deep Neural Network for Emotion Recognition

Deeptimaan Banerjee,Prateek Gothwal,Ashis Kumer Biswas

Main category: cs.CV

TL;DR: ExpressNet-MoE提出了一种结合CNN和MoE的混合深度学习模型，用于面部情绪识别（FER），通过动态选择专家网络和多尺度特征提取，显著提升了准确率。

Details

Motivation: 当前FER模型在真实场景中面临姿态变化、遮挡和光照等因素的挑战，且缺乏灵活性。本文旨在开发一种适应性强的模型，以提升情绪识别的泛化能力。

Contribution: 1. 提出ExpressNet-MoE，结合CNN和MoE框架；2. 动态选择专家网络以提升泛化能力；3. 多尺度特征提取捕获全局和局部特征。

Method: 模型包括多个CNN特征提取器、MoE模块（自适应特征选择）和残差网络（深度特征学习），通过多尺度提取特征并在多个数据集上验证。

Result: 在AffectNet（v7和v8）、RAF-DB和FER-2013数据集上分别达到74.77%、72.55%、84.29%和64.66%的准确率，优于现有方法。

Insight: 动态专家选择和多尺度特征是提升FER性能的关键，模型适用于复杂真实场景。

Abstract: In many domains, including online education, healthcare, security, and human-computer interaction, facial emotion recognition (FER) is essential. Real-world FER is still difficult despite its significance because of some factors such as variable head positions, occlusions, illumination shifts, and demographic diversity. Engagement detection, which is essential for applications like virtual learning and customer services, is frequently challenging due to FER limitations by many current models. In this article, we propose ExpressNet-MoE, a novel hybrid deep learning model that blends both Convolution Neural Networks (CNNs) and Mixture of Experts (MoE) framework, to overcome the difficulties. Our model dynamically chooses the most pertinent expert networks, thus it aids in the generalization and providing flexibility to model across a wide variety of datasets. Our model improves on the accuracy of emotion recognition by utilizing multi-scale feature extraction to collect both global and local facial features. ExpressNet-MoE includes numerous CNN-based feature extractors, a MoE module for adaptive feature selection, and finally a residual network backbone for deep feature learning. To demonstrate efficacy of our proposed model we evaluated on several datasets, and compared with current state-of-the-art methods. Our model achieves accuracies of 74.77% on AffectNet (v7), 72.55% on AffectNet (v8), 84.29% on RAF-DB, and 64.66% on FER-2013. The results show how adaptive our model is and how it may be used to develop end-to-end emotion recognition systems in practical settings. Reproducible codes and results are made publicly accessible at https://github.com/DeeptimaanB/ExpressNet-MoE.

[91] UniME-V2: MLLM-as-a-Judge for Universal Multimodal Embedding Learning

Tiancheng Gu,Kaicheng Yang,Kaichen Zhang,Xiang An,Ziyong Feng,Yueyi Zhang,Weidong Cai,Jiankang Deng,Lidong Bing

Main category: cs.CV

TL;DR: UniME-V2提出了一种基于MLLM的多模态嵌入学习方法，通过MLLM-as-a-Judge机制生成软语义匹配分数，用于硬负样本挖掘和表示学习，显著提升模型性能。

Details

Motivation: 现有方法难以捕捉候选样本间的细微语义差异，且负样本缺乏多样性；嵌入表示在区分错误和硬负样本时能力有限。

Contribution: 1) 引入MLLM-as-a-Judge机制生成软语义匹配分数；2) 提出全局检索构建硬负样本集；3) 结合软标签缓解一对一映射约束；4) 提出UniME-V2-Reranker进一步优化性能。

Method: 1) 全局检索构建硬负样本集；2) MLLM生成软语义匹配分数；3) 对齐相似度矩阵与软语义匹配分数矩阵；4) 联合成对和列表优化训练重排序模型。

Result: 在MMEB基准和多个检索任务中，UniME-V2平均性能达到SOTA。

Insight: 利用MLLM的高级理解能力生成软语义分数，可以有效挖掘高质量硬负样本并提升模型判别能力；软标签缓解了传统方法的刚性约束。

Abstract: Universal multimodal embedding models are foundational to various tasks. Existing approaches typically employ in-batch negative mining by measuring the similarity of query-candidate pairs. However, these methods often struggle to capture subtle semantic differences among candidates and lack diversity in negative samples. Moreover, the embeddings exhibit limited discriminative ability in distinguishing false and hard negatives. In this paper, we leverage the advanced understanding capabilities of MLLMs to enhance representation learning and present a novel Universal Multimodal Embedding (UniME-V2) model. Our approach first constructs a potential hard negative set through global retrieval. We then introduce the MLLM-as-a-Judge mechanism, which utilizes MLLMs to assess the semantic alignment of query-candidate pairs and generate soft semantic matching scores. These scores serve as a foundation for hard negative mining, mitigating the impact of false negatives and enabling the identification of diverse, high-quality hard negatives. Furthermore, the semantic matching scores are used as soft labels to mitigate the rigid one-to-one mapping constraint. By aligning the similarity matrix with the soft semantic matching score matrix, the model learns semantic distinctions among candidates, significantly enhancing its discriminative capacity. To further improve performance, we propose UniME-V2-Reranker, a reranking model trained on our mined hard negatives through a joint pairwise and listwise optimization approach. We conduct comprehensive experiments on the MMEB benchmark and multiple retrieval tasks, demonstrating that our method achieves state-of-the-art performance on average across all tasks.

[92] Learning Neural Parametric 3D Breast Shape Models for Metrical Surface Reconstruction From Monocular RGB Videos

Maximilian Weiherer,Antonia von Riedheim,Vanessa Brébant,Bernhard Egger,Christoph Palm

Main category: cs.CV

TL;DR: 论文提出了一种神经参数化的3D乳房形状模型（liRBSM），并基于该模型开发了一种低成本、易用的3D表面重建流程，可以从单目RGB视频中恢复精确的乳房几何形状。

Details

Motivation: 现有的商业3D乳房扫描解决方案成本高昂，而低成本替代方案通常依赖专用硬件或专有软件。作者希望通过一种无需特殊设备和软件的方法实现高质量的乳房几何重建。

Contribution: 1. 提出了局部化的神经参数3D乳房形状模型（liRBSM），相比全局模型（iRBSM）能提供更详细的表面重建。2. 开发了一种基于单目RGB视频的低成本3D乳房表面重建流程，误差小于2毫米。3. 模型和流程完全开源透明，可在设备上快速运行（少于6分钟）。

Method: 1. 利用现成的Structure-from-motion流程初步重建3D形状。2. 提出局部隐式神经表示模型（liRBSM），将乳房区域分解为多个局部区域，每个区域由基于解剖标志的局部神经SDF表示。3. 结合参数化模型进行鲁棒和度量正确的表面重建。

Result: 实验表明，liRBSM在重建质量上显著优于全局模型iRBSM，能够恢复高精度的3D乳房几何形状（误差小于2毫米），且速度快（少于6分钟）。

Insight: 1. 局部隐式表示比全局表示更适合复杂的解剖结构建模。2. 结合现成的SfM技术和参数化模型可以实现低成本、高精度的3D重建。

Abstract: We present a neural parametric 3D breast shape model and, based on this model, introduce a low-cost and accessible 3D surface reconstruction pipeline capable of recovering accurate breast geometry from a monocular RGB video. In contrast to widely used, commercially available yet prohibitively expensive 3D breast scanning solutions and existing low-cost alternatives, our method requires neither specialized hardware nor proprietary software and can be used with any device that is able to record RGB videos. The key building blocks of our pipeline are a state-of-the-art, off-the-shelf Structure-from-motion pipeline, paired with a parametric breast model for robust and metrically correct surface reconstruction. Our model, similarly to the recently proposed implicit Regensburg Breast Shape Model (iRBSM), leverages implicit neural representations to model breast shapes. However, unlike the iRBSM, which employs a single global neural signed distance function (SDF), our approach – inspired by recent state-of-the-art face models – decomposes the implicit breast domain into multiple smaller regions, each represented by a local neural SDF anchored at anatomical landmark positions. When incorporated into our surface reconstruction pipeline, the proposed model, dubbed liRBSM (short for localized iRBSM), significantly outperforms the iRBSM in terms of reconstruction quality, yielding more detailed surface reconstruction than its global counterpart. Overall, we find that the introduced pipeline is able to recover high-quality 3D breast geometry within an error margin of less than 2 mm. Our method is fast (requires less than six minutes), fully transparent and open-source, and – together with the model – publicly available at https://rbsm.re-mic.de/local-implicit.

[93] Accelerated Feature Detectors for Visual SLAM: A Comparative Study of FPGA vs GPU

Ruiqi Ye,Mikel Luján

Main category: cs.CV

TL;DR: 该论文首次比较了FPGA与GPU在视觉SLAM（V-SLAM）中加速特征检测器的性能，发现非学习型检测器（如FAST和Harris）在GPU上表现更优，而学习型检测器（如SuperPoint）在FPGA上更高效。

Details

Motivation: 随着SLAM技术在功耗受限平台（如无人机）上的广泛应用，特征检测的效率成为关键瓶颈。GPU和FPGA是常见的硬件加速选择，但缺乏对它们在V-SLAM中性能的系统比较。

Contribution: 论文首次对FPGA和GPU加速的特征检测器在V-SLAM中的性能进行了全面比较，揭示了学习型与非学习型检测器在不同硬件上的优劣。

Method: 通过在现代SoC（Nvidia Jetson Orin和AMD Versal）上实现FAST、Harris和SuperPoint的GPU与FPGA版本，并嵌入V-SLAM流程，对比运行时性能、能效和精度。

Result: 非学习型检测器（FAST和Harris）在GPU上更快更高效；学习型检测器（SuperPoint）在FPGA上表现更优（分别提升3.1倍和1.4倍）。GPU-SLAM精度更高，FPGA-SLAM在某些数据集上FPS更优。

Insight: 学习型检测器更适合FPGA加速，而非学习型更依赖GPU；硬件加速可减少全局优化模块的调用频率，从而在不牺牲精度的情况下提升SLAM性能。

Abstract: Feature detection is a common yet time-consuming module in Simultaneous Localization and Mapping (SLAM) implementations, which are increasingly deployed on power-constrained platforms, such as drones. Graphics Processing Units (GPUs) have been a popular accelerator for computer vision in general, and feature detection and SLAM in particular. On the other hand, System-on-Chips (SoCs) with integrated Field Programmable Gate Array (FPGA) are also widely available. This paper presents the first study of hardware-accelerated feature detectors considering a Visual SLAM (V-SLAM) pipeline. We offer new insights by comparing the best GPU-accelerated FAST, Harris, and SuperPoint implementations against the FPGA-accelerated counterparts on modern SoCs (Nvidia Jetson Orin and AMD Versal). The evaluation shows that when using a non-learning-based feature detector such as FAST and Harris, their GPU implementations, and the GPU-accelerated V-SLAM can achieve better run-time performance and energy efficiency than the FAST and Harris FPGA implementations as well as the FPGA-accelerated V-SLAM. However, when considering a learning-based detector such as SuperPoint, its FPGA implementation can achieve better run-time performance and energy efficiency (up to 3.1$\times$ and 1.4$\times$ improvements, respectively) than the GPU implementation. The FPGA-accelerated V-SLAM can also achieve comparable run-time performance compared to the GPU-accelerated V-SLAM, with better FPS in 2 out of 5 dataset sequences. When considering the accuracy, the results show that the GPU-accelerated V-SLAM is more accurate than the FPGA-accelerated V-SLAM in general. Last but not least, the use of hardware acceleration for feature detection could further improve the performance of the V-SLAM pipeline by having the global bundle adjustment module invoked less frequently without sacrificing accuracy.

[94] Modeling Cultural Bias in Facial Expression Recognition with Adaptive Agents

David Freire-Obregón,José Salas-Cáceres,Javier Lorenzo-Navarro,Oliverio J. Santana,Daniel Hernández-Sosa,Modesto Castrillón-Santana

Main category: cs.CV

TL;DR: 该论文提出了一个基于代理的流式基准测试，用于研究文化偏见和图像模糊如何影响面部表情识别（FER）的鲁棒性。通过实验发现不同文化群体在模糊条件下的表现存在不对称性，并量化了文化组成和交互结构对FER鲁棒性的影响。

Details

Motivation: 现有的FER评估大多假设数据同质且图像质量高，忽略了文化差异和感知退化条件的影响。本研究旨在揭示文化偏见和图像模糊如何共同影响FER的鲁棒性。

Contribution: 1) 引入了一个基于代理的流式基准测试，用于研究文化偏见和图像模糊的交互作用；2) 发现了不同文化群体在模糊条件下的不对称性能退化曲线；3) 量化了文化组成和交互结构对FER鲁棒性的影响。

Method: 研究者采用了基于CLIP特征空间的轻量级残差适配器，代理在该空间中移动和交互。实验设置了不同文化组成（单一文化、平衡/不平衡混合文化）和不同模糊程度的环境，分析了代理的性能表现。

Result: 实验结果表明：亚洲文化群体（JAFFE）在低模糊条件下表现更好，但在中等模糊时性能下降更快；西方文化群体（KDEF）的表现退化更均匀。混合文化群体在平衡条件下表现较好，但不平衡条件下会放大主流文化群体的弱点。

Insight: 文化偏见和图像模糊对FER的影响是复杂的，文化组成和环境条件会显著改变模型的鲁棒性。平衡的文化混合可能有助于缓解早期性能退化。

Abstract: Facial expression recognition (FER) must remain robust under both cultural variation and perceptually degraded visual conditions, yet most existing evaluations assume homogeneous data and high-quality imagery. We introduce an agent-based, streaming benchmark that reveals how cross-cultural composition and progressive blurring interact to shape face recognition robustness. Each agent operates in a frozen CLIP feature space with a lightweight residual adapter trained online at sigma=0 and fixed during testing. Agents move and interact on a 5x5 lattice, while the environment provides inputs with sigma-scheduled Gaussian blur. We examine monocultural populations (Western-only, Asian-only) and mixed environments with balanced (5/5) and imbalanced (8/2, 2/8) compositions, as well as different spatial contact structures. Results show clear asymmetric degradation curves between cultural groups: JAFFE (Asian) populations maintain higher performance at low blur but exhibit sharper drops at intermediate stages, whereas KDEF (Western) populations degrade more uniformly. Mixed populations exhibit intermediate patterns, with balanced mixtures mitigating early degradation, but imbalanced settings amplify majority-group weaknesses under high blur. These findings quantify how cultural composition and interaction structure influence the robustness of FER as perceptual conditions deteriorate.

[95] Fusion Meets Diverse Conditions: A High-diversity Benchmark and Baseline for UAV-based Multimodal Object Detection with Condition Cues

Chen Chen,Kangcheng Bin,Ting Hu,Jiahao Qi,Xingyue Liu,Tianpeng Liu,Zhen Liu,Yongxiang Liu,Ping Zhong

Main category: cs.CV

TL;DR: 论文提出了一种高多样性的无人机多模态目标检测数据集ATR-UMOD，并设计了基于条件提示的动态融合方法PCDF，以自适应地分配RGB和红外图像的贡献。

Details

Motivation: 现有无人机目标检测数据集难以覆盖真实世界的复杂成像条件，限制了模型的鲁棒性。

Contribution: 1. 提出了高多样性的ATR-UMOD数据集；2. 设计了基于条件提示的动态融合方法PCDF。

Method: PCDF通过文本提示编码成像条件，利用任务特定的软门控变换建模条件与多模态贡献的关系，并通过条件解耦模块增强实用性。

Result: 实验表明PCDF在ATR-UMOD数据集上表现优异。

Insight: 条件提示有助于动态融合多模态数据，提升模型在复杂环境中的适应能力。

Abstract: Unmanned aerial vehicles (UAV)-based object detection with visible (RGB) and infrared (IR) images facilitates robust around-the-clock detection, driven by advancements in deep learning techniques and the availability of high-quality dataset. However, the existing dataset struggles to fully capture real-world complexity for limited imaging conditions. To this end, we introduce a high-diversity dataset ATR-UMOD covering varying scenarios, spanning altitudes from 80m to 300m, angles from 0{\deg} to 75{\deg}, and all-day, all-year time variations in rich weather and illumination conditions. Moreover, each RGB-IR image pair is annotated with 6 condition attributes, offering valuable high-level contextual information. To meet the challenge raised by such diverse conditions, we propose a novel prompt-guided condition-aware dynamic fusion (PCDF) to adaptively reassign multimodal contributions by leveraging annotated condition cues. By encoding imaging conditions as text prompts, PCDF effectively models the relationship between conditions and multimodal contributions through a task-specific soft-gating transformation. A prompt-guided condition-decoupling module further ensures the availability in practice without condition annotations. Experiments on ATR-UMOD dataset reveal the effectiveness of PCDF.

[96] AVAR-Net: A Lightweight Audio-Visual Anomaly Recognition Framework with a Benchmark Dataset

Amjid Ali,Zulfiqar Ahmad Khan,Altaf Hussain,Muhammad Munsif,Adnan Hussain,Sung Wook Baik

Main category: cs.CV

TL;DR: 论文提出了AVAR-Net，一种轻量级音频-视觉异常识别框架，并引入了VAAR数据集，用于多模态异常识别研究。

Details

Motivation: 现有方法主要依赖视觉数据，在遮挡、低光照等挑战性条件下不可靠，且缺乏大规模同步音频-视觉数据集。

Contribution: 1. 提出了轻量级AVAR-Net框架；2. 引入了VAAR数据集；3. 在多个数据集上验证了性能提升。

Method: AVAR-Net包含四个模块：音频特征提取器（Wav2Vec2）、视频特征提取器（MobileViT）、融合策略和MTCN网络。

Result: 在VAAR和XD-Violence数据集上分别达到89.29%准确率和88.56%平均精度，优于现有方法。

Insight: 多模态融合显著提升了异常识别性能，同时轻量化设计使其适用于实际场景。

Abstract: Anomaly recognition plays a vital role in surveillance, transportation, healthcare, and public safety. However, most existing approaches rely solely on visual data, making them unreliable under challenging conditions such as occlusion, low illumination, and adverse weather. Moreover, the absence of large-scale synchronized audio-visual datasets has hindered progress in multimodal anomaly recognition. To address these limitations, this study presents AVAR-Net, a lightweight and efficient audio-visual anomaly recognition framework designed for real-world environments. AVAR-Net consists of four main modules: an audio feature extractor, a video feature extractor, fusion strategy, and a sequential pattern learning network that models cross-modal relationships for anomaly recognition. Specifically, the Wav2Vec2 model extracts robust temporal features from raw audio, while MobileViT captures both local and global visual representations from video frames. An early fusion mechanism combines these modalities, and a Multi-Stage Temporal Convolutional Network (MTCN) model that learns long-range temporal dependencies within the fused representation, enabling robust spatiotemporal reasoning. A novel Visual-Audio Anomaly Recognition (VAAR) dataset, is also introduced, serving as a medium-scale benchmark containing 3,000 real-world videos with synchronized audio across ten diverse anomaly classes. Experimental evaluations demonstrate that AVAR-Net achieves 89.29% accuracy on VAAR and 88.56% Average Precision on the XD-Violence dataset, improving Average Precision by 2.8% over existing state-of-the-art methods. These results highlight the effectiveness, efficiency, and generalization capability of the proposed framework, as well as the utility of VAAR as a benchmark for advancing multimodal anomaly recognition research.

[97] Challenges, Advances, and Evaluation Metrics in Medical Image Enhancement: A Systematic Literature Review

Chun Wai Chin,Haniza Yazid,Hoi Leong Lee

Main category: cs.CV

TL;DR: 这篇系统性文献综述分析了医学图像增强的关键挑战、最新进展和评估指标，总结了39项研究的结果，指出了低对比度和噪声是主要问题，并探讨了不同方法和评估指标的优劣。

Details

Motivation: 医学图像中常见的噪声、伪影和低对比度限制了其诊断潜力，因此需要有效的增强方法来提升图像质量和可解释性。

Contribution: 系统地总结了医学图像增强的挑战、方法和评估指标，指出了MRI和多模态成像研究较多，而其他模态研究不足的现状。

Method: 采用PRISMA方法分析了39项研究，对比了传统数学方法、深度学习和混合方法，并统计了65种图像质量评估指标。

Result: 低对比度和噪声是最常见的问题，MRI和多模态成像研究较多；传统方法占多数（29项），深度学习仅9项；非参考性评估指标更常用。

Insight: 深度学习方法在医学图像增强中潜力巨大但研究较少，未来可探索更多模态和混合方法，同时需要统一的评估标准。

Abstract: Medical image enhancement is crucial for improving the quality and interpretability of diagnostic images, ultimately supporting early detection, accurate diagnosis, and effective treatment planning. Despite advancements in imaging technologies such as X-ray, CT, MRI, and ultrasound, medical images often suffer from challenges like noise, artifacts, and low contrast, which limit their diagnostic potential. Addressing these challenges requires robust preprocessing, denoising algorithms, and advanced enhancement methods, with deep learning techniques playing an increasingly significant role. This systematic literature review, following the PRISMA approach, investigates the key challenges, recent advancements, and evaluation metrics in medical image enhancement. By analyzing findings from 39 peer-reviewed studies, this review provides insights into the effectiveness of various enhancement methods across different imaging modalities and the importance of evaluation metrics in assessing their impact. Key issues like low contrast and noise are identified as the most frequent, with MRI and multi-modal imaging receiving the most attention, while specialized modalities such as histopathology, endoscopy, and bone scintigraphy remain underexplored. Out of the 39 studies, 29 utilize conventional mathematical methods, 9 focus on deep learning techniques, and 1 explores a hybrid approach. In terms of image quality assessment, 18 studies employ both reference-based and non-reference-based metrics, 9 rely solely on reference-based metrics, and 12 use only non-reference-based metrics, with a total of 65 IQA metrics introduced, predominantly non-reference-based. This review highlights current limitations, research gaps, and potential future directions for advancing medical image enhancement.

[98] EditCast3D: Single-Frame-Guided 3D Editing with Video Propagation and View Selection

Huaizhi Qu,Ruichen Zhang,Shuqing Luo,Luchao Qi,Zhihao Zhang,Xiaoming Liu,Roni Sengupta,Tianlong Chen

Main category: cs.CV

TL;DR: EditCast3D通过视频生成基础模型，从单帧编辑传播到整个数据集，并结合视图选择策略，实现了高效且一致的3D编辑。

Details

Motivation: 现有基础模型在图像编辑中表现优异，但在3D编辑中扩展不足，计算量大且依赖闭源API，限制了其应用。

Contribution: 提出EditCast3D，通过视频传播和视图选择策略，减少了昂贵图像编辑的依赖，并解决了多视图对齐问题。

Method: 利用视频生成模型传播单帧编辑，引入视图选择策略以确保一致性，并采用前馈重建避免高成本细化。

Result: 在3D编辑数据集上优于现有方法，实现了高质量的编辑和高效率。

Insight: 结合视频模型和视图选择可为3D编辑提供一致且高效的新范式。

Abstract: Recent advances in foundation models have driven remarkable progress in image editing, yet their extension to 3D editing remains underexplored. A natural approach is to replace the image editing modules in existing workflows with foundation models. However, their heavy computational demands and the restrictions and costs of closed-source APIs make plugging these models into existing iterative editing strategies impractical. To address this limitation, we propose EditCast3D, a pipeline that employs video generation foundation models to propagate edits from a single first frame across the entire dataset prior to reconstruction. While editing propagation enables dataset-level editing via video models, its consistency remains suboptimal for 3D reconstruction, where multi-view alignment is essential. To overcome this, EditCast3D introduces a view selection strategy that explicitly identifies consistent and reconstruction-friendly views and adopts feedforward reconstruction without requiring costly refinement. In combination, the pipeline both minimizes reliance on expensive image editing and mitigates prompt ambiguities that arise when applying foundation models independently across images. We evaluate EditCast3D on commonly used 3D editing datasets and compare it against state-of-the-art 3D editing baselines, demonstrating superior editing quality and high efficiency. These results establish EditCast3D as a scalable and general paradigm for integrating foundation models into 3D editing pipelines. The code is available at https://github.com/UNITES-Lab/EditCast3D

[99] OmniGaze: Reward-inspired Generalizable Gaze Estimation In The Wild

Hongyu Qu,Jianan Wei,Xiangbo Shu,Yazhou Yao,Wenguan Wang,Jinhui Tang

Main category: cs.CV

TL;DR: OmniGaze是一个半监督框架，利用大规模无标签数据克服领域偏差，提升3D视线估计的泛化能力。通过伪标签奖励模型和多模态语言模型生成语义线索，选择高质量伪标签并进行加权损失计算，实验表明其在多种数据集上表现优异，具备零样本泛化能力。

Details

Motivation: 现有3D视线估计方法因标注数据稀缺和多样性不足，难以跨领域泛化。OmniGaze旨在利用无约束环境中的无标签数据解决这一问题。

Contribution: 提出了OmniGaze框架，结合伪标签奖励模型和多模态语言模型，提升视线估计的泛化能力；展示了其作为可扩展数据引擎的潜力。

Method: 1) 构建多样化无标签数据集；2) 设计奖励模型评估伪标签可靠性，结合视觉嵌入和语义线索；3) 选择高质量伪标签并加权损失计算。

Result: 在五个数据集上取得SOTA性能，跨领域和零样本泛化表现优异。

Insight: 通过多模态信息（视觉和语义）增强伪标签质量，是提升半监督学习泛化能力的有效方向。

Abstract: Current 3D gaze estimation methods struggle to generalize across diverse data domains, primarily due to i) the scarcity of annotated datasets, and ii) the insufficient diversity of labeled data. In this work, we present OmniGaze, a semi-supervised framework for 3D gaze estimation, which utilizes large-scale unlabeled data collected from diverse and unconstrained real-world environments to mitigate domain bias and generalize gaze estimation in the wild. First, we build a diverse collection of unlabeled facial images, varying in facial appearances, background environments, illumination conditions, head poses, and eye occlusions. In order to leverage unlabeled data spanning a broader distribution, OmniGaze adopts a standard pseudo-labeling strategy and devises a reward model to assess the reliability of pseudo labels. Beyond pseudo labels as 3D direction vectors, the reward model also incorporates visual embeddings extracted by an off-the-shelf visual encoder and semantic cues from gaze perspective generated by prompting a Multimodal Large Language Model to compute confidence scores. Then, these scores are utilized to select high-quality pseudo labels and weight them for loss computation. Extensive experiments demonstrate that OmniGaze achieves state-of-the-art performance on five datasets under both in-domain and cross-domain settings. Furthermore, we also evaluate the efficacy of OmniGaze as a scalable data engine for gaze estimation, which exhibits robust zero-shot generalization on four unseen datasets.

[100] CanvasMAR: Improving Masked Autoregressive Video Generation With Canvas

Zian Li,Muhan Zhang

Main category: cs.CV

TL;DR: CanvasMAR提出了一种改进的视频生成方法，通过引入画布机制和组合式无分类器引导，解决了传统掩码自回归模型中的慢启动问题和误差累积问题，显著提升了视频生成的效率和质量。

Details

Motivation: 传统掩码自回归视频生成模型存在慢启动问题和时空维度的误差累积，导致生成效率低且质量受限。CanvasMAR旨在通过画布机制和组合式引导优化这些问题。

Contribution: 1）提出画布机制，提供全局结构先验以加速生成；2）引入组合式无分类器引导，增强时空条件；3）通过噪声增强画布提升鲁棒性。

Method: 1）使用画布（模糊的全局预测）作为生成起点；2）组合式无分类器引导联合优化时空条件；3）噪声增强画布。

Result: 在BAIR和Kinetics-600数据集上，CanvasMAR以更少的自回归步骤生成高质量视频，性能在自回归模型中领先，可比扩散模型。

Insight: 画布机制为自回归生成提供了全局先验，有助于提升生成的连贯性和效率；组合式引导进一步优化了时空一致性。

Abstract: Masked autoregressive models (MAR) have recently emerged as a powerful paradigm for image and video generation, combining the flexibility of masked modeling with the potential of continuous tokenizer. However, video MAR models suffer from two major limitations: the slow-start problem, caused by the lack of a structured global prior at early sampling stages, and error accumulation across the autoregression in both spatial and temporal dimensions. In this work, we propose CanvasMAR, a novel video MAR model that mitigates these issues by introducing a canvas mechanism–a blurred, global prediction of the next frame, used as the starting point for masked generation. The canvas provides global structure early in sampling, enabling faster and more coherent frame synthesis. Furthermore, we introduce compositional classifier-free guidance that jointly enlarges spatial (canvas) and temporal conditioning, and employ noise-based canvas augmentation to enhance robustness. Experiments on the BAIR and Kinetics-600 benchmarks demonstrate that CanvasMAR produces high-quality videos with fewer autoregressive steps. Our approach achieves remarkable performance among autoregressive models on Kinetics-600 dataset and rivals diffusion-based methods.

[101] Seeing and Knowing in the Wild: Open-domain Visual Entity Recognition with Large-scale Knowledge Graphs via Contrastive Learning

Hongkuan Zhou,Lavdim Halilaj,Sebastian Monka,Stefan Schmid,Yuqicheng Zhu,Jingcheng Wu,Nadeem Nazer,Steffen Staab

Main category: cs.CV

TL;DR: 论文提出了一种知识引导的对比学习框架（KnowCoL），结合图像和文本描述，通过结构化知识图谱支持开放域视觉实体识别，显著提升了零样本识别的准确性。

Details

Motivation: 开放域视觉实体识别任务面临固定标签集的限制、训练数据稀疏和长尾分布的问题，亟需一种能够利用外部知识的方法来解决这些挑战。

Contribution: 提出KnowCoL框架，整合图像、文本和结构化知识图谱，显著提升开放域视觉实体识别性能，尤其是在零样本和长尾场景下。

Method: 通过对比学习将视觉和文本输入映射到共享语义空间，利用Wikidata中的实体描述、类型层次和关系上下文增强语义理解。

Result: 在OVEN数据集上，即使是较小的模型，对未见实体的识别准确率也比现有最佳方法提升了10.5%。

Insight: 结合多模态数据和结构化知识可以显著提升开放域视觉识别的性能，特别是在数据稀疏和语义模糊的场景下。

Abstract: Open-domain visual entity recognition aims to identify and link entities depicted in images to a vast and evolving set of real-world concepts, such as those found in Wikidata. Unlike conventional classification tasks with fixed label sets, it operates under open-set conditions, where most target entities are unseen during training and exhibit long-tail distributions. This makes the task inherently challenging due to limited supervision, high visual ambiguity, and the need for semantic disambiguation. In this work, we propose a Knowledge-guided Contrastive Learning (KnowCoL) framework that combines both images and text descriptions into a shared semantic space grounded by structured information from Wikidata. By abstracting visual and textual inputs to a conceptual level, the model leverages entity descriptions, type hierarchies, and relational context to support zero-shot entity recognition. We evaluate our approach on the OVEN benchmark, a large-scale open-domain visual recognition dataset with Wikidata IDs as the label space. Our experiments show that using visual, textual, and structured knowledge greatly improves accuracy, especially for rare and unseen entities. Our smallest model improves the accuracy on unseen entities by 10.5% compared to the state-of-the-art, despite being 35 times smaller.

[102] FlashWorld: High-quality 3D Scene Generation within Seconds

Xinyang Li,Tengfei Wang,Zixiao Gu,Shengchuan Zhang,Chunchao Guo,Liujuan Cao

Main category: cs.CV

TL;DR: FlashWorld 是一种生成模型，通过单张图像或文本提示在几秒内生成高质量3D场景，速度比现有方法快10~100倍，同时保持优越的渲染质量。它提出了一种3D导向的方法，取代了传统的多视图导向方法，并通过双模式预训练和跨模式后训练阶段实现了性能提升。

Details

Motivation: 现有的3D场景生成方法通常基于多视图重建范式，速度慢且容易出现3D不一致问题。FlashWorld旨在通过直接生成3D高斯表示来解决这些问题，同时提升生成速度和质量。

Contribution: 1. 提出了一种3D导向的生成范式，直接生成3D高斯表示；2. 设计了双模式预训练和跨模式后训练策略，结合多视图和3D导向方法的优势；3. 通过视频扩散模型的先验和单视图图像增强模型的泛化能力。

Method: 1. 预训练双模式多视图扩散模型，支持多视图和3D导向生成模式；2. 跨模式后训练通过匹配分布缩小3D导向模式的质量差距；3. 利用大规模单视图图像和文本提示增强模型泛化能力。

Result: 实验表明，FlashWorld在生成速度（快10~100倍）和质量上均优于现有方法，且能有效处理分布外输入。

Insight: 1. 3D导向生成范式可以显著提升效率；2. 跨模式训练策略能够在不牺牲3D一致性的情况下提升视觉质量；3. 利用视频扩散模型的先验和单视图数据有助于增强模型的适应性。

Abstract: We propose FlashWorld, a generative model that produces 3D scenes from a single image or text prompt in seconds, 10~100$\times$ faster than previous works while possessing superior rendering quality. Our approach shifts from the conventional multi-view-oriented (MV-oriented) paradigm, which generates multi-view images for subsequent 3D reconstruction, to a 3D-oriented approach where the model directly produces 3D Gaussian representations during multi-view generation. While ensuring 3D consistency, 3D-oriented method typically suffers poor visual quality. FlashWorld includes a dual-mode pre-training phase followed by a cross-mode post-training phase, effectively integrating the strengths of both paradigms. Specifically, leveraging the prior from a video diffusion model, we first pre-train a dual-mode multi-view diffusion model, which jointly supports MV-oriented and 3D-oriented generation modes. To bridge the quality gap in 3D-oriented generation, we further propose a cross-mode post-training distillation by matching distribution from consistent 3D-oriented mode to high-quality MV-oriented mode. This not only enhances visual quality while maintaining 3D consistency, but also reduces the required denoising steps for inference. Also, we propose a strategy to leverage massive single-view images and text prompts during this process to enhance the model’s generalization to out-of-distribution inputs. Extensive experiments demonstrate the superiority and efficiency of our method.

[103] Risk-adaptive Activation Steering for Safe Multimodal Large Language Models

Jonghyun Park,Minhyuk Seo,Jonghyun Choi

Main category: cs.CV

TL;DR: 该论文提出了Risk-adaptive Activation Steering (RAS)方法，通过增强跨模态注意力到安全关键图像区域，动态调整激活以生成安全且有用的响应，避免了迭代输出调整的开销。

Details

Motivation: 现代AI模型在处理多模态查询时，容易对嵌入图像的恶意意图作出不安全响应。传统方法需高成本的安全数据集训练或迭代调整输出，存在效率或准确性问题。

Contribution: 提出RAS方法，通过查询级别的风险评估和动态激活调整，显著降低了攻击成功率，同时保持了任务性能和推理速度。

Method: RAS通过增强对安全关键区域的跨模态注意力，在查询级别进行风险评估，并基于风险动态调整模型激活。

Result: 实验表明，RAS在多模态安全和实用性基准测试中显著优于现有推理时防御方法。

Insight: 动态激活调整是一种高效的多模态模型安全对齐方法，能够在不牺牲性能的情况下提升安全性。

Abstract: One of the key challenges of modern AI models is ensuring that they provide helpful responses to benign queries while refusing malicious ones. But often, the models are vulnerable to multimodal queries with harmful intent embedded in images. One approach for safety alignment is training with extensive safety datasets at the significant costs in both dataset curation and training. Inference-time alignment mitigates these costs, but introduces two drawbacks: excessive refusals from misclassified benign queries and slower inference speed due to iterative output adjustments. To overcome these limitations, we propose to reformulate queries to strengthen cross-modal attention to safety-critical image regions, enabling accurate risk assessment at the query level. Using the assessed risk, it adaptively steers activations to generate responses that are safe and helpful without overhead from iterative output adjustments. We call this Risk-adaptive Activation Steering (RAS). Extensive experiments across multiple benchmarks on multimodal safety and utility demonstrate that the RAS significantly reduces attack success rates, preserves general task performance, and improves inference speed over prior inference-time defenses.

[104] MVCustom: Multi-View Customized Diffusion via Geometric Latent Rendering and Completion

Minjung Shin,Hyunin Cho,Sooyeon Go,Jin-Hwa Kim,Youngjung Uh

Main category: cs.CV

TL;DR: MVCustom 是一个基于扩散模型的新框架，旨在同时实现多视角一致性和个性化生成。通过几何潜在渲染和补全技术，它在多视角相机姿态控制和个性化方面取得了显著成果。

Details

Motivation: 现有的多视角生成模型缺乏个性化功能，而个性化模型又无法实现明确的多视角控制。MVCustom 旨在填补这一空白，实现两者的统一。

Contribution: 提出 MVCustom 框架，实现多视角一致性和个性化生成；引入几何潜在渲染和补全技术，提升几何一致性和视角对齐准确性。

Method: 训练阶段使用特征场表示学习主题的身份和几何信息，并利用密集时空注意力增强的视频扩散模型；推理阶段通过深度感知特征渲染和一致性潜在补全确保几何一致性和视角对齐。

Result: 实验证明，MVCustom 是目前唯一能同时实现高质量多视角生成和个性化的框架。

Insight: 几何潜在渲染和补全技术为多视角生成和个性化任务提供了新的思路，展示了时空注意力在多视角一致性中的重要作用。

Abstract: Multi-view generation with camera pose control and prompt-based customization are both essential elements for achieving controllable generative models. However, existing multi-view generation models do not support customization with geometric consistency, whereas customization models lack explicit viewpoint control, making them challenging to unify. Motivated by these gaps, we introduce a novel task, multi-view customization, which aims to jointly achieve multi-view camera pose control and customization. Due to the scarcity of training data in customization, existing multi-view generation models, which inherently rely on large-scale datasets, struggle to generalize to diverse prompts. To address this, we propose MVCustom, a novel diffusion-based framework explicitly designed to achieve both multi-view consistency and customization fidelity. In the training stage, MVCustom learns the subject’s identity and geometry using a feature-field representation, incorporating the text-to-video diffusion backbone enhanced with dense spatio-temporal attention, which leverages temporal coherence for multi-view consistency. In the inference stage, we introduce two novel techniques: depth-aware feature rendering explicitly enforces geometric consistency, and consistent-aware latent completion ensures accurate perspective alignment of the customized subject and surrounding backgrounds. Extensive experiments demonstrate that MVCustom is the only framework that simultaneously achieves faithful multi-view generation and customization.

[105] Cyclic Self-Supervised Diffusion for Ultra Low-field to High-field MRI Synthesis

Zhenxuan Zhang,Peiyuan Jing,Zi Wang,Ula Briski,Coraline Beitone,Yue Yang,Yinzhe Wu,Fanwen Wang,Liutao Yang,Jiahao Huang,Zhifan Gao,Zhaolin Chen,Kh Tohidul Islam,Guang Yang,Peter J. Lally

Main category: cs.CV

TL;DR: 论文提出了一种循环自监督扩散框架（CSS-Diff），用于从低场MRI合成高质量的高场MRI图像，解决了解剖保真度和对比域差距问题，并通过实验验证了其优越性能。

Details

Motivation: 低场MRI成本低且安全性高，但分辨率低且信噪比差。合成高场MRI可以减少对昂贵设备的依赖，但目前方法在临床保真度和细节保留上仍有不足。

Contribution: 提出了CSS-Diff框架，结合了循环一致性约束、切片间隙感知网络和局部结构校正网络，实现了高保真度的高场MRI合成。

Method: 1. 采用循环自监督扩散模型，强化解剖结构保真度；2. 引入切片间隙感知网络对齐切片间不一致性；3. 设计局部结构校正网络增强局部特征恢复。

Result: 实验结果表明，CSS-Diff在PSNR、SSIM和LPIPS指标上表现最优，同时显著提升了解剖结构的准确性（如左脑白质误差从12.1%降至2.1%）。

Insight: 通过结合循环一致性和自监督学习，CSS-Diff能够在不依赖像素级监督的情况下，生成既可靠又解剖一致的高场MRI图像。

Abstract: Synthesizing high-quality images from low-field MRI holds significant potential. Low-field MRI is cheaper, more accessible, and safer, but suffers from low resolution and poor signal-to-noise ratio. This synthesis process can reduce reliance on costly acquisitions and expand data availability. However, synthesizing high-field MRI still suffers from a clinical fidelity gap. There is a need to preserve anatomical fidelity, enhance fine-grained structural details, and bridge domain gaps in image contrast. To address these issues, we propose a \emph{cyclic self-supervised diffusion (CSS-Diff)} framework for high-field MRI synthesis from real low-field MRI data. Our core idea is to reformulate diffusion-based synthesis under a cycle-consistent constraint. It enforces anatomical preservation throughout the generative process rather than just relying on paired pixel-level supervision. The CSS-Diff framework further incorporates two novel processes. The slice-wise gap perception network aligns inter-slice inconsistencies via contrastive learning. The local structure correction network enhances local feature restoration through self-reconstruction of masked and perturbed patches. Extensive experiments on cross-field synthesis tasks demonstrate the effectiveness of our method, achieving state-of-the-art performance (e.g., 31.80 $\pm$ 2.70 dB in PSNR, 0.943 $\pm$ 0.102 in SSIM, and 0.0864 $\pm$ 0.0689 in LPIPS). Beyond pixel-wise fidelity, our method also preserves fine-grained anatomical structures compared with the original low-field MRI (e.g., left cerebral white matter error drops from 12.1$%$ to 2.1$%$, cortex from 4.2$%$ to 3.7$%$). To conclude, our CSS-Diff can synthesize images that are both quantitatively reliable and anatomically consistent.

[106] Multi-Scale High-Resolution Logarithmic Grapher Module for Efficient Vision GNNs

Mustafa Munir,Alex Zhang,Radu Marculescu

Main category: cs.CV

TL;DR: 论文提出了一种新的图构建方法LSGC和混合CNN-GNN模型LogViG，通过多尺度高分辨率结构提升了视觉GNN的性能。

Details

Motivation: 现有的视觉图神经网络（ViG）在构建图时（如KNN方法）计算开销大，且SVGA等方法因尺度固定可能导致信息丢失或冗余连接。需要一种更高效的图构建方法。

Contribution: 1. 提出对数可扩展图构建（LSGC）方法，优化长程连接；2. 设计LogViG模型，结合CNN和GNN的优势；3. 引入多尺度高分辨率分支，增强特征融合。

Method: 1. 使用LSGC限制长程连接数量；2. 构建LogViG，融合CNN和GNN；3. 引入高分辨率分支并与低分辨率分支特征融合。

Result: LogViG在ImageNet-1K上达到79.9%的top-1准确率，比现有ViG高1.7%，参数和计算量分别减少24.3%和35.3%。

Insight: 通过优化长程连接和多尺度特征融合，ViG的性能可以超越当前主流方法，同时减少计算开销。

Abstract: Vision graph neural networks (ViG) have demonstrated promise in vision tasks as a competitive alternative to conventional convolutional neural nets (CNN) and transformers (ViTs); however, common graph construction methods, such as k-nearest neighbor (KNN), can be expensive on larger images. While methods such as Sparse Vision Graph Attention (SVGA) have shown promise, SVGA’s fixed step scale can lead to over-squashing and missing multiple connections to gain the same information that could be gained from a long-range link. Through this observation, we propose a new graph construction method, Logarithmic Scalable Graph Construction (LSGC) to enhance performance by limiting the number of long-range links. To this end, we propose LogViG, a novel hybrid CNN-GNN model that utilizes LSGC. Furthermore, inspired by the successes of multi-scale and high-resolution architectures, we introduce and apply a high-resolution branch and fuse features between our high-resolution and low-resolution branches for a multi-scale high-resolution Vision GNN network. Extensive experiments show that LogViG beats existing ViG, CNN, and ViT architectures in terms of accuracy, GMACs, and parameters on image classification and semantic segmentation tasks. Our smallest model, Ti-LogViG, achieves an average top-1 accuracy on ImageNet-1K of 79.9% with a standard deviation of 0.2%, 1.7% higher average accuracy than Vision GNN with a 24.3% reduction in parameters and 35.3% reduction in GMACs. Our work shows that leveraging long-range links in graph construction for ViGs through our proposed LSGC can exceed the performance of current state-of-the-art ViGs. Code is available at https://github.com/mmunir127/LogViG-Official.

[107] UniCalli: A Unified Diffusion Framework for Column-Level Generation and Recognition of Chinese Calligraphy

Tianshuo Xu,Kai Wang,Zhifei Chen,Leyi Wu,Tianshui Wen,Fei Chao,Ying-Cong Chen

Main category: cs.CV

TL;DR: UniCalli是一个统一的扩散框架，用于中文书法的列级生成与识别，通过联合训练生成和识别任务，实现了高质量的书法规整性和页面布局，同时在有限数据下表现优越。

Details

Motivation: 现有方法无法同时满足高质量字符生成和页面级美学（如连笔和间距）的需求，UniCalli旨在解决这一挑战。

Contribution: 提出了UniCalli框架，联合训练生成和识别任务，通过概念级抽象提升两者性能，并扩展至其他古代文字。

Method: 采用非对称噪声和栅格化框图为空间先验，结合合成、标注和未标注数据进行训练。

Result: 在生成质量和识别性能上达到SOTA，尤其在连笔连续性和布局保真度上表现突出。

Insight: 生成与识别任务的联合训练可以相互增强，尤其在数据有限的情况下效果显著。

Abstract: Computational replication of Chinese calligraphy remains challenging. Existing methods falter, either creating high-quality isolated characters while ignoring page-level aesthetics like ligatures and spacing, or attempting page synthesis at the expense of calligraphic correctness. We introduce \textbf{UniCalli}, a unified diffusion framework for column-level recognition and generation. Training both tasks jointly is deliberate: recognition constrains the generator to preserve character structure, while generation provides style and layout priors. This synergy fosters concept-level abstractions that improve both tasks, especially in limited-data regimes. We curated a dataset of over 8,000 digitized pieces, with ~4,000 densely annotated. UniCalli employs asymmetric noising and a rasterized box map for spatial priors, trained on a mix of synthetic, labeled, and unlabeled data. The model achieves state-of-the-art generative quality with superior ligature continuity and layout fidelity, alongside stronger recognition. The framework successfully extends to other ancient scripts, including Oracle bone inscriptions and Egyptian hieroglyphs. Code and data can be viewed in \href{https://github.com/EnVision-Research/UniCalli}{this URL}.

[108] InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialogue

Wenwen Tong,Hewei Guo,Dongchuan Ran,Jiangnan Chen,Jiefan Lu,Kaibin Wang,Keqiang Li,Xiaoxu Zhu,Jiakui Li,Kehan Li,Xueheng Li,Lumin Li,Chenxu Guo,Jiasheng Zhou,Jiandong Chen,Xianye Wu,Jiahao Wang,Silei Wu,Lei Chen,Hanming Deng,Yuxuan Song,Dinghao Zhou,Guiping Zhong,Ken Zheng,Shiyin Kang,Lewei Lu

Main category: cs.CV

TL;DR: InteractiveOmni 是一个统一的开源全模态大语言模型（4B-8B参数），专注于音频-视觉多轮对话，通过多阶段训练策略和精心设计的数据集，在跨模态理解和语音生成任务中表现优异。

Details

Motivation: 为了解决轻量级全模态模型在多轮音频-视觉交互中的能力不足问题，需要一种统一的模型来支持跨模态理解和语音生成。

Contribution: 1. 提出InteractiveOmni，一个统一的音频-视觉多轮交互模型；2. 设计多阶段训练策略；3. 构建多轮记忆和语音交互评测基准。

Method: 1. 集成视觉编码器、音频编码器、大语言模型和语音解码器；2. 通过预训练和后训练阶段增强跨模态能力；3. 使用多轮训练数据集提升长对话能力。

Result: InteractiveOmni在多项任务中表现优于开源竞品，4B模型性能接近更大的Qwen2.5-Omni-7B，8B模型仅需50%参数即可保留97%性能。

Insight: 轻量级模型通过统一设计和多阶段训练可以媲美更大模型，多轮数据集对长时记忆能力至关重要。

Abstract: We introduce InteractiveOmni, a unified and open-source omni-modal large language model for audio-visual multi-turn interaction, ranging from 4B to 8B parameters, designed to lead the field of lightweight models by offering comprehensive omni-modal understanding and speech generation capabilities. To achieve this, we integrate the vision encoder, audio encoder, large language model, and speech decoder into a unified model for understanding and generation tasks. We design a multi-stage training strategy to ensure robust cross-modal capabilities, including pre-training for omni-modal understanding, followed by post-training with speech conversation and audio-visual interaction. To enable human-like long-term conversational ability, we meticulously curate a multi-turn training dataset that enhances the model’s ability to handle complex and multi-turn interactions. To effectively evaluate the multi-turn memory and speech interaction capabilities, we construct the multi-modal multi-turn memory benchmark and the multi-turn speech interaction benchmark. Experiments demonstrate that InteractiveOmni significantly outperforms leading open-source models and provides a more intelligent multi-turn audio-visual experience, particularly in its long-term memory capabilities. Notably, InteractiveOmni-4B is comparable to the much larger model like Qwen2.5-Omni-7B on general benchmarks, and it can retain 97% of the performance of the InteractiveOmni-8B while utilizing only 50% of the model size. Achieving state-of-the-art results against similarly sized models across image, audio, video understanding, and speech generation tasks, InteractiveOmni is an accessible, open-source foundation for next-generation intelligent interactive systems.

[109] RECODE: Reasoning Through Code Generation for Visual Question Answering

Junhong Shen,Mu Cai,Bo Hu,Ameet Talwalkar,David A Ross,Cordelia Schmid,Alireza Fathi

Main category: cs.CV

TL;DR: RECODE提出了一种通过生成和执行代码来解决多模态大语言模型（MLLMs）在结构化视觉（如图表和图表）推理中缺乏验证机制的问题的新方法。该方法利用反渲染技术将视觉转换为可执行代码，并通过代码重构和验证提升推理的准确性，显著优于现有方法。

Details

Motivation: 多模态大语言模型在结构化视觉推理任务中表现不佳，主要原因是像素级感知缺乏验证机制。RECODE旨在通过将视觉内容转换为可执行代码，将模糊的感知任务转化为可验证的符号问题。

Contribution: 1. 提出了RECODE框架，利用反渲染技术将视觉内容转换为可执行代码；2. 通过生成候选程序并验证重构的准确性，提升推理的可验证性；3. 在多个视觉推理基准上显著优于现有方法。

Method: 1. 生成多个候选程序以重构输入图像；2. 使用评判器选择最忠实于原图的重构；3. 迭代优化代码，将视觉推理任务转化为可验证的符号问题。

Result: 在CharXiv、ChartQA和Geometry3K等基准测试中，RECODE显著优于不利用代码或仅将代码用于辅助线或裁剪的方法。

Insight: 通过将视觉感知任务转化为可执行代码，RECODE展示了在提升多模态推理准确性和可验证性方面的潜力。

Abstract: Multimodal Large Language Models (MLLMs) struggle with precise reasoning for structured visuals like charts and diagrams, as pixel-based perception lacks a mechanism for verification. To address this, we propose to leverage derendering – the process of reverse-engineering visuals into executable code – as a new modality for verifiable visual reasoning. Specifically, we propose RECODE, an agentic framework that first generates multiple candidate programs to reproduce the input image. It then uses a critic to select the most faithful reconstruction and iteratively refines the code. This process not only transforms an ambiguous perceptual task into a verifiable, symbolic problem, but also enables precise calculations and logical inferences later on. On various visual reasoning benchmarks such as CharXiv, ChartQA, and Geometry3K, RECODE significantly outperforms methods that do not leverage code or only use code for drawing auxiliary lines or cropping. Our work demonstrates that grounding visual perception in executable code provides a new path toward more accurate and verifiable multimodal reasoning.

[110] Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmark

Kai Zou,Ziqi Huang,Yuhao Dong,Shulin Tian,Dian Zheng,Hongbo Liu,Jingwen He,Bin Liu,Yu Qiao,Ziwei Liu

Main category: cs.CV

TL;DR: Uni-MMMU 是一个多学科多模态统一基准测试，旨在评估和理解统一多模态模型中视觉理解和生成能力的双向协同作用。

Details

Motivation: 当前的多模态基准测试很少真正检验视觉理解和生成的集成能力，而是将它们孤立对待或忽略耦合任务。

Contribution: 提出 Uni-MMMU 基准测试，覆盖八个推理密集型领域，系统地评估生成和理解能力的双向协同，并提供可验证的中间推理步骤和独特的地面真实值。

Method: 设计双向耦合任务，要求模型利用概念理解指导视觉合成，或通过生成为推理提供认知支架。

Result: 对最先进的统一模型进行评估，揭示了显著的性能差距和跨模态依赖性，为统一模型的进展提供了可靠基础。

Insight: 揭示了生成和理解能力在何时以及如何相互增强的新见解，强调了双向协同在多模态任务中的重要性。

Abstract: Unified multimodal models aim to jointly enable visual understanding and generation, yet current benchmarks rarely examine their true integration. Existing evaluations either treat the two abilities in isolation or overlook tasks that inherently couple them. To address this gap, we present Uni-MMMU, a comprehensive and discipline-aware benchmark that systematically unfolds the bidirectional synergy between generation and understanding across eight reasoning-centric domains, including science, coding, mathematics, and puzzles. Each task is bidirectionally coupled, demanding models to (i) leverage conceptual understanding to guide precise visual synthesis, or (ii) utilize generation as a cognitive scaffold for analytical reasoning. Uni-MMMU incorporates verifiable intermediate reasoning steps, unique ground truths, and a reproducible scoring protocol for both textual and visual outputs. Through extensive evaluation of state-of-the-art unified, generation-only, and understanding-only models, we reveal substantial performance disparities and cross-modal dependencies, offering new insights into when and how these abilities reinforce one another, and establishing a reliable foundation for advancing unified models.

[111] Scaling Vision Transformers for Functional MRI with Flat Maps

Connor Lane,Daniel Z. Kaplan,Tanishq Mathew Abraham,Paul S. Scotti

Main category: cs.CV

TL;DR: 该论文提出了一种将4D fMRI数据转换为2D平坦图视频的方法，通过Vision Transformers和时空掩码自编码器（MAE）框架进行训练，展示了数据规模对性能的提升遵循严格的幂律规律，并在下游分类任务中验证了模型的强大表征能力。

Details

Motivation: 现代深度学习架构如何适应功能磁共振成像（fMRI）数据的输入表示是一个关键问题。通过将4D fMRI数据转换为2D平坦图视频，弥合了fMRI与自然图像之间的模态差距。

Contribution: 主要贡献包括：1）提出将4D fMRI数据转换为2D平坦图视频的方法；2）使用时空掩码自编码器（MAE）框架训练Vision Transformers；3）验证数据集规模对性能提升的幂律规律；4）在下游任务中展示模型的表征能力。

Method: 方法包括：1）将4D fMRI数据转换为2D平坦图视频；2）利用Human Connectome Project的2.3K小时fMRI数据训练Vision Transformers；3）采用时空掩码自编码器（MAE）框架进行预训练。

Result: 实验结果显示：1）掩码fMRI建模性能随数据集规模提升遵循幂律规律；2）下游分类任务中，模型能够支持细粒度状态解码和跨被试的特质解码。

Insight: 研究提供了fMRI数据处理的新思路，通过2D平坦图视频和Vision Transformers的结合，展示了大数据规模对性能提升的重要性，为fMRI基础模型的构建提供了参考。

Abstract: A key question for adapting modern deep learning architectures to functional MRI (fMRI) is how to represent the data for model input. To bridge the modality gap between fMRI and natural images, we transform the 4D volumetric fMRI data into videos of 2D fMRI activity flat maps. We train Vision Transformers on 2.3K hours of fMRI flat map videos from the Human Connectome Project using the spatiotemporal masked autoencoder (MAE) framework. We observe that masked fMRI modeling performance improves with dataset size according to a strict power scaling law. Downstream classification benchmarks show that our model learns rich representations supporting both fine-grained state decoding across subjects, as well as subject-specific trait decoding across changes in brain state. This work is part of an ongoing open science project to build foundation models for fMRI data. Our code and datasets are available at https://github.com/MedARC-AI/fmri-fm.

[112] Adaptive Visual Conditioning for Semantic Consistency in Diffusion-Based Story Continuation

Seyed Mohammad Mousavi,Morteza Analoui

Main category: cs.CV

TL;DR: 该论文提出了AVC框架，用于基于扩散模型的故事延续任务，通过自适应视觉条件确保语义一致性。

Details

Motivation: 故事延续任务需要生成的图像与文本描述和先前图像保持连贯性，但如何有效利用先前的视觉上下文并确保当前的语义对齐是一个挑战。

Contribution: 引入了AVC框架，结合CLIP模型自适应检索语义对齐的图像，并限制不相关视觉信息的影响。

Method: 使用CLIP检索相关图像，在不相关时限制早期扩散阶段的视觉条件；通过LLM重标数据集提升数据质量。

Result: 定量和人工评估表明，AVC在连贯性、语义一致性和视觉保真度上优于基线方法。

Insight: 自适应限制视觉条件可避免误导信息，同时提升语义对齐；数据集重标有助于强化文本监督。

Abstract: Story continuation focuses on generating the next image in a narrative sequence so that it remains coherent with both the ongoing text description and the previously observed images. A central challenge in this setting lies in utilizing prior visual context effectively, while ensuring semantic alignment with the current textual input. In this work, we introduce AVC (Adaptive Visual Conditioning), a framework for diffusion-based story continuation. AVC employs the CLIP model to retrieve the most semantically aligned image from previous frames. Crucially, when no sufficiently relevant image is found, AVC adaptively restricts the influence of prior visuals to only the early stages of the diffusion process. This enables the model to exploit visual context when beneficial, while avoiding the injection of misleading or irrelevant information. Furthermore, we improve data quality by re-captioning a noisy dataset using large language models, thereby strengthening textual supervision and semantic alignment. Quantitative results and human evaluations demonstrate that AVC achieves superior coherence, semantic consistency, and visual fidelity compared to strong baselines, particularly in challenging cases where prior visuals conflict with the current input.

[113] NoisePrints: Distortion-Free Watermarks for Authorship in Private Diffusion Models

Nir Goren,Oren Katzir,Abhinav Nakarmi,Eyal Ronen,Mahmood Sharif,Or Patashnik

Main category: cs.CV

TL;DR: NoisePrints提出了一种轻量级的水印方案，利用扩散过程的随机种子验证作者身份，无需修改生成过程，适用于私有扩散模型，保护版权且无需访问模型权重。

Details

Motivation: 随着扩散模型在视觉内容生成中的广泛应用，证明作者身份和保护版权变得至关重要。现有方法需要访问模型权重且计算复杂，不实用且难以扩展。因此，需要一种更轻量级、高效的解决方案。

Contribution: 1. 提出了NoisePrints，一种利用随机种子作为水印的轻量级方案；2. 展示了从内容中恢复有效种子的不可行性；3. 演示了方法的鲁棒性，并利用零知识证明保护种子隐私。

Method: 通过将哈希函数引入噪声采样过程，确保初始噪声与生成内容高度相关且难以伪造。利用零知识证明在不泄露种子的情况下验证所有权。

Result: 实验在多组先进的扩散模型上验证了NoisePrints的高效性，仅需种子和输出即可验证，无需模型权重。

Insight: 初始噪声与生成内容的关联性可被用于轻量级版权保护，零知识证明进一步增强了隐私保护，为私有模型的版权验证提供了新思路。

Abstract: With the rapid adoption of diffusion models for visual content generation, proving authorship and protecting copyright have become critical. This challenge is particularly important when model owners keep their models private and may be unwilling or unable to handle authorship issues, making third-party verification essential. A natural solution is to embed watermarks for later verification. However, existing methods require access to model weights and rely on computationally heavy procedures, rendering them impractical and non-scalable. To address these challenges, we propose , a lightweight watermarking scheme that utilizes the random seed used to initialize the diffusion process as a proof of authorship without modifying the generation process. Our key observation is that the initial noise derived from a seed is highly correlated with the generated visual content. By incorporating a hash function into the noise sampling process, we further ensure that recovering a valid seed from the content is infeasible. We also show that sampling an alternative seed that passes verification is infeasible, and demonstrate the robustness of our method under various manipulations. Finally, we show how to use cryptographic zero-knowledge proofs to prove ownership without revealing the seed. By keeping the seed secret, we increase the difficulty of watermark removal. In our experiments, we validate NoisePrints on multiple state-of-the-art diffusion models for images and videos, demonstrating efficient verification using only the seed and output, without requiring access to model weights.

[114] Bee: A High-Quality Corpus and Full-Stack Suite to Unlock Advanced Fully Open MLLMs

Yi Zhang,Bolin Ni,Xin-Sheng Chen,Heng-Rui Zhang,Yongming Rao,Houwen Peng,Qinglin Lu,Han Hu,Meng-Hao Guo,Shi-Min Hu

Main category: cs.CV

TL;DR: 该论文提出了Bee项目，旨在通过高质量数据和全栈工具推动完全开放的多模态大语言模型（MLLMs）发展。其核心贡献包括Honey-Data-15M数据集、HoneyPipe数据处理流水线，以及训练出的Bee-8B模型，实现了与半开放模型竞争的性能。

Details

Motivation: 当前完全开放的MLLMs落后于专有模型的主要原因是监督微调（SFT）数据质量不足。现有开源数据集普遍存在噪声多、缺乏复杂推理数据（如Chain-of-Thought）的问题，限制了模型能力的发展。

Contribution: 1. 发布的Honey-Data-15M数据集，包含约1500万QA对，经过多重清理并采用双级CoT增强策略；2. HoneyPipe数据处理流水线及DataStudio框架，提供透明、灵活的数据处理方法；3. 训练出的Bee-8B模型，性能可与半开放模型媲美。

Method: 1. 使用多种清理技术处理Honey-Data-15M数据集；2. 提出双级CoT（短和长）增强策略；3. HoneyPipe和DataStudio实现数据处理的透明化和可扩展性；4. 训练8B参数的Bee-8B模型验证数据效果。

Result: Bee-8B在实验中表现优异，成为完全开放MLLMs的新SOTA，性能与InternVL3.5-8B等半开放模型相当甚至超越。

Insight: 论文表明，通过系统提升数据质量，完全开放的MLLMs可以具备与半开放模型竞争的潜力。HoneyPipe和DataStudio的开源还为社区提供了灵活的数据处理工具。

Abstract: Fully open multimodal large language models (MLLMs) currently lag behind proprietary counterparts, primarily due to a significant gap in data quality for supervised fine-tuning (SFT). Existing open-source datasets are often plagued by widespread noise and a critical deficit in complex reasoning data, such as Chain-of-Thought (CoT), which hinders the development of advanced model capabilities. Addressing these challenges, our work makes three primary contributions. First, we introduce Honey-Data-15M, a new SFT dataset comprising approximately 15 million QA pairs, processed through multiple cleaning techniques and enhanced with a novel dual-level (short and long) CoT enrichment strategy. Second, we introduce HoneyPipe, the data curation pipeline, and its underlying framework DataStudio, providing the community with a transparent and adaptable methodology for data curation that moves beyond static dataset releases. Finally, to validate our dataset and pipeline, we train Bee-8B, an 8B model on Honey-Data-15M. Experiments show that Bee-8B establishes a new state-of-the-art (SOTA) for fully open MLLMs, achieving performance that is competitive with, and in some cases surpasses, recent semi-open models such as InternVL3.5-8B. Our work delivers to the community a suite of foundational resources, including: the Honey-Data-15M corpus; the full-stack suite comprising HoneyPipe and DataStudio; training recipes; an evaluation harness; and the model weights. This effort demonstrates that a principled focus on data quality is a key pathway to developing fully open MLLMs that are highly competitive with their semi-open counterparts.

[115] Generative Universal Verifier as Multimodal Meta-Reasoner

Xinchen Zhang,Xiaoying Zhang,Youbin Wu,Yanbin Cao,Renrui Zhang,Ruihang Chu,Ling Yang,Yujiu Yang

Main category: cs.CV

TL;DR: 这篇论文提出了Generative Universal Verifier（生成式通用验证器），用于多模态推理中的视觉结果验证与优化，构建了ViVerBench基准并训练了OmniVerifier-7B，还提出了OmniVerifier-TTS测试时扩展方法。

Details

Motivation: 现有视觉语言模型在多模态推理中缺乏可靠的视觉验证能力，导致其生成的视觉结果与人类能力存在显著差距。

Contribution: 1. 构建了ViVerBench基准；2. 设计了OmniVerifier-7B通用验证器；3. 提出了OmniVerifier-TTS测试时扩展方法，提升了生成和编辑能力。

Method: 1. 通过自动化流水线构建大规模视觉验证数据；2. 训练OmniVerifier-7B用于通用验证；3. 提出迭代优化的测试时扩展方法。

Result: OmniVerifier在ViVerBench上提升8.3分，OmniVerifier-TTS在T2I-ReasonBench和GenEval++上分别提升3.7和4.3分。

Insight: 视觉验证的原子能力可以协同泛化，通用验证器不仅能提升生成质量，还可扩展到更多世界建模推理场景。

Abstract: We introduce Generative Universal Verifier, a novel concept and plugin designed for next-generation multimodal reasoning in vision-language models and unified multimodal models, providing the fundamental capability of reflection and refinement on visual outcomes during the reasoning and generation process. This work makes three main contributions: (1) We build ViVerBench, a comprehensive benchmark spanning 16 categories of critical tasks for evaluating visual outcomes in multimodal reasoning. Results show that existing VLMs consistently underperform across these tasks, underscoring a substantial gap from human-level capability in reliable visual verification. (2) We design two automated pipelines to construct large-scale visual verification data and train OmniVerifier-7B, the first omni-capable generative verifier trained for universal visual verification and achieves notable gains on ViVerBench(+8.3). Through training, we identify three atomic capabilities in visual verification and demonstrate how they generalize and interact synergistically. (3) We propose OmniVerifier-TTS, a sequential test-time scaling paradigm that leverages the universal verifier to bridge image generation and editing within unified models, enhancing the upper bound of generative ability through iterative fine-grained optimization. Beyond generation, we extend universal verifier to broader world-modeling interleaved reasoning scenarios. Empirically, OmniVerifier-TTS achieves improvements on T2I-ReasonBench(+3.7), and GenEval++(+4.3), outperforming existing parallel test-time scaling methods, such as Best-of-N. By endowing multimodal reasoning with reliable visual verification, OmniVerifier advances both reliable reflection during generation and scalable test-time refinement, marking a step toward more trustworthy and controllable next-generation reasoning systems.

[116] Reasoning in Space via Grounding in the World

Yiming Chen,Zekun Qi,Wenyao Zhang,Xin Jin,Li Zhang,Peidong Liu

Main category: cs.CV

TL;DR: 论文提出了一种名为GS-Reasoner的新型3D视觉基础模型，通过双路径池化机制统一语义和几何信息，实现了无需外部模块的自回归基础，并显著提升了空间推理能力。

Details

Motivation: 现有3D大型语言模型缺乏能统一语义和几何信息的表征，导致在视觉基础任务上表现不佳或依赖外部模块，阻碍了视觉基础与空间推理的无缝结合。

Contribution: 1. 提出了GS-Reasoner，首个无需外部模块的自回归3D视觉基础模型；2. 设计了双路径池化机制，统一表征语义和几何信息；3. 发布了GCoT数据集，结合视觉基础和逐步推理。

Method: 采用双路径池化机制，紧密对齐几何特征与语义及位置信息，构建基于图像块的统一3D表征，并利用此表征实现自回归基础与空间推理的无缝结合。

Result: GS-Reasoner在3D视觉基础和空间推理任务中达到最先进性能，验证了统一表征的有效性。

Insight: 统一的3D表征是实现高效视觉基础与空间推理的关键，双路径池化机制为解决这一问题提供了简单而有效的思路。

Abstract: In this paper, we claim that 3D visual grounding is the cornerstone of spatial reasoning and introduce the Grounded-Spatial Reasoner (GS-Reasoner) to explore the effective spatial representations that bridge the gap between them. Existing 3D LLMs suffer from the absence of a unified 3D representation capable of jointly capturing semantic and geometric information. This deficiency is manifested either in poor performance on grounding or in an excessive reliance on external modules, ultimately hindering the seamless integration of grounding and spatial reasoning. To address this, we propose a simple yet effective dual-path pooling mechanism that tightly aligns geometric features with both semantic and positional cues, constructing a unified image patch-based 3D representation that encapsulates all essential information without increasing the number of input tokens. Leveraging this holistic representation, GS-Reasoner is the first 3D LLM that achieves autoregressive grounding entirely without external modules while delivering performance comparable to state-of-the-art models, establishing a unified and self-contained framework for 3D spatial reasoning. To further bridge grounding and spatial reasoning, we introduce the Grounded Chain-of-Thought (GCoT) dataset. This dataset is meticulously curated to include both 3D bounding box annotations for objects referenced in reasoning questions and step-by-step reasoning paths that integrate grounding as a core component of the problem-solving process. Extensive experiments demonstrate that GS-Reasoner achieves impressive results on 3D visual grounding, which in turn significantly enhances its spatial reasoning capabilities, leading to state-of-the-art performance.

[117] Trace Anything: Representing Any Video in 4D via Trajectory Fields

Xinhang Liu,Yuxi Xiao,Donny Y. Chen,Jiashi Feng,Yu-Wing Tai,Chi-Keung Tang,Bingyi Kang

Main category: cs.CV

TL;DR: Trace Anything 提出了一种新的视频表示方法——轨迹场（Trajectory Field），通过预测像素的连续3D轨迹函数，单次前向传播即可生成全视频的动态表示，并在多个任务中展现出高效性和表现力。

Details

Motivation: 视频的有效时空表示对动态建模和理解至关重要。传统方法通常需要复杂的迭代优化，难以高效捕捉像素的连续运动轨迹。

Contribution: 1. 提出了轨迹场的概念，将视频表示为稠密的3D轨迹函数；2. 开发了Trace Anything模型，单次前向预测即可生成轨迹场；3. 展示了模型在多项任务中的高效性和涌现能力。

Method: 模型通过预测每个像素的控制点参数化B样条轨迹，支持任意时间点的3D位置查询。训练基于大规模4D数据，包括新平台的数据。

Result: 1. 在新提出的轨迹场估计基准上达到SOTA；2. 在经典点跟踪任务中表现优异；3. 展现出高效性和涌现能力（如目标条件操作、运动预测）。

Insight: 轨迹场提供了一种高效且统一的视频动态表示方法，支持多项任务的直接应用，无需额外优化或辅助模块。

Abstract: Effective spatio-temporal representation is fundamental to modeling, understanding, and predicting dynamics in videos. The atomic unit of a video, the pixel, traces a continuous 3D trajectory over time, serving as the primitive element of dynamics. Based on this principle, we propose representing any video as a Trajectory Field: a dense mapping that assigns a continuous 3D trajectory function of time to each pixel in every frame. With this representation, we introduce Trace Anything, a neural network that predicts the entire trajectory field in a single feed-forward pass. Specifically, for each pixel in each frame, our model predicts a set of control points that parameterizes a trajectory (i.e., a B-spline), yielding its 3D position at arbitrary query time instants. We trained the Trace Anything model on large-scale 4D data, including data from our new platform, and our experiments demonstrate that: (i) Trace Anything achieves state-of-the-art performance on our new benchmark for trajectory field estimation and performs competitively on established point-tracking benchmarks; (ii) it offers significant efficiency gains thanks to its one-pass paradigm, without requiring iterative optimization or auxiliary estimators; and (iii) it exhibits emergent abilities, including goal-conditioned manipulation, motion forecasting, and spatio-temporal fusion. Project page: https://trace-anything.github.io/.

[118] VisCoP: Visual Probing for Video Domain Adaptation of Vision Language Models

Dominick Reilly,Manish Kumar Govind,Le Xue,Srijan Das

Main category: cs.CV

TL;DR: VisCoP提出了一种新的视觉语言模型（VLM）领域自适应方法，通过可学习的视觉探针增强视觉编码器，避免了领域特定特征学习的局限性或灾难性遗忘问题。在多个挑战性领域自适应任务中，VisCoP表现优于现有方法。

Details

Motivation: 大规模视觉语言模型在新领域表现下降，现有领域自适应方法可能导致特征学习有限或灾难性遗忘。VisCoP旨在通过最小化预训练参数修改，高效实现领域自适应。

Contribution: 提出了VisCoP方法，通过在视觉编码器中引入可学习的视觉探针，实现了高效的领域自适应，并保留了源域知识。

Method: VisCoP通过在VLM的视觉编码器中添加少量可学习的视觉探针，实现领域特定的自适应，同时避免大规模参数调整。

Result: VisCoP在跨视角、跨模态和跨任务三种领域自适应任务中均取得优于现有方法的表现。

Insight: VisCoP展示了视觉探针的潜力，通过轻量化修改即可实现领域自适应，同时保留了模型的通用能力。

Abstract: Large Vision-Language Models (VLMs) excel at general visual reasoning tasks but exhibit sharp performance degradation when applied to novel domains with substantial distribution shifts from pretraining data. Existing domain adaptation approaches finetune different VLM components, but this often results in limited domain-specific feature learning or catastrophic forgetting of prior capabilities. To address these issues, we introduce Vision Contextualized Probing (VisCoP), which augments the VLM’s vision encoder with a compact set of learnable visual probes. These probes enable efficient domain-specific adaptation with minimal modification to pretrained parameters. We evaluate VisCoP across three challenging domain adaptation settings-cross-view (exocentric to egocentric), cross-modal (RGB to depth), and cross-task (human understanding to robot control). Experiments show that VisCoP consistently outperforms existing adaptation strategies, achieving superior performance on target domains while effectively retaining source-domain knowledge.

[119] PhysMaster: Mastering Physical Representation for Video Generation via Reinforcement Learning

Sihui Ji,Xi Chen,Xin Tao,Pengfei Wan,Hengshuang Zhao

Main category: cs.CV

TL;DR: PhysMaster提出了一种通过强化学习优化视频生成模型的物理表示的方法，旨在生成更符合物理规律的视频。

Details

Motivation: 目前的视频生成模型在视觉上可以生成逼真的视频，但往往无法遵循物理规律，限制了其作为'世界模型'的能力。因此，需要一种方法增强模型的物理感知能力。

Contribution: PhysMaster的核心贡献是通过PhysEncoder从输入图像中编码物理信息，并结合强化学习和人类反馈优化物理表示，从而提升视频生成的物理合理性。

Method: PhysMaster基于图像到视频任务，设计PhysEncoder从输入图像中提取物理信息（如物体相对位置和潜在交互）。使用强化学习和Direct Preference Optimization（DPO）优化物理表示的学习过程。

Result: PhysMaster在简单代理任务上展现了其物理感知能力，并能泛化到广泛的物理场景中，证明了其作为通用插件的可行性。

Insight: PhysMaster通过表示学习和强化学习的结合，为视频生成中的物理规律建模提供了一种通用解决方案，未来可扩展到更广泛的应用中。

Abstract: Video generation models nowadays are capable of generating visually realistic videos, but often fail to adhere to physical laws, limiting their ability to generate physically plausible videos and serve as ‘’world models’’. To address this issue, we propose PhysMaster, which captures physical knowledge as a representation for guiding video generation models to enhance their physics-awareness. Specifically, PhysMaster is based on the image-to-video task where the model is expected to predict physically plausible dynamics from the input image. Since the input image provides physical priors like relative positions and potential interactions of objects in the scenario, we devise PhysEncoder to encode physical information from it as an extra condition to inject physical knowledge into the video generation process. The lack of proper supervision on the model’s physical performance beyond mere appearance motivates PhysEncoder to apply reinforcement learning with human feedback to physical representation learning, which leverages feedback from generation models to optimize physical representations with Direct Preference Optimization (DPO) in an end-to-end manner. PhysMaster provides a feasible solution for improving physics-awareness of PhysEncoder and thus of video generation, proving its ability on a simple proxy task and generalizability to wide-ranging physical scenarios. This implies that our PhysMaster, which unifies solutions for various physical processes via representation learning in the reinforcement learning paradigm, can act as a generic and plug-in solution for physics-aware video generation and broader applications.

cs.CE [Back]

[120] Program of Thoughts for Financial Reasoning: Leveraging Dynamic In-Context Examples and Generative Retrieval

Subhendu Khatuya,Shashwat Naidu,Pawan Goyal,Niloy Ganguly

Main category: cs.CE

TL;DR: FINDER是一个新颖的两步框架，旨在提升大型语言模型(LLM)在金融数值推理任务中的能力，通过动态上下文示例和生成式检索技术，显著提升了FinQA和ConvFinQA数据集的性能。

Details

Motivation: 尽管LLM在多项任务中表现出色，但在金融数值推理任务中仍落后于最先进模型，因此需要开发更高效的方法。

Contribution: 提出了FINDER框架，结合生成式检索和动态上下文示例选择的技术，显著提升了LLM在FinQA和ConvFinQA数据集上的性能。

Method: FINDER分为两步：1)使用生成式检索器从非结构化数据中提取相关事实；2)采用上下文感知的程序化思维提示技术，动态选择上下文示例。

Result: FINDER在FinQA和ConvFinQA数据集上分别实现了执行准确率提升5.98%和4.05%，达到了新的SOTA性能。

Insight: 动态选择和生成式检索的结合可以有效提升LLM在复杂金融推理任务中的表现，为后续研究提供了新思路。

Abstract: Despite continuous advancements in the capabilities of large language models (LLMs), numerical reasoning remains a challenging area. Techniques like chain-of-thought prompting, tree-of-thought prompting, and program-of-thought prompting guide LLMs through intermediate reasoning steps. Although in-context learning with few-shot prompting has improved performance, LLMs still lag behind state-of-the-art models on financial numerical reasoning datasets such as FinQA and ConvFinQA. In this work, we introduce FINDER, a novel two-step framework, to enhance LLMs’ capabilities in financial numerical reasoning. The first step utilizes a generative retriever to extract relevant facts from unstructured data, including both text and tables. This is followed by context-aware Program of Thought prompting with dynamic selection of in-context examples. Our model FINDER achieves a new state-of-the-art performance on both the FinQA and ConvFinQA datasets, surpassing previous benchmarks with execution accuracy improvements of 5.98% and 4.05%, respectively.

cs.IR [Back]

[121] Improving Visual Recommendation on E-commerce Platforms Using Vision-Language Models

Yuki Yada,Sho Akiyama,Ryo Watanabe,Yuta Ueno,Yusuke Shido,Andre Rusli

Main category: cs.IR

TL;DR: 该论文提出了一种基于视觉-语言模型（VLM）的方法，用于提升电商平台的商品推荐效果。通过对SigLIP模型进行微调，并结合商品图像和标题数据，实验结果表明该方法在离线评估和在线测试中均显著优于基线模型。

Details

Motivation: 在大型电商平台上，用户需要高效地发现符合其偏好的商品。传统推荐系统可能难以捕捉视觉相似性，因此论文探索了VLM在商品推荐中的应用潜力。

Contribution: 1. 将视觉-语言模型（SigLIP）用于商品推荐任务；2. 通过微调和实践验证了该方法在电商平台上的有效性；3. 提供了离线和在线测试的详细结果。

Method: 1. 使用百万级商品图像-标题对微调SigLIP模型；2. 开发图像编码器生成商品嵌入；3. 通过离线日志分析和在线A/B测试评估模型性能。

Result: 离线nDCG@5提升9.1%，在线测试中点击率提升50%，转化率提升14%。

Insight: 视觉-语言模型可以有效捕捉商品间的视觉相似性，显著提升推荐系统性能，尤其在电商场景中表现突出。

Abstract: On large-scale e-commerce platforms with tens of millions of active monthly users, recommending visually similar products is essential for enabling users to efficiently discover items that align with their preferences. This study presents the application of a vision-language model (VLM) – which has demonstrated strong performance in image recognition and image-text retrieval tasks – to product recommendations on Mercari, a major consumer-to-consumer marketplace used by more than 20 million monthly users in Japan. Specifically, we fine-tuned SigLIP, a VLM employing a sigmoid-based contrastive loss, using one million product image-title pairs from Mercari collected over a three-month period, and developed an image encoder for generating item embeddings used in the recommendation system. Our evaluation comprised an offline analysis of historical interaction logs and an online A/B test in a production environment. In offline analysis, the model achieved a 9.1% improvement in nDCG@5 compared with the baseline. In the online A/B test, the click-through rate improved by 50% whereas the conversion rate improved by 14% compared with the existing model. These results demonstrate the effectiveness of VLM-based encoders for e-commerce product recommendations and provide practical insights into the development of visual similarity-based recommendation systems.

eess.IV [Back]

[122] Dedelayed: Deleting remote inference delay via on-device correction

Dan Jacobellis,Mateen Ulhaq,Fabien Racapé,Hyomin Choi,Neeraja J. Yadwadkar

Main category: eess.IV

TL;DR: Dedelayed是一种通过本地设备校正远程推理延迟的方法，能够在实时任务中减少延迟并保持高精度。

Details

Motivation: 远程推理虽然可以利用云端强大模型，但网络延迟会导致预测过时，不适合实时任务。Dedelayed旨在解决这一问题。

Contribution: 提出Dedelayed方法，通过结合本地轻量级模型和远程模型的过去帧特征，显著提高实时任务的精度。

Method: 使用轻量级本地模型处理当前帧，并融合远程模型对过去帧的特征，从而校正延迟。

Result: 在BDD100K数据集上，Dedelayed在所有超过33ms的延迟情况下优于纯本地或纯远程基线方法，尤其在100ms延迟下提升6.4 mIoU（相对于本地）和9.8 mIoU（相对于远程）。

Insight: 在高延迟和高动态场景中，Dedelayed的优势更加明显，适用于需要与当前世界状态保持一致的实时任务。

Abstract: Remote inference allows lightweight devices to leverage powerful cloud models. However, communication network latency makes predictions stale and unsuitable for real-time tasks. To address this, we introduce Dedelayed, a delay-corrective method that mitigates arbitrary remote inference delays, allowing the local device to produce low-latency outputs in real time. Our method employs a lightweight local model that processes the current frame and fuses in features that a heavyweight remote model computes from past frames. On video from the BDD100K driving dataset, Dedelayed improves semantic segmentation accuracy over the stronger of the local-only and remote-only baselines across all realistic communication network delays beyond 33 ms. Without incurring additional delay, it improves accuracy by 6.4 mIoU compared to fully local inference and 9.8 mIoU compared to remote inference, for a round-trip delay of 100 ms. The advantage grows under longer delays and higher-motion scenes, as delay-mitigated split inference sustains accuracy more effectively, providing clear advantages for real-time tasks that must remain aligned with the current world state.

cs.AI [Back]

[123] From Literal to Liberal: A Meta-Prompting Framework for Eliciting Human-Aligned Exception Handling in Large Language Models

Imran Khan

Main category: cs.AI

TL;DR: 论文提出了Rule-Intent Distinction (RID)框架，一种零样本的元提示技术，旨在解决LLMs在规则遵循上的僵化问题，使其更符合人类常识和意图，显著提升了模型的表现。

Details

Motivation: LLMs在处理显式规则时表现僵化，导致决策与人类常识和意图不符。现有的监督微调方法成本高昂且不易实现，需要一种低成本、高效的替代方案。

Contribution: 提出了RID框架，一种零样本的meta-prompting技术，能够在不依赖监督微调的情况下提升LLMs对人类意图的理解和例外处理能力。

Method: RID框架通过结构化认知模式解构任务、分类规则、权衡冲突结果并最终决策。相比基线和思维链(CoT)提示，RID表现更优。

Result: 在20个多领域场景的基准测试中，RID框架达到95%的人类对齐得分(HAS)，显著优于基线(80%)和CoT(75%)。

Insight: RID框架为LLMs提供了一种从字面指令遵循转向目标导向推理的实用方法，为更可靠和实用的AI代理奠定了基础。

Abstract: Large Language Models (LLMs) are increasingly being deployed as the reasoning engines for agentic AI systems, yet they exhibit a critical flaw: a rigid adherence to explicit rules that leads to decisions misaligned with human common sense and intent. This “rule-rigidity” is a significant barrier to building trustworthy autonomous agents. While prior work has shown that supervised fine-tuning (SFT) with human explanations can mitigate this issue, SFT is computationally expensive and inaccessible to many practitioners. To address this gap, we introduce the Rule-Intent Distinction (RID) Framework, a novel, low-compute meta-prompting technique designed to elicit human-aligned exception handling in LLMs in a zero-shot manner. The RID framework provides the model with a structured cognitive schema for deconstructing tasks, classifying rules, weighing conflicting outcomes, and justifying its final decision. We evaluated the RID framework against baseline and Chain-of-Thought (CoT) prompting on a custom benchmark of 20 scenarios requiring nuanced judgment across diverse domains. Our human-verified results demonstrate that the RID framework significantly improves performance, achieving a 95% Human Alignment Score (HAS), compared to 80% for the baseline and 75% for CoT. Furthermore, it consistently produces higher-quality, intent-driven reasoning. This work presents a practical, accessible, and effective method for steering LLMs from literal instruction-following to liberal, goal-oriented reasoning, paving the way for more reliable and pragmatic AI agents.

[124] DeepPlanner: Scaling Planning Capability for Deep Research Agents via Advantage Shaping

Wei Fan,Wenlin Yao,Zheng Li,Feng Yao,Xin Liu,Liang Qiu,Qingyu Yin,Yangqiu Song,Bing Yin

Main category: cs.AI

TL;DR: DeepPlanner 是一个端到端的强化学习框架，通过优势塑形提升深度研究代理的规划能力，在规划和动作生成阶段优化效果显著。

Details

Motivation: 当前基于大语言模型（LLMs）的方法在执行复杂任务时，规划的隐含性或显式性不足，导致决策点熵高且未充分优化。

Contribution: 提出了 DeepPlanner，通过熵优势塑形方法优化规划阶段的更新，并在七个基准测试中实现了最先进的结果且训练成本更低。

Method: 结合熵的优势塑形，对高熵规划标记分配更大的更新权重，并选择性加权规划密集型样本的优势。

Result: 在七个深度研究基准测试中，DeepPlanner 提升了规划质量，并以更低的训练成本取得了最优性能。

Insight: 通过熵塑形优化规划阶段的强化学习信号，能够显著提高深度研究代理的规划能力。

Abstract: Large language models (LLMs) augmented with multi-step reasoning and action generation abilities have shown promise in leveraging external tools to tackle complex tasks that require long-horizon planning. However, existing approaches either rely on implicit planning in the reasoning stage or introduce explicit planners without systematically addressing how to optimize the planning stage. As evidence, we observe that under vanilla reinforcement learning (RL), planning tokens exhibit significantly higher entropy than other action tokens, revealing uncertain decision points that remain under-optimized. To address this, we propose DeepPlanner, an end-to-end RL framework that effectively enhances the planning capabilities of deep research agents. Our approach shapes token-level advantage with an entropy-based term to allocate larger updates to high entropy tokens, and selectively upweights sample-level advantages for planning-intensive rollouts. Extensive experiments across seven deep research benchmarks demonstrate that DeepPlanner improves planning quality and achieves state-of-the-art results under a substantially lower training budget.

[125] Personalized Learning Path Planning with Goal-Driven Learner State Modeling

Joy Jia Yin Lim,Ye He,Jifan Yu,Xin Cong,Daniel Zhang-Li,Zhiyuan Liu,Huiqin Liu,Lei Hou,Juanzi Li,Bin Xu

Main category: cs.AI

TL;DR: 该论文提出了一种名为Pxplore的个性化学习路径规划框架，整合了强化学习范式和大语言模型驱动的教育架构，通过结构化学习者状态模型和自动奖励函数，实现了目标对齐的学习路径生成。

Details

Motivation: 传统的个性化学习方法缺乏目标对齐机制，难以生成符合个体目标的学习路径，因此需要一种新的框架来填补这一空白。

Contribution: 论文的主要贡献是提出了Pxplore框架，设计了一个结构化学习者状态模型和自动奖励函数，并结合了监督微调和GRPO训练方法，实现了目标驱动的学习路径规划。

Method: Pxplore采用强化学习范式和大语言模型驱动的教育架构，设计了结构化学习者状态模型和自动奖励函数，并通过监督微调和GRPO训练策略。

Result: 实验验证了Pxplore在生成连贯、个性化和目标驱动的学习路径方面的有效性。

Insight: 论文展示了如何将强化学习与大语言模型结合，以解决目标对齐的学习路径规划问题，为个性化教育领域提供了新的思路。

Abstract: Personalized Learning Path Planning (PLPP) aims to design adaptive learning paths that align with individual goals. While large language models (LLMs) show potential in personalizing learning experiences, existing approaches often lack mechanisms for goal-aligned planning. We introduce Pxplore, a novel framework for PLPP that integrates a reinforcement-based training paradigm and an LLM-driven educational architecture. We design a structured learner state model and an automated reward function that transforms abstract objectives into computable signals. We train the policy combining supervised fine-tuning (SFT) and Group Relative Policy Optimization (GRPO), and deploy it within a real-world learning platform. Extensive experiments validate Pxplore’s effectiveness in producing coherent, personalized, and goal-driven learning paths. We release our code and dataset to facilitate future research.

[126] EvoTest: Evolutionary Test-Time Learning for Self-Improving Agentic Systems

Yufei He,Juncheng Liu,Yue Liu,Yibo Li,Tri Cao,Zhiyuan Hu,Xinxing Xu,Bryan Hooi

Main category: cs.AI

TL;DR: 该论文提出了一个名为EvoTest的进化测试时间学习框架，用于提升AI代理在新环境中的即时学习能力，并通过J-TTL基准测试证明了其有效性。

Details

Motivation: 现有AI代理在新环境中缺乏即时学习能力，限制了实际应用效果，因此需要一种无需梯度微调的改进方法。

Contribution: 提出了J-TTL基准测试来衡量代理的动态学习能力，并开发了EvoTest框架，通过进化方式优化代理配置。

Method: EvoTest包含Actor Agent和Evolver Agent两个角色，前者执行任务，后者分析任务记录并提出改进配置。

Result: 在J-TTL基准测试中，EvoTest显著优于现有方法，是唯一能赢得两场游戏的框架。

Insight: 进化式学习方法能够在无需梯度更新的情况下显著提升代理的动态适应能力。

Abstract: A fundamental limitation of current AI agents is their inability to learn complex skills on the fly at test time, often behaving like “clever but clueless interns” in novel environments. This severely limits their practical utility. To systematically measure and drive progress on this challenge, we first introduce the Jericho Test-Time Learning (J-TTL) benchmark. J-TTL is a new evaluation setup where an agent must play the same game for several consecutive episodes, attempting to improve its performance from one episode to the next. On J-TTL, we find that existing adaptation methods like reflection, memory, or reinforcement learning struggle. To address the challenges posed by our benchmark, we present EvoTest, an evolutionary test-time learning framework that improves an agent without any fine-tuning or gradients-by evolving the entire agentic system after every episode. EvoTest has two roles: the Actor Agent, which plays the game, and the Evolver Agent, which analyzes the episode transcript to propose a revised configuration for the next run. This configuration rewrites the prompt, updates memory by logging effective state-action choices, tunes hyperparameters, and learns the tool-use routines. On our J-TTL benchmark, EvoTest consistently increases performance, outperforming not only reflection and memory-only baselines but also more complex online fine-tuning methods. Notably, our method is the only one capable of winning two games (Detective and Library), while all baselines fail to win any.

[127] Assessing LLM Reasoning Through Implicit Causal Chain Discovery in Climate Discourse

Liesbeth Allein,Nataly Pineda-Castañeda,Andrea Rocci,Marie-Francine Moens

Main category: cs.AI

TL;DR: 该论文通过评估大型语言模型（LLMs）在气候话语中生成隐式因果链的能力，揭示了LLMs在因果推理中的局限性及其模式匹配主导的特点，同时验证了其逻辑一致性。

Details

Motivation: 研究旨在探索LLMs是否能通过生成中间因果步骤来展示其因果推理能力，特别是在气候变化的争议性话语中。

Contribution: 提出了一个诊断评估框架，用于测试LLMs的因果推理能力；生成了一个基准数据集；并通过人类验证证实了LLMs生成的因果链的逻辑一致性。

Method: 使用了九个LLMs生成给定因果对之间的所有可能中间步骤，并对其一致性、置信度和推理模式进行了分析。

Result: LLMs生成的因果链数量和粒度不一，主要依赖关联模式匹配而非真正的因果推理，但其逻辑一致性得到了人类评估的验证。

Insight: LLMs在因果推理任务中表现有限，但其生成的逻辑一致的因果链为未来研究提供了基础。

Abstract: How does a cause lead to an effect, and which intermediate causal steps explain their connection? This work scrutinizes the mechanistic causal reasoning capabilities of large language models (LLMs) to answer these questions through the task of implicit causal chain discovery. In a diagnostic evaluation framework, we instruct nine LLMs to generate all possible intermediate causal steps linking given cause-effect pairs in causal chain structures. These pairs are drawn from recent resources in argumentation studies featuring polarized discussion on climate change. Our analysis reveals that LLMs vary in the number and granularity of causal steps they produce. Although they are generally self-consistent and confident about the intermediate causal connections in the generated chains, their judgments are mainly driven by associative pattern matching rather than genuine causal reasoning. Nonetheless, human evaluations confirmed the logical coherence and integrity of the generated chains. Our baseline causal chain discovery approach, insights from our diagnostic evaluation, and benchmark dataset with causal chains lay a solid foundation for advancing future work in implicit, mechanistic causal reasoning in argumentation settings.

[128] Hard2Verify: A Step-Level Verification Benchmark for Open-Ended Frontier Math

Shrey Pandit,Austin Xu,Xuan-Phi Nguyen,Yifei Ming,Caiming Xiong,Shafiq Joty

Main category: cs.AI

TL;DR: Hard2Verify是一个针对前沿开放数学问题的步骤级验证基准数据集，用于评估大语言模型（LLM）在生成和验证数学证明步骤时的能力。

Details

Motivation: 现有的LLM在数学推理任务中表现出色，但缺乏对步骤级错误的精准验证能力，尤其是在开放性问题中。因此，需要一个严格的基准数据集来评估和改进验证器的性能。

Contribution: 提出了Hard2Verify数据集，包含500多小时的人工标注，专注于评估前沿LLM在开放数学问题中的步骤级验证能力。

Method: 通过人工标注生成数据集，评估29种开源和闭源的生成式批评模型和处理奖励模型，分析验证器在步骤级任务中的表现和本质挑战。

Result: 研究发现，大多数开源验证器表现不佳，闭源模型表现较好，同时揭示了验证器计算扩展的影响以及自验证和验证-生成动态关系的本质问题。

Insight: 严格的步骤级验证是提升LLM数学推理能力的关键，同时开源验证器的性能差距表明需要进一步研究和改进的领域。

Abstract: Large language model (LLM)-based reasoning systems have recently achieved gold medal-level performance in the IMO 2025 competition, writing mathematical proofs where, to receive full credit, each step must be not only correct but also sufficiently supported. To train LLM-based reasoners in such challenging, open-ended settings, strong verifiers capable of catching step-level mistakes are necessary prerequisites. We introduce Hard2Verify, a human-annotated, step-level verification benchmark produced with over 500 hours of human labor. Hard2Verify is designed to rigorously assess step-level verifiers at the frontier: Verifiers must provide step-level annotations or identify the first error in responses generated by frontier LLMs for very recent, challenging, and open-ended math questions. We evaluate 29 generative critics and process reward models, demonstrating that, beyond a few standouts, open-source verifiers lag closed source models. We subsequently analyze what drives poor performance in step-level verification, the impacts of scaling verifier compute, as well as fundamental questions such as self-verification and verification-generation dynamics.

cs.LG [Back]

[129] Max It or Miss It: Benchmarking LLM On Solving Extremal Problems

Binxin Gao,Jingjun Han

Main category: cs.LG

TL;DR: 论文提出了一个名为ExtremBench的基准数据集，用于评估大型语言模型（LLM）在解决极值问题上的能力，发现当前模型的数学推理能力与极值问题解决能力之间存在不一致性。

Details

Motivation: 尽管LLM在数学推理方面表现优异，但其优化推理（如极值问题求解）的具体能力来源和机制尚不清晰。极值问题是规划、控制和资源分配等应用的基础，因此需要系统性评估。

Contribution: 1. 提出了ExtremBench基准数据集，包含93个标准化极值问题；2. 揭示了LLM在极值问题与一般数学推理能力之间的不一致性；3. 指出了现有评估方法的局限性。

Method: 1. 从中国数学奥林匹克竞赛题目中提取不等式问题并转换为标准极值问题；2. 对多个开源LLM（如Qwen3、GPT-OSS、DeepSeek）进行评估；3. 比较模型在ExtremBench与AIME25、MATH-500等基准上的表现。

Result: 研究发现，部分模型在一般数学推理上表现优异，但在极值问题上表现不佳，反之亦然，表明现有基准未能全面评估数学推理能力。

Insight: 极值问题求解能力可能与一般数学推理能力不同，需开发更全面的评估方法，以避免对模型能力的误判。

Abstract: Test-time scaling has enabled Large Language Models (LLMs) with remarkable reasoning capabilities, particularly in mathematical domains, through intermediate chain-of-thought (CoT) reasoning before generating final answers. However, the specific sources and mechanisms underlying these reasoning capabilities remain insufficiently understood. Optimization reasoning, i.e. finding extrema under constraints, represents a fundamental abstraction that underpins critical applications in planning, control, resource allocation, and prompt search. To systematically evaluate this capability, we introduce ExtremBench, a benchmark dataset for solving mathematical extremal problems, curated from inequality exercises used for Chinese Mathematical Olympiad and transformed into $93$ standardized extrema-finding problems. We conduct extensive evaluations across various state-of-the-art open-source model families, including the Qwen3, GPT-OSS, and DeepSeek. Our results reveal that LLMs’ extremal-solving reasoning capabilities do not always align with those of current mathematical benchmarks such as AIME25 and MATH-500, with some models showing strong general mathematical reasoning but poor extremal-solving skills, and vice versa. This discrepancy highlights a critical gap in current evaluation practices and suggests that existing benchmarks may not comprehensively capture the full spectrum of mathematical reasoning abilities.

[130] On the Reasoning Abilities of Masked Diffusion Language Models

Anej Svete,Ashish Sabharwal

Main category: cs.LG

TL;DR: 该论文探讨了掩码扩散模型（MDMs）在文本生成中的推理能力，证明了其在某些问题上的高效性，并与链式思维（CoT）和循环变换器（PLTs）进行了理论对比。

Details

Motivation: 研究掩码扩散模型（MDMs）的计算能力和并行生成的优势，现有研究对其推理能力和效率限制了解不足。

Contribution: 1. 揭示了MDMs与循环变换器（PLTs）在有限精度对数宽度设置下的等价性；2. 证明MDMs能解决所有CoT增强变换器可解决的问题；3. 展示了MDMs在某些问题（如正则语言）上比CoT变换器更高效。

Method: 将MDMs与链式思维（CoT）和循环变换器（PLTs）的理论框架连接，分析其在有限精度对数宽度设置下的计算等价性和效率。

Result: MDMs在某些推理问题中优于CoT变换器，并行生成为其提供了显著的效率优势。

Insight: 并行生成的MDMs在特定任务中具有潜力，但其局限性仍需进一步探索。

Abstract: Masked diffusion models (MDMs) for text offer a compelling alternative to traditional autoregressive language models. Parallel generation makes them efficient, but their computational capabilities and the limitations inherent to their parallelism remain largely unexplored. To this end, we characterize what types of reasoning problems MDMs can provably solve and how efficiently. We do this by connecting MDMs to the well-understood reasoning frameworks of chain of thought (CoT) and padded looped transformers (PLTs) in the finite-precision log-width setting: We show that MDMs and polynomially-padded PLTs are, in fact, equivalent in this setting, and that MDMs can solve all problems that CoT-augmented transformers can. Moreover, we showcase classes of problems (including regular languages) for which MDMs are inherently more efficient than CoT transformers, where parallel generation allows for substantially faster reasoning.

[131] UrbanFusion: Stochastic Multimodal Fusion for Contrastive Learning of Robust Spatial Representations

Dominik J. Mühlematter,Lin Che,Ye Hong,Martin Raubal,Nina Wiedemann

Main category: cs.LG

TL;DR: UrbanFusion 是一种地理基础模型（GeoFM），通过随机多模态融合（SMF）技术整合多种地理空间数据，提升了空间表示的学习能力，并在全球多项任务中展示了优异的泛化性能和预测效果。

Details

Motivation: 现有方法多为任务特异性模型，地理基础模型在多模态融合能力上受限，UrbanFusion 旨在解决这一问题。

Contribution: 提出了 UrbanFusion，支持多模态输入的统一表示学习，并实现了在训练和推理中灵活利用任意子集的多模态数据。

Method: 采用模态特定编码器处理输入数据，通过 Transformer 融合模块学习统一表示。

Result: 在全球 41 项任务中的表现优于现有 GeoAI 模型，泛化能力强。

Insight: 多模态融合能显著提升地理空间任务的性能，灵活的数据利用增强了模型的适用性。

Abstract: Forecasting urban phenomena such as housing prices and public health indicators requires the effective integration of various geospatial data. Current methods primarily utilize task-specific models, while recent foundation models for spatial representations often support only limited modalities and lack multimodal fusion capabilities. To overcome these challenges, we present UrbanFusion, a Geo-Foundation Model (GeoFM) that features Stochastic Multimodal Fusion (SMF). The framework employs modality-specific encoders to process different types of inputs, including street view imagery, remote sensing data, cartographic maps, and points of interest (POIs) data. These multimodal inputs are integrated via a Transformer-based fusion module that learns unified representations. An extensive evaluation across 41 tasks in 56 cities worldwide demonstrates UrbanFusion’s strong generalization and predictive performance compared to state-of-the-art GeoAI models. Specifically, it 1) outperforms prior foundation models on location-encoding, 2) allows multimodal input during inference, and 3) generalizes well to regions unseen during training. UrbanFusion can flexibly utilize any subset of available modalities for a given location during both pretraining and inference, enabling broad applicability across diverse data availability scenarios. All source code is available at https://github.com/DominikM198/UrbanFusion.

cs.SE [Back]

[132] TRUSTVIS: A Multi-Dimensional Trustworthiness Evaluation Framework for Large Language Models

Ruoyu Sun,Da Song,Jiayang Song,Yuheng Huang,Lei Ma

Main category: cs.SE

TL;DR: TRUSTVIS是一个多维度可信度评估框架，旨在自动化评估大语言模型（LLMs）的可信度，重点关注安全性和鲁棒性，并提供直观的可视化界面。

Details

Motivation: 随着LLMs在自然语言处理中的广泛应用，其可信度问题（如安全性和鲁棒性）引发了广泛关注，需要一个全面的评估工具。

Contribution: 提出了TRUSTVIS框架，集成了多种评估方法和交互式可视化，为大语言模型的可信度提供了自动化、可靠和直观的评估工具。

Method: 结合AutoDAN等扰动方法和多数投票机制，TRUSTVIS设计了交互式用户界面，直观展示可信度指标。

Result: 在Vicuna-7b、Llama2-7b和GPT-3.5等模型上的初步案例研究表明，TRUSTVIS能有效识别安全性和鲁棒性漏洞。

Insight: 通过交互式界面，用户可以深入探索评估结果，有针对性地改进模型，增强了大语言模型的可信度和可用性。

Abstract: As Large Language Models (LLMs) continue to revolutionize Natural Language Processing (NLP) applications, critical concerns about their trustworthiness persist, particularly in safety and robustness. To address these challenges, we introduce TRUSTVIS, an automated evaluation framework that provides a comprehensive assessment of LLM trustworthiness. A key feature of our framework is its interactive user interface, designed to offer intuitive visualizations of trustworthiness metrics. By integrating well-known perturbation methods like AutoDAN and employing majority voting across various evaluation methods, TRUSTVIS not only provides reliable results but also makes complex evaluation processes accessible to users. Preliminary case studies on models like Vicuna-7b, Llama2-7b, and GPT-3.5 demonstrate the effectiveness of our framework in identifying safety and robustness vulnerabilities, while the interactive interface allows users to explore results in detail, empowering targeted model improvements. Video Link: https://youtu.be/k1TrBqNVg8g

cs.SD [Back]

[133] UniMoE-Audio: Unified Speech and Music Generation with Dynamic-Capacity MoE

Zhenyu Liu,Yunxin Li,Xuanyu Zhang,Qixun Teng,Shenyuan Jiang,Xinyu Chen,Haoyuan Shi,Jinchao Li,Qi Wang,Haolan Chen,Fanbo Meng,Mingjun Zhao,Yu Xu,Yancheng He,Baotian Hu,Min Zhang

Main category: cs.SD

TL;DR: UniMoE-Audio提出了一种动态容量混合专家框架，统一了语音和音乐的生成，解决了领域冲突和数据不平衡问题。通过Top-P路由策略和混合专家设计，实现了高效的协同学习。

Details

Motivation: 目前听觉领域的通用音频生成模型面临语音和音乐分离的问题，主要原因是任务冲突和数据不平衡。UniMoE-Audio旨在解决这一问题。

Contribution: 1. 动态容量MoE框架；2. Top-P路由策略；3. 混合专家设计；4. 三阶段训练课程。

Method: 1. Top-P路由动态分配专家数量；2. 混合专家设计（路由专家、共享专家和空专家）；3. 三阶段训练策略。

Result: 在语音和音乐生成任务上达到SOTA，并展示了协同学习的优越性。

Insight: 动态MoE架构和分阶段训练策略可以有效解决领域冲突和数据不平衡问题，推动通用音频生成的发展。

Abstract: Recent advances in unified multimodal models indicate a clear trend towards comprehensive content generation. However, the auditory domain remains a significant challenge, with music and speech often developed in isolation, hindering progress towards universal audio synthesis. This separation stems from inherent task conflicts and severe data imbalances, which impede the development of a truly unified audio generation model. To address this challenge, we propose UniMoE-Audio, a unified speech and music generation model within a novel Dynamic-Capacity Mixture-of-Experts (MoE) framework. Architecturally, UniMoE-Audio introduces a Top-P routing strategy for dynamic expert number allocation, and a hybrid expert design comprising routed experts for domain-specific knowledge, shared experts for domain-agnostic features, and null experts for adaptive computation skipping. To tackle data imbalance, we introduce a three-stage training curriculum: 1) Independent Specialist Training leverages original datasets to instill domain-specific knowledge into each “proto-expert” without interference; 2) MoE Integration and Warmup incorporates these specialists into the UniMoE-Audio architecture, warming up the gate module and shared expert using a subset of balanced dataset; and 3) Synergistic Joint Training trains the entire model end-to-end on the fully balanced dataset, fostering enhanced cross-domain synergy. Extensive experiments show that UniMoE-Audio not only achieves state-of-the-art performance on major speech and music generation benchmarks, but also demonstrates superior synergistic learning, mitigating the performance degradation typically seen in naive joint training. Our findings highlight the substantial potential of specialized MoE architecture and curated training strategies in advancing the field of universal audio generation. Homepage: https://mukioxun.github.io/Uni-MoE-site/home.html

physics.med-ph [Back]

[134] An efficient approach with theoretical guarantees to simultaneously reconstruct activity and attenuation sinogram for TOF-PET

Liyang Hu,Chong Chen

Main category: physics.med-ph

TL;DR: 本文提出了一种基于最大似然估计的新数学模型，用于仅从飞行时间（TOF）-PET发射数据中同时重建活动和衰减正弦图。该方法利用衰减校正因子的指数形式，并在模型中考虑了活动总量约束，证明了其适定性。通过交替更新算法求解模型，数值实验表明该方法收敛性好、抗噪能力强，并在精度和效率上优于现有方法。

Details

Motivation: 传统PET中，衰减校正需要通过CT或MRI获取衰减图，这不仅增加辐射剂量和扫描时间，还可能导致运动引起的错位。因此，研究仅从PET发射数据中同时重建活动和衰减正弦图的方法具有重要意义。

Contribution: 1. 提出了一种新的数学模型，仅利用TOF-PET发射数据同时重建活动和衰减正弦图；2. 利用衰减校正因子的指数形式并引入活动总量约束；3. 证明了模型的适定性（存在性、唯一性和稳定性）；4. 提出了交替更新算法并分析其收敛性。

Method: 基于最大似然估计构建数学模型，利用衰减校正因子的指数形式，引入活动总量约束。通过交替更新算法求解模型，确保数值收敛性和稳定性。

Result: 数值实验表明，该方法具有数值收敛性和抗噪能力，在精度和效率上优于现有方法，并能实现自主衰减校正。

Insight: 该研究表明，仅通过PET发射数据即可实现高精度的活动和衰减正弦图重建，避免了额外扫描带来的问题，具有临床应用潜力。

Abstract: In positron emission tomography (PET), it is indispensable to perform attenuation correction in order to obtain the quantitatively accurate activity map (tracer distribution) in the body. Generally, this is carried out based on the estimated attenuation map obtained from computed tomography or magnetic resonance imaging. However, except for errors in the attenuation correction factors obtained, the additional scan not only brings in new radiation doses and/or increases the scanning time but also leads to severe misalignment induced by various motions during and between the two sequential scans. To address these issues, based on maximum likelihood estimation, we propose a new mathematical model for simultaneously reconstructing the activity and attenuation sinogram from the time-of-flight (TOF)-PET emission data only. Particularly, we make full use of the exclusively exponential form for the attenuation correction factors, and consider the constraint of a total amount of the activity in some mask region in the proposed model. Furthermore, we prove its well-posedness, including the existence, uniqueness and stability of the solution. We propose an alternating update algorithm to solve the model, and also analyze its convergence. Finally, numerical experiments with various TOF-PET emission data demonstrate that the proposed method is of numerical convergence and robust to noise, and outperforms some state-of-the-art methods in terms of accuracy and efficiency, and has the capability of autonomous attenuation correction.

cs.RO [Back]

[135] UNCAP: Uncertainty-Guided Planning Using Natural Language Communication for Cooperative Autonomous Vehicles

Neel P. Bhatt,Po-han Li,Kushagra Gupta,Rohan Siva,Daniel Milan,Alexander T. Hogue,Sandeep P. Chinchali,David Fridovich-Keil,Zhangyang Wang,Ufuk Topcu

Main category: cs.RO

TL;DR: UNCAP提出了一种基于自然语言的通信方法，用于协作自动驾驶车辆的规划，通过显式考虑感知不确定性并选择性融合信息，显著提升了通信效率和安全性。

Details

Motivation: 现有协作自动驾驶车辆的通信方法要么依赖高带宽的传感器数据流，要么忽略了共享数据中的感知不确定性，导致系统既不高效也不安全。

Contribution: UNCAP的核心贡献是通过自然语言通信和不确定性引导的策略，实现了高效、可解释且安全的协作规划。

Method: 采用两阶段通信协议：(1) 选择最相关的车辆；(2) 通过自然语言消息量化传输感知不确定性，并选择性融合信息以优化决策。

Result: 实验显示，通信带宽减少63%，驾驶安全评分提升31%，决策不确定性降低61%，碰撞距离边际提高四倍。

Insight: 通过自然语言通信和不确定性管理，能够显著提升多车协作的效率和安全性，为大规模自动驾驶系统提供了新思路。

Abstract: Safe large-scale coordination of multiple cooperative connected autonomous vehicles (CAVs) hinges on communication that is both efficient and interpretable. Existing approaches either rely on transmitting high-bandwidth raw sensor data streams or neglect perception and planning uncertainties inherent in shared data, resulting in systems that are neither scalable nor safe. To address these limitations, we propose Uncertainty-Guided Natural Language Cooperative Autonomous Planning (UNCAP), a vision-language model-based planning approach that enables CAVs to communicate via lightweight natural language messages while explicitly accounting for perception uncertainty in decision-making. UNCAP features a two-stage communication protocol: (i) an ego CAV first identifies the subset of vehicles most relevant for information exchange, and (ii) the selected CAVs then transmit messages that quantitatively express their perception uncertainty. By selectively fusing messages that maximize mutual information, this strategy allows the ego vehicle to integrate only the most relevant signals into its decision-making, improving both the scalability and reliability of cooperative planning. Experiments across diverse driving scenarios show a 63% reduction in communication bandwidth with a 31% increase in driving safety score, a 61% reduction in decision uncertainty, and a four-fold increase in collision distance margin during near-miss events. Project website: https://uncap-project.github.io/

[136] LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models

Senyu Fei,Siyin Wang,Junhao Shi,Zihao Dai,Jikun Cai,Pengfang Qian,Li Ji,Xinzhe He,Shiduo Zhang,Zhaoye Fei,Jinlan Fu,Jingjing Gong,Xipeng Qiu

Main category: cs.RO

TL;DR: 这篇论文对视觉-语言-动作（VLA）模型进行了系统的鲁棒性分析，揭示了模型在七种扰动维度下的脆弱性，发现尽管基准测试成绩优秀，但这些模型在实际扰动下表现极不稳定。

Details

Motivation: 尽管VLA模型在机器人操作任务中表现出色，但其在实际场景中的鲁棒性尚未得到充分验证。论文旨在通过系统的扰动分析揭示模型的潜在弱点。

Contribution: 论文的主要贡献是通过系统扰动分析，发现VLA模型在多种扰动维度（如相机视角和机器人初始状态）下表现脆弱，同时对语言指令的依赖极低。

Method: 研究通过在七个维度（物体布局、相机视角、机器人初始状态等）引入可控扰动，对多类先进的VLA模型进行了全面的鲁棒性测试。

Result: 结果显示，模型在扰动下性能显著下降（如从95%降至30%以下），且对语言指令几乎无响应。

Insight: 研究发现高基准测试分数并不等同于实际能力，强调了在真实场景下评估模型可靠性的重要性。

Abstract: Visual-Language-Action (VLA) models report impressive success rates on robotic manipulation benchmarks, yet these results may mask fundamental weaknesses in robustness. We perform a systematic vulnerability analysis by introducing controlled perturbations across seven dimensions: objects layout, camera viewpoints, robot initial states, language instructions, light conditions, background textures and sensor noise. We comprehensively analyzed multiple state-of-the-art models and revealed consistent brittleness beneath apparent competence. Our analysis exposes critical weaknesses: models exhibit extreme sensitivity to perturbation factors, including camera viewpoints and robot initial states, with performance dropping from 95% to below 30% under modest perturbations. Surprisingly, models are largely insensitive to language variations, with further experiments revealing that models tend to ignore language instructions completely. Our findings challenge the assumption that high benchmark scores equate to true competency and highlight the need for evaluation practices that assess reliability under realistic variation.

[137] Learning to Grasp Anything by Playing with Random Toys

Dantong Niu,Yuvan Sharma,Baifeng Shi,Rachel Ding,Matteo Gioia,Haoru Xue,Henry Tsai,Konstantinos Kallidromitis,Anirudh Pai,Shankar Shastry,Trevor Darrell,Jitendra Malik,Roei Herzig

Main category: cs.RO

TL;DR: 论文提出了一种通过使用随机组装物体（由四种基本形状组成）训练机器人抓取的方法，实现了对真实世界物体的零样本泛化，并提出了一种物体中心的视觉表示机制。

Details

Motivation: 受儿童通过简单玩具学习通用抓取技能的启发，研究机器人是否可以通过类似方式实现泛化能力。

Contribution: 主要贡献包括：1）使用随机组装物体训练机器人抓取的方法；2）提出物体中心的视觉表示机制；3）验证了零样本泛化的有效性。

Method: 方法基于四种基本形状（球体、长方体、圆柱体和环形）随机组装成训练物体，并提出检测池化机制以诱导物体中心的视觉表示。

Result: 在仿真和物理机器人实验中，模型在YCB数据集上实现了67%的抓取成功率，优于依赖更多领域数据的现有方法。

Insight: 训练物体的数量和多样性以及每个物体的演示次数对零样本泛化性能有显著影响，这为机器人学习的可扩展性和泛化性提供了新思路。

Abstract: Robotic manipulation policies often struggle to generalize to novel objects, limiting their real-world utility. In contrast, cognitive science suggests that children develop generalizable dexterous manipulation skills by mastering a small set of simple toys and then applying that knowledge to more complex items. Inspired by this, we study if similar generalization capabilities can also be achieved by robots. Our results indicate robots can learn generalizable grasping using randomly assembled objects that are composed from just four shape primitives: spheres, cuboids, cylinders, and rings. We show that training on these “toys” enables robust generalization to real-world objects, yielding strong zero-shot performance. Crucially, we find the key to this generalization is an object-centric visual representation induced by our proposed detection pooling mechanism. Evaluated in both simulation and on physical robots, our model achieves a 67% real-world grasping success rate on the YCB dataset, outperforming state-of-the-art approaches that rely on substantially more in-domain data. We further study how zero-shot generalization performance scales by varying the number and diversity of training toys and the demonstrations per toy. We believe this work offers a promising path to scalable and generalizable learning in robotic manipulation. Demonstration videos, code, checkpoints and our dataset are available on our project page: https://lego-grasp.github.io/ .

[138] InternVLA-M1: A Spatially Guided Vision-Language-Action Framework for Generalist Robot Policy

Xinyi Chen,Yilun Chen,Yanwei Fu,Ning Gao,Jiaya Jia,Weiyang Jin,Hao Li,Yao Mu,Jiangmiao Pang,Yu Qiao,Yang Tian,Bin Wang,Bolun Wang,Fangjing Wang,Hanqing Wang,Tai Wang,Ziqin Wang,Xueyuan Wei,Chao Wu,Shuai Yang,Jinhui Ye,Junqiu Yu,Jia Zeng,Jingjing Zhang,Jinyu Zhang,Shi Zhang,Feng Zheng,Bowen Zhou,Yangkun Zhu

Main category: cs.RO

TL;DR: 論文提出了一個名為InternVLA-M1的統一框架，通過空間引導的視覺-語言-動作訓練，提升機器人遵循指令的能力，實現通用的智能機器人政策。

Details

Motivation: 現有機器人政策在多任務和複雜場景中表現不足，尤其是空間推理和動作生成的結合不夠緊密。因此，作者提出基於空間引導的統一框架，以實現更具擴展性和泛化能力的機器人智能。

Contribution: 1. 提出了一個兩階段的空間引導訓練框架，將空間定位與動作生成緊密結合；2. 通過大規模數據預訓練和後訓練提升模型性能；3. 展示了在仿真和現實場景中的顯著性能提升。

Method: 1. 空間定位預訓練階段：使用230萬空間推理數據，學習指令與視覺空間位置的對齊；2. 空間引導動作後訓練階段：通過即插即用的空間提示生成動作，結合機器人具體形態。

Result: 在多個數據集和任務中表現優異：SimperEnv（+14.6%）、WidowX（+17%）、LIBERO Franka（+4.3%）。在真實場景中，合成數據聯合訓練提升了20.6%，長時序推理任務超過現有方法10%以上。

Insight: 空間引導訓練是實現可擴展和魯棒通用機器人的關鍵原則，尤其在複雜場景和長時序任務中表現出強大的優勢。

Abstract: We introduce InternVLA-M1, a unified framework for spatial grounding and robot control that advances instruction-following robots toward scalable, general-purpose intelligence. Its core idea is spatially guided vision-language-action training, where spatial grounding serves as the critical link between instructions and robot actions. InternVLA-M1 employs a two-stage pipeline: (i) spatial grounding pre-training on over 2.3M spatial reasoning data to determine where to act'' by aligning instructions with visual, embodiment-agnostic positions, and (ii) spatially guided action post-training to decide how to act’’ by generating embodiment-aware actions through plug-and-play spatial prompting. This spatially guided training recipe yields consistent gains: InternVLA-M1 outperforms its variant without spatial guidance by +14.6% on SimplerEnv Google Robot, +17% on WidowX, and +4.3% on LIBERO Franka, while demonstrating stronger spatial reasoning capability in box, point, and trace prediction. To further scale instruction following, we built a simulation engine to collect 244K generalizable pick-and-place episodes, enabling a 6.2% average improvement across 200 tasks and 3K+ objects. In real-world clustered pick-and-place, InternVLA-M1 improved by 7.3%, and with synthetic co-training, achieved +20.6% on unseen objects and novel configurations. Moreover, in long-horizon reasoning-intensive scenarios, it surpassed existing works by over 10%. These results highlight spatially guided training as a unifying principle for scalable and resilient generalist robots. Code and models are available at https://github.com/InternRobotics/InternVLA-M1.

cs.CY [Back]

[139] Toward LLM-Supported Automated Assessment of Critical Thinking Subskills

Marisa C. Peczuh,Nischal Ashok Kumar,Ryan Baker,Blair Lehman,Danielle Eisenberg,Caitlin Mills,Keerthi Chebrolu,Sudhip Nashi,Cadence Young,Brayden Liu,Sherry Lachman,Andrew Lan

Main category: cs.CY

TL;DR: 该论文探讨了利用大语言模型（LLM）自动化评估批判性思维子技能的可行性，通过学生议论文分析不同模型方法的表现，发现GPT-5在少量示例提示下效果最佳。

Details

Motivation: 批判性思维是教育中的核心能力，但目前缺乏有效的自动化评估方法。论文旨在通过LLM支持的学生议论文分析，填补这一空白。

Contribution: 提出了三种自动化评分方法（零样本提示、少量样本提示和监督微调），并在不同LLM上验证了其在评估批判性思维子技能中的表现。

Method: 基于学生议论文数据集，开发了编码标准，并比较了GPT-5、GPT-5-mini和ModernBERT三种模型的性能。

Result: GPT-5在少量样本提示下表现最佳，尤其在可分离且高频的子技能上；开源模型在实用性上表现较好但对少数类别敏感性较低。

Insight: 专有模型在高可靠性上表现优越但成本高，而开源模型在精确性和成本上更平衡；为高阶推理技能的自动化评估提供了初步探索。

Abstract: Critical thinking represents a fundamental competency in today’s education landscape. Developing critical thinking skills through timely assessment and feedback is crucial; however, there has not been extensive work in the learning analytics community on defining, measuring, and supporting critical thinking. In this paper, we investigate the feasibility of measuring core “subskills” that underlie critical thinking. We ground our work in an authentic task where students operationalize critical thinking: student-written argumentative essays. We developed a coding rubric based on an established skills progression and completed human coding for a corpus of student essays. We then evaluated three distinct approaches to automated scoring: zero-shot prompting, few-shot prompting, and supervised fine-tuning, implemented across three large language models (GPT-5, GPT-5-mini, and ModernBERT). GPT-5 with few-shot prompting achieved the strongest results and demonstrated particular strength on subskills with separable, frequent categories, while lower performance was observed for subskills that required detection of subtle distinctions or rare categories. Our results underscore critical trade-offs in automated critical thinking assessment: proprietary models offer superior reliability at higher cost, while open-source alternatives provide practical accuracy with reduced sensitivity to minority categories. Our work represents an initial step toward scalable assessment of higher-order reasoning skills across authentic educational contexts.

[140] Addressing the alignment problem in transportation policy making: an LLM approach

Xiaoyu Yan,Tianxing Dai,Yu,Nie

Main category: cs.CY

TL;DR: 该论文探讨了用大语言模型（LLMs）解决交通政策制定中的对齐问题，通过多智能体仿真模拟居民偏好投票，结果显示LLMs能近似集体偏好，但也存在模型特有的行为偏差。

Details

Motivation: 交通规划中，集体偏好与模型驱动的决策工具结果常不一致，导致政策实施延误或失败。作者希望通过LLMs的推理和决策模拟能力解决这一问题。

Contribution: 提出了一种基于LLM的多智能体仿真框架，用于模拟城市居民对交通政策的投票过程，并通过即时复选投票（IRV）聚合偏好。

Method: 使用GPT-4o和Claude-3.5构建LLM智能体代表不同社区居民，采用链式思考推理生成偏好，并通过IRV方法建模民主共识。

Result: LLM智能体能够近似合理的集体偏好并响应本地情境，但也表现出模型特有的行为偏差和与优化基准的轻微差异。

Insight: LLMs在交通决策中对齐问题中展现出潜力，但其行为偏差和局限性也需谨慎对待。

Abstract: A key challenge in transportation planning is that the collective preferences of heterogeneous travelers often diverge from the policies produced by model-driven decision tools. This misalignment frequently results in implementation delays or failures. Here, we investigate whether large language models (LLMs), noted for their capabilities in reasoning and simulating human decision-making, can help inform and address this alignment problem. We develop a multi-agent simulation in which LLMs, acting as agents representing residents from different communities in a city, participate in a referendum on a set of transit policy proposals. Using chain-of-thought reasoning, LLM agents provide ranked-choice or approval-based preferences, which are aggregated using instant-runoff voting (IRV) to model democratic consensus. We implement this simulation framework with both GPT-4o and Claude-3.5, and apply it for Chicago and Houston. Our findings suggest that LLM agents are capable of approximating plausible collective preferences and responding to local context, while also displaying model-specific behavioral biases and modest divergences from optimization-based benchmarks. These capabilities underscore both the promise and limitations of LLMs as tools for solving the alignment problem in transportation decision-making.