cs.CL [Total: 22]
cs.CV [Total: 82]
eess.IV [Total: 1]
cs.MA [Total: 1]
cs.IR [Total: 1]
cs.AI [Total: 4]
cs.RO [Total: 4]
cs.LG [Total: 5]
cs.SD [Total: 1]
math.NA [Total: 1]
cs.MM [Total: 1]
cs.AR [Total: 1]

cs.CL [Back]

[1] Indi-RomCoM: Code-Mixed Benchmark for Evaluating LLMs on Romanized Indic-English Instructions cs.CL | cs.AIPDF

Avisha Das, Mihir Parmar, Mohana Ramnath, Pulkit Verma

TL;DR: 该论文提出了Indi-RomCoM基准，用于系统评估大语言模型在罗马化印度语-英语混合指令上的表现。基准涵盖七种指令跟随任务、四种印度语言和三种代码混合强度级别。评估发现，LLMs在罗马化代码混合指令上表现不佳，且性能随混合密度增加而下降。

Details

Motivation: 罗马化代码混合已成为多语言社区的主要交流形式，但LLMs在单语和原生脚本基准上表现良好，其处理基于罗马化代码混合内容的能力尚未得到充分探索。

Result: 在零样本和少样本设置下评估了一系列专有、开源和印度语专用LLMs，模型在罗马化代码混合指令上持续表现不佳，性能随代码混合密度增加而下降；推理任务比检测任务（如毒性检测）受性能下降影响更小。

Insight: 创新点在于创建了首个系统评估LLMs处理罗马化印度语-英语混合指令的基准；客观分析表明，该基准有助于揭示LLMs在非标准语言变体上的能力缺陷，并为开发包容性多语言系统提供重要工具。

Abstract: Romanized Code Mixing (RCM), where bilingual speakers fluidly blend local languages with English in Roman script, has emerged as the dominant form of communication across multilingual communities. While Large Language Models (LLMs) perform strongly on monolingual and native-script benchmarks, their ability to follow instructions and reason over RCM-based content remains largely unexplored. To this end, we introduce the Indi-RomCoM benchmark for facilitating systematic evaluation on Indic Romanized Code-Mixed instructions. Our benchmark spans seven instruction-following tasks, four widely spoken Indic languages, and three controlled code-mixing intensity levels. We extensively evaluate a suite of LLMs covering proprietary, open-weight, and Indic-focused models under zero- and few-shot settings. LLMs consistently underperform on RCM instructions, with performance degrading as code-mixing density increases. Furthermore, reasoning tasks suffer less degradation than detection tasks (e.g., Toxicity) because the generated explanations offer necessary context. We believe Indi-RomCoM helps the community in developing inclusive multilingual systems.

[2] Test-Time Verification for Text-to-SQL via Outcome Reward Models cs.CL | cs.AI | cs.DBPDF

Mattia Tritto, Giuseppe Farano, Dario Di Palma, Gaetano Rossiello, Fedelucio Narducci

TL;DR: 本文提出了一种名为GradeSQL的可扩展框架，通过训练任务特定的结果奖励模型（ORMs）来改进文本到SQL任务中的测试时验证。该方法利用自动候选生成和基于执行的标签来训练验证器，无需人工标注，并将ORMs集成到验证驱动的最佳N采样流程中。在BIRD和Spider基准测试中，基于ORM的选择策略一致优于基于执行的最佳N采样和多数投票方法。

Details

Motivation: 针对文本到SQL等结构化推理任务，提高大型语言模型在推理时的可靠性是一个核心挑战。现有的测试时推理策略（如最佳N采样和多数投票）依赖于执行成功或输出频率等启发式信号，这些信号在候选输出之间提供的语义区分能力有限。

Result: 在BIRD和Spider基准测试上，基于ORM的选择策略相比基于执行的最佳N采样和多数投票方法，性能提升最高分别达到+4.33%和+2.10%。结果表明，ORM能有效扩展到更大的候选集，并在复杂查询上带来更强的改进。

Insight: 论文的创新点在于将结果奖励模型（ORMs）作为学习到的语义评分函数应用于文本到SQL的测试时验证，并提出了一个无需人工标注的可扩展训练框架GradeSQL。从客观角度看，该方法为启发式测试时选择策略提供了一个简单、有效且可扩展的替代方案，特别是在结构化查询生成任务中验证了ORM的潜力。

Abstract: Improving the reliability of large language models (LLMs) at inference time is a central challenge in structured reasoning tasks such as Text-to-SQL. Common test-time inference strategies, including Best-of-N sampling and Majority Voting, rely on heuristic signals such as execution success or output frequency, which provide limited semantic discrimination across candidate outputs. In this work, we study Outcome Reward Models (ORMs) as learned semantic scoring functions for test-time verification in Text-to-SQL. While ORMs have been previously explored for test-time scaling and alignment, their application to structured query generation remains underexplored. We introduce GradeSQL, a scalable framework for training task-specific ORMs via automated candidate generation and execution-based labeling, enabling verifier training without manual annotation. We integrate ORMs into a verification-driven Best-of-N pipeline and evaluate our approach on the BIRD and Spider benchmarks across multiple open-source LLM families. ORM-based selection consistently outperforms execution-based Best-of-N and Majority Voting, with gains of up to +4.33% on BIRD and +2.10% on Spider. We further show that ORMs scale effectively with larger candidate sets and yield stronger improvements on complex queries. Overall, our results demonstrate that ORM-based verification provides a simple, effective, and scalable alternative to heuristic test-time selection strategies for Text-to-SQL. Code datasets and models are publicly available.

[3] Using AI Agents to Automate Black-Box Audits of Personalization Algorithms at Scale cs.CL | cs.CY | cs.LG | cs.SIPDF

Alessandro Morosini, Sarah H. Cen, Andrew Ilyas, Hedi Driss, Aleksander Mądry

TL;DR: 本文提出了一种利用生成式AI代理进行个性化算法黑盒审计的框架，通过为合成账户配备基于人口统计和政治调查数据构建的固定人设AI代理，模拟真实用户行为与平台交互，从而实现对平台个性化推荐算法的反事实审计。

Details

Motivation: 现有审计方法难以在真实用户行为（成本高、控制难）与脚本化傀儡审计（可扩展但真实性有限）之间取得平衡，且无法有效解耦用户属性与行为，限制了因果理解个性化算法的能力。

Result: 在2024年美国大选后于X平台部署1,120个代理（涵盖14种人设和三种反事实条件），收集超过20万条内容曝光数据；发现算法推荐流相比时间顺序流放大了有毒、极化、政治化和右倾内容，且放大程度因用户意识形态差异显著；反事实分析显示人口统计信号对内容推送的影响因人设而异，总体效应微弱但子群体层面效应方向和强度存在差异。

Insight: 创新点在于利用生成式AI代理作为行为引擎，通过固定人设下的可控行为模拟，实现了用户属性与行为的解耦，支持反事实因果分析；该方法为大规模黑盒算法审计提供了可扩展且更接近真实的新工具。

Abstract: Personalization algorithms determine what content users encounter on online platforms. Auditing these systems is difficult because independent auditors have only black-box access to the algorithms, while personalization depends on users’ attributes, behavior, and evolving interaction histories. Existing auditing methods face a tradeoff: studies with real users capture realistic behavior but are costly and hard to control, whereas sock-puppet audits scale more easily but often rely on scripted behavior that limits realism. Beyond this, both approaches struggle to decouple user attributes from user behavior, limiting our ability to causally understand personalization. To address this gap, we introduce a framework for black-box audits of personalization algorithms using generative AI agents as behavioral engines for synthetic accounts. Each agent is instantiated with a fixed persona, grounded in demographic and political survey data, and interacts with a platform’s content by reasoning about it and choosing actions. Because behavior is fixed within each persona while platform-visible signals such as age, gender, or location can be experimentally perturbed, our design enables counterfactual auditing of how platforms respond to user attributes. As a case study, we deploy 1,120 agents on X shortly after the 2024 U.S. election, spanning 14 personas and three counterfactual conditions, collecting over 200,000 content exposures. We find that X’s algorithmic feed amplifies toxic, polarizing, political, and right-leaning content relative to the chronological feed, with amplification varying sharply by user ideology. Counterfactual analyses show that demographic signals affect content delivery in persona-dependent ways: pooled effects are largely null, while subgroup-level effects vary in direction and magnitude. Our work establishes GenAI-based agents as a new tool for algorithmic auditing.

[4] Training Therapeutic Judges and Multi-Agent Systems for Human-Aligned Mental Health Support cs.CL | cs.AI | cs.MAPDF

Mizanur Rahman, Abeer Badawi, Elahe Rahimi, Laleh Seyyed-Kalantari, Frank Rudzicz

TL;DR: 本文提出一个两阶段框架，通过将治疗性回复生成构建为基于多维人类对齐评估的决策优化问题来提升心理健康支持质量。第一阶段训练开源评估器TheraJudge，在7个心理维度上提供可靠评估；第二阶段开发多智能体系统TheraAgent，利用评估信号通过批评家、教练和治疗师角色的协同优化生成针对性改进回复。

Details

Motivation: 现有大语言模型在心理健康支持应用中，仅将评估作为被动指标难以有效提升治疗质量，需要将评估转化为可操作的优化信号以实现人类对齐。

Result: TheraJudge在7个心理维度上与临床医生评分达到高度一致（ICC=0.87-0.95），超越监督基线和闭源评估器；TheraAgent在盲测中使人类评分治疗质量提升0.43分（5分制），低质量回复（≤3分）提升2.45分且94%得到修复，临床医生评分者间信度达96%。

Insight: 创新点在于将评估从被动度量转变为主动优化信号，并通过角色化多智能体架构实现评估到生成的闭环优化；客观来看，该工作证明了在专业领域（心理健康）中，基于人类对齐评估的迭代优化比单纯增强生成能力更有效，为领域特异性LLM对齐提供了可借鉴的框架。

Abstract: Large language models show promise for mental health support, yet therapeutic quality improves only when evaluation functions as an actionable control signal rather than a passive metric. We introduce a framework that formulates therapeutic response generation as a decision-refinement problem driven by multi-dimensional, human-aligned evaluation. In Stage I, we introduce TheraJudge, an open-source therapeutic evaluator trained via preference-based optimization on human-annotated data to produce reliable judgments across 7 psychological dimensions. In Stage II, we introduce TheraAgent, which operationalizes TheraJudge’s evaluations through a coordinated refinement process with specialized Critic, Coach, and Therapist roles that translate evaluative signals into targeted response revisions. Empirically, TheraJudge achieves strong agreement with clinician ratings, with intraclass correlation coefficients (ICC = 0.87-0.95), surpassing supervised baselines and strong closed-source judges, particularly on critical dimensions such as Safety, Relevance, and Empathy. Acting on these evaluations, TheraAgent yields a +0.43 improvement in human-rated therapeutic quality (on a 5-point scale) under blind evaluation, with 96% clinician inter-rater reliability. Low-quality responses ($\leq 3$) improve by +2.45 points with a 94% recovery rate, demonstrating targeted correction of unsafe outputs. Overall, our results indicate that effective alignment of mental-health LLMs stems from acting on human-aligned evaluation, rather than relying solely on stronger generation. We release code at https://github.com/vis-nlp/TheraAlign.

[5] Measuring Judgment Quality in Natural-Language Explanations: Evidence from Forecasting Tournaments cs.CL | econ.GNPDF

Christopher W. Karvetski, Sheldon S. Huang, Simas Kučinskas, Nadja Flechner, Jingyu Hu

TL;DR: 本文提出了一种名为解释质量标记（EQMs）的方法，用于大规模评估自然语言解释的质量。该方法基于理论指导的60种推理模式，利用大语言模型（LLMs）进行评分，并在一个多年预测锦标赛的超过55,000个预测-理由对上进行了验证。研究发现EQMs能在预测和预测者层面有效预测准确性，优于传统的文本分析方法，并且其信号具有不对称性，能更可靠地识别表现不佳者。

Details

Motivation: 决策者经常依赖专家判断及其书面解释，但解释质量难以大规模衡量。预测锦标赛提供了一个自然的测试环境，其中概率判断与自然语言理由配对，并与实际结果进行评分。

Result: 在预注册分析中，EQMs在预测和预测者层面均能预测准确性，一致优于前LLM时代的文本分析方法。超过90%的统计显著模式级EQM-准确性相关性符合预期方向。在预测层面，EQMs是最强的预测指标；在预测者层面，其表现具有竞争力，但弱于先前的准确性。人类对理由质量的评分与准确性的相关性较弱且不一致，并过度看重理由长度。结果在一个独立的预测研究中得到了验证。

Insight: 创新点在于提出了一个可扩展、可解释的框架（EQMs），利用LLMs从书面解释中提取与判断相关的信息。客观来看，该方法将理论指导的推理模式与LLM评分相结合，为大规模评估解释质量提供了新工具，并揭示了人类评估与自动化评估在侧重点上的差异（如对文本长度的偏见）。

Abstract: Decision-makers routinely rely on expert judgments accompanied by written explanations, yet explanation quality is difficult to measure at scale. Forecasting tournaments offer a natural testing ground: probabilistic judgments are paired with natural-language rationales and scored against realized outcomes. We introduce Explanation Quality Markers (EQMs), a set of sixty theory-guided reasoning patterns scored by large language models (LLMs). In a pre-registered analysis of over 55,000 forecast-rationale pairs from a multiyear forecasting tournament, EQMs predict accuracy at both the forecast and forecaster levels, consistently outperforming pre-LLM text-analysis methods. More than 90% of statistically significant pattern-level EQM-accuracy correlations match our directional hypotheses. The signal is asymmetric: EQMs identify likely underperformers more reliably than they distinguish the very best forecasters. Benchmarked against traditional indicators of forecasting skill, EQMs are the strongest predictor at the forecast level and competitive at the forecaster level, though weaker than prior accuracy. Human ratings of rationale quality are less consistently correlated with accuracy and place disproportionate weight on rationale length. Results transfer to an independent forecasting study. EQMs provide a scalable, interpretable method for extracting judgment-relevant information from written explanations.

[6] Wait, am I Being Fair? Characterizing Deductive Stereotyping and Mitigating It with Fair-GCG cs.CL | cs.AIPDF

Naihao Deng, Yilun Zhu, Joan Nwatu, Clayton Scott, Rada Mihalcea

TL;DR: 本文提出了一种名为‘演绎刻板印象’的LLM推理失败模式，即模型将群体层面的统计规律错误地应用于个体案例，导致逻辑连贯但社会偏见性的推断。为引导模型进行公平感知推理，作者提出了一个推理时注入框架，并引入了Fair-GCG方法来自动发现有效的注入短语。

Details

Motivation: 尽管近期大语言模型的推理能力通常能提升公平性，但偏见问题依然存在。本文旨在识别并解决一种特定的推理失败模式——演绎刻板印象。

Result: Fair-GCG发现的注入短语在多个公平性基准测试中提升了性能，能够从小模型泛化到大模型，改善了推理层面的公平性，减少了开放式生成中的偏见，并能迁移到现实世界的公平敏感任务中。

Insight: 创新点在于对‘演绎刻板印象’这一新失败模式的定义与统计解释，以及提出的推理时注入框架和Fair-GCG方法，该方法能系统地发现可迁移、可泛化的公平性引导短语，而非依赖人工设计。

Abstract: Warning: This paper contains several toxic and offensive statements. While reasoning generally improves fairness in recent large language models (LLMs), failures persist. In this work, we identify a failure mode, deductive stereotyping, in which models apply population-level statistical regularities to individual cases, producing logically coherent yet socially biased inferences. We provide a statistical interpretation of this phenomenon. To steer models toward fairness-aware reasoning, we propose a reasoning-time injection framework. We further introduce Fair-GCG to systematically discover effective injection phrases. Injection phrases discovered by Fair-GCG improve performance across multiple fairness benchmarks, generalize from smaller to larger LLMs, improves reasoning-level fairness, reduces bias in open-ended generation, and transfer to real-world fairness-sensitive tasks.

[7] A Semantic-Layer-Mediated Agent for Natural Language to SQL over Heterogeneous Enterprise Databases cs.CLPDF

Ha Jeong Kim, Saksonita Khoeurn, Ye Ji Yoon

TL;DR: 本文提出了一种基于语义层中介的自然语言转SQL代理，用于处理异构企业数据库。该方法通过引入语义模型查询（SMQ）作为中间表示，将语义意图与物理SQL执行解耦，并利用确定性编译器将SMQ转换为特定方言的SQL。系统采用受限的思维-行动循环，支持多种数据库后端，并在Spider2-snow基准测试中取得了94.15%的执行准确率。

Details

Motivation: 解决真实企业环境中NL2SQL的挑战，包括大规模物理表、隐晦列名、异构SQL方言以及复杂分析工作负载（如嵌套聚合、时序推理和多表连接）带来的困难。

Result: 在包含547个任务的Spider2-snow基准测试上，使用Gemini 3 Pro模型实现了94.15%的执行准确率，在官方排行榜中排名第三，显著优于仅依赖原始模式的方法。

Insight: 创新点在于通过语义层和SMQ中间表示实现语义与执行的解耦，提供可验证的构建块；客观分析认为该方法通过约束推理循环和确定性编译，提升了复杂查询的鲁棒性和跨数据库兼容性，但需权衡语义层质量与过拟合风险。

Abstract: Natural language-to-SQL (NL2SQL) over real-world enterprise databases remains significantly more challenging than on academic benchmarks. Enterprise schemas often contain hundreds of physical tables with cryptic column names, heterogeneous SQL dialects, and complex analytical workloads requiring nested aggregations, temporal reasoning, and multi-table joins. We present a semantic-layer-mediated NL2SQL agent that decouples semantic intent from physical SQL execution. Rather than generating SQL directly over raw schemas, the agent reasons over a curated semantic layer through a compact intermediate representation called the Semantic Model Query (SMQ). A deterministic compiler translates each SMQ into dialect-specific SQL, providing verified building blocks that the agent composes into the final query. The system employs a constrained think-act loop, supports SQLite, BigQuery, and Snowflake backends, and is integrated into an end-to-end evaluation framework. Using Gemini 3 Pro, the system achieves 94.15% execution accuracy on the 547-task Spider2-snow benchmark, ranking third on the official leaderboard and substantially outperforming schema-only approaches. We describe the system architecture, SMQ representation, agent workflow, evaluation results, and discuss semantic-layer quality and the trade-off between improved grounding and overfitting.

[8] Building a Multimodal Dataset of Academic Paper for Keyword Extraction cs.CL | cs.DL | cs.HC | cs.IRPDF

Jingyu Zhang, Xinyi Yan, Yi Xiang, Yingyi Zhang, Chengzhi Zhang

TL;DR: 该研究构建了一个包含1000个样本的学术论文多模态数据集，每个样本包含论文文本、图像、音频和关键词。基于无监督和有监督的关键词提取方法，实验使用了论文文本以及从图像和音频中提取的文本，旨在探究不同模态信息及其融合对关键词提取任务性能的影响。

Details

Motivation: 当前关键词提取任务通常仅依赖文本数据，忽略了图像和音频模态的视觉细节与音频特征，导致信息丰富度不足并忽视了潜在关联，限制了模型学习数据表示的能力和预测准确性。此外，现有的多模态关键词提取数据集非常稀缺，进一步阻碍了该领域的研究进展。

Result: 实验结果表明，不同模态的文本在模型中表现出不同的特征。将论文文本、图像文本和音频文本进行拼接，可以有效提升学术论文关键词提取的性能。

Insight: 该研究的创新点在于构建了一个专门用于关键词提取任务的多模态学术论文数据集，填补了该领域数据资源的空白。从客观角度看，通过融合多模态文本信息来增强关键词提取性能，为多模态自然语言处理任务提供了新的思路和实验基础。

Abstract: Up to this point, keyword extraction task typically relies solely on textual data. Neglecting visual details and audio features from image and audio modalities leads to deficiencies in information richness and overlooks potential correlations, thereby constraining the model’s ability to learn representations of the data and the accuracy of model predictions. Furthermore, the currently available multimodal datasets for keyword extraction task are particularly scarce, further hindering the progress of research on multimodal keyword extraction task. Therefore, this study constructs a multimodal dataset of academic paper consisting of 1000 samples, with each sample containing paper text, images, audios and keywords. Based on unsupervised and supervised methods of keyword extraction, experiments are conducted using textual data from papers, as well as text extracted from images and audio. The aim is to investigate the differences in performance in keyword extraction task with respect to different modal information and the fusion of multimodal information. The experimental results indicate that text from different modalities exhibits distinct characteristics in the model. The concatenation of paper text, image text and audio text can effectively enhance the keyword extraction performance of academic papers.

[9] TAG-DLM: Diffusion Language Models for Text-Attributed Graph Learning cs.CL | cs.LGPDF

Lingjie Chen, Yuanchen Bei, Haobo Xu, Yanjun Zhao, Yuzhong Chen

TL;DR: 本文提出了一种名为TAG-DLM的扩散语言模型，用于处理文本属性图（TAGs）的学习任务。该方法通过将局部邻域线性化为token序列，并利用拓扑注意力掩码注入图结构，从而在统一的扩散语言模型框架内实现了文本推理和图消息传递。模型通过提示调整即可适应节点分类、链接预测和跨数据集迁移等多种任务，无需针对特定目标进行微调。

Details

Motivation: 现有方法在处理文本属性图时，通常将文本和图拓扑分开处理，如图神经网络仅使用浅层文本特征，而LLM与图的混合方法主要将语言模型用作文本编码器，结构学习则交由单独的图模块。本文旨在统一文本推理和图消息传递，以更有效地联合建模文本和图结构。

Result: 实验表明，该方法在两个任务的三个TAG基准测试中，均优于图神经网络、图Transformer和基于LLM的基线方法，相比最强基线提升了高达3.9个百分点。

Insight: 创新点在于将扩散语言模型与图学习相结合，通过拓扑注意力掩码实现图消息传递，使模型能够同时解释和生成文本，并通过提示调整灵活适应不同任务，实现了文本与图结构的统一建模，避免了任务特定的微调。

Abstract: Text-attributed graphs (TAGs), where each node carries a natural language description, require models to jointly reason over text and graph topology. Existing approaches often handle the two modalities separately: graph neural networks operate on shallow text features, while hybrids of LLMs and graphs use the language model mainly as a text encoder and delegate structure learning to a separate graph module. We propose method that unifies textual reasoning and graph message passing within a masked diffusion language model, a language model with bidirectional attention and generative decoding. For each graph instance, method linearises a sampled local neighbourhood into a token sequence and injects graph structure through a topology attention mask, which realises message passing over the graph. Because the diffusion language model can both interpret and generate text, the method adapts to different tasks simply by changing the prompt, supporting node classification, link prediction, and cross-dataset transfer with no target-specific fine-tuning. Experiments show that method outperforms graph neural networks, graph transformers, and LLM-based baselines on all three TAG benchmarks across two tasks, improving over the strongest baseline by up to 3.9 points.

[10] LOPA: Enhancing Spoken Language Assessment via Latent Ordinal Prototype Alignment cs.CL | cs.MMPDF

Hong-Yun Lin, Fu-An Chao, Bi-Cheng Yan, Berlin Chen

TL;DR: 本文提出了一种名为LOPA（潜在序数原型对齐）的新框架，用于增强口语语言评估（SLA）。该框架通过引入基于原型的正则化器，直接在潜在空间中强制执行序数几何先验，并结合语义锚定层路由（SALR）自适应地从冻结的Whisper编码器中提取多深度表示，从而在无需大规模多模态大语言模型（MLLMs）微调的情况下，实现了与十亿参数系统相媲美的性能。

Details

Motivation: 当前基于多模态大语言模型（MLLMs）的口语语言评估范式虽然有效，但往往忽视了语言习得固有的序数结构，且依赖于大规模模型。本文旨在绕过对大规模MLLMs的需求，通过引入序数感知的建模方法来解决这一问题。

Result: 在口语语言评估任务上，该框架实现了0.361的RMSE（均方根误差），其性能可与十亿参数系统相媲美，且无需基于LLM的微调。

Insight: 主要创新点包括：1. 提出LOPA，一种直接在潜在空间强制执行序数结构先验的原型对齐正则化方法；2. 提出SALR，一种自适应地从冻结编码器中路由多深度语义表示的技术；3. 两者的协同作用提供了可解释的、与评估标准对齐的偏好，为当前以规模为中心的SLA模型提供了一种高效且序数感知的替代方案。

Abstract: Fueled by increasing model scale and multimodal inputs, Multimodal Large Language Models (MLLMs) have emerged as a promising paradigm for Spoken Language Assessment (SLA). While effective, this paradigm often overlooks the intrinsic ordinal structure of language acquisition. This paper works around the necessity of large-scale MLLMs by introducing Latent Ordinal Prototype Alignment (LOPA) for SLA, a prototype-based regularizer that enforces an ordinal geometric prior directly on the latent space. Coupled with Semantic-Anchored Layer Routing (SALR), which adaptively harvests multi-depth representations from a frozen Whisper encoder, our framework achieves an RMSE of 0.361. This performance rivals billion-parameter systems without the need for LLM-based fine-tuning. Further analysis reveals that SALR’s synergy with LOPA offers interpretable, criterion-aligned preferences, thereby supporting an efficient and ordinal-aware modeling alternative to current scaling-centric models for SLA.

[11] Clinically Structured Rank-Gated LoRA for Cross-Benchmark Medical Question Answering cs.CLPDF

Yining Huang

TL;DR: 本文提出了一种名为BiRG-LoRA的单一适配器、秩门控的LoRA方法，用于医学多选题问答任务。该方法通过一个双轴门控机制，结合问题的隐藏语义证据与临床先验知识，为每个问题动态选择稀疏的秩原子子集，并控制适配器更新的强度，从而在多个异构医学基准上实现高效且有效的参数适应。

Details

Motivation: 医学多选题问答需要在不同的知识领域和推理操作之间进行参数高效的适应。不同类型的医学问题（如用药、诊断、公共卫生、护理操作）可能需要不同的低秩更新，而某些回忆类问题则应基本保留基础模型的表示，仅需轻微的适配器干预。现有方法在跨基准适应时存在效率或效果不足的问题。

Result: 在匹配的Qwen3-8B CMB-source协议下，BiRG-LoRA在CMB、CMExam、MedQA和MedMCQA四个基准上的宏平均准确率达到69.31%，在可训练的PEFT基线方法和匹配的路由控制方法中最高。相比MoELoRA，其准确率提升0.89个百分点，同时可训练参数减少28.1%。与普通LoRA r16和活动秩匹配的LoRA r4相比，宏平均准确率也分别提升0.83个百分点。

Insight: 创新点在于提出了一个临床结构化的秩门控机制，将问题语义与临床先验（专业领域和临床操作）及其交互结合起来，动态地为每个输入问题选择适配器更新的稀疏秩子集，实现了输入条件化的低秩适应。这为在异构医学任务上进行更精细、更高效的参数高效微调提供了新思路，且对适度的标签噪声具有鲁棒性。

Abstract: Medical multiple-choice question answering requires parameter-efficient adaptation across heterogeneous knowledge domains and reasoning operations. A medication question, a diagnostic decision, a public-health item, and a nursing-action item may require different low-rank updates, while some recall items should preserve the base model’s representation with only mild adapter intervention. We propose BiRG-LoRA, a single-adapter rank-gated LoRA method for medical question answering. BiRG-LoRA keeps one LoRA module per target layer but makes its rank dimension input-conditioned: for each question, a biaxial gate combines hidden semantic evidence with specialty/profession priors, clinical-operation priors, and their interaction to select a sparse top-$k$ subset of rank atoms. A scalar injection coefficient further controls the strength of the selected adapter update. Under a matched Qwen3-8B CMB-source protocol, BiRG-LoRA achieves the highest four-benchmark macro-average accuracy among trainable PEFT baselines and matched routing controls: 69.31% averaged over CMB, CMExam, MedQA, and MedMCQA. It improves over MoELoRA by 0.89 percentage points while using 28.1% fewer trainable parameters; a paired, benchmark-stratified bootstrap over final predictions gives a 95% confidence interval of [0.42, 1.37] for this macro-average gain. Basic controls show that BiRG-LoRA also improves over vanilla LoRA r16 and active-rank-matched LoRA r4 by 0.83 macro points, and an evaluation-time weak-axis perturbation check suggests that performance is not brittle to moderate tag noise. The results support a bounded claim: clinically structured rank allocation improves cross-benchmark medical QA under a matched single-seed protocol, while training-seed variance remains future work.

[12] CLExEval: A Human-in-the-Loop Framework for Qualitative Evaluation of LLM Clinical Reasoning cs.CLPDF

Ajmal M., Abin Roy, Afthab Salam Kanniyan, Jawadh Abdul Kabeer, Jerin James

TL;DR: 本文提出了一个名为CLExEval的人机协同框架，用于在渐进信息遮蔽条件下评估大型语言模型（LLM）的临床推理能力。该框架结合了5600个专家医师标注和200条来自40个罕见诊断病例的临床推理轨迹，揭示了LLM在临床推理中的三种常见失败模式，并指出自动化临床评估可能高估模型的可靠性。

Details

Motivation: 当前LLM在医学基准测试中表现优异，但其临床推理能力难以可靠评估，存在评估幻觉风险——即使最终诊断错误，流畅且结构良好的解释也可能显得具有临床说服力。

Result: 在CLExEval框架下，分析发现GPT-4o-mini在信息稀缺时诊断准确率从95.0%骤降至32.5%（冗长性偏差），专家模型最大诊断潜力达92.5%但无法在冗长上下文中可靠检索知识（隐藏知识悖论），且68.6%的病例存在推理轨迹正确但最终答案错误的不匹配现象。在人工验证的失败集（n=142）上，GPT-4o-mini错误批准了47.9%的临床不正确输出，而HuatuoGPT-o1批准了所有有效评分的失败案例并表现出自我偏好偏差。

Insight: 创新点在于提出了一个结合专家标注和渐进信息遮蔽的人机协同评估框架，系统性地识别了LLM临床推理的特定失败模式；客观来看，该研究强调了基于专家验证的必要性，揭示了当前自动化评估（包括LLM-as-a-Judge范式）在临床领域可能存在的严重局限性，为未来构建更可靠的医疗AI评估体系提供了重要见解。

Abstract: Large Language Models (LLMs) achieve strong results on many medical benchmarks, but their clinical reasoning remains difficult to evaluate reliably. A central risk is an evaluation illusion: fluent and well-structured explanations can appear clinically convincing even when the final diagnosis is incorrect. We introduce CLExEval, a human-in-the-loop framework for evaluating LLM clinical reasoning under progressive information masking. CLExEval combines 5,600 expert-physician annotations with 200 clinical reasoning traces derived from 40 rare diagnostic cases. Our analysis identifies three recurring failure patterns: (i) verbosity bias, where GPT-4o-mini’s diagnostic accuracy drops from 95.0% to 32.5% under information scarcity; (ii) a hidden knowledge paradox, where a specialist model reaches 92.5% maximum diagnostic potential but fails to retrieve that knowledge reliably in verbose contexts; and (iii) a 68.6% reasoning-to-output mismatch, where correct diagnoses appear in reasoning traces but are not reflected in final answers. We further evaluate the LLM-as-a-Judge paradigm on a human-verified failure set (n = 142). GPT-4o-mini approved 47.9% of clinically incorrect outputs, while HuatuoGPT-o1 approved all validly scored failures and showed a positive self-preference bias. These results suggest that standalone automated clinical evaluations can substantially overestimate clinical reliability without expert-grounded validation.

Nan Li, Albert Gatt, Massimo Poesio

TL;DR: 本文研究视觉语言模型（VLMs）在协作对话中区分‘可能共享’与‘已共享’信息的能力，通过基于HCRC MapTask对话的13,077条标注参考表达构建解释匹配任务，并系统控制对话上下文和地图信息访问。结果表明，提供真实地图图像虽提升整体性能，但导致模型过度预测对齐，而文本描述重现此偏差，非信息性图像则完全抑制对齐预测。校准分析和参考链追踪显示模型依赖地图静态参考线索而非跟踪对话历史中的共同基础建立过程，在Qwen3-VL-8B-Instruct等模型中观察到类似模式。

Details

Motivation: 解决协作对话中视觉语言模型是否能够区分参与者之间通过交互建立的共同基础（已共享信息）与仅基于共享感知的潜在共同基础（可能共享信息）的问题，以避免模型错误假设相互理解。

Result: 在HCRC MapTask基准上，提供真实地图图像使模型整体性能提升，但导致对齐案例的准确率下降，模型过度预测对齐；文本描述重现偏差，非信息性图像抑制对齐预测，表明偏差由任务相关地图内容驱动而非视觉通道。Qwen3-VL-8B-Instruct及另外四个模型均显示不同程度类似模式。

Insight: 创新点在于揭示VLMs在处理不对称对话时，将任务相关地图内容（无论视觉或文本形式）误视为已建立的共同基础证据，混淆潜在与已确立的共同基础；客观分析表明模型缺乏动态跟踪对话历史中共同基础建立过程的能力，依赖静态线索，这为改进VLMs的对话交互设计提供了关键见解。

Abstract: In collaborative dialogue, shared perception does not guarantee shared interpretation. Mutual understanding must be established through interaction. We investigate whether vision-language models (VLMs) can distinguish what could be shared from what has been shared between dialogue participants through grounding. We formulate this as an interpretation-matching task on 13,077 annotated reference expressions from HCRC MapTask dialogues, and evaluate VLMs under systematically controlled manipulations of dialogue context and map-information access. Our results show that providing authentic map images improves overall performance but shifts models toward over-predicting alignment. Textual descriptions of the same map content reproduce this bias, while non-informative images suppress alignment predictions entirely, indicating that the bias is driven by task-relevant map content, not the visual channel. This improvement comes at the cost of degraded accuracy on non-aligned cases. Calibration analysis and reference-chain tracking further suggest that models rely on static referential cues on the maps rather than tracking how grounding unfolds through dialogue history. We observe these patterns most clearly in Qwen3-VL-8B-Instruct and, to varying degrees, in four additional models from two architecture families. In models that exhibit the bias, map content, whether presented visually or textually, is treated as evidence of mutual understanding, conflating potential with established common ground.

[14] CHERRY: Compressed Hierarchical Experts with Recurrent Representational Yield cs.CL | cs.AIPDF

Dohyeon Kwon, Youngjin Park

TL;DR: 论文提出了三种互补的技术来训练计算高效的语言模型：选择性监督训练（SGT）通过仅监督约15%的关键语义标记来提高训练效率；深度压缩与循环恢复技术将深层模型压缩为浅层模型并通过循环展开恢复性能；以及压缩专家融合（MoEE）通过集成多个压缩模型进一步提升效果。这些技术在韩语基础模型CHERRY-1.8B上得到验证。

Details

Motivation: 旨在解决训练大型语言模型时计算成本高昂的问题，通过研究互补的技术来提升模型训练和推理的效率，同时保持或接近原始模型的性能。

Result: 在韩语数据和基于损失的指标上验证：SGT技术用15%的监督恢复了约67%的全序列损失减少，效率提升4.5倍；深度压缩技术将48层模型压缩至6层（参数减少2.5倍），通过循环恢复后损失（2.934）接近566M密集模型（2.926）；2专家MoEE将损失进一步降至2.789，优于单个最佳压缩模型（2.926）。

Insight: 创新点包括：1）选择性监督训练（SGT）利用自然语言的结构特性，通过正梯度耦合使无监督标记也受益；2）深度压缩结合循环恢复，实现参数大幅减少而性能损失极小；3）将多个压缩模型集成为混合高效专家（MoEE），以可比的活跃参数获得更好性能。这些方法为高效模型训练提供了新的技术路径。

Abstract: We study three complementary techniques for training compute-efficient language models. (1) Selective supervision and per-token efficiency. Selective Ground Truth Token Training (SGT) concentrates supervision on the ~15% of output tokens that carry semantic payload. Through positive gradient coupling in position-shared transformer weights – a token-level instance of auxiliary-task transfer – the remaining 85% of unsupervised tokens still improve substantially, giving a 4.5x per-supervised-token efficiency (at the step-100 eval optimum, ~67% of the full-sequence loss reduction is recovered from 15% of the supervision). We prove that this improvement on unsupervised tokens is guaranteed whenever the gradient coupling coefficient gamma-bar = 0.72 is positive (Theorem 1), and show the effect is a property of natural-language structure: it collapses on shuffled text. (2) Depth compression with recurrent recovery. A 48-layer, 1B-parameter transformer is compressed to 6 layers (227M) by averaging adjacent layers and restored through learned recurrent unrolling. With 34 effective recurrent layers it reaches a held-out loss of 2.934, within measurement noise of a 566M dense model at 2.926 – a 2.5x reduction in parameters. (3) Fusion of compressed experts. Assembling several compressed models as a Mixture of Efficient Experts (MoEE) with multi-token prediction improves over each single expert at comparable active parameters: a 2-expert MoEE reaches loss 2.789 versus 2.926 for the best single compressed model. We validate these techniques on CHERRY-1.8B, a Korean foundation model whose every trainable parameter derives from our own training runs. We are explicit throughout about the scope of the evidence (one model family, Korean data, loss-based metrics) and about which claims are established versus prospective.

[15] Theory of Mind and Persuasion Beyond Conversation: Assessing the Capacity of LLMs to Induce Belief States via Planning and Action cs.CLPDF

Ben Slater, Matteo G. Mecattaf, Lucy G. Cheke, John Burden, Winnie Street

TL;DR: 本文提出了一种评估大语言模型非对话式心智理论能力的新框架NCP-ExploreToM，通过要求模型执行规划与行动（如移动物体或引导角色）来诱导其他智能体产生特定信念状态，而非依赖传统的对话式说服。研究评估了包括GPT-5在内的六个前沿模型和人类参与者在600个任务实例上的表现，发现GPT-5在智能体设定下成功率约80%，是唯一超越人类表现的模型，但在跨情境鲁棒性上仍不及人类。

Details

Motivation: 现有LLM的心智理论评估通常基于被动的问答形式，而LLM在日益自主的智能体形式中部署，需要新的评估方法来衡量其通过行动而非对话来诱导他人信念状态的能力，这种非对话式规划心智理论能力对用户助手交互、教学等场景至关重要，但也可能带来操纵或虚假信息风险。

Result: 在NCP-ExploreToM框架的600个任务实例中，GPT-5在智能体设定下成功率约为80%，是唯一表现优于人类参与者的模型；所有模型（包括人类）在诱导真实信念状态的任务上表现均优于诱导错误信念状态的任务。

Insight: 创新点在于将心智理论评估从被动问答扩展到主动规划与行动领域，提出了NCP-ExploreToM这一新颖评估框架；客观来看，该研究揭示了LLM在非对话式社会推理任务上的新兴能力，并强调了基于智能体的评估对于理解自主社会智能体的安全性与对齐性的必要性，同时发现模型在诱导真实信念上表现更好，这对对齐努力是一个积极信号。

Abstract: Theory of Mind (ToM) benchmarks for Large Language Models (LLMs) typically rely on passive question-answering formats, but the deployment of LLMs in increasingly agentic and autonomous forms demands new evaluations. In this paper we evaluate an agent’s ability to induce specific belief states in other agents by taking actions rather than using conversational persuasion, a capability we call Non-Conversational Planning ToM (NCP-ToM). NCP-ToM is likely to be essential for many agent use-cases, including within user-assistant interactions and pedagogical contexts, but may also present manipulation or misinformation risks. Using a novel framework, NCP-ExploreToM, we subvert the conventional task structure by providing models with a set of belief state goals and requiring them to move objects or direct characters into rooms to achieve their goals. We evaluated six frontier models, including GPT-5, Gemini 2.5 Pro and the Claude 4 series, and a cohort of human participants, across 600 task instances. GPT-5 was successful on approximately 80% of tasks in the agentic setting, and was the only model to outperform human participants on our task, but was still less robust than humans across contexts. We additionally found that all models, like humans, performed better on tasks inducing true belief states than false belief states, which is a positive signal for alignment efforts. These findings highlight emerging social-reasoning capabilities in LLMs for non-conversational task completion and underscore the necessity of agentic evaluations for understanding the safety and alignment of autonomous social agents.

[16] LuxEmo: Expressive Text-to-Speech Corpus for Luxembourgish cs.CLPDF

Nina Hosseini-Kivanani, Sandipana Dowerah

TL;DR: 本文介绍了LuxEmo，一个针对低资源语言卢森堡语的21小时情感语音语料库，包含4种情感类别。论文提出了一种半自动化的语料库构建流程，并结合多种方法对五个情感文本转语音系统进行了基准测试。

Details

Motivation: 当前语音数据集主要集中于广泛使用的语言，而像卢森堡语这样的低资源语言在语音技术研究中代表性不足，因此需要构建相关资源以支持其发展。

Result: 在LuxEmo语料库上，对五种情感TTS系统（包括基于德语的跨语言迁移、多语言支持、卢森堡语适应和非参数韵律迁移）进行了基准测试，并通过客观指标和人工评估来衡量性能。

Insight: 创新点在于为低资源语言卢森堡语构建了一个高质量的情感语音语料库，并提出了一套结合自动检测与人工验证的半自动化语料库构建工作流程，为类似低资源语言的语音研究提供了可借鉴的方法。

Abstract: State-of-the-art speech datasets predominantly focus on widely spoken languages, often overlooking low-resource languages such as Luxembourgish, which remain underrepresented in speech technology research. In this work, we introduce LuxEmo, a 21-hour conversational expressive speech corpus for Luxembourgish with 4 emotion categories. LuxEmo is derived from Radio Télévision Luxembourg (RTL) youth broadcasts, using automated detection followed by human validation. We propose a semi-automatic curation workflow combining voice activity detection, denoising, language identification, LuxASR-based segmentation, automatic emotion prediction, lexical cues, and targeted human review. Additionally, we benchmark five expressive TTS systems covering German-based cross-lingual transfer, multilingual Luxembourgish support, Luxembourgish adaptation, and non-parametric prosody transfer. Performance is evaluated using both objective metrics and human evaluation.

[17] Explicit Fuzzy Logic in the Feed-Forward Layer: Self-Forgetting Quantifiers Discover Legible Grammatical-Licensing Detectors cs.CL | cs.LGPDF

Mark Oskin

TL;DR: 本文提出了一种可解释的模糊逻辑前馈层（NC-FFN），通过显式的模糊集合运算（交集和集合差）替代标准Transformer前馈子层，以增强模型的可解释性。在N位奇偶校验任务中，该方法展现了极高的参数效率；在125M参数的OpenWebText语言建模任务中，其困惑度与GELU基线相当，同时每个单元都携带显式的逻辑形式。通过引入具有可学习遗忘率的序列量词块，模型进一步解决了逻辑形式局部化和训练中退化的问题，并显著改善了语法许可检测能力，使前馈层能够直接解读为语法许可检测器。

Details

Motivation: 标准Transformer的前馈子层（FFN）虽然实现了注意力机制所收集的区分，但其计算过程缺乏可解释性。本文旨在设计一个参数中立、可解释的替代方案，以显式逻辑形式揭示前馈层在语法许可等任务中的计算机制。

Result: 在N位奇偶校验任务中，该方法在浅层深度下实现了最高的参数效率；在125M参数的OpenWebText语言建模基准上，其困惑度与GELU基线相当。在LAMBADA基准上表现略有领先。模型的前馈层单元被解读为语法许可检测器，能够检测比较级、被动分词、否定极性项等许可词，并预测被许可词。

Insight: 创新点在于提出了一种具有显式模糊逻辑运算（特别是带否定能力的集合差运算）的前馈层结构，增强了模型的可解释性。通过引入具有可学习遗忘率的软序列量词，解决了逻辑形式在训练中的局部化和退化问题，使模型能够稳定地学习并迁移语法许可机制，实现了参数中立且可解释的语言建模。

Abstract: A transformer’s feed-forward (FFN) sublayer materializes the distinctions attention gathers, yet gives no account of what it computes. In a parameter-neutral replacement, each hidden unit is an explicit fuzzy set operation on sigmoid-bounded [0,1] memberships: intersection AB and set-difference A(1-B), the latter a bounded positive negation (“A but not B”) that gated/bilinear units lack – a negation-capable FFN (NC-FFN). On N-bit parity they are the most parameter-efficient reasoning basis at shallow depth; at scale (125M, OpenWebText) NC-FFN ties the GELU baseline’s perplexity, every unit carrying explicit logical form. Two limits share one cause: two-operand logic localizes to layer 0 and erodes under training, and the one robust grammatical deficit concentrates in licensing and quantifiers, beyond within-token operators. We resolve both with a small block of sequence quantifiers: a soft existential and a soft proportion, each with a per-unit learned forgetting rate from a sticky init. This recovers the deficit at epoch one (halving the wider epoch-two gap), modestly leads on LAMBADA, and makes the FFN legible: the structure now holds and migrates into depth; the decay un-learns its stickiness (median half-life ~1.5 tokens; zero latch units); and at the semantic layers the units read, without dictionary learning, as grammatical licensing detectors: each fires on a licensor (a comparative, a passive participle, a negative-polarity item) and carries its memory forward to predict the licensed word (than, by, nor). This legibility is localized and free only up to a partition (a fully Boolean FFN diverges in training), but the result is a parameter-neutral, language-model-quality transformer with a readable, interpretable-by-construction grammatical mechanism – an account not just of what a feed-forward layer represents but how it licenses.

[18] DigitalCoach: Communication and Grounding Gaps in Human and Agentic Computer Use Coaching cs.CLPDF

Meng Chen, Anya Ji, Tsung-Han Wu, Tobias Maringgele, David M. Chan

TL;DR: 本文介绍了DigitalCoach数据集，包含72个人类专家-新手计算机使用指导会话，涵盖22,752轮对话和28.1小时的屏幕与输入事件记录，用于评估先进模型是否能有效指导人类使用软件。研究发现，模型在指导方式上与人类存在差异，提供更多直接指令但缺乏解释、错误诊断和知识检查，且视觉上下文基础薄弱，导致学习者被动跟随指令而缺乏深度参与。

Details

Motivation: 研究动机是探索智能体能否有效指导人类使用软件，解决当前模型在计算机使用教学中存在的沟通与基础差距问题。

Result: 在DigitalCoach数据集上的自动评估显示，模型与人类教练在指令风格上存在显著差异；交互评估证实模型指导导致学习者被动跟随，视觉基础能力不足。

Insight: 创新点在于构建了多模态计算机使用指导数据集，揭示了模型在教学中缺乏解释性和视觉基础的问题，为开发协作式主动指导智能体奠定了基础。

Abstract: Agents are increasingly capable of automating software tasks, but can they teach humans how to use software themselves? We introduce DigitalCoach, a multimodal dataset of 72 human expert-novice computer use coaching sessions consisting of 22,752 dialogue turns grounded in 28.1 hours of screen and input event recordings across five software applications. We use DigitalCoach to evaluate whether state-of-the-art models can teach humans how to use computers. Automated evaluation shows that models differ from humans in how they coach: models provide more direct instructions, but fewer explanations, error diagnoses, and knowledge-check questions. When we fix the coaching method, models produce utterances similar to human references yet poorly grounded in visual context. Interactive evaluation confirms that model coaches cause learners to passively follow instructions without deeper engagement and fall short in visual grounding. DigitalCoach lays a foundation for collaborative and proactive computer use coaching agents.

[19] Generative Skill Composition for LLM Agents cs.CLPDF

Xinyu Zhao, Zhen Tan, Vaishnav Tadiparthi, Nakul Agarwal, Kwonjoon Lee

TL;DR: 本文提出了一种名为SkillComposer的方法，用于解决LLM智能体在复杂任务中如何从技能库中自动选择、组合和执行技能序列的问题。该方法将技能组合形式化为结构化预测问题，通过约束自回归解码器联合预测技能子集、数量和顺序，从而提升任务执行效率。

Details

Motivation: 现有技能组合方法（如暴露全部技能库或基于嵌入检索）忽略了技能组合的结构性本质，即需要联合决定使用哪些技能、使用多少以及执行顺序，这三个维度不可分割。这已成为智能体复用技能库时的核心瓶颈。

Result: 在SkillsBench基准测试中，使用GPT-5.2-Codex和Gemini-3-Pro-Preview两个生产级编码智能体进行评估，SkillComposer将任务通过率分别提升了+23.1和+18.2个百分点，超越了top-3检索方法，并在更低提示令牌成本下匹配了黄金技能检索的上界性能。

Insight: 创新点在于将技能组合形式化为结构化序列预测问题，并设计了基于技能标识符的约束自回归解码器，使技能子集、数量和顺序在一次解码中联合生成，自然捕获了技能间的依赖关系。这为模块化智能体的技能复用提供了新的结构化决策框架。

Abstract: Recent LLM agents benefit from skills for solving complex tasks. Skills encapsulate modular packages of procedural knowledge and instructions for performing specialized tasks, such as setting up a sandboxed environment, running a test suite, or refactoring a function across multiple files. As skill libraries grow and become reusable across tasks and domains, selecting an appropriate skill composition has emerged as a central bottleneck. Existing approaches fall into two categories. One exposes the agent’s reasoning to the entire skill collection; the other performs skill retrieval via embeddings or LLM-based rerankers. Both provide useful insights; however, they miss the structural nature of skill composition, which is a joint decision over which skills, how many, and in what order – three dimensions that cannot be decoupled. We formalize this as structured skill composition: given a task and a skill library, predict an executable skill plan that jointly specifies the activated subset, count, and execution order. We propose SkillComposer, which instantiates structured skill composition as task-conditioned skill sequence prediction. SkillComposer uses a constrained autoregressive decoder over skill identifiers, so subset, count, and order emerge jointly from a single decoding pass, and dependencies between successive skills are captured naturally. We build a training set of task-composition pairs from a real, human-curated skill library. We then evaluate SkillComposer along two axes: composition quality on a held-out test set, and downstream task success on SkillsBench across two production-grade coding agents. On GPT-5.2-Codex, Gemini-3-Pro-Preview, SkillComposer raises the pass rate by +23.1, +18.2pp over the no-skill baseline, surpassing top-3 retrieval and matching the gold-skill retrieval upper bound at lower prompt-token cost.

[20] When LLMs Read Tables Carelessly: Measuring and Reducing Data Referencing Errors cs.CL | cs.AIPDF

Yuqing Yang, Qi Zhu, Zhen Han, Boran Han, Zhengyuan Shen

TL;DR: 本文系统评估了大语言模型在处理表格任务时出现的数据引用错误，发现所有测试模型都存在此类错误，并提出了通过批评机制来检测和减少这些错误的方法，显著提升了答案准确性。

Details

Motivation: 尽管大语言模型在表格任务上表现良好，但它们仍会犯数据引用错误，这些错误损害了中间推理步骤的正确性和可靠性，而先前研究对此仅有有限分析。

Result: 在所有测试模型（1.7B到20B参数）中都观察到数据引用错误；通过批评机制，答案准确性最高提升12.0%；训练了一个轻量级4B参数批评模型，平均F1分数达78.2%，能有效辅助更大模型的推理。

Insight: 创新点在于首次系统评估表格数据引用错误，并引入批评机制来检测和减少错误；客观分析认为，轻量级批评模型的设计可提升模型在表格任务中的可靠性和可解释性。

Abstract: While large language models (LLMs) perform well on table tasks, they still make data referencing errors (DREs), i.e., incorrectly citing or omitting table values, despite understanding the table structure. Beyond final-answer accuracy, DREs directly compromise the correctness and reliability of intermediate reasoning steps. Yet prior studies have only offered limited, small-scale analyses. In this work, we present the first systematic evaluation of tabular data referencing errors across different models and tasks. Our results show that DREs occur across all tested models (1.7B to 20B parameters). Furthermore, we demonstrate that incorporating data referencing as a critic significantly improves answer accuracy up to 12.0%, through critic-based filtering and rejection sampling. Finally, we trained a lightweight 4B-parameter critic model that achieves an average F1 score of 78.2% in detecting both in-distribution and out-of-distribution DREs, and effectively assists inference for larger models.

[21] Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs cs.CL | cs.AIPDF

Gabrielle Kaili-May Liu, Avi Caciularu, Gal Yona, Idan Szpektor, Arman Cohan

TL;DR: 本文提出了一种名为’带元认知反馈的强化学习（RLMF）’的新范式，旨在提升大型语言模型（LLMs）的元认知能力，特别是其忠实表达内在不确定性的能力。该方法通过利用模型对其自身表现的判断质量来优化偏好排序（RLMF）和选择高价值训练数据（元认知数据选择），从而校准模型表达的信心分数，并将其映射为自然的语言不确定性。

Details

Motivation: 当前LLMs在元认知方面存在系统性缺陷，例如高置信度地产生幻觉、无法识别知识边界以及错误表达其内部不确定性，这损害了模型的可靠性和可信度。论文旨在通过提升模型自我监控和调节的能力来解决这些问题。

Result: 在忠实校准（FC）任务上进行的广泛实验表明，RLMF在多种任务上实现了可泛化的、最先进的（SOTA）性能，同时保持了准确性。RLMF比标准强化学习（RL）性能提升高达63%，并增强了模型评估和表达自身能力极限的能力。

Insight: 论文的核心创新点在于将模型的元认知表现（即自我判断的质量）作为一种有效的强化学习信号，从而超越了先前内在反馈方法的局限。这为提升LLM的元认知能力、进而改善其能力和对齐性提供了一个有前景的范式。

Abstract: Metacognition is a critical component of intelligence that describes the ability to monitor and regulate one’s own cognitive processes. Yet LLMs exhibit systemic deficiencies in key metacognitive faculties: they hallucinate with high confidence, fail to recognize knowledge boundaries, and misrepresent their internal uncertainty–undermining trustworthiness and reliability. Since monitoring task performance and adapting behavior accordingly are central to metacognition, we posit that models capable of accurately judging their own performance are better positioned to improve it. We operationalize this idea via two novel mechanisms: reinforcement learning with metacognitive feedback (RLMF), a paradigm to refine completion rankings during preference optimization based on the quality of a model’s self-judgments of performance, and metacognitive data selection, which uses similar self-judgments to identify high-value training examples, outperforming naive active learning. We apply these innovations to the problem of faithful calibration (FC), a task that is itself fundamentally metacognitive: the goal is to align expressed with intrinsic uncertainty, difficult even for frontier LLMs. We adopt a two-stage, decoupled approach, first using these methods to calibrate the faithfulness of models’ self-reported confidence scores, then mapping to natural, context-adaptable linguistic uncertainty via targeted output editing. Extensive experiments show RLMF achieves generalizable, state-of-the-art FC on diverse tasks while preserving accuracy. Further, RLMF surpasses standard RL by up to 63% while enhancing models’ ability to assess and express their own capability limits. This positions RLMF as a promising paradigm to enhance LLM metacognition toward improved abilities and alignment, and suggests metacognitive performance as an effective RL signal to overcome limits of prior intrinsic feedback methods.

[22] Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervision cs.CL | cs.AI | cs.LGPDF

Zifan Carl Guo, Laura Ruis, Jacob Andreas, Belinda Z. Li

TL;DR: 本文研究语言模型在训练生成解释时如何实现忠实的内省而非表面模仿。通过使用反事实行为作为监督，发现即使基于固定反事实解释数据集训练，模型生成的解释也能与自身当前行为保持耦合，并跟踪行为变化。

Details

Motivation: 探究语言模型生成预测解释时，何时能实现忠实内省而非仅模仿表面模式，以解决解释忠实性问题。

Result: 在多个任务（如奉承和拒绝）中，模型基于固定反事实解释训练后，其解释与自身行为的耦合度高于训练目标，且能跟踪行为变化，对标签噪声具有鲁棒性。

Insight: 提出内省耦合概念，表明固定反事实解释数据集可作为可扩展的后训练信号，使解释自动适应模型行为变化，无需更新监督。

Abstract: When does training language models (LMs) to generate explanations of their predictions yield faithful introspection, rather than superficial imitation? We study LMs trained to explain which features of their inputs influenced their behavior, using models’ counterfactual behavior on modified inputs as supervision. Surprisingly, we find that LMs trained on fixed counterfactual explanations derived from earlier checkpoints of themselves, or even from behaviorally similar models in different families, frequently produce explanations more faithful to their own current behaviors than to those of their training targets. This “introspective” coupling between LM explanations and behaviors occurs when training explanations remain sufficiently correlated with current behaviors over the course of training, even as behaviors themselves shift. We also show that introspective coupling tracks behavior shifts: when explanation training is provided concurrently with other post-training objectives, explanations track those shifts without requiring updated supervision. This phenomenon appears in multiple tasks, including sycophancy and refusal, and is robust to label noise. Overall, our results show that even fixed datasets of counterfactual explanations can provide scalable and generalizable post-training signal for introspection.

cs.CV [Back]

[23] Streaming Gaussian Encoding for 4D Panoptic Occupancy Tracking cs.CV | cs.ROPDF

Maximilian Luz, Thomas Nürnberg, Yakov Miron, Abhinav Valada

TL;DR: 本文提出了一种用于4D全景占用跟踪（4D-POT）的流式高斯编码器，旨在解决现有基于掩码的方法在几何时间一致性上的不足。该方法通过维护一组持久化的潜在高斯查询来表示场景，利用自运动补偿进行传播，并在置信度引导的预算约束下更新，从而实现了表示级别的时间连贯性。

Details

Motivation: 现有的基于掩码的4D-POT方法通常在每个时间步重新计算底层体积表示，这限制了几何时间一致性，尤其是在遮挡和静态场景元素下。本文旨在通过一个持久的体积场景表示来解决这一局限性。

Result: 在Occ3D扩展的nuScenes和Waymo数据集上的大量实验表明，该方法为基于相机的4D-POT建立了新的最先进（SOTA）性能，以可忽略的计算开销提高了跟踪一致性。

Insight: 创新点在于提出了一个流式高斯编码器，将场景建模为一组固定大小的潜在高斯查询，并通过基于深度的监督来塑造高斯不透明度作为可见性的代理，使置信度能够作为持久场景支持的时间聚合度量进行累积。此外，基于预热的多帧训练策略进一步增强了时间连贯性，且该方法与现有基于掩码的流程完全兼容。

Abstract: Camera-based 4D panoptic occupancy tracking (4D-POT) is a promising paradigm for holistic scene understanding from multi-view imagery, enabling joint reasoning about geometry, semantics, and object identities across time. Recent mask-based pipelines achieve strong performance by propagating instance queries across frames. However, their underlying volumetric representations are typically recomputed at each timestep, limiting geometric temporal consistency, particularly under occlusion and for static scene elements. To address this limitation, we propose a streaming Gaussian encoder that maintains a persistent volumetric scene representation for 4D-POT. Our method models the scene as a fixed-size set of latent Gaussian queries that are propagated via ego-motion compensation and refreshed under a confidence-guided budget constraint. Crucially, we shape Gaussian opacities through depth-based supervision to serve as proxy for visibility, enabling confidence to accumulate as a temporally aggregated measure of persistent scene support. Together with a warmup-based multi-frame training strategy, this yields representation-level temporal coherence beyond decoder-only tracking. Extensive experiments on Occ3D-extended nuScenes and Waymo establish a new state-of-the-art for camera-based 4D-POT, improving tracking consistency with negligible computational overhead while remaining fully compatible with existing mask-based pipelines. We provide code and models at https://sge.cs.uni-freiburg.de.

[24] AVTok: 1D Unified Tokenization for Holistic Audio-Video Generation cs.CV | cs.MM | cs.SD | eess.ASPDF

Kien T. Pham, I Chieh Chen, Qifeng Chen, Long Chen

TL;DR: 本文提出了AVTok，一种用于整体音视频生成的统一标记化方法。该方法采用基于双流Transformer的架构，通过共享编码器-解码器和模态特定可学习查询，将音视频对编码为紧凑的一维潜在表示，并使用统一码本。为了解决异构信息不平衡问题，设计了分层训练策略。实验表明，AVTok在音视频重建及下游生成任务中表现优异。

Details

Motivation: 现有音视频生成方法多采用双分支设计，存在表示差距且计算资源需求高，本文旨在通过统一标记化解决这些问题。

Result: 大量实验证明，AVTok在音视频重建任务中表现出色，并在音频到视频、视频到音频及类别条件联合音视频生成的下游任务中集成后取得良好效果。

Insight: 创新点在于提出首个用于音视频生成的统一标记化方法，通过双流Transformer架构和分层训练策略有效处理模态差异，为构建统一多模态大模型提供了潜在方向。

Abstract: Audio-video generation has recently gained unprecedented research attention, aiming to synthesize high-quality sounding video content with fine-grained synchronization and semantic alignment between the auditory and visual components. The preceding methods predominantly adopt a dual-branch design with separate tokenization and generation modules per modality, neglecting the representation gap while necessitating intensive computational resources for proper training. Inspired by recent advancements in one-dimensional visual tokenization, we present \textbf{AVTok}, a novel unified tokenizer designated for holistic audio-video generation. AVTok features a dual-stream transformer-based architecture with shared encoder-decoder and modal-specific learnable queries to efficiently and effectively encode an audio-video pair into a compact one-dimensional latent representation with a unified codebook. To cope with the heterogeneous information imbalance that hinders AVTok from exploiting aligned audio-visual information, we devise a hierarchical training strategy to progressively realize reconstruction capabilities for each modality. Extensive experiments demonstrate that AVTok excels both in audio-video reconstruction and when integrated into downstream pipelines for audio-to-video, video-to-audio, and class-conditional joint audio-video generation. AVTok paves the way for the challenge of joint audio-video tokenization and provides a potential direction to build unified large multimodal models for audio-video generation.

[25] Simple Supervision Is Hard to Beat: A Bitter Lesson from Sparse Target Labels in Domain-Adaptive Object Detection cs.CVPDF

Lijun Zhang, Ruinian Xu, Mudit Agrawal

TL;DR: 本文研究了在稀疏目标标签下的无源域自适应目标检测问题。作者提出了一种名为RTSM的简单方法，通过在有标签的少量目标图像上使用监督检测损失来增强模型，同时保持原有的无标签自适应分支不变。实验表明，RTSM在多种设置下均能显著提升纯无源域自适应方法的性能，而更复杂的稀疏标签反馈机制带来的额外增益有限。

Details

Motivation: 在无源域自适应目标检测中，当存在少量均匀采样的目标域标注图像时，如何有效利用这些稀疏标签来提升自适应性能是一个关键问题。

Result: 在四个SFDA-OD方法、两个目标检测器、多个自适应任务以及1%到10%的目标标签预算下，RTSM将纯SFDA方法的性能提升了1.7到18.3 AP50。相比之下，十种稀疏标签反馈插件带来的增益有限且依赖于具体方法。

Insight: 核心创新点在于提出了RTSM这一简单而有效的锚点方法，它通过直接监督损失利用稀疏标签，避免了复杂反馈机制的设计。客观来看，论文揭示了在稀疏标签设置下，简单的监督信号往往比复杂的自训练引导策略更有效，这一发现对领域自适应研究具有重要启示。

Abstract: Source-free domain adaptive object detection adapts a source-trained detector to an unlabeled target domain, typically through teacher-student self-training with pseudo-labels. We revisit this setting when a small, uniformly sampled subset of target images is labeled. We introduce Random-Target Supervised Mixing (RTSM), a simple anchor that incorporates these annotations through a supervised detection loss while leaving the original unlabeled adaptation branch unchanged. Across evaluations spanning four SFDA-OD methods, two object detectors, multiple adaptation tasks, and target-label budgets from 1% to 10%, RTSM consistently improves pure SFDA by 1.7 to 18.3 AP50. We then examine whether the same annotations can provide further gains by steering unlabeled self-training. To this end, we evaluate ten sparse-label feedback plugins covering pseudo-label selection, object completion, and optimization control, which yield limited and method-dependent gains over RTSM. These results reveal a bitter lesson for sparse-label SFDA-OD: simple supervision is hard to beat. RTSM therefore provides a simple yet effective anchor for sparse-label SFDA-OD.

[26] SyncCache: Exploiting Asymmetric Dynamics for Fast Audio-Driven Portrait Animation cs.CV | cs.SD | eess.ASPDF

Juncheng Ma, Yuxuan Du, Yanan Sun, Zhening Xing, Changlin Li

TL;DR: 本文提出SyncCache，一种针对基于扩散变换器（DiT）的音频驱动肖像动画的无训练缓存加速方法。该方法通过利用音频驱动动画中固有的空间和模态不对称动态，优先缓存和重用动态人体区域的高频变化，同时轻量级地重新计算音频块以保持精确的唇形同步，从而在保持近乎无损视觉质量和音频对齐的同时，实现了显著的推理加速。

Details

Motivation: 现有的无训练扩散缓存方法主要针对文本条件生成设计，忽视了音频驱动肖像动画中固有的空间（动态人体区域与静态背景）和模态（音频与视觉）不平衡问题，导致基于DiT的动画模型计算成本高、推理延迟大。

Result: 在HunyuanVideo-Avatar和Wan-S2V基准测试上，SyncCache分别实现了高达4.12倍和3.75倍的加速，同时保持了近乎无损的视觉保真度和精确的音频对齐，取得了优越的速度-质量权衡。

Insight: 创新点在于明确利用不对称动态：通过空间不对称探测优先处理动态人体区域的误差敏感性，以及通过模态解耦缓存重用稳定的块间残差并持续轻量级重算音频块以保持唇同步。此外，将内存自适应的缓存选择公式化为离线动态规划问题，避免了在线开销。

Abstract: Diffusion Transformers (DiTs) have significantly advanced audio-driven portrait animation, but their high computational cost leads to substantial inference latency. Although training-free diffusion caching accelerates inference significant, existing methods are primarily developed for text-conditioned generation and overlook the spatial and modality imbalances inherent in audio-driven portrait animation. In this paper, we propose SyncCache, a training-free caching acceleration method tailored for DiT-based portrait animation that explicitly exploits asymmetric dynamics. Specifically, high-frequency dynamics driven by audio conditions and concentrated in human regions are more challenging and critical to cache and reuse than the low-frequency visual background in portrait animation. First, we introduce Spatially-Asymmetric Probing to prioritize error sensitivity in dynamic human region. Second, through Modality-Decoupled Caching, we bypass heavy DiT block by reusing stable inter-block residuals, while continuously recomputing lightweight audio blocks to preserve precise lip synchronization. Furthermore, we introduce a cache ratio to control cache capacity and formulate memory-adaptive cache selection as an offline dynamic programming problem without online overhead. Extensive experiments demonstrate that SyncCache achieves superior speed-quality trade-offs, delivering up to 4.12x acceleration on HunyuanVideo-Avatar and 3.75x on Wan-S2V with near-lossless visual fidelity and precise audio alignment.

[27] The Label Imitation Game: Turing Test Network for Zero-Shot Pseudo-Label Pruning cs.CV | cs.AI | cs.LGPDF

Brent A. Griffin, Jason J. Corso

TL;DR: 该论文提出了一种名为标签模仿游戏（LIG）的框架，用于解决基础模型零样本伪标注中的幻觉错误问题。其核心是训练一个任务无关的图灵测试网络（TTN），通过对抗性审问的方式，在数据集范围内评估并修剪伪标签，而无需额外监督或重新训练。

Details

Motivation: 动机在于解决基础模型进行零样本推理生成伪标签时，标准阈值方法无法有效过滤幻觉错误，从而损害下游任务性能的问题。

Result: 在四个不同数据集上的实验表明，该方法能持续提升三种SOTA视觉语言模型的标签准确性，在目标检测任务上，F1分数提升最高达44%，并实现了独特的零样本任务迁移能力。

Insight: 创新点在于将伪标签修剪形式化为对抗性审问游戏，并证明学习到的语义上下文逻辑可以替代空间几何验证，成为一种鲁棒的、可跨任务迁移的零样本伪标签质量评估机制。

Abstract: Foundation model pseudo-labeling - labeling data strictly via zero-shot inference - enables massive scale, but performance is undermined by hallucinations that evade standard thresholds. To eliminate these errors, we introduce the Turing-inspired Label Imitation Game (LIG), a framework that formalizes pseudo-label pruning as an adversarial interrogation. Rather than filtering labels via isolated thresholds, we use the LIG to train a Turing Test Network (TTN), a task-agnostic “judge” that evaluates candidate pseudo-labels within a dataset-wide context. Experiments across four diverse datasets demonstrate the TTN’s robustness, consistently enhancing label accuracy for three state-of-the-art vision-language models without costly supervision or retraining. Crucially, we demonstrate that learned semantic-contextual logic is a robust alternative to spatial-geometric verification, enabling a unique zero-shot task transfer capability - a TTN trained strictly on image classification datasets can effectively prune complex object detection pseudo-labels. This pruning yields F1-score gains of 28% for the worst-performing baseline categories and 44% with task-specific fine-tuning. Significantly, we also observe Category Revival, where the TTN pruning “detoxifies” the training signal for downstream models and enables them to recover from zero recall on transfer-vulnerable classes. The pre-trained TTN models and code are available at https://github.com/voxel51/ttn.

[28] GRAPE: Graph-Augmented Prototype Explanations for Interactive Medical Image Diagnosis cs.CVPDF

Rasul Khanbayov, Hasan Kurban

TL;DR: GRAPE是一种用于交互式医学图像诊断的图增强原型解释架构，旨在解决基于原型的医学图像分类器存在的三个临床限制：将发现视为独立、隐式放大不安全的医生反馈以及需要完全重新训练以添加新发现。该架构通过图注意力任务头建模解剖概念共现、概念不匹配安全检查机制以及开放词汇原型锚定技术，在保持低延迟的同时提升了诊断性能和安全性。

Details

Motivation: 解决现有基于原型的医学图像分类器在临床应用中存在的三个关键问题：忽略解剖概念间的关联性、无法有效识别医生标注错误以及难以灵活扩展新发现类别。

Result: 在TBX11K数据集上，图注意力任务头将宏F1分数提升了13.8个百分点；概念不匹配安全检查机制能以零额外推理成本捕获85%的错误标注（MC-Dropout仅为51%）；在NIH ChestX-ray14和TBX11K上，开放词汇原型锚定分别实现了全监督定位精度和2.6倍于端到端基线的病灶定位性能，整体延迟仅增加约1毫秒。

Insight: 创新性地将图注意力网络引入原型解释框架以建模医学概念间的结构关系；首次在基于原型的医学分类器中设计了实时标注错误检测机制；通过视觉-文本对齐实现了仅需单样本即可扩展新发现类别的开放词汇能力，为可解释医疗AI提供了模块化安全增强方案。

Abstract: Prototype-based medical image classifiers present three clinical limitations: they treat findings as independent, silently amplify unsafe physician feedback, and require full retraining whenever a new finding is needed. We present GRAPE (Graph-Augmented Prototype Explanations), a unified architecture that addresses all three challenges. First, a Graph Attention Task Head models anatomical concept co-occurrence, boosting macro-F1 by +13.8,pp over the prototype baseline on TBX11K. Second, a Concept-Mismatch Safety Check - the first such mechanism in prototype-based medical classifiers - warns when the model’s dominant finding inside a doctor-drawn region conflicts with the claimed label, catching 85% of erroneous annotations versus 51% for MC-Dropout with no extra inference cost. Third, Open-Vocabulary Prototype Anchoring aligns visual prototypes to clinical text, allowing a new finding to be added from a single labeled image without modifying any other component. On NIH ChestX-ray14, one Effusion example recovers full-supervision localization accuracy; on TBX11K, prototype maps achieve 2.6x better lesion localization than end-to-end baselines. All three capabilities add only +1~ms latency at interactive batch size. The project page is https://github.com/KurbanIntelligenceLab/GRAPE.

[29] No Adaptation Without Observation: Observability-Constrained Test-Time Prompt Tuning for LiDAR Semantic Segmentation cs.CVPDF

Linlian Jiang, Wentao Ju, Sadman Rakib Pinon, Jianwei Xian, Zhixiang Chi

TL;DR: 本文提出一种用于激光雷达语义分割的几何约束测试时提示调优框架，以应对实际部署中因感知条件变化导致的性能下降问题。该方法通过深度一致性光束终止点和邻域支持来估计每个位置的感知可靠性，并以此重新加权空间监督，同时将适配过程限制在轻量级提示适配器中，以避免不可靠区域扰动全局共享表示。

Details

Motivation: 激光雷达语义分割在真实世界部署中常因感知条件变化而性能下降，而重新收集标注数据不切实际。现有测试时适配方法在激光雷达数据上直接应用时，由于伪标签可靠性在空间上存在异方差性，可能导致不稳定梯度并破坏适配过程。

Result: 在标准激光雷达基准测试上的实验表明，该方法在无需额外标注的情况下，能够提升部署变化下的适配稳定性和分割性能。

Insight: 创新点在于引入几何感知的可靠性估计来加权空间监督，并采用轻量级提示适配器与空间门控机制来隔离不可靠区域的影响，同时通过时间平滑的原型对齐策略积累可靠语义证据以稳定在线更新。

Abstract: LiDAR semantic segmentation often degrades under real-world deployment due to evolving sensing conditions, while collecting new annotations for retraining is impractical. Test-time adaptation (TTA) updates model parameters online using pseudo-label supervision, but directly applying standard TTA strategies to LiDAR data is challenging. Because pseudo-label reliability is spatially heteroscedastic under range-dependent sparsity and occlusion, uniform updates on globally shared parameters can inject unstable gradients and destabilize adaptation. We propose a geometry-constrained test-time prompt tuning framework for LiDAR semantic segmentation. Our method estimates per-location sensing reliability from depth-consistent beam terminations and neighborhood support, and uses it to reweight spatial supervision. Adaptation is confined to lightweight prompt adapters inserted into a frozen backbone, with spatial gating to prevent unreliable regions from perturbing globally shared representations. A temporally smoothed prototype alignment strategy further stabilizes online updates by accumulating reliable semantic evidence over time. Experiments on standard LiDAR benchmarks demonstrate improved adaptation stability and segmentation performance under deployment variations without additional annotations.

[30] Learning Where to Look: A Reinforcement Learning Framework for Robust Micro-Ultrasound Prostate Cancer Detection cs.CV | cs.AI | cs.LGPDF

Mohammad Mahdi Abootorabi, Sina Namazi, Armin Saadat, Lyuyang Wang, Obed Dzikunu

TL;DR: 本文提出了一种名为Prost-RL的强化学习框架，用于微超声（μUS）前列腺癌检测。该方法将检测任务重新定义为空间感知的策略驱动推理问题，通过先学习‘看哪里’再进行解码，生成可解释的空间注意力图作为软提示，用于预测癌症可能性热图和图像级分类。

Details

Motivation: 微超声（μUS）是一种新兴的前列腺癌检测成像方式，但准确识别可疑组织高度依赖临床经验，导致观察者间差异大。现有机器学习方法面临监督信号稀疏（仅有核心级病理结果，缺乏像素级标注）和严重类别不平衡的挑战。

Result: 在一个包含5个临床站点、693名患者、6607个活检核心的数据集上，Prost-RL在核心级检测上达到了79.0±3.5 AUROC，在80%特异性下灵敏度为64.6±6.3%，优于最强基线（AUROC提升2.1，灵敏度提升4.5个百分点）；在临床显著癌症分类上达到79.3±5.8 AUROC。

Insight: 主要创新点包括：1) 将检测任务重构为强化学习策略驱动的空间注意力学习问题；2) 提出了自适应策略优化（APO）以稳定混合监督-强化学习训练；3) 设计了结合对称交叉熵和负熵正则化的噪声鲁棒目标函数，以缓解弱标签噪声并鼓励精确的定位。该方法提供了透明、基于空间位置的证据，增强了模型的可解释性。

Abstract: Micro-ultrasound ($μ$US) is a new, emerging, and promising imaging modality for prostate cancer (PCa) detection, but accurate identification of suspicious tissue remains highly dependent on clinical experience, leading to substantial inter-observer variability. Machine-learning assistance can reduce this variability; however, training reliable deep models is challenging because supervision is sparse and noisy – typically limited to core-level histopathology outcomes (e.g., cancer grade and its percentage in a biopsy core) without pixel-level lesion annotations and under severe class imbalance. We introduce Prost-RL, which reframes $μ$US PCa detection as a spatially aware, policy-driven inference problem by learning where to look before decoding. Prost-RL integrates a lightweight reinforcement-learning policy into a foundation-model encoder-decoder to generate interpretable spatial attention maps that act as soft prompts for both cancer-likelihood heatmap prediction and image-level classification. We further propose Adaptive Policy Optimization (APO) to stabilize hybrid supervised-RL training and a noise-robust objective combining symmetric cross-entropy with negative-entropy regularization to mitigate weak-label noise and encourage sharp localization. On a cohort of 6,607 biopsy cores from 693 patients across five clinical sites, Prost-RL achieves $79.0\pm3.5$ AUROC with $64.6\pm6.3$% sensitivity at 80% specificity for core-level detection (+2.1 AUROC and +4.5 sensitivity points over the strongest baseline), and $79.3\pm5.8$ AUROC for clinically significant cancer classification. The learned policy highlights biopsy-aligned regions, providing transparent, spatially grounded evidence alongside quantitative risk predictions. Code is available at: https://github.com/DeepRCL/Prost-RL.

[31] Auditing Generalization in AI-Generated Video Detection: A Six-Control Protocol and the VidAudit Toolkit cs.CVPDF

Mert Onur Cakiroglu, Zhihe Lu, Mehmet Dalkilic, Hasan Kurban

TL;DR: 本文提出了一种包含六项标准控制的审计协议（VidAudit Toolkit），用于评估AI生成视频检测方法的泛化能力，并揭示了现有基准测试中未受控制的因素会夸大泛化性能。通过该协议，作者重新评估了六个代表性特征源，发现简单分类器性能崩溃，而WaveRep检测器在分布外泛化中表现优异。

Details

Motivation: 现有AI生成视频检测基准（如GenVidBench和AIGVDBench）的评估协议存在未受控制的混杂因素，导致报告的泛化性能被夸大，缺乏对方法真实泛化能力的可靠衡量。

Result: 在审计协议下，简单三特征分类器的LOGO AUC从0.998降至0.529（接近随机水平）；WaveRep检测器在分布外LOGO AUC达到0.996，且真实视频间一致性处于随机水平；在可部署的FPR为0.1%时，多个高AUC方法的召回率降至个位数，排行榜顺序发生变化。

Insight: 创新点包括提出六控制审计协议以消除评估混杂因素，推荐使用审计元组（AUC、高于基线余量、操作点召回率和校准）替代单一指标，以及引入跨基底特征融合（XSFF）来增强检测器的互补性和鲁棒性。

Abstract: AI-generated video detection benchmarks such as GenVidBench and AIGVDBench are the de facto leaderboards, yet most evaluation protocols leave uncontrolled confounds that can inflate reported generalization. As an existence proof, a three-feature clip-length classifier reaches a leave-one-generator-out (LOGO) AUC of 0.998 on GenVidBench under unaudited evaluation, while measuring nothing about motion. A 20-paper survey finds none applying all six standard controls that would catch this, so we combine them into an audited protocol and apply it to six representative feature sources (three published detectors and three repurposed signal sources), re-running it cross-dataset on AIGVDBench. The audit both debunks and certifies: the trivial classifier collapses to near chance (0.529), a CLIP baseline is caught carrying dataset identity, and the 2025 forensic detector WaveRep clears the floor at out-of-distribution LOGO AUC 0.996 with chance-level real-vs-real coherence. At a deployable FPR of 0.1%, multiple high-AUC methods fall to single-digit recall and the leaderboard order changes, so we recommend an audited tuple (AUC, above-floor margin, operating-point recall, and calibration) over a single number. As a white-box positive control, we add TemporalSpec (codec motion vectors); via cross-substrate feature fusion (XSFF), a second substrate adds genuine complementarity that survives the audit. We release VidAudit, to our knowledge the largest unified and audited detector collection for this task, providing 14 detectors behind one plugin API, a leaderboard, and Croissant metadata, available at https://github.com/KurbanIntelligenceLab/vidaudit. Together, the protocol and toolkit move evaluation from leaderboard rank toward whether a result measures what it claims.

[32] Dense Structural Priors for Sparse Functional Landmark Localization in Surgical Videos cs.CVPDF

Chenyan Jing, Hao Ding, Lalithkumar Seenivasan, Jacob M. Delgado López, Mathias Unberath

TL;DR: 本文提出了一种利用SAM 3作为结构先验的轻量级细化框架，用于手术视频中稀疏功能标志点（如器械尖端和锚点）的定位。该方法通过零样本点提示生成器械级结构掩码，无需像素级标注，并将掩码与视觉及热图特征融合以优化预测。

Details

Motivation: 现有视觉基础模型（如SAM 3）能提供可迁移的对象级结构，但其分割输出未明确编码与手术动作相关的功能语义（如剪切、抓握所需的器械尖端或锚点），因此需要一种无需手动掩码标注、能利用结构先验进行精确定位的方法。

Result: 在来自YouTube、Cholec80等数据集的7,867个手术视频片段上评估，模型在无像素级掩码标注训练的情况下，实现了尖端定位72.4%和锚点定位58.0%的总体F1分数。

Insight: 创新点在于利用SAM 3的零样本能力生成非预言掩码作为结构先验，并通过预测驱动的先验细化和辅助监督提供有效的中间结构指导，避免了直接将掩码施加于热图目标导致的偏向工具宽泛区域的问题，为动作依赖的标志点预测提供了新思路。

Abstract: Vision foundation models such as SAM 3 can provide transferable object-level structure across diverse surgical video conditions, but segmentation outputs do not explicitly encode the action-conditioned semantics that define functional surgical landmarks. Estimating instrument extent and geometry differs from localizing the tip or anchor relevant to clipping, grasping, or dissecting. We investigate vision foundation model-enabled sparse action-aware landmark localization, using zero-shot, point-prompted structural masks to provide dense instrument-level context without manual pixel-level mask annotations. We propose a lightweight refinement framework that uses SAM 3 as a structural prior. A coarse multi-frame network predicts tip and anchor prompts, generating non-oracle masks that are fused with visual and heatmap features to refine functional landmark predictions. We compare direct mask-augmented supervision, prediction-derived mask-prior refinement, and auxiliary mask supervision to examine how vision foundation model-derived structure should enter a precision-oriented localization system. Experiments on 7,867 clips from 60 surgical videos spanning YouTube, Cholec80, HeiChole, SurgVU, and CRCD evaluate the approach under heterogeneous conditions. Without manual pixel-level mask annotations for training, the proposed model achieves overall F1 scores of 72.4% for tip and 58.0% for anchor localization. Directly imposing masks on heatmap targets biases learning toward broad tool regions, whereas prediction-derived priors and auxiliary supervision provide effective intermediate structural guidance for action-dependent landmark prediction.

[33] Dual Sparse Aggregation Transformer for Multispectral Object Detection cs.CVPDF

Wencong Wu, Xiuwei Zhang, Hanlin Yin, Hongxi Zhang, Yanning Zhang

TL;DR: 本文提出了一种名为双稀疏聚合Transformer（DSAFormer）的新方法，用于多光谱目标检测。该方法通过空间稀疏多头交叉注意力和通道稀疏多头交叉注意力机制，选择性地捕获跨模态互补信息并消除无关交互，同时引入可学习的加法融合块来有效融合多模态特征，从而提升检测性能。

Details

Motivation: 现有的基于Transformer的多光谱检测方法在计算相似性时倾向于使用所有可用token，导致来自无关区域的冗余信息交互，从而降低了检测性能。本文旨在克服这一挑战，通过引入稀疏性来更有效地建模跨模态关系。

Result: 在MFAD、FLIR、M³FD和LLVIP四个公共数据集上的大量实验结果表明，所提出的DSAFormer方法优于最先进的方法，实现了更好的检测性能。

Insight: 核心创新点在于提出了双稀疏Transformer（DSFormer），它结合了空间级和通道级的稀疏交叉注意力机制，以选择性地捕获互补信息并抑制冗余。此外，可学习的加法融合块（LAFB）通过特征重加权来聚合模态内和模态间的特征信息，也是一种有效的多模态融合策略。

Abstract: Transformer-based approaches have obtained excellent performance in multispectral object detection tasks due to their ability to model long-range dependencies and capture complementary information. However, previous transformer-based multispectral detection methods tend to use all available tokens for similarity calculation, which results in redundant information interaction from irrelevant areas, leading to degraded detection performance. To overcome this challenge, we propose a novel Dual Sparse Aggregation Transformer (DSAFormer) for multispectral object detection, which consists of a Dual Sparse Transformer (DSFormer) and a Learnable Addition Fusion Block (LAFB). Specifically, the DSFormer is designed to exploit and boost cross-modal complementary information, thereby improving detection performance. It incorporates three key components: A Spatial Sparse Multi-Head Cross-Attention (SSMHCA) mechanism selectively captures cross-modal relationships at the spatial level by reserving only the high query-key similarity scores, eliminating irrelevant interactions. A Channel Sparse Multi-Head Cross-Attention (CSMHCA) mechanism performs similar sparse calculations at the channel level to enhance feature representation and filter out low matching query-key. A Multi-Scale Feature Refinement Layer (MSFRL) is developed to aggregate hierarchical features and suppress redundant information. To effectively fuse multimodal features, the LAFB is introduced to aggregate intramodal and intermodal feature information by feature reweighting. Extensive experimental results have demonstrated that our proposed DSAFormer achieves better detection performance against state-of-the-art methods on four public datasets, including the MFAD, FLIR, M$^3$FD, and LLVIP. The source code of our DSAFormer will be released at https://github.com/WenCongWu/DSAFormer.

[34] WarpI2I: Image Warping for Image-to-Image Translation cs.CVPDF

Shen Zheng, Anurag Ghosh, Gaurav Parmar, Srinivasa Narasimhan

TL;DR: 本文提出WarpI2I，一种用于图像到图像（I2I）转换的显著性引导的扭曲-反扭曲框架。该方法通过扭曲输入图像，将空间表示重新分配至显著区域，从而在保持潜在分辨率不变的情况下更好地保留结构细节。该框架与模型无关，无需修改架构，计算开销可忽略，并可自然扩展到视频处理。

Details

Motivation: 现有紧凑型潜在扩散模型（LDMs）在编码时将高分辨率输入压缩至空间下采样的潜在空间，导致难以保留细粒度结构。本文旨在解决I2I转换中结构细节丢失的问题。

Result: 在人体重光照、驾驶场景重光照和场景转换任务上的实验表明，该方法在结构保持、光照忠实度和图像质量方面均有提升，具有良好的时间稳定性。

Insight: 创新点在于提出了一种简单且模型无关的显著性引导扭曲-反扭曲框架，以及一种基于外绘的高质量合成数据生成流程，用于图像重光照的配对数据生成。该方法在不增加潜在分辨率或计算开销的情况下有效提升了细节保留能力。

Abstract: Image-to-image (I2I) translation has achieved strong results in tasks like human relighting and driving scene translation using latent diffusion models (LDMs). However, compact LDMs often struggle to preserve fine-grained structures because the encoder compresses high-resolution inputs into a spatially downsampled latent space. To address this issue, we propose a simple saliency-guided warp-unwarp framework that reallocates spatial representation toward salient regions before encoding, enabling better preservation of structural details without increasing latent resolution. The warped image is processed by the original diffusion model and then mapped back via an inverse warp. In addition, we propose a simple and efficient outpainting-based synthetic data generation pipeline to produce high-quality paired data for image relighting. Our method is model-agnostic, requires no architectural modification, and introduces negligible computational overhead. Experiments on human relighting, driving scene relighting, and translation demonstrate improved structural preservation, lighting faithfulness, and image quality, with our framework extending naturally to video via frame-by-frame application with good temporal stability. Project Webpage: https://shenzheng2000.github.io/WarpI2I.github.io

[35] Learning Video Dynamics with Predictive Differentiable Rendering cs.CV | cs.AIPDF

Yujin Tang, Tian Zhou, Xin Lin, Cheng Tan, Yifan Hu

TL;DR: 本文提出了一种名为预测可微分渲染（PDR）的新型端到端视频预测范式，通过引入基于2D高斯表示的轻量级适配器PredGS，弥合了离散像素空间与连续表示之间的鸿沟。该方法结合L1和SSIM损失进行优化，有效克服了传统MSE损失导致的预测模糊问题，在多个真实世界基准测试中实现了更优的细节保留和预测精度。

Details

Motivation: 现有确定性视频预测模型在离散像素空间中运行，主要使用像素级均方误差（MSE）进行优化，这通常导致预测结果过度平滑且缺乏细粒度视觉细节。论文旨在解决这一问题，以更准确地预测高保真度的未来世界。

Result: 在TaxiBJ、WeatherBench、KTH和Human3.6M等多个真实世界基准测试上进行的大量实验表明，PDR方法在细节保留、视觉保真度和预测准确性方面持续超越现有方法，达到了SOTA水平。

Insight: 主要创新点在于提出了预测可微分渲染（PDR）范式，并引入了基于2D高斯表示的轻量级、即插即用适配器PredGS，该适配器可与现有像素空间预测器无缝集成，以可忽略的计算开销显著改善空间细节保留。此外，开发了支持任意通道的CUDA加速可微分2D高斯渲染器predgsplat，渲染速度比基线快10倍，并通过结合L1和SSIM损失优化克服了MSE的模糊倾向。

Abstract: How to accurately predict a high-fidelity future world? While the visual world is inherently continuous, existing deterministic video prediction models operate in discrete pixel space and are mainly optimized with pixel-wise mean squared error (MSE), which often leads to over-smoothed predictions and a lack of fine-grained visual details. To address these limitations, we propose Predictive Differentiable Rendering (PDR), a novel end-to-end video prediction paradigm that bridges the gap between discrete and continuous representations. Inspired by recent progress in 3D reconstruction with 3D Gaussian Splatting, we introduce PredGS, a lightweight and plug-and-play adapter based on 2D Gaussian representation, which could be seamlessly integrated with existing pixel space predictors, significantly improving spatial detail preservation with negligible computational overhead. Furthermore, we develop predgsplat, a CUDA-accelerated differentiable 2D Gaussian renderer supporting arbitrary channels. Each Gaussian is defined by 5 + C learnable parameters (position, scale, rotation, and C channel amplitudes) and achieves up to 10x faster rendering than the baseline. Optimized by a combined L1 and SSIM loss, PDR overcomes the inherent blurring tendencies of MSE Loss, significantly enhancing the prediction performance. Extensive experiments on diverse real-world benchmarks, including TaxiBJ, WeatherBench, KTH, and Human3.6M, demonstrate that PDR consistently surpasses existing methods, delivering superior detail preservation, visual fidelity, and predictive accuracy.

[36] ADAPT: Attention Dynamics Alignment with Preference Tuning for Faithful MLLMs cs.CV | cs.AI | cs.CL | cs.MMPDF

Zhiyuan Yao, Zheren Fu, Zhixiao Zheng, Jiajun Li, Yi Tu

TL;DR: 本文提出ADAPT框架，通过干预多模态大语言模型（MLLMs）生成过程中的文本到图像交叉注意力动态来减少幻觉问题。该方法包含三个核心组件：基于早期解码优化的交叉注意力视觉锚点、在线检测和纠正注意力漂移的注意力监督推理机制，以及引导模型偏好于视觉基础响应的视觉注意力指导DPO。实验表明，ADAPT能显著降低多种主流模型在多个幻觉基准上的幻觉率，同时保持通用多模态能力。

Details

Motivation: 多模态大语言模型存在严重的幻觉问题，生成与图像不一致的内容。作者发现幻觉的内部特征在于生成过程中文本到图像交叉注意力的渐进退化，导致注意力不集中或偏差等特定失败模式，而现有缓解策略多为结果驱动，未明确针对此失败模式。

Result: ADAPT的每个组件均有助于减少幻觉，完整框架在多个幻觉基准上取得了新的最佳结果，在主流骨干模型上将幻觉率降低了40%-60%，同时保持了通用多模态能力。

Insight: 创新点在于从注意力动态的内部视角出发，通过稳定空间基础的视觉锚点、在线注意力监督和基于偏好的对齐，直接干预交叉注意力的退化过程，为缓解幻觉提供了新的基于注意力的方法。

Abstract: Multimodal Large Language Models (MLLMs) are critically hampered by hallucination, generating content inconsistent with the provided image. In this paper, we identify an internal signature of hallucination: progressive degradation of text-to-image cross-attention during generation, leading to specific failure patterns like unfocused or biased attention. Existing mitigation strategies are largely outcome-driven and do not explicitly target this failure mode. To address this problem, we propose ADAPT (Attention Dynamics Alignment with Preference Tuning), an attention-based framework that intervenes directly on text-to-image cross-attention dynamics. We propose ADAPT with three key contributions: a cross-attention visual anchor refined from early decoding to provide stable spatial grounding, an attention-supervised inference mechanism that detects and corrects attention drift online, and a Visual Attention Guidance DPO that aligns preferences toward visually grounded responses. Experiments show that each component of ADAPT contributes to hallucination reduction, and the full framework achieves new best results across multiple hallucination benchmarks, reducing hallucination rates by 40%-60% across mainstream backbones while preserving general multimodal capabilities. Our work provides an attention-based perspective on mitigating hallucinations by exploring the model’s internal text-to-image cross-attention behaviors. Code is available at https://github.com/yao-ustc/ADAPT

[37] Diffusion-Based Material Regularization for Physics-Based Inverse Rendering cs.CVPDF

Jingwang Ling, Lifan Wu, Feng Xu, Shuang Zhao

TL;DR: 本文提出了一种基于扩散模型的材料正则化方法，用于物理基础的逆向渲染。该方法利用扩散模型的预测作为相似性核进行优化，通过正则化损失惩罚材料在扩散预测恒定区域内的偏差，从而联合重建几何、材料和光照。

Details

Motivation: 物理基础的逆向渲染虽然提供准确的图像形成模型，但严重欠约束，导致光照被烘焙到材料中，重建结果在新视角和光照下泛化能力差；而数据驱动的扩散模型预测的材料虽视觉合理，但通常不满足渲染方程，无法直接用于物理渲染。

Result: 在Synthetic4Relight、Stanford-ORB和DTC-Synthetic数据集上，该方法在重建精度和重光照质量方面显著优于最先进的基线方法。

Insight: 创新点在于将扩散模型预测作为相似性核用于优化正则化，而非直接作为目标材料值，从而桥接了物理渲染与数据驱动模型，实现了高质量、可物理渲染的资产重建。

Abstract: Reconstructing physics-based 3D assets – geometry, materials, and illumination – from multi-view images is a core problem in computer graphics and vision, and a prerequisite for realistic relighting and editing. Physics-based inverse rendering offers an accurate image-formation model, but is severely underconstrained: without strong priors, illumination is baked into materials, and reconstructions generalize poorly to novel views and lighting. Data-driven diffusion models, in contrast, predict visually plausible materials, yet their predictions rarely satisfy the rendering equation and are not directly usable for physics-based rendering. We bridge these two paradigms rather than replacing either. Our key idea is to treat the predictions of a state-of-the-art diffusion model not as target material values but as a similarity kernel for optimization: we introduce a regularization loss that penalizes deviations in the optimized material over surface regions where the diffusion predictions are near-constant, while leaving the optimization free to match the input images. Built on this regularizer, our end-to-end pipeline jointly reconstructs geometry, materials, and illumination, yielding high-quality assets that drop into standard rendering pipelines and relight faithfully. On the Synthetic4Relight, Stanford-ORB, and DTC-Synthetic datasets, our method significantly outperforms state-of-the-art baselines in both reconstruction accuracy and relighting quality.

[38] Hybrid Unet-Transformer Model for Generating Stress and Strain Fields from Composite Geometrics cs.CVPDF

Shrey Patel

TL;DR: 本文提出了一种结合UNet与Transformer的混合架构，用于直接从复合材料微观结构几何图像中预测复杂的应力与应变场分布，作为有限元模拟的高效替代模型。该模型在多种两相复合材料配置（如方形、六边形和三角形镶嵌）及不同边界条件下，对十种应力应变场类型均表现出良好的预测性能。

Details

Motivation: 传统有限元方法在预测分层复合材料微观结构的应力应变场时计算成本高昂，难以大规模应用，因此需要开发高效的数据驱动替代模型来加速物理信息材料设计。

Result: 在多数子数据集上模型预测准确度高，其中周期性镶嵌几何的峰值精度达到R2=0.9991、SSIM=0.9936和MAE=0.0050；在八个评估子数据集中的六个上，归一化像素尺度上的MAE保持在0.05以下。但在不规则方形网格几何（含稀疏软相夹杂）上性能下降，S11法向应力子数据集的R2为0.7735、SSIM为0.7126。

Insight: 创新点在于将UNet的局部特征提取能力与Transformer的全局注意力机制结合，构建了高效的力学场预测替代模型；通过Grad-CAM分析表明模型无需显式结构监督即可关注力学关键区域（如相界、连接处和接触区），但平滑损失图像翻译模型在再现尖锐应力不连续性方面存在已知局限。

Abstract: Accurate prediction of stress and strain fields in hierarchical composite microstructures is critical for physics-informed material design, yet conventional finite element method (FEM) simulations are computationally prohibitive at scale, requiring minutes to days per evaluation. In this work, we propose a hybrid UNet-Transformer architecture that predicts complex mechanical field distributions directly from composite microstructure geometry images, serving as an efficient surrogate for FEM across ten distinct stress and strain field types spanning diverse two-phase composite configurations including square, hexagonal, and triangular tessellations, multiple boundary conditions, and high-resolution geometries. Results demonstrate that the proposed architecture achieves strong predictive performance across the majority of subdatasets, with peak accuracy on periodic tessellation geometries reaching R2=0.9991, SSIM=0.9936, and MAE=0.0050 on the boundary condition subdataset and the triangular tessellation subdataset respectively. Across six of the eight evaluated subdatasets, MAE remains below 0.05 on the normalized [0,1] pixel scale. Encoder attention analysis via Grad-CAM and Grad-CAM++ confirms that the model develops physically meaningful internal representations, localizing attention at mechanically critical regions including phase boundaries, ligament junctions, and indenter contact zones without explicit structural supervision. Performance degrades on irregular square-grid geometries with sparse soft-phase inclusions, with the S11 normal stress subdataset yielding R2=0.7735 and SSIM=0.7126, consistent with the known limitation of smooth-loss image translation models in reproducing sharp stress discontinuities.

Meng Yang, Zizhuo Li, Linfeng Tang, Fan Fan, Jiayi Ma

TL;DR: 本文提出了AnyMatch框架，通过利用大规模单视图图像，低成本生成具有3D几何一致性的多模态训练数据，以解决多模态图像匹配中训练数据稀缺的问题。该方法集成了单目深度估计、3D重投影、基于扩散模型的修复和跨模态图像转换技术，构建了Any-syn合成数据集。实验表明，在Any-syn上微调的匹配网络（如LoFTR、EDM、RoMa）在多模态基准测试中性能显著提升，展现出优异的泛化性和鲁棒性。

Details

Motivation: 多模态图像匹配对于视觉定位和多传感器融合至关重要，但现有真实世界数据集成本高昂、场景多样性有限且存在SfM-MVS流程误差，而合成方法难以保持3D几何一致性或实现逼真外观，因此需要一种低成本、高保真的数据生成方案。

Result: 在Any-syn数据集上微调的匹配网络（如LoFTR、EDM、RoMa）在多模态基准测试中取得了显著的性能提升，表现出优于现有数据训练模型的泛化能力和鲁棒性。

Insight: 创新点在于通过显式3D重投影确保几何标注的严格一致性，避免了SfM-MVS的误差累积，并利用扩散模型和跨模态转换实现外观真实感；框架具有强可扩展性，可通过调整输入和相机参数控制场景多样性和标注难度，为多模态匹配提供了高质量、可扩展的数据合成范式。

Abstract: Multi-modal image matching is essential for visual localization and multi-sensor fusion, but it is hindered by the scarcity of large-scale training data with precise geometric annotations. Existing real-world datasets suffer from prohibitive costs, limited scene diversity, and errors in SfM-MVS pipelines, while synthetic methods struggle to maintain 3D geometric consistency or achieve photorealistic appearance. To address this, we propose AnyMatch, a novel framework that leverages abundant, easily accessible single-view images at minimal cost to generate rich multi-modal training data. AnyMatch integrates monocular depth estimation, 3D reprojection, diffusion-based inpainting, and crossmodal image translation to synthesize multi-view, multi-modal image pairs with 3D geometric fidelity. Crucially, our method provides annotations that strictly adhere to 3D geometric consistency through explicit 3D reprojection, avoiding SfM-MVS error accumulation. Furthermore, AnyMatch offers strong scalability, enabling controllable scene diversity and annotation difficulty via adjustable input and camera parameters. We construct Any-syn, a large-scale synthetic multi-modal dataset using AnyMatch. Experimental results show that matching networks (e.g., LoFTR, EDM, RoMa) fine-tuned on Any-syn achieve substantial performance gains on multi-modal benchmarks, exhibiting superior generalization and robustness compared to models trained on existing data.

[40] Towards Flexible, Natural, Efficient Interaction for Conversational Talking Face Generation cs.CVPDF

Baiqin Wang, Sen Chen, Jiankuo Zhao, Xiangyu Liu, Zhen Lei

TL;DR: 本文提出了InterTalk，一个用于高度交互式对话人脸生成的新型高效框架。该框架旨在解决多轮对话中人脸视频合成的灵活性、自然性和效率三大核心挑战，通过显式建模参与者间的对话动态、引入运动反馈和迭代生成策略，以及解耦面部运动组件来实现更自然的交互行为。

Details

Motivation: 现有方法在平衡对话人脸生成的灵活性（支持任意数量参与者的多轮对话）、自然性（保持连贯动作和恰当的非语言反馈）和效率（实现实时生成和低计算开销）方面存在不足，因此需要一种能够同时满足这三方面要求的解决方案。

Result: 大量实验表明，InterTalk在保持30 FPS实时性能的同时，实现了卓越的交互质量。

Insight: 创新点包括：1）显式建模多轮对话动态以支持任意参与者数量的灵活性；2）引入多参与者运动反馈和迭代生成策略以增强交互自然性；3）将运动解耦为多个面部组件，实现针对性的细化（如精确唇同步和逼真眨眼）；4）构建了新的多人对话数据集并采用基于3D人脸的数据增强方法。

Abstract: Conversational talking face generation has recently attracted increasing attention, aiming to synthesize interactive talking videos where characters speak, listen, and respond dynamically to each other. This task presents three core challenges: 1) Flexibility: enabling multi-round dialogues with an arbitrary number of participants; 2) Naturalness: maintaining coherent motion and appropriate non-verbal feedback throughout the interaction; and 3) Efficiency: achieving real-time generation and low computation overhead for long-term continuous online conversation. Despite recent advances, existing methods still fall short in balancing all three requirements. To bridge this gap, we introduce InterTalk, a novel and efficient framework designed for highly interactive conversational talking face generation. Built upon a motion-based architecture, InterTalk supports real-time conversation synthesis. Our method achieves strong flexibility by explicitly modeling multi-round conversational dynamics among each participant, eliminating constraints on their numbers. To enhance interactivity, we incorporate motion feedback from multiple participants and introduce an iterative generation strategy for more natural behaviors. Besides, we disentangle motion into several facial components, enabling targeted refinements for natural response such as precise lip sync and realistic eye blinking. Finally, we construct a new multi-person conversational dataset and enrich it with 3D face-based data augmentation. Extensive experiments demonstrate that InterTalk achieves superior interaction quality while maintaining real-time performance at 30 FPS.

[41] Anchoring on Reality: Breaking the Pseudo-Target Ceiling in Makeup Transfer cs.CVPDF

Bo Wei, Xianhui Lin, Yi Dong, Zhongzhong Li, Zonghui Li

TL;DR: 本文提出了一种名为ART的两阶段框架，用于解决化妆迁移任务中因缺乏真实配对数据而导致的细节退化、合成伪影和身份漂移问题。该方法通过伪目标初始化建立语义对齐，再通过可微循环将监督从伪目标转向真实参考图像，以重建细节并消除伪影。同时，作者还发布了首个2K分辨率野外化妆肖像数据集MakeupFaces2K。

Details

Motivation: 化妆迁移任务缺乏真实配对训练数据，现有方法依赖弱先验或大规模编辑模型生成的合成伪目标，导致细节丢失、合成伪影和身份漂移，因此需要一种能锚定真实参考的监督机制来提升迁移质量。

Result: 在广泛实验中，该方法在化妆保真度、背景稳定性和身份保持方面表现优异，尤其在复杂化妆风格上效果显著，但未明确提及具体基准或是否达到SOTA水平。

Insight: 创新点包括两阶段框架中的现实锚定细化循环，将监督从伪目标转向真实参考以重建细节；同时，发布了首个高分辨率野外化妆数据集MF2K，为领域提供了宝贵资源。

Abstract: Makeup transfer applies a reference cosmetic style to a source face while preserving its identity and geometry. However, this task is severely hindered by the lack of real paired training data. Current methods rely on either weak priors or synthetic pseudo-targets from large-scale editing models. These paradigms provide suboptimal guidance, often leading to degraded fine-grained details, synthetic artifacts, and identity drift. To this end, we propose Anchoring on Reality Makeup Transfer (ART), a two-stage framework with a reality-anchored refinement cycle. In Stage I, the model is initialized with pseudo-targets to establish basic semantic alignment and global makeup placement. Crucially, Stage II shifts supervision from pseudo-targets to the real reference, reconstructing it from its bare-skin counterpart through a differentiable cycle that penalizes any omitted detail and overrides synthetic artifacts. Furthermore, we introduce MakeupFaces2K (MF2K), the first 2K-resolution in-the-wild makeup portrait dataset comprising 8,573 images. Extensive experiments demonstrate that our method achieves superior makeup fidelity, strong background stability, and robust identity preservation, especially for complex makeup styles.

[42] PiLoT v2: Pixel-to-Orthogonal Map Alignment for Free-view UAV Geo-localization cs.CVPDF

Xinyi Liu, Xiaoya Cheng, Rouwan Wu, Zhaochen Wang, Shen Yan

TL;DR: 本文提出了PiLoT v2系统，用于解决GNSS拒止环境下无人机实时无漂移地理定位问题。该系统通过将参考数据从繁重的3D网格转换为2.5D正交地图（TDOMs和DSMs），并训练一个跨视角特征配准网络来对齐无人机倾斜视角图像与正交地图，从而大幅降低了存储和计算开销。

Details

Motivation: 解决前代PiLoT系统因依赖重型3D网格而带来的巨大存储开销、复杂地图获取以及高昂计算渲染成本的问题，使其难以部署在嵌入式平台上。

Result: 实验结果表明，PiLoT v2在性能上达到甚至超越了其前代Pixel-to-3D方法，同时显著降低了存储和计算成本。

Insight: 创新点在于将地理定位范式从像素到3D配准转变为像素到正交地图配准，并利用新颖的大规模几何标注数据集训练跨视角特征网络。此外，将机载传感器先验（重力方向和单点激光测距）直接集成到姿态优化流形中，以增强对跨视角视觉退化的鲁棒性。

Abstract: Real-time, drift-free UAV geo-localization is essential for autonomous missions in GNSS-denied environments. The pioneering system, PiLoT, achieves high precision via Neural Pixel-to-3D Registration, aligning UAV video streams with a single rendered reference view from 3D meshes. However, its reliance on heavy 3D meshes incurs massive storage overheads, complex map acquisition, and significant computational rendering costs, severely hindering deployment on embedded platforms. To address these bottlenecks, we propose PiLoT v2, a lightweight yet robust evolution that shifts the paradigm to direct pixel-to-orthogonal map registration for free-view UAV geo-localization. By leveraging True Digital Orthophoto Maps (TDOMs) and Digital Surface Models (DSMs) as the reference substrate, PiLoT v2 replaces GPU-intensive 3D rendering with a highly efficient, CPU-friendly map cropping operation. To bridge the severe geometric discrepancy between these 2.5D orthogonal crops and free-view oblique UAV imagery, we train a cross-view feature registration network using a novel, large-scale geometrically annotated dataset. Furthermore, we integrate onboard sensor prior–specifically gravity direction and single-point laser rang–directly into the pose optimization manifold to enhance robustness against cross-view visual degradation. Experimental results demonstrate that PiLoT v2 achieves performance comparable to, or even exceeding, its Pixel-to-3D predecessor, while offering drastically lower storage and computational costs.

[43] TaxoMIL: Taxonomy-Constrained Learning for Hierarchical Whole Slide Image Analysis cs.CVPDF

Chaeyeon Lee, Khang Nguyen Quoc, Jinsol Song, Yosep Chong, Kwangil Yim

TL;DR: 本文提出TaxoMIL，一个用于全切片图像（WSI）分析的、受分类学约束的框架。它将WSI诊断重新定义为多粒度文本生成任务，通过双头Transformer解码器生成粗粒度和细粒度的诊断文本，并引入分类学引导的目标来结构化标签嵌入空间并约束视觉表示。在三个不同的WSI数据集上的实验表明，TaxoMIL在准确性和层次感知方面优于现有方法。

Details

Motivation: 传统的WSI诊断方法将其视为对离散标签的扁平分类任务，这与临床推理固有的、从粗到细的层次化本质相矛盾。现有的层次分类器或视觉语言模型要么未能捕捉相关诊断间的语义连续性，要么存在不受约束的文本生成问题，导致分类学幻觉和父子标签违规。

Result: 在三个不同的WSI数据集上进行的大量实验表明，TaxoMIL在准确性和层次感知方面，始终优于最先进的多示例学习分类器和基于视觉语言模型的生成方法。

Insight: 核心创新在于将WSI诊断重新定义为受分类学约束的多粒度文本生成任务，并设计了分类学引导的目标函数来显式地结构化标签空间并严格约束视觉表示，从而生成准确且符合层次结构的诊断。这为解决扁平分类与层次化临床知识之间的不匹配提供了新思路。

Abstract: Whole slide image (WSI) analysis is central to computational pathology, with multiple instance learning (MIL) emerging as the standard pipeline for slide-level diagnosis. However, conventional approaches formulate WSI diagnosis as a flat classification task over discrete labels, contradicting the inherently hierarchical, coarse-to-fine nature of clinical reasoning. Although recent hierarchical classifiers and vision-language models (VLMs) have sought to address this structural gap, they either fail to capture semantic continuity between related diagnoses or suffer from unconstrained text generation that produces taxonomic hallucinations and parent-child label violations. To address these limitations, we propose TaxoMIL, a taxonomy-constrained framework that reformulates WSI diagnosis as a multi-granularity text generation task. TaxoMIL utilizes a dual-head Transformer decoder to generate coarse- and fine-level diagnostic text, and introduces taxonomy-guided objectives that explicitly structure the label embedding space and strictly ground slide-level visual representations within the clinical taxonomy. Extensive experiments across three diverse WSI datasets demonstrate that TaxoMIL consistently outperforms state-of-the-art MIL classifiers and VLM-based generative methods, yielding accurate and hierarchy-aware diagnostic predictions. The code is released at https://github.com/QuIIL/TaxoMIL

[44] InfiniVerse: Occupancy Guided Unbounded Scene Generation for Autonomous Driving cs.CVPDF

Xiaoyu Ye, Leheng Li, Xinyu Ji, Yingjie Cai, Hongda He

TL;DR: 本文提出InfiniVerse，一个用于自动驾驶场景生成的统一框架，能够从单帧多视角图像重建3D占据表示，并沿任意轨迹自回归扩展生成动态城市场景。该方法采用分层草图-精炼范式，通过视频扩散模型将粗糙占据网格转化为逼真、时空一致的视频序列，并在Waymo和nuScenes数据集上达到SOTA性能。

Details

Motivation: 解决自动驾驶领域生成真实、可控且时序一致的动态城市场景这一关键但尚未解决的挑战，实现从单帧输入进行长距离、2D-3D对齐的可控场景合成。

Result: 在Waymo Open Dataset和nuScenes数据集上评估，FID为6.4，FVD为67.97，在持续时间和稳定性方面显著优于现有基准，达到SOTA水平。

Insight: 创新点包括：1）基于3D占据表示的自回归场景扩展框架；2）分层草图-精炼范式实现视觉与空间域的跨模态对齐与相互增强；3）将视频扩散模型与3D重建相结合生成时空一致序列。

Abstract: Generating realistic, controllable, and temporally coherent urban environments is a critical yet unresolved challenge in the autonomous driving community. In this paper, we introduce InfiniVerse, a unified pipeline for long-range, 2D-3D-aligned, and controllable synthesis of dynamic urban scenes from a single frame. In practice, our approach first reconstructs a 3D occupancy representation from the input multi-view frame. This representation serves as a foundation for autoregressive scene extension along arbitrary trajectories. Subsequently, a video diffusion model translates the coarse occupancy grid into realistic, spatiotemporally consistent video sequences. Moreover, we propose a hierarchical sketch-and-refine paradigm, in which the generated videos are re-projected as image-conditioned feedback to enhance the 3D occupancy representation, establishing cross-modal alignment and mutual enhancement between the visual and spatial domains. Extensive evaluations on the Waymo Open Dataset and nuScenes demonstrate that InfiniVerse achieves state-of-the-art performance, with a FID of 6.4 and FVD of 67.97, significantly outperforming existing benchmarks in both duration and stability.

[45] JacobianAvatar: Temporally Consistent Semi-rigid Avatar Reconstruction from a Monocular Video cs.CVPDF

Changyeon Won, Min-Gyu Park, Seonghwan Park, Ju Hong Yoon, Hae-Gon Jeon

TL;DR: 本文提出JacobianAvatar，一种从单目视频重建时序一致半刚性人体化身的方法。该方法利用神经雅可比场表示姿态依赖的变形，通过求解泊松方程自监督训练网络预测雅可比矩阵。针对单目输入的自遮挡和不可见表面问题，引入了约束泊松求解器、基于符号距离的雅可比正则化和变形引导的残差流损失三个关键组件，以抑制边界伪影、恢复常被遮挡区域并增强运动中的时序一致性。

Details

Motivation: 在单目设置下，生成具有复杂运动（如衣物动力学）的真实感人体化身需要对全局和局部变形进行建模，这仍然具有挑战性。本文旨在解决单目视频中半刚性化身重建的时序一致性问题。

Result: 在基准数据集和真实世界视频上的实验表明，该方法能生成时序稳定且几何连贯的化身，性能优于现有最先进方法。

Insight: 创新点在于利用神经雅可比场表示半刚性变形，并针对单目重建难点提出了三个关键技术组件：约束泊松求解器、符号距离正则化和变形引导残差流损失，有效解决了边界伪影、遮挡区域恢复和时序一致性问题。

Abstract: Generating realistic human avatars in complex motions–such as clothing dynamics–requires modeling of global and local deformations which remains challenging in monocular settings. We address this problem by leveraging neural Jacobian fields (NJFs) for representing semi-rigid deformations. We train self-supervised neural networks for predicting Jacobian matrices that give the pose-dependent deformations, by solving a Poisson equation. However, monocular input presents several difficulties such as self-occluded regions and invisible surfaces. To address these issues, we introduce three key components: a constrained Poisson solver, signed distance-based Jacobian regularization, and a deformation-guided residual flow loss, which together suppress boundary artifacts, recover frequently occluded regions such as armpits and thighs, and enforce temporal consistency during motion. Experiments on benchmark and in-the-wild videos demonstrate that our method generates temporally stable and geometrically coherent avatars, outperforming state-of-the-art approaches.

[46] SkillSpotter: Pose-Aware Multi-View Skilled Action Detection and Grading in Ego-Exo Videos cs.CV | cs.AI | cs.LGPDF

Björn Braun, Christian Holz

TL;DR: 本文提出SkillSpotter，一个用于第一人称-第三人称视频中技能动作检测与评级的姿态感知多视角架构。该系统通过自适应时序抑制、门控3D姿态融合和双向跨视角注意力三个模块，联合检测技能动作并评估其执行质量（正确或需改进）。在Ego-Exo4D基准测试中显著提升了检测与评级的性能。

Details

Motivation: 为实现基于增强现实眼镜或固定摄像头的个性化实时教练系统（如体育、烹饪、音乐领域），需要系统不仅能识别人的行为，还能评估其执行质量。Ego-Exo4D的熟练度演示基准将此任务形式化为同时检测个体技能动作并对其正确性进行分类。

Result: 在Ego-Exo4D基准上，SkillSpotter将类特定mAP从12.40提升至21.82（提升76%），平衡准确率从55.99%提升至60.40%，显著优于现有方法。该方法在HoloAssist数据集上也展现出良好的泛化能力。

Insight: 创新点包括：1) 自适应时序抑制模块处理不同活动中技能动作密度变化；2) 门控3D人体姿态融合将身体运动学作为视觉特征的补充信号；3) 双向跨视角注意力有效融合第一人称和第三人称视图。这些模块可迁移至其他时序动作检测模型并带来一致性能提升。

Abstract: To enable personalized, real-time coaching using Augmented Reality glasses or fixed camera setups in domains such as sports, cooking, or music, a system must understand not just what a person does, but how well they execute an activity. In an ego-exo video setting, this requires simultaneously detecting individual skilled actions and classifying each as correct or needing improvement, which Ego-Exo4D’s proficiency demonstration benchmark formalized. We first adapt seven state-of-the-art temporal action detection architectures to this task, extend the evaluation protocol to disentangle detection from grading, and show that existing methods grade near-randomly. We then introduce SkillSpotter, a pose-aware multi-view architecture that jointly detects and grades skilled actions through three task-specific modules: (1) adaptive temporal suppression to handle the varying density of skilled actions across diverse activities, (2) gated 3D body pose fusion to leverage body kinematics as a complementary signal to visual features, and (3) bidirectional cross-view attention to combine ego and exo views effectively. SkillSpotter improves class-specific mAP from 12.40 to 21.82 (+76%) and balanced accuracy from 55.99% to 60.40% over the best baseline. SkillSpotter’s modules transfer to other temporal action detection models with consistent gains, and our method generalizes beyond Ego-Exo4D to HoloAssist. Code: https://github.com/eth-siplab/SkillSpotter

[47] FROST: Training-Free Few-Shot Segmentation with Frozen Features and Nonparametric Statistics cs.CVPDF

Junghwan Park

TL;DR: 本文提出了一种名为FROST的训练无关小样本分割方法，该方法利用冻结的自监督特征（DINOv3）和非参数统计技术，将参考图像的前景和背景视为单位球面上的点云，通过计算密度比来标注查询图像的每个token，无需任何训练即可实现高效分割。

Details

Motivation: 解决遥感图像小样本分割中标签稀缺、图像与自然图像差异大，以及现有方法（如基于训练的方法或单原型/聚类原型方法）会损失类别内部多模态结构或过度依赖训练分布的问题。

Result: 在17个遥感基准测试中，FROST超越了所有训练无关和基于学习的方法，在单个标注示例下领先5.6 mIoU，且随着支持集增大优势扩大，同时模型规模保持最小之一。

Insight: 创新点在于将类别视为分布而非点估计，利用冻结自监督特征的非参数密度比进行分割，所有参数（如核带宽）均从支持集读取而非调优，从而在保留类别内部多模态结构的同时实现高效、无需训练的分割。

Abstract: Few-shot segmentation asks a model to delineate a target class in a query image from only a handful of annotated examples, a setting most acute in remote sensing, where labels are scarce and the imagery departs sharply from the natural images on which vision backbones are pretrained. Prevailing approaches either train a segmenter on labelled episodes, which raises accuracy within the training distribution but binds the model to it, or reduce each class to a lossy summary of frozen features, a single prototype, a few cluster prototypes, or a discrete clustering, none of which preserves the internal structure of a multimodal class. We argue that a class is better described by a distribution than by a point, and that frozen self-supervised features already carry enough structure to estimate that distribution directly. We introduce FROST, a training-free few-shot segmenter that treats the reference foreground and background as two point clouds on the unit sphere of frozen DINOv3 features and labels each query token by a nonparametric density ratio, with a threshold the Bayes rule fixes at zero under equal priors. Because the variance of a density estimate shrinks as its sample grows, the decision sharpens as references accumulate, and every remaining quantity from the kernel bandwidth to the spatial gate is read from the support set rather than tuned. We develop FROST for overhead imagery, where a class is typically a scatter of many small and dissimilar instances that a density tracks but a lossy summary blurs. Across seventeen remote-sensing benchmarks FROST surpasses both training-free and learning-based methods, leading by 5.6 mIoU from a single annotated example and widening its lead as the support set grows, all while remaining among the smallest models compared. Code is available at https://github.com/jhpark-ai/FROST.

Gabriel Clinger

TL;DR: 本文提出了一种名为LINet（线性集成网络）的多流神经网络（MSNN），用于RGB-D场景分类。该网络通过一种新颖的线性集成卷积（LIConv2d）算子，在每一层实现连续的跨模态学习，解决了现有多模态架构中特征融合方式（如早期、晚期或混合融合）存在的结构性问题。

Details

Motivation: 当前多模态架构将特征融合视为离散的、临时性的事件，例如早期融合过早地纠缠表示，晚期融合则将其隔离到最后一层，而混合或基于注意力的方法需要猜测中间融合块的位置。LINet旨在解决这种结构上的折衷。

Result: 在SUN RGB-D 19类场景分类任务上从头训练，LINet在ResNet18规模下达到了45.2%的平均类别准确率，优于先前的从头训练结果；使用领域内RGB-D（ScanNet）预训练后，准确率提升至49.6%。

Insight: 核心创新点是提出了线性集成卷积（LIConv2d）算子，在非线性激活之前实现连续的跨模态信号集成，其灵感来源于神经元放电决策前的体细胞整合。此外，论文揭示了连续集成中关键的初始化病理（Kaiming初始化会扰乱梯度流），并提出了1/N常数初始化来缓解；同时采用了渐进式模态丢弃策略，以防止负协同学习导致的通路崩溃。

Abstract: We present LINet (Linear Integration Network), a Multi-Stream Neural Network (MSNN) for RGB-D scene classification. Current multi-modal architectures treat feature fusion as a discrete, ad-hoc event: early fusion entangles representations prematurely, late fusion isolates them until the final layer, and hybrid or attention-based methods require architectural guesswork to place intermediate fusion blocks. LINet addresses this structural compromise by maintaining three dedicated parallel streams (RGB, depth, and integration) where a novel Linear Integration Convolution (LIConv2d) operator enables continuous cross-modal learning at every layer. The integration stream receives raw filtered signals from both modality streams and combines them before the nonlinear activation threshold, conceptually inspired by somatic integration preceding the neuronal firing decision. Implementing continuous integration exposes a critical initialization pathology: Kaiming initialization of the bridging weights scrambles gradients before they reach the stream backbones, producing a failure mode that resembles overfitting but is corrupted gradient flow. A 1/N constant initialization mitigates this. We employ progressive modality dropout, a curriculum adapted to continuous fusion in which blanking probability increases from zero, preventing pathway collapse, a form of negative co-learning, by forcing robust independent stream representations. Trained from scratch on SUN RGB-D 19-class scene classification, LINet reaches 45.2% mean class accuracy at ResNet18 scale, outperforming prior from-scratch results, and rises to 49.6% with in-domain RGB-D (ScanNet) pretraining.

[49] WaterGen: Decoupling Scene and Medium in Underwater Image Generation cs.CVPDF

Jiayi Wu, Tianfu Wang, Tianyi Xiong, Dehao Yuan, Xiaomin Lin

TL;DR: 本文提出了WaterGen方法，用于生成大规模、真实且多样化的水下图像，该方法能够独立控制场景内容和水体介质条件。通过将水下图像生成分解为两个阶段：首先利用无退化水下图像微调潜在扩散U-Net以生成多样化的场景内容，然后通过条件解码过程添加物理准确的水体退化效果。实验表明，利用WaterGen生成的合成数据能够有效提升水下图像恢复和语义分割等下游任务的性能。

Details

Motivation: 水下计算机视觉任务（如检测、恢复和分割）受限于大规模多样化训练数据的稀缺，现有方法要么在可控性上牺牲真实性和多样性，要么生成真实场景但无法准确独立建模水体介质效应。

Result: 利用WaterGen构建的大规模合成水下数据集在场景结构多样性和水体效应准确性方面表现出色，实验证明该合成数据能持续提升水下图像恢复和语义分割的下游任务性能。

Insight: 核心创新在于将场景生成与介质建模在潜在扩散框架中解耦，通过两阶段设计（场景内容生成与条件介质退化合成）实现了对场景多样性和水体外观的独立精确控制，突破了现有方法在可控性与真实性/多样性之间的权衡限制。

Abstract: Underwater computer vision tasks, such as detection, restoration, and segmentation, are limited by the scarcity of large-scale and diverse training data. We introduce WaterGen, a method for generating large-scale, realistic, and diverse underwater images that provides independent control of the scene and water medium conditions. Our approach treats underwater image generation as the decoupled control of two factors: realistic and diverse scene content (what is in the image), and accurate and controllable water medium effects (what the water does to the image). Existing methods generally achieve only part of this objective: they either provide controllability with limited realism or diversity, or generate realistic scenes without accurately and independently modeling water-medium effects. Our key insight, that allows us to avoid this compromise, is that scene generation and medium modeling can be decoupled within a latent diffusion framework, enabling diverse scene generation together with accurate and controllable underwater appearance. To do this, we decompose underwater image synthesis into two stages. First, we fine-tune the latent diffusion U-Net using degradation-free underwater images so that it learns to generate diverse and realistic latent embeddings of underwater scene content without medium-induced degradation. Second, we formulate the physically accurate medium degradation synthesis as a conditional decoding process applied to these latent embeddings. This decoupled design allows our model to generate diverse scenes with full control of underwater appearance. We leverage WaterGen to build large-scale synthetic underwater datasets that are diverse in scene structures and accurate in water effects and pseudo-labels. We demonstrate that our synthetic data consistently improve downstream performance in underwater restoration and semantic segmentation.

[50] PruneGround: Plug-and-play Spatial Pruning for 3D Visual Grounding cs.CV | cs.AI | cs.CLPDF

Duc Cao Dinh, Khai Le-Duc, Florent Draye, Chris Ngo, Terry Jingchen Zhang

TL;DR: 本文提出PruneGround，一种即插即用的3D视觉定位框架，通过语言引导的空间剪枝、多视图条件描述重构和LLM-Grounder三个核心组件，在减少计算开销的同时提升定位精度。

Details

Motivation: 现有3D视觉定位方法通常对整个场景进行推理，导致预测模糊且计算成本高；作者观察到许多指代表达依赖于局部空间上下文，因此旨在设计一个能聚焦相关区域的高效框架。

Result: 在三个最流行的点云基准测试（ScanRefer、Nr3D/Sr3D）上进行了广泛实验，该方法在ScanRefer的所有设置以及Nr3D/Sr3D的10个设置中的9个上达到了最先进的性能。

Insight: 创新点包括：利用冻结的视觉语言模型进行语言引导的空间剪枝以缩小搜索空间；通过多视图推理重构描述以增强空间线索；将检测预训练的空间大语言模型重新用于语言条件定位，实现了点云与语言表征的对齐。

Abstract: 3D Visual Grounding (3DVG) aims to localize target objects in 3D scenes given natural language descriptions. Existing approaches typically perform reasoning over the entire scene, leading to ambiguous predictions and high computational cost, especially in cluttered environments. We observe that many referential expressions rely on local spatial context and often correspond to restricted spatial regions rather than the full scene. Motivated by this insight, we propose PruneGround, an effective plug-and-play framework for 3DVG built upon three key components. First, we introduce Language-Guided Spatial Pruning (LGSP), which leverages a frozen Vision Language Model (VLM) to identify language-relevant regions, thereby reducing spatial computation and grounding candidates in the narrower search space. Second, we propose MultiView-Conditioned Description Reformulation (MCDR), which decomposes complex expressions into simplified target-anchor relations and augments missing spatial cues through multi-view reasoning. Finally, we propose LLM-Grounder, which repurposes a detection-pretrained spatial LLM into a language-conditioned grounding model by aligning point cloud and linguistic representations within the pruned region. Extensive experiments on the three most popular point cloud benchmarks demonstrate that our method achieves state-of-the-art results on all three ScanRefer settings and on 9 out of 10 Nr3D/Sr3D settings. Code and models are publicly available: https://github.com/leduckhai/PruneGround

[51] Rethinking Foundation Model Collaboration: Enhancing Specialized Models through Proxy Task Reasoning cs.CVPDF

Hongyi Lin, Yang Liu, Jinhua Zhao, Xiaobo Qu

TL;DR: 本文提出FAT框架，通过任务分解而非模型替换的方式，让基础模型与专用模型协作。FAT将结构化预测任务分解为专用模型预测、信息空间重构和基础模型代理推理三个步骤，其中专用模型生成几何和物理上有效的假设，基础模型则在重构的多模态候选上进行有界的代理任务（如选择或验证）。

Details

Motivation: 基础模型在具身智能系统中日益普及，但直接让其执行需要精确几何和数值估计的结构化预测任务时，其能力往往不如专用模型。这引发了一个关键问题：基础模型是应该取代任务特定的预测器，还是应该通过与自身优势更匹配的任务进行协作？

Result: 作者将FAT框架实例化为ProxySelect方法，并在2D目标检测、3D目标检测、轨迹预测和语义分割等多个任务上进行了评估。ProxySelect一致地提升了专用基线的性能，并且以更低的计算成本显著优于直接使用基础模型进行回归的方法。

Insight: 论文的核心创新在于提出了一个通用的协作原则：专用模型保留任务特定的结构，而基础模型则通过上下文代理推理来优化其假设。这提供了一种有效结合两类模型优势的新范式，即让基础模型专注于其擅长的语义理解和推理任务，而非不擅长的精确几何回归。

Abstract: Foundation models are increasingly integrated into embodied intelligence systems, but directly assigning them structured prediction tasks requires precise geometric and numerical estimation, where specialized models often remain stronger. This capability mismatch raises a key question: should foundation models replace task-specific predictors, or should they collaborate through tasks better aligned with their strengths? We propose FAT, a foundation-model-augmented task-specific reasoning framework that treats collaboration as task decomposition rather than model replacement. FAT decomposes structured prediction into specialist prediction, information-space reconstruction, and foundation-model proxy reasoning. The specialist generates geometrically and physically valid hypotheses in the native output space, while the foundation model performs a bounded proxy task, such as selection or verification, over reconstructed multimodal candidates. We instantiate this principle as ProxySelect with a vision–language model. Across 2D object detection, 3D object detection, trajectory prediction, and semantic segmentation, ProxySelect consistently improves specialized baselines and substantially outperforms direct foundation-model regression at lower computational cost. These results suggest a general collaboration principle: specialized models preserve task-specific structure, while foundation models refine their hypotheses through contextual proxy reasoning.

[52] Reasoning-aware Speculative Decoding for Efficient Vision-Language-Action Models in Autonomous Driving cs.CVPDF

Anh Dung Dinh, Simon Khan, Flora Salim

TL;DR: 本文提出了一种推理感知的推测解码方法，用于加速自动驾驶中视觉-语言-动作（VLA）规划器的推理步骤。该方法将传统的因果链（CoC）推理分解为两个专门化的路径：一个处理可预测常规延续的‘常规推理器’，以及一个处理需要新视觉证据的意外情况的‘审慎推理器’。通过引入FlatRoPE位置编码和动作感知强化学习（AARL）后训练技术，系统在推理步骤的运行时间上相比原始Alpamayo规划器实现了约4倍的加速。

Details

Motivation: 自动驾驶VLA规划器在生成轨迹前需要进行自回归的因果链推理，这主导了推理延迟，而延迟是自动驾驶的关键操作约束。因此，加速推理步骤是本文要解决的核心问题。

Result: 在推理步骤上，所提出的双推理器系统相比原始的Alpamayo规划器，运行时间减少了约4倍。

Insight: 创新点在于将推理任务分解为两个专门化的路径（常规与审慎），并利用推测解码框架进行协作。具体技术贡献包括：1) FlatRoPE，一种在草稿模型中打破目标模型3D M-RoPE旋转对称性的1D旋转位置编码，以将注意力从视觉令牌重定向到轨迹历史令牌；2) AARL，一种结合动作质量奖励和静态参考KL锚的后训练阶段。这不同于标准推测解码使用目标模型的小型副本，而是通过功能专业化来提升效率。

Abstract: Modern Vision-Language-Action (VLA) planners for autonomous driving emit a chain-of-causation (CoC) reasoning step \emph{before} producing a trajectory. The reasoning is autoregressive and dominates inference latency, while the trajectory head is parallel and cheap. Latency is an operational constraint in autonomous driving, so accelerating the reasoning step is the central problem we address. We observe that CoC reasoning has two qualitatively different needs: most tokens continue routine setup that follows naturally from the ego-trajectory history, and a small fraction encode commitments that require fresh visual evidence about an unexpected situation. We split this reasoning into two specialized paths: a \emph{routine reasoner} that handles the predictable continuation by attending to trajectory history, and a \emph{deliberative reasoner} (the unmodified VLA target) that handles novel cases by attending to current visual evidence, using the speculative decoding framework as the architectural template for how the two paths cooperate. Unlike standard speculative decoding, our routine reasoner is not a smaller replica of the target; the two reasoners are deliberately specialized to read different parts of the prompt. We propose two techniques to realize this. First, we introduce \textbf{FlatRoPE}, a 1D rotary positional embedding in the draft that breaks the rotational symmetry of the target’s 3D M-RoPE, redirecting attention away from visual tokens and onto trajectory-history tokens. Second, we introduce \textbf{Action-aware RL (AARL)}, a post-training stage that uses an action-quality reward together with a static-reference KL anchor. Together, our two-reasoner system reduces the reasoning-step running time by approximately $4\times$ relative to the original Alpamayo planner.

[53] Beyond Single Character: Evaluating MLLMs for Sentence-Level Oracle Bone Inscription Understanding cs.CVPDF

Ziqi Li, Zijian Chen, Tingzhu Chen, Guangtao Zhai

TL;DR: 本文提出了S-OBI基准，用于评估多模态大语言模型在句子级甲骨文理解上的能力。该基准通过字形替换和组合，合成清晰、标准化的句子级甲骨文实例，并设计了语义匹配、语义槽提取和上下文推理等任务。实验表明，当前MLLMs在句子级甲骨文理解上表现不佳，其理解仍严重依赖于字符级识别。

Details

Motivation: 现有AI辅助甲骨文研究主要集中于字符级识别，忽略了完整卜辞中的长文本连贯性和上下文依赖性。MLLMs强大的视觉感知能力为甲骨文信息处理提供了新可能，但缺乏评估其句子级理解能力的基准。

Result: 实验揭示了当代MLLMs在句子级甲骨文理解任务上的劣势。具体而言，未遮盖区域的视觉感知错误会通过推理链传播，导致对遮盖字符的错误预测，表明当前模型的理解仍强烈依赖于字符级识别。

Insight: 创新点在于构建了一个诊断性基准S-OBI，通过合成标准化视觉输入来隔离低层畸变影响，专注于评估结构化、句子级的理解能力。这为评估MLLMs能否超越孤立字符识别、实现上下文连贯的铭文理解提供了新工具。

Abstract: Existing AI-assisted oracle bone inscription (OBI) visual recognition and understanding studies mainly focus on character-level, ignoring the long-form textual coherence and contextual dependencies embedded in complete divination charges. Recently, the powerful visual perception capabilities of multimodal large language models (MLLMs) have opened new possibilities for OBI information processing. In this work, we introduce S-OBI, a novel benchmark for evaluating MLLMs in Sentence-level OBI understanding. Instead of using noisy and incomplete rubbings as the visual input, S-OBI synthesizes clear and standardized sentence-level OBI instances through glyph substitution and composition. According to 95 original rubbings with translations that have been identified, corrected, and verified by experts, we replace characters in the original rubbings with corresponding clean glyph samples sourced from existing OBI datasets while preserving the overall inscriptional structure and semantic organization. This mitigates the influence of low-level distortions and enables a more focused evaluation of sentence-level OBI understanding. Based on this, we design semantic matching, semantic slot extraction, and contextual reasoning tasks and obtain 695 question-answer pairs. Experiments reveal the inferiority of contemporary MLLMs on sentence-level OBI understanding. In particular, visual perception errors in unmasked regions propagate through the reasoning chain, leading to erroneous predictions for masked characters, which indicates that sentence-level OBI understanding in current models remains strongly dependent on character-level recognition. Overall, S-OBI provides a diagnostic benchmark for evaluating whether MLLMs can move beyond isolated character recognition toward structured inscription-level understanding.

[54] Learning to Deny: Action Denial in Multimodal Large Language Models cs.CVPDF

Raiyaan Abdullah, Shehreen Azad, Yogesh Singh Rawat

TL;DR: 本文针对多模态大语言模型在视频理解中存在的‘动作否认’能力不足问题，提出了UCF101-AD基准数据集，用于评估模型识别动作未发生的能力。研究发现，现有SOTA模型在正面动作识别上表现良好，但在动作否认任务上准确率大幅下降，揭示了模型倾向于确认看似合理的动作而非验证其是否真实发生的盲点。作者进一步提出了CausalAct因果图方法，通过自然语言提示整合场景上下文、交互和运动信息，有效提升了模型的否认推理能力。

Details

Motivation: 当前多模态大语言模型在标准视频理解基准上取得了显著进展，但其在强上下文线索下识别动作未发生（即‘动作否认’）的能力尚未被充分探索。本文旨在填补这一空白，评估并提升模型在因果推理方面的能力。

Result: 在提出的UCF101-AD基准上评估了20个SOTA多模态大语言模型。结果显示，模型在正面动作类别的准确率超过85%，但在对应的动作否认任务上准确率骤降至50%以下。提出的CausalAct因果图方法通过整合因果线索，显著减少了误报。

Insight: 论文的核心创新点在于提出了首个专门评估‘动作否认’能力的大规模基准UCF101-AD，并揭示了现有模型在因果推理上的一个关键缺陷。从方法上，将因果图形式化（CausalAct）并通过自然语言提示来表达场景结构，证明了否认是一种可学习的推理技能，为提升多模态模型的因果理解能力提供了新思路。

Abstract: Multimodal large language models (MLLMs) have rapidly advanced video understanding, achieving strong zero-shot and few-shot recognition across standard benchmarks. Yet their ability to deny an action by recognizing when an activity is not happening despite strong contextual cues remains largely unexplored. We introduce UCF101-AD, a large-scale benchmark consisting of paired Action-Presence and Action-Denial clips, designed to evaluate this capacity for denial. Each negative video in UCF101-AD preserves the same contextual and motion cues, including persons, objects, and locations, as its positive counterpart, but the defining action itself is explicitly absent. Evaluating 20 state-of-the-art MLLMs reveals a consistent failure: models that exceed 85% accuracy on the positive action classes collapse below 50% on their action-denial counterparts, indicating a strong inclination to affirm plausible actions rather than verify that they truly occur. This exposes a critical blind spot in modern video understanding: the inability to reason causally about whether a motion actually happens. To probe this issue, we explore a causal graph formulation, CausalAct, which expresses scene structure through natural-language prompts linking context, interaction, and motion. Incorporating such causal cues substantially reduces false positives, demonstrating that denial is a learnable reasoning skill. UCF101-AD provides a new lens for diagnosing and improving causal reasoning in multimodal models. Dataset and relevant code: https://github.com/raiyaan-abdullah/Learn-to-Deny.

[55] AC3S: Adaptive Conditioning for 3D-Aware Synthetic Data Generation cs.CVPDF

Eric Ji, Qiran Hu, Wufei Ma, Sarthak Jain, Yingying Li

TL;DR: 本文提出了一种名为AC3S的扩散模型框架，用于生成具有精确3D结构和姿态一致性的高质量合成数据。该框架通过自监督视觉提示调制器动态调整ControlNet的条件强度以避免过拟合，并结合多智能体视觉语言模型生成与几何结构对齐的详细提示，从而在保证照片级真实感的同时增强数据多样性和语义一致性。

Details

Motivation: 现有基于扩散模型的合成数据生成方法在利用边缘图等视觉提示引导时，常因过拟合导致图像真实感下降和数据集质量受限，难以精确执行3D结构和姿态一致性。

Result: 大量实验表明，该方法在图像质量和下游任务实用性方面均有显著提升，能够生成具有准确2D和3D标注的高质量合成数据集。

Insight: 创新点在于提出了自适应条件调节机制来平衡结构引导与生成自由度，并利用多智能体VLM框架生成3D感知的详细提示，这为可控且高质量的3D感知图像合成提供了一种有效方案。

Abstract: Synthetic data generation has emerged as a powerful tool for improving data scalability in computer vision. Recent diffusion-based pipelines have demonstrated strong photorealism. However, how to enforce precise 3D structure and pose consistency in generated images remains challenging. Existing methods leverage visual prompts such as edge maps to guide diffusion models, but often suffer from over-conditioning artifacts that degrade image realism and limit dataset quality. In this paper, we present a diffusion-based image generation framework that enforces 3D structural alignment while preserving photorealism through adaptive conditioning. Our framework, Adaptive Conditioning for 3D-Aware Synthetic Data Generation (AC3S), introduces a self-supervised visual prompt modulator that dynamically adjusts the strength of ControlNet conditioning, preventing over-conditioning and enabling the diffusion model to retain its generative expressiveness. To further enhance diversity and semantic consistency, we develop a multi-agent vision language model framework that composes detailed and 3D-aware prompts aligned with the underlying geometric structure. Together, these components enable the scalable generation of high-quality synthetic datasets with accurate 2D and 3D annotations. Extensive experiments demonstrate that our method significantly improves image quality and downstream utility.

[56] Distilling Temporal Coherence into 2D Networks for Transrectal Ultrasound Prostate Video Segmentation cs.CV | cs.AIPDF

Dong Yeong Kim, JunGyu Lee, Jaewon Choi, June Young Seo, Myeongseop Kim

TL;DR: 本文提出了一种用于经直肠超声前列腺视频分割的时序一致性学习框架，旨在将时序连贯性知识蒸馏到2D网络中，以在保持实时单帧推理效率的同时解决传统2D方法因忽略时序上下文而产生的帧间不一致问题。该方法通过置信度加权时序一致性目标和双尺度原型对齐模块，选择性地利用可靠区域的信息并增强语义连贯性，同时利用基于几何等变性的伪标注技术减少对密集逐帧标注的依赖。

Details

Motivation: 解决经直肠超声前列腺实时视频分割中，传统2D方法因忽略时序信息导致帧间分割不一致，而3D方法又因计算延迟过高无法满足实时性要求的矛盾。

Result: 在SUN-SEG和新提出的TRUS-V基准（包含2679帧）上进行了广泛实验，结果表明该方法在保持实时速度的同时，达到了最先进的（SOTA）分割精度和时序一致性。

Insight: 创新点在于将临床观察（前列腺几何稳定而周围声学环境波动）转化为算法设计，提出了置信度加权时序一致性目标以选择性地约束可靠区域，并结合双尺度原型对齐模块增强语义连贯性；同时，利用几何等变性生成伪标签进行知识蒸馏，减少了对密集标注的依赖。

Abstract: Real-time video segmentation of the prostate in Transrectal Ultrasound (TRUS) is essential for image-guided interventions. While conventional 2D methods suffer from inter-frame inconsistencies by disregarding temporal context, 3D architectures incur prohibitive latency. To resolve this dilemma, we present a Temporally Consistent Learning Framework that distills temporal coherence into a 2D network during training, preserving single-frame inference efficiency. Our design is driven by a key clinical observation: the prostate exhibits geometric stability, whereas the surrounding acoustic environment fluctuates due to physiological motion and transducer pressure. Because conventional temporal constraints propagate erroneous gradients from these unstable regions, we introduce a Confidence-Weighted Temporal Consistency objective derived from optical flow warping residuals, selectively attenuating contributions from unreliable regions. Complementing this pixel-wise constraint, a Dual-scale Prototype Alignment Module enforces semantic coherence through contrastive optimization of local boundary and global semantic features. Furthermore, to eliminate the need for dense per-frame video annotations, we employ geometric equivariance-based pseudo-labeling with knowledge distillation from a pretrained teacher. Extensive experiments on SUN-SEG and our newly introduced TRUS-V benchmark (2,679 frames) demonstrate state-of-the-art accuracy and temporal consistency at real-time speed. Code and dataset are available at https://github.com/DYDevelop/DTC-TRUS.

[57] AA: A Multi-view Multimodal Dataset for Screen-based Gaze Estimation cs.CV | cs.HCPDF

Chang Liu, Jiaqi Liu, Zhoutong Ye, Xinjie Shen, Chun Yu

TL;DR: 本文介绍了AA数据集，这是一个用于基于屏幕的视线估计的多视角多模态数据集。它通过八个固定在屏幕上的摄像头和两个侧视摄像头同步采集面部观测数据，并与受控注视条件下收集的精确屏幕空间注视目标配对。每个样本包含多视角面部观测以及结构化的面部区域裁剪，支持从全局和局部视觉线索进行多模态学习。

Details

Motivation: 现有视线估计数据集多为单视角，难以应对视角变化和遮挡。AA旨在提供一个多视角、多模态的数据集，以支持更鲁棒的视线估计建模。

Result: 数据集提供了独立于受试者的评估划分和标准化的数据处理流程，以支持可复现的研究。

Insight: 创新点在于同时提供屏幕安装和侧面安装的多视角覆盖，并结合了全局面部观测与局部面部区域裁剪，实现了多模态学习，有望提升视线估计在视角变化和遮挡情况下的鲁棒性。

Abstract: We present AA, a multi-view multimodal dataset for screen-based gaze estimation. The dataset captures synchronized facial observations from eight fixed screen-mounted cameras and two additional side-view cameras, paired with precise screen-space gaze targets collected under controlled fixation conditions. Each sample contains multi-view face observations together with structured facial region crops, enabling multimodal learning from both global and local visual cues. Unlike existing single-view gaze datasets, AA provides multi-view coverage from both screen-mounted and side-mounted perspectives, enabling more robust modeling under viewpoint variation and occlusion. The dataset includes subject-independent evaluation splits and a standardized data processing pipeline to support reproducible research in gaze estimation.

Bo Wu, Ruoshen Mo, Justin Yue, Yanyu Zhang, Janice Nguyen

TL;DR: 本文介绍了CooperScene，一个高保真的协同自动驾驶数据集，包含真实世界的C-V2X通信特性。该数据集涵盖多种场景（如交叉路口、高速公路匝道、停车场），涉及三辆联网自动驾驶车辆和一个路边单元，均配备多模态传感器和商用C-V2X通信设备，并提供全局一致的3D标注。

Details

Motivation: 现有数据集常忽略现实部署的复杂性，如有限的通信带宽及其动态性、异构感知模态以及超越单一合作方的可扩展性。

Result: 数据集包含59K帧、总计344K个对象的标注，并基于严格的传感器/代理同步、厘米级定位与空间对齐、精确跨模态校准以及符合3GPP标准的C-V2X通信，为评估多智能体扩展性和实际性能建立了严格基准。

Insight: 创新点在于首次在协同自动驾驶数据集中集成了真实世界的C-V2X通信特性，并考虑了多车辆与基础设施的异构多模态协同，为现实可部署环境下的评估提供了更全面的基准。

Abstract: Cellular vehicle-to-everything (C-V2X) enables cooperative perception, prediction, and planning beyond the field of view of individual agents. However, existing datasets often overlook the complexities of real-world deployment, such as limited communication bandwidth and its dynamics, heterogeneous sensing modalities, and scalability beyond a single cooperative partner. In this paper, we introduce CooperScene, a high-fidelity cooperative autonomy dataset with real-world C-V2X communication characterization. The dataset is organized into diverse scenes, including intersections, highway ramps, and parking lots. These scenes involve three connected and autonomous vehicles (CAVs) and one infrastructure roadside unit (RSU), all equipped with multi-modal sensors and commercial off-the-shelf C-V2X communication radios. All scenes are annotated with globally consistent 3D labels at 10 Hz, totaling 344K objects across 59K frames, underpinned by tight sensor- and agent-synchronization, centimeter-level localization and spatial alignment, precise cross-modality calibration, and 3GPP-standard-compliant C-V2X communication. CooperScene establishes a rigorous benchmark for evaluating multi-agent scaling and actual performance in real-world deployable settings. Project website for data and benchmark: https://cisl.ucr.edu/CooperScene

[59] HyperVLP: Enhancing Hierarchical Surgical Video-Language Pre-training in Hyperbolic Space cs.CVPDF

Yaojun Hu, Kun Yuan, Nassir Navab, Haochao Ying, Jian Wu

TL;DR: 本文提出HyperVLP，一种在双曲空间中增强层次化手术视频-语言预训练的框架。它通过缓解程序上下文引起的结构假阴性并强制父阶段与其子步骤之间的语义一致性，来显式保留手术知识的层次结构。

Details

Motivation: 现有手术视觉-语言基础模型通常将多维度、层次化的知识（如细粒度动作、中级关键步骤和全局程序上下文）压缩到单一的平坦嵌入空间中，忽略了跨层次的语义包含关系，这削弱了长程依赖建模。

Result: 在多个手术基准测试上的广泛实验表明，该方法在跨程序和机构的零样本和少样本阶段识别任务上取得了一致的性能提升。

Insight: 创新点在于将层次化语义结构建模引入手术视频-语言预训练，并利用双曲空间的几何特性来更好地表示这种层次关系，从而更有效地编码手术知识。

Abstract: Surgical vision-language foundation models typically adopt educational materials, such as surgical lecture videos, to transfer surgical knowledge encoded in language into visual representations. These knowledge are multi-dimensional and hierarchical: fine-grained action cues appear in narration, mid-level key steps are summarized in subsection headings, and global procedural context, such as patient history and surgical strategy, is described in abstract texts. Prior work largely collapses these heterogeneous signals into a single flat embedding space, implicitly assuming independence across hierarchy levels. However, this is suboptimal because it ignores cross-level semantic containment, e.g., actions belong to steps, steps compose phases, weakens long-range dependency modeling. To this end, we propose a hyperbolic surgical video-language pre-training framework that explicitly preserves the hierarchical structure by mitigating structural false negatives induced by procedural context and enforcing semantic consistency between parent phases and their constituent child steps. Extensive experiments on multiple surgical benchmarks show consistent gains in zero- and few-shot phase recognition across procedures and institutions.

[60] Rethinking the Role of Feature Engineering and Learning Strategies in Few-Shot Hidden Emotion Recognition cs.CVPDF

Xiaochuan Guo, Jihao Gu, Haixu Liu, Yuxin Liu, Qi Wang

TL;DR: 本文介绍了团队在EI-MIGA-IJCAI挑战赛Track 3中获得第一名的解决方案，针对长视频中微弱稀疏的隐式情绪识别任务，提出了一个紧凑的多模态时序建模框架。该框架整合了骨架、表情、视觉基础模型、视频特征和语义先验等多源特征，并设计了跨注意力机制与自适应池化方法来消除静态偏差并抑制背景噪声。

Details

Motivation: 解决长视频中隐式情绪证据微弱且稀疏的挑战，提升少样本隐藏情绪识别的性能。

Result: 在EI-MIGA-IJCAI挑战赛Track 3中取得第一名，测试准确率达到0.76923。

Insight: 创新点包括：1) 以静态姿态特征为Query、动态微运动差分特征为Key/Value的跨注意力机制，可消除个体体型和身份相关的静态偏差；2) 基于多示例学习的自适应池化方法，能提取瞬时情绪并抑制长序列背景噪声；3) 揭示了通用视觉基础模型在微动态任务中的表征崩溃现象，并分析了由捷径学习和机械记忆导致的伪泛化机制。

Abstract: In this paper, we present the solution developed by our team, XInsight Lab, which achieved first place in Track 3 of the 4th EI-MIGA-IJCAI Challenge with a test accuracy of 0.76923. To address the challenge of weak and sparse implicit emotion evidence in long videos, this paper extends the winning solution from the previous competition and proposes a compact multi-modal temporal modeling framework. The framework integrates and evaluates the effects of multi-source features, including 2D/3D skeletons, facial expression Blendshapes, DINOv2/v3 vision foundation models, X-CLIP video features, and Gemini semantic priors. Architecturally, we propose a cross-attention mechanism that utilizes static pose features, denoted as Base, as the Query and dynamic micro-motion differential features, denoted as Offset, as the Key and Value. By capturing local relative velocities, this mechanism eliminates static biases related to individual body shape and identity. Concurrently, an adaptive pooling method based on Multiple Instance Learning is employed to extract instantaneous emotions while suppressing background noise in long sequences. Finally, the paper reveals the representation collapse phenomenon of general vision foundation models in micro-dynamic tasks, and analyzes the underlying mechanisms where networks fall into public-leaderboard-driven pseudo-generalization due to shortcut learning and rote memorization.

[61] Learning from Failure: Inference-Time Self-Improvement for Computer-Use Agents cs.CV | cs.AI | cs.CL | cs.CY | cs.LGPDF

Xueqiao Sun, Xiaohan Wang, Ludwig Schmidt, Serena Yeung-Levy, Yuhui Zhang

TL;DR: 本文提出了一种基于失败轨迹的推理时自我改进方法，用于提升计算机使用智能体的性能。该方法利用大语言模型（LLM）分析智能体失败的任务轨迹，诊断失败模式，并提出推理时的解决方案（如代码补丁），从而在不增加训练成本的情况下改进智能体。

Details

Motivation: 当前开发计算机使用智能体的主流方法依赖于收集大规模、高质量的成功轨迹数据进行自我改进，但忽视了失败轨迹所包含的关于模型弱点的丰富信息。本文旨在探索一种以失败为驱动的、数据中心的自我改进范式，作为基于成功轨迹方法的补充。

Result: 在OSWorld基准测试上，该方法将最先进的OpenCUA-72B模型的成功率从42.3%提升至48.9%，获得了6.6个百分点的增益，且没有额外的训练成本，仅有适度的推理开销。

Insight: 创新点在于将失败轨迹视为有价值的改进资源，并设计了一个由LLM驱动的、在推理时进行诊断和打补丁的自动化流程。这为智能体改进提供了一种高效、低成本的新思路，补充了传统的基于成功轨迹的微调方法。

Abstract: Computer-use agents, which leverage multimodal large language models (MLLMs) to operate computers and complete tasks, have attracted significant attention for their utility and versatility. A major challenge in developing these agents is collecting large-scale, high-quality trajectories. The standard approach generates synthetic data through a self-improving loop: an agent is placed in a verifiable environment and iteratively fine-tuned on its successful trajectories. Despite its effectiveness, this paradigm exploits only successful trajectories and discards the failed ones, even though failures carry rich information about a model’s weaknesses. In this work, we explore a complementary failure-driven self-improvement loop, a data-centric paradigm that turns failed trajectories into agent improvements. Specifically, we employ an LLM to diagnose failure modes, propose inference-time solutions, and generate code patches – lightly verified by humans – that upgrade the agent. We validate this approach with the state-of-the-art OpenCUA-72B model on the OSWorld benchmark, improving the success rate from 42.3% to 48.9%, a gain of 6.6 percentage points, without any additional training cost and with only modest inference overhead. Our results demonstrate that failure-driven self-improvement is a viable complement to success-based pipelines, enabling more efficient agent improvement.

[62] Editing Everything Everywhere All at Once cs.CVPDF

Fabio Quattrini, Carmine Zaccagnino, Enis Simsar, Marta Tintoré Gazulla, Rita Cucchiara

TL;DR: 本文提出了一种名为MICE（Multi-Instance Concurrent Editing）的训练免费策略，用于在单次前向传播中同时编辑图像中的多个区域。该方法通过修改联合注意力中的加性偏置来调控不同编辑指令之间的交互，从而解决多指令编辑时的语义干扰问题，实现属性绑定并保持全局视觉一致性。

Details

Motivation: 解决在单次前向传播中同时编辑图像多个区域时，由于语义干扰导致的属性泄漏和编辑解耦不良的问题，尤其是在编辑数量增加时。

Result: 在LoMOE-Bench和新提出的更富挑战性的MICE-Bench（平均每图8.5个并发编辑）上评估，实验表明MICE在视觉质量保持和编辑指令忠实度方面优于强基线及近期竞争对手。

Insight: 创新点在于通过调控注意力机制（允许实例内注意力、惩罚相邻区域令牌交互、抑制无关的跨实例注意力）来管理多指令交互，这是一种无需训练的策略，可提升多实例编辑的可扩展性和效果。

Abstract: Editing multiple elements of an image in a single forward pass is a practical alternative to multi-turn image manipulation, offering improved efficiency and potentially better harmonization. However, when several instructions target different regions, semantic interference often leads to attribute leakage and poor edit disentanglement, especially as the number of edits increases. In this work, we propose MICE (Multi-Instance Concurrent Editing), a training-free strategy for scalable multi-instance image editing with Multimodal Diffusion Transformers. MICE modifies the additive bias of joint attention to regulate interactions between instance-specific edit instructions, latent, and context tokens identified via user-provided segmentation masks. Specifically, MICE allows intra-instance attention, penalizes interactions between neighboring region tokens, and suppresses unrelated cross-instance attention. As a result, our method enforces attribute binding while preserving global visual consistency. We evaluate MICE on LoMOE-Bench and introduce MICE-Bench, a more challenging benchmark with an average of 8.5 concurrent edits per image. The experiments demonstrate that our approach outperforms strong baselines and recent competitors in terms of visual quality preservation and faithfulness to the editing instructions.

[63] Decodable Is Not Grounded: A Vision-Ablation Arbiter for VLM Spatial Reasoning cs.CVPDF

Chih-Ting Liao, Fei Shen, Xin Cao, Tat-Seng Chua

TL;DR: 本文揭示视觉语言模型（VLM）在空间推理任务中，通过线性探测和导向恢复等标准方法评估的潜在知识可能被系统性高估，因为模型解码出的答案未必真正基于图像内容。作者提出一种简单的因果控制方法——将图像替换为灰色空白，以此作为仲裁器来区分三种不同的基础机制：真正基于视觉的、基于先验的以及感知倒置的。该研究在涵盖六个语言模型家族、规模从2B到27B的十四个VLM上验证了这一分类的普适性，并展示了如何通过低成本的自校准仲裁器清晰分离这些机制。

Details

Motivation: 标准方法（如线性探测结合导向恢复）在评估VLM的潜在知识时，可能系统性地高估模型对图像的实际基础理解，尤其是在空间推理任务中，这种误差难以被现有探测方法察觉。

Result: 在空间推理任务上，探测方法解码出的答案准确率在73%到97%之间，而通过无训练投影可将一个接近随机（59%）的轴提升至79%，显示出解锁潜在知识的迹象。然而，空白图像仲裁器反驳了这一点，揭示了三种基础机制：真正基础于视觉的（正确）、基于先验的（视觉无关）以及感知倒置的（解码可控但符号错误，导致准确率低于随机）。这一分类在研究的十四个VLM中普遍成立，例如水平轴是基础于视觉的，垂直轴是基于先验的，深度轴是倒置的，且倒置现象在模型规模扩大时出现。解码与部署的倒置在五个家族的八个模型中的七个上复现，而纠正所需的编辑复杂度因模型几何结构而异。

Insight: 论文的核心创新在于提出了一种简单、低成本且自校准的因果控制方法（空白图像仲裁器），能够清晰地区分VLM中真正基于视觉的感知、感知倒置以及先验替代，这为评估VLM的潜在知识和导向声明提供了一个应成为默认控制的重要工具。从客观角度看，该研究揭示了VLM空间推理中解码能力与真实基础之间的关键脱节，并提供了一个可操作的框架来诊断和纠正这种脱节，对理解和完善VLM的感知能力具有重要借鉴意义。

Abstract: The standard way to read latent knowledge out of a model, a linear probe confirmed by a steering recovery, can systematically overstate what a vision-language model (VLM) actually grounds in the image. We show this on spatial reasoning, where the error is invisible to both probing and steering yet exposed by a one-line causal control: replacing the image with a gray blank. Probes decode the within-axis answer at 73–97% across axes, and a training-free projection lifts a near-chance axis from 59% to 79%, exactly the signature of unlocking latent knowledge. The blank-image arbiter refutes it, revealing three grounding regimes that probing conflates: an axis can be grounded (vision-dependent, correct), a prior (vision-independent, with its decode and its apparent recovery a directional default rather than perception), or, surprisingly, inverted: decodable, causally controllable, but deployed with the wrong sign, so the model scores below chance and the error requires looking. The taxonomy holds across the studied VLMs: in fourteen models spanning six language-model families and 2B–27B, horizontal is grounded, vertical is a prior, and depth is inverted, with the inversion emerging at scale within families. The decode-versus-deploy inversion replicates on seven of eight models across five families, and the minimal edit that re-deploys it varies with geometry: a training-free rotation matches a trained edit on the cleanest model, while distributed inversions need a trained low-rank edit, tracing a per-model correction-complexity spectrum. The cheap, self-calibrating arbiter cleanly separates grounded perception, inverted perception, and prior substitution; we argue it should be a default control for latent-knowledge and steering claims in VLMs.

[64] Bridging Video Understanding and Generation in a Unified Framework cs.CVPDF

Yuqi Wang, Runyi Li, Ruoyu Feng, Renjie Chen, Wenfeng Lin

TL;DR: 本文提出Vega框架，首次将视频理解与生成统一建模。该框架采用共享词汇表联合建模文本与视觉表示，并设计混合架构结合自回归预测与扩散渲染。实验表明Vega在视频生成与理解基准上均取得优异性能。

Details

Motivation: 现有统一建模研究多集中于图像领域，视频领域因理解任务需要紧凑语义表示而生成任务需保留细节与时序连贯性，存在建模鸿沟。视频同时包含空间语义与时间动态特性，是比静态图像更适合统一多模态建模的模态。

Result: 在VBench视频生成基准和VideoMME视频理解基准上的大量实验表明，Vega取得强劲性能，验证了框架在两类任务上的有效性。

Insight: 创新点在于：1) 提出首个统一视频理解与生成的框架；2) 设计自回归预测关键帧语义token与扩散渲染高分辨率帧的混合架构，通过结构化表示引导细节生成；3) 构建共享词汇表实现跨模态统一表示学习。

Abstract: Recently, unified image generation and understanding have been extensively explored. However, extending such unified modeling paradigms to the video domain remains largely underexplored. A central challenge is that video understanding favors compact, discriminative semantic representations, whereas video generation requires dense signals that preserve visual details and temporal coherence. Videos naturally capture both spatial semantics and temporal dynamics, making them a more suitable modality for unified multimodal modeling compared to static images. In this paper, we propose Vega, a unified framework that bridges video understanding and generation. Vega leverages a shared vocabulary to jointly model text and visual representations and employs a hybrid architecture combining autoregressive (AR) prediction with diffusion-based rendering. Specifically, the AR model focuses on predicting semantically meaningful visual tokens for keyframes, providing a structured representation that guides the diffusion module in rendering dense, high-resolution video frames. Extensive experiments demonstrate that Vega achieves strong performance on video generation benchmarks such as VBench and video understanding benchmarks like VideoMME.

[65] Language-Assisted Super-Resolution from Real-World Low-Resolution Patches cs.CVPDF

Joonkyu Park, Kyoung Mu Lee

TL;DR: 本文提出LA-SR，一种利用语言辅助的单图像超分辨率新框架，旨在解决真实世界低分辨率图像重建问题。该方法通过从单张高质量图像中提取不同深度区域作为非配对的真实LR-HR对，并利用视觉语言模型在语义空间中弥合LR与HR之间的差距，从而克服了传统方法依赖合成退化数据导致的泛化能力不足。

Details

Motivation: 动机在于解决单图像超分辨率训练中真实配对HR-LR数据难以获取的问题，传统方法使用手工合成的退化核无法捕捉真实LR图像的复杂性，导致实际应用泛化性能差。

Result: 论文表明LA-SR框架能够有效超分辨真实LR输入，生成逼真的输出，克服了基于合成数据训练方法的局限性，但摘要中未提及具体的定量结果或基准测试对比。

Insight: 创新点在于将非配对超分辨率问题重新定义在语言空间中，利用视觉语言模型提取语义丰富的表示，并设计了语言引导的内容损失和质量损失，以实现语义保真度和感知真实性的提升，这为利用多模态信息解决图像复原问题提供了新思路。

Abstract: Single image super-resolution aims to reconstruct high-resolution (HR) images from low-resolution (LR) inputs. Training SR models typically requires paired HR-LR data, which is difficult to obtain in reality. As a result, most methods synthesize LR images by artificially degrading HR images with handcrafted kernels or camera ISP adjustments. However, these synthetic degradations fail to capture the complexity of real LR images, leading to poor generalization in practice. To address this, we observe that even within a single high-quality image, regions at different depths exhibit varying resolutions, where distant regions act as LR patches and closer ones as HR patches. This allows the extraction of real, degradation-induced LR patches from real images. Since these LR patches lack paired HR counterparts, we propose LA-SR (Language Assistant for SR), a novel framework for unpaired SR. The key idea of LA-SR is to redefine unpaired SR in the language space, using vision-language models to bridge the LR-HR gap. LA-SR projects images into a semantically rich space representing both content and quality, and applies two language-guided losses: linguistic content loss to preserve semantic fidelity, and linguistic quality loss to enhance perceptual realism. With this alignment, LA-SR effectively super-resolves real LR inputs, producing realistic outputs that overcome the limitations of synthetic-data-trained methods.

[66] MS-Resampler: Multi-Scope Visual Resampling for Efficient Multimodal LLMs cs.CVPDF

Zhongyang Li, Yaqian Li, Faming Fang, Rinyoichi Takezoe, Zi-Hao Bo

TL;DR: 本文提出MS-Resampler，一种用于多模态大语言模型（MLLMs）的多尺度视觉重采样框架，通过注入明确的空间尺度先验，实例化多个特定尺度的重采样器，分别从局部到全局聚合视觉信息，并自适应融合其输出，以在固定令牌预算下更好地捕捉局部细节和全局上下文。

Details

Motivation: 现有基于重采样的投影器通常采用单一固定的全局交叉注意力聚合范围，这可能会模糊细粒度的局部证据，并限制在固定令牌预算下同时捕获局部细节和全局上下文的能力。

Result: 在十个公共多模态基准测试上的广泛实验表明，MS-Resampler相比传统的单尺度重采样器，持续提升了视觉理解和多模态推理能力，同时仅引入了极小的计算开销。

Insight: 创新点在于将显式的空间尺度先验注入重采样注意力，以实例化多个特定尺度的重采样分支，并自适应融合其输出，从而在计算高效的前提下，实现了从局部到全局的多粒度视觉信息聚合，提升了MLLMs的视觉表征能力。

Abstract: Multimodal large language models (MLLMs) typically employ resampling-based projectors to transform dense visual features into a compact token sequence for language modeling. Most existing resamplers adopt a single, fixed aggregation scope via global cross-attention, which can blur fine-grained local evidence and limit the ability to capture both local details and global context within a fixed token budget. In this work, we propose MS-Resampler, a multi-scope visual resampling framework for MLLMs. MS-Resampler instantiates multiple scope-specific resamplers by injecting explicit spatial scope priors into the resampling attention, enabling each branch to aggregate visual information at a particular granularity from local to global. The outputs of these scope-specific resamplers are then adaptively fused to produce the final visual representations for language modeling. Extensive experiments on ten public multimodal benchmarks show that MS-Resampler consistently improves visual understanding and multimodal reasoning over conventional single-scope resamplers, while introducing only minimal computational overhead.

[67] Visual Semantic Entropy: Do Vision Language Models Recognize Visual Ambiguity? cs.CV | cs.AI | cs.CLPDF

Ta Duc Huy, Trang Nguyen, Townim Chowdhury, Ankit Yadav, Minh-Son To

TL;DR: 本文提出视觉语义熵（VSE）方法，用于评估视觉语言模型（VLMs）在视觉模糊输入下的不确定性。该方法通过仅扰动图像并保持文本查询不变来探测视觉变化，从而更准确地捕捉视觉歧义，避免了现有方法因文本变化主导而导致的偏差。

Details

Motivation: 现有基于熵的不确定性估计方法（如语义熵）在视觉模糊输入下，由于视觉嵌入的过度自信会抑制输出多样性，导致低估不确定性；而基于输入扰动的方法又往往被文本变化主导，无法准确反映视觉证据的歧义。

Result: 在五个现代视觉语言模型和五个不同的VQA基准测试上进行广泛评估，结果表明VSE能有效捕捉视觉歧义，在VLM不确定性估计任务上达到了新的最先进（SOTA）水平。

Insight: 创新点在于提出仅扰动图像的视觉语义熵（VSE），通过聚类生成答案到语义原型并计算其质量加权离散度来量化不确定性，确保了不确定性估计直接源于视觉证据而非文本提示的敏感性。

Abstract: Vision-language models can produce confident answers on visually ambiguous inputs, resulting in biased predictions. Common entropy-based methods, such as Semantic Entropy (SE), rely on output diversity. Yet our analysis shows that overconfident visual embeddings suppress output diversity under stochastic decoding, causing SE to underestimate uncertainty in such cases. Recent methods instead probe output diversity through input perturbations, including textual paraphrasing or joint text-image perturbations, and show improved performance. We study these approaches and reveals that the resulting variability is often dominated by textual changes rather than visual evidence, causing uncertainty estimates to reflect prompt sensitivity rather than visual ambiguity. We therefore propose Visual Semantic Entropy (VSE), which perturbs only the image to probe nearby visual variations while keeping the text query fixed. VSE measures uncertainty by clustering generated answers into semantic prototypes and computing the mass-weighted dispersion among them. Extensive evaluation across five modern vision-language models and five diverse VQA benchmarks demonstrates that VSE effectively captures visual ambiguity, establishing a new state-of-the-art for VLM uncertainty estimation.

[68] One Video, One World: Turning Monocular Video into Physical 4D Scenes cs.CVPDF

Junhao Chen, Boran Zhang, Mingjin Chen, Henghaofan Zhang, Saining Zhang

TL;DR: OVOW是首个无需训练的系统，能够从单目视频中重建实例级、可用于仿真的4D网格场景。它通过四阶段流程实现：视觉语言模型发现、标记和运动分类实例；类别感知重建为刚性物体生成单实例网格，为可变形物体生成拓扑一致的网格序列；迭代渲染-匹配-优化过程恢复度量尺度和6自由度姿态轨迹；基于物理的组装确保地面接触和物体间支撑。该系统直接通过顶点变形建模所有运动，无需类别先验或骨骼绑定，生成可用于下游物理仿真和编辑的水密网格场景。

Details

Motivation: 现有4D重建方法（如隐式场、高斯图元或点云）缺乏物理仿真和具身AI所需的水密拓扑、实例分离和标准化物理接口，OVOW旨在填补这一空白。

Result: 在两个合成基准测试（静态和4D）中，OVOW在所有基线中实现了最佳的整体布局和几何精度，以及最低的光度和语义误差；在单目视频上运行速度比基线快一到两个数量级，下游物理仿真证实了其物理稳定性。

Insight: 创新点包括：首个无需训练的实例级4D网格重建系统；通过直接顶点变形统一建模刚性和非刚性运动，避免类别先验；建立了首个结构化Video-to-4D评估基准，并可作为合成配对视频到4D仿真数据的可扩展引擎。

Abstract: We introduce \textbf{OVOW}, the first training-free system that reconstructs \emph{instance-level, simulation-ready} 4D mesh scenes from a single monocular video. Recent 4D reconstruction achieves impressive rendering quality, but its outputs (\eg, implicit fields, Gaussian primitives, or point clouds) lack the watertight topology, instance separation, and standardized physical interfaces required by physics simulators and embodied AI. OVOW closes this gap with a four-stage pipeline: a vision-language model discovers, labels, and motion-classifies all instances; category-aware reconstruction yields per-instance meshes for rigid objects and topology-consistent mesh sequences for deformable ones; an iterative render-match-optimize procedure recovers metric scale and 6-DoF pose trajectories; and physics-grounded assembly enforces ground contact and inter-object support. Crucially, we model all motion, rigid and non-rigid, through direct vertex deformation without category-specific priors or skeleton rigging, producing watertight mesh scenes ready for downstream physics simulation and editing. We further establish the first benchmark for \emph{structured Video-to-4D} evaluation, with metrics for geometric correctness, instance separation, and physical plausibility beyond visual fidelity; the same pipeline doubles as a scalable engine for \emph{synthesizing} paired video-to-4D simulation data for future 4D world models and embodied AI. Across two synthetic benchmarks (static and 4D), OVOW attains the best overall layout and geometry accuracy and the lowest photometric and semantic error among all baselines, and on monocular video runs one to two orders of magnitude faster than the baselines, while downstream physics simulation confirms its physical stability.

[69] Temporal Preservation over Processing: Diagnosing and Designing Spatiotemporal Single-Stage Video Detectors cs.CV | cs.AIPDF

Karam Tomotaki-Dawoud, Anna Hilsmann, Peter Eisert, Sebastian Bosse

TL;DR: 本文提出TemporalLens诊断框架和YOLO-3D架构，旨在评估和设计单阶段视频目标检测器的时空推理能力。TemporalLens通过受控扰动揭示模型是否真正利用时间信息，而YOLO-3D则通过骨干网络保持时间深度来提升性能。

Details

Motivation: 现有单阶段视频目标检测器在时间关键应用中部署时，其是否真正进行时空推理尚不明确，标准指标无法区分模型是依赖单帧信息还是跨时间推理，存在评估盲区。

Result: 在YOLO-3D架构中，通过在骨干网络中保持时间深度，实现了平均跨尺度mAP@50提升3.7个百分点（32帧条件下）；诊断显示堆叠2D模型在目标帧移除时崩溃，而时空模型能从早期帧恢复预测。

Insight: 创新点包括提出模型无关的时空依赖诊断框架TemporalLens，以及设计模块化实时时空检测器YOLO-3D，强调骨干网络的时间深度保留是性能提升的关键驱动因素，将时空推理问题转化为可测量、可操作的研究方向。

Abstract: Single-stage video object detectors are increasingly deployed in time-critical applications, yet it remains unclear whether these models genuinely reason over temporal context or merely exploit a single informative frame-a gap hidden by standard metrics, which reward correct predictions regardless of how they are reached. We address this from two complementary directions: first, we propose TemporalLens, a model-agnostic diagnostic framework probing temporal dependence through controlled perturbations, structured occlusions, temporal shuffling, redundancy injection, and resolution degradation, revealing whether a detector actually uses information across time. Applied to stacked-frame 2D detectors and our YOLO-3D architecture, it exposes behavioural differences invisible to mAP: stacked 2D models collapse when the target frame is removed, while spatiotemporal models recover predictions from earlier frames, a signature of real temporal reliance. Second, we detail YOLO-3D, a modular real-time spatiotemporal detector built on YOLOv8, and show that simply preserving temporal depth through the backbone is the dominant performance driver (+3.7 pp mAP@50 at 32 frames averaged across scales). Together, the diagnostics and architecture turn “does this detector reason over time?” into a measurable, actionable question.

[70] AeroVerse-SatAgent: UAV-Satellite Collaborative Spatial Reasoning Inspired by the Dual Visual Pathway Theory of Cognitive Neuroscience cs.CVPDF

Wenyi Zhang, Fanglong Yao, Youzhi Liu, Peng Hu, Zhengqiu Zhu

TL;DR: 本文提出SatAgent模型，这是一种受人类视觉双通路理论启发的无人机-卫星协同空间推理模型。该模型通过联合利用卫星和无人机视角，在复杂城市环境中实现鲁棒且准确的空间推理。作者还构建了首个大规模无人机-卫星协同多视角空间推理数据集SatAgent-SR130K。

Details

Motivation: 现有基于无人机的空间推理方法存在单视角感知易受遮挡和透视畸变影响，以及大多数视觉语言模型缺乏显式几何建模、依赖语义线索导致视角和尺度变化下推理不一致的问题。

Result: 实验表明，SatAgent在多样化任务上分别超越了最先进的通用基础模型和专用空间推理模型25.91%和11.69%，在复杂几何关系推理中达到了特别高的准确率。

Insight: 创新点包括：1) 受认知神经科学双视觉通路理论启发的协同推理框架；2) 将2D无人机特征提升为显式3D空间表示的几何感知3D重建编码器；3) 在统一BEV坐标系中集成跨视角特征的多视角拓扑-语义对齐模块；4) 鼓励视角不变表示的多视角一致性损失；5) 构建了首个大规模无人机-卫星协同多视角空间推理数据集。

Abstract: With the rapid advancement of aerospace embodied intelligence, enabling Unmanned Aerial Vehicles (UAVs) to autonomously understand and reason about complex environments has become increasingly important. However, existing UAV-based spatial reasoning approaches face critical limitations: single-view perception renders them vulnerable to occlusions and perspective distortions, while most VLMs lack explicit geometric modeling, relying on semantic cues and yielding inconsistent reasoning under viewpoint and scale variations. To address these challenges, we propose SatAgent, a UAV-Satellite collaborative spatial reasoning model inspired by the dual-pathway mechanism of the human visual system. By jointly leveraging satellite and UAV perspectives, SatAgent enables robust, accurate reasoning in complex urban environments. We first introduce a Geometric-Aware 3D Reconstruction Encoder that elevates 2D UAV features into explicit 3D spatial representations. Next, we design a multi-view topology-semantic alignment module integrating cross-view features within a unified BEV coordinate system. We further introduce a multi-view consistency loss encouraging viewpoint-invariant representations. Finally, we construct SatAgent-SR130K, the first large-scale UAV-Satellite collaborative multi-view spatial reasoning dataset. Experiments show SatAgent outperforms state-of-the-art general-purpose foundation models and specialized spatial reasoning models by 25.91% and 11.69%, respectively, across diverse tasks, achieving particularly high accuracy in complex geometric relationship reasoning.

[71] Think While You Map: Asynchronous Vision-Language Agents for Incremental 3D Scene Graphs cs.CVPDF

Deniz Bickici, Michael Pabst, Shohei Mori, Dieter Schmalstieg

TL;DR: 本文提出了一种异步架构的开放词汇3D场景图构建方法，将轻量级的在线建图与重量级的语义细化并行执行。该方法通过概率体素骨干网络增量式地维护稳定的物体身份，同时利用后台视觉语言模型（VLM）智能体逐步丰富场景图的语义信息，包括解决物体轨迹重复、附加细粒度视觉属性以及推导物体间空间关系。一个多目标帧调度器通过选择一小部分信息丰富的帧来分摊VLM的计算成本，使得在探索过程中场景图即可查询，且语义丰富度随时间增长。

Details

Motivation: 现有的开放词汇3D场景图方法通常采用‘先重建，后丰富’的两阶段顺序流程，导致在探索过程中场景图无法查询。作者认为这种顺序耦合是不必要的，因此提出异步架构来解决此问题。

Result: 在语义分割任务上（ScanNet, Replica），该方法与现有开放词汇3D场景图方法相当或更优。在三个视觉定位基准测试（Sr3D+, Nr3D, ScanRefer）上，其性能超越了先前的最先进方法，A@0.25指标提升了15.3到18.8个百分点。

Insight: 核心创新在于异步架构设计，将在线建图与语义细化解耦并行，实现了探索过程中的实时可查询性。此外，通过语义回环闭合解决重复物体跟踪、多目标帧调度器分摊VLM计算成本，以及增量式丰富语义信息，都是值得借鉴的技术点。

Abstract: Open-vocabulary 3D scene graph methods typically operate in two stages: first reconstruct, then enrich with vision-language models, leaving the graph unqueryable during exploration. We argue that this sequential coupling is unnecessary and propose an asynchronous architecture in which lightweight online mapping runs concurrently with heavyweight semantic refinement. A probabilistic voxel-based backbone maintains stable object identities incrementally, while background VLM agents progressively enrich the graph. This framework resolves duplicate object tracks through semantic loop closure, attaches fine-grained visual attributes and derives spatial relations between objects. A multi-target frame scheduler amortizes VLM cost by selecting a small set of informative frames that jointly cover multiple targets. The resulting scene graph is queryable during exploration and grows in semantic richness over time. Our method matches or outperforms existing open-vocabulary 3D scene graph methods on semantic segmentation (ScanNet, Replica) and surpasses the prior state-of-the-art across three visual grounding benchmarks (Sr3D+, Nr3D, ScanRefer) by 15.3 to 18.8 A@0.25. Project page: https://denizbickici.github.io/thinkgraphs/

[72] HVPNet: A Bio-Inspired Network for General Salient and Camouflaged Object Detection cs.CVPDF

Jiawei Xu, Qiangqiang Zhou, Zhouping Li, Yanjiao Shi, Yugen Yi

TL;DR: 本文提出HVPNet，一种受生物视觉启发的通用网络架构，用于显著目标检测（SOD）和伪装目标检测（COD）。该网络通过模拟视网膜多层信息整合的视网膜整合模块（RIM）和模拟视觉皮层分层处理的皮层解码器（CD），实现了简单而有效的多模态特征融合与解码。该方法在7个任务、22个数据集上取得了优异的精度-效率权衡。

Details

Motivation: 当前多模态SOD和COD研究通常依赖复杂的跨模态特征融合与解码结构，导致模型参数量过大且性能受限。受人类视觉系统高效处理显著与伪装目标的启发，本文旨在探索一种更简单、受生物启发的建模策略，以实现准确且高效的目标检测。

Result: HVPNet在涵盖7个任务（包括RGB、RGB-D、RGB-T、光流等4种模态）的22个基准数据集上进行了评估，结果表明其在精度和效率之间取得了出色的平衡，无需复杂技巧即可实现通用且高性能的检测。

Insight: 主要创新点在于从人类视觉处理过程（视网膜多层整合和视觉皮层分层处理）中汲取概念灵感，设计了RIM和CD模块，从而用更简洁的架构实现了有效的多模态特征集成与解码。这为构建通用、轻量且高性能的视觉检测模型提供了新的生物启发式设计思路。

Abstract: In recent years, most research on multimodal salient object detection (SOD) and camouflaged object detection (COD) typically aims to improve performance through complex cross-modal feature fusion and decoding structures. However, this approach leads to an excessively large model parameter scale and often fails to deliver satisfactory detection performance due to structural redundancy. In contrast, the human visual process is able to efficiently perform salient and camouflaged object identification without such complex structures. This contrast raises an important question: Can we draw conceptual inspiration from the human visual process to achieve a simpler modeling strategy, and still realize accurate and efficient object detection? To answer this question, we propose HVPNet, a simple yet general bio-inspired computational architecture. Drawing on the multi-layered information integration of the retina as a conceptual metaphor, we designed a Retinal Integration Module (RIM), which effectively integrates multimodal features through a level-specific multi-stage integration strategy. To fully exploit these features, we further design a cortical decoder (CD) that breaks down the decoding process into low- and high-level visual stages, abstracting the hierarchical processing in the human visual cortex. Benefiting from these designs, HVPNet can readily extend to seven tasks across four modalities. Without bells and whistles, it establishes an excellent accuracy-efficiency trade-off across 22 datasets spanning these seven tasks. Our code is available at https://github.com/jiaweiXu1029/HVPNet.

[73] SimpleSearch-VL: A Simple Recipe for Multimodal Agentic Deep Search cs.CVPDF

Ming Dai, Zhihong Lu, Jinjie Gu, Jiedong Zhuang, Yefeng Liu

TL;DR: 本文提出了SimpleSearch-VL，一个用于多模态智能体深度搜索的高效、可靠且实用的框架。其核心思想是通过改进智能体自身的搜索与验证过程来提升性能，而非依赖扩大数据、工具或辅助模型规模。具体通过因子化自适应采样（FAR）提升效率，通过证据验证推理确保可靠性，并通过轻量级工具接口和网页自总结实现实用性。

Details

Motivation: 旨在解决多模态智能体搜索任务中，传统方法过度依赖扩展数据、工具或模型组件的问题，转而聚焦于优化智能体自身的核心搜索与验证流程，以构建更高效、可靠且实用的系统。

Result: 仅使用5K条监督工具交织轨迹和2K条强化学习数据进行训练，SimpleSearch-VL将Qwen3-VL智能体基线在8B和30B-A3B变体上的平均性能分别提升了15.8和16.0个点。SimpleSearch-VL-30B-A3B模型取得了与智能体Gemini-3-Pro相竞争的性能水平。

Insight: 主要创新点在于将性能提升的核心从外部资源扩展转向智能体内部流程优化，具体包括：因子化自适应采样（FAR）策略以高效利用样本；显式的证据验证推理链（chain-of-thought verification）来评估检索信息的相关性；以及集成网页自总结的轻量级工具接口设计，增强了系统的独立性和实用性。

Abstract: We present SimpleSearch-VL, an efficient, reliable, and practical framework for multimodal agentic search. Its core idea is to improve the agent’s own search-and-verification process rather than scaling data, tools, or auxiliary model components. For efficiency, Factorized Adaptive Rollout (FAR) improves sampling efficiency by forming more informative training groups while using redundant samples to mitigate long-tail latency and expose hard samples. For reliability, SimpleSearch-VL performs evidence-verified reasoning, explicitly using chain-of-thought verification to assess the relevance of retrieved visual and textual cues to the original context. For practicality, SimpleSearch-VL keeps a lightweight tool interface and performs webpage self-summary within the agent, requiring no additional external dependencies. With only 5K supervised tool-interleaved trajectories and 2K RL data, SimpleSearch-VL improves Qwen3-VL agentic baselines by 15.8 and 16.0 average points for the 8B and 30B-A3B variants, respectively. The SimpleSearch-VL-30B-A3B model further achieves performance competitive with agentic Gemini-3-Pro.

[74] MV-GEL: Language-Driven Multi-View Geometric Entity Localization on Meshes cs.CVPDF

Kartik Bali, Roland Aydin

TL;DR: MV-GEL是一个用于在三维网格模型上根据自然语言查询定位精细几何实体（如面、边、体）的框架。其核心是通过一个名为GELviews的模块，根据语言提示的几何实体可观察性来优先选择最佳视角，然后利用基于视觉语言模型的分割主干进行处理，并通过几何感知的光线投射将预测结果映射回网格。该方法不依赖于CAD格式，仅需三维网格，在实验中显著优于基线方法。

Details

Motivation: 在CAD、机器人操作和科学仿真等领域，识别并定位三维对象中的精确几何实体（如边缘、平面区域、曲面）是基础任务。尽管现代视觉语言模型在图像领域的指代分割取得了进展，但将这种语言驱动的定位能力扩展到结构化三维几何中面临巨大挑战，因为三维对象的外观对视角高度敏感，单一视角可能导致目标实体被严重遮挡或变形。

Result: 实验表明，在面级别的IoU指标上，MV-GEL相比原始基线方法有高达1.7倍的提升；在边级别的F1分数上，有超过4.5倍的增益。该方法显著优于基于CLIP的视图采样和随机视图采样方法，尤其是在处理薄结构和视角敏感结构时表现突出。

Insight: 论文宣称的创新点在于提出了GELviews模块，这是一个基于语言提示条件对视角进行排序的模块，其核心思想是可靠的CAD实体定位依赖于选择能使查询实体最易于解释的视角。从客观角度看，该框架将语言驱动的可解释性视角选择与VLM分割及几何感知的后处理相结合，为在通用三维网格上进行细粒度几何实体定位提供了一种新颖且有效的解决方案。

Abstract: Identifying and grounding precise geometric entities, such as edges, planar regions, and curved surfaces within 3D objects, is foundational to computer-aided design (CAD), robotic manipulation, and scientific simulation. Although modern Vision Language Models (VLMs) have advanced referring segmentation (RIS) in the image domain, extending such language-driven localization to structured 3D geometry is substantially harder. The 3D object appearance is highly sensitive to viewpoints; a single perspective may render a target entity clearly observable, while another may suffer from severe occlusion or foreshortening. In this work, we attempt to solve these challenges with MV-GEL (Multi-View Geometric Entity Localization), a framework for localizing fine-grained geometric entities on polygon meshes from natural language queries. Our key insight is that reliable CAD entity (i.e., faces, edges or solids) localization depends on selecting views that make the queried entity maximally interpretable. We introduce GELviews, a prompt-conditioned ranking module that prioritizes viewpoints based on language prompted observability of geometric CAD entities. Selected views are processed by a VLM-based reasoning segmentation backbone, and predicted masks are lifted to the corresponding meshes via geometry-aware ray casting. Our framework is completely CAD agnostic and relies only on 3D meshes. Experiments show up to a 1.7X improvement in face-level IoU and over 4.5X gains in edge-level F1 compared to vanilla baselines, substantially outperforming CLIP-based and random view sampling, particularly for thin and view-sensitive structures.The dataset, code and trained checkpoints are available at https://github.com/kbali1297/MV-GEL.

[75] Localized Conformal Prediction for Image Classification with Vision-Language Models cs.CV | cs.LGPDF

Clément Fuchs, Tim Bary, Benoît Macq

TL;DR: 本文研究了视觉语言模型在自然图像分类任务中的局部化共形预测方法，通过开源实现评估了现有算法，发现直接使用余弦相似度效果有限，进而提出了一种非线性变换方法以在保持边际覆盖保证的同时显著减少预测集大小。

Details

Motivation: 共形预测在不确定性量化领域具有重要价值，但其完全条件保证难以实现；现有方法通过样本相似性调整预测集，但在图像分类任务中应用不足，尤其是在视觉语言模型背景下。

Result: 在自然图像分类基准测试中，提出的非线性变换方法在保持边际覆盖保证的前提下，实现了统计显著的预测集平均大小减少，优于非局部化基线方法。

Insight: 创新点在于针对视觉语言模型特征，设计了一种简单的非线性相似度变换，有效提升了局部化共形预测的性能；客观分析表明该方法为视觉任务中的不确定性量化提供了实用改进。

Abstract: Conformal predictions have attracted significant attention in the field of uncertainty quantification, mainly because of their strong marginal coverage guarantees. Full conditional guarantee is not an attainable goal, a well known fact in conformal predictions literature. As a result, several approaches have tried to approximate this behavior by adapting the conformal sets of test-time samples according to their similarity to calibration examples. Although the latter has gained traction and shown impressive performances for regression problems, its application to image classification remains under-explored. We conduct an extensive benchmarking on natural image classification tasks with vision-language models (VLMs), using our open source implementation of a recent localized conformal prediction algorithm. We show that straightforward usage of the cosine similarity between test-time and calibration visual features, an intuitive choice for VLMs, is not sufficient to improve over the non-local baselines. In response, we propose a simple non-linear transformation of the cosine similarities, which conserves marginal coverage guarantees and achieves statistically significant mean set sizes reduction. Code is available at https://github.com/cfuchs2023/lcp-vlm/.

[76] AugSplat: Radiance Field-Informed Gaussian Splatting for Sparse-View Settings cs.CVPDF

Lorenzo Lazzaroni, Riccardo Bollati, Daniel Barath, Michael Niemeyer, Keisuke Tateno

TL;DR: 本文提出了AugSplat框架，旨在解决稀疏视角下3D高斯溅射方法因初始几何质量不佳而导致重建质量下降的问题。其核心思想是利用神经辐射场从稀疏输入视图合成额外的虚拟视角图像，为高斯溅射的优化过程提供额外的监督信号，从而提升重建质量。

Details

Motivation: 神经辐射场在稀疏视角下重建鲁棒但渲染慢，而高斯溅射渲染快但其优化对初始几何（如点云）质量高度敏感，在稀疏视角下尤其成问题。本文旨在结合两者优势，用辐射场增强高斯溅射在稀疏视角下的性能。

Result: 在稀疏视角的mip-NeRF 360场景上的实验表明，AugSplat相比标准高斯溅射提升了重建质量。其中，分阶段版本（Staged AugSplat）取得了最强的平均性能，而双监督版本（Dual AugSplat）性能接近，且两者在推理时均保持了实时渲染能力。

Insight: 创新点在于提出了一种利用预训练辐射场进行视角增强的简单框架，为数据稀缺的高斯溅射优化提供了有效的辅助监督。其核心洞察是将辐射场作为数据增强器来弥补稀疏视角的监督不足，而非直接用于最终渲染，从而兼顾了重建质量和渲染速度。

Abstract: Generating high-quality novel views at real-time frame rates remains a central challenge in 3D vision, particularly in sparse-view scenarios. Neural radiance fields have demonstrated robust reconstruction from limited observations, but their reliance on volumetric rendering leads to high computational cost and slow inference. In contrast, Gaussian Splatting methods achieve real-time rendering through rasterization, but their optimization is highly sensitive to the quality of the initial geometry. This sensitivity becomes especially problematic in sparse-view settings, where limited observations often lead to incomplete or noisy point-cloud reconstructions. In this work, we present AugSplat, a simple framework for improving Gaussian Splatting in sparse-view regimes using radiance-field-based view augmentation. We first train a radiance field on the sparse input views and use it to synthesize additional images from nearby novel viewpoints, increasing the effective view-space coverage available for supervision. These synthetic views are then used as auxiliary supervision during Gaussian Splatting optimization. We study two variants: Staged AugSplat, which uses synthetic views for an initial optimization phase before switching to real images, and Dual AugSplat, which jointly trains on real and synthetic views with a decaying synthetic loss weight. Experiments on sparse-view mip-NeRF 360 scenes show that AugSplat improves reconstruction quality over standard Gaussian Splatting. Staged AugSplat achieves the strongest average performance, while Dual AugSplat provides a closely performing formulation that keeps real-image supervision active throughout training, and both variants preserve real-time rendering at inference.

[77] DPPE: Rethinking Camera-Based Positional Encoding for Scaling Multi-View Transformers cs.CV | cs.AIPDF

Shun Kenney, Teppei Suzuki

TL;DR: 本文提出了一种名为解耦姿态位置编码（DPPE）的新方法，用于改进多视图Transformer中的相机感知位置编码。该方法通过显式解耦旋转和平移信息，解决了传统基于相机参数的位置编码在模型训练规模扩大时出现的性能瓶颈问题，从而实现了更稳定的长期训练和更好的泛化能力。

Details

Motivation: 当扩大新视图合成（NVS）模型的训练规模时，传统的基于相机参数（如外参或投影矩阵）的位置编码会导致模型性能在训练后期停滞，这阻碍了Transformer在3D计算机视觉中的可扩展性。

Result: 在NVS任务上的广泛评估表明，DPPE即使在扩大训练设置下也能实现稳定的长期训练，并在外推设置（如处理更多视角和放大场景）中表现出优异的泛化性能。

Insight: 论文的创新点在于揭示了传统位置编码将旋转和平移信息存储在值向量的同一维度中会导致其独立识别的不确定性，从而限制了训练可扩展性；提出的DPPE通过显式解耦这两个分量来解决此问题，为多视图Transformer的缩放提供了更有效的位置编码方案。

Abstract: The remarkable scalability of Transformers has expanded their application to 3D computer vision, where camera-aware positional encoding is crucial for providing spatial cues in multi-view geometry. Recent advancements have established the practice of using camera parameters – such as extrinsics or projection matrices – as relative positional encoding into the query, key, and value vectors of the attention mechanism. However, when scaling up the training recipe of novel view synthesis (NVS) models with the camera-based positional encoding, we observe a significant issue: model performance stagnates in the late stages of training. In this paper, we investigate the cause of the performance bottleneck when scaling up and demonstrate that storing rotation and translation given by the positional encoding in the same dimensions of the value vector causes indeterminacy in their independent identification, hindering training scalability. To address this, we propose Decoupled Pose Positional Encoding (DPPE), a novel camera-based positional encoding that explicitly decouples rotation and translation. Extensive evaluations on NVS tasks demonstrate that DPPE enables stable long-term training even in scaled-up training setup. Furthermore, it exhibits superior generalization performance in extrapolation settings, such as handling an increased number of viewpoints and zoom-in scenarios.

[78] Token-Sparse Medical Multimodal Reasoning via Dual-Stream Reinforcement Learning cs.CV | cs.AIPDF

Kaitao Chen, Weiqian Zhao, Jiamin Wu, Qihao Zheng, Shangquan Sun

TL;DR: 本文提出了一种名为ViToS的双流强化学习框架，用于解决医学多模态推理中视觉证据稀疏的问题。该框架通过主动视觉令牌剪枝（VTP）来减少无关视觉令牌，并同时进行基于剪枝后的令牌稀疏推理，从而提升推理性能和效率。

Details

Motivation: 现有结合强化学习的视觉语言模型在医学图像推理中表现不佳，因为医学图像中关键的视觉证据通常非常稀疏。虽然剪枝非关键区域的视觉令牌能提升推理，但缺乏一个统一的强化学习框架来同时优化令牌剪枝和问答任务。

Result: 在七个医学基准测试上，ViToS方法将视觉令牌序列长度减少至原始的77%，同时在Lingshu-7B和HuatuoGPT-Vision-7B模型上分别实现了108.27%和104.16%的相对性能提升，达到了优越的性能和推理加速。

Insight: 创新点在于提出了一个双流强化学习框架，通过两个任务分支（定位和稀疏推理）共享策略模型，并引入交叉反馈顺序优化来解决耦合策略学习问题，避免了梯度冲突并促进了模型收敛，为医学多模态推理建立了高效范式。

Abstract: Vision-language models (VLMs) combining reinforcement learning (RL) ignite remarkable progress in multimodal reasoning, yet still struggle with medical images, which typically exhibit extremely sparse visual evidence to inform clinical decision-making. We recognize that pruning visual tokens outside the grounding region greatly enhances medical reasoning. However, a united RL framework for active visual token pruning (VTP) and medical multimodal reasoning remains unestablished. Here, we propose a dual-stream RL framework, ViToS, to fulfill token pruning and question answering. ViToS trains one policy model with two task branches, where one focuses on grounding while the other conducts token-sparse reasoning after VTP. Furthermore, we solve the coupled policy learning problem by introducing the cross-feedback sequential optimization, avoiding gradient conflict and facilitating convergence of the shared policy model. Evaluated on seven medical benchmarks, our method reduces visual tokens to 77% of the original sequence length while achieving a 108.27% relative performance on Lingshu-7B and 104.16% relative performance on HuatuoGPT-Vision-7B. Overall, ViToS delivers superior performance and inference speedup, establishing an efficient paradigm for medical multimodal reasoning.

[79] Technical Report of RoboSpatial Challenge at CVPR 2026: Selective Reasoning Activation and Reference-Frame Disambiguation for Embodied Spatial Reasoning cs.CVPDF

Yuxiang Xie, Qi Lv, Jianming Xing, Zijian Hong, Xiang Deng

TL;DR: 本文介绍了RoboSpatialBrain系统，这是为CVPR 2026 Embodied Reasoning in Action Workshop中的RoboSpatial挑战赛设计的解决方案。该系统基于RoboBrain2.5-8B-NV模型，通过两种无需训练的推理时机制——选择性推理激活策略和显式参考系消歧管道——来提升具身任务中的空间推理能力，并在RoboSpatial-Home基准上取得了80.9%的成功率，赢得了挑战赛第一名。

Details

Motivation: 动机在于解决视觉语言模型在具身任务中普遍存在的空间推理能力不足的问题，特别是针对RoboSpatial挑战赛中上下文和兼容性任务的具体需求。

Result: 在RoboSpatial挑战赛中取得了第一名，在RoboSpatial-Home基准上的整体成功率达到80.9%。

Insight: 创新点在于提出了两种无需额外训练、在推理时应用的机制：一是通过强制前缀激活和任务特定后提示来激发对任务的审慎推理；二是通过显式的参考系重定向管道来解决相机中心与物体中心视角的歧义问题。此外，论文还探索了在兼容性数据上微调基础模型及其与提示策略的交互作用。

Abstract: Vision-language models achieve strong general perception but often struggle with the spatial reasoning required for embodied tasks. We present RoboSpatialBrain, our submission to the RoboSpatial Challenge at the Embodied Reasoning in Action Workshop, CVPR 2026, built on RoboBrain2.5-8B-NV. RoboSpatialBrain combines two training-free, inference-time mechanisms: a forced prefix activation strategy paired with a task-specific post-prompt that elicits deliberate reasoning on context and compatibility tasks, and an explicit reference-frame redirection pipeline that resolves camera-centric and object-centric ambiguity for context tasks. We additionally explore fine-tuning RoboBrain2.5 on compatibility data and present a detailed analysis of its interaction with prompting. RoboSpatialBrain achieved first place in the RoboSpatial Challenge, with an overall success rate of 80.9% on RoboSpatial-Home. Code is available at https://github.com/YuxiangXie2003/RoboSpatialBrain.

[80] What Memory Do GUI Agents Really Need? From Passive Records to Active Task-Driving States cs.CVPDF

Chen Liu, Ling Chen, Hanzhang Zhou, Xu Zhang, Quyu Kong

TL;DR: 本文针对移动GUI智能体在长视野任务中面临的信息记忆与利用问题，提出了一种主动任务驱动记忆（ATMem）方法，将记忆从被动存储转变为主动维护的执行状态，并引入了在线强化学习方法STR-GRPO来学习选择性使用记忆。

Details

Motivation: 现有GUI智能体的记忆方法主要将记忆视为被动存储，在需要时检索过去观察到的数据，但检索到的值无法直接揭示其在当前工作流中的角色和状态，导致在长轨迹任务中容易因相似字段、重复值、干扰项和过时状态而产生重复或遗漏操作。

Result: 为评估智能体在长视野执行中能否完成所有范围内工作并避免范围外操作，作者构建了一个具有挑战性的移动基准测试，要求智能体从一系列几乎相同的条目中，对满足指令的每个条目执行操作，并拒绝违反约束的条目。

Insight: 核心创新在于将GUI智能体的记忆概念从被动记录转变为主动维护的任务驱动状态，使每个值都与其角色和当前状态相关联；同时，STR-GRPO方法通过对比使用记忆与不使用记忆的轨迹来估计记忆使用的收益，并结合记忆成本感知的奖励来鼓励高效记忆使用。

Abstract: Mobile GUI agents increasingly face long-horizon tasks that require reading, updating, and reusing task-relevant data across pages and applications. Existing memory methods treat memory largely as passive storage, where past observations are accumulated and retrieved when needed. Yet retrieving a value does not reveal its current role in the workflow. The agent must still infer from accumulated records whether the value should be used now, has already been used, or must wait for a later dependency. This implicit reconstruction becomes unreliable in long trajectories with similar fields, repeated values, distractors, and outdated states, causing repeated or missed operations. We propose Active Task Driving Memory (ATMem), which shifts GUI-agent memory from passive storage to an actively maintained execution state. ATMem maintains task-relevant information as a continually updated execution state that links each value to its role and current status, enabling action selection based on the current workflow state. We therefore introduce \textbf{STR-GRPO}, an online reinforcement learning method that learns to use ATMem selectively according to its contribution to task completion. STR-GRPO contrasts memory-on and memory-off rollouts to estimate when memory use improves execution, while memory-cost-aware reward discourages costly memory usage that does not improve execution. To evaluate whether agents can complete all in-scope work while avoiding out-of-scope actions over long-horizon execution, we build a challenging mobile benchmark. From a list of near identical entries, agents must act on every entry that satisfies the instruction and reject entries that violate its constraints.

[81] Robust Autonomous UAV Landing on Maritime Platforms via Multimodal Agentic AI and Active Wave Compensation cs.CV | cs.ROPDF

Francisco S. Neves, Pedro N. Pereira, Raul D. S. G. Campilho, Andry M. Pinto

TL;DR: 本文提出了一种解耦的多飞行器着陆框架，用于在随机海况下实现无人机在海上平台的自主安全着陆。该框架通过同步配备3-RPU稳定平台的无人水面艇和鲁棒无人机，利用两个独立的深度强化学习代理分别进行高频波浪运动补偿和无人机最终进近控制。

Details

Motivation: 解决海上基础设施自主空中检查中因随机海况导致的高动能冲击、着陆后翻倒和传感器遮挡等风险问题。

Result: 在高保真度海上模拟中，系统在从平静到恶劣的不同波浪状态下进行了15次试验，实现了100%的着陆成功率；在恶劣条件下，平均稳定效能为87.8%，着陆面在96%的任务时间内保持在水平面1度以内。

Insight: 创新点在于采用解耦的多飞行器架构，并结合了用于平台稳定补偿的SAC代理和用于无人机进近的多模态RL代理；从客观角度看，这种将主动波浪补偿与多模态智能体决策相结合的方法，为动态环境下的精确着陆提供了新思路。

Abstract: Autonomous aerial inspection of marine infrastructure is frequently compromised by stochastic sea states, introducing risks of high-kinetic impacts, post-landing toppling, and sensory occlusion. This paper proposes a decoupled, multi-vehicle landing framework synchronizing an Unmanned Surface Vehicle (USV) equipped with a 3-RPU stabilized platform with a robust Unmanned Aerial Vehicle (UAV). The architecture utilizes two independent Deep Reinforcement Learning (DRL) agents: a Soft Actor-Critic (SAC) agent providing high-frequency wave-motion compensation for the landing deck, and a multimodal RL agent for the UAVs final approach. Evaluated in high-fidelity maritime simulations, the system achieved a 100% landing success rate across 15 trials in wave states varying from calm to rough. Results show a mean stabilization efficacy of 87.8%, maintaining the landing surface within 1 degree of the horizontal plane for 96% of the mission duration in rough conditions, effectively contributing to safer landings.

[82] WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Models cs.CV | cs.AIPDF

Ting-Bing Xu, Jiacheng Sui, Zhe Gao, Kewei Shi, Wenjin Yang

TL;DR: 论文提出了WorldRoamBench，一个用于评估交互式世界模型（IWMs）长时程稳定性的开放世界基准测试。该基准从动作、视觉、物理和记忆四个维度进行综合评估，包含600多个测试用例，覆盖自然、城市和室内场景。评估发现，现有模型均无法在所有维度上可靠地满足要求。

Details

Motivation: 现有基准测试仅在轨迹层面评估动作跟随，忽略了记忆和交互物理，无法全面衡量交互式世界模型在长时程交互中的稳定性。

Result: 在WorldRoamBench上评估了10多个开源和闭源模型，结果显示没有一个模型能可靠地满足所有维度；即使最好的模型也只取得了中等分数。

Insight: 创新点在于提出了一个多维度、细粒度的评估框架：设计了绕过跨模型语义尺度差异的逐帧动作度量、捕捉非单调序列崩溃的基于片段的视觉漂移度量、基于可控性门控的物理一致性评估，以及解耦动作的场景与主体记忆评估协议。这为构建稳定、物理可信、记忆忠实且可部署的交互式世界模型提供了关键的评估工具。

Abstract: Despite rapid progress in interactive world models (IWMs), existing benchmarks evaluate action following only at trajectory level and ignore memory and interaction physics. We introduce WorldRoamBench, an open-world benchmark for long-horizon stability across four dimensions, each with tailored innovations: (i) Action: per-frame action metric bypassing cross-model semantic scale disparity and exposing failures hidden by trajectory; (ii) Vision: segment-based drift metric capturing non-monotonic mid-sequence collapse missed by start-vs-end comparisons; (iii) Physics: controllability-gated evaluation over mechanics, optics, and 3D consistency, scoring plausibility under faithful action execution; (iv) Memory: action-decoupled protocol evaluating scene memory via transition-localized 3D point-cloud reconstruction and subject memory via tracking-plus-VLM reasoning. The benchmark comprises 600+ test cases across Nature, Urban, and Indoor scenes in first/third-person views with WASD 10-60s continuous interaction. Evaluating 10+ open/closed-source models reveals none reliably satisfies all dimensions; even the best achieves only moderate scores. Advances on WorldRoamBench are steps toward IWMs that are stable, physically grounded, memory-faithful, and deployable in real-world applications.

[83] MemLearner: Learning to Query Context memory for Video World Models cs.CVPDF

Jiwen Yu, Jianxiong Gao, Jianhong Bai, Yiran Qin, Kaiyi Huang

TL;DR: MemLearner是一种用于视频世界模型的学习型自适应上下文查询方法，通过查询令牌桥接上下文与预测令牌，以解决长时视频生成中因缺乏记忆导致的场景不一致问题。

Details

Motivation: 现有基于规则的上下文帧检索方法在场景遮挡和动态对象场景下泛化能力不足，因此需要一种能够自适应查询上下文记忆的方法来提升视频世界模型的长期一致性。

Result: 在包含遮挡和动态对象的挑战性场景下，MemLearner在场景一致性和记忆能力方面显著优于先前的视频世界模型。

Insight: 创新点在于利用视频生成模型自身的预训练视觉先验进行上下文查询，无需从头训练额外模块，并提出了结合标注渲染视频与无标注真实视频的多数据集训练策略。

Abstract: Video World Models are interactive video generation models that predict future world states based on user actions and history video frames. A critical challenge in video world models is the lack of memory, causing inconsistent generated scenes over extended durations. Previous methods explored rule-based context frame retrieval as memory, but they fail to generalize in scenarios with scene occlusions and dynamic objects. We propose MemLearner, a learning-based adaptive context query method using query tokens to bridge context and predicted tokens. By leveraging the video generation model itself for context querying, MemLearner exploits pre-trained visual priors without training additional modules from scratch, and incorporates efficient strategies for training and inference. We collect a dataset of long videos with scene occlusions and dynamic objects, paired with camera pose annotations, and propose a multi-dataset training strategy leveraging both annotated rendered and unannotated real-world videos. Extensive experiments demonstrate that MemLearner significantly outperforms prior video world models in terms of scene consistency and memory, particularly under challenging occlusion and dynamic scenarios.

[84] REDI: Corpus Aware Patch Ranking for DINOv3 Token Reduction cs.CVPDF

Chanjong Im, Sebastian Diem, Thomas Mandl

TL;DR: REDI是一种针对DINOv3视觉Transformer的token缩减方法，通过结合类别特定的语料库统计量（监督TF-IDF）和图像特定的注意力图来对图像块进行排序和分配预算，在保持或提升分类精度的同时显著减少序列长度。

Details

Motivation: 研究如何在固定的token预算下，为图像分类任务最优地分配图像块，以探索准确性和效率之间的有利权衡。

Result: 在ImageNet-1K上，基于传入注意力质量的REDI变体实现了84.706%的Top-1准确率，优于稠密基线（83.514%）、仅使用注意力（82.634%）和仅使用监督TF-IDF（81.796%），并将序列长度从201个token减少到107个（减少46.8%）。

Insight: 创新点在于将类别特定的语料库统计信息（通过监督TF-IDF量化视觉词汇）与图像特定的注意力信号相结合，为patch排序提供了互补且更有效的信号；该方法表明预计算的语料库知识可以增强动态的注意力机制，实现更高效的token缩减。

Abstract: Most token reduction methods for Vision Transformers seek favorable tradeoffs between accuracy and efficiency by pruning, merging, or pooling patch tokens. REDI (Relevance for DINOv3 Token Reduction) studies this question through a controlled supervised reference: how should a fixed token budget be allocated across patches for image classification? REDI quantizes final block DINOv3 patch representations into a visual vocabulary and derives class conditioned corpus scores using supervised TF-IDF over visual words. For each validation image, the ground truth class selects a row of the TF-IDF table, and four transformed views produce a TF-IDF map aligned to a reference center crop. A separate dense pass on the same crop provides an attention map. After independent min max normalization, their elementwise product defines the REDI score. A fixed keep, merge, and compress operator then uses score rank to assign patch roles and score magnitude to weight merging and compression. With precomputed REDI scores, a frozen DINOv3 ViT-B/16 backbone, and the same linear classifier used for dense evaluation, the operator reduces the sequence length from 201 to 107 tokens, a 46.8% sequence reduction. The REDI variant based on incoming attention mass achieves 84.706% Top-1 accuracy on ImageNet-1K, compared with 83.514% for the dense baseline, 82.634% for incoming attention mass alone, and 81.796% for supervised TF-IDF alone. The same corpus term also improves reduced classification for three alternative attention formulations relative to their attention only counterparts. Together, these controlled comparisons indicate that class specific corpus statistics and image specific attention provide complementary signals for patch ranking in this setting.

[85] UniCoder: Unified Visual-to-Code Generation via Symbolic Rewards and Reference-Guided Code Optimization cs.CVPDF

Yaozhi Zheng, Yilei Jiang, Manyuan Zhang, Yuxuan Wan, Kaituo Feng

TL;DR: 本文提出了UniCoder，一个统一的强化学习框架，用于解决视觉到代码生成任务中像素级对齐的难题。该框架通过引入符号属性对齐来提供细粒度奖励，并利用参考引导的代码优化策略来克服探索停滞问题，从而显著提升了模型在多个基准测试上的性能。

Details

Motivation: 标准的多模态大语言模型仅通过监督微调难以实现视觉到代码生成所需的像素级精确对齐，而强化学习又面临奖励粗糙和探索停滞两大障碍。

Result: 在ChartMimic、UniSVG、Design2Code和ScreenBench基准测试上，其8B参数的模型超越了所有开源基线，并达到了与专有模型相当的最先进水平。

Insight: 创新点在于提出了符号属性对齐机制，利用轻量级辅助LLM解析代码为离散视觉属性以计算密集奖励，以及参考引导的代码优化策略，将真实轨迹动态注入低性能组以引导策略改进，为通用视觉到代码合成建立了新范式。

Abstract: Visual-to-Code generation, which transforms scientific plots, vector graphics, and webpages into executable scripts, demands a level of pixel-precise alignment that standard Multimodal Large Language Models (MLLMs) fail to achieve through Supervised Fine-Tuning (SFT) alone. While Reinforcement Learning (RL) offers a theoretical pathway to bridge this gap, its application is hindered by two fundamental obstacles: (1) \textit{Reward Coarseness}, where semantic metrics like CLIP scores fail to penalize fine-grained element deviations, and (2) \textit{Exploration Stagnation}, where the sparse, heterogeneous code search space prevents the policy from bootstrapping valid trajectories. To overcome these limitations, we introduce UniCoder, a unified RL framework that integrates two novel mechanisms. First, we propose \textbf{Symbolic Attribute Alignment}, which employs a lightweight auxiliary LLM to parse generated code into discrete visual attributes (e.g., hex colors, coordinate limits), enabling dense, element-wise reward computation. Second, to escape local optima, we devise \textbf{Reference-Guided Code Optimization}, a strategy that dynamically injects ground-truth trajectories into low-performing rollout groups, transforming blind exploration into guided policy improvement. Extensive experiments on ChartMimic, UniSVG, Design2Code and ScreenBench benchmarks demonstrate that our 8B-parameter model not only surpasses all open-source baselines but also achieves state-of-the-art performance comparable to proprietary models, establishing a new paradigm for generalized visual-to-code synthesis.

[86] Rhythm-Structured Predictive Learning for Remote Photoplethysmography cs.CVPDF

Ba-Thinh Nguyen, Huu-Dung Nguyen, Thi-Duyen Ngo, Thanh-Ha Le

TL;DR: 本文提出RhythmJEPA，一种用于远程光电容积描记术（rPPG）的节奏结构化联合嵌入预测学习框架。该框架通过预测潜在教师表征而非重建RGB帧，鼓励在嵌入空间中进行生理感知的表征学习，并引入循环节奏状态规划器（CRSP）和双序Mamba编码器（DOM）来显式建模脉冲信号的循环时间结构，以捕获局部时间连续性和长程节奏一致性依赖。

Details

Motivation: 现有自监督rPPG方法主要重建掩码像素或低级视觉表征，可能导致模型偏向面部外观而非潜在生理动态；同时，大多数基于Mamba的方法仅按时间顺序扫描面部视频标记，限制了利用脉冲信号循环结构的能力。

Result: 在PURE、UBFC-rPPG和MMPD数据集上的实验表明，该方法在代表性rPPG方法中具有竞争力。

Insight: 创新点包括：采用联合嵌入预测学习以避免外观偏差；设计CRSP通过动态规划和约束转移语法解码循环状态路径来显式建模生理节奏；结合时间顺序和状态顺序扫描的双序Mamba编码器（DOM）有效捕获时间依赖；轻量级空间脉冲混合器（SPM）在复杂性和性能间取得平衡。

Abstract: Remote photoplethysmography (rPPG) estimates physiological signals from facial videos by analyzing subtle pulse induced skin color variations. Despite recent progress, existing self-supervised rPPG methods mainly reconstruct masked pixels or low-level visual representations, which can bias the model toward facial appearance rather than latent physiological dy namics. Moreover, most recent Mamba-based approaches scan facial video tokens only in chronological order, limiting their ability to exploit the cyclic structure of pulse signals. To ad dress these limitations, we propose RhythmJEPA, a rhythm structured joint-embedding predictive learning framework for rPPG. Instead of reconstructing RGB frames, RhythmJEPA predicts latent teacher representations from masked facial videos, thereby encouraging physiology-aware representation learning in the embedding space. To explicitly model pulse-related tem poral structure, we introduce a Cyclic Rhythm-State Plan ner (CRSP), which estimates frame-wise latent physiological states and decodes the most plausible cyclic state path via dynamic programming with a constrained transition grammar. Guided by the decoded states, we further design a Dual Order Mamba Encoder (DOM), which combines conventional chronological scanning with state-ordered scanning to capture both local temporal continuity and long-range rhythm-consistent dependencies. Finally, a lightweight Spatial Pulse Mixer (SPM) extracts compact pulse-sensitive facial tokens with a favorable balance between complexity and performance. Experiments on PURE, UBFC-rPPG, and MMPD show competitive performance over representative rPPG methods. The codes are available at https://github.com/deconasser/RhythmJEPA.

[87] Generative Lane Topology Reasoning via Autoregressive Model with Geometry Prior cs.CVPDF

Jiahui Fu, Zehao Huang, Han Li, Naiyan Wang, Si Liu

TL;DR: 本文提出TopoGPT，一种基于自回归序列建模的生成式框架，用于从车载传感器观测中构建车道拓扑图。该方法通过在大规模地图数据集上预训练，学习车道图结构的几何先验，并利用感知适配器将先验知识迁移到基于传感器的车道图预测任务中。

Details

Motivation: 现有车道拓扑推理方法遵循检测与关联范式，独立处理每个车道实例，导致连接端点处的几何不一致以及因视觉遮挡而产生的不完整图。本文旨在解决这些问题。

Result: 在OpenLane-V2基准测试中，TopoGPT在车道级指标上平均提升+6.4，在点级指标上平均提升+11.6，优于现有方法，并能生成几何一致且结构完整的车道图。

Insight: 创新点在于将车道图构建视为序列生成问题，通过自回归建模学习几何先验，并使用感知适配器实现从地图先验到传感器观测的迁移，从而提升几何一致性和结构完整性。

Abstract: Lane topology reasoning aims to construct a lane graph from onboard sensor observations. Existing methods follow a detection and association paradigm that treats each lane instance independently, leading to geometric inconsistency at connected endpoints and incomplete graphs due to visual occlusions. To address these issues, we propose TopoGPT, a generative framework that learns the geometry prior from typical lane graph structures through autoregressive sequence modeling. Specifically, we construct a large-scale map dataset comprising 3.3M scenes. For each lane graph, a lane tokenizer serializes it into discrete tokens, while a scene context encoder converts it into a rasterized image and extracts global features as scene tokens. We pre-train an autoregressive lane sequence transformer via scene-conditioned next-token prediction, endowing the model with the geometry prior over lane graph structures. Building upon this prior, a perception adapter aligns BEV features from multi-view images with the pre-trained scene condition, transferring the learned geometry prior to sensor-based lane graph prediction. On the OpenLane-V2 benchmark, TopoGPT outperforms existing methods by an average of +6.4 on lane-level and +11.6 on point-level metrics, and produces geometrically consistent and structurally complete lane graphs.

[88] JL1-CC&QA: Extending the JL1-CD Benchmark with Change Captioning and Question Answering cs.CV | cs.AIPDF

Ziyuan Liu, Ruifei Zhu, Ouqiao Ma, Yuantao Gu

TL;DR: 本文提出了JL1-CC&QA基准，它扩展了原有的JL1-CD遥感变化检测数据集，增加了变化描述和变化问答两个语义理解任务。该基准基于5000对吉林一号卫星获取的双时相图像，包含超过1.7万条质量验证的变化描述和超过2万条涵盖八种问题类型的变化问答对，旨在推动遥感领域从像素级变化检测向语义级变化理解的转变。

Details

Motivation: 传统遥感变化检测仅关注像素级的二元分割，只能定位变化区域，无法解释变化的类型和原因，存在语义鸿沟。为了解决这一问题，作者希望建立一个能支持细粒度、交互式变化理解的基准。

Result: 该工作构建了一个包含变化描述（JL1-CC）和变化问答（JL1-QA）两个子任务的多任务基准，其标注通过一个结合多模态大语言模型生成、视觉基础LLM判断和人类专家验证的三阶段流程完成。

Insight: 创新点在于将遥感变化检测从单纯的像素分割任务扩展为集成了变化掩码、自然语言描述和问答的多任务理解框架。其标注流程巧妙地利用多模态LLM进行初步生成和判断，再辅以人工验证，为构建高质量、大规模语义标注数据集提供了高效可行的技术路径。

Abstract: Remote sensing change detection (CD) traditionally focuses on pixel-level binary segmentation, which identifies where changes occur but neither what nor why. To bridge this semantic gap, we introduce JL1-CC&QA, a multi-task benchmark that extends the JL1-CD dataset with two complementary annotation layers: change captioning (CC) and change question answering (QA). Built upon 5,000 bi-temporal image pairs acquired by the Jilin-1 satellite at 0.5-0.75m ground sample distance, the benchmark comprises: (i) JL1-CC, providing 17,021 quality-verified captions that describe diverse land-cover transformations; and (ii) JL1-QA, offering 20,060 question-answer pairs across eight question types, enabling fine-grained, interactive interrogation of surface changes. All annotations are produced via a three-stage pipeline consisting of multi-modal large language model (LLM) generation, vision-grounded LLM judging, and human expert verification. We hope that JL1-CC&QA, as a benchmark unifying binary change masks, change captions, and change-oriented QA over the same image set, will serve as a valuable resource for the community to advance multi-task change understanding in remote sensing. The dataset is available at https://github.com/circleLZY/JL1-CD.

[89] MuSViT: A Foundation Vision Model for Sheet Music Representation cs.CVPDF

Carlos Penarrubia, Antonio Rios-Vila, Eliseo Fuentes-Martinez, Juan C. Martinez-Sevilla, Francisco J. Castellanos

TL;DR: 本文提出了MuSViT，首个面向乐谱表示的基础视觉模型，它是一个通过掩码自编码器在IMSLP的970万页乐谱上预训练的ViT编码器。模型采用两阶段课程学习策略，先在合成排版乐谱上预热，再在大规模真实乐谱上训练。在四个下游任务（全页/谱行乐谱识别、音乐符号检测、乐谱难度分类）上，无论是线性探测还是微调，MuSViT均表现出色，并证明其表示空间直接编码了符号化的音乐结构。

Details

Motivation: 基础模型在视觉和语言领域提供了可跨任务重用的丰富表示，但乐谱作为一种音乐语言的视觉编码，缺乏强大的领域专用骨干网络。本文旨在填补这一空白，为乐谱理解构建一个基础视觉模型。

Result: 在线性探测（冻结编码器）场景下，MuSViT持续优于现代通用视觉编码器，表明通用表示在捕捉乐谱结构化符号特性方面存在系统性不足。在微调场景下，MuSViT通常能超越特定任务的SOTA方法。嵌入-转录一致性分析进一步证实其表示空间与音乐符号内容直接相关。

Insight: 创新点在于首次为乐谱领域构建了大规模预训练的基础视觉模型，并采用两阶段课程学习策略处理真实乐谱的复杂性。其核心洞察是，通用视觉表示无法充分捕捉乐谱特有的结构化符号语义，而领域专用的预训练能直接学习到符号音乐结构，这为其他结构化视觉领域（如图表、公式）的表示学习提供了借鉴。

Abstract: Foundation models have transformed vision and language processing by providing rich, reusable representations that transfer across diverse tasks. Sheet music, as a visual encoding of musical language, lacks such a strong domain-specific backbone. We introduce MuSViT (Music Score Vision Transformer): the first foundation vision model for sheet music representation – a ViT encoder pre-trained via Masked Autoencoders on 9.7 million pages from the IMSLP. To handle the complexity of real-world scores, we adopt a two-stage curriculum: a synthetic warm-up on typeset scores followed by large-scale training on the full IMSLP corpus. We evaluate MuSViT on four downstream tasks – full-page and staff-level music score recognition, music symbol detection, and score difficulty classification – under two scenarios: linear probing (frozen encoder) and fine-tuning. Under linear probing, MuSViT consistently outperforms modern vision encoders, revealing that general-purpose representations, regardless of scale, fall systematically short on the structured symbolic properties of musical notation. Under fine-tuning, MuSViT generally improves upon task-specific state-of-the-art methods. An additional embedding-transcription consistency analysis reveals that MuSViT encodes symbolic musical structure directly in its representation space – unlike other encoders, whose embeddings do not correlate with music notation content. These results establish MuSViT as a foundation backbone for sheet music understanding.

[90] Breaking Failure Cascades: Step-Aware Reinforcement Learning for Medical Multimodal Reasoning cs.CV | cs.AIPDF

Junha Jung, Minbyul Jeong, Suhyeon Lim, Sungwook Jung, Jaehoon Yun

TL;DR: 本文提出了一种名为医学推理感知策略优化（MRPO）的强化学习算法，旨在解决医学视觉问答（VQA）中因早期推理错误导致的级联失败问题。该方法通过引入逐步过程奖励，在最终答案错误时对早期无效推理步骤中的标记施加指数级惩罚，从而优化推理过程。实验表明，MRPO在多个多模态大语言模型骨干上均优于现有基线方法，显著减少了早期推理失败并提升了最终答案准确率。

Details

Motivation: 现有临床图像推理的多模态大语言模型训练流程主要关注最终答案正确性或序列级偏好，存在信用分配稀疏的问题，难以优化对临床应用至关重要的推理过程。分析发现，医学VQA基准测试中早期推理失败导致的级联错误是预测错误的主要原因。

Result: 在三个多模态LLM骨干网络上，MRPO均一致优于标准GRPO和最近的RL基线方法；在Qwen3-VL-8B-Instruct模型上，其表现甚至大幅超越HuatuoGPT-Vision-34B等更大的医学MLLM模型2.79个点。此外，MRPO将早期推理失败率从64.0%降低至13.0%。

Insight: 论文的创新点在于提出了一个逐步过程奖励机制，通过指数惩罚早期无效推理步骤来打破失败级联，而不影响成功路径。从客观角度看，这种针对推理过程而非仅最终结果的细粒度优化策略，为改善多模态模型在安全关键领域（如医疗）的可靠性和可解释性提供了新思路。

Abstract: Recent multimodal large language models have shown great promise in clinical image reasoning, but existing post-training pipelines remain predominantly outcome-centric, relying on final answer correctness or sequence-level preferences. This suffers from sparse credit assignment, making it difficult to optimize the reasoning process essential for clinical applications. Our analysis reveals that cascading errors from early-stage reasoning failures are a leading cause of incorrect predictions in medical visual question answering (VQA) benchmarks. Motivated by this, we propose Medical Reasoning-aware Policy Optimization (MRPO), an RL algorithm that incorporates step-wise process rewards. When the final answer is incorrect, MRPO assigns exponentially larger penalties to tokens in earlier invalid reasoning steps, breaking failure cascades without compromising successful paths. Across three multimodal LLM backbones, MRPO consistently outperforms standard GRPO and a recent RL baseline, and on Qwen3-VL-8B-Instruct even surpasses substantially larger medical MLLMs such as HuatuoGPT-Vision-34B by 2.79 points. Moreover, MRPO reduces early-stage reasoning failures from 64.0% to 13.0%, showing that targeted mitigation of cascading failures improves both reasoning quality and final answer accuracy. Our code is available at https://github.com/dmis-lab/MRPO

[91] Real-Time Source-Free Object Detection cs.CV | cs.AIPDF

Sairam VCR, Varun Gopal, Poornima Jain, Vineeth N Balasubramanian, Muhammad Haris Khan

TL;DR: 本文提出了一种实时无源目标检测方法RT-SFOD，旨在解决现有SFOD方法在域适应时模型庞大、速度慢的问题。该方法基于YOLOv10双头检测器，通过提出DHF伪标签融合策略和MARD多尺度自适应表征多样化损失，在保持推理结构不变的情况下，显著提升了域适应性能、推理速度和模型紧凑性。

Details

Motivation: 现实世界中的自动驾驶、监控和机器人等应用的目标检测器必须在严格的延迟和内存限制下处理域偏移，而现有的无源目标检测方法依赖于重型架构，只关注精度，牺牲了速度和模型大小。本文旨在打破这种权衡。

Result: 在多个域偏移基准测试上，RT-SFOD方法取得了1.4%到3.5%的mAP提升，吞吐量提高了1.3倍，参数数量比之前最先进的SFOD方法减少了约2倍，从而推进了速度-精度-模型大小的帕累托前沿。

Insight: 创新点在于：1）针对双头检测器提出了DHF伪标签融合策略，通过选择性融合O2O和O2M头的预测来优化域适应中的监督信号；2）提出了MARD损失，通过在多尺度特征图上施加检测感知的方差和协方差约束，缓解域偏移导致的多尺度特征判别性崩溃。这两个模块仅在训练时使用，不影响推理效率。

Abstract: Real-world detectors for autonomous driving, surveillance, and robotics must handle domain-shifts under strict latency and memory constraints, yet existing source-free object detection (SFOD) methods rely on heavyweight architectures that prioritize accuracy alone. We show this trade-off is unnecessary: building on YOLOv10, an NMS-free dual-head detector, we achieve state-of-the-art adaptation accuracy while being faster and more compact. We observe that directly applying vanilla mean-teacher self-training to dual-head detectors leads to suboptimal adaptation performance due to two key factors. First, simple pseudo-label generation strategies, such as using a single head or directly combining high-confidence predictions from both heads, yield suboptimal supervision under domain-shift. We propose DHF (Dual-Head Pseudo-Label Fusion) which selectively admits one-to-one (O2O) and one-to-many (O2M) head predictions, preserving precision and recovering missed objects. Second, we observe domain-shift collapses multi-scale feature discriminability. We propose the use of our MARD (Multi-scale Adaptive Representation Diversification) loss which mitigates this by enforcing detection-aware variance and covariance constraints on multi-scale feature maps. Both modules are training-time only, leaving inference unchanged. Across domain-shift benchmarks, our method, RT-SFOD yields 1.4 to 3.5% mAP gains, 1.3$\times$ higher throughput, with $\sim$2$\times$ fewer parameters than prior state-of-the-art SFOD methods, thus advancing the Pareto frontier of the speed-accuracy-model size trade-off. We report main results with YOLOv10, and demonstrate generalizability with additional YOLO- and DETR-based dual-head detectors. Code is available here: https://github.com/Sairam13001/RT-SFOD/

[92] Towards Voxel Spacing Consistency for Medical Image Segmentation cs.CVPDF

Xin You, Runze Yang, Minghui Zhang, Hanxiao Zhang, Han Li

TL;DR: 本文提出了一种名为Consispace的语义感知重采样框架，旨在解决医学图像分割中体素间距不一致的问题。该框架通过结合基于ODE的解剖约束和密集特征语义相关性，实现了轴向体素间距的一致性，同时保持解剖和语义的连续性，从而提升下游分割任务的性能。

Details

Motivation: 当前医学图像分割研究主要关注网络架构设计，而忽略了体素间距等物理属性对分割的影响；图像重采样作为常见的预处理步骤，其与下游分割任务的交互尚未被系统性地探索。

Result: 在多个数据集上的实验表明，Consispace在重建质量和感知保真度上表现优异，能生成更平滑的切片间解剖结构，并作为预处理步骤时显著提升了分割性能。

Insight: 创新点包括：引入基于ODE的连续插值器建模切片间动态，以处理复杂解剖过渡；利用预训练视觉模型的密集特征构建切片内语义相关性图，并通过特征重加权注入类别语义一致性；将约束集成到隐式神经网络中，支持任意尺度的重采样。

Abstract: Volumetric medical image segmentation is essential for both preoperative diagnosis and intraoperative guidance. While recent years have witnessed rapid progress in segmentation architectures, comparatively little attention is paid to the physical voxel spacing of anatomical data. Indeed, volumetric image resampling is a ubiquitous preprocessing step before segmentation, yet its interaction with downstream segmentation has not been systematically exploited. In this work, we study the correlation between image resampling and segmentation, and propose Consispace, a semantic-aware resampling framework that achieves consistent voxel spacing in the axial direction while preserving anatomical and semantic consistency. Consispace introduces an ODE-based anatomical constraint to model inter-slice dynamics with a continuous interpolator, enabling faithful reconstruction under complex anatomical transitions beyond discrete interpolation. To further couple resampling with segmentation objectives, we leverage dense features from a pretrained vision model to build intra-slice semantic correlation maps and inject class-wise semantic consistency via feature reweighting during resampling. Both intra-slice and inter-slice constraints are integrated into an implicit neural network, supporting arbitrary-scale resampling. Extensive experiments on multiple datasets demonstrate that Consispace achieves superior reconstruction quality and perceptual fidelity, produces smoother inter-slice anatomy, and improves downstream segmentation performance when used as a preprocessing step.

[93] SENSE-VAD: Sentient and Semantic Video Anomaly Detection for Autonomous Driving cs.CVPDF

Nghia T. Nguyen, Lokman Bekit, Yasin Yilmaz

TL;DR: SENSE-VAD是首个专门针对自动驾驶中社会性复杂异常设计的合成视频异常检测基准。它利用CARLA模拟器和Unreal Engine生成包含个体行为、群体行为、人-物交互等多类异常场景的视频数据集，并提供了真实世界视频用于模拟到真实的迁移测试。论文评估了现有先进方法，发现社会性异常构成了一个独特且尚未解决的挑战。

Details

Motivation: 当前自动驾驶系统的异常检测主要关注基于运动或外观的异常，但无法有效识别由智能体间社会关系构成的风险场景（如儿童脱离监护人、追逐行为等），这些场景在孤立运动信号上可能并不明显。

Result: 在SENSE-VAD基准上评估了最先进的视频异常检测基线方法，结果表明社会性复杂异常构成了一个独特且当前方法未能解决的挑战。

Insight: 创新点在于明确区分社会性异常与运动/外观异常，并构建了首个专注于社会语境异常的合成数据集；其数据生成框架和场景设计原则为研究社会感知的异常检测提供了新方向。

Abstract: Autonomous vehicles (AVs) must navigate not only motion-based hazards but also socially complex situations whose danger is constituted by inter-agent relationships rather than movement statistics alone. A child running away from a guardian, a person being carried by another, or a pursuer chasing a pedestrian across a sidewalk are all anomalous in social context, yet none produces an obvious motion signal that current anomaly detectors are equipped to flag. We introduce SENSE-VAD, the first synthetic video anomaly detection benchmark for autonomous driving explicitly designed around socially complex anomalies. Using the CARLA simulator and Unreal Engine (UE), we generate distinct anomaly scenarios across multiple categories: individual behaviors, group behaviors, person–object interactions, cyclist interactions, vehicle & agent, each annotated with per-frame binary labels. A key design principle is the separation of social anomaly from motion-based or appearance-based anomaly: many scenarios involve motion of objects that appears unremarkable in isolation but is anomalous in relational context. We additionally provide real-world normal and anomalous videos as a sim-to-real transfer probe. We evaluate state-of-the-art video anomaly detection baselines and demonstrate that socially complex anomalies constitute a distinct and currently unsolved challenge. Our dataset, annotations, and generation code are publicly available.

Shaozu Ding, Linan Song, Marco De Vincenzi, Dajiang Suo

TL;DR: 该论文提出了一个名为RESOLVE的大规模真实世界基准数据集，用于系统评估路边3D检测与跟踪中的单模态及融合架构。该数据集包含多分辨率激光雷达和同步相机-激光雷达感知数据，覆盖多种光照和天气条件，旨在探究传感器配置和场景感知条件导致的点云稀疏性变化下不同架构的性能。

Details

Motivation: 当前研究对单模态及相机-激光雷达融合架构在激光雷达点云稀疏性变化（由传感器配置和场景依赖的感知条件引起）下的行为缺乏深入探索，因此需要系统评估以指导高效路边多模态感知设计。

Result: 广泛的基准实验结果表明，多模态融合能够补偿激光雷达点云稀疏性，为设计成本效益高的路边多模态感知系统提供了线索。

Insight: 创新点在于引入了首个支持可控评估多分辨率激光雷达的基准数据集，允许在点云分布偏移（如分辨率变化、感知距离、训练-推理分辨率不匹配）下进行公平的跨架构比较，从而深入理解融合架构对稀疏性的鲁棒性。

Abstract: LiDAR has increasingly been integrated into traffic cameras to expand coverage and mitigate occlusion in roadside cooperative perception. However, how unimodal and camera-LiDAR fusion architectures behave under variations in LiDAR point sparsity induced by sensor configurations and scene-dependent sensing conditions remains underexplored. We introduce RESOLVE, a large-scale real-world benchmark dataset featuring multi-resolution roadside LiDAR and synchronized camera-LiDAR sensing for systematic evaluation of unimodal and fusion-based architectures in roadside 3D detection and tracking. RESOLVE contains over 100k images and 26k point cloud frames with 220k manually annotated bounding boxes, captured at a real-world urban intersection across diverse lighting and weather conditions and spanning 10 classes of traffic participants. In particular, RESOLVE enables controlled evaluation across three LiDAR resolution levels while keeping all other sensing and environmental factors fixed. This allows fair cross-architecture comparisons under point cloud distribution shifts resulting from resolution variations, sensing distance, and training-inference resolution mismatches. Results from extensive benchmark experiments reveal insights into how multimodal fusion can compensate for LiDAR point sparsity, offering clues for designing cost-efficient roadside multimodal perception. The dataset and benchmark codes are available at https://github.com/ASU-Suo-Lab/RESOLVE.

[95] InstanceControl: Controllable Complex Image Generation without Instance Labeling cs.CVPDF

Xiaoyu Liu, Huan Wang, Fan Li, Zhixin Wang, Jiaqi Xu

TL;DR: 本文提出InstanceControl，一种无需实例标注的多实例可控图像生成方法，通过视觉语言模型自动建立文本提示与视觉条件间的实例级对应关系，并引入自适应掩码细化策略提升生成质量。

Details

Motivation: 现有可控生成方法（如ControlNet）在处理复杂多实例场景时容易出现属性混淆问题，而依赖人工实例标注的方法又过于耗时费力。

Result: 在多个基准测试上的大量实验表明，该方法优于现有最先进方法，实现了更高的保真度和精确的实例级控制。

Insight: 创新点在于利用视觉语言模型自动解析文本中的实例描述并预测视觉条件中的实例掩码，从而建立实例级对应；同时通过自适应掩码细化策略动态优化噪声掩码，提升了复杂场景下的生成可控性。

Abstract: Controllable image generation methods, such as ControlNet, have demonstrated a remarkable capacity to introduce visual conditions(e.g., depth maps) to guide image generation. However, these methods often struggle with complex multi-instance scenes, frequently leading to attribute confusion among instances. While recent approaches attempt to mitigate this via manual instance labeling, such requirements are labor-intensive. In this paper, we propose InstanceControl, a novel multi-instance controllable generation method that eliminates the need for instance labeling. We identify the primary bottleneck in existing methods as the inability to accurately associate instance descriptions with their corresponding regions within visual conditions. To address this, we leverage the Vision-Language Model (VLM) to establish instance-level correspondences between text prompts and visual conditions. Specifically, the VLM automatically parses instance descriptions from the text prompts and simultaneously predicts instance masks based on the visual conditions. Furthermore, since the predicted masks may contain noise, we introduce an adaptive mask refinement strategy that dynamically refines these instance masks during the generation process. Extensive experiments demonstrate that our approach outperforms state-of-the-art methods, achieving superior fidelity and precise instance-level control.

[96] Attend, Transform, or Silence: Operator-Level Visual Skipping for Efficient Multimodal LLM Inference cs.CV | cs.AIPDF

Zhaoyang Luo, Runmin Dong, Miao Yang, Fan Wei, Yushan Lai

TL;DR: 本文提出了一种针对多模态大语言模型（MLLM）推理效率的细粒度加速方法。该方法从答案可观测的视角出发，发现后期视觉令牌更新对答案令牌表征影响甚微，存在‘答案沉默冗余’。基于此，作者将Transformer层分解为注意力（Attention）和前馈网络（FFN）算子，并提出一个算子级别的视觉令牌跳过框架，在保留完整视觉令牌序列的同时，选择性地绕过冗余的注意力、FFN或两者。

Details

Motivation: 现有MLLM加速方法通常粗粒度地移除视觉令牌或跳过整个层的更新，这可能导致丢弃细粒度证据或同时抑制有用与冗余算子。本文旨在解决这种计算冗余问题，实现更精细、高效的推理。

Result: 在三种MLLM架构和10个VQA基准测试上的实验表明，该方法实现了优异的效率-精度权衡。例如，在Qwen3-VL模型上，该方法减少了33.7%的TFLOPs，同时保留了99.5%的原始模型性能。

Insight: 核心创新点在于从算子层面（而非整个层或令牌层面）分析并跳过冗余的视觉令牌计算。其关键洞察是‘答案沉默冗余’现象，以及有用视觉计算具有算子主导和层依赖的特性，这为细粒度模型加速提供了新思路。

Abstract: Multimodal large language models (MLLMs) increasingly process long visual-token sequences, increasing the overall inference computation. Existing acceleration methods usually remove visual tokens or skip visual-token updates in entire layers, but these coarse strategies may discard fine-grained evidence or suppress useful operators together with redundant ones. In this paper, we study visual-token computation from an answer-observable perspective and find that late visual-token updates can remain large while having little effect on answer-token representations. Motivated by this answer-silent redundancy, we decompose each Transformer layer into attention and FFN operators and show that useful visual computation is often operator-dominant and layer-dependent. We propose an operator-level visual-token skipping framework that preserves the full visual-token sequence while selectively bypassing redundant attention, FFN, or both. Experiments across three MLLM architectures and 10 VQA benchmarks show that our method achieves strong efficiency-accuracy trade-offs, reducing \textbf{33.7%} TFLOPs on Qwen3-VL while retaining \textbf{99.5%} of the vanilla model performance.

[97] DriveWeaver: Point-Conditioned Video Inpainting for Controllable Vehicle Insertion in Autonomous Driving Simulation cs.CVPDF

Junzhe Jiang, Zipei Ma, Zijie Pan, Li Zhang

TL;DR: 本文提出DriveWeaver框架，用于自动驾驶仿真中的可控车辆插入。该方法基于视频修复技术，利用车辆点云作为条件，在掩码区域生成高质量、时序一致的车辆，并通过全局到局部的分层修复策略确保长期生成的一致性。同时，通过城市重建流程提取显式3D高斯表示以实现实时渲染。

Details

Motivation: 解决自动驾驶仿真中插入前景车辆时，现有方法依赖预重建3D资产导致的照明不一致问题，以及有限的手工资产阻碍大规模部署的挑战。

Result: 在多个数据集上的广泛实验表明，该方法在视觉真实感和几何一致性方面优于现有基线，为可扩展的自动驾驶场景增强提供了鲁棒工具。

Insight: 创新点在于将点云条件的视频修复用于车辆插入，确保了前景与背景的无缝融合和优越的泛化能力；全局到局部的分层修复策略保证了车辆身份和外观的一致性；结合3D高斯表示实现实时渲染，提升了仿真的实用性。

Abstract: A pivotal step in autonomous driving simulation involves inserting foreground vehicles with predefined trajectories into simulated scenes. This process enhances scene diversity and facilitates the creation of various corner cases for testing and improving autonomous driving models. However, existing methods often rely on pre-reconstructed 3D assets, which frequently lead to lighting inconsistencies between the inserted foreground and the background. Moreover, the reliance on limited, manually-curated 3D assets hinders large-scale deployment. To address these challenges, we propose DriveWeaver, a novel framework for controllable vehicle insertion in autonomous driving simulation. Specifically, for a masked target insertion area, DriveWeaver performs video inpainting conditioned on vehicle point clouds to generate high-quality, temporally consistent vehicles. This video-inpainting-based approach ensures seamless blending between the foreground and background, while the readily available point cloud conditions enable superior generalization. To support long-term generation, we further design a global-to-local hierarchical inpainting strategy, ensuring the consistent identity and appearance of the inserted vehicles. Meanwhile, we extract explicit 3D Gaussian representations of the inserted vehicles through an urban reconstruction pipeline to enable real-time rendering for autonomous driving simulation. Extensive experiments across diverse datasets demonstrate that our method outperforms existing baselines in visual realism and geometric consistency, providing a robust tool for scalable autonomous driving scene augmentation.

[98] No Place to Hide: Benchmarking Video Hallucination with Background-Controlled Pairs cs.CVPDF

Haojian Huang, Harold Haodong Chen, Meng Luo, Junjia Du, Shanqing Xu

TL;DR: 本文提出了VidPair-Halluc，一个用于在严格受控条件下评估大型视频模型（LVMs）视频幻觉问题的新基准。该基准通过PairFlow流程构建，包含背景高度相似但前景语义截然不同的视频对，以及相应的时空问答对，旨在精确归因模型错误于真实幻觉而非背景变化。评估表明主流LVMs在对抗性设置下对细粒度视频理解存在持续困难。

Details

Motivation: 现有基准主要依赖基于文本的扰动或对抗性问题，而忽略了视觉背景的一致性，难以精确评估模型是否产生真实幻觉。本文旨在通过控制背景变化，构建一个能更准确评估视频模型幻觉问题的基准。

Result: 在主流大型视频模型上的评估显示，模型在对抗性设置下（即背景相似但前景语义不同的视频对上）进行鲁棒的细粒度视频理解存在持续困难。

Insight: 核心创新在于通过构建背景高度相似但前景语义不同的“对抗性视频对”，将模型错误更精确地归因于幻觉，而非背景变化带来的混淆。这为评估视频模型的幻觉问题提供了一个更严谨、可控的基准构建方法论。

Abstract: We introduce VidPair-Halluc, a new benchmark for evaluating video hallucination in large video models (LVMs) under rigorous and controlled conditions. Unlike previous benchmarks that primarily rely on text-based perturbations or adversarial questions while neglecting the consistency of visual backgrounds, VidPair-Halluc features video pairs with highly similar backgrounds but distinctly different foreground semantics, enabling precise attribution of model errors to genuine hallucination rather than background variation. The benchmark is constructed through PairFlow, a pipeline that leverages recent advances in text-to-image and video generation to systematically compose stories, generate coherent video clips, and assemble them into adversarial pairs. Covering both spatial and temporal reasoning across ten semantic aspects, VidPair-Halluc comprises 1K high-quality adversarial video pairs and 11K spatio-temporal QA pairs with control over background and foreground variations. Evaluations on mainstream LVMs show persistent difficulty with robust fine-grained video understanding in adversarial settings, and code and data are available at the https://jethrojames.github.io/VidPair-Halluc/.

[99] World Narrative Model for Highly Controllable Video Generation: A Paradigm Shift from Pixel Sampling to Physical World Orchestration cs.CVPDF

Ye Chen, Xuanhong Chen, Yupeng Zhu, Liming Tan, Zhewen Wan

TL;DR: 本文提出了世界叙事模型（WNM），一种从像素分布采样转向物理世界编排的视频生成新范式。WNM将视频生成解耦为‘渲染什么’（结构化的物理叙事）和‘如何渲染’（像素生成过程），通过协作智能体将稀疏多模态输入转化为包含场景几何、物体布局、角色运动、摄像机轨迹和光照的完全可编辑4D世界表示，作为确定性蓝图驱动现有视频基础模型生成最终视频。

Details

Motivation: 现有工业级视频生成模型缺乏可控性，它们将视频视为像素分布采样问题，绕过了显式的、实例级的4D物理世界，导致创作者无法以确定、定量的方式指定几何、运动、摄像机参数或光照，陷入低效的‘抽卡’循环。

Result: 实验表明，WNM极大地减少了概率性的‘抽卡’调用，并生成了在布局、运动和摄影方面紧密遵循创作者意图的视频。

Insight: 核心创新在于范式转变：从端到端的黑盒采样转变为基于编排的4D预可视化，将视频生成解耦为世界叙事与神经渲染两个阶段。这提供了一个开放、模块化的框架，允许世界表示、控制智能体和适配器等组件独立改进，并将基础模型转变为忠实的‘神经着色器’。

Abstract: The fundamental obstacle to industrial grade video generation is the lack of controllability: existing models treat video as a pixel distribution sampling problem, bypassing the explicit, instance level $4D$ $(3D + T)$ physical world. Consequently, content creators cannot specify geometry, motion, camera parameters, or lighting in a deterministic, quantitative way, leading to the infamous ‘’gacha’’ loop that makes professional content creation prohibitively inefficient and expensive. To address this, we introduce the World Narrative Model (WNM), a paradigm that decouples what to render – the structured physical narrative – from how to render – the pixel generation process. WNM replaces end-to-end black-box sampling with orchestrated $4D$ pre-visualization for media generation. Collaborative agents translate sparse multimodal inputs, including text, reference videos, and sketches, into a fully editable world representation with scene geometry, object layouts, character/animal skeleton motion, trajectories, camera motion, and lighting at quantitative, physically meaningful granularity. This representation acts as a deterministic structural blueprint that drives existing video foundation models, either frozen or lightly adapted, to render final footage, turning the base model into a faithful neural shader. Built on this engine, our human-AI platform supports automatic world generation and pre-visualization aligned with professional filmmaking pipelines, while director consoles enable seamless human refinement. Experiments show that WNM greatly reduces probabilistic ``gacha’’ calls and produces videos whose layout, motion, and cinematography closely follow creator intent. The framework is open and modular, allowing each component, such as world representation, control agents, and adapters, to be independently improved. Project website: https://glassroom.sjtu.edu.cn/WNM/.

[100] LUNA: Learning Universal 3D Human Animation Beyond Skinning cs.CV | cs.AIPDF

Peng Li, Rawal Khirodkar, Junxuan Li, Yuan Dong, Chen Cao

TL;DR: LUNA是一种无需线性混合蒙皮（LBS）的通用神经动画模型，它能够直接将多种2D控制信号（如图像、关键点、草图）映射为3D高斯形变，从而生成逼真且可动画化的3D人体化身。该方法通过基于Transformer的运动回归器分离全局刚体运动与局部非刚性动态，并利用混合监督策略结合有限拟合数据与大量未标注视频进行训练。

Details

Motivation: 现有基于单目图像创建逼真3D人体化身的方法主要依赖线性混合蒙皮（LBS）和参数化人体模型，这些方法限制了表达能力并常因拟合不完美而产生伪影。LUNA旨在绕过显式人体拟合，直接从2D控制信号生成3D动画，以提升表达性和通用性。

Result: 大量实验表明，LUNA在视觉保真度上与基于LBS的方法相当，同时能生成逼真的人体运动，并支持跨身份零样本泛化，适用于多种驱动模态。

Insight: 创新点在于提出了首个支持隐式2D驱动的端到端3D可动画模型，其核心是通过Transformer架构解耦运动表示，并设计了结合教师模型知识蒸馏与无标签视频训练的混合监督策略，有效解决了2D到3D提升的模糊性问题并实现了大规模数据扩展。

Abstract: Creating photorealistic, animatable 3D human avatars from monocular images still largely depends on Linear Blend Skinning (LBS) and parametric body models, which constrain expressivity and often introduce artifacts due to imperfect fitting. We propose LUNA, an LBS-free universal neural animation model that directly maps multiple 2D controls like images, keypoints, sketches, and unseen characters into 3D Gaussian deformations, bypassing explicit body fitting. At its core, a transformer-based motion regressor disentangles global rigid motion from fine-grained local dynamics to capture both coherent movement and subtle non-rigid effects. To resolve the inherent ambiguity of 2D-to-3D lifting while scaling beyond fitted datasets, we introduce hybrid supervision that distills soft structural priors from an LBS teacher and a loss that supports training on both limited fitted data and large in-the-wild unlabeled videos. Extensive experiments show LUNA achieves competitive visual fidelity compared to LBS-based approaches, while delivering realistic human motion and zero-shot cross-identity generalization across diverse driving modalities. To the best of our knowledge, LUNA is the first end-to-end 3D animatable model that supports implicit 2D driving.

[101] ERA: Entropy-Guided Visual Token Pruning with Rectified Attention for Efficient MLLMs cs.CVPDF

Yuhao Wang, Mu Qiao, Haiwen Diao, Yunzhi Zhuge, Pingping Zhang

TL;DR: 本文提出了一种名为ERA的无需训练的高效多模态大语言模型视觉令牌剪枝框架，旨在解决现有方法因注意力分布扭曲而导致的’注意力对数塌缩’问题。该框架通过双视角熵剪枝、偏置感知令牌回收和对数保持注意力校正三个核心组件，在保持视觉证据的同时实现高效的令牌压缩，适用于单图、多图和视频等多种场景。

Details

Motivation: 多模态大语言模型由于处理长视觉令牌序列导致推理成本高昂，现有无需训练的视觉令牌剪枝方法会扭曲注意力分布，引发注意力对数塌缩现象，因此需要一种能有效校正该问题的高效剪枝方案。

Result: ERA框架在广泛的MLLMs上，在单图像、多图像和视频设置中均实现了鲁棒的性能，即使在激进的压缩下也能保持视觉证据，为高效MLLMs提供了实用的加速方案。

Insight: 创新点在于首次将保持对数作为视觉令牌剪枝的原则性框架，通过熵引导的剪枝与偏置校正的注意力机制，统一了理论基础、算法设计和实际部署，解决了注意力对数塌缩这一关键问题。

Abstract: Multimodal Large Language Models (MLLMs) incur prohibitive inference costs due to long visual token sequences. Training-free visual token reduction provides an efficient solution. However, existing methods distort attention distributions, giving rise to a phenomenon we term Attention Logit Collapse. To address this issue, we propose ERA, an Entropy-guided visual token pruning framework with Rectified Attention for efficient MLLMs. Specifically, ERA comprises three crucial components: Dual-view Entropy Pruning (DEP), Bias-aware Token Recycling (BTR), and Logit-preserving Attention Rectification (LAR). First, DEP identifies representative anchor tokens by jointly modeling visual diversity and head-wise saliency. BTR then recycles pruned tokens into their corresponding anchors while estimating a cluster-level logit bias. Building upon this, LAR injects the estimated bias into attention logits, effectively rectifying the collapse induced by token reduction. Together, these components preserve visual evidence even under aggressive compression, enabling robust performance across single-image, multi-image, and video settings on a wide range of MLLMs. Beyond delivering practical acceleration, ERA establishes logit-preserving visual token pruning as a principled framework for efficient MLLMs, unifying theoretical foundation, algorithmic design, and practical deployment. The code is at https://github.com/924973292/ERA.

Lianyu Hu, Shengqian Qin, Zeqin Liao, Qing Guo, Liang Wan

TL;DR: 本文提出了CoLT（Chain of Latent Thoughts）框架，旨在通过隐式思维链而非显式文本链来提升多模态大语言模型（MLLMs）的推理效率。该方法利用轻量级外部解码器进行前向和后向监督，以稳定训练隐式推理步骤，并在推理时移除解码器以保持高效。实验表明，CoLT在多个基准测试中超越了现有隐式推理方法，并显著减少了推理时间和文本解码时间。

Details

Motivation: 解决基于文本的思维链（CoT）推理在推理时速度慢、受自然语言表达能力限制的问题，旨在实现更高效、更紧凑的多模态推理。

Result: 在八个基准测试中，CoLT不仅超越了CODI、SIM-CoT等现有隐式推理方法，也优于依赖昂贵标注辅助图像的隐式视觉推理方法；与文本CoT方法相比，推理时间减少了10.1倍，文本解码时间减少了22.6倍。

Insight: 创新点在于用隐式思维链替代显式文本链进行推理，并通过前向/后向解码监督和内部监督来稳定训练过程；可借鉴之处在于高效隐式表示的监督训练机制，以及推理时移除辅助模块以保持效率的设计思路。

Abstract: Chain-of-thought (CoT) reasoning has enabled multi-modal large language models (MLLMs) to tackle complex visual reasoning tasks by generating explicit intermediate reasoning steps in natural language. However, this text-based reasoning paradigm is inherently slow at inference time with even thousands of tokens and fundamentally constrained by the expressiveness of natural language. In this paper, we propose CoLT, (Chain of Latent Thoughts), a novel framework that teaches multi-modal models to reason through a chain of latent thought representations instead of verbose text tokens, which can perform thinking with as few as 3 steps. Naively forcing the model to think with latent states easily produces meaningless semantics and makes training unstable. To effectively regulate the latent reasoning process, we introduce a lightweight external decoder that provides step-level supervision for each latent reasoning step in two complementary directions: a forward mode that decodes latent thoughts into the textual reasoning of the next step, and a backward mode that aligns decoder hidden states with the model’s latent thoughts given preceding textual context. We further incorporate internal supervision that encourages coherent step-by-step latent transitions. The decoder and internal supervision are removed during inference to maintain high efficiency of latent reasoning. Extensive experiments on eight benchmarks demonstrate that CoLT not only outperforms existing latent reasoning methods such as CODI and SIM-CoT, but also surpasses latent visual reasoning approaches that rely on auxiliary images with costly annotation requirements. Compared to text CoT methods, CoLT can notably reduce the inference time by 10.1$\times$ and text decoding time by 22.6$\times$. Code is released at https://github.com/hulianyuyy/CoLT.

[103] Automated Background Swapping for Robustness against Spurious Backgrounds cs.CV | cs.LGPDF

Cesar Roder, Kajetan Schweighofer

TL;DR: 本文提出了自动化背景交换（AutoBackSwap）方法，旨在减少深度神经网络分类器对虚假背景的依赖。该方法通过一个辅助网络分离图像的前景和背景，利用修复技术合成完整背景，并通过组合不同前景与修复背景来增强训练数据。实验表明，仅需少量样本的块级标注即可训练辅助网络，并在多个具有虚假背景的图像分类任务中有效提升模型鲁棒性。

Details

Motivation: 深度神经网络分类器在训练中可能依赖虚假相关性（如背景与标签的伪关联），导致泛化性能差。本文旨在解决图像分类中因虚假背景导致的模型脆弱性问题。

Result: 在多个具有虚假背景的图像分类任务中，AutoBackSwap方法一致优于先前方法，即使在训练数据中不存在任何打破虚假相关性的样本时也表现有效。

Insight: 创新点在于提出了一种自动化数据增强流程，通过前景-背景解耦与修复合成来减少模型对虚假背景的依赖；客观分析认为其优势在于仅需少量标注即可实现全数据集的自动增强，且不依赖打破虚假相关性的训练样本。

Abstract: Classifiers based on Deep Neural Networks exhibit strong performance across domains, yet can fail catastrophically if they rely on spurious correlations, i.e., features that are predictive of the target label in the training data but are not causally linked and thus fail to generalize. For the vision domain, many such spurious correlations manifest themselves within the background of the image, where only the foreground is predictive of the class label. In this paper, we introduce Automated Background Swapping (AutoBackSwap) to reduce the reliance of classifiers on such spurious backgrounds. AutoBackSwap uses a secondary network to disentangle the foreground and background, followed by infilling to synthesize complete backgrounds, and finally combines different foregrounds and inpainted backgrounds to augment the training data. We find that patch-wise labeling of just a few hundred samples suffices to train the secondary network and automatically augment the full training dataset on challenging image classification tasks. In contrast to many previous methods, AutoBackSwap proves very effective even if there is not a single sample in the training data breaking the spurious correlation. Across a range of image classification tasks with spurious backgrounds, AutoBackSwap consistently outperforms prior methods.

[104] SpheRoPE: Zero-Shot Optimization-Free 360 Panorama Generation with Spherical RoPE cs.CVPDF

Or Hirschorn, Aaron Olender, Eli Alshan, Ianir Ideses, Lior Fritz

TL;DR: 本文提出SpheRoPE，一种无需训练、无需优化的零样本360度全景图像和视频生成框架。该方法通过将球面先验直接注入预训练的扩散Transformer中，用球面旋转位置编码替换标准RoPE，并结合语义扭曲分类器引导，解决了等距柱状投影的拓扑约束问题。

Details

Motivation: 现有方法要么依赖稀缺全景数据的昂贵微调限制了泛化能力，要么使用多步优化导致推理延迟过高。作者观察到现有生成模型虽具备一定全景先验，但无法满足等距柱状投影的严格拓扑约束。

Result: 在Flux.1、Flux.2和LTX-Video等骨干网络上进行文本到全景生成实验，取得了与基线方法相当的性能，同时保持了无需训练的优势。

Insight: 创新点在于将RoPE的低频通道重新参数化为3D笛卡尔坐标以编码球面流形，高频通道进行谐波量化以确保周期性；结合语义扭曲CFG显式引导几何结构，实现了无需重新训练即可继承SOTA模型的全部生成能力。

Abstract: We present a zero-shot, training-free and optimization-free framework for generating 360 panoramic images and videos by directly injecting spherical priors into pre-trained diffusion transformers. Existing methods either rely on costly fine-tuning on scarce panoramic data that limits generalization, or leverage multi-step optimization that incurs prohibitive inference latency. We observe that contemporary generative models natively exhibit some panoramic priors from large-scale training. However, these emergent capabilities are insufficient, as the models fundamentally fail to satisfy the rigorous topological constraints imposed by equirectangular projection (ERP). We introduce a zero-shot and optimization-free approach that resolves these constraints at inference time. Spherical RoPE replaces standard rotary position embeddings: low-frequency channels are re-parameterized as 3D Cartesian coordinates to natively encode the spherical manifold, while high-frequency channels are harmonically quantized to enforce exact periodicity. Coupled with complementary Semantic Distortion classifier-free guidance (CFG) that explicitly steers geometry, we avoid retraining and inherit the full creative breadth of state-of-the-art models. Our approach generalizes across diverse backbones and 360 generation modalities. We demonstrate this across text-to-panorama using Flux.1, Flux.2, and LTX-Video backbones, achieving competitive performance against baselines, all while remaining training-free. Project page: https://orhir.github.io/SpheRoPE

eess.IV [Back]

[105] Accelerating Merge with Motion Vector Difference via Filter Difference Analysis for VVenC eess.IV | cs.CVPDF

Xinmin Feng, Shengyang Xu, Jianhua Chen, Li Li, Dong Liu

TL;DR: 本文提出了一种基于分数运动矢量滤波器差异分析的新型快速MMVD算法，用于VVenC编码器。该方法通过用2抽头滤波器近似8抽头插值滤波器，推导出基于空间梯度和预测残差的准则来估计MMVD候选的潜在增益，并引入了对称偏移推断和十字形下采样点积计算等优化以减少开销。

Details

Motivation: MMVD是VVC中提高运动预测精度的关键编码工具，但其穷举搜索策略给编码器带来了巨大的计算负担。本文旨在解决MMVD搜索计算复杂度高的问题，以加速VVenC编码器的编码过程。

Result: 在VVenC的快速预设下，与现有快速MMVD算法相比，该方法将平均MMVD搜索比率从21.07%降低到11.05%，并将效率-复杂度度量η从11.79降低到7.10。

Insight: 创新点在于通过滤波器近似和梯度/残差分析来快速评估MMVD候选，避免了耗时的全搜索，并引入了针对硬件友好的实现优化（如对称推断和下采样计算），在保持编码效率的同时显著降低了计算复杂度。

Abstract: Merge with Motion Vector Difference (MMVD) is a key coding tool in Versatile Video Coding for improving motion prediction accuracy. However, its exhaustive search strategy imposes a significant computational burden on the encoder. To address this issue, we propose a novel fast MMVD algorithm for the VVenC encoder based on fractional motion vector filter difference analysis. By approximating the 8-tap interpolation filter with a 2-tap filter, we derive a criterion based on spatial gradients and prediction residuals for estimating the potential gain of MMVD candidates. We further generalize this criterion to accommodate both shifted integer reference samples and 2D separable filtering. To minimize the overhead of the proposed method, we introduce implementation optimizations, including symmetric offset inference and cross-shaped downsampled dot-product computation. Compared with existing fast MMVD algorithms in VVenC, our method reduces the average MMVD search ratio from 21.07% to 11.05% and decreases the efficiency-complexity metric $η$ from 11.79 to 7.10 under the fast preset.

cs.MA [Back]

[106] MECoBench: A Systematic Study of Multimodal Agent Collaboration in Embodied Environments cs.MA | cs.AI | cs.CL | cs.CVPDF

Qingyun Liu, Jiwen Zhang, Jingyi Hu, Siyuan Wang, Zhongyu Wei

TL;DR: 本文介绍了MECoBench，这是一个用于评估多模态大语言模型在具身环境中协作能力的系统性基准测试平台。该平台涵盖了多样化的现实世界任务、两种合作结构和三种协作模式。通过广泛的实验，研究发现协作通常能提升具身任务完成度，但其效果取决于协作收益与协调复杂性之间的平衡，且沟通至关重要。

Details

Motivation: 当前多模态大语言模型作为具身智能体潜力巨大，但其在视觉接地环境中的协作能力尚未得到充分探索。为了填补这一研究空白，作者构建了MECoBench基准来系统性地评估和探索多模态具身协作的机制与极限。

Result: 在MECoBench基准上对多种MLLM进行了广泛实验。结果表明，协作普遍提升了具身任务完成率，最佳协作模式取决于团队规模和模型能力，并且协作在存在噪声先验和探索条件下能提高鲁棒性。

Insight: 论文的主要创新点是提出了首个系统性的多模态具身协作基准MECoBench，它整合了任务、结构和模式的多样性。客观来看，其核心洞察在于揭示了协作收益与协调成本之间的权衡关系，并明确了沟通在协作中的关键作用，为未来具身智能体的协作策略设计提供了实证依据。

Abstract: Recent multimodal large language models (MLLMs) have strong potential as embodied agents, but their ability to collaborate in visually grounded environments remains underexplored. To address this gap, we introduce MECoBench, a multimodal embodied cooperation benchmark with an evaluation platform spanning diverse real-world tasks, two cooperation structures, and three collaboration modes. Through extensive experiments across various MLLMs, we summarize three key findings: (i) Collaboration generally improves embodied task completion, but its benefits depend on balancing collaborative gains against coordination complexity. (ii) Communication is essential to collaboration gains, while the best collaboration mode depends on team size and model capability. (iii) Moreover, collaboration improves robustness under noisy priors and exploration conditions. Generally, MECoBench provides a systematic testbed for understanding the mechanisms and limits of multimodal embodied collaboration. Code and dataset are available at https://github.com/q-i-n-g/MECoBench.

cs.IR [Back]

Runhao Li, Xiaoxu Ma, Zhenyu Weng, Yue Zhang, Guibo Luo

TL;DR: 本文提出了一种名为全局-邻域对齐哈希（GNAH）的无监督跨模态哈希方法，旨在解决现有方法依赖大规模图像-文本对的问题。该方法通过原型锚定全局对齐模块将视觉-语言基础模型的连续语义结构迁移到紧凑的二进制汉明空间，并利用对比随机邻域对齐模块缓解稀疏成对相关性导致的过拟合，从而在数据受限条件下实现高效的跨模态检索。

Details

Motivation: 现有无监督跨模态哈希方法通常依赖大规模且收集成本高昂的图像-文本对，限制了实际应用；本文旨在开发一种仅需有限数据即可保持语义结构的高效跨模态检索方法。

Result: 大量实验表明，GNAH在数据受限设置下持续优于现有的无监督跨模态检索方法，为实际应用提供了实用解决方案。

Insight: 创新点在于结合了原型锚定的全局结构对齐与建模随机邻域关系的对比学习，有效将基础模型的丰富语义信息迁移至二进制空间，同时缓解了数据稀疏时的过拟合问题。

Abstract: Compared to supervised cross-modal hashing (CMH), unsupervised CMH reduces the reliance on manual labeling by learning binary codes from unlabeled image-text pairs. However, existing unsupervised CMH methods often rely on large-scale image-text pairs, which are costly to collect. To address this limitation, we propose Global-Neighborhood Alignment Hashing (GNAH), a novel approach that preserves the semantic structure of vision-language foundation models within a compact binary Hamming space using only a limited number of image-text pairs. Specifically, GNAH captures global structural information from the continuous latent space and transfers it into the binary Hamming space through a Prototype-Anchored Global Alignment module. In addition, GNAH extends conventional pairwise contrastive learning by modeling stochastic neighborhood relationships via a Contrastive Stochastic Neighborhood Alignment module, thereby alleviating overfitting to sparse pairwise correlations. Extensive experiments demonstrate that GNAH consistently outperforms existing unsupervised cross-modal retrieval methods under data-constrained settings, offering a practical solution for real-world CMH applications.

cs.AI [Back]

[108] When Does Learning to Stop Help? A Cost-Aware Study of Early Exits in Reasoning Models cs.AI | cs.CL | cs.LGPDF

Zhe Dong, Fang Qin, Manish Shah

TL;DR: 本文研究了在推理模型中学习停止规则（LearnStop）的有效性，通过多特征预测前缀正确性来提前终止推理。研究发现，学习停止规则的价值取决于任务类型：在自由形式数学任务（如GSM8K）上能提升固定预算下的性能，但在多项选择或极难任务上，简单的标量规则（如置信度、熵）已足够竞争。

Details

Motivation: 动机在于探究学习停止规则何时能优于简单的置信度或收敛阈值，以优化推理模型的计算效率，避免不必要的计算开销。

Result: 在GSM8K等自由形式数学任务上，LearnStop在固定预算前沿上实现后验峰值适应增益+0.157，验证选择的操作点保持正增益，相比最强标量基线配对增益+0.028；在多项选择和极难任务上，标量规则表现竞争或更强。

Insight: 创新点在于提出无需隐藏状态的检查点停止器LearnStop，并系统分析学习停止规则的价值取决于轨迹结构，而非普遍替代标量规则；提供了成本核算、验证操作点等实用工具。

Abstract: Reasoning models spend different amounts of useful computation across instances, but it remains unclear when a learned stopping rule improves over simple confidence or convergence thresholds. We study this question with LearnStop, a hidden-state-free checkpoint stopper for reasoning language models. At fixed budget checkpoints, LearnStop probes a short answer from the current reasoning prefix and predicts prefix correctness from online features such as answer confidence, entropy, prefix vote share, answer stability, and backtracking-marker density. Across 18 task-model settings spanning GSM8K, MATH-500, MMLU-Pro, AIME-90, GPQA, Qwen3, and DeepSeek-R1 distillations, the answer is task-dependent. On free-form math, learned multi-feature stopping improves the fixed-budget frontier and often beats scalar exits: on GSM8K with Qwen3-32B, the empirical frontier reaches a post-hoc peak adapt gain of +0.157, validation-selected operating points preserve positive gains, and the paired gain over the strongest scalar baseline is +0.028. On multiple-choice and very hard settings, scalar confidence, entropy, or stability rules are competitive or stronger. We therefore frame learned stopping not as a universal replacement for scalar exits, but as a tool whose value depends on trajectory structure. We further provide validation-selected operating points, paired bootstrap tests, finite-grid lost-correct risk calibration, cost accounting under KV-fork, prefix-cache, and black-box regimes, H100 serving profiles, checkpoint-schedule sweeps, transfer analyses, and robustness checks. The main practical finding is that learned stopping is useful when many questions become correct before full budget but do not exhibit a single reliable scalar stopping signal; its benefits largely disappear when confidence or answer convergence already solves the stopping problem.

[109] HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents cs.AI | cs.CL | cs.CVPDF

Qianchu Liu, Sheng Zhang, Guanghui Qin, Jeya Maria Jose Valanarasu, Maximilian Rokuss

TL;DR: 本文介绍了HealthAgentBench，一个包含54个医疗代理任务、覆盖7个类别和多种模态的统一基准测试套件，旨在模拟端到端临床工作流，对前沿AI代理进行严格评估。

Details

Motivation: 随着AI代理在复杂、长程推理方面能力增强，需要严谨、全面的评估来衡量其在真实世界医疗应用中的进展，现有基准在模拟真实医疗环境和工作流方面存在不足。

Result: 在HealthAgentBench上评估前沿代理发现，总体任务成功率较低，最强的代理Codex GPT-5.5成功率仅约42%；代理在处理EHR数据的研究建模管道方面表现出潜力，但医学影像任务尤其具有挑战性，且所有当前代理在结合大搜索空间与组合推理需求的任务上均表现不佳。

Insight: 创新点在于构建了一个统一、现实且具有挑战性的医疗代理基准套件，其任务设计强调从原始数据探索到多步骤执行的端到端工作流，超越了简单的提示工程，为评估和推动医疗AI代理发展提供了重要工具。

Abstract: As AI agents become increasingly capable of complex, long-horizon reasoning, rigorous and holistic evaluation is essential for measuring progress toward real-world healthcare applications. We introduce HealthAgentBench, a suite of 54 agentic healthcare tasks across 7 categories each with its unique environment. The benchmark suite spans diverse workflows throughout the patient journey and a broad range of modalities. Each task is designed to replicate an end-to-end clinical workflow: given minimal instructions, an agent must explore raw healthcare data, operate within a complex environment, and execute multi-step solutions that go beyond naive prompting. A final task success rate is reported to provide a single, interpretable metric for HealthAgentBench overall performance for each agent. Evaluating frontier agents on HealthAgentBench, we find that overall task success rate remains low, underscoring the difficulty of the suite. The strongest and the most cost effective agent, Codex GPT-5.5, achieves only approximately 42% success rate. Beyond aggregate performance, HealthAgentBench reveals nuanced strengths and weaknesses across task categories. Frontier agents show promise in automatically developing research modeling pipelines over EHR data, but medical imaging remains especially challenging, particularly for Claude Code models, while Codex GPT-5.5 shows emerging capability. Tasks that combine large search spaces with compositional reasoning requirements remain difficult for all current agents. Together, these results suggest that HealthAgentBench provides a challenging and realistic benchmark with substantial room for future progress. We release our benchmark at https://github.com/microsoft/HealthAgentBench.

[110] Modality-Driven Search with Holistic Trace Judging for ARC-AGI-2 cs.AI | cs.CL | cs.LGPDF

Johan Land

TL;DR: 本文提出了一种用于ARC-AGI-2视觉推理基准测试的求解器。其核心是两大原则：一是将推理模态（文本、图像、代码）视为独立的搜索算子以生成多样化的候选答案；二是采用保留上下文的整体判断，即在一个长上下文提示中联合比较所有候选推理路径。该方法在ARC Prize半私有评估集上取得了72.9%的准确率，超越了GPT-5.2 Pro和Gemini 3 Pro等前沿模型。

Details

Motivation: 解决大型语言模型在抽象推理任务中可能产生流畅但错误答案的问题，其核心挑战在于如何从多个候选答案中进行有效选择，而不仅仅是生成答案。

Result: 在ARC Prize半私有评估集上达到72.9%的准确率（每次任务成本38.99美元），超越了GPT-5.2 Pro（54.2%）和Gemini 3 Pro（54.0%），是当前已核实排行榜的最高分。在公开评估集上达到76.1%的准确率（每次任务成本19.69美元）。

Insight: 创新点在于将多模态（文本、图像、代码）作为独立的搜索算子来生成多样化候选，并采用保留上下文的整体判断来比较所有候选，这比自洽性或多数投票法更能可靠地发现正确的少数派假设。研究还发现，规定性的提示模板和迭代精炼会系统性降低假设多样性并损害性能。

Abstract: Large language models can produce fluent, internally coherent reasoning traces for abstract reasoning tasks while still being confidently wrong - making selection among candidates, not just generation, the central challenge. I present a solver for ARC-AGI-2, a few-shot visual reasoning benchmark, built around two principles: (i) treating reasoning modalities as search operators, generating diverse candidates independently across text, image, and code channels, and (ii) context-preserving holistic judging, in which a judge model jointly compares all candidate reasoning traces within a single long-context prompt. Unlike self-consistency or majority voting, this approach reliably recovers correct minority hypotheses on tasks where the modal answer is wrong. On the ARC Prize semi-private evaluation set, the solver achieves 72.9 percent at USD 38.99 per task - the highest score on the verified leaderboard at the time of writing, exceeding the best standalone frontier models, GPT-5.2 Pro at 54.2 percent and Gemini 3 Pro at 54.0 percent, by +18.7 percentage points. On the public evaluation set, it achieves 76.1 percent at USD 19.69 per task. I release the full source code and document extensive negative results, including the finding that prescriptive prompting templates and iterative refinement systematically reduce hypothesis diversity and degrade performance.

[111] Harnessing Textual Refusal Directions for Multimodal Safety cs.AI | cs.CV | cs.LGPDF

Moreno D’Incà, Massimiliano Mancini, Nicu Sebe

TL;DR: 本文提出了一种无需多模态安全数据的轻量级免训练方法MARS，通过利用从LLM主干提取的文本拒绝方向来提升多模态大语言模型的安全性，该方法通过激活重定心、自适应调整引导强度并选择最优干预层，在多个基准测试中实现了安全性的提升且不影响模型实用性。

Details

Motivation: 针对多模态大语言模型缺乏安全多模态数据的问题，研究文本拒绝方向是否能够跨模态（如图像、视频）泛化，以规避收集不安全多模态数据的困难。

Result: 在五个SOTA MLLMs上进行评估，涵盖安全、实用性和视频越狱基准测试，MARS实现了持续的安全性提升，同时保持了实用性。

Insight: 揭示了安全相关结构在模态间共享，文本拒绝方向是多模态对齐中一个强大且未被充分探索的基础；通过激活重定心纠正模态错位，在几何定义的信任区域内自适应缩放引导强度，并在首个生成token处选择最优干预层，实现了免训练的安全注入。

Abstract: To improve safety in Large Language Models (LLMs) we can either perform post-training alignment or exploit refusal directions in the activation space. Both strategies are less feasible in Multimodal LLMs (MLLMs) as they require unsafe multimodal data, harder to collect than their unimodal counterpart. In this work, we relax this constraint and investigate whether textual refusal directions, extracted directly from the LLM backbone, generalize across modalities (i.e., image, video). Preliminary findings confirm this ability, though effectiveness is conditioned by layer selection, steering strength, and cross-modal alignment, with the latter causing safe multimodal inputs to be spuriously steered toward refusal. Building on this, we introduce Modality-Agnostic Refusal Steering (MARS), a light-weight training-free approach that injects multimodal safety without the need for multimodal safety data. MARS corrects modality misalignment via activation re-centering, adaptively scales steering strength within a geometrically defined trust region, and selects the optimal intervention layer, operating at the first generated token. Evaluated on five SOTA MLLMs across safety, utility, and video jailbreak benchmarks, MARS achieves consistent safety gains while preserving utility. These results reveal that safety-relevant structure is shared across modalities and that textual refusal directions are a powerful and underexplored foundation for multimodal alignment.

cs.RO [Back]

Kaier Liang, Hengde Dai, Cristian-Ioan Vasile

TL;DR: 本文提出ViTL框架，通过将自然语言指令编译为线性时序逻辑（LTL）公式并转换为确定性有限自动机（DFA），结合视觉语言模型（VLM）生成的方向性得分，实现了在未知环境中执行具有时序约束的零样本长视野自然语言导航任务。

Details

Motivation: 解决现有零样本物体导航方法只能处理单目标任务，无法应对现实世界中包含时序和逻辑约束（如“先清洁椅子或沙发，然后打开电视”）的多目标自然语言指令的挑战。

Result: 在Habitat-Matterport 3D (HM3D)数据集上的实验表明，该完整框架能成功完成具有时序约束的零样本长视野自然语言导航任务，并且其提出的方向性得分相比基线方法提高了单目标导航的准确性和效率。

Insight: 创新点在于将自然语言指令形式化为LTL/DFA以协调多通道价值地图和动态重规划，以及在导航层面引入方向性得分（而非全视野无方向价值）来提升VLM的引导精度；这为处理复杂、结构化指令的零样本机器人导航提供了可借鉴的层级化解决方案。

Abstract: Enabling robots to follow natural language commands to complete zero-shot long-horizon tasks remains challenging. It requires extracting implicit temporal and logical constraints from natural language commands and executing multiple sub-tasks accordingly. Recent zero-shot object navigation methods use vision-language models (VLMs) to guide frontier-based exploration in unknown environments, but they are limited to single-target tasks. Real-world commands such as “Clean either the chair or the couch, then turn on the tv.” require navigating to multiple targets in a temporally constrained order, which no existing zero-shot system can handle. We present ViTL, a framework that addresses this gap at two levels. At the task level, we use a large language model (LLM) to compile natural language commands into Linear Temporal Logic (LTL) formulas, which are then converted into Deterministic Finite Automata~(DFA) that coordinate multi-channel value maps and trigger dynamic replanning when new objects are detected. At the navigation level, we introduce directional score: rather than producing a direction-agnostic value across the entire field of view, we label frontier directions on the observation image and extract per-direction scores from the VLM. Experiments on Habitat-Matterport 3D (HM3D) show that the full framework enables zero-shot long-horizon completion of natural language navigation tasks with temporal constraints, and that directional score improves single-target navigation accuracy and efficiency over the baseline.

[113] RCT: A Robot-Collected Touch-Vision-Language Dataset for Tactile Generalization cs.RO | cs.AI | cs.CL | cs.CVPDF

Jingbo He, Michael Färber, Roberto Calandra

TL;DR: 本文介绍了RCT数据集，这是一个由机器人收集的触觉-视觉-语言数据集，包含来自7个类别122种工业参考材料的29,279个触觉帧，用于研究机器人触觉表征在未见材料上的泛化能力。

Details

Motivation: 为了解决机器人在操作开放世界物体时，触觉表征必须能够泛化到未见材料的问题，作者构建了一个新的数据集以支持可重复的、基于接触序列和材料留出的评估。

Result: 实验表明，在编码器固定的情况下，移除接触序列重叠会使触觉到文本的Recall@1下降17.7个百分点；当在训练时额外留出材料时，性能急剧下降，留出材料的Recall@1平均为25.1 +/- 6.1%。在公开的TVL/HCT数据集上，原始像素最近邻在98.3%的情况下能恢复正确的接触序列。

Insight: 论文的创新点在于构建了一个强调接触序列和材料留出评估的机器人触觉数据集，揭示了接触序列重叠对评估的潜在影响，并将未见材料泛化确立为机器人触觉感知的核心挑战。均匀采样按压过程可以改进对比训练，且RCT训练的嵌入能改进对未见材料的类别探测。

Abstract: For robots manipulating open-world objects, tactile representations must generalize to unseen materials. We introduce RCT (Robotic Contact Tactile), a robot-collected touch-vision-language dataset with 29,279 tactile frames from full robot presses on 122 industrial reference materials in 7 categories, recorded with three DIGIT sensors at multiple contact positions. RCT preserves each press as a contact sequence, enabling held-out evaluation across materials, categories, sensors, contact positions, and contact sequences. Frames from one press are strongly correlated: frame-random splits can place near-duplicate observations of the same physical interaction in both training and test. With the encoder held fixed, removing contact-sequence overlap reduces tactile-to-text Recall@1 by 17.7 percentage points. When materials are additionally held out at training time, performance drops sharply, leaving held-out-material Recall@1 at 25.1 +/- 6.1% averaged over three held-out draws. The public TVL/HCT split shows the same structure: every test contact sequence appears in training, and raw-pixel nearest neighbors recover the correct sequence in 98.3% of cases. Uniformly sampling a press improves contrastive training, and RCT-trained embeddings improve category probes on unseen materials. RCT makes contact-sequence-aware, held-out-material evaluation reproducible and exposes novel-material generalization as a central challenge for robotic tactile perception. The RCT dataset is open-sourced at https://faerber-lab.github.io/RCT/

Wen Jiang, Hanfang Liang, Li Wang, Kangyao Huang, Wang Xu

TL;DR: 本文提出DynFly，一个动态感知的连续轨迹生成框架，旨在解决无人机视觉语言导航中从高层导航意图到可执行无人机连续运动之间的建模鸿沟。该方法通过B样条控制点空间表示专家轨迹，并利用Spline-DiT生成器通过流匹配学习条件轨迹生成，同时引入面向无人机的动态感知监督。

Details

Motivation: 现有无人机视觉语言导航方法主要预测离散动作、局部目标或稀疏航点，未能充分建模从导航意图到可执行无人机连续运动的过渡，这限制了生成轨迹的连续性、稳定性和可执行性。

Result: 在OpenUAV UAV-VLN基准测试的Test Unseen Full划分上，DynFly显著提升了导航性能和轨迹质量，相比最强基线在NDTW、SDTW、SR和OSR指标上分别提升了4.69、2.40、2.14和4.87个百分点，同时将NE降低了4.51米。

Insight: 核心创新在于提出了一个轻量级的轨迹生成层，将高层导航推理与连续无人机运动桥接起来。其通过将轨迹表示在B样条控制点空间并使用流匹配进行条件生成，并结合了针对无人机动力学特性（位置、速度、加速度、航向一致性、局部目标对齐）的监督信号，从而生成更符合实际飞行约束的平滑轨迹。该框架可与现有UAV-VLN系统集成，保留其原有的视觉-语言推理流程。

Abstract: Recent advances in multimodal large models have significantly improved UAV vision-language navigation (UAV-VLN) by enhancing high-level perception and reasoning. However, existing methods mainly focus on predicting discrete actions, local targets, or sparse waypoints, while the continuous transition from navigation intent to executable UAV motion remains weakly modeled. This motion-interface gap limits the continuity, stability, and executability of generated UAV trajectories. To address this gap, we propose DynFly, a dynamic-aware continuous trajectory generation framework that bridges high-level navigation reasoning and executable UAV motion. DynFly bridges high-level navigation intent and continuous UAV motion through a lightweight trajectory generation layer. Specifically, it represents expert trajectories in B-spline control-point space and employs a Spline-DiT generator to learn conditional trajectory generation via flow matching. Furthermore, we introduce UAV-oriented dynamic-aware supervision over position, finite-difference velocity, finite-difference acceleration, heading consistency, and local target alignment, enabling the generated trajectories to better satisfy UAV motion characteristics. And our trajectory generation framework can also be integrated with an existing UAV-VLN framework while preserving its original visual-language reasoning pipeline. Extensive experiments on the OpenUAV UAV-VLN benchmark show that DynFly improves both navigation performance and trajectory quality. On the Test Unseen Full split, DynFly improves the strongest baseline by 4.69 NDTW, 2.40 SDTW, 2.14 SR points and 4.87 OSR points, while reducing NE by 4.51 m.

[115] MVP-Nav: Multi-layer Value Map Planner Navigator cs.RO | cs.AI | cs.CVPDF

Wenyuan Xie, Shaokai Wu, Yijin Zhou, Yanbiao Ji, Guodong Zhang

TL;DR: 本文提出了MVP-Nav，一个仅使用RGB图像的物理感知零样本目标导航框架。该框架利用3D基础模型从单目观测中重建显式物理占据，并通过多层价值图（MVM）将高层语义推理与底层物理约束统一到一个共享代价空间中，实现基于几何的规划。

Details

Motivation: 解决仅使用RGB感知的零样本目标导航（ZSON）中，因缺乏显式深度信息而导致的严重物理不确定性和语义-物理错位问题。现有方法要么缺乏几何基础，要么缺乏显式物理约束，导致语义合理但物理不安全的行为。

Result: 在零样本目标导航基准测试上的大量实验表明，MVP-Nav显著优于现有的无深度方法，达到了最先进的性能水平。

Insight: 核心创新在于通过3D基础模型从2D语义实例重建3D定向边界框，形成全局空间语义表示，并设计了多层价值图（MVM）来统一语义优先级与重建几何，将物理先验结构化为可规划的代价空间，从而有效补偿了主动深度传感器的缺失。

Abstract: Zero-shot Object Goal Navigation (ZSON) with RGB-only perception poses a fundamental challenge for embodied agents, as the absence of explicit depth information introduces severe physical uncertainty and semantic-physical misalignment. Existing approaches either rely on high-level semantic reasoning without geometric grounding or learn end-to-end policies that lack explicit physical constraints, often resulting in semantically plausible but physically unsafe behaviors. In this paper, we propose MVP-Nav, a physical-aware RGB-only navigation framework that aligns perception, planning, and control with the real 3D world. MVP-Nav reconstructs explicit physical occupancy from monocular observations by leveraging 3D foundation models to project 2D semantic instances into 3D oriented bounding boxes, forming a global spatial semantic representation. To unify high-level semantic reasoning and low-level physical constraints, we introduce a Multi-layer Value Map (MVM) that integrates semantic priorities and reconstructed geometry into a shared cost space, enabling physically grounded geometric planning. Extensive experiments on zero-shot object navigation benchmarks demonstrate that MVP-Nav significantly outperforms existing depth-free methods, achieving state-of-the-art performance and validating that structured physical priors can effectively compensate for the absence of active depth sensors.

cs.LG [Back]

[116] From Search to Synthesis: Training LLMs as Zero-Shot Workflow Generators cs.LG | cs.AI | cs.CLPDF

Gan Luo, Zihan Qin, Bin Dong, Wotao Yin

TL;DR: 本文提出了MetaFlow，一种将工作流生成视为元学习问题的方法，通过两阶段训练（监督微调和基于可验证奖励的强化学习）使大语言模型能够学习组合解决方案策略，生成具有结构一致性的工作流。该方法在问答、代码生成和数学推理等基准测试中，在领域内任务上达到与最先进基线相当的性能，并在领域外任务和操作符集上展现出强大的零样本泛化能力。

Details

Motivation: 大语言模型虽然擅长多种任务，但其针对具体实例的解决方案缺乏结构一致性，难以可靠部署。手动设计可复用、鲁棒且可解释的工作流需要大量专业知识和努力，限制了其广泛应用。现有自动工作流生成方法要么无法学习任务级模式，要么泛化能力有限。

Result: 在问答、代码生成和数学推理等基准测试中，MetaFlow在领域内任务上通过单次推理达到了与最先进基线（SOTA）相当的性能。同时，在领域外任务和未见过的操作符集上，它展现出了卓越的零样本泛化能力。

Insight: 核心创新点在于将工作流生成形式化为元学习问题，并采用两阶段训练策略（SFT + RLVR），利用跨问题实例的执行反馈来提升端到端成功率。这使模型能够学习任务层面的算法模式，从而生成可复用、鲁棒的工作流，并实现强大的零样本泛化。

Abstract: Large language models (LLMs) excel across a wide range of tasks, yet their instance-specific solutions often lack the structural consistency needed for reliable deployment. Workflows that encode recurring algorithmic patterns at the task level provide a principled framework, offering robustness across instance variations, interpretable traces for debugging, and reusability across problem instances. However, manually designing such workflows requires significant expertise and effort, limiting their broader application. While automatic workflow generation could address this bottleneck, existing methods either produce instance-specific solutions without learning task-level patterns, or cannot generalize beyond their training configurations. We present MetaFlow, which casts workflow generation as a meta-learning problem: given a task and an operator set, the model learns to compose solution strategies. MetaFlow trains in two stages: supervised fine-tuning on synthetic workflow data, followed by reinforcement learning with verifiable rewards (RLVR) that uses execution feedback across problem instances in the task to improve end-to-end success. The resulting model produces effective workflows for trained tasks and exhibits strong generalization to untrained tasks and novel operator sets. Across benchmarks in question answering, code generation, and mathematical reasoning, MetaFlow achieves performance comparable to state-of-the-art baselines on in-domain tasks with single inference, while demonstrating remarkable zero-shot generalization capabilities on out-of-domain tasks and operator sets.

[117] Fork-Think with Confidence cs.LG | cs.CLPDF

Zena Al-Khalili, Rafi Hakim, Dietrich Klakow, Ji-Ung Lee

TL;DR: 本文提出了一种名为Fork-think with confidence的高效LLM推理方法，它采用先决策后思考的范式，通过模型置信度在单一路径中识别分叉点，然后采样多个延续路径并聚合结果。该方法在三个模型和三个推理基准测试中显著减少了token消耗和运行时间，同时性能与并行思维方法相当或更优。

Details

Motivation: 现有并行思维方法遵循先思考后决策的范式，首先生成多个推理路径，这会导致过度生成，然后需要剪枝或停止不必要的路径来弥补。相比之下，先决策后思考的范式（即先识别可能产生理想生成的点）尚未得到充分探索。

Result: 在三个模型和三个推理基准测试中，Fork-think将token消耗减少了高达30%，运行时间减少了高达57%，同时性能与并行思维方法相当或更好。结合早期停止和加权投票等现有机制后，性能可进一步提升，与现有最先进方法相当，且无需预热或离线训练。

Insight: 创新点在于提出了先决策后思考的范式，通过模型置信度动态识别分叉点，从而减少不必要的计算开销。客观分析表明，该方法能识别与下游任务相关的有意义分叉点，且在后期位置采样可以显著改善生成质量，为高效LLM推理提供了新的研究方向。

Abstract: Parallel thinking has enjoyed great success for boosting LLM performance on reasoning tasks without the need for any re-training. However, existing methods follow a think-first-then-decide paradigm, i.e., they first sample multiple reasoning paths, which inevitably leads to overgeneration, then prune or stop unnecessary paths to compensate. In contrast, decide-first-then-think, i.e., first identifying points that are likely to lead to desirable generations, has been underexplored so far. Following this paradigm, we propose Fork-think with confidence, that first identifies forking points using model confidence in a single seeding path, then triggers thinking, sampling multiple continuations and aggregating them for the final response. Our experiments across three models and three reasoning benchmarks show that Fork-think reduces the token consumption by up to 30% and run-time by up to 57%, while performing comparable to or better than parallel thinking. Our analysis reveals that Fork-think is able to identify forking points that are meaningful with respect to the downstream task and that sampling at later positions can lead to substantially better generations. Finally, we demonstrate how combining Fork-think with existing mechanisms such as early stopping and weighted voting can further boost the performance and perform comparably to existing state-of-the-art methods, without requiring any warm-up or offline training. Our results establish pre-determined forking as a promising research direction for efficient LLM reasoning.

[118] Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers cs.LG | cs.CLPDF

Ying Fan, Anej Svete, Kangwook Lee

TL;DR: 本文提出LOTUS（Looped Transformers with parallel supervision on latents），一种基于循环（或循环深度）Transformer的潜在思维链（latent CoT）方法。该方法通过权重复用的循环结构，在模型的隐藏状态中进行多步推理，旨在弥合潜在推理与显式推理（explicit CoT）之间的性能差距，并在保持高效的同时提升推理能力。

Details

Motivation: 现有潜在思维链方法在参数规模超过10亿后性能显著落后于显式思维链，且差距随规模扩大而加剧。循环Transformer通过复用权重增加计算深度而不增加参数量，天然适合潜在推理，因此研究其是否能弥合这一性能差距。

Result: 在30亿参数规模下，LOTUS首次使潜在CoT性能与显式CoT相当，同时在从紧凑数学表达式到自然语言的推理任务中，将思维阶段延迟降低了2.5倍至6.9倍。消融实验证实循环主干和对黄金CoT标记的并行监督均至关重要。

Insight: 核心创新在于结合了循环Transformer架构与对潜在位置黄金推理步骤的并行交叉熵监督，这使模型在潜在空间中进行高效、可解释且与思维链对齐的多步推理。客观来看，该方法通过结构设计有效利用了计算深度，为高效大模型推理提供了新思路。

Abstract: Language models typically reason via explicit chain-of-thought (CoT), generating intermediate steps token-by-token. Latent CoT offers an alternative: it performs multi-step reasoning in the model’s hidden states, replacing decoded tokens with continuous representations for greater efficiency. However, existing latent CoT methods underperform explicit CoT beyond 1B parameters, and the gap widens with scale. Looped, or recurrent-depth, Transformers, which reuse their weights to increase computation depth without adding parameters, are a natural fit for latent reasoning. We therefore ask whether looped Transformers can bridge this gap. We answer affirmatively with a simple recipe: a looped padded Transformer that processes K latent blocks in parallel for R iterations, with a cross-entropy loss on each latent position’s gold CoT-step token, similar to explicit CoT supervision. We instantiate it as LOTUS (Looped Transformers with parallel supervision on latents). LOTUS is, to our knowledge, the first latent-CoT method to bridge the gap to explicit CoT at the 3B scale, while cutting thought-phase latency by 2.5x-6.9x from compact math expressions to natural language. Projecting LOTUS’s post-loop latents through the base LM head recovers the gold reasoning steps and even surfaces alternative valid intermediate steps, evidence that its latent space is interpretable and CoT-aligned. Ablations confirm that both the looped backbone and the parallel supervision on gold CoT tokens are essential.

[119] QVal: Cheaply Evaluating Dense Supervision Signals for Long-Horizon LLM Agents cs.LG | cs.AI | cs.CLPDF

Sergio Hernández-Gutiérrez, Matteo Merler, Ilze Amanda Auzina, Joschka Strüber, Ameya Prabhu

TL;DR: 本文提出QVal，一种无需训练即可直接评估长视野LLM智能体密集监督信号质量的测试框架。通过衡量状态-动作对评分与强参考策略Q值的对齐程度，QVal在四个多样化环境和七个方法家族上对21种密集监督方法进行了大规模基准测试。研究发现简单提示基线优于近期文献方法，且性能按方法家族呈现强聚类趋势。

Details

Motivation: 解决长视野LLM智能体任务中，仅依赖稀疏结果奖励难以评估中间动作质量的问题；现有密集监督方法评估依赖下游训练流程，导致评估成本高、混淆监督质量与工程因素、不同方法家族难以公平比较。

Result: 在QVal-v1.0测试平台上，基于六个开源模型主干进行了超过1200次评估实验，发现简单提示方法持续优于近期文献中的密集监督方法；性能表现按方法家族呈现明显聚类，该结论在不同模型规模、环境和观察模态中均成立。

Insight: 创新点在于提出训练无关的密集监督信号直接评估框架，通过Q对齐度量实现方法间的公平比较；客观分析表明该框架将监督信号质量与训练工程解耦，为密集监督方法的快速迭代提供了可扩展的基准测试平台。

Abstract: LLM agents increasingly act over long horizons, where a single trajectory can contain hundreds or thousands of actions. In these settings, outcome-only rewards provide too sparse guidance, failing to inform the model about the goodness of intermediate actions. Dense supervision methods aim to solve this problem by scoring intermediate steps, from intrinsic confidence to self-distillation and embedding similarities. However, it is common practice to evaluate them by measuring the downstream performance of a training pipeline that integrates them. This is expensive, conflates supervision quality with training engineering confounders, and renders different methodological families requiring distinct training setups incomparable. As a result, dense supervision methods are rarely benchmarked on common ground. We introduce QVal, a training-free testbed for directly evaluating dense supervision signals. Given a state-action pair, QVal measures how well a method’s score is Q-aligned: whether it orders actions according to the Q-values of a strong reference-policy. This lets us compare signals before any training run and separate signal quality from other engineering choices. We instantiate QVal as QVal-v1.0, benchmarking 21 dense supervision methods across four diverse environments and seven methodological families, with over 1.2K evaluation experiments across six open-weight model backbones. We find that simple prompting baselines consistently outperform recent dense supervision methods from the literature, and that performance clusters strongly by family. These findings hold across model sizes, environments, and observation modalities. QVal is designed to be easily extensible to new environments and methods, enabling researchers to iterate on dense supervision methods before any training run.

[120] CoMet: Context and Multiplicity Decomposition for Multimodal Uncertainty Estimation cs.LG | cs.CVPDF

Sanghyuk Chun, William Yang, Amaya Dharmasiri, Olga Russakovsky

TL;DR: 本文提出CoMet方法，用于多模态大语言模型（MLLMs）中的不确定性估计。该方法将不确定性分解为上下文特定项和多重性特定项，分别捕捉由给定上下文（如任务或提示）引起的歧义性以及由上下文决定的、与输入兼容的合理答案数量。通过训练一个轻量级的事后不确定性模块来估计这些量，实现了无需自回归答案生成或重复采样的高效不确定性估计。

Details

Motivation: 不确定性估计是AI模型中长期存在的挑战，尤其在多模态大语言模型（MLLMs）中变得日益重要，因为不确定性可能源于多种来源及其相互关系，以及在开放端设置中无界答案的复杂性。

Result: 在多个开放端多模态基准测试、幻觉检测和多项选择视觉问答基准测试上的实验表明，CoMet在不确定性估计方面持续优于现有基线方法，同时在实际应用中保持高效性。

Insight: 创新点在于将MLLMs中的不确定性分解为上下文和多重性两个组成部分，并通过轻量级事后模块实现高效估计，避免了计算密集的自回归生成或采样过程，为模型的可信度评估提供了新思路。

Abstract: Uncertainty estimation has been a long-standing challenge in AI models; it amounts to “knowing what you don’t know,” and metacognition is notoriously difficult even for humans (cf. the Dunning-Kruger effect). Although it is still far from solved even in simpler classification systems, tackling it in multimodal large language models (MLLMs) is becoming increasingly important. Within MLLMs, uncertainty can stem from any of the diverse sources as well as from their relationships, and further can stem from the unbounded answers in the open-ended setting. To tackle the issues, we propose CoMet, an MLLM uncertainty estimation method by decomposing uncertainty into a context-specific term and a multiplicity-specific term. The former captures ambiguity induced by the given context (e.g., task or prompt), while the latter captures how many plausible answers determined by the context remain compatible with the given input. We train a lightweight post-hoc uncertainty module to estimate these quantities, which enables efficient uncertainty estimation without autoregressive answer generation or repeated sampling. Experiments on various open-ended multimodal benchmarks, hallucination detection, and multiple-choice visual question answering benchmarks show that CoMet consistently improves uncertainty estimation over existing baselines while remaining efficient in practice. Code is available at https://github.com/princetonvisualai/comet_uncertainty

cs.SD [Back]

[121] ASR-Agnostic Multimodal Spectrotemporal Modeling for Early Dementia Detection cs.SD | cs.AI | cs.CL | eess.ASPDF

Chukwuemeka Ugwu, Oluwafemi Richard Oyeleke

TL;DR: 本文提出了一种不依赖自动语音识别（ASR）的多模态谱时域建模框架，用于早期痴呆症检测。该方法直接从梅尔频谱图中提取谱时域位移场作为认知衰退的数字生物标志物，并通过跨注意力机制融合CNN-ConvGRU声学嵌入，最后使用Transformer编码器和可学习查询池进行聚合。

Details

Motivation: 解决现有基于语音的痴呆检测系统依赖转录、丢弃录音内时间结构，且仅在单一存在已知录音伪影的英语语料库上验证的问题，旨在开发一种语言无关、能有效利用多模态时序信息的鲁棒检测框架。

Result: 在斯洛伐克EWA-DB语料库上达到83.9%的准确率，西班牙语Ivanova语料库上也有提升，而英语DementiaBank基线仅为53.2%，证实了其已知伪影问题。消融研究表明多模态融合的价值取决于语料库：当信号分布在多模态时至关重要，当单一模态主导时可能适得其反，当无有效信号时则无关紧要。

Insight: 创新点在于提出ASR无关的谱时域位移场作为认知衰退的生物标志物，以及使用跨注意力机制进行多模态特征融合。客观分析认为，其揭示了多模态融合策略的有效性高度依赖于具体数据集的数据特性，这为跨语言、跨数据集的模型设计提供了重要洞见，同时辅助时序损失收敛到语言无关值也表明了架构的跨语言稳定性。

Abstract: Speech recruits the same executive, attentional, and working memory processes underlying instrumental activities of daily living, or IADLs, providing a non-invasive proxy for cognitive assessment. Yet most speech-based dementia detection systems depend on transcription, discard within-recording temporal structure, and are validated on a single English corpus with known recording artifacts. We propose an ASR-agnostic framework operating directly on Mel spectrograms. Our key contribution is extracting spectrotemporal displacement fields from consecutive spectrogram frames, capturing shifting spectral energy patterns as digital biomarkers of cognitive decline. These features are fused with CNN-ConvGRU acoustic embeddings via a learned cross-attention mechanism and aggregated using a Transformer encoder with learnable query pooling. A composite temporal loss enforces smoothness and contrastive coherence across segments. We train independent models on English DementiaBank, Slovak EWA-DB, and Spanish Ivanova corpora, using clinical elicitation protocols taxing IADL-relevant cognitive domains. The Slovak model achieves 83.9% accuracy, and Spanish achieves, while the English baseline yields 53.2%, confirming known artifacts. Cross-lingual ablation studies reveal distinct fusion regimes: removing cross-attention collapses Spanish performance to 53.7%, below unimodal models, while the Slovak audio encoder alone outperforms the full model, 93.7% vs. 83.9%, and all English configurations remain near chance. Thus, multimodal fusion’s value is corpus-dependent: essential when signal is distributed across modalities, counterproductive when one dominates, and irrelevant when no signal exists. Auxiliary temporal losses converge to language-invariant values, indicating cross-lingual architectural stability.

math.NA [Back]

[122] Online TT-ALS for Streaming Tensor Decomposition with Incremental Orthogonalization math.NA | cs.CV | cs.LGPDF

Hiroki Takeda, Yuto Miyatake, Daisuke Furihata

TL;DR: 本文提出了一种在线TT-ALS算法，用于流式张量分解，通过顺序施加正交性约束，实现了核心张量的高效精确更新。该方法在保证高重构精度的同时，将计算复杂度从二次降低到线性，适用于低延迟实时处理。

Details

Motivation: 针对流式数据场景，现有批量方法内存消耗大，而在线方法计算效率高但精度不足，本文旨在设计一种兼具高精度和高效率的在线张量分解算法。

Result: 实验表明，该方法在数学近似精度和基于人类感知的视频质量指标上均优于现有在线技术，并且相比近期基于深度学习的范式，实现了数个数量级的加速。

Insight: 创新点在于通过顺序正交化约束保证目标函数单调下降和时间平滑性，其确定性单次扫描更新将秩依赖从二次降至线性，是一种高效且精确的代数方法。

Abstract: Tensor Train (TT) decomposition is a powerful technique for analyzing high-dimensional data. Existing algorithms for computing TT decompositions can be categorized into two main types: conventional batch-based approaches and recursive online methods. In the context of streaming data, batch methods typically achieve higher reconstruction accuracy but often suffer from memory exhaustion, while online methods provide greater computational efficiency. In this work, we introduce Online TT-ALS (Alternating Least Squares), an algorithm that sequentially enforces orthogonality constraints. This approach allows for efficient and exact updates of the core tensor while maintaining high reconstruction accuracy. Theoretically, we prove that enforcing these orthogonal gauge constraints guarantees monotonic decrease of the local objective function and temporal smoothness. Computationally, our deterministic single-sweep update reduces the rank dependence from quadratic to linear, achieving an overall complexity of $\mathcal{O}(I^{n-1} r)$. Experimental results demonstrate that the proposed method outperforms existing online techniques not only in terms of mathematical approximation accuracy but also in human perception-based video quality metrics. Furthermore, compared to recent deep learning-based paradigms, our algebraic approach achieves speedups of several orders of magnitude. Consequently, our method exhibits high computational efficiency and is suitable for low-latency real-time processing applications.

cs.MM [Back]

[123] Evidence Triangulation for Multimodal Fact-Checking in the Wild cs.MM | cs.CVPDF

Stefanos-Iordanis Papadopoulos, Zacharias Chrysidis, Christos Koutlis, Symeon Papadopoulos, Panagiotis C. Petrantonakis

TL;DR: 该论文针对社交媒体中图像强化虚假声明的多模态错误信息问题，提出了一个名为X-POSE的真实世界多模态事实核查基准，以及一个名为TRENT的新型多模态事实核查模型。TRENT模型通过三个并行交叉注意力流和关系融合机制进行证据三角验证，以捕捉帖子与外部证据之间的细微关系。

Details

Motivation: 当前多模态事实核查研究依赖于合成训练数据和精心策划的基准，无法捕捉真实世界数据的复杂性，且现有检测模型要么依赖受限的模态内一致性，要么采用无约束的全对全融合，难以建模帖子与外部证据之间的微妙关系。

Result: 广泛的评估表明，TRENT模型在提出的X-POSE基准上持续优于最先进的专用模型和商业视觉语言模型（VLMs），达到了SOTA水平。

Insight: 论文的创新点在于构建了首个社区标注的真实世界多模态帖子基准X-POSE，并提出了TRENT模型，其核心是证据三角验证框架，通过三个并行交叉注意力流和显式建模蕴含与矛盾关系的关系融合机制，实现了对多模态证据更精细的关联建模。

Abstract: The proliferation of multimedia content on social platforms has fueled multimodal misinformation, where images are used to reinforce false claims. Consequently, Multimodal Fact-Checking (MFC) has emerged as an increasingly important research area. However, current progress is hindered by a reliance on synthetic training data and curated benchmarks that fail to capture the complexity of in-the-wild data. Furthermore, existing detection models rely on restricted intra-modality consistency or unconstrained all-to-all fusion, failing to capture nuanced relations between posts and external evidence. To address these limitations, we introduce X-POSE, a benchmark of real-world, community-annotated multimodal posts from X (formerly Twitter), augmented with full-length news articles retrieved via VLM-optimized search. Additionally, we propose TRENT, a novel MFC model that performs evidence triangulation using three parallel cross-attention streams alongside a relational fusion mechanism that explicitly models entailment and contradiction. Extensive evaluations demonstrate that TRENT consistently outperforms state-of-the-art specialized models and commercial VLMs. The code, prompt templates, and dataset are available at https://github.com/stevejpapad/evidence-triangulation

cs.AR [Back]

[124] FlexViT: A Flexible FPGA-based Accelerator for Edge Vision Transformers cs.AR | cs.CV | cs.DC | cs.LGPDF

Hubert Dymarkowski, Xingjian Fu, Rappy Saha, Jude Haris, José Cano

TL;DR: 本文提出了FlexViT，一种基于FPGA的可重构加速器，用于在资源受限的边缘设备上高效部署视觉Transformer（ViT）模型。它通过硬件-软件协同设计，将全连接层和卷积层映射到统一的INT8 GEMM引擎，并采用动态双模式数据流和深度优先分块策略来优化性能和内存带宽。

Details

Motivation: 在边缘平台上部署ViT模型面临挑战，因为其计算需求高，且现代混合ViT模型架构异构（包含全连接和卷积层），导致张量形状变化大，需要灵活高效的FPGA加速方案。

Result: 在PYNQ-Z2 FPGA上实现并评估了多种ViT模型，FlexViT在加速器执行层上实现了最高2.74倍的加速，相比仅用CPU执行，端到端加速最高达1.40倍。

Insight: 创新点包括：基于SECDA-TFLite框架的硬件-软件协同设计，使用运行时im2col变换统一处理全连接和卷积层；提出动态双模式数据流，通过运行时重配置计算阵列在输入和权重复用间切换；引入深度优先分块策略，单次完成累加以减少片外部分和传输及内存带宽需求。

Abstract: Deploying Vision Transformer (ViT) models on edge platforms remains challenging due to their high computational demands and the architectural heterogeneity of modern hybrid ViT models, which incorporate both fully connected and convolutional layers. This heterogeneity leads to significant variation in tensor shapes, requiring flexible and efficient FPGA-based acceleration. In this paper, we present FlexViT, a reconfigurable FPGA accelerator for efficient ViT inference on resource-constrained edge devices. Built on the SECDA-TFLite framework, FlexViT employs a hardware-software co-design approach that maps both fully connected and convolutional layers onto a unified high-throughput INT8 GEMM engine using a runtime im2col transformation. To efficiently support diverse layer configurations, we propose a dual-mode dataflow that dynamically switches between input and weight reuse by reconfiguring the compute array at runtime. We further introduce a depth-first tiling strategy that completes accumulation in a single pass, eliminating off-chip partial-sum transfers and reducing memory bandwidth requirements. We implement FlexViT on a PYNQ-Z2 FPGA and evaluate it across a representative set of ViT models. FlexViT achieves up to 2.74x speedup on accelerator-executed layers, translating into up to 1.40x end-to-end speedup compared to CPU-only execution. The code is available at: https://github.com/gicLAB/FlexViT

Table of Contents

cs.CL [Back]

[1] Indi-RomCoM: Code-Mixed Benchmark for Evaluating LLMs on Romanized Indic-English Instructions cs.CL | cs.AIPDF

[2] Test-Time Verification for Text-to-SQL via Outcome Reward Models cs.CL | cs.AI | cs.DBPDF

[3] Using AI Agents to Automate Black-Box Audits of Personalization Algorithms at Scale cs.CL | cs.CY | cs.LG | cs.SIPDF

[4] Training Therapeutic Judges and Multi-Agent Systems for Human-Aligned Mental Health Support cs.CL | cs.AI | cs.MAPDF

[5] Measuring Judgment Quality in Natural-Language Explanations: Evidence from Forecasting Tournaments cs.CL | econ.GNPDF

[6] Wait, am I Being Fair? Characterizing Deductive Stereotyping and Mitigating It with Fair-GCG cs.CL | cs.AIPDF

[7] A Semantic-Layer-Mediated Agent for Natural Language to SQL over Heterogeneous Enterprise Databases cs.CLPDF

[8] Building a Multimodal Dataset of Academic Paper for Keyword Extraction cs.CL | cs.DL | cs.HC | cs.IRPDF

[9] TAG-DLM: Diffusion Language Models for Text-Attributed Graph Learning cs.CL | cs.LGPDF

[10] LOPA: Enhancing Spoken Language Assessment via Latent Ordinal Prototype Alignment cs.CL | cs.MMPDF

[11] Clinically Structured Rank-Gated LoRA for Cross-Benchmark Medical Question Answering cs.CLPDF

[12] CLExEval: A Human-in-the-Loop Framework for Qualitative Evaluation of LLM Clinical Reasoning cs.CLPDF

[13] Seeing Is Not Sharing: Some Vision-Language Models Overestimate Common Ground in Asymmetric Dialogue cs.CL | cs.AIPDF

[14] CHERRY: Compressed Hierarchical Experts with Recurrent Representational Yield cs.CL | cs.AIPDF

[15] Theory of Mind and Persuasion Beyond Conversation: Assessing the Capacity of LLMs to Induce Belief States via Planning and Action cs.CLPDF

[16] LuxEmo: Expressive Text-to-Speech Corpus for Luxembourgish cs.CLPDF

[17] Explicit Fuzzy Logic in the Feed-Forward Layer: Self-Forgetting Quantifiers Discover Legible Grammatical-Licensing Detectors cs.CL | cs.LGPDF

[18] DigitalCoach: Communication and Grounding Gaps in Human and Agentic Computer Use Coaching cs.CLPDF

[19] Generative Skill Composition for LLM Agents cs.CLPDF

[20] When LLMs Read Tables Carelessly: Measuring and Reducing Data Referencing Errors cs.CL | cs.AIPDF

[21] Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs cs.CL | cs.AIPDF

[22] Introspective Coupling: Self-Explanation Training Tracks Behavioral Change Despite Fixed Supervision cs.CL | cs.AI | cs.LGPDF

cs.CV [Back]

[23] Streaming Gaussian Encoding for 4D Panoptic Occupancy Tracking cs.CV | cs.ROPDF

[24] AVTok: 1D Unified Tokenization for Holistic Audio-Video Generation cs.CV | cs.MM | cs.SD | eess.ASPDF

[25] Simple Supervision Is Hard to Beat: A Bitter Lesson from Sparse Target Labels in Domain-Adaptive Object Detection cs.CVPDF

[26] SyncCache: Exploiting Asymmetric Dynamics for Fast Audio-Driven Portrait Animation cs.CV | cs.SD | eess.ASPDF

[27] The Label Imitation Game: Turing Test Network for Zero-Shot Pseudo-Label Pruning cs.CV | cs.AI | cs.LGPDF

[28] GRAPE: Graph-Augmented Prototype Explanations for Interactive Medical Image Diagnosis cs.CVPDF

[29] No Adaptation Without Observation: Observability-Constrained Test-Time Prompt Tuning for LiDAR Semantic Segmentation cs.CVPDF

[30] Learning Where to Look: A Reinforcement Learning Framework for Robust Micro-Ultrasound Prostate Cancer Detection cs.CV | cs.AI | cs.LGPDF

[31] Auditing Generalization in AI-Generated Video Detection: A Six-Control Protocol and the VidAudit Toolkit cs.CVPDF

[32] Dense Structural Priors for Sparse Functional Landmark Localization in Surgical Videos cs.CVPDF

[33] Dual Sparse Aggregation Transformer for Multispectral Object Detection cs.CVPDF

[34] WarpI2I: Image Warping for Image-to-Image Translation cs.CVPDF

[35] Learning Video Dynamics with Predictive Differentiable Rendering cs.CV | cs.AIPDF

[36] ADAPT: Attention Dynamics Alignment with Preference Tuning for Faithful MLLMs cs.CV | cs.AI | cs.CL | cs.MMPDF

[37] Diffusion-Based Material Regularization for Physics-Based Inverse Rendering cs.CVPDF

[38] Hybrid Unet-Transformer Model for Generating Stress and Strain Fields from Composite Geometrics cs.CVPDF

[39] AnyMatch: Supercharging Universal Multi-Modal Image Matching with Large-Scale Single-View Images cs.CVPDF

[40] Towards Flexible, Natural, Efficient Interaction for Conversational Talking Face Generation cs.CVPDF

[41] Anchoring on Reality: Breaking the Pseudo-Target Ceiling in Makeup Transfer cs.CVPDF

[42] PiLoT v2: Pixel-to-Orthogonal Map Alignment for Free-view UAV Geo-localization cs.CVPDF

[43] TaxoMIL: Taxonomy-Constrained Learning for Hierarchical Whole Slide Image Analysis cs.CVPDF

[44] InfiniVerse: Occupancy Guided Unbounded Scene Generation for Autonomous Driving cs.CVPDF

[45] JacobianAvatar: Temporally Consistent Semi-rigid Avatar Reconstruction from a Monocular Video cs.CVPDF

[46] SkillSpotter: Pose-Aware Multi-View Skilled Action Detection and Grading in Ego-Exo Videos cs.CV | cs.AI | cs.LGPDF

[47] FROST: Training-Free Few-Shot Segmentation with Frozen Features and Nonparametric Statistics cs.CVPDF

[48] MSNN-LINet: Cross-Modal Learning via Continuous Linear Integration cs.CV | cs.LGPDF

[49] WaterGen: Decoupling Scene and Medium in Underwater Image Generation cs.CVPDF

[50] PruneGround: Plug-and-play Spatial Pruning for 3D Visual Grounding cs.CV | cs.AI | cs.CLPDF

[51] Rethinking Foundation Model Collaboration: Enhancing Specialized Models through Proxy Task Reasoning cs.CVPDF

[52] Reasoning-aware Speculative Decoding for Efficient Vision-Language-Action Models in Autonomous Driving cs.CVPDF

[53] Beyond Single Character: Evaluating MLLMs for Sentence-Level Oracle Bone Inscription Understanding cs.CVPDF

[54] Learning to Deny: Action Denial in Multimodal Large Language Models cs.CVPDF

[55] AC3S: Adaptive Conditioning for 3D-Aware Synthetic Data Generation cs.CVPDF

[56] Distilling Temporal Coherence into 2D Networks for Transrectal Ultrasound Prostate Video Segmentation cs.CV | cs.AIPDF

[57] AA: A Multi-view Multimodal Dataset for Screen-based Gaze Estimation cs.CV | cs.HCPDF

[58] CooperScene: Multi-Modal Cooperative Autonomy Benchmark with C-V2X Communication Characterization cs.CVPDF

[59] HyperVLP: Enhancing Hierarchical Surgical Video-Language Pre-training in Hyperbolic Space cs.CVPDF

[60] Rethinking the Role of Feature Engineering and Learning Strategies in Few-Shot Hidden Emotion Recognition cs.CVPDF

[61] Learning from Failure: Inference-Time Self-Improvement for Computer-Use Agents cs.CV | cs.AI | cs.CL | cs.CY | cs.LGPDF

[62] Editing Everything Everywhere All at Once cs.CVPDF

[63] Decodable Is Not Grounded: A Vision-Ablation Arbiter for VLM Spatial Reasoning cs.CVPDF

[64] Bridging Video Understanding and Generation in a Unified Framework cs.CVPDF

[65] Language-Assisted Super-Resolution from Real-World Low-Resolution Patches cs.CVPDF

[66] MS-Resampler: Multi-Scope Visual Resampling for Efficient Multimodal LLMs cs.CVPDF

[67] Visual Semantic Entropy: Do Vision Language Models Recognize Visual Ambiguity? cs.CV | cs.AI | cs.CLPDF

[68] One Video, One World: Turning Monocular Video into Physical 4D Scenes cs.CVPDF

[69] Temporal Preservation over Processing: Diagnosing and Designing Spatiotemporal Single-Stage Video Detectors cs.CV | cs.AIPDF

[70] AeroVerse-SatAgent: UAV-Satellite Collaborative Spatial Reasoning Inspired by the Dual Visual Pathway Theory of Cognitive Neuroscience cs.CVPDF

[71] Think While You Map: Asynchronous Vision-Language Agents for Incremental 3D Scene Graphs cs.CVPDF

[72] HVPNet: A Bio-Inspired Network for General Salient and Camouflaged Object Detection cs.CVPDF

[73] SimpleSearch-VL: A Simple Recipe for Multimodal Agentic Deep Search cs.CVPDF

[74] MV-GEL: Language-Driven Multi-View Geometric Entity Localization on Meshes cs.CVPDF

[75] Localized Conformal Prediction for Image Classification with Vision-Language Models cs.CV | cs.LGPDF

[76] AugSplat: Radiance Field-Informed Gaussian Splatting for Sparse-View Settings cs.CVPDF

[77] DPPE: Rethinking Camera-Based Positional Encoding for Scaling Multi-View Transformers cs.CV | cs.AIPDF