cs.CL [Total: 31]
cs.CV [Total: 81]
cs.GR [Total: 1]
cs.LO [Total: 1]
eess.IV [Total: 1]
cs.IR [Total: 1]
eess.AS [Total: 2]
q-bio.NC [Total: 1]
cs.DB [Total: 1]
cs.LG [Total: 7]
cs.AI [Total: 4]
cs.MM [Total: 1]
cs.RO [Total: 4]

cs.CL [Back]

[1] IdiomX A Multilingual Benchmark for Idiom Understanding, Retrieval, and Interpretation cs.CL | cs.AI | cs.IRPDF

Ayman Ali Sharara

TL;DR: 本文介绍了IdiomX，一个用于习语理解、检索和解释的大规模多语言基准测试。它通过一个可复现的多阶段流程构建，包含超过19万个语境化示例，涵盖1.2万多个习语，并提供了英语、阿拉伯语和法语的对齐语义表示、习语与字面用法标签以及丰富的语言元数据。基于此资源，论文定义了一个统一的四任务基准，涵盖习语检测、语境到习语检索、阿拉伯语到英语习语检索以及习语解释，将评估从比喻识别扩展到语义基础和可解释的意义检索。

Details

Motivation: 习语表达因其含义通常是非组合性、依赖于上下文且难以跨语言对齐，一直是自然语言处理的持续挑战。现有习语资源通常在规模、语境多样性或多语言覆盖方面有限，限制了其在现代语言模型中的实用性。

Result: 实验表明，上下文Transformer模型显著提高了习语检测的性能，而混合检索和重排序架构则显著增强了单语和跨语言习语检索。结果进一步证明，习语解释可以有效地建模为语义检索任务，从而将可解释性引入作为补充的基准维度。

Insight: 论文的主要创新点在于构建了一个大规模、多语言、语境丰富的习语基准数据集IdiomX，并定义了一个从检测到检索再到语义解释的统一评估框架。从客观角度看，其将习语解释任务形式化为语义检索，以及提供可扩展、模块化的框架以支持更多语言和比喻推理任务，是具有借鉴意义的创新之处。

Abstract: Idiomatic expressions remain a persistent challenge for natural language processing because their meanings are often non-compositional, context-dependent, and difficult to align across languages. Existing idiom resources are often limited in scale, contextual diversity, or multilingual coverage, restricting their utility for modern language models. We introduce IdiomX, a large-scale multilingual benchmark for idiom understanding, retrieval, and interpretation, constructed through a reproducible multi-stage pipeline combining lexical resource extraction, large-scale normalization, controlled large language model enrichment, and structured validation. The resulting dataset contains over 190K contextualized examples spanning 12K+ idioms, with aligned English, Arabic, and French semantic representations, idiomatic and literal usage labels, and rich linguistic metadata. Building on this resource, we define a unified four-task benchmark covering idiom detection, context-to-idiom retrieval, Arabic-to-English idiom retrieval, and idiom interpretation, extending evaluation from figurative recognition to semantic grounding and explainable meaning retrieval. Experiments show that contextual transformer models substantially improve idiom detection, while hybrid retrieval and reranking architectures significantly strengthen both monolingual and cross-lingual idiom retrieval. Results further demonstrate that idiom interpretation can be effectively modeled as a semantic retrieval task, introducing interpretability as a complementary benchmark dimension. Overall, IdiomX provides a scalable benchmark for studying idiomatic language as a progression from detection to retrieval and semantic interpretation, and offers a modular framework extensible to additional languages and figurative reasoning tasks

[2] Economy of Minds: Emerging Multi-Agent Intelligence with Economic Interactions cs.CL | cs.AI | cs.MAPDF

Zhenting Qi, Huangyuan Su, Ao Qu, Chenyu Wang, Yu Yao

TL;DR: 本文提出了一种基于经济交互的多智能体系统，灵感来源于哈耶克的去中心化市场协调理论。通过拍卖竞争、支付交换和财富积累等简单经济信号，系统实现了去中心化的信用分配和规划，无需全局协调或显式通信协议。智能体群体通过经济选择进化：有效智能体积累财富并通过利用进行变异，无效智能体破产并通过探索被替换。

Details

Motivation: 研究如何在没有集中控制的情况下，使智能体群体自我组织和自我适应，形成更强的集体智能。

Result: 在数学推理、金融研究、科学研究、加速器设计和分布式系统优化等五个智能体任务中，该方法优于更强的单体基线模型，并涌现出多步推理策略。

Insight: 创新点在于将经济理论引入多智能体系统，通过设计去中心化的激励结构自动涌现协调行为，而非直接设计协调机制。这为多智能体智能提供了一条新路径，将局部激励与长期全局性能联系起来。

Abstract: How can a population of agents self-orchestrate and self-adapt into stronger collective intelligence without centralized control? Inspired by Friedrich Hayek’s economic theory of decentralized coordination in markets, we study this question through an agent economy in which agents compete via auctions for the right to act, exchange payments, and accumulate wealth from environmental rewards. These simple economic signals induce decentralized credit assignment, driving planning without global orchestration or explicit communication protocols. The population evolves through economic selection: effective agents accumulate wealth and are mutated via exploitation, while ineffective ones go bankrupt and are replaced via exploration. We show that, initialized with weak agents, the economy produces emergent multi-step reasoning strategies and outperforms stronger monolithic baselines across five agentic tasks, including mathematical reasoning, financial research, scientific research, accelerator design, and distributed-system optimization. We further provide theoretical insights into how economic dynamics shape agent behaviors, linking local incentives to long-term global performance. Our results suggest a new path to multi-agent intelligence: rather than engineering coordination, we can design decentralized incentive structures under which it automatically emerges.

[3] Adaptive Latent Agentic Reasoning cs.CL | cs.AIPDF

Dongwon Jung, Peng Shi, Yi Zhang, Junshan Zhang, Muhao Chen

TL;DR: 本文提出了一种名为自适应潜在智能体推理（ALAR）的双模式框架，旨在解决大型语言模型（LLM）智能体在多轮交互中推理效率低下的问题。该框架通过使用紧凑的潜在推理处理常规步骤，并仅在需要深度思考时切换到显式的思维链推理，从而显著减少了生成的文本量。

Details

Motivation: 当前LLM智能体在每个决策步骤都生成冗长的文本推理，且在各轮次间近乎均匀地分配推理努力，导致多轮智能体轨迹的效率低下。

Result: 在智能体搜索和工具使用基准测试上的实验表明，ALAR在保持相当或更好任务准确率的同时，将生成的令牌数量大幅减少了高达43.6%（搜索）和84.6%（工具使用），改善了LLM智能体的准确率-效率权衡。

Insight: 核心创新点是引入了基于智能体动作作为监督锚点来学习潜在推理的双模式框架，并优化了在任务成功足够时使用潜在推理、为更困难决策保留显式思维链的机制，从而实现了推理过程的动态自适应和效率提升。

Abstract: Large reasoning models improve performance by generating extended chain-of-thought (CoT) reasoning, but this behavior becomes inefficient when applied to LLM agents. Current LLM agents often generate verbose textual reasoning at every decision step and allocate reasoning effort nearly uniformly across turns, leading to substantial inefficiency in multi-turn agentic trajectories. We propose Adaptive Latent Agentic Reasoning (ALAR), a dual-mode framework that uses compact latent reasoning for routine turns and selectively escalates to explicit chain-of-thought when deeper deliberation is needed. ALAR learns latent reasoning by using the agent’s actions as supervision anchors and is further optimized to use latent reasoning when it is sufficient for task success and reserve explicit CoT for harder decisions. Experiments on agentic search and tool-use benchmarks show that ALAR maintains comparable or better task accuracy while substantially reducing generated tokens by up to 43.6% in search and 84.6% in tool use. These results demonstrate that ALAR improves the accuracy-efficiency trade-off of LLM agents by reducing unnecessary textual reasoning while preserving explicit deliberation for harder decision steps.

[4] Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States cs.CL | cs.AIPDF

Subramanyam Sahoo, Vinija Jain, Aman Chadha, Divya Chaudhary

TL;DR: 该论文通过线性探测分析Qwen3-14B模型在LogiQA 2.0、ARC-Challenge和αNLI三个基准测试中的隐藏状态，发现高探测精度主要源于任务格式的混淆因素，而非模型内部不同的推理模式。

Details

Motivation: 旨在检验线性探测方法是否真能揭示大语言模型对不同推理类型（演绎、归纳、溯因）学习到不同的表征，还是仅仅捕捉了任务格式的差异。

Result: 在去除源身份、选项数量和响应长度等格式混淆因素后，线性探测准确率降至随机水平；因果干预实验也显示几何结构与推理模式之间无显著功能联系（p=0.286）。

Insight: 论文的创新点在于系统揭示了线性探测结果可能被任务格式严重混淆，强调了在机制可解释性研究中进行常规格式去混淆的必要性，并提出了使用残差化和因果控制等方法进行验证。

Abstract: Linear probing of large language model (LLM) hidden states is widely used to claim that models learn distinct representations for different reasoning types. We test this by probing Qwen3-14B on three benchmarks spanning the classical trichotomy: LogiQA 2.0 (deductive), ARC-Challenge (inductive), and $α$NLI (abductive). At layer 32 of 40, linear probes achieve 100% cross-validated accuracy with well-separated geometry (intrinsic dimensionalities: 20.6, 28.5, 33.6; convex hull contamination $\leq$1.5%). However, this separation is entirely driven by format confounds. Residualizing source identity, option count, and response length reduces accuracy to chance. Trace-anchor similarity indicates largely shared reasoning across tasks (42.5% agreement vs.\ 33.3% chance), and causal steering with random controls ($n=20$) shows no functional link between geometry and reasoning mode ($p=0.286$). Thus, high probe accuracy reflects task format rather than computational structure, motivating routine format deconfounding in mechanistic interpretability.

[5] WRIT: Write-Read Intensive Trajectory Synthesis for Multi-Turn User-Facing Agents cs.CL | cs.AIPDF

Hengrui Gu, Xiaotian Han, Kaixiong Zhou

TL;DR: 本文提出WRIT（Write-Read Intensive Trajectory Synthesis）方法，用于合成多轮用户交互智能体的训练轨迹。该方法通过同时增加任务中的决策数量和每个决策所需的证据负担来提升轨迹复杂性，旨在训练智能体不仅能执行长序列任务，还能在高信息负载下进行稳健的、基于证据的决策。

Details

Motivation: 现有训练轨迹合成方法主要通过组合多个用户请求来生成长任务（即写密集型轨迹），但忽略了单个决策本身的难度，即智能体在做出决策前需要收集和比较大量工具读取的证据。仅靠写密集型数据无法解决这一挑战。

Result: 实验表明，仅使用2K条WRIT合成的轨迹训练的4B模型，在τ²-bench基准上超越了GPT-5.1 no-think模型，并显著减少了推理时的token使用量。

Insight: 创新点在于提出了一个沿两个复杂度轴（决策数量和每个决策的证据负担）合成轨迹的框架，并引入了用户行为指令多样化以模拟真实对话变化。客观来看，该方法的核心洞察是将昂贵的测试时推理部分转化为高效的智能体行为，通过紧凑的监督微调数据实现性能提升。

Abstract: Multi-turn user-facing agents must infer user intent from incomplete requests, collect missing information through dialogue and tools, and execute valid actions. A training trajectory records this process as an interleaved sequence of user messages, agent responses, tool calls, etc. Synthesizing sufficiently complex trajectory has become a central route to train agents: existing pipelines often increase difficulty by composing multiple user requests into longer tasks, producing write-intensive trajectories that train sequential execution. We argue that a single write decision can itself be difficult when the agent must gather and compare substantial read-tool evidence before its arguments become identifiable, a challenge that write-intensive data alone cannot address. Guided by this insight, we propose WRIT (\uline{W}rite-\uline{R}ead \uline{I}ntensive \uline{T}rajectory Synthesis), a pipeline for synthesizing multi-turn agent training trajectories along two complexity axes: the number of write decisions in a task and the evidence burden of each individual decision. WRIT first generates write-intensive and read-heavy tasks. It then diversifies user behavior instructions to reflect realistic conversational variation, and finally simulates agent-user interactions in an executable environment to produce complete training trajectories. The resulting data trains agents not only for longer task execution, but also for robust, evidence-grounded decision making under high information load. With only 2K synthesized trajectories, a 4B model trained on WRIT outperforms GPT-5.1 no-think on $τ^2$-bench and substantially reduces inference-time token usage, showing that compact SFT data can convert part of expensive test-time reasoning into efficient agent behavior.

[6] EURO-5K: When Does Domain Pretraining Matter? Benchmarking Transformers for EU Reporting Obligation Extraction cs.CLPDF

Marios Koniaris, Vasileios Kotronis, Eugenia Giannini, Panayiotis Tsanakas

TL;DR: 本文介绍了EURO-5K数据集，这是一个用于从欧盟立法中提取报告义务的句子级语料库，包含来自136项法案的正负样本。研究系统比较了判别式（BERT风格）和生成式（LLM）模型，以及全微调与参数高效调优（QLoRA）等策略，评估了领域预训练的价值。

Details

Motivation: 从欧盟立法中准确提取报告义务对于评估和减轻监管负担至关重要，但现有法律NLP方法缺乏专门的、带有清晰指南的数据集，以及对不同提取范式和领域适应策略的比较评估。

Result: 在EURO-5K上，全微调的通用和法律BERT模型性能相似（F1 0.89），微调的LLM在句子级提取上达到编码器精度。法律预训练对生成模型增益有限，但在参数高效调优（如QLoRA）下，Legal-BERT明显优于通用BERT。学习曲线分析表明，约3000个样本后性能趋于饱和。在两个外部监管语料库上的跨数据集评估表明，模型是专门的报告义务提取器，而非通用监管分类器。

Insight: 核心创新在于构建了高质量的专用数据集EURO-5K，并进行了全面的范式与适应策略对比。关键洞见是：在数据充足时，法律领域预训练的优势有限；但在参数高效调优等受限场景下，其价值显著，能加速早期学习。研究为监管合规自动化提供了实用的模型选择与训练策略指导。

Abstract: Extracting reporting obligations from EU legislation is critical for assessing and reducing regulatory reporting burden. However, distinguishing reporting requirements from structurally similar provisions requires specialised legal understanding. Current legal NLP methods lack specialised datasets with clear guidelines and comparative evaluation of extraction paradigms and domain adaptation strategies. We curate EURO-5K, a corpus of sentence-level reporting obligations and challenging negative examples from 136 EU legislative acts. On this dataset, we train and compare discriminative token-classification models (BERT-style) and generative span-extraction models (LLMs), evaluating both full fine-tuning and parameter-efficient QLoRA against baselines (pattern and dependency-based extraction, few-shot prompting). Results show that fully fine-tuned generic and legal BERT models achieve similar performance (0.89 F1), while fine-tuned LLMs match encoder accuracy for sentence-level extraction. Legal pretraining offers only small gains for generative models. In contrast, it is clearly beneficial when adaptation capacity is constrained, as parameter-efficient tuning of Legal-BERT outperforms its generic counterpart. Learning curve analysis demonstrates that legal pretraining accelerates early learning with minimal data. All approaches converge around 3K samples with diminishing returns thereafter, validating dataset sufficiency. Cross-dataset evaluation on two external regulatory corpora shows that our models behave as specialised reporting obligation extractors rather than generic regulatory classifiers. We release EURO-5K, trained models, and an interactive demo with explainability visualizations and structured RDF export. These demonstrate that both paradigms and parameter-efficient training provide practical tools for regulatory compliance automation.

[7] Predicting Inference-Time Scaling Gains from Labeled Validation-Set Output Statistics cs.CLPDF

Luyang Zhang, Jingyan Li

TL;DR: 本文提出了一种预测最佳N推理（Best-of-N）缩放增益的方法，通过分析验证集输出统计特征（如一致性、多样性、置信度等）来构建紧凑的预测器，无需进行端到端的奖励模型评分，从而在配置筛选阶段降低成本。

Details

Motivation: 解决当前预测最佳N推理缩放增益需要运行完整端到端流程的问题，旨在利用验证集输出统计特征提前预测增益，以降低候选配置筛选的成本。

Result: 在三个基础模型家族、六种后训练方法和数学推理任务域上，基于核心三特征（提示级一致性分布、标签辅助的首个正确样本位置、完成长度方差）加熵增补的紧凑岭预测器，与奖励模型验证下的实际最佳N增益达到Spearman ρ=0.90的相关性。

Insight: 创新点在于通过稳定性分析（如bootstrap-Lasso）识别出稳定的核心特征集，构建线性近似残差的集中分析，提供了一种低成本预测推理缩放增益的有效方法，可应用于模型配置的验证集筛选。

Abstract: Best-of-$N$ inference scaling (drawing $N$ candidate answers from a language model and returning the one a reward model ranks highest) improves accuracy by an amount that varies across models, but predicting that amount in advance currently requires running the procedure end-to-end. Prior work links cheap statistics of a model’s sampled outputs and validation-set correctness (how often samples agree, how diverse they are, how confident the model is, and where correct samples appear) to model behavior, but does not isolate which of these form a stable, compact predictor of best-of-$N$ gain. We fit ridge predictors on features computed from a single labeled validation-set sampling pass, use bootstrap-Lasso as a stability analysis of the candidate feature set, and give a concentration analysis with an explicit linear-approximation residual. Across three base-model families, six post-training methods, and math and reasoning task domains, the stability analysis identifies a strict three-feature core spanning prompt-level agreement spread, label-assisted first-correct-sample position, and completion-length variance; a compact ridge predictor built from this core plus an entropy add-on reaches Spearman $ρ= 0.90$ with actual best-of-$N$ gain under a reward-model verifier. The intended use is labeled validation-set screening of candidate configurations before paying the full reward-model scoring cost.

[8] Hint-Guided Diversified Policy Optimization for LLM Reasoning cs.CLPDF

Zhiyu Cao, Kaixin Wu, Mingjie Zhong, Peifeng Li, Xiaobo Li

TL;DR: 本文提出了一种名为Hint-Guided Diversified Policy Optimization (HDPO)的新方法，用于增强大型语言模型(LLM)的推理能力。该方法受人类解题过程的启发，通过一个‘提出-选择-思考’的轨迹，引导模型首先生成多样化的候选解决方案大纲作为提示，然后选择最可靠的一个进行深入推理。实验表明，HDPO能有效提升LLM的推理性能、解决方案的多样性以及模型识别可靠方案的能力。

Details

Motivation: 现有基于可验证奖励的强化学习(RLVR)方法仅关注结果层面的正确性，缺乏引导模型考虑多样化解决方案的显式信号，这与人类评估多种潜在方法并选择最可靠方案的认知过程不符。

Result: 实验结果表明，HDPO有效提升了LLM的推理能力，增强了候选解决方案的多样性以及模型识别可靠解决方案的能力。

Insight: 核心创新在于将‘生成多样化候选方案’作为中间奖励信号引入强化学习框架，通过‘结构化推理冷启动’和‘提示引导的多样化强化学习’两阶段训练，显式激励模型遵循‘提出-选择-思考’的推理轨迹，这模仿了人类的认知过程，是对现有RLVR范式的有益扩展。

Abstract: Recent developments in Large Language Models (LLMs) have showcased impressive reasoning capabilities, with Reinforcement Learning with Verifiable Rewards (RLVR) being a promising enhancement strategy. However, existing reward mechanisms are constrained to the outcome-level correctness and lack explicit signals to guide the model to consider diverse solutions. In contrast, human problem solving typically involves evaluating multiple potential approaches and selecting the most reliable solution, a cognitive process that current RLVR frameworks do not explicitly incentivize. Inspired by this, we propose Hint-Guided Diversified Policy Optimization (HDPO), allowing the model to first list all potential candidate solution outlines as hints and then select the most reliable one for further reasoning. HDPO comprises two stages of Cold Start for Structured Reasoning and Hint-Guided Diversified Reinforcement Learning to incentivize the model to generate diverse and reliable solutions following the ``propose-select-think’’ trajectory. Experimental results show that HDPO effectively boosts LLM reasoning and enhances the diversity of candidate solutions as well as the LLM’s ability to identify reliable solutions.

[9] PhotoCraft: Agentic Reasoning with Hierarchical Self-Evolving Memory for Deep Image Search cs.CL | cs.AIPDF

Kailin Lyu, Zhiqiang Yuan, Jianwei He, Qiwei Yan, Xuanbo Su

TL;DR: 本文提出了PhotoCraft，一种用于深度图像搜索的无训练、分层记忆系统。该系统受人类认知启发，为多模态大语言模型配备了工作、情景和语义记忆，以在多步推理中保持逻辑一致性并促进知识迁移，从而解决现有无状态智能体在长时上下文维护和跨任务经验转移方面的不足。

Details

Motivation: 现有基于LLM的智能体大多是无状态和反应式的，缺乏持久记忆来维持长时上下文或在任务间转移经验，这导致了执行漂移和经验隔离问题。本文旨在解决深度图像搜索中多步推理对丰富上下文线索的需求与智能体记忆能力不足之间的矛盾。

Result: 在DISBench基准上的大量实验表明，PhotoCraft能持续提升不同MLLM骨干网络的情境感知检索能力，最高可获得18.5%的性能增益，有效缓解了无记忆深度图像搜索中的关键瓶颈。

Insight: 主要创新点在于将人类认知中的分层记忆结构（工作、情景、语义记忆）引入多模态智能体，以无训练的方式动态调用，从而在多步推理中实现逻辑一致性和知识可迁移性。这为构建可靠且可泛化的多模态搜索智能体提供了一条实用路径。

Abstract: Deep Image Search requires multi-step reasoning over rich contextual cues, such as time, location, and event relations. However, most existing LLM-based agents are stateless and reactive, lacking persistent memory to maintain long-horizon context or transfer experience across tasks, which often leads to execution drift and experience isolation. To address these limitations, we propose PhotoCraft, a training-free, hierarchical memory system for photo-search agents. Inspired by human cognition, PhotoCraft equips MLLMs with working, episodic, and semantic memory, which are dynamically invoked during reasoning to preserve logical consistency and knowledge transferability throughout multi-step reasoning and answer generation. Extensive experiments on DISBench demonstrate that PhotoCraft consistently improves context-aware retrieval across diverse MLLM backbones, achieving gains of up to 18.5% and effectively mitigating key bottlenecks in memoryless deep image search, offering a practical path toward reliable and generalizable multimodal search agents.

[10] DMT-CBT: Longitudinal Therapeutic State Modeling for CBT Counseling cs.CLPDF

Chang Liu, Shuyi Zhang, Changsheng Ma, Yongfeng Tao, Minqiang Yang

TL;DR: 本文提出了DMT-CBT框架，用于在认知行为疗法咨询中对动态演化的治疗状态进行纵向建模。该框架通过整合多模态行为基础和工具增强干预，支持跨会话的自适应治疗推理，并构建了包含演化治疗状态的多会话多模态合成数据集DMTCorpus。实验表明，该方法在咨询保真度、治疗联盟、纵向情感轨迹和治疗状态保持方面优于事后提取方法。

Details

Motivation: 现有基于大语言模型的CBT咨询方法通常将咨询视为局部响应生成问题，仅关注短文本或单次会话内的共情回复，这与真实心理治疗作为跨会话的纵向过程本质不匹配。真实的CBT涉及多模态推断和跨会话的延迟干预效应，需要在部分可观测性下捕捉纵向治疗状态的演化。

Result: 实验结果表明，DMT-CBT在咨询保真度和治疗联盟方面有所提升，能产生更有利的纵向情感轨迹，并且比事后提取方法更忠实地保持治疗状态。

Insight: 创新点在于将CBT咨询建模为跨会话的纵向动态治疗状态演化问题，而非孤立的回复生成；提出了结合结构化治疗状态、多模态行为基础和工具增强干预的框架；构建了包含连续干预和多模态信息的合成数据集，以支持对治疗过程的更真实建模。

Abstract: Large language models (LLMs) have shown growing potential for Cognitive Behavioral Therapy (CBT) counseling. However, most existing approaches still formulate counseling as a local response generation problem, focusing on empathetic replies within short, text-only, or single-session interactions. We argue that this formulation fundamentally mismatches the nature of real psychotherapy. In clinical CBT, therapy is a longitudinal process in which therapists continuously infer, update, and intervene on evolving therapeutic states across sessions. Realistic CBT further involves multimodal inference and delayed cross-session intervention effects, requiring models to capture longitudinal therapeutic state evolution under partial observability. We propose DMT-CBT, a framework for Dynamic Modeling of evolving Therapeutic states in CBT counseling. DMT-CBT maintains structured therapeutic states across sessions while incorporating multimodal behavioral grounding and tool-augmented intervention to support adaptive therapeutic reasoning. Based on this framework, we construct DMTCorpus, a synthetic multi-session multimodal CBT counseling dataset featuring evolving therapeutic states, image-grounded client behaviors, and cross-session intervention continuity. Experimental results show that DMT-CBT improves counseling fidelity and therapeutic alliance, produces more favorable longitudinal affective trajectories, and preserves therapeutic states more faithfully than post-hoc extraction approaches.

[11] MemTrain: Self-Supervised Context Memory Training cs.CLPDF

Ziheng Li, Xingrun Xing, Haoqing Wang, Zhi-Hong Deng, Yehui Tang

TL;DR: 本文提出MemTrain，一种用于增强LLM智能体上下文记忆能力的自监督训练框架。该框架通过两个在未标注维基百科语料上的代理任务（掩码重建和中间记忆召回）进行联合优化，旨在提升智能体在长序列交互中的记忆维护与压缩能力。

Details

Motivation: 现有基于强化学习的记忆智能体方法需要高质量标注数据，成本高且多样性不足，难以覆盖通用记忆行为。MemTrain旨在通过自监督方式，利用无标注文本增强LLM智能体的通用记忆能力，以支持更有效的下游任务后训练。

Result: 在长文本QA和基于搜索的QA基准测试上的广泛实验表明，MemTrain能持续提升不同模型在下游记忆密集型推理任务上的性能，相比直接的任务特定后训练最高可获得17.67分的性能提升。

Insight: 创新点在于设计了两个耦合的自监督代理任务，分别从最终结果和中间过程角度鼓励记忆的维护与忠实压缩；采用GRPO进行联合优化，提供了一种无需下游标注数据即可通用增强LLM记忆能力的方法。

Abstract: Memory is an indispensable capability for long-horizon LLM agents, enabling them to preserve and utilize information accumulated across extended interactions. Existing memory-agent approaches are typically trained end-to-end with reinforcement learning on downstream tasks. However, collecting high-quality annotated problems for memory-intensive scenarios is costly, and the resulting training data often lack sufficient diversity to cover general memory behaviors. In this work, we propose MemTrain, a self-supervised training framework for generally enhancing the context-memory capability of LLM agents for more effective downstream post-training. MemTrain introduces two coupled proxy tasks over unlabeled Wikipedia corpora: (1) an end-to-end masked reconstruction objective, which requires the model to recover masked entities after multiple rounds of memory updates, thereby encouraging memory maintenance from the final outcome perspective; and (2) an intermediate memory recall objective, which requires the model to reconstruct masked historical information using intermediate memory states, encouraging faithful compression and memory completeness throughout the interaction process. The two objectives are jointly optimized using GRPO. Extensive experiments on long-text QA and search-based QA benchmarks demonstrate that MemTrain consistently improves downstream memory-intensive reasoning performance across different models, achieving gains of up to 17.67 points over direct task-specific post-training.

[12] HyperPatch: Sequential Knowledge Editing Under n-ary Structural Drift cs.CLPDF

Yu-Kai Chan, Wen-Sheng Lien, Dong-Ting Yao, Bo-Kai Ruan, Kwan-Yeung Lin

TL;DR: 论文提出HyperPatch框架，用于解决大语言模型在非平稳环境中进行顺序知识编辑时出现的n元结构漂移问题。该框架将顺序知识编辑重新定义为超图流形上的稳定性问题，通过结构先验初始化、顺序拓扑编辑和结构条件推理三个阶段，保持事件完整性并避免知识转移失败。

Details

Motivation: 解决大语言模型在现实世界知识编辑中面临的核心问题：真实知识本质上是n元的，而顺序更新复杂关系会导致n元结构漂移，即将n元事件二元化为三元组会破坏关系原子性，进而引发结构条件知识转移失败（常被误诊为参数幻觉）。

Result: 在MQuAKE-CF和MQuAKE-T基准测试中，HyperPatch相比最强基线分别实现了96.24%和21.06%的Hop-wise Accuracy相对提升。消融实验显示其在连续n元更新流中具有优越的可靠性，而基于标准知识图谱的变体因结构错位导致H-Acc下降高达88.3%。

Insight: 创新点包括：1）将顺序知识编辑重新定义为超图流形稳定性问题；2）提出三阶段框架（结构先验初始化、顺序拓扑编辑、结构条件推理）保持事件完整性；3）采用基于SimHash的拓扑对齐快速解决冲突，结合拓扑LoRA适配跟踪漂移而无需主干网络重训练；4）融合语言和结构流形的全局一致证据进行推理。

Abstract: Large Language Models (LLMs) rely on Knowledge Editing (KE) to maintain temporal validity, yet real-world knowledge is inherently n-ary. We demonstrate that in non-stationary environments, sequential updates to complex relations induce N-ary Structural Drift, a phenomenon where the binary reification of n-ary events into triples fractures relational atomicity. This precipitates Structure-Conditioned Knowledge Transfer Failure, a systematic mis-grounding of the retriever frequently misdiagnosed as parametric hallucination. To tackle this, we propose HyperPatch, a parameter-preserving framework that reformulates sequential KE as a stability problem over hypergraph manifolds. HyperPatch preserves event integrity through three phases: (i) Structural Prior Initialization, establishing a topology-aware embedding space via contrastive learning on a Hypergraph Neural Network (HGNN) to capture high-order correlations; (ii) Sequential Topology Editing, utilizing a dual-stage mechanism that employs SimHash-based Topological Alignment for rapid conflict resolution and Topological LoRA Adaptation to track drift without backbone retraining; and (iii) Structure-Conditioned Reasoning, which integrates globally consistent evidence from fused linguistic and structural manifolds. On the MQuAKE-CF and MQuAKE-T benchmarks, HyperPatch achieves relative gains in Hop-wise Accuracy (H-Acc) of 96.24% and 21.06% over the strongest baseline, respectively. Further ablations demonstrate superior reliability under continuous n-ary update streams, whereas the standard KG-based variant suffers H-Acc collapses of up to 88.3% due to structural misalignment.

[13] WebRISE: Requirement-Induced State Evaluation for MLLM-Generated Web Artifacts cs.CL | cs.AIPDF

Yuxin Meng, Yuhan Suo, Junjie Wang, Yuhan Sun, Yiyao Yu

TL;DR: 本文提出了WebRISE，一个用于评估多模态大语言模型（MLLM）生成的网页制品的新基准。它将任务需求编译为交互契约图（ICG），包含可观察状态、用户意图转换和DOM/视觉断言，以进行与实现无关的浏览器执行。该基准包含442个任务，涵盖五种输入模态，并揭示了当前顶级MLLM在状态转换有效性和需求覆盖方面存在显著不足。

Details

Motivation: 现有评估MLLM生成网页制品的基准主要依赖局部证据进行交互评估，忽略了决定页面是否正常工作的、由需求诱导的状态和状态转换。

Result: 在评估的14个MLLM中，即使最强的模型在状态转换有效性上也仅达到65.6%，需求覆盖率为66.3%。视频模态提供了最强的交互信号（比文本模态在隐式约束覆盖上高10.6个百分点）。基于ICG的评分检测状态错误的效率是检查点式评估的2到16倍。

Insight: 核心创新在于提出了交互契约图（ICG）这一抽象，将用户陈述的功能与隐式产品级约束分离，实现了对网页制品功能行为的细粒度、与实现无关的评估。研究还揭示了视觉质量不能替代行为正确性，并且不同输入模态对交互能力的影响存在差异。

Abstract: Existing benchmarks for MLLM-generated web artifacts assess interaction through local evidence and miss the requirement-induced states and transitions that determine whether a page works. We introduce WebRISE, which compiles task requirements into Interaction Contract Graphs (ICGs) of observable states, user-intent transitions, and DOM/visual assertions for implementation-agnostic browser execution. WebRISE spans 442 tasks across five input modalities (Text, Markdown, Sketch, Image, Video), with 5,495 transitions and 5,271 requirement checks that separate user-stated functions from implicit product-level constraints. Across 14 MLLMs, even the strongest model reaches only 65.6% transition validity and 66.3% requirement coverage, and visual quality is no proxy for behavior (Qwen3.6-35B-A3B on Markdown: V=80.8 yet T=15.5). Video gives the strongest interaction signal (+10.6 pp implicit coverage over Text), while implicit constraints persist; defect injection shows ICG-based scoring detects state errors at 2-16x the rate of checkpoint-style evaluation.

[14] ARBOR: Online Process Rewards via a Reusable Rubric Buffer for Search Agents cs.CLPDF

Zheng Liu, Longxiang Zhang, Xintong Wang, Zhiang Xu, Shaoxiong Zhan

TL;DR: 本文提出ARBOR框架，通过维护一个可重用的评分标准缓冲区为基于LLM的搜索代理提供在线过程奖励。该方法从对比轨迹中提取查询局部草稿，将其整合为跨查询的通用评分标准，并通过稀疏成对判断对轨迹进行评分，从而在结果奖励均匀时仍能提供过程级梯度。

Details

Motivation: 现有基于LLM的搜索代理主要依赖结果奖励进行训练，当所有采样轨迹结果相同时（结果同质组），奖励信号退化为零梯度。现有过程监督方法要么训练成本高昂的验证器，要么生成单次使用且跨查询不一致的评分标准。

Result: 在四个多跳问答基准测试中，ARBOR consistently优于GRPO和DAPO基线，将LLM-judge准确率平均提升高达4.2个百分点，并将高达42%原本零梯度的训练组转化为有信息量的训练组。

Insight: 创新点在于提出可重用的跨查询评分标准缓冲区机制，通过对比轨迹诱导和评分标准整合实现持续的过程监督，采用稀疏成对判断降低计算成本，有效解决了结果同质组中的梯度消失问题。

Abstract: LLM-based search agents are trained predominantly with outcome-only reward, leaving the search process itself unsupervised. This signal degenerates on outcome-homogeneous groups where all sampled trajectories share the same correctness, yielding zero within-group advantage and no gradient. Existing process supervision either trains a costly verifier or generates per-query rubrics that are inconsistent across queries and discarded after one use. We propose ARBOR (Adaptive Rubric Buffer for Online Reward), a reusable process-reward framework that maintains a rubric memory shared across queries. Query-local drafts induced from contrastive trajectories are admitted, consolidated into cross-query common rubrics, and retired as the policy evolves. A small active subset of common rubrics scores trajectories via sparse pairwise judging, and the resulting scores are added to the base reward, providing process-level gradient even when outcome reward is uniform. ARBOR consistently outperforms GRPO and DAPO baselines on four multi-hop QA benchmarks, raising average LLM-judge accuracy by up to 4.2 points and converting up to 42% of otherwise-zero-gradient training groups into informative ones.

[15] SEA-NLI: Natural Language Inference as a Lens into Southeast Asian Cultural Understanding cs.CLPDF

Peerawat Chomphooyod, Jian Gang Ngui, Yosephine Susanto, Attapol T. Rutherford, Alham Fikri Aji

TL;DR: 本文介绍了SEA-NLI，一个针对东南亚八国、涵盖英语和本土语言的原生文化推理自然语言推理基准测试。研究发现，前沿LLM在西方语境表现良好，但在该基准上表现不佳，失败主要源于缺乏东南亚文化知识。

Details

Motivation: 现有NLI基准大多以西方为中心、由翻译衍生或单语种，无法有效衡量基于文化的推理能力，而前沿LLM在东南亚等代表性不足文化中的表现尚未得到充分测试。

Result: 在17个编码器和解码器模型上的测试结果显示，所有模型性能都较低，尤其在语言、科学技术等知识密集型类别上。文化适配模型和文化感知提示能提升性能，而思维链提示增益有限。

Insight: 创新点在于构建了一个由母语者验证、文化原生的多语言NLI基准，揭示了LLM在非西方文化理解上的显著知识缺口，并证明针对特定文化的模型适配比通用推理技术更有效。

Abstract: Frontier LLMs perform well in Western contexts, but remain poorly tested on underrepresented cultures such as those in Southeast Asia (SEA). Existing NLI benchmarks are largely Western-centric, translation-derived, or monolingual, limiting their ability to measure culturally grounded reasoning. We introduce SEA-NLI, a native, culturally grounded NLI benchmark covering eight SEA countries in English and native regional languages, verified by native speakers. Across 17 encoder and decoder models, we observe a low performance from all models, especially for knowledge-intensive categories such as Languages and Science and Technology. Our analysis shows that failure cases mainly stem from missing SEA cultural knowledge: SEA-adapted models and culture-aware prompting improve performance, while CoT prompting offers limited gains.

[16] SagaQA: A Multi-hop Reasoning Benchmark for Long-form Narrative Understanding in TV Series cs.CL | cs.CVPDF

Galann Pennec, Zhengyuan Liu, Nicholas Asher, Philippe Muller, Nancy F. Chen

TL;DR: 论文提出了SagaQA，一个针对全长电视剧进行多跳推理的长视频基准测试。该基准旨在弥补现有视频推理基准侧重于局部理解的不足，要求模型对跨剧集的长距离、多模态叙事进行高层级理解。论文进一步研究了不同规划策略（并行、顺序和混合）处理此类复杂推理的能力，发现混合规划器能产生更高质量的计划。

Details

Motivation: 现有视频推理基准通常强调对相邻帧或片段的局部理解，缺乏对长篇幅、多模态叙事进行高层级、长距离推理的评估。SagaQA旨在填补这一空白，推动模型对完整电视剧中跨剧集事件和行动的整体理解。

Result: 在SagaQA基准上的实验结果表明，混合规划器（Hybrid planners）能够持续产生更高质量、更连贯完整的推理计划，并在处理电视剧中复杂、高层级的叙事理解任务时展现出更强的能力。

Insight: 论文的核心创新点在于构建了一个需要跨剧集进行长距离多跳推理的长视频基准（SagaQA），其推理步骤的粒度独特。从方法角度看，对规划策略（并行、顺序、混合）的系统性分类与评估，为处理复杂叙事理解任务提供了有价值的见解，表明混合规划策略更具优势。

Abstract: We introduce SagaQA, a long-form video benchmark for multi-hop reasoning over full-length TV series. Existing video reasoning benchmarks often emphasize local understanding of adjacent frames or clips. SagaQA addresses this gap by requiring high-level comprehension of extended multimodal narratives in entire TV shows. A distinguishing feature of SagaQA is the granularity of its reasoning steps. Our dataset necessitates long-range reasoning hops to connect information across completely different episodes. This requires models to reason over entire events and actions, demanding a deep understanding of the show’s narration and progression at a multimodal level. Motivated by recent progress in agentic methods, we further study how different planning strategies handle such complex reasoning. We categorize these approaches into three classes-Parallel, Sequential, and Hybrid planners-and evaluate their ability to generate coherent and complete reasoning plans. Our results on SagaQA suggest that hybrid planners consistently produce higher-quality plans and exhibit stronger capabilities for complex, high-level narrative understanding in TV shows.

[17] From Script to Semantics: Prompting Strategies for African NLI cs.CL | cs.LGPDF

Anuj Tiwari, Terry Oko-odion, Hannah Nwokocha

TL;DR: 本文系统研究了针对斯瓦希里语、约鲁巴语和豪萨语等低资源非洲语言的自然语言推理（NLI）提示策略，使用AfriXNLI基准测试评估了五种策略在Llama3.2-3B和Gemma3-4B模型上的表现，发现对比提示是最可靠且能稳定提升性能的策略。

Details

Motivation: 大型语言模型在多语言环境中的评估日益增多，但其在低资源非洲语言上的推理行为，特别是在无需微调的纯提示场景下，仍未得到充分探索。

Result: 在AfriXNLI基准上，对比提示策略被证明是最可靠且能跨语言和模型稳定提升性能的方法，其类别行为更平衡，整体准确率增益更好；精心构建的提示足以超越使用少样本提示和思维链提示的更强大基线模型。

Insight: 研究强调了提示构建对低资源语言多语言NLI任务至关重要，语言感知的决策结构能有效增强资源受限环境下的模型鲁棒性；同时，通过消除少样本和思维链的影响，成功分离并凸显了提示设计本身的关键作用。

Abstract: Large language models (LLMs) are increasingly evaluated in multilingual settings, yet their inference behavior in low-resource African languages remains underexplored especially under pure prompting without fine-tuning. We present a systematic study of prompting strategies for Natural Language Inference (NLI) in Swahili, Yoruba, and Hausa using the AfriXNLI benchmark. We evaluate five prompting strategies Baseline (zero-shot), Script-Aware, Language Specific, Contrastive, and Native-Label Self-Translation (NL-STP) across two mid-sized open weight models (Llama3.2-3B and Gemma3-4B). To isolate the effect of prompt design, the effect of few-shot examples and Chain-of-Thought reasoning is eliminated in our study. We find a significant difference in performance of class wise across strategies with highly neutral class collapse and high prediction skew in some configurations. Contrastive prompting proves to be the most reliable and steadily improving strategy over language and model and has better balance of class behavior and balance of overall accuracy gains. Notably, well-constructed prompts are sufficient to beat more powerful baselines that are provided with few-shot prompts and Chain-of-Thought prompts. We have found that prompt formulation is essential to multilingual NLI with low-resource languages and that language aware decision structuring can be used to meaningfully enhance robustness in resource challenged settings.

[18] Evaluating LLMs’ Effectiveness on Real-World Consumer Device Repair Questions cs.CL | cs.AIPDF

Atm Mizanur Rahman, Md Arid Hasan, Syed Ishtiaque Ahmed, Sharifa Sultana

TL;DR: 本文评估了大型语言模型在真实世界消费设备维修问题上的有效性，引入了一个包含991个Reddit维修问题的基准测试集，涵盖手机维修、电脑维修和数据恢复，并提供了孟加拉语翻译以评估跨语言性能。研究评估了六种最先进的LLM在英语和孟加拉语中的表现，使用正确性、完整性、实用性和安全性四个维修特定标准。结果显示，虽然LLM能提供有用的维修帮助，但在高风险维修任务中仍不可靠，尤其是在手机维修领域，所有模型在板级诊断、维修优先级和安全恢复程序方面存在显著错误，且孟加拉语响应表现普遍差于英语，其中GPT-5.4整体表现最佳。

Details

Motivation: 消费设备维修是LLM的一个重要但未充分探索的测试领域，维修任务需要在不完整问题描述、硬件特定诊断、可操作故障排除和安全关键决策上进行推理，错误建议可能导致设备损坏、电池危险或永久数据丢失。

Result: 在英语和孟加拉语中评估了六种SOTA LLM，使用维修特定标准（正确性、完整性、实用性、安全性），结果显示GPT-5.4整体表现最佳，但所有模型在手机维修等高风险领域存在显著错误，孟加拉语响应性能一致差于英语。

Insight: 论文创新点在于引入了一个真实世界维修基准测试集，强调跨语言评估（孟加拉语翻译），并提出了维修特定评估标准；客观分析认为，研究突出了LLM在安全关键应用中的局限性，为未来开发更可靠的维修辅助系统提供了方向，特别是在板级诊断和安全程序方面需要改进。

Abstract: Consumer device repair is an important but underexplored testbed for large language models (LLMs). Repair tasks require reasoning over incomplete problem descriptions, hardware-specific diagnostics, actionable troubleshooting, and safety-critical decisions, where incorrect advice can cause device damage, battery hazards, or permanent data loss. We introduce a benchmark of 991 real-world repair questions from Reddit spanning phone repair, computer repair, and data recovery, each paired with technician-written reference solutions, and provide Bangla translations to evaluate cross-lingual performance. We evaluate six state-of-the-art LLMs in English and Bangla using four repair-specific criteria: correctness, completeness, practicality, and safety. Our results show that while LLMs can provide useful repair assistance, they remain unreliable for high-risk real-world repair tasks without rigorous evaluation and explicit safety safeguards. Phone repair is the most difficult and safety-sensitive domain, and all models make substantial errors in board-level diagnosis, repair prioritization, and safe recovery procedures. Across domains and models, Bangla responses consistently perform worse than English responses. Among the evaluated models, GPT-5.4 performs best overall.

[19] Lingo_Research_Group at SemEval-2026 Task 9: Evaluating Prompt Variants for Polarization Detection cs.CL | cs.LGPDF

Pritam Kadasi, Anuj Tiwari, Mayank Singh

TL;DR: 本文介绍了Lingo_Research_Group团队在SemEval-2026 Task 9（多语言文本分类挑战：极化检测）中的提交方案，该方案覆盖了二元极化检测、极化类型分类和极化表现形式识别三个子任务。团队采用了一种系统性的提示词研究方法，设计了12种在术语清晰度、定义细节、推理指导和上下文示例使用上不同的提示，并基于aya-101和Gemma3-27B模型进行实验，最终选择性能更优的Gemma3-27B模型提交。

Details

Motivation: 该研究的动机是探索提示工程在复杂多语言社会语言学分类任务（特别是文本极化检测）中的有效性，旨在通过系统性地设计不同变体的提示词来提升大语言模型在此类任务上的表现。

Result: 在官方测试集（22种语言平均）上，该系统在子任务1、2、3上分别取得了0.762、0.587和0.444的平均宏F1分数，以及0.819、0.678和0.498的平均准确率。实验表明，基于提示的方法能有效检测粗粒度极化，但在细粒度和多标签分类任务上遇到更多困难。

Insight: 论文的创新点在于对提示词设计进行了系统性研究，考察了术语清晰度、定义细节、推理指导和上下文示例等多个维度的影响。从客观角度看，其跨任务和跨语言的分析为提示工程在复杂社会语言学任务中的应用提供了有价值的实证见解，揭示了其在不同粒度任务上的能力边界。

Abstract: Our submission presented in this paper is for SemEval-2026 Task 9: Multilingual Text Classification Challenge - Polarization Detection and it covers all three subtasks: (1) binary polarization detection, (2) polarization type classification and (3) polarization manifestation identification. We adopt a systematic approach of research on short designed prompts by considering twelve designed prompts that are different in terminology clarity, detail of the definition, guidance of reasoning and in-context examples use. The experiments are conducted using aya-101 and Gemma3-27B, with the latter chosen for the submission at the end of the development through performance considerations. Our system has an average macro level F1-score of 0.762 on Subtask 1, 0.587 on Subtask 2 and 0.444 on Subtask 3 with the average accuracy of 0.819, 0.678 and 0.498, respectively, on the official test set averaged among 22 languages, respectively. With cross-task and cross-lingual analysis, we demonstrate that prompt-based approaches can be used effectively to detect coarse grained polarization but encounter more and more difficulties as far as fine-grained and multi-label sociolinguistic classification is concerned.

[20] The Unsampled Truth: Psychometrics in SLMs Measure Prompt Artifacts, Not Psychological Constructs cs.CL | cs.AIPDF

Nils Schwager, Christoph Hau, Simon Münker, Achim Rettinger

TL;DR: 本文通过一个提示词变体框架，评估了13个开放权重的SLM（0.6B到140亿参数）在心理测量评估中的表现。研究发现，提示词中的伪影（如人物设定、指令、选项符号等）产生的方差经常压倒语义信号，导致模型输出主要反映的是对提示的遵从性，而非模拟的心理特质。这限制了SLM在心理测量学中的应用，但该框架为识别有害伪影和分离语义理解提供了诊断工具。

Details

Motivation: 动机在于检验一个前提假设：当提示SLM进行心理测量评估时，研究者的默认假设是模型输出反映了语义推理。本文旨在评估这一前提是否成立。

Result: 在13个不同规模的开放权重模型上，通过系统性地改变提示词元素，发现伪影方差经常主导语义信号，表明模型输出主要受提示词形式影响，而非内在的语义理解。

Insight: 主要创新点在于提出了一个诊断框架，能够分离提示词伪影和语义信号，揭示了当前SLM在心理测量任务中的局限性，并为未来前沿模型研究提供了识别和隔离有害伪影的方法。

Abstract: When prompting SLMs for psychometric assessments, researchers assume the outputs reflect semantic reasoning. We evaluate this premise across 13 open-weights models (0.6B to 14B parameters) using a prompt variation framework that separates semantic signals from prompt artifacts. By systematically varying personas, instructions, items, and option symbols, we find that artifactual variance frequently overpowers the semantic signal. In these cases, models predominantly reflect prompt compliance rather than simulated psychological traits. While these findings limit SLM utility in psychometrics, our framework provides a diagnostic tool to identify destructive artifacts and isolate semantic understanding for future frontier-model research.

Honghui Zhang, Chenmeinian Guo, Yichen Yu, Guanyu Liu, Yongming Qin

TL;DR: 本文提出了See-Infer-Intervene（SII）框架和Proactive Intent World Model（PIWM）模型，旨在构建面向目标的社会智能体，使其能够在零售场景中主动识别顾客意图并选择干预措施。研究构建了GuidanceSalesBench基准测试，并在真实商店试点中验证了模型的有效性。

Details

Motivation: 解决多模态零售智能体在顾客明确请求之前，如何主动识别其行为、推断潜在意图并决定是否及如何提供协助的问题。

Result: 在GuidanceSalesBench基准测试中，当给定真实顾客状态时，PIWM在30个保留测试视频上取得了0.641的宏平均F1分数，优于零样本Qwen2.5-VL-7B基线模型；仅基于视频的端到端选择性能降至0.295。初步真实商店试点在20个完全标注视频上达到了0.579的动作宏平均F1。

Insight: 创新点在于将SII框架与结合AIDA购买阶段和BDI心理场的世界模型（PIWM）相结合，用于建模顾客状态和意图转移，并定义了五类响应动作。研究还揭示了从视频到状态（video-to-state）的落地是部署时的主要瓶颈。

Abstract: Multimodal retail agents should not only recognize what a customer is doing, but also decide whether and how to assist before an explicit request is made. We study this setting through the See–Infer–Intervene (SII) framework, where a device must see pre-interaction behavior, infer latent customer intent, and act by selecting an appropriate service intervention or choosing to wait. We instantiate SII with the Proactive Intent World Model (PIWM), which represents customer state with AIDA (Attention, Interest, Desire, Action) purchasing phases and BDI (belief, desire, intention) psychological fields, predicts action-conditioned intent transitions, and selects from five response classes: Greet, Elicit, Inform, Recommend, and Hold. We further construct GuidanceSalesBench, a smart-retail benchmark containing state manifests, pre-interaction videos, candidate responses, action-conditioned outcomes, and best-action labels. When conditioned on ground-truth customer state to isolate action selection, PIWM achieves 0.641 macro F1 on 30 held-out target videos, outperforming a zero-shot Qwen2.5-VL-7B baseline and training variants without balanced action supervision; end-to-end video-only selection drops to 0.295, below the 5-class balanced random baseline of 0.414, identifying video-to-state grounding as the dominant deployment-time bottleneck. A preliminary staged real-store pilot (recorded with paid participants performing scripted customer behaviors) reaches 0.579 action macro F1 on 20 fully annotated videos, with 10 additional accessible videos released with index-level labels.

[22] Beyond the Literal: Decomposing Pragmatic Intent in Multimodal Meme Understanding cs.CLPDF

Zhengyi Zhao, Shubo Zhang, Zezhong Wang, Luyao Ye, Huimin Wang

TL;DR: 论文提出Intent Projection框架，将多模态模因理解重构为字面意义与语用意图的分解问题。该框架在单一大型视觉语言模型（LVLM）内部，通过表征、输出和目标三个层面分离这两种信号，以解决现有模型倾向于描述图像表面内容而非作者真实沟通意图的问题。

Details

Motivation: 现有大型视觉语言模型在理解模因或讽刺性帖子时，往往只描述图像的字面内容，而忽略了作者试图传达的语用意图。标准的指令微调将帖子的字面内容与其语用含义纠缠在一起，导致表面细节污染了最终响应。

Result: 在六个多模态基准测试中，Intent Projection框架持续优于开源基线模型，并缩小了与专有模型的差距，在字面意义与语用意图差异较大的帖子上取得了最显著的性能提升。

Insight: 创新点在于将模因理解形式化为字面-语用分解任务，并提出了一个在表征层通过正交投影模块去除主导的单模态信号、在输出层外化结构化推理链、在目标层使用对比奖励明确惩罚重复字面描述的端到端框架。这为解决多模态理解中的意图鸿沟问题提供了新思路。

Abstract: When asked what a meme or sarcastic post means, Large Vision Language Models (LVLMs) tend to describe what the image shows rather than what the author is trying to communicate. Standard instruction tuning entangles a post’s literal content with its pragmatic meaning, letting surface-level details contaminate the final response. We reframe meme understanding as a problem of literal-pragmatic decomposition and propose \textbf{Intent Projection}, a framework that separates the two signals at the representation, output, and objective levels within a single LVLM backbone. At the representation level, an orthogonal projection module removes dominant unimodal directions from the fused image-text representation, retaining only the pragmatic residual, while a surface-real affect classifier anchors the decoder with a discrete tag that names the polarity gap. At the output level, the model externalizes a structured reasoning chain, and at the objective level a contrastive reward explicitly penalizes answers that restate the literal description. Across six multimodal benchmarks, Intent Projection consistently outperforms open-source baselines and narrows the gap to proprietary models, with the largest gains on high-divergence posts where literal collapse is most damaging.

[23] Does Language Shift Break Medical Vision-Language Models? Indonesian Radiology Visual Question Answering Case Study cs.CL | cs.CVPDF

Pieter Christy Yan Yudhistira, Dzaki Rafif Malik, Novanto Yudistira

TL;DR: 本文通过构建印尼语放射学视觉问答数据集IndoRad-VQA，评估了医学视觉语言模型在非英语临床语言环境下的鲁棒性。研究发现，模型在英语基准上的优异表现并不能直接迁移到印尼语场景，存在显著的性能差距。

Details

Motivation: 当前医学视觉语言模型主要在英语放射学视觉问答基准上进行评估，其在非英语临床语言（如印尼语）下的推理能力尚未得到充分探索。

Result: 在IndoRad-VQA数据集上，不同模型在英语和印尼语提示设置下的性能存在8%到25%的差距，具体取决于评估指标。

Insight: 研究揭示了医学多模态基础模型在语言迁移时存在的鲁棒性问题，强调了进行更具包容性的多语言评估的必要性。其构建高质量翻译数据集的方法（基于自评估的质量控制）也值得借鉴。

Abstract: Medical Vision-Language Models (VLMs) are typically evaluated on English radiology visual question answering benchmarks, leaving their robustness under non-English clinical language largely unexplored. We introduce IndoRad-VQA, an Indonesian adaptation of VQA-RAD, to assess whether medical VLMs retain radiology reasoning ability when questions are asked in Bahasa Indonesia. Radiology question-answer pairs are translated into Indonesian with self-evaluation-based quality control to preserve clinical meaning, terminology consistency, and answer equivalence. We evaluate general-purpose, Southeast Asian multilingual, and medical-specific VLMs under English and Indonesian prompting settings. Beyond accuracy, we quantify the language robustness gap between English and Indonesian inputs. We also conduct an error analysis to identify failure modes of question answering, such as yes/no flips, laterality errors, and output-language mismatches. Our findings show that strong performance on English medical VQA benchmarks does not necessarily translate to robust behavior in Indonesian clinical contexts. We observe a performance gap of 8 to 25 percent between the English and Indonesian settings, depending on the evaluation metric. These results highlight the need for more inclusive multilingual evaluation of medical multimodal foundation models. The dataset is available at https://huggingface.co/datasets/Lab-IS/IndoRad-VQA.

[24] HybridThinker: Efficient Chain-of-Thought Reasoning via Compressed Memory and Transient Thought Steps cs.CLPDF

Xin Liu, Runsong Zhao, Xinyu Liu, Junhao Ruan, Pengcheng Huang

TL;DR: 本文提出了HybridThinker方法，旨在解决思维链推理中扩展轨迹带来的高计算和内存成本问题。该方法通过压缩记忆和临时保留思维步骤，在保持推理效率的同时，减少了因信息丢失导致的错误。

Details

Motivation: 现有思维链压缩方法通过记忆令牌将思维步骤压缩为紧凑表示，但在推理时仅保留这些表示，导致细粒度信息丢失，使得后续步骤更容易出错。

Result: 在四个推理基准测试上，HybridThinker的性能与未压缩基线相当，在推理时间相似的情况下，将思维链压缩的平均准确率提升了5.8个百分点，达到了该领域的先进水平。

Insight: 创新点在于同时保留压缩记忆表示和临时思维步骤以提供细粒度细节，并引入混合训练方案，强制模型通过记忆令牌进行压缩和检索，从而有效平衡信息保留与计算效率。

Abstract: Extended chain-of-thought (CoT) traces improve LLM reasoning but incur substantial computational and memory costs. While existing CoT compression methods mitigate this by condensing thought steps into compact representations via memory tokens and retaining only these representations at inference time, the loss of fine-grained information makes subsequent steps more error-prone. To alleviate this, we propose \textbf{HybridThinker}, where in addition to preserved these representations, thought steps are also temporarily retained to provide fine-grained details. However, we observe that naively keeping thought steps accessible to subsequent steps \emph{during training} lets the model bypass memory tokens by retrieving information directly from these steps, leaving the model’s ability to compress and retrieve information through memory tokens insufficiently trained. We therefore introduce a hybrid training scheme, in which only some thought steps are directly accessible through attention to subsequent steps, while the other thought steps are masked, forcing the model to use memory tokens for compression and retrieval. Across 4 reasoning benchmarks, HybridThinker matches the uncompressed baseline, advancing the state of the art in CoT compression by 5.8 points on average accuracy with similar inference time. Ablation studies confirm that both temporary thought-step retention and the hybrid training scheme contribute to these gains.

[25] Framing Migration News with LLMs: Structured CoT as a Support for Human Interpretation cs.CLPDF

David Alonso del Barrio, Jing Wen, Daniel Gatica-Perez

TL;DR: 本文研究如何利用本地可部署的开源大语言模型（Llama3-8B）作为辅助工具，支持对移民新闻进行可解释的框架分析。作者提出了一种结构化思维链提示方法，通过分步推理将模型输出与预定义的框架类别关联，以提高分类性能并支持人工审核。研究通过自动评估和人本评估验证了该方法的有效性，并探讨了LLM辅助框架分析的潜力与风险。

Details

Motivation: 移民新闻的框架分析是一项具有社会影响的任务，现有基于LLM的方法依赖专有API和大模型，存在数据隐私、可复现性和公平访问等问题。本研究旨在探索一种可在本地部署的开源LLM，作为透明、可审计且资源友好的辅助工具，以支持媒体研究者进行可解释的框架分析。

Result: 在移民相关新闻数据集上的评估表明，结构化思维链方法在零样本和少样本基线基础上提升了分类性能，且可在单GPU上运行。人本评估显示，模型推理的连贯性平均得分为4.1/5，并能促使用户反思初始解读，尽管不同文本间存在显著差异。

Insight: 创新点在于提出了结构化思维链提示方法，将模型的逐步推理与预定义分类框架绑定，增强了输出的可追溯性，并支持人机协同的批判性解读。从客观角度看，该工作强调了在资源受限的学术环境中开发负责任、可访问的计算工具的重要性，并通过本地部署和人在回路的交互设计，为分析社会影响力大的媒体叙事提供了新思路。

Abstract: Frame analysis of migration news is a socially consequential task: media scholars and researchers who study how migration is narrated need tools that are not only accurate, but transparent, auditable, and accessible within the resource constraints typical of academic research groups. Existing LLM-based approaches rely on proprietary APIs and large models that raise concerns about data privacy, reproducibility and equitable access among media researchers. This work studies how a locally deployable open-source LLM can support interpretable frame analysis as an assistive tool. We introduce a Structured Chain-of-Thought (SCoT) prompting approach using Llama3-8B, enabling step-by-step justifications grounded in predefined framing categories. This structured design allows users to audit model outputs and examine alternative interpretations in a task that is inherently subjective. We evaluate our approach on a dataset of migration-related news and show that SCoT improves classification performance over zero-shot and few-shot baselines while remaining feasible on a single GPU. Then, we conduct a human-centered evaluation in which annotators assess the coherence and influence of “the model’s reasoning”. Results indicate that SCoT explanations are generally perceived as logical (mean score 4.1/5, though with notable variation across texts) and can prompt reflection on initial interpretations, even when disagreement persists. Our findings highlight both the potential and risks of LLM-assisted frame analysis. While structured reasoning can increase the traceability of model outputs and support critical interpretation, it can also influence human judgment in subtle ways. By enabling local deployment and emphasizing human-in-the-loop interaction, this work contributes to discussions on responsible and accessible computational tools for the study of socially impactful media narratives.

[26] Reasoning over Grammar: Can Synthetic Linguistic Reasoning Traces Enhance Low-Resource Machine Translation? cs.CLPDF

Renhao Pei, Yihong Liu, Sampo Pyysalo, Hinrich Schütze, Shaoxiong Ji

TL;DR: 本文研究如何利用结构化语言推理轨迹提升低资源机器翻译性能，提出从通用依存树库、词典和语法规则库自动生成分步语言推理轨迹的流程，并在Xibe和Chintang语言上评估了上下文学习、监督微调和强化微调三种设置的效果。

Details

Motivation: 针对大语言模型在低资源机器翻译中难以有效利用语法信息的问题，受思维链推理启发，探索通过引入语言分析和语法推理的中间步骤来提升翻译性能。

Result: 在上下文学习设置中，可靠的句子特定推理轨迹显著提升了大多数模型、语言和指标上的翻译性能；而将推理轨迹用作训练数据则收益较小且不稳定，模型虽学会轨迹格式但常生成错误内容。

Insight: 创新点在于自动生成结构化语言推理轨迹的流程，并系统评估其在不同微调范式中的作用；核心发现是LLMs在推理时给予可靠语言分析能有效利用语法信息，但学习生成此类分析仍是主要瓶颈。

Abstract: Large language models (LLMs) offer a promising approach to machine translation (MT) for extremely low-resource languages by incorporating linguistic resources through in-context learning. However, LLMs often struggle to apply grammatical information effectively during translation. Inspired by recent progress in chain-of-thought reasoning, we investigate whether low-resource MT can benefit from structured intermediate steps of linguistic analysis and grammatical reasoning. We propose a pipeline for automatically generating step-by-step linguistic reasoning traces from Universal Dependencies treebanks, dictionaries, and grammar-rule banks. We evaluate these traces in three settings: in-context learning (ICL), supervised fine-tuning (SFT), and reinforcement fine-tuning (RFT), on Xibe and Chintang as test cases. Our results show that linguistic reasoning traces are most effective as inference-time guidance: in ICL, reliable sentence-specific traces substantially improve translation performance across most models, languages, and metrics. In contrast, using the linguistic reasoning traces as training data yields smaller and less consistent gains, as models learn the trace format but often generate erroneous content. These findings suggest that LLMs can leverage grammatical information for low-resource MT when given reliable linguistic analyses, while learning to generate such analyses remains a major bottleneck.

Hashmat Shadab Malik, Muzammal Naseer, Salman Khan

TL;DR: 本文系统研究了多语言多模态大语言模型（MLLMs）的对抗鲁棒性和安全性对齐问题，重点关注12种不同语言。研究发现，基于梯度的对抗攻击具有跨语言可迁移性，且模型在多语言安全行为上存在显著差异，低资源语言的安全性可能源于模型的理解或视觉基础能力不足（即‘因失败而安全’），而非真正的安全对齐。

Details

Motivation: 现有关于MLLM鲁棒性的研究主要集中于英语任务，多语言环境下的对抗鲁棒性和多模态安全性尚未得到充分探索，本文旨在填补这一空白。

Result: 实验表明，对抗图像在不同语言间具有强可迁移性。在安全性方面，当有害指令通过文本发出时，语言基础更强的语言更容易引发不当响应；当有害内容以排版文本形式嵌入图像时，英语脚本能被可靠识别并遵循，而非英语脚本则很少被视觉编码器解析。相比之下，像Qwen3-VL这样在多个训练阶段构建多语言能力的模型，展现出真正的跨语言安全性。

Insight: 论文的核心创新在于揭示了多语言MLLMs中‘因失败而安全’的现象，即低资源语言表面上的安全性可能源于模型的理解或视觉基础能力缺陷。研究强调，仅通过指令微调进行浅层多语言适配可能导致虚假的安全表象，而在整个训练阶段进行深度整合才能实现真正的多语言安全对齐。

Abstract: Multimodal Large Language Models integrate visual perception into language reasoning, introducing a continuous attack surface susceptible to adversarial attacks. Prior work on MLLM robustness has focused largely on English-centric tasks, leaving multilingual behaviour unexplored. We address this gap through a systematic study of adversarial robustness and multimodal safety across 12 diverse languages, evaluating open-source MLLMs that acquire multilingual capability through instruction tuning. Gradient-based attacks reveal a transferable multilingual vulnerability: adversarial images optimized in one language continue to induce failure in others, demonstrating strong cross-lingual transferability. Multilingual safety further varies with how effectively a model retrieves or interprets harmful instructions. When harmful intent is issued through text, languages with stronger linguistic grounding more often elicit misuse-enabling responses, while weaker languages produce fewer unsafe outputs. When embedded in the image as typographic content, English scripts are reliably recognised and followed, whereas non-English scripts are rarely parsed by the vision encoder. Lower-resource languages may therefore appear safer, but this is an artefact of comprehension and visual-grounding failures rather than genuine alignment, a phenomenon we term safety-by-failure. In contrast, MLLMs that build multilingual capability throughout their training stages rather than only at instruction tuning, such as Qwen3-VL, exhibit genuine cross-lingual safety, maintaining active refusal across languages rather than masking comprehension failure. Shallow multilingual adaptation, such as fine-tuning on translated instruction data, may produce surface-level understanding that creates illusory safety in low-resource languages; deeper integration across training stages leads to genuine multilingual safety alignment.

[28] Agentic Chain-of-Thought Steering for Efficient and Controllable LLM Reasoning cs.CL | cs.AIPDF

Yu Xia, Zhouhang Xie, Xin Xu, Byungkyu Kang, Prarit Lamba

TL;DR: 本文提出了一种名为Agentic Chain-of-Thought Steering（ACTS）的方法，旨在高效且可控地引导大型语言模型（LLM）的思维链推理。该方法将推理引导建模为马尔可夫决策过程，由一个控制器智能体在推理过程中自适应地引导一个冻结的推理器，通过观察推理轨迹和剩余思考预算来发出包含推理策略和引导短语的指令，从而实现预算感知的策略控制。

Details

Motivation: 现有思维链推理方法虽然能提高最终答案的准确性，但通常存在令牌使用效率低下和推理过程缺乏实时控制的问题。现有高效推理方法主要通过缩短、提前停止或压缩推理轨迹来控制思考长度，但未显式控制模型的思考方式。

Result: 在多个基准测试上的实验表明，ACTS方法在实现与完整思考链相当性能的同时，能显著节省令牌使用量，并且能够在不同推理器和任务上实现可控的准确性与效率权衡。

Insight: 创新点在于将推理引导形式化为一个由智能体控制的决策过程，实现了对推理策略和预算的显式、自适应控制。该方法通过合成轨迹初始化和基于预算条件的强化学习奖励塑形来优化控制器，在保持推理器生成连续性的同时，提供了灵活的准确率-效率权衡机制。

Abstract: Large language models improve final-answer accuracy through extended chain-of-thought reasoning, but often spend tokens inefficiently and offer little inference-time control. Existing efficient reasoning methods control thinking length by shortening, early-stopping, or compressing traces, leaving how the model thinks implicit. In this paper, we propose Agentic Chain-of-Thought Steering (ACTS), which formulates reasoning steering as a Markov decision process where a controller agent adaptively steers a frozen reasoner during inference. At each step, the controller observes the reasoning trace and remaining thinking budget, then issues a steering action consisting of a reasoning strategy and a steering phrase that initiates the next reasoner step. This enables budget-aware strategy control for efficient reasoning while preserving the reasoner’s generation continuity. We initialize the controller agent from our constructed synthetic steering trajectories with multi-budget augmentation, and further optimize it via reinforcement learning with budget-conditioned reward shaping. Experiments across multiple benchmarks show that ACTS matches full-thinking performance with substantial token savings, and enables controllable accuracy-efficiency trade-offs across different reasoners and tasks. The code is available at https://github.com/Andree-9/ACTS.

[29] Synthesize and Reward – Reinforcement Learning for Multi-Step Tool Use in Live Environments cs.CL | cs.AI | cs.LGPDF

Ibrahim Abdelaziz, Asim Munawar, Kinjal Basu, Maxwell Crouse, Chulaka Gunasekara

TL;DR: 本文提出了PROVE框架，用于解决训练LLM进行多步工具调用的三大障碍：真实有状态执行环境构建成本高、合成训练查询与服务器实际状态脱节、以及基于召回率的RL奖励鼓励冗长的工具调用模式。PROVE包含一个包含20个有状态MCP服务器（提供343个工具）的库、一个基于实时服务器状态生成已验证多轮工具调用轨迹的自动化数据合成管道，以及一个无需外部评判模型的多组件程序化奖励函数。

Details

Motivation: 动机是解决训练LLM编排多步工具调用时面临的三个相互关联的障碍：构建真实有状态执行环境成本高昂、合成训练数据与服务器实际状态脱节导致工具调用失败，以及基于召回的强化学习奖励会鼓励冗长而非高效的调用模式。

Result: 在Qwen3-4B、Qwen3-8B、Qwen2.5-7B和Granite-4.1-8B四个模型上使用GRPO和约13K训练样本进行训练，在BFCL Multi-Turn、tau2-bench和T-Eval三个基准测试上，PROVE框架分别带来了最高+10.2、+6.8和+6.5分的提升，表明其紧凑的程序化奖励能在两个模型家族上为多步工具编排带来一致的性能增益。

Insight: 创新点在于提出了一个集成了已验证执行环境、状态感知的数据合成以及多组件程序化奖励的完整框架。其核心洞察是：通过依赖图引导的、基于实时服务器状态的对话模拟来生成数据，并结合一个精心设计的、无需外部模型的程序化奖励（包含有效性评分、依赖感知覆盖率、自适应效率惩罚等），可以有效解决多步工具调用训练中的数据-状态脱节和奖励信号问题，从而提升模型性能。

Abstract: Training LLMs to orchestrate multi-step tool calls is held back by three coupled obstacles: realistic stateful execution environments are costly to build, synthetic training queries are often detached from the server’s actual state (so the generated tool calls fail to execute), and recall-based RL rewards incentivize verbose tool-calling patterns. We present PROVE (Programmatic Rewards On Verified Environments), a framework with three contributions: (1) a library of 20 stateful MCP (Model Context Protocol) servers exposing 343 tools, enabling live-execution RL training with session-scoped state isolation; (2) an automated data synthesis pipeline that generates validated multi-turn tool-call trajectories against these servers via dependency-graph-guided conversation simulation grounded in live-sampled server state, so every generated query references entities that actually exist; and (3) a multi-component programmatic reward - graduated validity scoring, dependency-aware coverage, an adaptive efficiency penalty with a complexity-scaled call budget, a tool-name signal, and an argument-value matching bonus - requiring no external judge model. We train four models (Qwen3-4B, Qwen3-8B, Qwen2.5-7B, Granite-4.1-8B) with GRPO using identical reward hyperparameters and ~13K training examples; only learning rate is tuned per model family from a three-point sweep. On BFCL Multi-Turn, tau2-bench, and T-Eval, PROVE yields improvements of up to +10.2, +6.8, and +6.5 points respectively, demonstrating that a compact programmatic reward yields consistent gains on multi-step tool orchestration across two model families.

[30] Quantifying Faithful Confidence Expression in Large Reasoning Models cs.CL | cs.AIPDF

Areeb Gani, Asal Meskin, Gabrielle Kaili-May Liu, Arman Cohan

TL;DR: 本文提出了一种量化大型推理模型忠实置信度表达的新框架，通过分析语言决断性与内部不确定性的三个来源（词元概率、隐藏状态和采样响应一致性）来评估模型内在置信度与语言表达置信度之间的对齐程度。研究发现，忠实置信度表达是大型推理模型面临的一个显著挑战，推理行为并不能自动转化为改进的忠实校准，且不同置信度估计器对同一推理轨迹的评估结果存在分歧。

Details

Motivation: 解决大型推理模型在忠实置信度表达方面的挑战，即模型内在置信度与语言表达置信度之间的对齐问题，这对于模型在关键任务中的可信度至关重要。

Result: 应用该框架对多种领先模型、数据集和提示进行评估，发现忠实置信度表达是大型推理模型的一个显著失败模式，且先前评估方法存在脆弱性。

Insight: 创新点在于提出了一个系统量化大型推理模型忠实置信度的框架，并设计了前缀条件采样方法来控制推理轨迹间的条件和结构变异，从而更准确地评估模型的不确定性表达。

Abstract: Reliable uncertainty communication is critical to the trustworthiness of LLMs, yet faithful calibration (FC)–the alignment between models’ intrinsic and (linguistically) expressed confidence–is a persistent failure mode. This challenge is key for large reasoning models (LRMs), whose extended reasoning traces are often interpreted by users as evidence of deliberation, competence, and confidence. Despite the importance of FC and wide usage of LRMs, the extent to which LRMs can faithfully express their confidence remains poorly understood. Moreover, the prevailing paradigm to measure FC does not generalize well to the long chain-of-thought outputs generated by LRMs, which tend to lack clear step boundaries, involve inconsistent step structure, and encode complex conditional dependencies throughout the trace–complicating estimation of intrinsic confidence. To address this challenge, we introduce a novel framework to systematically quantify FC of LRMs. Our framework analyzes linguistic decisiveness relative to three sources of internal uncertainty, based on token probabilities, hidden states, and sampled response consistency. We also devise a prefix-conditioned sampling approach to control for conditional and structural variation across traces. Applying our framework to a diverse suite of leading models, datasets, and prompts, we find that faithful confidence expression is a significant challenge for LRMs. Reasoning behaviors do not automatically translate to improved FC, and prompt interventions for non-reasoning models do not improve faithfulness in the reasoning setting. Different confidence estimators further produce divergent assessments of the same traces, revealing fragility in prior evaluation methodologies. Taken together, our work establishes FC as a distinct reliability and alignment target for LRMs, particularly as such systems are increasingly deployed in high-stakes contexts.

[31] QUBRIC: Co-Designing Queries and Rubrics for RL Beyond Verifiable Rewards cs.CL | cs.AIPDF

Rongzhi Zhang, Rui Feng, Zhihan Zhang, Jingfeng Yang, Qingyu Yin

TL;DR: 论文提出了QUBRIC框架，通过协同设计查询（queries）和评分标准（rubrics）来解决基于评分标准的强化学习（rubric-based RL）中查询结构限制评分标准质量的问题。该方法将开放式查询重写为基于场景的可评估问题，并利用对比性评分标准生成和可学习性过滤来优化查询-评分标准对，从而在无需严格可验证奖励的任务上提升模型性能。

Details

Motivation: 现有基于评分标准的强化学习方法在优化评分标准时，将查询分布视为固定，导致开放式查询产生模糊的评分标准，而简单收窄查询又会引入无法验证的虚构参考，使得训练缺乏奖励信号。论文旨在突破这一结构性瓶颈，通过协同设计查询和评分标准来提升强化学习在超越可验证奖励任务中的实用性。

Result: 在ArenaHard基准上，QUBRIC相比监督微调（SFT）基线取得了+5.5分的提升。仅使用指令遵循数据训练后，该方法进一步迁移到三个涵盖法律、道德和叙事推理的保留基准上，平均提升+6.3分，且改进主要集中在推理相关维度。

Insight: 创新点在于提出查询与评分标准的协同设计框架，包括基于教师模型关键点重写查询、利用教师-策略差距生成对比性评分标准，以及通过可学习性过滤筛选信息丰富的查询-评分标准对。这为超越严格可验证任务的强化学习提供了一种实用补充途径。

Abstract: Rubric-based RL is a promising route for extending reinforcement learning beyond verifiable rewards, yet existing methods optimize rubrics while treating the query distribution as fixed. We identify a structural bottleneck: rubric quality is constrained by query structure. Open-ended queries yield vague rubrics; naively narrowing them introduces fabricated references that no model can verify, so all responses fail and training receives no reward signal. We present QUBRIC, a framework that co-designs queries and rubrics. Teacher-derived key points ground the rewriting of open-ended queries into scenario-based, evaluable questions. Contrastive rubric generation then turns teacher-policy gaps into query-level criteria, and learnability filtering retains only informative query-rubric pairs for GRPO training. QUBRIC achieves a +5.5 point gain on ArenaHard over the SFT baseline. Trained only on instruction-following data, it further transfers to three held-out benchmarks spanning legal, moral, and narrative reasoning (+6.3 points on average), with improvements concentrated in reasoning-related dimensions. These results provide evidence that co-designing queries and rubrics can make rubric-based RL a practical complement to RLVR beyond strictly verifiable tasks.

cs.CV [Back]

[32] AVTrack: Audio-Visual Tracking in Human-centric Complex Scenes cs.CV | cs.AIPDF

Yaoting Wang, Yun Zhou, Zipei Zhang, Henghui Ding

TL;DR: 本文提出了AVTrack，一个面向动态真实场景的以人为中心的视听实例分割（AVIS）数据集，旨在解决现有数据集在复杂、动态场景下评估视听说话人跟踪能力的不足。该数据集包含相机运动、视觉遮挡和位置变化等多样且具有挑战性的条件，并提供了一个简单有效的基线方法。

Details

Motivation: 现有视听说话人跟踪数据集大多局限于简单或同质的场景，且标注粗糙，导致评估偏向于静态的视听共现，无法严格评估在复杂动态场景下的鲁棒时空建模和跨模态推理能力。

Result: 在AVTrack数据集上评估现有代表性AVIS方法时，发现其性能显著下降，这确立了AVTrack作为复杂环境下鲁棒的以人为中心视听场景理解的一个具有挑战性的基准。

Insight: 主要创新点是构建了一个专注于复杂动态真实场景的、以人为中心的视听实例分割数据集，弥补了现有数据集的不足，并为该领域提供了一个更严格的评估基准和基线方法。

Abstract: Audio-visual speaker tracking aims to localize and track active speakers by leveraging auditory and visual cues, enabling fine-grained, human-centric scene understanding. This capability is essential for real-world applications such as intelligent video editing, surveillance, and human-computer interaction. However, existing datasets are largely limited to simple or homogeneous audio-visual scenes with coarse annotations. Such oversimplified settings bias evaluation toward static audio-visual co-occurrence, rather than rigorously assessing robust spatiotemporal modeling and cross-modal reasoning in complex, dynamic scenes. To address these limitations, we introduce AVTrack, a human-centric audio-visual instance segmentation (AVIS) dataset designed for dynamic real-world scenarios. AVTrack features diverse and challenging conditions, including camera motion, visual occlusions, and position changes. Evaluations of representative AVIS methods on AVTrack reveal substantial performance degradation, establishing AVTrack as a challenging benchmark for robust human-centric audio-visual scene understanding in complex environments. We further provide a simple yet effective baseline to facilitate future research. Project website: https://FudanCVL.github.io/AVTrack/

[33] Plan2Map: A Multimodal Benchmark for Document-Grounded Geospatial Boundary Reconstruction from Planning Records cs.CV | cs.AIPDF

Fabian Degen, Oishi Deb, Jindong Gu, Junchi Yu, Samuele Marro

TL;DR: 论文提出了Plan2Map基准测试集和GeoPlanAgent系统，用于解决从规划记录文档中重建地理空间边界的问题。Plan2Map是一个包含208个案例的多模态基准，要求系统仅根据源规划文档（文本、表格、地图图版等）重建出有效的GeoJSON边界。GeoPlanAgent是一个基于文档、利用地理空间工具循环的系统，通过任务分解显著提升了性能。

Details

Motivation: 规划记录定义了地理区域的限制，但其源文档通常只提供间接的空间证据，而非机器可读的边界，这给自动化处理带来了挑战。

Result: 在Plan2Map基准上，GeoPlanAgent取得了0.736的平均IoU和0.904的中位数IoU，其中67.8%的预测IoU达到或超过0.8，大幅优于直接的视觉语言模型（VLM）到GeoJSON的基线方法。

Insight: 创新点在于构建了一个具体的多模态地理空间重建基准，并提出了一个将复杂任务分解为证据提取、定位、地图配准、边界分割、投影和验证等多个子步骤的系统框架。诊断分析表明，直接的VLM预测不可靠，而监督式的边界分割能显著提升掩码质量，这为类似任务提供了可借鉴的工程化思路。

Abstract: Planning records define restrictions over geographic areas, but their source documents often provide only indirect spatial evidence rather than machine-readable boundaries. We introduce Plan2Map, a 208-case multimodal benchmark for document-grounded geospatial boundary reconstruction from UK planning records. Given only a source planning document, systems must reconstruct a valid geospatial boundary from notice text, schedules, map plates, map labels, and boundary annotations; the reference GeoJSON is held out for scoring. We propose GeoPlanAgent, a document-grounded, geospatial-tool-in-the-loop system that decomposes the task into evidence extraction, localisation, map registration, boundary segmentation, projection, and verification. On Plan2Map, GeoPlanAgent achieves 0.736 mean IoU and 0.904 median IoU, with 67.8% of predictions at or above 0.8 IoU, substantially outperforming direct VLM-to-GeoJSON baselines. Diagnostic analysis shows that direct VLM prediction remains unreliable, while remaining errors are concentrated in localisation and map registration, and supervised boundary segmentation substantially improves pixel-level mask quality. Plan2Map provides a concrete testbed for multimodal geospatial reconstruction from public planning records. Project page: https://odeb1.github.io/Plan2Map_Project_Page/.

[34] Consistent Yet Wrong: Evidence Insensitivity in Spatial Vision-Language Models cs.CVPDF

S Divakar Bhat, Toshihiko Yamasaki

TL;DR: 这篇论文挑战了视觉语言模型（VLMs）中跨视图一致性作为几何理解代理的常见假设。作者发现，领先的VLMs在度量距离查询上经常产生视图不变且一致的答案，即使这些答案是错误的，表明预测与特定视图的视觉证据之间耦合较弱。为此，他们引入了ViewDiag，一个从Hypersim、ScanNet和KITTI360构建的受控多视图评估协议，用于评估模型在度量准确性、分布集中性和内部崩溃的潜在特征探测三个轴上的表现。

Details

Motivation: 空间推理对机器人学、自主系统和具身AI至关重要，但现代视觉语言模型在度量距离查询上仍不可靠。论文旨在测试一个常见假设，即跨视图的一致性预测反映了几何基础，结果发现该假设不成立。

Result: 在ViewDiag基准测试中，多种模型表现出高预测稳定性与显著错误并存的模式，聚类在强一致性但低准确性的区域。这挑战了使用跨视图一致性作为几何理解代理的做法。

Insight: 论文的创新点在于揭示了VLMs中稳定预测可能反映的是先验驱动的崩溃，而非证据敏感的推理。ViewDiag提供了一个超越准确性的受控基准和诊断框架，用于评估空间VLMs，有助于区分决策崩溃和表示崩溃。

Abstract: Spatial reasoning is fundamental to robotics, autonomy, and embodied AI, yet modern vision-language models (VLMs) remain unreliable on metric distance queries. A common assumption is that consistent predictions across viewpoints reflect geometric grounding. We test this assumption and find the opposite: leading VLMs often produce view-invariant and consistent answers even when those answers are incorrect, indicating weak coupling between predictions and viewpoint-specific visual evidence. We introduce \textbf{ViewDiag}, a controlled multi-view evaluation protocol built from Hypersim, ScanNet, and KITTI360, comprising 176 object-pair tracks across 80 scenes with 2–10 views per track. The protocol evaluates models along three axes: metric accuracy, distributional concentration, and a latent feature probe for internal collapse that distinguishes decision collapse from representation collapse. Across diverse models, we observe a consistent pattern of high prediction stability paired with substantial error, clustering in a regime characterized by strong consistency but low accuracy. \noindent These results challenge the common use of cross-view consistency as a proxy for geometric understanding. Instead, we show that stable predictions may reflect prior-driven collapse rather than evidence-sensitive reasoning. ViewDiag provides a controlled benchmark and diagnostic framework for evaluating spatial VLMs beyond accuracy alone. The code and data can be found \href{https://github.com/SDivakarBhat/Consistent_Yet_Wrong.git}{here}

[35] MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Data cs.CV | cs.AIPDF

Teng Hu, Mingchun Lu, Yating Wang, Jiangning Zhang, Jinkun Hao

TL;DR: MetaWorld提出了一种从单视角视频数据直接扩展到多智能体视频世界模型的新框架，解决了多智能体场景下数据稀缺和世界状态对齐的挑战。该方法通过单目世界状态展开分解相机运动和主体轨迹，利用主体感知世界生成器进行外观驱动模拟，并通过世界状态对齐机制确保多视角间静态几何和动态运动的一致性。

Details

Motivation: 现有视频世界模型局限于单智能体单视角，难以扩展到多智能体场景，主要面临多视角协调数据收集成本高昂以及独立生成视频流无法保证共享物理环境一致性演化的挑战。

Result: 大量实验表明，MetaWorld在跨视角一致性和身份保真度方面表现优异，为多智能体视频世界建模建立了一个高度可扩展、物理驱动的范式。

Insight: 创新点包括：1）通过相机-轨迹分解从单目视频中提取同步的多智能体运动数据，无需多相机设置；2）引入主体感知世界生成器实现基于身份图像的外观驱动模拟；3）提出世界状态对齐机制，在视频DiT的每一层插入跨分支交叉注意力，联合同步去噪过程以强制几何和运动一致性。

Abstract: Video world models are a foundational generative technology for embodied AI and the Metaverse, yet existing approaches are inherently limited to a single agent observing from a single perspective. Extending these models to multi-agent settings introduces two critical challenges: data scarcity (coordinated multi-view recordings are prohibitively expensive to collect for general open-domain scenarios) and world state alignment (independently generated video streams cannot ensure that shared physical environments and events evolve consistently across views). To address these challenges, we propose MetaWorld, a novel framework that scales multi-agent video world models to open-domain environments directly from single-view videos. First, we introduce Monocular World-State Unrolling (MWSU) to explicitly decompose monocular footage into the camera operator’s ego-motion and the visible subject’s spatial trajectory. This camera-trajectory decomposition naturally extracts synchronized multi-agent motion data within a shared 3D space, completely bypassing the need for multi-camera setups. Second, for precise visual control, we develop the Subject-Aware World Generator to enable appearance-driven simulation conditioned on per-agent identity images. Finally, to ensure both views are grounded in the identical physical reality, we propose World-State Alignment, a per-frame inter-branch cross-attention mechanism inserted at every transformer layer of the video DiT. By jointly synchronizing the denoising process, WSA enforces both static geometric consistency and dynamic motion consistency, encouraging that the shared 3D environment and physical events remain well-aligned across both egocentric views. Extensive experiments demonstrate that MetaWorld achieves superior cross-view consistency and identity fidelity, establishing a highly scalable, physics-driven paradigm for multi-agent video world modeling.

[36] Diagnosis of Human Object Interaction Detectors for Real World Educational Applications cs.CVPDF

Divya Mereddy, Ashwin Tudur Sadashiva, Marcos Quinones-Grueiro, Gautam Biswas

TL;DR: 本文提出了一种诊断驱动框架，用于改进人-物交互（HOI）检测器在现实世界教育应用（如医疗培训）中的性能。该框架结合了HOI错误分类和错误因素归因分析，以指导预训练模型的针对性优化。

Details

Motivation: 尽管SOTA HOI检测器在基准数据集上表现良好，但在真实教育环境中（如CCATT混合现实医疗培训）部署时，由于特定领域物体、遮挡和复杂视觉条件，性能会显著下降。

Result: 在CCATT数据集上，该方法通过基于诊断错误因素的针对性优化，将预训练CDN模型的macro-F1分数从48.6提升至90.2。

Insight: 创新点在于将HOI错误模式诊断与模型优化策略相结合，通过系统分析失败原因（如错误分类）来指导领域自适应，而非盲目微调，这为现实场景中HOI模型的部署提供了可借鉴的诊断-优化框架。

Abstract: Human-object interaction (HOI) recognition is critical for automatically analyzing student behavior in complex educational environments. Although state-of-the-art (SOTA) HOI detectors perform well on benchmark datasets, their performance often degrades when deployed in real-world training environments due to domain-specific objects, occlusions, and complex visual conditions. In this paper, we introduce a diagnosis-driven framework that integrates a triplet-level HOI error taxonomy with error-factor attribution analysis for real-world educational video data. We study this problem in the context of Critical Care Air Transport Team (CCATT) mixed-reality medical training. Based on an analysis of HOI failure modes and their causes, we develop a diagnosis-informed refinement strategy for adapting pretrained HOI models to the target domain. Experiments on the CCATT dataset show that this approach improves the macro-F1 score of a pretrained CDN model from 48.6 to 90.2 through targeted refinement guided by diagnosed error factors. These results highlight the value of detailed diagnostic analysis for informing targeted adaptation of HOI models in real-world educational environments.

[37] GeoDrive-Bench: Benchmarking Region-Specific Multimodal Reasoning in Autonomous Driving cs.CVPDF

Yingzi Ma, Chaowei Xiao, Ming Jiang

TL;DR: 该论文提出了GeoDrive-Bench基准，用于系统评估自动驾驶视觉语言模型（VLMs）处理不同地区特定交通规则和文化背景的推理能力。基准包含来自六个国家的5053个人工验证的问答对，涵盖感知、预测、规划和区域推理四个驾驶任务。论文还设计了一种知识蒸馏算法，将区域特定交通规则知识注入VLM内部表示，以提升其跨区域推理性能。

Details

Motivation: 现有自动驾驶视觉语言模型在应对全球不同地区的特定交通规则和文化差异方面能力不足，这阻碍了其实际部署。因此，需要建立一个基准来系统评估和提升模型在这方面的区域感知推理能力。

Result: 在九个最先进的VLM上的实验表明，模型在不同地理驾驶文化下的各项任务性能存在显著差异。论文提出的基线模型（通过知识蒸馏增强）在所有区域都表现出改进的地理文化推理能力。

Insight: 创新点在于创建了一个专注于区域特定多模态推理的自动驾驶诊断基准，并提出了通过知识蒸馏将区域规则知识隐式注入模型表示的方法，而非依赖显式国家标签，这有助于模型更好地将视觉场景理解与本地驾驶策略对齐。

Abstract: Vision-language models (VLMs) for autonomous driving have shown promising performance, but their ability to handle region-specific traffic rules remains underexplored, raising uncertainties about their deployment across diverse global settings. We therefore introduce GeoDrive-Bench, a novel benchmark that enables the systematic investigation of VLMs’ geo-culturally grounded driving reasoning. We curated 5,053 human-validated multiple-choice QA pairs across six countries covering diverse driving cultures. Specifically, we emphasize four driving tasks: perception, prediction, planning, and region reasoning. Each question requires models to infer the correct driving behavior from visual evidence and local traffic conventions without explicit country labels. Beyond evaluation, we further design a distillation algorithm that injects region-specific traffic-rule knowledge into the internal representations of VLMs, enabling models to better align visual scene understanding with local driving policies. Experiments on nine state-of-the-art VLMs show substantial performance variations across geo-driving cultures for each task, while our proposed baseline models exhibit improved geo-cultural reasoning across regions. These results suggest that current VLMs still lack robust region-aware driving intelligence and highlight GeoDrive-Bench as a diagnostic and training-oriented testbed for deployable autonomous driving foundation models.

[38] Cosmos 3: Omnimodal World Models for Physical AI cs.CV | cs.AI | cs.LG | cs.MM | cs.ROPDF

Aditi, Niket Agarwal, Arslan Ali, Jon Allen, Martin Antolini

TL;DR: Cosmos 3是一个全能模态世界模型家族，采用统一的混合Transformer架构，能够联合处理和生成语言、图像、视频、音频和动作序列。它通过支持高度灵活的输入输出配置，将视觉语言模型、视频生成器、世界模拟器和世界-动作模型统一到一个框架中，旨在作为具身智能体的可扩展通用主干。

Details

Motivation: 旨在为物理人工智能（Physical AI）创建一个统一的多模态世界模型，以解决不同模态（语言、图像、视频、音频、动作）在理解和生成任务中各自为政的问题，为具身智能体提供通用基础。

Result: 评估表明，Cosmos 3在多样化的理解和生成任务套件上建立了新的最先进水平（SOTA）。其发布后训练的模型在技术报告撰写时，被Artificial Analysis评为最佳开源文本到图像和图像到视频模型，并被RoboArena评为最佳策略模型。

Insight: 主要创新点在于提出了一个统一的混合Transformer架构，实现了高度灵活的全能模态（语言、图像、视频、音频、动作）联合处理与生成，将多种模型范式整合进单一框架，为物理AI提供了一个可扩展的通用世界模型主干。

Abstract: We introduce Cosmos 3, a family of omnimodal world models designed to jointly process and generate language, image, video, audio, and action sequences within a unified mixture-of-transformers architecture. By supporting highly flexible input-output configurations, Cosmos 3 seamlessly unifies critical modalities for Physical AI – effectively subsuming vision-language models, video generators, world simulators, and world-action models into a single framework. Our evaluation demonstrates that Cosmos 3 establishes a new state-of-the-art across a diverse suite of understanding and generation tasks, demonstrating omnimodal world models as scalable, general-purpose backbones for embodied agents. Our post-trained Cosmos 3 models were ranked as the best open-source Text-to-Image and Image-to-Video models by Artificial Analysis, and the best policy model by RoboArena at the time the technical report was written. To accelerate open research and deployment in Physical AI, we make our code, model checkpoints, curated synthetic datasets, and evaluation benchmark available under the Linux Foundation’s OpenMDW-1.1 https://openmdw.ai/license/1-1/ License at https://github.com/nvidia/cosmos}{github.com/nvidia/cosmos and https://huggingface.co/collections/nvidia/cosmos3 . The project website is available at https://research.nvidia.com/labs/cosmos-lab/cosmos3 .

[39] Automated Report-Derived Oncology VQA Benchmark for Evaluating Vision-Language Models on 3D Medical Imaging cs.CVPDF

Bo Liu, Hanxue Gu, Xiangru Li, Zheren Zhu, Jacob Ellison

TL;DR: 本文提出了一种自动化代理驱动流程，用于从配对的私人放射学报告和3D肿瘤影像中直接生成多项选择视觉问答数据集，从而创建了一个无需人工标注、实例污染可控的医学VLM评估基准。该流程生成两种互补的问题类型：基于临床报告模式确定性生成的RADS风格问题，以及由LLM从放射科医生发现中生成并经源报告验证的报告衍生问题。在四个内部癌症队列上应用后，对六个VLM的零样本评估显示没有主导模型，且所有类别均有显著提升空间。

Details

Motivation: 现有医学VLM评估基准存在规模有限、依赖人工标注或可能泄露到预训练数据中的问题，需要创建临床基础扎实、可扩展且能控制评估混淆因素的基准。

Result: 在四个内部癌症队列上构建了基准，对六个VLM的零样本评估显示没有模型占主导地位，且存在显著提升空间。消融实验表明视觉依赖性高度依赖于具体数据集：肝脏报告衍生问题确实需要图像，而肺部CT问题基本可以在不看图像的情况下解决（领先的闭源模型在盲测时甚至超过了其有视觉输入的准确率）。

Insight: 创新点在于提出了一个全自动、可扩展的基准生成流程，能直接从临床数据生成受控的VQA数据集，避免了人工标注和实例污染。客观来看，该方法揭示了即使使用私有临床数据，也不能保证对模型视觉能力的评估是纯净的，因为某些数据集可能被语言模型“走捷径”解决，这为未来构建更可靠的医学VLM评估基准提供了重要洞见。

Abstract: Evaluating vision-language models (VLMs) on medical images requires benchmarks that are clinically grounded, scalable, and controlled for evaluation confounds. Existing public benchmarks are limited in scale, manually annotated, or potentially leaked into VLM pretraining corpora. We present an automated agent-driven pipeline that generates multiple-choice VQA datasets directly from paired private radiology reports and 3D oncology imaging, producing two complementary question types: RADS-style questions deterministically derived from clinician-defined reporting schemas, and radiology report-derived questions generated by an LLM from radiologist findings and verified against the source report. Applied to four in-house cancer cohorts, the pipeline yields an instance-contamination-controlled benchmark without per-question human annotation. Zero-shot evaluation of six VLMs reveals no dominant model and substantial headroom across all cells. A blind ablation reveals that visual reliance is highly dataset-specific: liver Report-derived questions genuinely require the image, while Lung CT is essentially solvable without it - the leading closed model exceeds its sighted accuracy on Lung CT when blinded - indicating that even private clinical data does not guarantee a contamination-controlled read of visual capability. The pipeline is released as an open agent skill for in-house redeployment.

[40] Pathway-Structured Privileged Distillation for Deployable Computational Pathology cs.CVPDF

Yongxin Guo, Hao Lu, Onur Koyun, Zhengjie Zhu, Muhammet Demir

TL;DR: 本文提出了一种名为Mixture of Pathway Experts（MoPE）的知识蒸馏框架，旨在解决在常规临床环境中RNA测序数据稀缺的问题。该框架通过将多模态学习重新定义为特权蒸馏，仅使用组织病理学切片（WSI）进行推理，同时利用RNA衍生的通路信息进行监督训练。MoPE通过内存使用对齐机制，将分子层面的监督信息转移到基于通路的病理学专家模型中，从而在多个公共基准和独立乳腺癌队列中提升了仅使用WSI的推理性能。

Details

Motivation: 整合转录组学和组织病理学可以改进癌症风险建模，但在常规临床实践中，RNA测序数据的可用性有限，这限制了多模态学习的实际应用。因此，本文旨在开发一种方法，能够在训练阶段利用分子信息，而在推理阶段仅依赖组织病理学图像，以克服数据可用性的约束。

Result: 在多个公共基准和两个独立的乳腺癌队列上，MoPE相对于基线方法，在仅使用WSI的推理任务中，性能得到了一致提升。通路使用分析和人工审核的视觉检查为模型行为和候选的形态学相关读出提供了有界的可解释性分析。

Insight: MoPE的创新点在于将多模态学习重构为特权蒸馏问题，通过RNA衍生的通路结构来引导组织病理学模型的训练，从而在推理时无需RNA数据。从客观角度看，该方法通过内存使用对齐机制，有效地将分子监督信息转移到病理学模型中，为在数据受限场景下利用分子信息提供了可借鉴的框架。

Abstract: Integrating transcriptomics and histopathology can improve cancer risk modelling, yet practical use is constrained by the limited availability of RNA profiling in routine settings. Here we introduce Mixture of Pathway Experts (MoPE), a knowledge-distillation framework that reframes multimodal learning as privileged distillation for histology-only inference. MoPE is motivated by the partial observability between RNA profiles and whole-slide images: histology can capture morphology-linked consequences of certain molecular programmes, but cannot be expected to reconstruct the full transcriptomic state. MoPE encodes RNA-derived pathways and transfers the molecular supervision to pathway-indexed pathology experts through memory-usage alignment. Across diverse public benchmarks and two independent breast cancer cohorts, MoPE consistently improved WSI-only inference performance relative to baseline methods. Pathway-usage analyses and human-audited visual inspection provide bounded inspection of model behaviour and candidate morphology-linked readouts. These results support pathway-structured privileged distillation as a promising route to using molecular information during training while preserving RNA-free inference.

[41] Tiny Collaborative Inference for Occlusion-Robust Object Detection cs.CVPDF

Chieh-Tung Cheng, Mustafa Aslanov, Eiman Kanjo

TL;DR: 该论文研究了在资源受限的边缘设备上进行遮挡鲁棒的目标检测，通过结合MCUNet骨干网络、YOLOv2检测头和TensorFlow Lite量化技术，在SRAM小于1MB的设备上评估了两种协作推理策略：特征级融合和基于加权框融合（WBF）的决策级融合。实验表明，在遮挡场景下，WBF优于特征级融合，并在非对称遮挡中提升mAP达0.2736，扩展到三视图可进一步提升精度（mAP增益0.3827），同时通信开销较小。硬件部署在Coral Dev Board Micro上验证了WBF的可行性，在自主会话中帧级覆盖率提升29.8%。

Details

Motivation: 解决在超低端边缘设备（如IoT监控节点和搜救平台）上运行本地计算机视觉时面临的内存计算限制、协作通信成本以及遮挡导致的准确率下降问题。

Result: 在遮挡设置下，WBF决策级融合优于特征级融合，在非对称遮挡场景中mAP提升高达0.2736；扩展到三视图时mAP增益达0.3827，通信开销约1.3KB/次。在Coral Dev Board Micro的Wi-Fi点对点部署中，WBF在设备上运行，通信能耗相对于推理较小，在301.9秒的自主会话中帧级覆盖率提升29.8%。

Insight: 创新点在于将决策级融合（WBF）应用于超低端硬件的协作推理，以提升遮挡鲁棒性，同时实现无主机的多板操作；客观分析表明该方法在资源受限环境下平衡了精度与通信开销，为小规模边缘目标检测提供了实用方案。

Abstract: Small edge devices such as IoT surveillance nodes and search-and-rescue (SAR) platforms are increasingly expected to run computer vision locally. On ultra-low-end hardware, however, object detection is limited by available memory and compute, by communication costs when several devices cooperate, and by the loss of accuracy caused by occlusion. The work evaluates occlusion-robust object detection on devices with less than 1 MB SRAM by combining an MCUNet backbone, a YOLOv2 detection head, and TensorFlow Lite quantisation. We evaluate two collaborative inference strategies: feature-level fusion, which concatenates intermediate feature maps, and decision-level fusion via Weighted Boxes Fusion (WBF). Under the tested occlusion settings, WBF outperforms feature-level fusion and gives gains of up to +0.2736 mAP in asymmetric occlusion scenarios. Extending fusion to three views improves accuracy further (up to +0.3827 mAP) while adding communication overhead (approximately 1.3 KB per exchange). The hardware experiments start with a host-assisted USB-relay baseline and then move to a Wi-Fi peer-to-peer deployment on two Coral Dev Board Micro units, where WBF runs on-device and communication energy remains small relative to inference. In a representative 301.9 s autonomous session comprising 108 frames, fused output is observed on 61 frames compared with 47 for Board 2 alone, a frame-level coverage gain of +29.8%. We also include a small exploratory decentralised federated learning (DFL) feasibility note, but do not treat it as a main result because performance remains limited under non-iid local data. The results support decision-level fusion as a viable option for improving occlusion robustness in small-scale edge object detection, including host-free multi-board operation on ultra-low-end hardware.

[42] Pixel Cube: Diffusion-based Portrait Video Relighting Through Realistic Lighting Reproduction cs.CVPDF

Yufan Zhang, Yu Ji, Ayo Ajiboye, Rundi Wu, Yu Guo

TL;DR: 本文提出了一种基于扩散模型的动态肖像视频重光照方法，通过真实光照模拟和混合训练数据集，实现了高真实感、时序一致的重光照效果。该方法利用预训练视频扩散模型的先验知识，以高动态范围环境图作为光照控制，并引入合成背景图像以调节相机曝光和色调，从而生成与目标环境和谐且保留人物身份特征的视频。

Details

Motivation: 解决动态肖像视频重光照中真实感、时序一致性和身份特征保持的挑战，通过构建真实捕获与渲染的混合数据集，以及LED光照系统模拟真实光照条件，以提升重光照的视觉效果。

Result: 在野外视频重光照实验中，使用多种环境图进行测试，结果表明该方法在真实感、光照和谐性和时序一致性方面达到了最先进的性能水平。

Insight: 创新点包括构建混合训练数据集和LED光照系统以模拟真实光照，利用预训练视频扩散模型先验并结合HDR环境图与合成背景图像进行多维度控制，实现了对未见数据（如人物外观、运动和光照条件）的良好泛化能力，为肖像摄影提供了实用应用。

Abstract: We present a diffusion-based method for relighting dynamic portrait videos with photorealism and temporal consistency. Our method is fueled by a hybrid training dataset that consists of real-captured and rendered dynamic portrait videos with diverse subject appearances, facial motions, head poses, and known lighting conditions. Specifically, we construct an LED-based lighting system for realistic lighting emulation and high-speed video relighting data acquisition. By leveraging the image priors embedded in pre-trained video diffusion models, and using per-frame high dynamic range (HDR) environment map as lighting control, we train a high-performance generative model for realistic and identity-preserving dynamic portrait video relighting. In addition to the environment map control, our model uses a synthesized background image to enable control on the camera’s exposure level and color tone. Our model can produce temporally consistent relit portrait video that looks realistic and harmonious under a provided new environment and faithfully preserve the subject’s expression and fine facial features, including skin tone, wrinkles, and facial hair. Our model generalizes well to unseen data, in terms of the subject appearance, motion, and lighting condition. We perform extensive experiments on relighting in-the-wild videos with various environment maps and demonstrate practical applications on portrait photography. Results show that our method achieves state-of-the-art performance in photorealism, lighting harmony, and temporal consistency.

[43] Any2Poster: Any-Source Poster Generation Across Modalities and Domains cs.CVPDF

Amogh Vinaykumar, Aiden Li, Suozhi Huang, Shilong Liu

TL;DR: 本文提出了Any2Poster Bench基准测试和Any2Poster Agent智能体，用于评估和实现跨模态、跨领域的任意源海报生成。基准测试涵盖八种输入模态和五个内容领域，通过基于测验的探测和视觉语言模型评估来综合衡量信息保真度和视觉传达效果。智能体作为一个端到端的参考系统，能够解析异构源、组织内容、规划布局、渲染海报并进行迭代优化。

Details

Motivation: 当前自动海报生成的进展难以衡量，因为现有评估通常局限于纯文本输入、狭窄领域或表面视觉相似性。本文旨在解决这一评估缺口，为多模态、领域通用的海报生成提供一个全面、可复现的评估框架。

Result: 在提出的Any2Poster Bench上，Any2Poster Agent在所有输入模态和内容领域的平均准确率分别达到87.25%和87.28%。在仅限论文输入（PaperQuiz风格）的评估中，该智能体将整体准确率从先前最佳模型（PosterAgent-4o）的51.06-51.33%提升至72.58%，密度增强分数从116-121提升至145.16，实现了显著的性能提升。

Insight: 论文的主要创新点在于构建了一个涵盖广泛输入模态和内容领域的综合性评估基准，并提出了一个端到端的参考智能体作为强基线。从客观角度看，其将基于测验的事实保留与理解评估，与基于VLM的视觉质量、布局、可读性等多维度判断相结合，为评估生成式系统的信息保真度和视觉沟通能力提供了新的、更全面的方法论。

Abstract: Visual posters are a compact medium for communicating dense information, yet progress on automatic poster generation remains difficult to measure because existing evaluations are often restricted to paper-only inputs, narrow domains, or surface-level visual similarity. We introduce Any2Poster Bench, a benchmark for any-source poster generation that evaluates systems across eight input modalities–PDFs, URLs, PPTX, DOCX, Markdown, LaTeX, notebooks, and videos–and five content domains. Any2Poster Bench pairs each source with quiz-based probes of verbatim factual retention and interpretive understanding, together with VLM-based judgments of visual quality, layout, readability, content completeness, and logical flow, enabling reproducible assessment of both information fidelity and visual communication. To instantiate and validate this benchmark, we further present Any2Poster Agent, an end-to-end reference agent that parses heterogeneous sources, organizes salient content, plans poster layouts, renders posters, and iteratively refines them using visual feedback. On Any2Poster Bench, Any2Poster Agent achieves 87.25% average accuracy across input modalities and 87.28% across content domains. On PaperQuiz-style evaluation, where prior paper-to-poster agents are directly comparable, Any2Poster Agent improves over PosterAgent-4o from 51.06-51.33% to 72.58% overall accuracy and from 116-121 to 145.16 in density-augmented score. Together, Any2Poster Bench and Any2Poster Agent provide a reusable evaluation resource and a competitive baseline for studying multimodal, domain-general poster generation.

[44] The Road Ahead in Autonomous Driving: The KITScenes Multimodal Dataset cs.CV | cs.LG | cs.ROPDF

Richard Schwarzkopf, Fabian Immel, Alexander Blumberg, Jonas Merkert, Nils Rack

TL;DR: 本文介绍了KITScenes Multimodal数据集，这是一个欧洲的高保真多模态自动驾驶数据集，包含高分辨率相机、超400米长程激光雷达、4D成像雷达和冗余定位系统，并提供了目前最完整的HD地图，所有交通元素均以3D映射且具有完整拓扑连接。该数据集在具有不规则街道布局和混合交通模式的城市中采集，扩展了地理多样性，并引入了四个基准任务以推进具身AI的空间学习。

Details

Motivation: 现有自动驾驶数据集在传感器保真度、地图完整性和地理多样性方面存在不足，因此需要构建一个更全面、高保真的多模态数据集来推动自动驾驶研究。

Result: 数据集通过开源软件在自动驾驶试验中验证了其HD地图的完整性，并首次在公开数据集中实现了所有驾驶相关交通元素的3D精确重投影映射。

Insight: 创新点在于集成了高保真传感器套件和极其完整的3D HD地图，首次实现了交通元素的拓扑连接映射，并通过四个新基准任务（如在线HD地图构建和长程深度估计）促进了具身AI的空间学习能力。

Abstract: Existing autonomous driving datasets have enabled major progress, but fall short in sensor fidelity, map completeness, or geographic diversity. We present KITScenes Multimodal, a European dataset built around high-fidelity sensors and maps. Our fully synchronized sensor suite combines high-resolution global-shutter cameras, long-range lidar beyond 400m, 4D imaging radar, and redundant GNSS/INS localization. Our HD maps are, to our knowledge, the most complete of any sensor dataset, validated through autonomous driving trials on open-source software. For the first time in a public dataset, all driving-relevant traffic elements, such as traffic lights, are mapped in 3D to a reprojection-accurate level with full topological connectivity. Recorded in cities with irregular street layouts and mixed traffic modes, our dataset complements existing datasets by broadening the available geographic diversity. We also introduce four benchmarks, each advancing spatial learning for embodied AI: online HD map construction, long-range depth estimation, novel view synthesis, and end-to-end driving. Project page: https://kitscenes.com/

[45] Hand Trajectory Fusion for Egocentric Natural Language Query Grounding cs.CV | cs.AI | cs.HC | eess.IVPDF

Enmin Zhong, Carlos R. del-Blanco, Fernando Jaureguizar, Narciso García

TL;DR: 本文提出了一种用于第一人称视角自然语言查询（NLQ）定位的新方法，该方法通过引入手部轨迹编码器来融合手部运动信息。该方法将手部骨骼序列转换为高语义的手部运动学特征，并通过带有自适应门控的交叉注意力策略与预训练的视频-文本特征进行对齐和融合。

Details

Motivation: 现有方法在融合视频外观与查询文本时忽略了手部运动信息，而Ego4D NLQ数据集中约41%的查询答案与手-物交互或其直接结果相关，因此需要整合手部轨迹以提供更丰富的定位线索。

Result: 在Ego4D NLQ v2验证集上，该方法在Hand-Object Interaction查询和Quantity/State查询上取得了最显著的提升，分别提高了+2.54和+4.32的R1@IoU=0.3分数。

Insight: 创新点在于设计了一个专门的手部轨迹编码器来提取手部运动学特征，并通过自适应门控的交叉注意力机制将其与视觉-文本特征融合，这为第一人称视频理解任务提供了超越外观的手部交互动态线索。

Abstract: Egocentric Natural Language Query (NLQ) grounding asks a model to localize, in a long first-person video, the temporal interval that answers a free-form text query. Existing methods fuse video appearance with the query but ignore hand motion, despite the fact that roughly 41% of Ego4D NLQ queries are answered at a moment of hand–object manipulation or their immediate outcomes.We propose a hand-trajectory encoder for converting a sequence of hand skeletons into highly-semantic hand kinematic features, which are then aligned and combined with pretrained video–text features through a cross-attention fusion strategy with adaptive gating. On the Ego4D NLQ v2 validation split, the clearest gains appear for Hand-Object Interaction queries (+2.54 R1@IoU=0.3) and Quantity/State queries (+4.32 R1@IoU=0.3), indicating that hand trajectory provides grounding cues beyond appearance alone.

[46] Towards Compact Autonomous Driving Perception with Balanced Learning and Multi-sensor Fusion cs.CV | cs.AI | cs.ROPDF

Oskar Natan, Jun Miura

TL;DR: 本文提出了一种紧凑的多任务深度学习模型，用于在单次前向传播中同时处理自动驾驶感知任务，包括语义分割、深度估计、LiDAR分割和鸟瞰图投影。该模型通过自适应损失加权算法解决多任务学习不平衡问题，并利用数据预处理和中间传感器融合技术整合RGB相机、动态视觉传感器和LiDAR等多模态输入，以提升对动态环境的理解。

Details

Motivation: 解决自动驾驶感知中多任务模型参数量大、计算效率低的问题，同时处理多传感器输入融合与多任务学习不平衡的挑战。

Result: 在CARLA仿真数据集和真实世界nuScenes-lidarseg数据集上，模型在保持更好性能的同时参数量显著减少，推理速度更快且GPU内存占用更低，通过消融实验和对比研究验证了其有效性。

Insight: 创新点包括紧凑的多任务网络架构、自适应损失加权算法以平衡多任务学习，以及中间传感器融合技术整合多模态输入，实现了高效且性能均衡的自动驾驶感知系统。

Abstract: We present a novel compact deep multi-task learning model to handle various autonomous driving perception tasks in one forward pass. The model performs multiple views of semantic segmentation, depth estimation, light detection and ranging (LiDAR) segmentation, and bird’s eye view projection simultaneously without being supported by other models. We also provide an adaptive loss weighting algorithm to tackle the imbalanced learning issue that occurred due to plenty of given tasks. Through data pre-processing and intermediate sensor fusion techniques, the model can process and combine multiple input modalities retrieved from RGB cameras, dynamic vision sensors (DVS), and LiDAR placed at several positions on the ego vehicle. Therefore, a better understanding of a dynamically changing environment can be achieved. Based on the ablation study, the model variant trained with our proposed method achieves a better performance. Furthermore, a comparative study is also conducted to clarify its performance and effectiveness against the combination of some recent models. As a result, our model maintains better performance even with much fewer parameters. Hence, the model can inference faster with less GPU memory utilization. Moreover, the result tends to be consistent in 3 different CARLA simulation datasets and 1 real-world nuScenes-lidarseg dataset. To support future research, we share codes and other files publicly at https://github.com/oskarnatan/compact-perception.

[47] MUSE: A Unified Agentic Harness for MLLMs cs.CV | cs.AIPDF

Jianglin Lu, Hailing Wang, Xu Ma, Qihua Dong, Mingyuan Zhang

TL;DR: 本文提出了MUSE，一种多模态统一结构化执行框架，通过组合任务表示、视觉处理、感知工具使用、结构化解析、确定性验证和验证器引导修复等模块，在不重新训练模型的情况下提升冻结多模态大语言模型（MLLM）的能力。

Details

Motivation: 尽管多模态大语言模型（MLLMs）发展迅速，但在人类轻松解决的任务（如从截图导航网格迷宫或选择正确的拼图块）上仍存在失败。研究旨在探索在不重新训练模型的前提下，通过改进其执行框架能激发多少模型潜力。

Result: 在涵盖视觉空间规划、视觉感知、多模态推理和细粒度视觉辨别的多个基准测试中，使用多个最先进的MLLMs进行评估，MUSE在所有设置下均比原始模型带来一致提升，在挑战性实例上提升最大。

Insight: 创新点在于将MLLM的失败归因于框架层面的不足而非模型根本缺陷，并通过验证器引导修复等结构化代理框架提供了一种与模型中心优化正交的改进途径，强调了代理多模态框架作为关键但未充分探索的设计维度。

Abstract: Despite rapid progress, multimodal large language models (MLLMs) still fail on tasks that humans solve effortlessly, such as navigating a grid maze from a screenshot or selecting the correct puzzle piece. Rather than retraining the model, we ask a complementary question: how much capability can be elicited from a frozen MLLM purely by improving the execution scaffold around it? We introduce MUSE, a multimodal unified structured execution harness that wraps any off-the-shelf MLLM with composable modules for task representation, visual processing, perception tool use, structured parsing, deterministic verification, and verifier-guided repair, without any model retraining. We evaluate MUSE across diverse benchmarks spanning visual spatial planning, visual perception, multimodal reasoning, and fine-grained visual discrimination, using multiple state-of-the-art MLLMs. MUSE delivers consistent gains over the bare model in all settings, with the largest jumps on challenging instances. Further analysis reveals that many MLLM failures arise from harness-level shortcomings rather than fundamental model deficits, and can be addressed through verifier-guided repair without touching the model. These findings highlight the agentic multimodal harness as a critical yet underexplored design dimension, offering an orthogonal avenue for improving MLLMs beyond model-centric optimization.

[48] Hierarchical Federated Learning with Dynamic Clustering and Adaptive Regularization for Robust Infrastructure Inspection cs.CVPDF

Yuhu Feng, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama

TL;DR: 本文提出了一种新颖的层次化联邦学习框架，通过动态聚类和自适应正则化来解决基础设施结构健康监测中的‘双重异构性’挑战。该框架在宏观层面通过基于梯度的动态聚类将客户端聚合为专家小组，在微观层面通过动态区域自适应近端正则化模块校准本地更新，从而在保护隐私的同时提升模型的鲁棒性和专业性。

Details

Motivation: 动机是解决在严格隐私法规下，数据驱动的结构健康监测模型面临的数据孤岛问题，以及联邦学习在应用于全国性基础设施网络时遇到的‘双重异构性’挑战——即宏观层面的物理结构差异和微观层面的本地数据统计不平衡。

Result: 在一个大规模真实世界结构检测数据集上的综合评估表明，该框架成功中和了双重异构性，为复杂基础设施检测生成了高度鲁棒和专业的诊断模型。

Insight: 创新点在于将宏观的动态梯度聚类与微观的动态区域自适应近端正则化进行层次化集成，前者无需先验地理元数据即可根据结构退化轨迹进行聚类，后者通过实时计算非独立同分布强度分数并自适应调整近端惩罚，有效缓解了客户端漂移和少数损伤类别的灾难性遗忘问题。

Abstract: The deployment of data-driven computer vision models for structural health monitoring (SHM) is heavily constrained by the data silo dilemma due to stringent privacy and security regulations. While federated learning (FL) offers a privacy-preserving collaborative alternative, its application to nationwide infrastructure networks is severely hindered by the challenge of ``double heterogeneity’’: macro-level physical divergence across disparate structural types and micro-level statistical imbalances within local datasets. To overcome this challenge, this paper proposes a novel hierarchical federated learning framework. The framework orchestrates a synergistic two-tier optimization strategy. At the macro-level, a dynamic gradient-based clustering mechanism autonomously aggregates distributed clients into specialized expert groups based on their structural degradation trajectories, circumventing the need for prior geographical metadata. Concurrently, at the micro-level, an intra-cluster Dynamic Region-Adaptive Proximal Regularization (DRAPR) module computes a real-time statistical Non-IID Intensity Score for each client. By adaptively modulating a proximal penalty based on local label skewness and gradient divergence, DRAPR effectively calibrates local updates, mitigates client drift, and prevents the catastrophic forgetting of minority damage classes. Comprehensive evaluations on a large-scale, real-world structural inspection dataset demonstrate that the hierarchical integration of macro-clustering and micro-regularization successfully neutralizes dual-level heterogeneity, yielding highly robust and specialized diagnostic models for complex infrastructure inspection.

[49] TGV-KV: Text-Grounded KV Eviction for Vision-Language Models cs.CVPDF

Jizhihui Liu, Ruizi Han, Miao Zhang, Rui Shao, Xuebo Liu

TL;DR: 本文提出了一种针对视觉语言模型（VLMs）的文本引导KV缓存淘汰方法TGV-KV，旨在解决VLMs推理时KV缓存内存消耗随上下文长度线性增长的问题。该方法通过分析文本与视觉模态间的差异，设计了三个子模块：文本-视觉预算分配、文本加权排序和文本优先保留策略，以在减少内存占用的同时保持模型性能。

Details

Motivation: VLMs继承了自回归生成范式并缓存所有先前token的键值（KV）以加速推理，导致内存消耗随上下文长度线性增长，且视觉模态中存在大量冗余。现有的KV缓存淘汰方法多针对语言模型设计，忽视了文本与视觉间的固有差距，在VLMs中应用时会导致显著的性能下降。

Result: 在VizWiz-VQA任务上，TGV-KV在LLaVA-NeXT模型上仅保留5%的KV缓存时，仍能保持99.2%的全KV缓存准确率，并将端到端吞吐量提升了52.6%。该方法在五种不同规模和架构的模型上进行了评估，均显示出有效性。

Insight: 创新点在于首次系统分析了VLMs中的模态差距，并主张视觉信息的重要性应基于文本引导来评估。据此设计的TGV-KV方法通过文本加权注意力对视觉重要性进行排序，并策略性地保留文本KV以避免关键信息丢失，为VLMs的高效推理提供了新思路。

Abstract: Vision-Language Models (VLMs) inherit the auto-regressive generation paradigm and cache the keys and values (KV) of all previous tokens to accelerate inference, resulting in memory consumption that scales linearly with context length. This issue is particularly pronounced in VLMs due to substantial redundancy in the visual modality. Although KV cache eviction approaches can effectively reduce inference memory, they often incur significant performance degradation in VLMs, as most are designed for language models and overlook the inherent gap between text and vision. By systematically analyzing the modality gap in VLMs in this work, we argue that the importance of visual information should be grounded in textual guidance and accordingly propose a Text-Grounded KV Eviction method for VLMs (TGV-KV). TGV-KV comprises three submodules: (1) Text-Vision Budgeting (TVB) assigns budget to each layer based on the mutual information interaction. (2) Text-Weighted Ranking (TWR) assesses the priority of text and ranks vision importance based on weighted text-image attention. (3) Text-Prioritised Retention (TPR) policy strategically preserves text KV to avoid acute information loss. We evaluate TGV-KV across five models with different sizes and architectures, showing that TGV-KV preserves 99.2% full-KV accuracy on the VizWiz-VQA task with LLaVA-NeXT and boosts end-to-end throughput by 52.6% with an extreme retention budget of 5%. Code is available at https://github.com/Danielement321/TGV-KV.

[50] Zero-Shot 3D Question Answering via Hierarchical View-to-Token Transportation cs.CV | cs.LGPDF

Dongsheng Wang, Dawei Su, Hui Huang

TL;DR: 本文提出了一种名为KeyVT的分层方法，用于零样本3D问答任务，通过结合2D视觉语言模型（VLMs）处理3D点云。该方法在视图和令牌两个层次上收集输入上下文：首先基于语义内容和几何位置选择空间一致且任务相关的视图，然后在最优传输（OT）框架下识别代表性令牌以减少冗余。实验表明，该方法在多个基准测试上显著优于现有免调优方法，性能接近基于训练的方法。

Details

Motivation: 动机是利用2D视觉语言模型（VLMs）进行零样本3D场景理解，但现有方法在有限输入预算下难以保留任务相关的3D细节，因此需要提高输入上下文的质量。

Result: 在三个广泛使用的基准测试上评估，该方法相比现有免调优方法有显著改进，性能与基于训练的方法相当。

Insight: 创新点包括：结合像素特征和相机参数进行视图重要性评估，确保空间一致性和任务相关性；在最优传输框架下识别代表性令牌以减少跨视图冗余，提升上下文效率。从客观角度看，分层处理视图和令牌的策略有效平衡了细节保留与计算成本。

Abstract: Recently, zero-shot 3D scene understanding via 2D Vision-Language Models (VLMs) has gained increasing research interest due to their promising spatial reasoning capabilities. Typically, multiple 2D views are sampled from a 3D point cloud and fed into pre-trained VLMs to answer a given question. This paradigm highlights the critical role of input context quality and raises the challenge of retaining as many task-relevant 3D details as possible under a limited input budget. We propose \texttt{KeyVT}, a hierarchical approach for input context collection at both the view and token levels. Specifically, we combine pixel features with camera parameters and assess view importance based on both semantic content and geometric position, resulting in spatially consistent and task-relevant views. Furthermore, we address redundancy among patches across selected views by identifying representative tokens under the optimal transport (OT) framework, where view tokens and key tokens are formulated as two discrete distributions in the embedding space. These key tokens are expected to cover all view features by minimizing the OT distance. We evaluate our framework on three widely used benchmarks, demonstrating significant improvements over existing tuning-free methods and performance comparable to training-based approaches.

[51] KC-3DGS: Kurtosis-Constrained Gaussian Splatting for High-Fidelity View Synthesis cs.CVPDF

Vivekjyoti Banerjee, Abhay Yadav, Rama Chellappa, Aniket Roy

TL;DR: 论文提出KC-3DGS方法，通过在3D高斯溅射（3DGS）的训练中引入基于自然图像统计的小波域监督，以解决标准像素空间损失导致的过度平滑和结构伪影问题，从而提升高保真视图合成质量。

Details

Motivation: 标准3DGS使用的像素空间损失（如L1、SSIM）仅约束整体重建误差，允许优化过程在不同频率尺度上重新分配误差，这会导致在稀疏视图等监督有限的情况下产生过度平滑和结构伪影。

Result: 在MipNeRF360、Tanks&Temples、MVImgNet、DeepBlending和WRIVA-ULTRRA等多个基准测试中，该方法在感知质量上取得了一致性提升。在具有挑战性的WRIVA-ULTRRA户外数据集上，KC-3DGS将DreamSim指标提升了9.48%，同时PSNR、SSIM和LPIPS也有所改善；在仅有12张训练图像的稀疏视图设置下，MipNeRF360上的PSNR提升了高达0.5 dB。

Insight: 创新点在于引入了多尺度小波系数对齐损失、监督峰度集中损失和跨带协方差惩罚三项小波域监督组件，从理论上分析了像素空间损失的局限性，并提出了一种联合优化目标以排除退化解，该方法可作为即插即用的正则化策略无缝集成到现有3DGS流程中。

Abstract: 3D Gaussian Splatting (3DGS) enables real-time novel view synthesis by representing scenes as collections of anisotropic Gaussians optimized via differentiable rasterization. However, standard pixel-space losses (L1, SSIM) constrain only aggregate reconstruction error, permitting the optimization to redistribute error across frequency scales. This leads to oversmoothing and structural artifacts, particularly in sparse-view settings where supervision is limited. We propose KC-3DGS, which augments 3DGS training with wavelet-domain supervision based on natural image statistics. Our method combines three components: (1) a multi-scale wavelet coefficient alignment loss that explicitly penalizes missing high-frequency detail, (2) a supervised kurtosis concentration loss that encourages rendered images to match the heavy-tailed frequency statistics of ground-truth images, and (3) a cross-band covariance penalty that promotes frequency specialization. We provide theoretical analysis showing that pixel-space losses admit a family of indistinguishable perturbations under wavelet redistribution, and that our joint objective excludes degenerate solutions. Experiments across MipNeRF360, Tanks&Temples, MVImgNet, DeepBlending, and WRIVA-ULTRRA demonstrate consistent improvements in perceptual quality. On the challenging WRIVA-ULTRRA outdoor dataset, KC-3DGS achieves a 9.48% improvement in DreamSim while also improving PSNR, SSIM, and LPIPS. In sparse-view settings with only 12 training images, our method improves PSNR by up to 0.5 dB on MipNeRF360 while maintaining perceptual quality. The approach integrates seamlessly into existing 3DGS pipelines as a plug-and-play regularization strategy.

[52] FAF-CD: Frequency-Aware Fusion for Change Detection under Imperfect Multimodal Remote Sensing cs.CVPDF

Yufan Wang, Sokratis Makrogiannis, Chandra Kambhamettu

TL;DR: 论文提出了FAF-CD，一种用于不完美多模态遥感变化检测的频率感知融合框架。该框架采用DINOv3预训练的ConvNeXt编码器和基于线性复杂度VMamba的解码器，通过一个包含可变形空间对齐、傅里叶和哈尔小波比较的三分支融合模块，自适应地聚合跨尺度的互补信息。

Details

Motivation: 解决现实世界遥感监测中，由于前后事件图像异步、跨传感器或受光照、季节、模态偏移影响，导致的不完美异质观测条件下的变化检测难题，尤其是在EO-SAR灾害制图中，干扰变化可能与结构损坏相似。

Result: 在BRIGHT验证集上，针对匹配的异质EO-SAR数据，其干净和扰动后的tc-mIoU/tc-mAP指标优于NeXt2Former-CD。在二元光学变化检测任务上，在LEVIR-CD和WHU-CD数据集上分别达到0.924和0.955的cF1分数，并在伪变化对齐压力测试中，在两个二元数据集上的平均扰动cIoU/cF1优于M-CD和NeXt2Former-CD。同时，相比NeXt2Former-CD减少了约24 GFLOPs的计算成本。

Insight: 创新点在于提出了一个频率感知的混合框架，其核心是结合了可变形空间对齐与频域（傅里叶）和时频域（哈尔小波）比较的三分支融合模块，并采用自适应门控机制进行多尺度信息融合。从客观角度看，该方法在频域和空间域进行联合建模以应对复杂干扰，并有效平衡了精度与计算效率。

Abstract: Remote sensing change detection for real-world monitoring often relies on imperfect heterogeneous observations, where pre- and post-event images may be asynchronous, cross-sensor, or affected by illumination, seasonal, and modality shifts. This setting is especially challenging for EO-SAR disaster mapping, where nuisance variation can resemble structural damage. We propose FAF-CD, a frequency-aware hybrid framework with a DINOv3-pretrained ConvNeXt encoder and a linear-complexity VMamba-based decoder. Its rectification-aware tri-branch fusion module combines deformable spatial alignment with Fourier and Haar-wavelet comparisons, using adaptive gating to aggregate complementary cues across scales. On BRIGHT validation, a matched heterogeneous EO-SAR adaptation improves clean and perturbed tc-mIoU/tc-mAP over NeXt2Former-CD. FAF-CD also generalizes to binary optical CD, achieving 0.924 cF1 on LEVIR-CD and 0.955 cF1 on WHU-CD, and obtains the best average perturbed cIoU/cF1 on both binary datasets among M-CD and NeXt2Former-CD under pseudo-change-aligned stress tests. It further reduces cost by approximately 24 GFLOPs relative to NeXt2Former-CD while maintaining or improving accuracy.

[53] Disentangling Visual and Factual Correctness in LVLMs’ Visualization Literacy cs.CVPDF

Soohyun Lee, Jaeyoung Kim, Seokhyeon Park, Sihyeon Lee, Jiwon Song

TL;DR: 该论文提出了一个名为CVLAT的框架，旨在分离大型视觉语言模型（LVLMs）在可视化解读任务中的视觉正确性和事实正确性，以评估其是否真正基于视觉证据进行推理，还是依赖于训练中记忆的事实先验。研究发现，现有评估方法（如VLAT）可能高估模型能力，而随机数据测试（reVLAT）可能低估，通过CVLAT和视觉-事实依赖指数（VFRI）揭示了模型在视觉与事实冲突时的不同倾向。

Details

Motivation: 当前对LVLMs可视化素养的评估混淆了视觉推理和事实先验记忆，无法确定模型正确回答是源于对视觉证据的真实理解还是训练数据的记忆，这限制了可靠评估模型在视觉分析中整合能力。

Result: 在15个SOTA LVLMs上的三个实验表明：1）多个模型在标准测试（VLAT）上达到人类水平，但可能反映事实回忆而非视觉理解；2）使用CVLAT和VFRI指标，将模型分类为视觉导向型（多数）和事实知识导向型（少数），人类基线在冲突时主要遵循图表；3）提示干预可改变优先级，但效果高度依赖模型且方向不对称，高图表阅读能力不预示提示可控性。

Insight: 创新点在于提出了CVLAT框架和VFRI指标来解耦视觉与事实正确性，客观揭示了模型在冲突时的仲裁行为；可借鉴之处在于强调评估LVLMs时需同时考虑可视化素养和模型在视觉证据与事实先验分歧时的权衡机制，而非仅依赖准确率。

Abstract: Large Vision-Language Models (LVLMs) show strong visualization interpretation, yet it is unclear whether their responses reflect genuine reasoning over visual evidence or factual priors learned during training. Current evaluations mix these two sources, obscuring when correct visual interpretation is overridden by memorized facts. We present a framework that isolates visual correctness from factual correctness, revealing validity limitations in existing visualization literacy assessments. Across three experiments with 15 state-of-the-art LVLMs: (1) several models reach human-level performance on standard tests (VLAT), but this may reflect factual recall rather than visual understanding, while randomized-data tests (reVLAT) underestimate literacy when correct visual interpretation is superseded by factual priors. (2) Using our Counterfactual Visualization Literacy Assessment Test (CVLAT) with capability-normalized arbitration metrics, we classify models by the sign of their visual-factual reliance index (VFRI), revealing a visualization-oriented majority and a factual knowledge-oriented minority, though several near-zero cases warrant caution. A human baseline (N=30) on the same counterfactual items confirms that people overwhelmingly follow the chart under conflict, providing a human reference point. (3) Prompt-based intervention can shift prioritization, but its effectiveness is highly model-dependent and direction-asymmetric, and high chart-reading capability does not predict prompt-controllability. Overall, high visualization accuracy is not sufficient evidence of faithful visual reasoning: reliable integration into visual analytics requires evaluating not only visualization literacy but also how models arbitrate between visual evidence and factual priors when the two diverge. Benchmark and code: https://github.com/JaeyoungKim-HCIL/CVLAT

[54] JAVEDIT: Joint Audio-Visual Instruction-Guided Video Editing with Agentic Data Curation cs.CVPDF

Yinan Chen, Chuming Lin, Zhennan Chen, Yuxiang Zeng, Junwei Zhu

TL;DR: 本文提出了JAVEdit，一个用于指令引导的联合视听视频编辑的基线模型，并构建了首个大规模高质量数据集JAVEdit-100k和标准化评估基准JAVEditBench。该工作旨在解决联合视听编辑领域缺乏专用数据集和基准的问题。

Details

Motivation: 指令引导的视频编辑已取得显著进展，但联合视听编辑由于缺乏专用数据集和基准而受到限制。本文旨在填补这一空白。

Result: 实验表明，JAVEdit模型在六项评估指标中的五项上优于所有基线模型。

Insight: 主要创新点包括：1）构建了首个大规模、高质量的指令引导联合视听编辑数据集JAVEdit-100k，专注于以人为中心的视频，并通过精心设计的生成流程和智能体在环质量控制机制构建；2）引入了标准化的综合评估基准JAVEditBench；3）提出了该领域的首个基线模型JAVEdit。

Abstract: While instruction-based video editing has seen significant progress, joint audio-visual editing remains constrained by the absence of dedicated datasets and benchmarks. To bridge this gap, we present JAVEdit-100k, the first large-scale, high-quality dataset tailored for instruction-guided joint audio-visual editing. Focusing on human-centric videos, JAVEdit-100k comprises approximately 100K editing triplets spanning five distinct categories, including subject editing and speech editing. This dataset is rigorously constructed via four meticulously designed generation pipelines, seamlessly paired with an agent-in-the-loop quality control mechanism. Furthermore, to address the lack of standardized evaluation within the field, we introduce JAVEditBench, a comprehensive benchmark featuring curated source videos and human-aligned instructions across all editing categories. Finally, we propose JAVEdit, a pioneering baseline model for instruction-guided joint audio-visual editing. Experiments show that \model\ outperforms all baselines on five of six evaluation metrics.

[55] NVIDIA OmniDreams: Real-Time Generative World Model for Closed-Loop Autonomous Vehicle Simulation cs.CV | cs.AI | cs.ROPDF

NVIDIA, :, Aarti Basant, Amlan Kar, Despoina Paschalidou

TL;DR: 本文介绍了NVIDIA OmniDreams，一个用于闭环自动驾驶仿真的实时生成世界模型。该模型基于Cosmos扩散模型进行中后训练，能够自回归地生成以动作为条件的视频，合成传统仿真器难以捕捉的复杂、未见现象（如极端天气和不可预测的动态智能体行为）。在部署中，它与Alpamayo 1策略模型和AlpaSim编排器协同工作，作为一个高响应、反应式的环境，为下一代自动驾驶策略的训练和评估提供可扩展的全面解决方案。

Details

Motivation: 随着自动驾驶能力的发展，在长尾场景中安全评估驾驶策略仍然是一个关键瓶颈。基于重建的神经仿真器虽然能提供照片级真实感，但受限于初始捕获数据，难以泛化到高度动态或新颖场景。

Result: 初步结果表明，从OmniDreams后训练得到的世界-动作模型（WAM）在Physical AI Autonomous Vehicles NuRec数据集上取得了强劲性能，超越了基于VLA的Alpamayo 1.5研究策略模型，同时仅使用其总参数的五分之一。

Insight: 核心创新在于提出了一个实时生成世界模型，它利用大规模驾驶场景数据（21k小时）进行中后训练，能够自回归地、以动作为条件生成照片级传感器观测，从而克服了传统仿真器在泛化性和动态场景建模上的限制。此外，该模型本身也可作为策略架构的骨干网络，展示了生成世界模型在闭环仿真和策略学习中的双重潜力。

Abstract: As autonomous vehicle capabilities advance, the safe evaluation of driving policies in long-tail scenarios remains a critical bottleneck. In closed-loop simulation, the driving policy model actively interacts with the environment, where its actions dynamically update the simulator state and directly influence the next set of generated sensor observations. While recent reconstruction-based neural simulators offer photorealism, they are fundamentally constrained by their initial captured data and struggle to generalize to highly dynamic or novel scenes. To overcome these limitations, we introduce OmniDreams, a foundation generative world model mid- and post-trained from the Cosmos diffusion model to autoregressively generate action-conditioned videos in real time. By leveraging the rich visual priors of Cosmos and mid- and post-training on 21k hours of driving scenarios, OmniDreams synthesizes complex, unobserved phenomena that are hard for traditional simulators to capture, such as extreme weather and unpredictable dynamic agent behaviors. Crucially, it autoregressively conditions its photorealistic sensor generation on past frames, the current simulator state, and immediate driving actions. Deployed in a closed-loop system with the Alpamayo 1 policy model and AlpaSim orchestrator, OmniDreams acts as a highly responsive, reactive environment, providing a scalable and comprehensive solution for training and evaluating next-generation autonomous driving policies. We additionally show preliminary results indicating that a world-action model (WAM) post-trained from OmniDreams achieves strong performance on the Physical AI Autonomous Vehicles NuRec dataset, surpassing the VLA-based Alpamayo 1.5 research policy model while using only 1/5 the total parameters. These results highlight the potential for a real-time world model like OmniDreams to also serve as a backbone for policy architectures.

[56] GLINT: Sparsely Gated Vision-Language Alignment for Fine-Grained Radiology Representations cs.CV | cs.CL | cs.LGPDF

Jonggwon Park, Seongeun Lee, Junhyun Park, Hannah Yun, Hyunwoong Kim

TL;DR: 本文提出了GLINT框架，通过稀疏门控对齐机制，解决了放射学视觉语言模型中全局图像-报告配对与局部病灶区域不匹配的问题。该方法在2D胸片和3D胸部CT上均适用，实现了无需掩码监督的零样本分割，并在分类、报告生成和分割任务上优于现有方法。

Details

Motivation: 放射学视觉语言模型面临的核心挑战是：图像中的病灶仅占小区域，但监督信号仅存在于全局图像-报告层面，导致模型权重分散而非集中于与查询相关的稀疏区域。

Result: GLINT在零样本分类、定位和分割任务上表现优异，尤其在需要稀疏、查询特定定位的零样本定位和分割任务上提升显著。在下游评估中，其在分类、报告生成和分割任务上均超越了自监督学习编码器和医学视觉语言模型。

Insight: 创新点在于引入了稀疏门控对齐架构，通过门控机制显式激活与文本查询相关的图像块，并结合密集特征正则化来保持细粒度特征。这是首个在3D CT体积上实现无需掩码监督的零样本分割的方法。

Abstract: Vision-language models (VLMs) for radiology have emerged as a scalable paradigm by leveraging image-report pairs naturally produced in clinical workflows. However, this pairing reveals a mismatch in scale: each finding occupies only a small region of the image, yet supervision is provided only at the global image-report level. This poses a central challenge: prior approaches spread weight densely across all patches rather than concentrating on the sparse subset relevant to a given query. To address this, we present GLINT (Gated Language-Image alignmeNT), a framework that explicitly models this sparse correspondence. On the alignment side, we introduce Sparsely Gated Alignment, a novel architecture in which a sigmoid gate over a separate gate embedding space activates only the patches relevant to each textual query, enforcing explicit sparsity. On the representation side, we add Dense Feature Regularization, which anchors the trainable encoder’s intermediate features to a frozen self-supervised learning (SSL) teacher, preserving the fine-grained patch features that the gate relies on. The same recipe applies to both 2D chest X-ray (CXR) and 3D chest computed tomography (CT), built with DINOv3 and V-JEPA 2.1, respectively. GLINT enables zero-shot classification, grounding, and segmentation from free-text queries, and to our knowledge is the first to demonstrate zero-shot segmentation on 3D CT volumes without mask supervision. Notably, the most pronounced gains arise on zero-shot grounding and segmentation, where sparse, query-specific localization is required, consistent with our design intent. In downstream evaluation, GLINT outperforms both SSL encoders and medical VLMs on classification, report generation, and segmentation.

[57] Reinforcement Learning from Cross-domain Videos with Video Prediction Model cs.CV | cs.AIPDF

Zhao Yang, Xinrui Zu, Jacob E. Kooi, Thomas Delliaux, He Liu

TL;DR: 本文提出XIPER（跨域视频预测奖励）模型，用于从视觉差异大的专家视频中进行强化学习。该方法通过训练一个跨域视频预测模型，将智能体观测映射到专家域，并利用预测似然作为奖励信号，以解决跨域视频学习中的奖励信号缺失和域差距问题。

Details

Motivation: 解决从视觉差异大的专家视频中进行强化学习时，因缺乏奖励信号和存在域差距（如颜色、形态差异或仿真到现实的差距）而面临的挑战。

Result: 在DMC Color Suite（8个任务）和DMC Body Suite（3个任务）上的实验表明，XIPER在存在颜色和形态等域差距的情况下，始终优于基线方法。在仿真到现实转移数据集上的进一步分析显示，它能为真实机器人观测提供有意义的奖励信号，即使只使用仿真专家视频。

Insight: 创新点在于利用跨域视频预测模型将观测映射到专家域，并将预测似然作为奖励信号，这为跨域模仿学习提供了一种新颖的奖励建模方法，有效处理了视觉差异和域差距问题。

Abstract: Reinforcement learning from expert videos across visually distinct domains is challenging due to the absence of reward signals and the presence of domain gaps. We introduce XIPER (Cross-domain Video Prediction Reward), a reward model for learning from expert videos collected in a visually different domain, where the agent’s appearance differs due to factors such as color, morphology, or the sim-to-real gap. More specifically, XIPER trains a cross-domain video prediction model that maps agent observations into the expert domain and uses the prediction likelihood as a reward signal. Experiments on the DMC Color Suite (8 tasks) and DMC Body Suite (3 tasks) show that XIPER consistently outperforms baselines despite domain gaps such as differences in agent color and morphology. We further analyze XIPER on a sim-to-real transfer dataset, demonstrating that it produces meaningful reward signals for real-robot observations given only simulated expert videos. Code, pretrained models, datasets and video demonstrations can be found on our project webpage: https://sites.google.com/view/xiper

Xunyi Zhao, Sihao Lin, Gengze Zhou, Zerui Li, Shijie Li

TL;DR: 本文针对实例目标导航任务中的歧义问题，提出了一种成本感知的开放式交互方法。通过将交互导航重新定义为成本敏感的不确定性降低问题，作者构建了一个包含不同问题类型成本的诊断基准，并提出了加权成功率指标。此外，论文还引入了一种零样本多模态大语言模型导航器，该导航器仅在预期不确定性降低超过交互成本时进行选择性查询。

Details

Motivation: 现有交互式导航方法在处理自然语言描述模糊的实例目标导航时，未能区分不同查询类型的成本，导致代理可能通过重复高信息量问题来提升成功率，而非高效解决底层歧义。

Result: 作者构建了一个用于诊断交互行为和效率的基准，并提出了加权成功率指标来惩罚不同成本的查询。提出的零样本MLLM导航器在成本感知的交互设置下表现高效。

Insight: 创新点在于将交互导航形式化为成本敏感的不确定性降低问题，并通过信息增益分析从现有导航语料中推导出紧凑的问题类型集和权重，从而实现了更高效的交互策略。这为构建更贴近实际应用的、考虑交互成本的导航系统提供了新思路。

Abstract: Instance Goal Navigation (IGN) requires an embodied agent to find a specific object instance among distractors from an underspecified natural-language description. Such ambiguity often cannot be resolved from perception and language alone, making interaction with an oracle a natural mechanism for disambiguation. Prior interactive methods allow oracle queries but treat lightweight clarification and route-level guidance alike, letting agents boost success rate through repeated high-information questions rather than by resolving the underlying ambiguity efficiently. We recast interactive IGN as a cost-sensitive uncertainty-reduction problem, where the agent should ask the question whose answer provides the largest reduction in navigation uncertainty relative to its penalty. To this end, we apply an information-gain analysis on existing navigation corpora to identify which cues reduce navigation uncertainty, yielding a compact set of question types and data-derived weights.However, existing interactive navigation benchmarks do not model the cost of different question types or evaluate how efficiently agents use interaction, making them unsuitable for studying cost-sensitive interaction. Based on this taxonomy, we construct a benchmark for diagnosing interaction behavior and efficiency, together with a Weighted Success Rate metric that penalizes each query by its derived cost. We further propose a zero-shot MLLM navigator that selectively queries at each decision step only when the expected uncertainty reduction justifies the interaction cost.

[59] Follow-Your-Preference++: Rethinking Preference Alignment for Image Inpainting cs.CVPDF

Junkun Yuan, Yutao Shen, Toru Aonishi, Hideki Nakayama, Yue Ma

TL;DR: 本文重新审视了图像修复中的偏好对齐问题，采用直接偏好优化框架，利用公开奖励模型构建偏好训练数据，通过实证研究发现奖励模型存在偏差且易导致奖励黑客问题，并提出简单的集成方法以缓解偏差，最终在不修改模型架构或引入额外数据集的情况下，显著超越了现有SOTA模型。

Details

Motivation: 研究图像修复中的偏好对齐问题，从第一性原理出发重新评估其核心挑战，而非提出新方法。

Result: 在标准指标、大型视觉语言模型评估和人工评估中，模型大幅超越了先前的最先进模型。

Insight: 奖励模型存在明显的亮度、构图和色彩方案等偏差，易诱导奖励黑客；简单的奖励模型集成能有效缓解偏差并提升鲁棒性和泛化性能；偏好对齐可迁移至对象移除任务。

Abstract: We study preference alignment for image inpainting. Rather than proposing yet another method, we revisit the problem from first principles and reassess its core challenges. We adopt the widely used direct preference optimization framework and construct preference training data with publicly available reward models. Our empirical study spans nine reward models, two benchmarks, and two baseline inpainting models that differ in architecture and generative mechanism. Our main findings are: (1) Most reward models provide valid signals for preference data construction, although some are unreliable as evaluators. (2) Across models and benchmarks, preference data exhibits consistent trends under both candidate and sample scaling. (3) Reward models display pronounced biases–particularly in brightness, composition, and color scheme–that make them prone to inducing reward hacking. (4) A simple ensemble of reward models mitigates such biases and yields robust, generalizable performance. {\color{rebuttal_blue}(5) Preference alignment is transferable to the object removal task, where the goal shifts from open-ended creative generation to coherent background completion. (6) Further analysis reveals that a calibrated ensemble method further mitigates hacking and improves robustness.} Without modifying model architectures or introducing additional datasets, our models substantially outperform prior state-of-the-art models on standard metrics, large vision-language model evaluations, and human assessments. Our code is available at: https://github.com/shenytzzz/Follow-Your-Preference.

[60] PaddleOCR-VL-1.6: Expanding the Frontier of Document Parsing with Under-Optimized Region Refinement and Progressive Post-Training cs.CVPDF

Zelun Zhang, Hongen Liu, Suyin Liang, Yubo Zhang, Yiqing Xiang

TL;DR: 本文介绍了PaddleOCR-VL-1.6，这是一个基于PaddleOCR-VL-1.5升级的紧凑型文档解析模型。该模型通过引入区域感知数据优化框架来识别并增强前代模型的弱区域，并采用基于精选数据选择和强化学习的渐进式后训练方法，从而提升了模型性能。

Details

Motivation: 为了解决PaddleOCR-VL-1.5模型中存在的错误集中在模型行为不稳定、数据覆盖稀疏或监督信号不可靠的未优化区域的问题，而不是盲目扩大训练语料库。

Result: PaddleOCR-VL-1.6在OmniDocBench v1.6基准测试上取得了96.33%的新SOTA分数，并展现出与顶级视觉语言模型（VLMs）的强大竞争力。

Insight: 创新点在于提出了区域感知数据优化框架，针对性地增强模型弱区域，以及采用渐进式后训练方法，通过分阶段优化提升性能，为PaddleOCR-VL系列提供了实用的后训练方案。

Abstract: We introduce PaddleOCR-VL-1.6, an upgraded compact document parsing model built upon PaddleOCR-VL-1.5. Although PaddleOCR-VL-1.5 establishes a strong 0.9B baseline, its remaining errors concentrate in under-optimized regions where model behavior is unstable, data coverage is sparse, or supervision is unreliable. Rather than expanding the training corpus indiscriminately, PaddleOCR-VL-1.6 introduces a region-aware data optimization framework that identifies weak regions from the previous model, applies targeted enhancement to these regions, and improves the reliability of supervision signals. It further adopts a progressive post-training recipe based on curated data selection and reinforcement learning, pushing model performance to a higher level through staged optimization. PaddleOCR-VL-1.6 achieves a new state-of-the-art score of 96.33% on OmniDocBench v1.6, demonstrates strong competitiveness against top-tier VLMs, and provides a practical post-training recipe for the PaddleOCR-VL series.

[61] MemoGen: Can Past Experience Improve Future Text-to-Image Generation? cs.CVPDF

Wenshuo Chen, Kuimou Yu, Bowen Tian, Jianfei Song, Shaofeng Liang

TL;DR: 本文提出MemoGen，一种无需训练的框架，通过为现有文本到图像生成器添加智能进化层，使系统能够从自身生成经验中持续学习。该框架通过推断视觉需求、检索外部证据、转化为可执行约束、评估结果，并将任务理解、参考选择、视觉反馈、成功策略和失败教训存储为可重用经验记忆，从而在测试时实现自我进化，无需更新模型参数。

Details

Motivation: 现有文本到图像模型在处理需要隐含视觉约束、关系推理或外部知识的提示时仍不可靠，且现有检索增强和智能生成方法通常将每次生成视为孤立事件，未能系统性地保存过去成功或失败的经验以供未来使用。

Result: 在知识密集型和推理导向的基准测试（如WISE和Mind-Bench）上进行广泛实验，仅经过两轮进化后，基于开源Qwen-Image骨干的MemoGen超越了Nano Banana Pro和GPT-Image-1等强大的专有系统。

Insight: 创新点在于引入显式经验记忆作为持续学习信号，通过智能进化层实现测试时自我进化，无需参数更新；客观分析认为，将生成过程视为可积累经验的连续任务，并系统化存储和复用成功策略与失败教训，是提升文本到图像生成可靠性的有效途径。

Abstract: Modern text-to-image models have achieved strong visual synthesis, yet remain unreliable when prompts require implicit visual constraints, relational reasoning, or external knowledge. Existing retrieval-augmented and agentic generation methods mitigate this issue by acquiring external knowledge, references, or refined prompts for the current request, yet they typically treat each generation as an isolated episode and do not systematically preserve past successes or failures for future use. In this work, we ask whether a text-to-image system can continually improve from its own generation experience without updating the underlying generator. We propose MemoGen, a training-free framework that augments existing image generators with an agentic evolution layer. For each task, MemoGen explicitly infers visual requirements, retrieves external evidence and references when necessary, translates them into executable generation constraints, evaluates the generated result, and stores task understanding, reference choices, visual feedback, successful strategies, and failure lessons as reusable experience memory. Across evolution rounds, the agent retrieves relevant experience to improve similar future generations, selectively repairing previously failed cases while preserving successful ones, thereby enabling test-time self-evolution without parameter updates. Extensive experiments on knowledge-intensive and reasoning-oriented benchmarks demonstrate the effectiveness of this paradigm: after only two evolution rounds, MemoGen built upon the open-source Qwen-Image backbone surpasses strong proprietary systems such as Nano Banana Pro and GPT-Image-1 on WISE and Mind-Bench, showing that explicit experience memory can serve as a powerful continual learning signal for reliable text-to-image generation.

[62] VistaHop: Benchmarking Multi-hop Visual Reasoning for Visual DeepSearch cs.CV | cs.AI | cs.CLPDF

Hang He, Chuhuai Yue, Chengqi Dong, Chengcheng Wan, Ting Su

TL;DR: 本文提出了VistaHop基准测试，用于评估视觉深度搜索中的多跳视觉推理能力。该基准包含300张高分辨率图像、25个视觉搜索场景和350个多跳问答任务，要求模型从视觉锚点追踪证据链或融合多路径信息。同时开发了VistaArena评估环境，支持工具增强推理。实验表明当前多模态大推理模型在视觉锚定、证据回溯和长链推理等方面仍存在明显不足。

Details

Motivation: 现有基准主要关注单步视觉理解或静态图像问答，缺乏对迭代图像检查、视觉锚点定位和多跳证据整合的评估，无法充分衡量视觉深度搜索所需的多跳视觉推理能力。

Result: 在七个代表性多模态大推理模型上的实验显示，当前模型远未解决VistaHop任务，最佳模型SenseNova-MARS-32B仅获得24.31%的Pass@1准确率。

Insight: 创新点在于构建了首个专注于视觉深度搜索中多跳推理的基准测试，并设计了支持工具交互的统一评估环境。该工作揭示了当前模型在视觉锚定、证据回溯、长链推理和多锚点信息融合等核心能力上的系统性缺陷，为未来模型训练和评估提供了重要方向。

Abstract: Visual DeepSearch requires multimodal large reasoning model (MLRM) agents to answer complex visual queries by repeatedly inspecting image regions, grounding intermediate reasoning in visual evidence, and connecting fine-grained clues across long reasoning chains. However, existing benchmarks mainly focus on single-step visual understanding or static image-question answering, offering limited evaluation of iterative image inspection, visual-anchor grounding, and multi-hop evidence integration. In this work, we introduce VistaHop, a benchmark for evaluating vision-centric search and multi-hop visual reasoning in Visual DeepSearch. VistaHop contains 300 high-resolution images, 25 visual search scenarios, and 350 multi-hop QA tasks that require models to follow evidence chains from visual anchors or fuse information across multiple image-grounded reasoning paths. We further develop VistaArena, a unified evaluation environment that supports tool-augmented reasoning with text search, image search, image cropping, and evidence-based answer validation. Experiments on seven representative MLRMs show that current models remain far from solving VistaHop: the best model, SenseNova-MARS-32B, achieves only 24.31% Pass@1. These results reveal persistent limitations in visual grounding, evidence revisiting, long-chain reasoning, and multi-anchor information fusion, highlighting the need for stronger benchmarks and training methods for Visual DeepSearch.

[63] Cross-Modality Feature Fusion Based on Structured State Space Duality for Multimodal Image Registration Network cs.CVPDF

Zhikang Li, Yan Wu, Xin Hu, Yi Dai, Ming Li

TL;DR: 本文提出了一种名为RegNetMamba-2的新型多模态图像配准算法，其核心创新在于将结构化状态空间对偶性（SSD）引入到从粗到精的匹配流程中，以高效提取局部和全局结构特征。算法通过在不同尺度应用SSD进行特征提取，并设计了基于SSD的跨模态特征融合模型（包含跨模态交互模块和多尺度融合模块），最终在多个数据集上实现了性能与效率的良好平衡。

Details

Motivation: 多模态图像配准的主要挑战在于提取共享的结构信息。与Transformer相比，结构化状态空间对偶性（SSD）在训练和推理时能以更高效率提供更强的全局结构特征提取能力，因此本文旨在利用SSD的优势来解决多模态配准问题。

Result: 在VIS-SAR、VIS-IR和VIS-NIR等多个数据集上的大量实验表明，与当前最先进的基于深度学习的算法相比，RegNetMamba-2在多模态图像配准任务中，在性能和效率方面均取得了良好效果。

Insight: 论文的主要创新点是将高效的SSD架构引入多模态配准，并设计了专门的跨模态特征交互（CMI）和多尺度特征融合（MSF）模块。从客观角度看，其将SSD的全局建模能力与对局部边缘/结构信息的强调相结合，并构建了系统的跨尺度融合流程，为多模态特征对齐提供了一种新颖且高效的解决方案。

Abstract: In multi-modal image registration, the primary challenge lies in shared structural information extraction. Compared to Transformers, Structured State Space Duality (SSD) offers greater global structural feature extraction with higher efficiency during training and inference. Inspired by these advantages, we propose a novel algorithm for multi-modal image registration, named RegNetMamba-2. Our algorithm incorporates SSD into coarse-to-fine matching process to extract local and global structural features effectively. Firstly, SSD is applied in three different scales for multi-modal feature extraction in our network. To strengthen local representation, we pay more attention on foreground edge and structural information by feature scaling function of SSD. Secondly, for shared feature extraction of input images and multi-modal feature fusion in all scales, we propose cross-modality feature fusion model based on SSD, consisting of Cross-Modality feature Interaction (CMI) module and Multi-Scale feature Fusion (MSF) module. CMI module is designed for cross-modality feature extraction of each scale by SSD in cross form. MSF module is designed to employ a progressive upward fusion in feature-level to obtain fine features, consisting of multi-modal features in all scales. Following coarse-to-fine, the features in 1/8 scale from CMI and 1/2 scale from MSF are collected to calculate matching probability scores. Then we respectively establish matching process by correspondences of pixel-wise. Extensive experiments demonstrate that comparing with state-of-the-art deep-learning based algorithms, RegNetMamba-2 has achieved good effects in both performance and efficiency for multi-modal image registration on the following datasets: VIS-SAR (OSDataset), VIS-IR (LGHD/RoadSence) and VIS-NIR (RGB-NIR sense).

[64] Beyond Semantics: Modeling Factual and Affective Perceptual Experiences from Vision-Language Data cs.CV | cs.CL | cs.CYPDF

Youssef Mohamed, Kenneth Ward Church, Mohamed Elhoseiny

TL;DR: 本文提出了P-Topics建模这一新问题，旨在从视觉-语言数据中理解图像如何被感知，包括客观事实和主观情感两个方面。作者引入了PercepT（Perception topic Transformer）这一两阶段架构，用于发现和建模数据集中的感知体验，并将图像关联到相关的感知簇。

Details

Motivation: 动机在于超越传统的语义理解，研究图像如何被不同文化和情感背景的人所感知，即同时捕捉图像中的客观事实和主观情感体验。

Result: 在ArtELingo数据集上，PercepT在形成感知簇方面取得了0.97的轮廓系数（Silhouette score），显著优于基线方法的0.37；在映射图像到感知簇方面取得了0.94的AUC分数，优于基线的0.77，达到了SOTA水平。人类评估也证实了其有效性。

Insight: 创新点在于将感知建模形式化为一个包含事实和情感两个维度的主题发现与映射问题，并提出了一个无监督的两阶段Transformer架构，能够动态确定感知簇的数量以匹配数据集的感知丰富度。

Abstract: We present P-Topics (Perception Topics) modeling, a novel problem for understanding how images are perceived affectively and across cultures. The goal is to (1) discover and model the different perception experiences in a dataset of images and captions, where each experience is defined by an objective factual and a subjective affective aspect, and (2) associate images to their relevant perception experiences. We introduce PercepT (Perception topic Transformer), a two-stage architecture that tackles P-Topics modeling. In the formation stage, percepT discovers P-Topics as visual-textual clusters using an unsupervised training objective, and dynamically selects the number of clusters to match the perceptual richness of the dataset. In the mapping stage, it learns P-Topic mapping functions via attention pooling to associate images to their respective clusters. On ArtELingo, PercepT achieves a silhouette score of 0.97 compared to 0.37 from the closest baseline reflecting better perceptual clusters. PercepT also achieves an AUC score of 0.94 compared to 0.77 showing better mapping to perceptual clusters. Human evaluation confirms that PercepT captures semantically meaningful perception experiences and significantly outperforms existing methods. Our implementation will be made public.

[65] SynCred-Bench: Benchmarking Synthetic Credibility in AI-Generated Visual Misinformation cs.CV | cs.AIPDF

Junxiao Yang, Minghao Zhang, Xiaoce Wang, Haoran Liu, Shiyao Cui

TL;DR: 该论文提出了SynCred-Bench基准测试，用于评估AI生成的视觉虚假信息中的‘合成可信度’问题。该基准包含600张AI生成的虚假信息图像，涵盖六种可信形式类别和七种细粒度传播风格，并提供了一个450张真实图像的负样本集（FP450）以测量误报率。评估发现，现有检测系统（包括MLLMs、开源AIGC检测器和商业API）以及人类标注者在识别此类合成可信度方面表现均不佳，突显了这是一个严重且未被充分探索的挑战。

Details

Motivation: 动机在于解决由生成模型（能生成带有逼真嵌入文本和布局的视觉内容）所引发的新型虚假信息威胁——‘合成可信度’，即利用看似可信的视觉形式（如新闻标题、收据）来传播虚假信息。

Result: 在5%的误报率约束下，评估结果显示：15个MLLMs的平均真正例率（TPR）仅为10.5%，开源AIGC检测器的TPR低于5%，商业API的TPR达到57.6%，而人类标注者的TPR也只有63%。这些结果表明现有系统在SynCred-Bench基准上均不可靠。

Insight: 论文的创新点在于首次系统性地定义并构建了针对‘合成可信度’的基准测试（SynCred-Bench），强调了需要开发超越表面可信度线索进行推理的检测器。从客观角度看，该研究为评估和推动针对新型AI生成视觉虚假信息的检测技术提供了重要的数据集和性能基线。

Abstract: Recent generative models can now produce visual artifacts with realistic embedded text and layouts, creating a new misinformation threat: synthetic credibility. We introduce SYNCRED-Bench, a benchmark of 600 AI-generated misinformation images balanced across six credible-form categories and seven fine-grained circulation styles, together with FP450, a real-image negative set for measuring false positives. Extensive evaluation shows that existing systems remain unreliable: under a 5% false-positive-rate constraint, 15 MLLMs achieve only 10.5% true positive rate (TPR), open-source AIGC detectors achieve less than 5%, and commercial APIs reach 57.6%. Human annotators also struggled to identify synthetic credibility, reaching only 63% TPR. These findings establish synthetic credibility as a severe and underexplored visual misinformation challenge, and provide a benchmark for developing detectors that reason beyond superficial credibility cues.

[66] P\textsuperscript{2}-DPO: Grounding Hallucination in Perceptual Processing via Calibration Direct Preference Optimization cs.CV | cs.AI | cs.CL | cs.LGPDF

Ruipeng Zhang, Zhihao Li, Haozhang Yuan, C. L. Philip Chen, Tong Zhang

TL;DR: 该论文提出了一种名为P²-DPO的新型训练范式，旨在解决大型视觉语言模型中的幻觉问题。该方法通过让模型生成并学习自己的偏好对，直接针对视觉感知瓶颈（如关注区域和图像退化鲁棒性）进行优化，并引入了校准损失来精确对齐视觉信号与文本生成。

Details

Motivation: 现有直接偏好优化方法未能专门针对视觉感知瓶颈（如关注区域感知不足和图像退化下的视觉鲁棒性不足），且现有偏好对通常是视觉无关的、离策略的，限制了其指导模型学习的有效性。

Result: 在可比的训练数据和成本下，P²-DPO在基准测试上超越了依赖昂贵人类反馈的强基线模型。在注意力区域保真度和图像退化场景下的评估验证了其在解决关注区域感知瓶颈和提升对退化输入的视觉鲁棒性方面的有效性。

Insight: 创新点在于提出了一种在策略的偏好对构建方法，专门针对“聚焦与增强”感知和视觉鲁棒性，并设计了一个校准损失来精确对齐视觉与文本的因果生成，从而更直接地解决视觉幻觉的根源问题。

Abstract: Hallucination has recently garnered significant research attention in Large Vision-Language Models (LVLMs). Direct Preference Optimization (DPO) aims to learn directly from the corrected preferences provided by humans, thereby addressing the hallucination issue. Despite its success, this paradigm has yet to specifically target the perceptual bottleneck in attended regions or address insufficient Visual Robustness against image degradation. Furthermore, existing preference pairs are often vision-agnostic and their inherently off-policy nature limits their effectiveness in guiding model learning. To address these challenges, we propose Perceptual Processing Direct Preference Optimization (P\textsuperscript{2}-DPO), a novel training paradigm in which the model generates and learns from its own preference pairs, thereby directly addressing the identified visual bottlenecks while inherently avoiding the issues of vision-agnostic and off-policy data. It introduces: (1) an on-policy preference pairs construction method targeting Focus-and-Enhance perception and Visual Robustness, and (2) a well-designed Calibration Loss to precisely align visual signals with the causal generation of text. Experimental results demonstrate that with a comparable amount of training data and cost, P\textsuperscript{2}-DPO outperforms strong baselines that rely on costly human feedback on benchmarks. Furthermore, evaluations on Attention Region Fidelity (ARF) and image degradation scenarios validate the effectiveness of P\textsuperscript{2}-DPO in addressing perceptual bottleneck in attended regions and improving Visual Robustness against degraded inputs.

[67] Towards Characterizing Scientific Image Utility and Upgradability cs.CVPDF

WenZhe Li, Qihang Yan, Liang Chen, Junying Wang, Farong Wen

TL;DR: 本文提出SIU^2A框架，用于评估科学图像的实用性和可升级性，以应对AI生成内容对科学图像完整性带来的威胁。该框架包含两个维度：实用性（错误检测与修复可行性）和可升级性（修正质量），并构建了SIU^2A-Benchmark数据集进行验证。实验表明当前多模态系统在科学错误评估和忠实修正方面存在显著局限。

Details

Motivation: 解决现有评估范式（如感知质量指标和语言模型）无法有效评估科学图像科学有效性的问题，以应对AI生成内容引入的微妙但严重的错误对研究通信中关键证据的威胁。

Result: 在SIU^2A-Benchmark数据集上的实验显示，当前多模态系统在科学错误评估和忠实修正方面表现不佳，揭示了视觉感知与科学可用性之间的根本差距。

Insight: 创新点在于提出了专门针对科学图像的实用性和可升级性评估框架，将图像损坏分为四种基本类型（细节失真、不完整性、虚假内容和实体混淆），并设计了两阶段评估协议（实用性和可升级性阶段），为科学图像完整性评估提供了新范式。

Abstract: Scientific images function as critical evidence in research communication, yet their integrity faces unprecedented threats from AI-generated content that introduces subtle but consequential errors. Existing evaluation paradigms prove inadequate: perceptual quality metrics poorly correlate with scientific validity, while language models lack domain-specific verification capabilities. To address this gap, we propose the \textbf{S}cientific \textbf{I}mage \textbf{U}tility and \textbf{U}pgradability \textbf{A}ssessment (\textbf{SIU$^2$A}) framework, which introduces two complementary dimensions for scientific image evaluation. \textbf{Utility} encompasses \textit{error detection} (identifying scientific inaccuracies) and \textit{correction feasibility} (assessing whether errors can be reliably repaired). \textbf{Upgradability} measures the quality of correction. We categorize scientific image corruption into four fundamental types: Detail Distortion, Incompleteness, False Content, and Entity Confusion. Based on this taxonomy, we construct SIU$^2$A-Benchmark, a dataset with expert annotations for error identification and repair. The framework implements a two-stage evaluation protocol: the \textit{Utility} stage evaluates error detection capability and repair instruction generation, while the \textit{Upgradability} stage assesses whether corrections faithfully restore scientific validity without compromising existing accurate information. Experiments reveal that current multimodal systems exhibit significant limitations in both scientific error assessment and faithful correction, exposing a fundamental gap between visual perception and scientific usability.

[68] Mamba-Enhanced Implicit Motion Learning for Audio-Driven Portrait Animation cs.CVPDF

Xuan Wei, Jiahui Chen, Kaiheng Li, Mingyu Shao, Qingqi Hong

TL;DR: 本文提出了一种新颖的隐式运动框架，用于从单张静态图像和音频生成真实且时序一致的人体运动视频。该方法采用两阶段流程，将运动预测与渲染解耦：第一阶段整合外观先验和分层深度线索到区域感知注意力机制中，以建模潜在运动特征；第二阶段采用Mamba增强的扩散模型直接从音频和源图像预测这些特征，实现了细粒度运动模式的无监督学习。

Details

Motivation: 解决现有基于关键点的方法在捕捉细微运动动态方面的不足，旨在生成更真实、时序更连贯的音频驱动人体动画。

Result: 在一个新的380小时高质量数据集上训练，该方法在多个公共基准测试和自收集数据上，在准确性、自然度和时序一致性方面均优于先前工作，达到了新的最先进水平。

Insight: 主要创新点在于将运动预测与渲染解耦的两阶段架构，以及将Mamba模型集成到扩散过程中以增强对时序音频特征的建模能力，从而实现了对细粒度、隐式运动模式的无监督学习。

Abstract: Audio-driven human motion video generation aims to synthesize realistic and temporally coherent human animations from a single static image, with applications in talking-head synthesis, co-speech gesture generation, and dynamic presentations. Moving beyond conventional keypoint-based methods that often struggle to capture subtle motion dynamics, We propose a novel implicit-motion framework for generating realistic and temporally coherent human motion videos from a single static image and audio. Our approach uses a two-stage pipeline that decouples motion prediction from rendering. The first stage integrates appearance priors and hierarchical depth cues into a region-aware attention mechanism to model latent motion features. The second stage employs a Mamba-enhanced diffusion model to directly predict these features from audio and the source image, enabling unsupervised learning of fine-grained motion patterns. This decoupled architecture enhances flexibility and efficiency. Trained on a new 380-hour high-quality dataset, our method outperforms prior work across multiple public benchmarks and our collected data in accuracy, naturalness, and temporal coherence, setting a new state-of-the-art.

[69] SAMatcher: Co-Visibility Modeling with Segment Anything for Robust Feature Matching cs.CVPDF

Xu Pan, Qiyuan Ma, Mingyue Dong, He Chen, Wei Ji

TL;DR: SAMatcher是一个基于Segment Anything Model（SAM）的特征匹配框架，通过显式建模跨视图的共同可见区域来解决图像对应点估计问题。它首先预测共同可见区域掩码和边界框作为结构化先验，然后利用对称的跨视图交互机制进行特征对齐和匹配。

Details

Motivation: 现有基于学习的方法大多在像素或图像块级别操作，缺乏对跨视图共同可见区域的显式建模，这限制了在存在大视角和尺度变化时的鲁棒性。论文旨在通过引入共同可见性建模来提升特征匹配的可靠性。

Result: 在具有挑战性的基准测试上进行了广泛实验，结果表明SAMatcher相比现有匹配流程有显著提升，特别是在大视角和尺度变化下表现优异。

Insight: 核心创新在于将原本为单目分割设计的基础模型（SAM）通过显式的共同可见性建模扩展到多视图对应关系推理中，并提出了统一的掩码预测与边界框定位联合监督方案。这为图像匹配的结构化表示学习提供了新视角。

Abstract: Reliable correspondence estimation is a fundamental problem in image processing, underpinning applications such as Structure from Motion, visual localization, and image registration. Existing learning-based methods have significantly improved local feature representations, yet most still operate at the pixel or patch level and lack explicit modeling of regions that are jointly visible across views. We propose SAMatcher, a feature matching framework that formulates correspondence estimation through co-visibility modeling. Instead of directly matching local features, SAMatcher first predicts co-visible region masks and bounding boxes as structured priors for correspondence estimation. Built upon the Segment Anything Model (SAM), it introduces a symmetric cross-view interaction mechanism that enables bidirectional feature exchange and cross-view semantic alignment. We further develop a unified supervision scheme that jointly optimizes mask prediction and box localization through mask learning, box regression, and mask-box consistency constraints. Extensive experiments on challenging benchmarks demonstrate substantial improvements over existing matching pipelines, particularly under large viewpoint and scale variations. Our results show that foundation models originally designed for monocular segmentation can be effectively extended to multi-view correspondence reasoning through explicit co-visibility modeling, offering a new perspective on structured representation learning for image matching. Code and project page: https://xupan.top/Projects/samatcher

[70] A unified multi-task framework enables interpretable chest radiograph analysis cs.CVPDF

Lijian Xu, Ziyu Ni, Xinglong Liu, Xiaosong Wang, Hongsheng Li

TL;DR: 本文提出了IMT-CXR（可解释多任务胸部X光分析Transformer）框架，通过模拟放射科医生的诊断流程，在统一架构中依次执行疾病分类、病灶定位、解剖分割和报告生成四个临床任务。该框架采用医学领域指令调优优化，在十个CXR基准测试中展现出竞争力，并在多中心临床评估中生成66%可媲美或超越原始临床报告的AI报告。

Details

Motivation: 针对现有黑盒医疗影像分析系统多局限于孤立任务、忽视临床诊断多任务流程中可信度敏感性的问题，旨在构建一个可追溯决策路径的、模拟放射科医生工作流的可解释多任务框架。

Result: 在十个胸部X光基准测试的直接推理和微调设置下均表现出有竞争力的性能；在四家医疗中心160份历史报告的盲评中，三位放射科医生认为66%的AI生成报告在诊断清晰度上可比或优于原始临床报告。

Insight: 创新性地将放射科医生诊断流程建模为证据驱动的三阶段（疾病识别、属性表征、证据整合报告生成），并通过统一Transformer架构和医学指令调优实现多任务协同；其核心在于建立了从解剖发现到结论的可追溯诊断路径，弥合了AI技术指标与临床效用之间的鸿沟。

Abstract: While multimodal deep learning has advanced medical imaging analysis, existing black-box systems \textcolor{black}{may remain confined to isolated tasks, often overlooking} the trust-sensitive nature of clinical diagnosis as a multi-task process. We propose IMT-CXR (Interpretable Multi-task Transformer for Chest X-ray Analysis), a framework that emulates radiologists’ diagnostic workflow through three evidence-driven stages: 1) Disease recognition; 2) Attribute characterization (e.g., size, location, severity quantification); 3) Evidence-integrated report generation with traceable decision pathways. The framework employs a unified transformer architecture optimized via medical-domain instruction tuning, sequentially executing four clinical tasks: multi-label disease classification, lesion localization, anatomical segmentation, and radiology report generation. Experimental validation demonstrates competitive performance on ten CXR benchmarks under direct inference and fine-tuning settings. In a blinded evaluation of 160 historical reports from four medical centers, three radiologists rated 66% of AI-generated reports as comparable to or surpassing original clinical reports in diagnostic clarity, highlighting the framework’s translational potential. By establishing traceable diagnostic pathways from anatomical findings to conclusions, this work bridges the gap between AI technical metrics and clinical utility, advancing trustworthy AI systems in medical imaging.

[71] Enginuity: A Dataset and Benchmark for Vision-Language Understanding of Engineering Diagrams cs.CVPDF

Abhishek Kumar, Isha Motiyani, Tilak Kasturi, Ethan Seefried, Prahitha Movva

TL;DR: 论文提出了Enginuity，这是首个用于评估视觉语言模型在复杂工程图表理解能力的公开数据集和基准。该基准包含两个任务：从美国军事维修手册中提取结构化零件表（Task 1）和进行自由形式的视觉问答（Task 2）。研究评估了多个前沿VLM模型，揭示了它们在零件识别与描述准确性以及事实推理方面存在的系统性差距。

Details

Motivation: 工程图表因其密集的空间布局、领域特定符号以及视觉标注与结构化零件表之间的交叉引用，对视觉语言模型构成了独特挑战。然而，目前缺乏公开的基准来评估VLM在此领域的性能，现有数据集主要关注流程图、科学图表或商业文档。

Result: 在Task 1中，模型在Recall@all指标上达到0.61-0.87，但Token F1pen仅为0.03-0.18，表明零件识别与描述准确性之间存在巨大差距。Task 2显示所有模型在事实推理方面均存在一致性的不足。分析还发现，基于词符重叠的指标相对于语义相似度，低估了模型在技术描述上的能力达2-6倍。

Insight: 论文的主要创新点是创建了首个专注于工程图表理解的公开数据集和基准（Enginuity），填补了该领域的空白。客观分析认为，其提出的任务设计（结构化提取与自由形式VQA）能有效暴露VLM在技术文档处理中的系统性弱点，并且倡导使用LLM-as-judge进行领域特定评估的校准方法，为后续研究提供了重要方向。

Abstract: Engineering diagrams pose a distinct challenge for vision-language models: unlike natural images or general documents, they encode information through dense spatial layouts, domain-specific symbols, and cross-references between visual callouts and structured parts tables. Despite their centrality to service, repair, and design workflows, there is no public benchmark for measuring VLM capabilities in this domain; existing datasets primarily focus on flowcharts, scientific figures, or business documents. To address this gap, we introduce Enginuity, the first open dataset and benchmark for evaluating VLMs on complex engineering diagrams. We define two tasks over a corpus of U.S. military service and repair manuals: structured parts-table extraction (Task 1) and free-form visual diagram question answering (VQA)(Task 2) for benchmarking. We evaluate four frontier VLMs (GPT-5.2 Chat, Claude Opus 4.7, Gemma 4, Qwen3-VL-32B-Instruct) under zero-shot and chain-of-thought prompting. On Task 1, models reach Recall@all of 0.61-0.87 but Token F1pen of only 0.03-0.18, exposing a systematic gap between part identification and description fidelity. Task 2 reveals a consistent factual-reasoning gap across all models. A supporting analysis shows that token-overlap metrics under-report model capability on technical descriptions by 2-6x relative to semantic similarity, motivating LLM-as-judge calibration for domain-specific evaluation. We release the dataset, annotations, evaluation harness, and per-sample model outputs to support a reproducible study of VLM capability on engineering content.

[72] IDO: Incongruity-aware Distribution Optimization for Multimodal Fake News Detection cs.CVPDF

Hengyang Zhou, Rongman Hong, Yuxuan Zhou, Jing Wang, Zhaoyan Pan

TL;DR: 本文提出了一种名为IDO（Incongruity-aware Distribution Optimization）的方法，用于提升多模态假新闻检测的性能。该方法从事实不一致性和模态不一致性两个角度出发，通过通道重加权策略和高斯分布建模来处理事实不一致性，并利用不一致性对比学习来学习跨模态语义信息。实验表明，IDO在假新闻检测任务上达到了最先进的性能。

Details

Motivation: 现有方法主要关注跨模态一致性，但未能显式建模虚假多模态内容中的语义不一致性，而错误信息往往包含与事实不符的语义信息，因此需要解决这一挑战。

Result: 实验证明IDO在假新闻检测任务上取得了最先进的（SOTA）性能。

Insight: 创新点在于明确建模了事实不一致性和模态不一致性，具体通过通道重加权策略和高斯分布来捕捉事实不一致性带来的不确定相关性，并利用不一致性对比学习来增强跨模态语义理解。

Abstract: Multimodal fake news detection aims to identify the authenticity of news. Existing multimodal fake news detection methods mainly focus on cross-modal consistency, but often fail to explicitly model the semantic incongruity that characterizes deceptive multimodal content. However, misinformation often contains semantic information incongruity with the facts. To address these challenges, we propose Incongruity-aware Distribution Optimization (IDO) to improve the performance of fake news detection from the perspectives of factual incongruity and modality incongruity. For factual incongruity, we introduce a channel-wise reweighting strategy to obtain semantically discriminative embeddings and utilize gaussian distribution to model the uncertain correlation caused by factual incongruity. For modality incongruity, we utilize incongruity contrastive learning to learn cross-modal semantic information. Experiments demonstrate that IDO achieves state-of-the-art performance.

[73] PRISM: Synergizing Vision Foundation Models via Self-organized Expert Specialization cs.CV | cs.AIPDF

Ying Tang, Dong Li, Youjia Zhang, Zikai Song, Junqing Yu

TL;DR: 论文提出PRISM框架，通过自组织专家专业化协同多种视觉基础模型（VFMs）。该框架采用双流混合专家（MoE）架构，分两阶段实现：首先进行专业知识解构，通过教师条件路由器引导专家专注于不同表示子空间以减少干扰；随后进行动态重组，路由器学习为下游任务组装专家形成定制化计算路径。

Details

Motivation: 旨在解决将多种视觉基础模型的互补优势统一到单个高效模型时，由于整体蒸馏导致的负迁移和特征冲突问题。

Result: 在PASCAL-Context和NYUD-v2基准测试上取得了新的最先进（SOTA）性能。

Insight: 创新点在于提出模块化专业化方法，通过稀疏、涌现的专家专业化来整合多样化的视觉知识，这是一种可扩展的集成策略。

Abstract: Unifying the complementary strengths of diverse Vision Foundation Models (VFMs) into a single efficient model is highly desirable but challenged by the negative transfer inherent in monolithic distillation. To address these feature conflicts, we introduce \textbf{PRISM}, a novel dual-stream Mixture-of-Experts (MoE) framework that synergizes VFMs via modular specialization. We propose a two-stage paradigm: (1) expertise deconstruction, where a teacher-conditional router guides experts to specialize in distinct representational subspaces to mitigate interference, followed by (2) dynamic recomposition, where the router learns to assemble these experts into tailored computational pathways for downstream tasks. Experiments on PASCAL-Context and NYUD-v2 show that \textbf{PRISM} establishes a new state of the art, validating that sparse, emergent specialization is a scalable approach for integrating diverse visual knowledge.

[74] Mixed-Modality Dual Face-Hair Retrieval cs.CVPDF

Quoc-Anh Bui-Huynh, Mai-Tuyen Lam, Dai-Anh-Tuan Nguyen, Thanh Duc Ngo

TL;DR: 该论文提出了双人脸-发型检索（DFHR）任务，这是一个新的混合模态双参考图像检索任务，查询由一个指定身份的人脸图像和一个以图像或文本表达的发型参考组成。论文构建了首个混合模态人脸-发型检索基准DFHR-Bench，包含超过18万个标注三元组，并提出了统一框架MFHC，通过令牌注入和多视图监督融合解耦的身份和发型嵌入。

Details

Motivation: 解决现有检索设置中无法同时处理来自异构模态的两个语义独立属性（身份和发型）进行跨组件推理的问题，需要实现局部特征解耦、跨模态语义对齐和混合模态组合。

Result: 在构建的DFHR-Bench基准上评估，该基准包含双图像和图像-文本两种设置，确保了语义和身份完整性，为任务建立了新的评估范式。

Insight: 创新点在于提出了混合模态双参考检索任务DFHR，它要求在一个统一的嵌入空间中实现身份和发型的解耦与组合；提出的MFHC框架通过令牌注入和多视图监督有效融合多模态信息，为身份感知、属性可控的跨模态视觉检索建立了新范式。

Abstract: We introduce Dual Face-Hair Retrieval (DFHR), a new mixed-modality dual-reference task in image retrieval where a query consists of a face image specifying identity and a hairstyle reference expressed as either an image or text. Unlike prior retrieval settings, DFHR requires cross-component reasoning between two semantically independent attributes – identity and hairstyle – originating from heterogeneous modalities. This formulation demands localized feature disentanglement, cross-modal semantic alignment, and mixed-modality composition within a unified embedding space. We construct DFHR-Bench, the first benchmark for mixed-modality face-hair retrieval, comprising over 180K annotated triplets across dual-image and image-text settings, built via a multi-stage annotation protocol ensuring semantic and identity integrity. We further propose MFHC (Multimodal Face-Hair Combiner), a unified framework that fuses disentangled identity and hairstyle embeddings through token injection and multi-view supervision. DFHR and DFHR-Bench together establish a new paradigm for identity-aware, attribute-controllable visual retrieval across modalities.

[75] From 3D Perception to Safety Reasoning: A Graph-Based Framework for Real-Time Underground Mine Monitoring cs.CVPDF

Pasindu Ranasinghe, Simit Raval, Dibyayan Patra, Bikram Banerjee, Ismet Canbulat

TL;DR: 本文提出了一种基于图结构的实时地下煤矿安全监控框架，该框架将彩色3D点云转化为结构化、可追溯的安全推理输出。它集成了3D语义感知、基于不确定性的异常检测、基于规则的危害检查、设备端LLM推理和基于GraphRAG的记忆分析，以识别即时危险并解释长期安全模式。

Details

Motivation: 解决地下煤矿共享、密闭、照明不良空间中，传统监控系统（如固定摄像头和基于规则的接近警报）因缺乏3D场景理解和上下文记忆，而难以预测复杂或动态演变危险（如设备违规接近、结构不稳定和遮挡盲区）的问题。

Result: 感知模型在30 FPS下达到92.7%的准确率且内存占用低。在115个危险场景中，基于规则的检查覆盖率为57%，结合上下文LLM推理提升至76%，使用历史记录进行基于记忆的推理后达到93%。

Insight: 创新点在于将图（场景图和时间图）作为显式知识结构，连接感知与各推理阶段，并融合了多模态感知、不确定性分析、LLM推理和GraphRAG记忆，实现了从3D感知到分层安全推理的闭环。为解决标注数据稀缺，采用了真实扫描、受控物体放置、高保真模拟生成多样场景，并结合自监督预训练提升分割性能。

Abstract: Underground coal mining requires personnel and heavy equipment to operate within shared, confined, and poorly illuminated spaces where hazards such as equipment proximity violations, structural instabilities, and occluded blind spots are difficult to anticipate. Conventional monitoring systems, including fixed cameras and rule-based proximity alerts, can detect predefined events but lack the 3D scene understanding and contextual memory needed to identify complex or evolving hazards. This paper presents a continuous monitoring framework that converts colourised 3D point clouds into structured and traceable safety reasoning outputs. The framework combines 3D semantic perception, uncertainty-based anomaly detection, rule-based hazard checks, on-device LLM reasoning, and GraphRAG -based memory analysis to identify immediate hazards and interpret longer-term safety patterns. Scene and temporal graphs serve as the explicit knowledge structure, linking perception outputs across reasoning stages. To overcome the scarcity of labeled underground data, real roadway scans, controlled object placement, and high-fidelity longwall simulation were combined to generate diverse hazard scenarios, while self-supervised pretraining improved segmentation from limited annotations. The perception model achieved 92.7% accuracy at 30 FPS with low memory usage. Across 115 hazard scenarios, rule-based checks achieved 57% coverage, increasing to 76% with contextual LLM reasoning and 93% with memory-based reasoning using historical records. Qualitative results show uncertainty-derived anomaly signals support the interpretation of out-of-distribution hazards beyond predefined classes. Overall, graph-based knowledge representation combined with 3D perception and layered safety reasoning provides a practical foundation for intelligent decision support in underground mine monitoring.

Zuhao Ge, Xiaosong Jia, Chao Wu, Yuchen Zhou, Zuxuan Wu

TL;DR: 本文提出了EvoMemNav，一种用于零样本具身导航的高效自演化细粒度记忆框架。该方法构建视觉语义记忆图（VSMGraph），将原始视图作为一等记忆，并通过轻量级语义线索和拓扑关系组织成房间-视图-对象层次结构，以保留细粒度细节用于消歧和停止验证。通过预算化的由粗到精策略来扩展记忆，并在每个子任务后进行反思驱动的写回操作，以更新图附着的先验知识，从而无需重新训练即可优化未来决策。

Details

Motivation: 解决零样本具身导航中长时程规划的记忆构建问题。现有方法（如以检测器为中心的场景图）会压缩观测为稀疏节点，丢弃细粒度视觉证据并累积噪声，而基于3D重建的方法计算成本过高。

Result: 在GOAT-Bench和HM3D基准测试中，针对物体、文本描述和图像目标等多种模态，SR/SPL指标均取得一致提升，表现出更好的多实例消歧能力、更少的过早停止以及更强的零样本泛化能力。

Insight: 创新点在于将原始视图作为一等记忆进行保留和层次化组织，并引入了预算化的由粗到精策略以及反思驱动的记忆写回机制，实现了无需重新训练的自演化记忆更新，从而在效率和细粒度信息保留之间取得了平衡。

Abstract: Building memory is essential for long-horizon planning in zero-shot embodied navigation. Detector-centric scene graphs often compress observations into sparse nodes, discarding fine-grained visual evidence and accumulating noise, while 3D reconstruction-based methods remain computationally prohibitive. We present EvoMemNav, an efficient, self-evolving, fine-grained memory framework for zero-shot embodied navigation. EvoMemNav constructs a Visual-Semantic Memory Graph (VSMGraph) that keeps raw views as first-class memory and organizes them with lightweight semantic cues and topological relations into a room-view-object hierarchy, preserving fine-grained details for disambiguation and Stop verification. To scale to growing memory, we introduce a budgeted coarse-to-fine policy: a coarse stage compresses the search space into promising regions, and a fine stage invokes a VLM only for targeted verification and decision. Beyond static memories, EvoMemNav performs reflection-driven write-back after each subtask, updating graph-attached priors that encode accumulated environmental knowledge to refine future decisions without retraining. Experiments on GOAT-Bench and HM3D across object, text-description, and image-goal modalities show consistent gains in SR/SPL, with better multi-instance disambiguation, fewer premature stops, and stronger zero-shot generalization.

[77] PersistGS: Differentiable Physics for Object Permanence in 4D Gaussian Splatting cs.CV | cs.GRPDF

Adrian Ramlal, John S. Zelek

TL;DR: 本文提出PersistGS方法，将可微分的刚体动力学模拟与4D高斯泼溅（3DGS）相结合，以解决动态场景重建中物体被完全遮挡时的‘物体恒存性’问题。该方法通过分解场景为各物体的高斯表示和碰撞网格，利用可微模拟从遮挡前轨迹估计摩擦和速度，从而预测物体在遮挡期间的SE(3)轨迹，实现物理正确的运动建模。

Details

Motivation: 现有动态3DGS方法在物体被完全遮挡时，由于缺乏光度监督信号，其高斯表示会退化；而基于生成先验的方法往往牺牲物理正确性以追求视觉合理性。本文旨在通过引入可微物理模拟，在遮挡期间恢复物体的物理合理运动轨迹。

Result: 在合成场景的实验中，PersistGS在训练时未使用的、能观测到遮挡物体的相机视角下进行评估。相比恒定速度外推法，PSNR提升了+2.46dB，且与真实轨迹上限的差距仅为0.19dB；其轨迹误差比仅使用光度监督的方法降低了40%。

Insight: 核心创新在于将可微分刚体动力学模拟与3DGS耦合，通过物理约束而非纯视觉先验来维持遮挡期间的对象表示；提出的质心轮廓损失函数能有效分离位置梯度与外观噪声，提升了轨迹预测的准确性。这种方法为神经场景重建中的不完全观测问题提供了物理可信的解决方案。

Abstract: Dynamic 3D Gaussian Splatting (3DGS) methods reconstruct time-varying scenes from synchronized multi-camera video using photometric supervision. When a moving object becomes fully occluded from all training cameras, this supervision vanishes: the Gaussians representing it receive no gradient signal and degrade. Existing approaches to incomplete observations in neural reconstruction rely on learned generative priors that prioritize visual plausibility over physical correctness. We propose $\textbf{PersistGS}$, a method that restores object permanence during occlusion by coupling differentiable rigid body simulation with 3D Gaussian Splatting. Our approach decomposes the scene into per-object Gaussians and collision meshes, estimates friction and velocity from the observed pre-occlusion trajectory via differentiable simulation, and uses the resulting SE(3) trajectory to position object Gaussians throughout the occlusion period. Because the predicted trajectory satisfies the governing equations of rigid body dynamics, it faithfully captures contact events (bounces, friction-based deceleration, direction changes) that kinematic extrapolation cannot model. We introduce a centroid silhouette loss that isolates positional gradients from appearance noise, yielding 40% lower trajectory error than photometric supervision. We evaluate using cameras withheld from training that observe the object during its occlusion. Experiments on synthetic scenes show that PersistGS outperforms constant velocity extrapolation by +2.46dB PSNR and comes within 0.19dB of a ground-truth trajectory upper bound.

[78] TrAction: Action Recognition with Sparse Trajectories cs.CVPDF

Jan F. Meier, Felix B. Mueller, Alexander Ecker, Timo Lüddecke

TL;DR: 本文提出了一种基于稀疏点轨迹的高效动作识别方法TrAction，该方法使用简单的Transformer架构和掩码轨迹预训练，旨在通过专注于运动信息而非外观或背景捷径来提升识别性能。

Details

Motivation: 现代动作识别模型通常依赖计算密集的密集RGB视频，并容易利用外观和背景等捷径信息（如根据物体或场景预测动作），而非真正的运动特征。本文旨在探索一种高效且能避免此类偏见的替代输入模态——稀疏点轨迹。

Result: 该方法在Something-Something V2数据集上达到45%的top-1准确率，在EPIC-Kitchens-100上达到54%，并在时间反转敏感性上超越了V-JEPA模型。更重要的是，轨迹特征与最先进的外观特征（如DINOv2和V-JEPA 2）互补，融合后能显著提升Something-Something V2的准确率。

Insight: 论文的核心创新在于将稀疏点轨迹作为主要输入模态，并设计了相应的Transformer架构和掩码预训练策略，这有效避免了外观偏见并专注于运动建模。其关键洞察是轨迹特征与外观特征具有互补性，融合两者能实现性能的显著提升。

Abstract: Modern action recognition models operate on memory- and compute-intensive dense RGB video volumes and frequently exploit appearance and background shortcuts, for example, predicting actions from objects or scenes instead of characteristic motion. We investigate an efficient alternative input modality that is largely free of such biases by construction: sparse point trajectories. To this end, we develop a simple transformer architecture for 2.5D trajectory-based recognition together with a masked-trajectory pretraining, which we show to substantially improve downstream action recognition accuracy. Despite using only a fraction of the dense RGB input, our method reaches 45% top-1 on Something-Something V2 and 54% on EPIC-Kitchens-100, and surpasses V-JEPA on time-reversal sensitivity. More importantly, we find trajectory features to be complementary to state-of-the-art appearance-based features. Fusing our pretrained model with DINOv2 and V-JEPA 2 improves top-1 accuracy on Something-Something V2 by 8.7 and 1.6 points, respectively. Code: https://github.com/ecker-lab/TrAction

[79] Knowledge-Preserved Model Tuning in Null-Space for Robust Spatio-Temporal Video Grounding cs.CVPDF

Haoxuan Chen, Xianqin Liu, Jian-Fang Hu

TL;DR: 本文提出了一种名为Null-Space Tuning（NST）的模型调优框架，用于解决时空视频定位任务中低质量视频输入导致的性能下降问题。该方法通过利用冻结权重零空间的几何特性，向输入特征注入可学习的残差，从而在适应低质量输入的同时，有效保留预训练模型对高质量输入的知识。

Details

Motivation: 现有时空视频定位方法主要针对高质量视频输入设计，忽略了现实场景中广泛存在的低质量视频。虽然像LoRA这样的调优方法可以适应退化输入，但不可避免地会破坏预训练知识。

Result: 在作者构建的Mixed-Quality基准测试上进行的广泛实验表明，NST方法优于最先进的（SOTA）方法。

Insight: 核心创新点在于利用冻结权重零空间的几何特性，通过质量自适应单元和双空间重参数化技术，合成选择性对预训练主干网络“不可见”的输入残差，从而实现了在适应低质量输入的同时不干扰高质量输入下的预训练知识。这是一种新颖的知识保留型模型适应策略。

Abstract: Spatio-Temporal Video Grounding aims to localize object tubes based on textual queries. While recent methods have achieved remarkable success, they mainly focus on high-quality(HQ) inputs, neglecting the widespread presence of low-quality(LQ) videos in real-world scenarios. Although tuning methods like LoRA can adapt to degraded inputs, they inevitably disrupt pre-trained knowledge. To address this, we propose Null-Space Tuning (NST). This framework exploits the geometric property that adding vectors within the null-space of frozen weights to the layer input does not affect the output. Leveraging this, NST injects learnable residuals into input features that can be selectively invisible to the pre-trained backbone. Specifically, NST combines the Quality-Adaptive Unit and Dual-Space Reparameterization to synthesize these residuals by confining components for HQ inputs to the null-space, while directing restoration components for LQ inputs to the non-null space. As the frozen weights eliminate null-space components, we effectively rectify degraded inputs while preserving pre-trained knowledge for HQ inputs. Extensive experiments show that NST outperforms state-of-the-art methods on our Mixed-Quality benchmark.

[80] Attend to Anything: Foundation Model for Unified Human Attention Modeling cs.CVPDF

Wenzhuo Zhao, Ronghao Xian, Keren Fu, Qijun Zhao

TL;DR: 本文提出了Attend to Anything Model (AAM)，一个用于统一人类注意力建模的多模态基础模型。它通过将注意力重新定义为一种在双曲空间中用分层嵌入语言提示实现的、从一般到具体的认知蕴含关系，并采用流体动力学视角将视频帧注意力建模为受福克-普朗克方程控制的扩散时间演化，从而统一了图像、视频和视听任务中的注意力建模。

Details

Motivation: 现有的人类注意力（显著性）建模方法在不同模态、场景和任务表述上高度碎片化，导致模型通常是场景依赖和任务特定的，难以在实际应用中泛化。本文旨在解决这一根本限制。

Result: 在16个基准测试上的广泛实验表明，AAM在各种场景中始终优于最先进的方法，平均提升6%，同时在视频推理上实现了约4倍的速度提升。

Insight: 主要创新点包括：将注意力重新定义为层次化的认知蕴含关系，并使用双曲空间嵌入和语言提示来实现；从流体动力学角度统一静态图像和动态视频注意力，用福克-普朗克方程建模时间演化。这为注意力相关任务提供了一个原则性的统一基础。

Abstract: Existing human attention (saliency) modeling methods persist as highly fragmented across modalities, scenes, and task formulations. Consequently, even with increasing model capacity and data scale, current models predominantly remain scene-dependent and task-specific, failing to practically generalize in real-world applications. To address the fundamental limitations, we present the Attend to Anything Model (AAM), a multi-modal foundation model that unifies attention modeling across various image, video, and audio-visual tasks and scenes. AAM reformulates attention as a cognitive entailment relationship organized in a general-to-specific hierarchy, implemented through language prompts with hierarchical embeddings in hyperbolic space. Furthermore, to unify static image and dynamic video attention, we adopt a fluid-dynamics perspective, formulating video-frame attention as a diffusive temporal evolution governed by the Fokker–Planck equation. Extensive experiments on 16 benchmarks demonstrate that AAM consistently outperforms state-of-the-art methods by an average of 6% across various scenarios, while achieving approximately a 4$\times$ speedup in video inference. Overall, these results demonstrate that AAM provides a principled foundation for future research on attention and saliency-related tasks. The dataset and code will be available at https://github.com/wz-zhao/Attend-to-Anything.

[81] \textsc{CR-Seg}: Attention-Guided and CoT-Enhanced Coarse-to-Refined Reasoning Segmentation cs.CV | cs.AIPDF

Yifan Cao, Xiaocui Yang, Faxian Wan, Shi Feng, Daling Wang

TL;DR: 本文提出了CR-Seg，一个用于推理分割的两阶段框架。该方法通过EAP模块提取注意力图进行粗定位并选择信息点，输入SAM进行掩码细化，并引入GLCoT策略从全局到局部进行渐进式推理，以解决现有方法跨模态对齐困难或丢失语义的问题。

Details

Motivation: 现有推理分割方法要么依赖学习的语义令牌导致跨模态对齐困难，要么依赖显式空间提示（如边界框）可能丢失整体响应语义，本文旨在解决这些局限性。

Result: 在推理分割基准上的大量实验证明了CR-Seg的有效性，但摘要中未提及具体的定量结果或是否达到SOTA水平。

Insight: 创新点在于设计了EAP模块进行粗定位与信息点选择，并引入了全局到局部的思维链（GLCoT）策略来缓解推理-答案不一致性，实现渐进式推理。

Abstract: Reasoning segmentation aims to segment target objects described by complex language through joint visual-textual reasoning. Existing methods typically rely on either learned semantic tokens to bridge Multimodal Large Language Models (MLLMs) and segmentation models, suffering from difficult cross-modal alignment, or explicit spatial prompts such as bounding boxes, which may lose holistic response semantics. To address these limitations, we propose Attention-Guided and CoT-Enhanced Coarse-to-Refined Reasoning Segmentation, termed CR-Seg, a two-stage framework for coarse-to-refined reasoning segmentation. Specifically, we design an Extract Attention Maps and Points (EAP) module to extract attention maps for coarse target localization and select informative points, both of which are fed into SAM for mask refinement. To alleviate reasoning–answer inconsistency, we further introduce Global-to-Local Chain-of-Thought (GLCoT), which guides the model to reason progressively from global scene context to local target details. Extensive experiments on reasoning segmentation benchmarks demonstrate the effectiveness of CR-Seg.

[82] When Attention Collapses: Stage-Aware Visual Token Pruning from Structure to Semantics cs.CV | cs.AIPDF

Jiahui Wang, Kai Zhang, Mai Han, Huanghe Zhang

TL;DR: 本文提出了一种名为Structure-to-Semantics (STS)的两阶段视觉令牌剪枝框架，旨在解决现有基于注意力分数的视觉令牌剪枝方法因注意力分数塌缩到语义相似区域而导致特征多样性和上下文细节丢失的问题。该框架通过解耦剪枝过程，第一阶段采用基于排斥的采样机制最大化空间和结构多样性，第二阶段利用指令感知的交叉注意力精确过滤与提示无关的令牌，从而在保持几何覆盖的同时根据语义相关性精炼保留的视觉令牌。

Details

Motivation: 视觉语言模型在推理时存在显著的计算开销，而现有的视觉令牌剪枝方法主要依赖初始注意力分数，这种单一度量范式存在关键缺陷：高注意力分数会塌缩到语义相似区域，严重降低特征多样性并丢弃关键上下文细节。

Result: 广泛的评估表明，STS缓解了基于注意力选择带来的冗余，提高了保留视觉令牌的结构多样性和细粒度任务对齐性能。

Insight: 核心创新在于将剪枝过程明确解耦为结构和语义两个阶段，通过排斥采样确保几何覆盖，再通过指令感知交叉注意力进行语义精炼，这种两阶段协同机制有效克服了传统单度量剪枝的局限性，提升了剪枝后令牌的代表性和任务相关性。

Abstract: Vision-Language Models (VLMs) have demonstrated remarkable capabilities but suffer from significant computational overhead during inference. While visual token pruning offers a promising solution, existing methods predominantly rely on initial attention scores. This single-metric paradigm presents a critical flaw: high attention scores inherently collapse onto semantically similar regions, thereby severely reducing feature diversity and discarding vital contextual details. To address this, we introduce Structure-to-Semantics (STS), a novel two-stage visual token pruning framework that explicitly decouples the pruning process. The first stage employs a repulsion-based sampling mechanism to maximize spatial and structural diversity. The second stage leverages instruction-aware cross-attention to precisely filter out prompt-irrelevant tokens. This two-stage synergy constitutes the core of STS, first ensuring geometric coverage and then refining the retained tokens according to semantic relevance. Extensive evaluations demonstrate that STS mitigates the redundancy caused by attention-based selection, improving both structural diversity and fine-grained task alignment of the preserved visual tokens.

[83] Eliciting Complex Spatial Reasoning in MLLMs through Wide-Baseline Matching cs.CVPDF

Hao Zhong, Muzhi Zhu, Shenyan Zeng, Anzhou Li, Cong Chen

TL;DR: 该论文针对多模态大语言模型在宽基线匹配任务中空间推理能力的不足，提出了一个分层评估基准ReasonMatch-Bench，并开发了一个可扩展的数据生成管道和动态对应强化学习方法DCRL，以提升模型在复杂空间推理任务上的性能。

Details

Motivation: 当前MLLMs在需要整合几何理解、视角变化、细粒度感知和遮挡推理的宽基线匹配任务上缺乏系统性的评估和训练框架，而该能力对于物理环境部署至关重要。

Result: 在困难的90样本子集上，人类标注者达到84.0 F1，而现有最佳基线仅为37.2；提出的DCRL方法在ReasonMatch-Bench上取得显著提升，并能泛化到相关空间基准，同时在其他多个基准上保持或略微提升通用视觉理解性能。

Insight: 创新点在于构建了一个分层、可扩展的宽基线匹配评估基准和自动数据生成管道，并提出了结合图像级视角递进和点级对应课程学习的动态对应强化学习方法，无需显式思维链监督即可通过可验证奖励改进训练。

Abstract: Wide-baseline matching (WBM) requires integrating geometric understanding, viewpoint changes, fine-grained perception, and occlusion reasoning, making it a challenging testbed for spatial reasoning in multimodal large language models (MLLMs) deployed in physical environments. However, current MLLMs lack systematic evaluation and training frameworks for these capabilities. We introduce ReasonMatch-Bench, a benchmark stratified by viewpoint displacement and matching granularity across indoor, outdoor, and object-centric scenarios, and show that current MLLMs still struggle with fine-grained wide-baseline correspondence: on a difficult 90-sample subset, human annotators achieve 84.0 F1, while the best existing baseline reaches 37.2. To bridge this gap, we build a scalable data-generation pipeline that automatically extracts wide-baseline view pairs from large-scale video-3D corpora, including RGB-D videos and SfM reconstructions, yielding diverse and verifiable supervision. We further propose Dynamic Correspondence Reinforcement Learning (DCRL), which combines Image-Level Viewpoint Progression and Point-Level Correspondence Curriculum to improve WBM training through verifiable rewards without explicit CoT supervision. Extensive experiments show that DCRL substantially improves ReasonMatch-Bench and transfers to related spatial benchmarks, while maintaining general visual understanding performance with modest gains on several benchmarks.

[84] Efficient Transformer-Based Localized Patch Sampling for Choroid Plexus Segmentation in Multiple Sclerosis cs.CV | cs.AIPDF

Po-Jui Lu, Alessandro Cagol, Mario Ocampo-Pineda, Federico Spagnolo, Marina Mastantuono

TL;DR: 本文提出了一种基于SwinUNETR的局部小补丁采样方法，用于从MRI图像中自动分割多发性硬化症（MS）相关的侧脑室脉络丛（LVCP）。该方法在扩展测试集上实现了0.868的Dice相似系数，显著优于UXNET模型，并将计算成本降低了99%。

Details

Motivation: 手动分割LVCP非常繁琐，限制了其在临床试验和纵向评估中的广泛应用，因此需要开发一种自动、准确的自动分割方法。

Result: 在扩展测试集（n=388）上，结合MPRAGE和FLAIR模态，SwinUNETR模型取得了0.868的平均DSC，显著优于UXNET（0.858，p<0.0001），并且计算量（GFLOPs）降低了99%。仅使用FLAIR模态时，SwinUNETR仍保持高精度（DSC: 0.863），而UXNET的空间定位性能显著下降（HD95: 3.00 mm vs. 1.86 mm）。

Insight: 创新点在于将局部（脑室内及周围）小补丁采样策略与SwinUNETR架构相结合，在保证分割精度的同时，极大降低了计算复杂度，为临床广泛应用提供了高效、鲁棒的解决方案。

Abstract: Background: The lateral ventricle choroid plexus (LVCP) is gaining recognition as a key imaging biomarker for multiple sclerosis (MS) related to physical disability and neuroinflammation. Yet, manual segmentation of the LVCP is highly tedious, restricting its use in broad clinical trials and longitudinal assessments. This research aims to develop a SwinUNETR-driven pipeline that leverages targeted intra- and peri-ventricular small patch sampling to automatically segment the LVCP in MS from both standalone and multi-modal MRI inputs. Methods: We retrospectively assessed 3T MRI scans across three sets of data stemming from two separate MS-dominant cohorts (Dataset 1: n=177; Dataset 2: n=177; expanded test set: n=388). Our method employed a SwinUNETR architecture trained on 32x32x32 voxel patches, benchmarking it against the 3D UXNET model. The primary metric for evaluation was the Dice Similarity Coefficient (DSC), supplemented by computational demand (GFLOPs) and the 95th percentile Hausdorff Distance (HD95). Results: On the extended test set, the SwinUNETR model secured a mean DSC of 0.868 (95% CI: 0.863-0.872) with MPRAGE and FLAIR combined, showing a statistically significant gain over UXNET (DSC: 0.858 [95% CI: 0.853-0.862], p<0.0001). When restricted to standalone FLAIR inputs, the transformer-based approach sustained a high DSC of 0.863, while the spatial localization of UXNET worsened considerably (HD95: 1.86 vs. 3.00 mm). Importantly, the proposed framework lowered computational load by 99% (91.8 vs. 22,080 GFLOPs). By integrating localized patch sampling with a SwinUNETR architecture, this methodology offers an accurate, robust, and statistically superior alternative to current leading models for LVCP segmentation. Its vast reduction in computational cost makes it ideal for widespread implementation in clinical and research environments.

[85] UnsOcc: 3D Semantic Occupancy Prediction in Unstructured Scene via Rendering Fusion cs.CV | cs.ROPDF

Ye Wu, Ruiqi Song, Baiyong Ding, Nanxin Zeng, Junjie Cheng

TL;DR: 本文提出了一种名为UnsOcc的多模态3D语义占据预测框架，专门用于解决非结构化场景（如露天矿场）中的感知挑战。其核心创新包括一个基于渲染的融合模块RenderFusion，通过双向渲染监督增强跨模态特征对齐，以及一个基于高斯泼溅的细节感知辅助监督方法GSRefinement，用于改善长尾类别的预测性能。

Details

Motivation: 非结构化场景（如存在不规则障碍物和稀疏布局的露天矿场）对传统3D目标检测方法构成挑战，而现有的3D语义占据预测方法直接应用于此类场景时，会因场景稀疏性阻碍有效的跨模态融合，以及更严重的长尾分布问题导致性能下降。

Result: 在专门构建的露天矿场数据集和公开的nuScenes数据集上进行的广泛实验表明，该方法显著优于现有的最先进方法。

Insight: 论文的创新点在于提出了RenderFusion模块，利用双向渲染监督来增强跨模态（如相机和激光雷达）特征对齐，以及GSRefinement方法，通过将稀疏的3D占据预测投影为密集的2D语义分割图，从而为长尾类别提供有效的监督信号。这为解决非结构化场景中数据稀疏和类别不平衡问题提供了新思路。

Abstract: Unstructured scenes present unique challenges for autonomous driving, as irregular obstacles and sparse scene layouts undermine the effectiveness of traditional perception methods such as 3D object detection. 3D semantic occupancy prediction has emerged as a prominent focus due to its ability to provide dense spatial representations by assigning semantic labels to individual voxels in 3D space. However, directly applying 3D semantic occupancy prediction to unstructured scenes remains challenging because scene sparsity hinders effective cross-modal fusion and the more severe long-tail distribution in these scenarios further degrades prediction performance. To validate the effectiveness of our approach, we construct a dedicated dataset of unstructured scenes collected from open-pit mines. Based on this, we propose UnsOcc, a multi-modal 3D semantic occupancy prediction framework that improves robustness in unstructured environments. At its core, we introduce a rendering-based fusion module, RenderFusion, which enhances cross-modal feature alignment through bidirectional rendering supervision. Furthermore, we propose GSRefinement, a detail-aware auxiliary supervision method based on Gaussian Splatting that projects sparse 3D occupancy predictions into dense 2D semantic segmentation maps, enabling effective supervision for long-tail categories. Extensive experiments on both the open-pit mine dataset and the nuScenes dataset demonstrate that our method significantly outperforms existing state-of-the-art approaches.

[86] World Models Meet Language Models: On the Complementarity of Concrete and Abstract Reasoning cs.CV | cs.CLPDF

Yucheng Zhou, Wei Tao, Yiwen Guo, Jianbing Shen

TL;DR: 该论文探讨了世界模型与多模态大语言模型在从静态视觉观察预测未来结果方面的互补性。世界模型能生成具体的视觉推演，而MLLMs能进行抽象推理。论文提出了受控具体推理框架，通过PF-OPSD方法学习调用、验证和整合视觉未来模拟与抽象推理，并在两个基准测试上验证了其有效性。

Details

Motivation: 解决世界模型生成的视觉推演具有随机性、可能视觉合理但任务错误的问题，需要确定视觉模拟何时有用、推演是否可信以及如何影响最终答案。

Result: 在VRQABench和OpenWorldQA两个基准测试上，PF-OPSD方法分别比基线模型提升了10.6%和10.9%的性能，并增强了对噪声或矛盾推演的鲁棒性。

Insight: 创新性地提出了受控具体推理框架，将具体视觉模拟与抽象推理相结合；提出了PF-OPSD训练方法，利用真实未来视频作为特权上下文进行策略内蒸馏，而部署模型在测试时不观察真实未来，实现了有效的模拟整合与鲁棒性提升。

Abstract: World models and multimodal large language models (MLLMs) provide complementary capabilities for predicting future outcomes from static visual observations. World models can generate concrete visual rollouts of possible futures, while MLLMs can reason abstractly over questions, goals, and rules. However, generated rollouts are stochastic and may be visually plausible but task-incorrect, making it necessary to determine when visual simulation is useful, whether a rollout is credible, and how it should influence the final answer. We formulate this problem as controlled concrete reasoning, where a model learns to invoke, verify, and integrate visual future simulation alongside abstract reasoning. To study this setting, we construct two human-verified benchmarks, VRQABench for controllable spatial lookahead and OpenWorldQA for open-domain physical prediction, and propose Privileged-Future On-Policy Self-Distillation (PF-OPSD). During training, PF-OPSD uses ground-truth future videos and answers only as teacher-side privileged context to evaluate on-policy concrete-reasoning trajectories, while the deployable student never observes true futures at test time. Experimental results show that PF-OPSD outperforms baseline by 10.6% and 10.9% on VRQABench and OpenWorldQA, respectively, while increasing robustness to noisy or conflicting rollouts. Our code and dataset are available at https://github.com/yczhou001/PF-OPSD.

[87] TurtleAI: Benchmarking Multimodal Models for Visual Programming in Turtle Graphics cs.CV | cs.AI | cs.CYPDF

Chao Wen, Jacqueline Staub, Adish Singla

TL;DR: 本文介绍了TurtleAI基准测试，用于评估多模态模型在Turtle Graphics领域的教育导向视觉编程能力。该基准包含823个基于真实任务的任务，要求模型感知几何模式、推理空间关系并生成Python代码。评估了20多个VLM模型（如GPT-5、GPT-4o等），发现它们表现不佳（成功率低于30%），并提出一种基于少量种子样本的数据生成技术，通过微调Qwen2-VL-72B模型将真实任务性能提升约20%。

Details

Motivation: 解决当前视觉语言模型在教育导向视觉编程中性能不足的问题，并探究限制其表现的因素，以填补现有研究在生产力导向之外的空白。

Result: 在TurtleAI基准上，大多数VLM模型成功率低于30%；通过提出的数据生成技术微调Qwen2-VL-72B后，在真实任务上性能提升约20%。

Insight: 创新点包括引入教育导向的视觉编程基准TurtleAI，以及一种高效的数据生成技术；客观分析表明，模型在空间推理和精确视觉复制方面存在瓶颈，而微调主要改善了视觉推理与代码实现的对齐。

Abstract: Vision-language models (VLMs) have been explored for visual programming, where they generate code to solve visual tasks. However, most prior work focuses on visual programming for productivity; it remains unclear how well current VLMs perform on education-oriented visual programming and what factors limit their performance. To bridge this gap, we introduce TurtleAI, a benchmark containing 823 tasks curated based on real-world visual programming tasks in the Turtle Graphics domain. Solving these tasks requires models to perceive geometric patterns, reason about spatial relationships, and synthesize Python code that faithfully reproduces geometric patterns. We evaluate 20+ VLMs, including GPT-5, GPT-4o, and Qwen2-VL-72B, and find that they struggle significantly, with most achieving success rates below 30%. To address these limitations, we propose a data generation technique that requires only a small set of seed samples. Fine-tuning Qwen2-VL-72B on the resulting synthetic data yields an improvement of about 20% on real-world tasks. Our failure analysis reveals that GPT-4o struggles with spatial reasoning and precise visual replication, whereas fine-tuning primarily improves the alignment between visual reasoning and code implementation.

Haoliang Meng, Xiaopeng Hong, Yabin Wang, Wangmeng Zuo

TL;DR: 本文构建了首个半监督多模态人群计数基准，包括任务设定、标准化协议、多样化的基线方法以及性能评估，旨在为这一未探索领域奠定基础。

Details

Motivation: 解决半监督多模态人群计数任务缺乏基准和标准化评估的问题，为研究提供统一的实验框架和数据划分。

Result: 在提出的基准上评估了多种基线方法（包括全监督多模态和半监督单模态方法），但摘要未具体提及定量结果或SOTA比较。

Insight: 创新点在于首次系统性地构建半监督多模态人群计数基准，制定了标准化协议和数据划分，为后续研究提供了可复现的基础。

Abstract: This paper constructs the first benchmark on semi-supervised multi-modal crowd counting. To lay the foundation for this unexplored task, we first formulate the semi-supervised multi-modal setting and a standardized protocol that specifies the labeled-unlabeled data partition across different labeled ratios. Next, to establish solid reference points, we carefully tailor a diverse set of representative baselines, including existing fully supervised multi-modal methods and semi-supervised single-modal methods. Then, we carefully evaluate their performance under our proposed benchmark. Codes and the data partition will be released on https://github.com/HenryCilence/Semi-supervised-Multimodal-Crowd-Counting.

[89] VidMsg: A Benchmark for Implicit Message Inference in Short Videos cs.CV | cs.AIPDF

Issar Tzachor, Michael Green, Rami Ben-Ari

TL;DR: 本文提出了VidMsg基准测试，用于评估短视频中隐含信息的理解能力。该基准包含400个YouTube视频片段，涵盖9个主题领域和52个细粒度目标信息，支持双向信息-视频检索和诊断性多项选择题问答任务。实验表明，现有视频语言和检索模型在VidMsg上表现不佳，因为任务需要语用推理、上下文线索整合以及对语义相近信息的区分能力。

Details

Motivation: 为了解决短视频理解中超越可见物体和动作识别的深层次问题，即捕捉视频制作者在片段中蕴含的潜在信息或意图，作者构建了VidMsg基准。

Result: 在VidMsg基准上的实验表明，当前先进的视频语言和检索模型表现不佳，突显了该任务的挑战性。作者提出的基线方法VidVec-Msg在面向信息的检索任务上有所改进，但仍为未来工作留下了很大提升空间。

Insight: 创新点在于提出了首个专注于短视频隐含信息理解的基准测试，并采用了‘信息优先’的构建流程（LLM生成间接搜索场景再人工筛选），这为评估模型对视频整体语义和语用意图的理解能力提供了新标准。

Abstract: Understanding short online videos involves more than identifying visible objects and actions; video makers often include an underlying message or purpose in the clip. We introduce VidMsg, a benchmark for evaluating implicit message understanding in short, internet-native video clips. VidMsg contains 400 YouTube-derived clips across 9 practical topic areas and 52 fine-grained target messages, covering domains such as career and finance, education, health and well-being, culture, safety, sustainability, and lifestyle. VidMsg is constructed through a message-first pipeline: an LLM first translates target messages into indirect search scenarios, which are used to retrieve candidate clips. Human annotators then retain clips that convey the intended message without being overly explicit. VidMsg is designed primarily for bidirectional message-clip retrieval for scalable applications such as video search and recommendation, where systems must capture holistic video understanding. In addition to retrieval, VidMsg includes a diagnostic multiple-choice QA benchmark, where models select the intended message of a clip from semantically related alternatives. Experiments with contemporary video-language and retrieval models show that strong models often fail on VidMsg, because the task requires pragmatic inference, integration of contextual cues, and discrimination among semantically close messages. We also introduce VidVec-Msg, a baseline method that improves message-oriented retrieval while leaving substantial headroom for future work.

Hashmat Shadab Malik, Muzammal Naseer, Salman Khan

TL;DR: 本文系统研究了多模态大语言模型（MLLMs）的对抗鲁棒性。研究发现，大规模多模态对抗预训练是提升MLLMs视觉对抗鲁棒性的关键，通过端到端多模态训练整合鲁棒视觉编码器，在图像描述和视觉问答任务上显著提升了模型在强对抗攻击下的性能。此外，研究还探索了轻量级测试时防御方法，并证明鲁棒模型能有效减少在白盒视觉越狱攻击下的有害生成。

Details

Motivation: MLLMs通过视觉编码器（如CLIP）整合视觉输入，极大地扩展了攻击面，使其容易受到视觉对抗扰动的影响。现有防御方法通常在对抗微调时强制对齐CLIP的原始嵌入空间以保持与预训练MLLMs的兼容性，但这种约束从根本上限制了可实现的鲁棒性。

Result: 在强对抗攻击下，与受约束的即插即用基线相比，通过端到端多模态训练整合鲁棒视觉编码器，在图像描述任务上平均提升28个CIDEr点，在视觉问答任务上提升11.7%的准确率。从鲁棒骨干网络进行端到端对抗训练可带来额外的1.9个CIDEr点和4.3%的VQA准确率提升。对于非鲁棒MLLMs，轻量级测试时视觉随机变换防御能将其对抗性能从接近零提升到与鲁棒模型相当的水平。

Insight: 创新点在于提出了一个诊断性的CLIP对齐协议，可在完整MLLM训练前预测哪些鲁棒视觉编码器能有效迁移到多模态场景，并揭示大规模多模态对抗预训练（而非单模态规模）是强鲁棒性迁移的关键因素。客观来看，研究强调了鲁棒视觉表示是MLLMs对抗防御的先决条件，并为训练时和测试时防御提供了系统性的见解和有效方案。

Abstract: Multi-modal Large Language Models (MLLMs) achieve strong performance on vision-language tasks, but incorporating visual inputs through a vision encoder (e.g., CLIP) substantially expands the attack surface, making these models vulnerable to visual adversarial perturbations. Prior defenses typically preserve compatibility with pretrained MLLMs by enforcing strict alignment to CLIP’s original embedding space during adversarial fine-tuning; while practical, this constraint fundamentally limits achievable robustness. We present a systematic investigation of adversarial robustness in MLLMs. We first introduce a diagnostic CLIP-alignment protocol that predicts, prior to full MLLM training, which robust vision encoders will transfer effectively to the multimodal setting, revealing that large-scale multimodal adversarial pretraining, rather than unimodal scale alone, is the critical factor for strong robustness transfer. Integrating such encoders into MLLMs via end-to-end multimodal training yields average gains of 28 CIDEr points on captioning and 11.7% VQA accuracy under strong adversarial attacks compared to constrained plug-and-play baselines. We further show that adversarial training applied directly to a standard non-robust MLLM degrades both clean and adversarial performance, establishing robust visual representations as a strict prerequisite, while end-to-end adversarial training from a robust backbone delivers additional gains of 1.9 CIDEr points and 4.3% VQA accuracy. Beyond training-time defenses, lightweight test-time visual stochastic transformations serve as an effective black-box defense for non-robust MLLMs, elevating adversarial performance from near-zero to levels comparable with robust models. Finally, we show that our robust models substantially reduce toxic generation under white-box visual jailbreak attacks. Code and pretrained weights will be released publicly.

[91] Beyond False Stability: High-Noise Drift Gating for Test-Time Adversarial Defenses in Vision-Language Models cs.CVPDF

Hashmat Shadab Malik, Muzammal Naseer, Salman Khan

TL;DR: 本文提出了一种名为高噪声漂移门控的训练无关、即插即用防御机制，用于提升视觉语言模型（如CLIP）在测试时对抗对抗攻击的鲁棒性。该方法通过分析发现，在强噪声扰动下，对抗样本的特征表示会比干净样本表现出更显著的不稳定性，从而提供了一个清晰的区分信号。利用这一高噪声特征漂移作为轻量级门控信号，仅在检测到类似对抗样本的不稳定性时才触发现有的测试时防御方法，从而在保持高干净准确率的同时显著提升鲁棒性。

Details

Motivation: CLIP等视觉语言模型在零样本泛化上表现优异，但对对抗攻击高度脆弱。对抗训练提升鲁棒性但计算成本高昂，因此测试时防御成为研究重点。现有基于随机扰动的测试时防御方法虽能提升鲁棒性，但往往以牺牲干净准确率为代价，导致不利的“干净-鲁棒”权衡。本文旨在重新审视随机测试时防御，并探索更优的权衡策略。

Result: 在13个数据集上的实验表明，该方法能持续改善“干净-鲁棒”权衡。在8个细粒度数据集上，对于反攻击防御，平均干净+对抗准确率从65.7%提升至71.4%；对于噪声锚定防御，从68.4%提升至73.2%。在ImageNet及其四个分布偏移变体上，分别从56.1%提升至66.2%和从62.1%提升至67.6%。

Insight: 核心创新点在于发现并利用了CLIP表示空间中一个未被充分探索的噪声机制转变：在超越弱噪声机制后，对抗样本的特征表示会变得比干净样本明显更不稳定，这为区分对抗样本提供了更清晰的信号。基于此，提出的高噪声漂移门控机制是一种轻量级、训练无关的插件，能智能地触发现有防御，有效改善了现有测试时防御方法在“干净-鲁棒”权衡上的不足。

Abstract: Vision-language models (VLMs) such as CLIP show strong zero-shot generalization but remain highly vulnerable to adversarial attacks. Adversarial training improves robustness but is computationally expensive, motivating test-time defenses. Recent approaches exploit how CLIP’s visual representations respond to stochastic perturbations: aggregating predictions across noisy views, constructing Gaussian noise-averaged anchors and interpolating features toward them, or applying counter-perturbations. These strategies improve robustness but often degrade clean accuracy, yielding an unfavorable clean-robust trade-off. We revisit stochastic test-time defenses and identify an underexplored noise-regime transition in CLIP’s representation space. Prior work explored perturbations mainly in the weak-noise regime, where adversarial examples can appear unusually stable (false stability). Our analysis shows this reverses as perturbation strength grows: beyond the weak-noise regime, adversarial representations become markedly more unstable than clean ones, giving a clearer separation signal. The transition is consistent across uniform and Gaussian noise, photometric and geometric transforms, datasets, and diverse attacks. It largely disappears in adversarially trained models, suggesting it is tied to the fragile local-basin geometry of adversarial representations in non-robust CLIP. We propose a training-free, plug-in drift-gated mechanism that uses high-noise feature drift as a lightweight gating signal to trigger existing test-time defenses only when adversarial-like instability is detected. Across 13 datasets it consistently improves the clean-robust trade-off. On eight fine-grained datasets, mean clean+adversarial accuracy rises from 65.7% to 71.4% for counterattack defenses and 68.4% to 73.2% for noise-anchoring; on ImageNet and four shifted variants, from 56.1% to 66.2% and 62.1% to 67.6%.

[92] Ultralytics YOLO26: Unified Real-Time End-to-End Vision Models cs.CV | cs.AIPDF

Glenn Jocher, Jing Qiu, Mengyu Liu, Shuai Lyu, Fatih Cagatay Akyon

TL;DR: 本文提出了Ultralytics YOLO26，一个统一、实时的端到端视觉模型家族，旨在解决现有YOLO系列模型在推理时依赖非极大值抑制、检测头因分布焦点损失而臃肿、训练周期长以及对小物体标签分配不足等问题。通过协调的架构和训练改进，YOLO26实现了无需NMS的端到端推理，并支持检测、实例分割、姿态估计、分类和定向检测等多种任务。

Details

Motivation: 解决现有实时视觉模型（尤其是YOLO系列）在部署效率、推理简化、训练速度和小物体检测方面的局限性，旨在构建一个更准确、高效且易于部署的统一模型家族。

Result: 在COCO数据集上，YOLO26系列模型在T4 TensorRT延迟1.7-11.8毫秒下达到40.9-57.5 mAP，提升了精度-延迟帕累托前沿；其开放词汇扩展版本YOLOE-26x在LVIS minival上通过文本提示达到40.6 AP。

Insight: 主要创新点包括：采用双头设计实现原生无NMS的端到端推理并移除DFL以简化检测头；引入结合Muon-SGD的混合优化器MuSGD、逐步将监督转向推理头的渐进损失以及保证小物体正标签分配的STAL策略；为多种视觉任务设计了统一的特定任务头和损失函数，实现了多任务扩展和性能提升。

Abstract: Real-time vision demands models that are accurate, efficient, and simple to deploy across diverse hardware. The YOLO family has become widely deployed for this reason, yet most YOLO detectors still rely on non-maximum suppression at inference, carry heavy detection heads due to Distribution Focal Loss, require long training schedules, and can leave the smallest objects without positive label assignments. We present Ultralytics YOLO26, a unified real-time vision model family that addresses these limitations through coordinated architecture and training advances. YOLO26 uses a dual-head design for native NMS-free end-to-end inference and removes DFL entirely, yielding a lighter head with unconstrained regression range. Its training pipeline combines MuSGD, a hybrid Muon-SGD optimizer adapted from large language model training; Progressive Loss, which shifts supervision toward the inference-time head; and STAL, a label assignment strategy that guarantees positive coverage for small objects. Beyond detection, YOLO26 introduces task-specific head and loss designs for instance segmentation, pose estimation, and oriented detection, producing consistent gains across tasks and scales. The family spans five scales (n/s/m/l/x) and supports detection, instance segmentation, pose estimation, classification, and oriented detection in a single pipeline, with an open-vocabulary extension, YOLOE-26, for text-, visual-, and prompt-free inference. Across all scales, YOLO26 achieves 40.9-57.5 mAP on COCO at 1.7-11.8 ms T4 TensorRT latency, advancing the accuracy-latency Pareto front over prior real-time detectors, while YOLOE-26x reaches 40.6 AP on LVIS minival under text prompting. Code and models are available at https://github.com/ultralytics/ultralytics.

[93] SLU-2K: A Question-Based Benchmark for Semantic Evaluation of Sign Language Translation cs.CVPDF

Zeno Testa, Antonino Furnari, Lorenzo Baraldi, Natalia Díaz-Rodríguez

TL;DR: 该论文提出了一个名为SLU-2K的基准数据集，用于评估手语翻译系统的语义理解能力，而非传统的表面形式指标。它基于PHOENIX-2014T和CSL-Daily数据集构建，包含2,350个封闭式视频问答对，涵盖7个语义类别。论文通过评估多模态大语言模型和现有SOTA系统，揭示了当前系统在语义理解上的显著不足。

Details

Motivation: 当前手语翻译评估主要依赖BLEU和ROUGE等表面形式指标，这些指标奖励词汇重叠，但无法直接衡量翻译是否保留了源手语序列的语义，这与将SLT集成到辅助技术的最终目标不符。因此，需要将评估重点从手语翻译转向手语理解，特别是语义理解。

Result: 在SLU-2K基准上的评估结果显示，多模态大语言模型表现接近随机水平，而经过领域内数据精心微调的最先进翻译系统（MMSTL和SpaMo）的准确率也仅在56.7%到75.2%之间，表明存在显著的语义鸿沟。

Insight: 论文的核心创新在于提出了一个基于问答的语义评估基准SLU-2K及其自动化生成流程，将评估重点从词汇匹配转向对动作、人物、对象等关键语义成分的理解恢复。这揭示了当前SLT评估协议高估了真实理解能力，并指出未来进展应同时衡量流畅性、n-gram重叠和语义正确性。

Abstract: Sign Language Translation (SLT) is typically evaluated with surface-form metrics such as BLEU and ROUGE, which reward lexical overlap but do not directly measure whether a translation preserves the meaning of the source sign sequence. This is in contrast with the final objective of integrating SLT in assistive technology. In this work, we shift the focus from Sign Language Translation (SLT) to Sign Language Understanding (SLU), with particular emphasis on semantic understanding. Specifically, we evaluate systems based on their ability to correctly recover, from the input video, key semantic aspects of the original sentence, such as actions taking place and facts about people and objects. To enable this evaluation systematically, we propose SLU-2K, a dataset of 2,350 closed-ended video question-answer pairs based on the popular PHOENIX-2014T and CSL-Daily datasets. To obtain SLU-2K, we propose and extensively evaluate an automated data generation pipeline which produces questions across 7 categories, namely actions, locations, numbers, objects, people, time, and weather conditions. We show the potential of SLU-2K by evaluating popular Multimodal Large Language Models (MLLMs) and two representative state-of-the-art systems, MMSTL and SpaMo. Our results show that MLLMs reach near-random performance, highlighting the need for a more systematic integration of SLU in current AI systems. Furthermore, state-of-the-art translation systems carefully fine-tuned on in-domain data still exhibit a substantial semantic gap, with results ranging from 56.7% to 75.2%. These findings suggest that current SLT evaluation protocols overestimate true understanding and that future progress should be measured not only by fluency and n-gram overlap, but also by semantic correctness. Code, prompts, and benchmark files are available at https://github.com/ZenoTsT/SLU-2K

[94] Beyond Compression: Quantifying Spectral Accessibility in Vision Representations cs.CVPDF

Akayou A. Kitessa, Yijun Zhao

TL;DR: 本研究通过空间频率可访问性分析视觉表征在视觉语言模型中的变化，引入残差谱损失（RSL）来量化超越维度压缩的效应，发现CLIP和DINOv2模型在ImageNet和MS-COCO数据集上均呈现频率依赖的可访问性变化，且中间层和池化机制是谱变换的主要驱动因素。

Details

Motivation: 探究视觉语言模型中学习到的投影层如何改变视觉信息的结构，特别是通过空间频率可访问性来量化表征变化，以隔离维度减少和优化混淆效应。

Result: 实验显示CLIP和DINOv2在ImageNet和MS-COCO上有一致的频率依赖可访问性变化，谱可访问性在深度上呈非单调轨迹，在中间层达到峰值后下降；CLIP的投影层在谱上中性，变化由压缩解释，而DINOv2的[CLS]池化导致结构化谱损失。

Insight: 创新点在于引入RSL指标来分离维度压缩效应，并发现中间层和池化机制是视觉编码器中谱变换的关键，这为理解模型表征的频谱特性提供了新视角。

Abstract: Vision-language models map visual features into a shared embedding space through learned projection layers, yet it remains unclear how these transformations alter the structure of visual information. This study examines changes in representation through spatial-frequency accessibility, measured by the linear recoverability of band-limited Fourier energy from model representations. To isolate effects beyond dimensionality reduction, we introduce Residual Spectral Loss (RSL), which evaluates changes relative to a dimension-matched random projection baseline. To reduce confounding effects from optimization, the analysis uses pretrained models with all parameters frozen. The experimental results show consistent frequency-dependent changes in accessibility across CLIP and DINOv2 on ImageNet and MS-COCO datasets. Spectral accessibility follows a non-monotonic trajectory across depth, peaking at intermediate layers before decreasing toward the output representation. The final transformation differs across architectures: CLIP’s learned projection is spectrally neutral, with changes explained by compression, whereas DINOv2’s [CLS] pooling induces a structured loss across the spectrum. These findings identify intermediate layers and pooling mechanisms as primary drivers of spectral transformation in modern vision encoders.

[95] Where Do We (Not) Need Temporal Context in Low-Resource Video Task Adaptation? cs.CVPDF

Luc P. J. Sträter, Hazel Doughty

TL;DR: 本文系统研究了视频理解任务中的模型适应策略，重点关注参数高效微调（PEFT）和探针方法在低资源场景下的应用。通过比较不同方法在基于外观、基于运动和空间密集任务中的表现，探讨了时间上下文在模型主干、PEFT模块和探针之间的最佳分配方式。

Details

Motivation: 解决视频理解中参数高效适应方法缺乏系统比较的问题，特别是时间上下文在模型不同组件中的分配策略不明确，这对于标注和计算成本高昂的视频任务至关重要。

Result: 研究在有限数据场景下评估了多种适应策略，结果表明时间上下文的合理分配对视频适应效果具有关键影响，为不同任务设置下的PEFT和探针方法提供了新的性能见解。

Insight: 创新点在于首次系统比较了图像预训练模型适应与视频表示直接适应的差异，并提出了时间上下文分配框架；客观分析认为其核心贡献是为低资源视频任务建立了适应策略的选择指南。

Abstract: Parameter-efficient fine-tuning (PEFT) and probing enable adaptation of foundation models using only a small number of trainable parameters, making it attractive for video understanding where annotation and computation are expensive. However, video PEFT has focused on adapting image-pretrained models, while standard PEFT methods can also be applied to video representations. These settings are rarely compared and both confine temporal reasoning to a single component of the model, leaving open how temporal context should be distributed across backbone, PEFT and probe. In this work we provide a systematic study of model adaptation strategies for video understanding. We evaluate methods across appearance-focused, motion-focused and spatially dense settings, with a particular focus on scenarios with limited data where parameter-efficiency is most beneficial. Our results provide new insights into PEFT and probing across settings and demonstrate the importance of temporal context allocation for effective video adaptation

[96] TeX-1500: A Paired Real-World LWIR Hyperspectral Dataset and Benchmark for Temperature-Emissivity-Texture Decomposition cs.CVPDF

Cheng Dai, Jiale Lin, Hongyi Xu, Bingxuan Song, Ziyang Xie

TL;DR: 本文提出了TeX-1500，一个用于温度-发射率-纹理分解的大规模配对长波红外高光谱数据集和基准。该数据集包含1522个真实场景的校准配对样本，涵盖不同地点、季节、时间和传感器，并提供了校准的辐射立方体、波长位置以及对齐的温度、发射率和纹理监督。

Details

Motivation: 现有温度-发射率-纹理分解方法主要是针对特定场景的逆向求解器，缺乏配对的LWIR HSI-TeX监督数据，限制了基于学习的方法发展。本文旨在填补这一空白。

Result: 在DARPA IH推扫式场景的保留集上进行实验，并零样本/少样本迁移到FTIR场景，结果表明TeX-1500提供了可用的配对监督和一个可测量的基准，用于数据驱动的以物理属性为中心的热感知。

Insight: 创新点在于构建了首个大规模、真实场景、多源、校准的配对LWIR HSI-TeX数据集，并提供了一个波长感知的基线模型TeX-UNet，为数据驱动的热物理属性分解建立了基准。

Abstract: Temperature-emissivity-texture (TeX) decomposition seeks to recover object heat state, material spectral response, and visible-like geometric texture from long-wave infrared hyperspectral imaging (LWIR HSI). Existing TeX pipelines are mainly scene-specific inverse solvers, and the lack of paired LWIR HSI-TeX supervision has limited learning-based decomposition. To address this gap, we introduce TeX-1500, a large-scale paired LWIR HSI-TeX dataset and benchmark for supervised HSI-to-TeX decomposition. TeX-1500 contains 1,522 calibrated real-scene pairs from DARPA Invisible Headlights (DARPA IH) pushbroom imagery and our FTIR acquisitions, covering five locations, four seasons, diverse acquisition times, heterogeneous wavelength layouts, and two sensor families. Each sample stores a calibrated valid-band radiance cube, calibrated wavelength positions, and aligned temperature, emissivity, and texture supervision constructed through a consistent restoration and TeX-construction protocol. We further provide TeX-UNet, a simple wavelength-aware baseline that maps calibrated HSI bands and wavelength positions to TeX fields. Experiments on the held-out DARPA IH pushbroom scenes and zero-/few-shot transfer to FTIR scenes show that TeX-1500 provides usable paired supervision and a measurable benchmark for data-driven physical-property-centered thermal perception.

[97] Visual Instruction Tuning Aligns Modalities through Abstraction cs.CV | cs.CL | cs.LGPDF

Luis Palacios, Lorenzo Basile, Diego Doimo, Alberto Cazzaniga

TL;DR: 该论文研究了视觉指令调优如何将视觉特征嵌入到大型语言模型（LLM）的分层抽象结构中。研究发现，指令调优主要作为桥梁，将视觉特征直接嵌入到LLM的中间语义层，绕过早期单模态处理层。这些中间层是多模态处理的语义核心，对性能至关重要，且微调通过扩展和加强现有抽象阶段来对齐视觉与文本特征。

Details

Motivation: 旨在探究视觉指令调优如何将视觉信息整合到预训练LLM的层次抽象结构中，以理解多模态整合的具体机制。

Result: 通过限制微调仅作用于中间层，在视觉中心基准测试上保持了与完全微调相当的性能，同时减少了训练时间。

Insight: 多模态整合是一个局部化现象，由LLM内部抽象引擎的重新利用驱动；中间语义层是关键对齐点，这为高效微调提供了新策略。

Abstract: Visual instruction tuning effectively adapts a pre-trained Large Language Model (LLM) to process image information alongside text. Yet, it remains unclear how visual features are embedded into the layer-wise hierarchy of abstractions of the LLM backbone. Across a diverse set of vision-language architectures, we show that instruction tuning primarily serves as a bridge, embedding visual features directly into the intermediate semantic layers of the LLM, bypassing the early layers devoted to unimodal processing. With probing analyses and causal interventions, we show that these intermediate layers are the semantic core of vision-language processing and play a critical role in the performance on a broad set of multimodal benchmarks. In addition, by comparing the geometry of semantically equivalent visual and textual representations, we find that fine-tuning extends and strengthens the existing abstraction phase, aligning visual features with pre-existing textual ones. Finally, we confirm the functional role of this localized alignment by restricting fine-tuning to intermediate layers alone: this strategy preserves the performance of full fine-tuning on vision-centric benchmarks while reducing training time. Our results suggest that multimodal integration is a localized phenomenon driven by the repurposing of the internal abstraction engine of the LLM.

[98] Unified Video-Action Joint Denoising for Dexterous Action and Data Generation cs.CVPDF

Dingrui Wang, YuAn Wang, Jinkun Liu, Yue Zhang, Mattia Piccinini

TL;DR: 本文提出了一种名为Donk的统一视频-动作联合去噪模型，用于灵巧手操作。该模型能够在语言、初始图像和初始手部状态条件下，采样生成未来视频和双手MANO轨迹作为动作策略；在没有图像条件时，则从文本条件分布中采样生成配对的视频-动作序列，从而将对齐的视频先验转化为数据生成引擎。

Details

Motivation: 现有方法通常将对齐的先验窄化为基于观测的未来动作策略分布，而本文从分布视角重新审视视频基础模型与可执行机器人动作的对齐问题，旨在通过建模交互视频和可执行手部轨迹的联合空间，保持更广泛的分布，以提升灵巧操作的性能和数据生成能力。

Result: 在动作、视频和纯文本生成评估中，Donk提高了灵巧轨迹的准确性，保持了较强的视频保真度，并在统一的训练方案下生成了平滑的文本条件动作序列。

Insight: 创新点在于提出了一个统一的联合去噪架构，将视频生成与动作策略学习整合，并利用多条件机制（语言、图像、手部状态）实现从策略到数据生成的灵活转换，这为利用视频先验进行机器人动作学习和合成数据提供了新思路。

Abstract: Recent world action models leverage video foundation models by aligning broad visual-dynamics priors with executable robot actions. We revisit this alignment from a distributional perspective. Existing formulations typically narrow the aligned prior into an observation-conditioned policy distribution over future actions. In contrast, we keep the distribution broader by modeling the joint space of interaction videos and executable hand trajectories under multiple conditioning regimes. We propose Donk, a unified video-action denoising model for dexterous hands. With language, an initial image, and the initial hand state, Donk samples future videos and bimanual MANO trajectories as an action policy. Without the image condition, the same denoising architecture samples paired video-action rollouts from a text-conditioned distribution, turning the aligned video prior into a data engine. Across action, video, and text-only generation evaluations, Donk improves dexterous trajectory accuracy, preserves strong video fidelity, and produces smooth text-conditioned action rollouts under the same unified training recipe.

[99] DyaPlex: Full-Duplex Speech-Motion Model for Dyadic Interaction cs.CV | cs.ROPDF

Koki Nagano, Hongyu Liu, Seonwook Park, Tianye Li, Amrita Mazumdar

TL;DR: DyaPlex是一个为双人交互设计的流式全双工语音-动作模型，能够同时感知和生成语音与身体动作，实现完全同步的多模态交互。该模型基于一个基础的全双工语音模型，通过双塔Transformer架构和统一的交互令牌交错机制，将动作路径与语音特征对齐。在4000小时的Seamless Interaction数据集上训练后，在单人和双人交互基准测试中达到了新的最先进性能。

Details

Motivation: 为了捕捉人类交流的连续性和互惠性，解决在流式交互中同时处理语音和动作的同步问题，以更自然地模拟双人对话。

Result: 在单人和双人人类交互基准测试中建立了新的最先进（SOTA）性能，有效捕捉了跨说话者依赖关系。

Insight: 创新点包括全双工流式设计、双塔Transformer架构保持基础语音模型的零样本推理能力，以及通过时间对齐的语音-动作RoPE引导交叉注意力，实现动作与语音潜在特征的有效对齐。

Abstract: We present DyaPlex, a streaming, full-duplex speech-and-motion model designed for dyadic interaction. To capture the continuous and reciprocal nature of human communication, this full-duplex capability empowers the agent to simultaneously perceive and generate both speech and physical motion in a streaming fashion. At its core, our method leverages the strong priors of a foundational full-duplex speech model and integrates a novel motion pathway, thereby achieving fully synchronized multi-modal interaction. Specifically, we design a dual-tower Transformer architecture that preserves the zero-shot conversational reasoning of a frozen base speech model while constructing a deeply coupled, streaming motion pathway. By introducing a unified dyadic token interleaving mechanism and guiding cross-attention via a time-aligned speech-motion RoPE, our model effectively aligns autoregressive motions with rich latent speech features. Trained on the 4,000-hour Seamless Interaction dataset, our model effectively captures cross-speaker dependencies and establishes new state-of-the-art performance across both monadic and dyadic human interaction benchmarks.

[100] MLP Splatting: Object-Centric Neural Fields cs.CVPDF

Shinjeong Kim, Yuzhou Cheng, Xin Kong, Paul H. J. Kelly, Andrew J. Davison

TL;DR: 本文提出了MLP Splatting方法，通过少量表达力强的光场基元实现场景分解，同时提供逼真的新视角合成。该方法将每个基元建模为一个独立的紧凑MLP，具有局部空间支持，可预测辐射和透明度。仅使用RGB监督即可训练基元，使其代表局部场景区域（通常对应物体或部件），从而实现无需分割掩码的交互式物体级编辑。

Details

Motivation: 现有方法（如3D高斯泼溅和神经辐射场）虽能实现逼真的新视角合成，但难以将场景元素轻松分解为少量基元，需要额外的分割或分组才能进行物体级操作。本文旨在解决这一场景分解与编辑的局限性。

Result: 相比现有方法，MLP Splatting在内存使用上大幅降低（1/15倍），渲染速度更快（3倍）。在实验中，与语义3DGS方法相比，展现出显著优势。

Insight: 创新点在于将神经基元（MLP）作为局部化、高表达力的场景表示单元，仅需RGB监督即可实现物体级分解与编辑。通过稀疏体素合成实现高效渲染，并可结合语义特征蒸馏支持开放词汇场景交互与开放集即时分割。

Abstract: 3D representations are fundamental to scene rendering, understanding, and interaction. Recent approaches, such as 3D Gaussian Splatting and Neural Radiance Fields, achieve impressive photorealistic novel-view synthesis, but lack the ability to easily decompose scene elements into a few primitives, requiring additional segmentation or grouping for object-level manipulation. We present MLP-Splatting, a method that enables scene decomposition via a few expressive light-field primitives while providing photorealistic novel-view synthesis. MLP-Splatting models each primitive as an independent compact MLP with localized spatial support that predicts radiance and opacity. In contrast to low-level Gaussian primitives or a single global radiance field, our neural primitives provide greater expressive capacity while remaining spatially localized. Rendering is performed through efficient sparse volumetric compositing over ray-primitive interactions. Our primitives are supervised using RGB supervision alone, which yields primitives that represent local scene regions often corresponding to objects or object parts, enabling interactive object-level editing without segmentation masks by selecting a handful of primitives. Our method, augmented with optional semantic feature distillation, enables open-vocabulary scene interaction and open-set instant segmentation. Compared to state-of-the-art methods, we achieve substantially lower memory usage (1/15$\times$) and faster rendering (3$\times$), as we show in our experiments compared to semantic 3DGS methods. Project Page: https://shinjeongkim.com/mlp-splatting

[101] Beyond Encoder Accumulation: Measuring Encoder Roles in Multi-Encoder VLMs cs.CV | cs.AIPDF

Wei Ding, Yudong Zhang, Ruobing Xie, Xingwu Sun, Jiansheng Chen

TL;DR: 本文通过重新训练和评估五种常见视觉编码器的31个子集，研究了多编码器视觉语言模型（VLM）中编码器在联合训练下的角色。研究发现，编码器的排名在从头训练与固定检查点掩码下存在差异，并提出了Capacity和Necessity两个维度来分解编码器贡献。此外，编码器在投影层前的有效秩解释了剩余性能差异，最强编码器对由在联合训练中保持高秩的锚点编码器和秩扩展的补充编码器组成。

Details

Motivation: 随着基础模型融合更多异构视觉流，理解不同编码器在联合训练下的交互对于原则性设计至关重要，但目前大型视觉语言模型缺乏相关工具，且参数高效的编码器配置在训练前难以识别。

Result: 在16个基准的Cambrian-1套件上，通过统一流程重新训练和评估所有编码器子集（总计约20k GPU小时），发现最高Capacity编码器配对并非最优，而高Capacity锚点与自适应补充编码器的配对能匹配完整五编码器模型。

Insight: 创新点在于提出了Capacity和Necessity分解框架以及投影层前有效秩分析，揭示了编码器交互机制，为多编码器VLM设计提供了方法论指导和具体优化原则，如锚点-补充编码器配对策略。

Abstract: As foundation models scale toward fusing more heterogeneous visual streams, understanding how diverse encoders interact under joint training becomes a prerequisite for principled design. Yet large vision-language models (LVLMs) currently lack the tools to do so, and parameter-efficient encoder configurations remain hard to identify before training. To re-examine encoder roles under joint training, on the 16-benchmark Cambrian-1 suite we retrain and evaluate all 31 non-empty subsets of five common vision encoders under a unified pipeline (~20k GPU-hours total), and report three findings. First, retraining each subset from scratch reveals encoder rankings that differ from those obtained by masking encoders on a fixed checkpoint, including which encoder ranks first overall. Second, we decompose each encoder’s contribution into two axes, Capacity, the score an encoder reaches on its own, and Necessity, the drop when it is removed from the full pool. The two axes are not interchangeable. Pairing the two highest-Capacity encoders is suboptimal, while pairing a high-Capacity anchor with an adaptive complement matches the full five-encoder model. Adding further encoders beyond this pair yields only marginal gains. Third, at fixed parameter count, per-encoder pre-projector effective rank explains the residual score variation. The strongest pairs combine an anchor whose rank survives joint training with a complement whose rank expands under it, suggesting that higher-rank, less-collapsed projector inputs correspond to a more favorable optimization regime at the encoder-projector interface. Together, the Capacity-Necessity decomposition and the pre-projector rank analysis, along with comprehensive evaluation through retraining, expose a methodological gap in multi-encoder LVLM design, and offer concrete primitives for closing it.

[102] Bootstrap Your Generator: Unpaired Visual Editing with Flow Matching cs.CVPDF

Yoad Tewel, Yuval Atzmon, Gal Chechik, Lior Wolf

TL;DR: 本文提出了一种名为Bootstrap Your Generator（ByG）的无配对视觉编辑框架，通过利用预训练基础模型的知识并结合循环一致性来保持结构，实现了无需外部信号或大规模配对数据的高效图像和视频编辑。

Details

Motivation: 现有生成模型在视觉编辑任务中通常依赖大量配对数据，这在视频编辑等场景下成本高昂且难以扩展，因此需要一种无需配对数据的训练方法。

Result: 在数据稀缺的图像和视频编辑任务上，该方法取得了最先进（SOTA）的结果，并通过用户研究和广泛评估证明其能够泛化到未见领域，甚至优于基于数百万样本训练的监督基线。

Insight: 创新点在于提出梯度路由机制来弥合训练与推理之间的差距，并从基础模型中提取语义线索作为鲁棒的训练信号，从而无需外部奖励模型，实现了高效的无配对编辑训练。

Abstract: Modern generative models possess a deep understanding of visual content, yet training them for image editing typically requires massive datasets of paired examples. This limits scalability, especially for video editing where collecting paired data is prohibitively expensive. We propose Bootstrap Your Generator (ByG), a general framework for unpaired training of flow matching editing models. It leverages the base model’s knowledge without any external signal. Our approach pairs instruction-following cues extracted from the frozen model with cycle-consistency for structure preservation. To make this tractable, we propose to route gradients from downstream losses over clean predictions to noisy training states. We demonstrate state-of-the-art results on challenging data-scarce image and video editing scenarios. Extensive evaluations and user studies show that our method effectively generalizes to unseen domains and outperforms supervised baselines trained on millions of samples. Analysis reveals that our gradient routing bridges the train-inference gap, and extracting semantic cues from a base model provides a robust training signal that obviates the need for external reward models.

[103] OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMs cs.CVPDF

Yifei Li, Pengyiang Liu, Yuhang Zang, Zhongyue Shi, Qi Fu

TL;DR: 论文提出了OVO-S-Bench，一个用于评估多模态大语言模型（MLLMs）流式空间智能的层次化基准测试。该基准包含1,680个基于348个第一人称视角视频的问题，覆盖从瞬时感知到空间推理的四个抽象层次。评估发现，即使是顶尖模型（如Gemini-3.1-Pro）也与人类专家存在显著差距，且专门针对流式或空间微调的模型表现不佳。

Details

Motivation: 现有的基准测试要么评估离线完整视频，要么关注事件而非空间结构，无法满足机器人、AR和自动驾驶等领域中智能体需要从连续的第一人称视角流中推理地点和布局的需求。

Result: 在38个专有和开源MLLMs上的测试结果显示，Gemini-3.1-Pro得分59.2，落后于人类专家（86.6）27个百分点，其中以自我为中心到以客体为中心（allocentric）的映射是主要瓶颈。专门针对流式或空间微调的模型表现甚至不如其骨干模型。

Insight: 创新点在于构建了一个完全人工标注、具有查询时间戳和证据区间的流式空间智能基准，并揭示了当前MLLMs在流式空间推理上的核心缺陷，如脱离视频流的思维链推理会放大错误，这为下一代模型的发展指明了方向。

Abstract: Multimodal agents in robotics, AR, and autonomous driving must reason about places and layouts from continuous egocentric streams, often using evidence outside the current view. Existing benchmarks either evaluate offline over full videos or target events rather than spatial structure. We introduce OVO-S-Bench, a fully human-annotated benchmark for streaming spatial intelligence, comprising 1,680 questions over 348 source videos. Annotation involves 12 trained annotators, each also serving as a blind cross-reviewer, across roughly 804 person-hours of multi-round quality assurance. Each question carries a query timestamp and an evidence interval, and at evaluation, the model sees only the prefix preceding the query. Questions span four levels of increasing abstraction: instantaneous egocentric perception, spatiotemporal context tracking, spatial simulation and reasoning, and allocentric mapping. Across 38 proprietary and open-source MLLMs, Gemini-3.1-Pro trails human experts by 27 points, 59.2 vs. 86.6, with allocentric mapping as the dominant bottleneck. Notably, streaming and spatially fine-tuned MLLMs underperform their own backbones. We further find that chain-of-thought reasoning amplifies spatial errors when ungrounded in the stream. By exposing these limitations, OVO-S-Bench establishes a demanding testbed for next-generation streaming spatial MLLMs.

[104] Benchmarking Visual State Tracking in Multimodal Video Understanding cs.CVPDF

Sihyun Yu, Nanye Ma, Pinzhi Huang, Hyunseok Lee, Shusheng Yang

TL;DR: 该论文提出了一个名为VSTAT的视觉状态追踪基准测试，用于评估多模态大语言模型在视频理解中对实体、状态和事件进行连续追踪的能力。研究发现，尽管现有MLLMs在传统视频基准上表现良好，但在VSTAT上远低于人类水平，且其失败主要源于视觉感知而非文本推理的缺陷。

Details

Motivation: 当前对多模态大语言模型的评估忽视了视频理解中至关重要的视觉状态连续追踪能力，因此需要一个新的基准来诊断模型在这方面的不足。

Result: 在VSTAT基准（包含834个视频片段和1500个问题）上，最先进的MLLMs表现远低于人类，仅略高于答案先验基线，且初步评估表明基于MLLM的智能体方法也未能有效解决此问题。

Insight: 论文的创新点在于构建了首个专注于视觉状态追踪的视频基准VSTAT，并揭示了MLLMs在视频理解中的核心瓶颈是视觉感知而非文本推理，这为未来模型改进指明了方向。

Abstract: Understanding a video requires more than recognizing isolated moments, as humans continuously track entities, states, and events over time. This capacity for visual state tracking is fundamental to video understanding, yet remains underexplored in current evaluations of Multimodal Large Language Models (MLLMs). We introduce Visual STAte Tracking benchmark (VSTAT), a video-based benchmark designed to diagnose visual state tracking in MLLMs. VSTAT consists of 834 clips drawn from both synthetic and real-world videos, paired with 1,500 questions that cannot be answered from any single frame or short segment, requiring continuous perception and integration of events across the entire video stream. Despite their strong performance on existing video benchmarks, we find that state-of-the-art MLLMs perform far below humans and only modestly above answer-prior baselines. To analyze this gap, we compare MLLMs’ thinking traces with the underlying video stream to understand why and when MLLMs fail on VSTAT. We find that MLLMs reason and track correctly in text, but fail at visually perceiving the events they need to track. Finally, our preliminary evaluation suggests that recent agentic approaches, including MLLM-based video agents and coding agents, do not readily resolve these failures, still falling short on VSTAT.

[105] Demo2Tutorial: From Human Experience to Multimodal Software Tutorials cs.CVPDF

Zechen Bai, Zhiheng Chen, Yiqi Lin, Kevin Qinghong Lin, Difei Gao

TL;DR: Demo2Tutorial是一个将人类在数字环境中的屏幕录制和交互日志转化为结构化、多模态软件教程的框架。它通过多模态动作解析器重建感知、动作和意图，再通过步骤规划器抽象为层次化任务图，最后生成可重用的图文教程。该框架旨在利用人类经验中的程序性知识，服务于人类学习和智能体训练。

Details

Motivation: 人类在数字环境中的经验是一个未被充分探索的、包含丰富程序性知识的真实交互资源。论文旨在解决如何自动将这些原始、未经剪辑的交互转化为结构化、可复用的多模态软件教程的问题，以同时促进人类学习和智能体能力提升。

Result: 在基于官方软件文档构建的新基准测试上，Demo2Tutorial生成的教程质量超过了人工编写的教程，并显著优于基线方法。实验表明，该框架能带来更快的人类任务完成速度和改进的GUI智能体规划与泛化能力。

Insight: 论文的核心创新在于提出了一个从原始人类经验中蒸馏结构化知识表示的端到端框架，将多模态解析、层次化任务抽象和教程生成相结合。其宣称的亮点是证明了这种从经验中提取的结构化教程能作为有效的知识表示，同时提升人类学习和智能体能力，实现了双重受益。

Abstract: Human experience in digital environments offers a vast, underexplored resource of authentic, untrimmed interactions that contain rich procedural knowledge. We introduce Demo2Tutorial, a framework that transforms this experience captured via screen recordings and interaction logs into structured, multimodal software tutorials for teaching both humans and agents. Demo2Tutorial first collects human experience via a dedicated recorder, then parses raw experience using a multimodal Action Parser to reconstruct perception, action, and intent. A Step Planner then abstracts these steps into hierarchical task graphs representing goals and steps. Finally, a Tutorial Composer transforms the parsed experience into structured, reusable image-text instructions. We evaluate the tutorial generation quality on a new benchmark derived from official software documentation. We further demonstrate that this distilled representation benefits (i) human learning, by automatically generating multimodal tutorials, and (ii) agent learning, by improving downstream GUI-agent planning and generalization. Experiments show Demo2Tutorial produces high-quality tutorials that surpass human-authored ones and significantly outperform baseline methods, while enabling both faster human task completion and improved GUI agent planning, demonstrating that structured tutorials distilled from human experience can serve as effective knowledge representations for advancing both human learning and agent capabilities. Code and data will be available at https://github.com/showlab/Demo2Tutorial.

[106] VLESA: Vision-Language Embodied Safety Agent for Human Activity Monitoring cs.CV | cs.LG | cs.ROPDF

Hanjiang Hu, Yiyuan Pan, Jiaxing Li, Xusheng Luo, Alexander Robey

TL;DR: 本文提出了VLESA（视觉语言具身安全智能体）框架，用于通过第一人称视角视频监控人类活动，并在预测到危险行为时触发实时安全干预。该框架解决了意图依赖的安全性问题，即相同行为在不同情境下可能安全或危险，并引入了包含目标条件安全标注的数据集。

Details

Motivation: 随着AI系统越来越多地协助人类完成物理任务，确保安全变得至关重要，因为物理行为会带来即时且不可逆的后果。论文旨在解决意图依赖的安全监控问题，即需要根据上下文推断意图来判断行为的安全性。

Result: 在ASIMOV-2.0基准测试中，VLESA在精确匹配真实帧的干预准确率上优于基线方法；通过GRPO训练的目标条件Q过滤器，通过目标条件约束解码将动作安全性提高了超过41个百分点。

Insight: 创新点在于提出了一个结合目标推断与动作预测的联合框架，并引入了目标条件安全Q过滤器，无需重新训练即可根据推断的意图评估动作安全性，实现了对上下文敏感的安全监控。

Abstract: As AI systems increasingly assist humans in physical tasks, ensuring safety becomes paramount – physical actions carry immediate and irreversible consequences that digital errors do not. We introduce the Vision-Language Embodied Safety Agent (VLESA), a framework that monitors human activities from egocentric video and triggers real-time safety interventions when dangerous actions are predicted. VLESA addresses intent-dependent safety where identical actions can be safe or dangerous depending on context. A dataset pairing egocentric frames with goal-conditioned safety annotations is introduced, enabling a goal-conditioned safety Q-filter trained via GRPO that evaluates actions with respect to inferred intent without retraining. On top of that, an intent-action prediction agent is proposed to jointly infer goals and predict future actions from video. On the ASIMOV-2.0 benchmark, VLESA achieves higher intervention accuracy at the exact ground-truth frame compared to baselines, while the GRPO-trained Q-filter improves action safety by over 41 percentage points through goal-conditioned constrained decoding. Code is available at https://github.com/HanjiangHu/VLESA.

[107] Video-Mirai: Autoregressive Video Diffusion Models Need Foresight cs.CVPDF

Yonghao Yu, Lang Huang, Runyi Li, Zerun Wang, Toshihiko Yamasaki

TL;DR: 本文提出了Video-Mirai，一种仅通过训练改进、无需改变推理架构的自回归视频扩散模型方法。该方法通过引入一个冻结的前瞻编码器和一个轻量级预测器，在训练期间将未来帧的信息蒸馏到因果状态中，以弥补标准训练导致的‘表示级规划鸿沟’，从而提升长视频生成的一致性和质量。

Details

Motivation: 标准的自回归视频扩散模型训练只要求每个因果状态解释当前片段，这可能导致状态丢弃了对未来一致性（如身份、布局和运动）至关重要的信息，造成表示级规划鸿沟。论文旨在解决这一问题，使模型在保持因果推理的同时获得前瞻能力。

Result: 在5秒VBench基准测试中，Video-Mirai将强基线Causal-Forcing的总分从83.8提升至84.6。在超出训练范围的30秒生成长度上，主体一致性从84.9提升至88.5，背景一致性从90.2提升至91.9。消融实验表明未来条件目标是关键因素。

Insight: 核心创新在于提出了一种训练时注入前瞻信息、推理时保持原始架构和计算开销不变的方法。其关键思想是‘因果性应约束推理，而非表示监督’，即通过未来帧监督中间表示而非生成器输入，从而在不牺牲效率的前提下提升长程一致性。这为自回归视觉模型的训练范式提供了新思路。

Abstract: Causal video generators must predict from the past, but they need not learn only from it. In streaming autoregressive video diffusion, each emitted segment becomes a commitment that future segments must preserve. Standard training, however, only asks each causal state to explain the present. This creates what we call a representation-level planning gap: states that fit the current segment may discard identity, layout, and motion information needed for a consistent future. We introduce Video-Mirai, a training-only method that closes this gap without changing causal inference: the generator rolls out causally, a frozen foresight encoder reads the completed rollout non-causally, and a lightweight predictor distills the resulting stopped-gradient targets into causal states. Future frames supervise representations, never generator inputs. At inference, the encoder and predictor are discarded, leaving the original architecture, per-step FLOPs, and KV-cache behavior unchanged. Video-Mirai improves a strong Causal-Forcing baseline on 5-second VBench from 83.8 to 84.6 in terms of Total Score. On 30-second rollouts beyond the training horizon, subject consistency improves from 84.9 to 88.5 and background consistency from 90.2 to 91.9. Ablations identify future-conditioned targets as the key ingredient, and probes show that future frames become more decodable from current features. Causality should constrain inference, not representation supervision. Our study highlights that visual autoregressive models need foresight. Project page: https://y0uroy.github.io/Video-Mirai.

[108] AAD-1: Asymmetric Adversarial Distillation for One-Step Autoregressive Video Generation cs.CVPDF

Haobo Li, Yanhong Zeng, Yunhong Lu, Jiapeng Zhu, Hao Ouyang

TL;DR: 本文提出了AAD-1，一种用于一步自回归图像到视频生成的非对称对抗蒸馏框架。该框架通过非对称的生成器-判别器架构和分阶段的训练策略，解决了现有对抗蒸馏方法中存在的运动崩溃和训练不稳定问题，从而生成动态连贯的视频。在VBench基准测试中取得了最先进的性能。

Details

Motivation: 现有最先进的对抗蒸馏方法在自回归视频生成中面临运动崩溃和训练不稳定的挑战，导致生成的视频趋于静态。本文旨在解决这些问题，以生成高质量、动态连贯的一步自回归视频。

Result: 在VBench基准测试上进行的广泛实验表明，AAD-1在一步自回归视频生成任务中取得了最先进的（SOTA）性能。

Insight: 核心创新点在于非对称架构设计：生成器保持因果性以维持自回归采样能力，而判别器则利用双向的时空上下文对整个视频序列产生一个整体的真实感评分，从而有效检测导致运动崩溃的全局时序故障。此外，分阶段训练策略（先通过分布匹配引导稳定的一步生成器，再进行对抗蒸馏）有助于稳定训练过程。

Abstract: We present AAD-1, an Asymmetric Adversarial Distillation framework for One-step autoregressive image-to-video generation. State-of-the-art methods adopt adversarial distillation but suffer from motion collapse and training instability, resulting in static videos. AAD-1 addresses these challenges through two key designs in architecture and training strategy. Our key architectural insight is to break the symmetry between generator and discriminator. While the generator remains causal to preserve autoregressive sampling capability, the discriminator attends bidirectionally over the full spatiotemporal context and produces a single holistic realism score for the entire video sequence. This asymmetric design enables the discriminator to effectively detect global temporal failures and long-range drift that cause motion collapse in autoregressive generation. To stabilize training, we introduce a phased strategy that first uses distribution matching to bootstrap a stable one-step generator, providing a warm-up phase that brings the student distribution closer to the teacher before adversarial distillation begins. Extensive experiments on VBench demonstrate that AAD-1 achieves state-of-the-art performance in one-step autoregressive video generation.

[109] Formalizing the Binding Problem cs.CV | cs.AI | cs.LG | q-bio.NCPDF

Lianghuan Huang, Yihao Li, Saeed Salehi, Yingshan Chang, Ansh Soni

TL;DR: 本文通过信息论方法形式化定义了绑定问题，并提出了一种探测方法来测量模型表示中的绑定信息。研究在Vision Transformers（ViTs）上进行了实验，测量了不同架构组件（如[CLS]令牌和空间令牌）的绑定信息，并使用了具有不同绑定挑战的数据集（如特征共享、遮挡和自然特征）。

Details

Motivation: 尽管已有研究表明Vision Transformers知道哪些图像块属于同一物体，但当前深度学习模型是否学习并表现出特征级别的绑定信息尚不清楚，尤其是在多物体场景中错误分配特征（如将特征归因于错误物体）是ViT架构的常见失败案例。

Result: 实验在多个预训练ViT模型上进行，使用了包含特征共享、遮挡和自然特征等挑战的数据集，以比较不同模型的绑定信息测量结果。研究表明绑定是强大视觉识别和推理的关键要素。

Insight: 创新点在于使用信息论形式化绑定问题，并引入一种通用的探测方法来量化模型表示中的绑定信息，这为评估和改进模型在多物体场景下的理解能力提供了新工具。

Abstract: Representations of the world, arguably, contain information about features (e.g. something is blue, something is a circle) but also information about which features are part of the same object (e.g. the circle is blue), which we call binding information. Any system with the ability to understand scenes with multiple objects must be able to solve the binding problem: it needs to know which features belong together. However, despite work showing that Vision Transformers (ViTs) know which patches belong together, it is not known whether current deep learning models learn to exhibit binding information, i.e., for features. We may believe that there is not much binding information, after all misattributing features to wrong objects is a common failure of ViT-based architectures, especially in scenes with objects sharing features. Here we formalize the binding problem with an information-theoretic approach, and introduce a probing method to measure binding information in model representations. We perform experiments on ViTs, measuring binding from different components of the architecture, such as the image summary token [CLS] or the spatial tokens. We use datasets with different binding challenges, such as feature sharing, occlusion, and natural features, while comparing the performance of several pre-trained ViTs. Overall, our research demonstrates binding as a key ingredient to strong visual recognition and reasoning.

[110] NewtPhys: Do Foundation Models Understand Newtonian Physics? cs.CVPDF

Sebastian Cavada, Soumava Paul, Tuan-Hung Vu, Andrei Bursuc, Raoul de Charette

TL;DR: 该论文提出了NewtPhys数据集，这是一个基于真实世界多视角图像构建的4D物理标注数据集，用于评估基础模型对牛顿力学的理解能力。通过该数据集，作者系统评估了56个视觉语言模型和10个视觉基础模型，揭示了它们在低级物理推理方面的局限性。

Details

Motivation: 现有评估基础模型物理推理能力的数据集多采用合成或半合成场景，缺乏视觉真实性和对低级牛顿力学理解的细粒度评估，因此需要构建更贴近真实世界复杂性的数据集。

Result: 在NewtPhys数据集上对66个模型（包括54个开源模型和2个闭源前沿模型）的评估结果显示，现有模型在低级物理推理方面存在显著不足。

Insight: 创新点在于构建了一个结合真实视觉复杂性与物理模拟的密集细粒度标注数据集，为物理基础视觉研究和下一代物理感知评估提供了新的基准和工具。

Abstract: Previous work has evaluated physics reasoning in foundation models using synthetic or semi-synthetic scenes and visual question-answering tasks. However, these benchmarks emphasize high-level events and lack the visual fidelity required to assess true low-level Newtonian understanding. We introduce NewtPhys, a 4D physically annotated dataset built from multiview images of real-world scenes with physics-grounded simulations. The dataset provides dense, fine-grained annotations across timesteps – including 3D forces and amodal per-pixel quantities covering physics, tracking, semantics and geometry – bridging the gap between simplistic synthetic setups and realistic visual complexity. Using NewtPhys, we systematically evaluate 56 VLMs, including 54 open-weight models and 2 closed-source frontier models, and 10 VFMs and reveal limitations in low-level physics reasoning. Beyond benchmarking, our dataset enables future research in physics-grounded vision and the development of next-generation physics-aware evaluations. Code and datasets are available at https://astra-vision.github.io/NewtPhys.

[111] Exploring Easy Boosts for Lidar Semantic Scene Completion cs.CV | cs.ROPDF

Tetiana Martyniuk, Jonathan Seele, Alexandre Boulch, Gilles Puy, Renaud Marlet

TL;DR: 本文研究了无需复杂架构重设计即可提升激光雷达语义场景补全（SSC）性能的“免费午餐”策略。通过为输入点云添加来自现成分割器的语义伪标签，以及区分空与未知空间的可见性信息，显著提升了现有模型的性能，使旧模型仍能与最先进系统竞争甚至超越。

Details

Motivation: 解决在不进行复杂架构改动的情况下，如何有效提升激光雷达语义场景补全性能的问题，探索简单易行的增强策略。

Result: 在SSC基准测试中，使用语义伪标签和可见性信息增强后，旧模型性能显著提升，达到与最先进（SOTA）系统竞争甚至超越的水平。

Insight: 创新点在于利用现成分割器提供的语义先验和可见性信息作为简单增强，证明了高质量语义先验是性能提升的主要驱动力，为SSC任务提供了低成本高效的改进途径。

Abstract: This paper investigates “free lunch” strategies to boost the performance of lidar semantic scene completion (SSC) without requiring complex architectural redesigns. We first demonstrate that endowing input point clouds with semantic pseudo-labels from off-the-shelf segmentors significantly improves the performance of existing architectures. By evaluating these models against an oracle, we establish that high-quality semantic priors are a primary driver of mIoU gains. Furthermore, we equip the input lidar scan with visibility information that distinguishes between empty and unknown spaces, which provides a secondary performance boost across the tested architectures. Using these simple enhancements, we observe that older models remain competitive with state-of-the-art systems, and can even outperform them. Our code is available at https://github.com/astra-vision/SSC-Priors.

[112] PixVOD: Pixel-Distributed Direct Visual Odometry and Depth Estimation cs.CVPDF

Shinjeong Kim, Ignacio Alzugaray, Callum Rhodes, Paul H. J. Kelly, Andrew J. Davison

TL;DR: 论文提出了一种名为PixVOD的视觉里程计与深度估计方法，该方法将计算完全分布到像素层面，利用传感器内处理器通过高斯置信传播（GBP）在像素间交换信息，以达成相机运动共识并从每个像素的光度观测与表面法线先验中推断深度。为了在优化过程中保持几何稳定性，引入了类似关键帧的锚定机制来调节帧间的有效基线。

Details

Motivation: 传统计算机视觉算法以2D像素阵列作为标准输入，但将原始、冗余且噪声大的像素数据传输离传感器效率低下，因此推动向焦平面传感器-处理器发展，使大量计算直接在像素内完成，旨在让像素本地合成更高级信号，减轻下游负载并为高级视觉任务提供更丰富的输入。

Result: 该方法在真实数据集上进行了评估，证明了基于GBP的像素级分布式里程计与深度估计结合关键帧锚定在传感器上实现的可行性。

Insight: 创新点在于提出了一种完全可并行化的像素分布式视觉里程计与深度估计框架，利用GBP进行像素间信息交换以达成共识，并引入关键帧锚定机制来维持优化过程中的几何稳定性，这为在传感器端实现高效、低通信的视觉计算提供了新思路。

Abstract: Images composed of 2D pixel arrays are the standard input to computer vision algorithms, yet many underlying computations can be distributed across pixels. Transmitting raw, redundant, and noisy pixel data off the sensor remains inefficient, motivating a shift toward focal-plane sensor-processors that perform a significant part of the computation directly within each pixel. We envision pixels synthesizing higher-level signals locally, reducing downstream load, and providing richer inputs for higher-level vision tasks. We propose a fully parallelizable form of visual odometry and depth estimation across pixels, where sensor-processors exchange information through Gaussian Belief Propagation (GBP) to achieve consensus about camera motion and infer depth from per-pixel photometric observations and a surface normal prior. To maintain geometric stability during optimization, we introduce a keyframe-like anchoring mechanism that regulates the effective baseline between frames, enabling consistent motion and depth updates. Our method is evaluated on realistic datasets, demonstrating the feasibility of GBP-based pixel-level distributed odometry and depth estimation with keyframe anchoring on-sensor. Project Page: https://www.shinjeongkim.com/pixvod/

cs.GR [Back]

[113] PaintBench: Deterministic Evaluation of Precise Visual Editing cs.GR | cs.CV | cs.LGPDF

Kai Xu, Ellis Brown, Shrikar Madhu, Rob Fergus, He He

TL;DR: 本文提出了PaintBench，一个用于评估精确视觉编辑能力的动态可扩展基准测试，涵盖几何变换、结构操作、颜色变化和符号推理四大类共20种基本编辑操作。该基准通过程序化生成具有可配置复杂度的测试集，实现了无限、抗污染且确定性的像素级评估。在测试11个图像编辑模型后，发现整体性能较低，当前最佳模型仅获得17.1%的mIoU得分。此外，研究还创建了数据可视化编辑基准TinyGrafixBench，验证了PaintBench分数与应用任务性能之间的强相关性。

Details

Motivation: 当前多模态模型在开放式视觉编辑方面表现良好，但在执行精确的单答案编辑任务上仍存在重大障碍。为了解决这一挑战，需要建立一个能够系统评估模型精确编辑能力的基准。

Result: 在PaintBench上测试的11个图像编辑模型整体表现不佳，当前性能最高的行业领先模型仅获得17.1%的平均交并比（mIoU）。通过任务分解发现，几何变换、大多数结构操作和基于公式的颜色变化等操作类型尤其具有挑战性。此外，创建的TinyGrafixBench基准显示，其性能与PaintBench分数存在强线性相关（R² = 0.91, p < 0.001）。

Insight: 论文的创新点在于提出了一个程序化生成、抗污染且支持确定性像素级评估的精确视觉编辑基准PaintBench，避免了依赖有偏见的评判模型。其细粒度诊断能力可以揭示模型在特定操作类型和场景变化（如物体数量、背景复杂度）下的性能退化，为多模态视觉编辑的进展提供了严谨的衡量基础。

Abstract: While current multimodal models are proficient at open-ended visual editing, executing precise single-answer edits remains an important obstacle. To probe this challenge, we introduce PaintBench, a dynamically scalable benchmark targeting 20 fundamental precise visual editing operations across four categories: geometric transformation, structural manipulation, color change, and symbolic reasoning. Procedural generation with configurable complexity enables an effectively infinite, contamination-resistant evaluation suite, and deterministic pixel-level evaluation eliminates reliance on bias-prone judge models. Across 11 image editing models, we find overall low performance, with the current highest-performing industry leader scoring only 17.1% (mIoU). Task decomposition reveals especially challenging operation types (geometric transformation, most structural manipulation, formula-based color change) and model-specific specializations. Fine-grained benchmark diagnostics further show performance degradations induced by scene variations in object count, background complexity, color scheme, and edit-region size. To test generalization of PaintBench scores to applied task performance, we create a procedural, deterministic evaluation for data visualization editing (TinyGrafixBench) and find strong linear correlation with PaintBench scores ($R^2 = 0.91$, $p < 0.001$). Altogether, PaintBench provides a rigorous foundation for measuring and driving progress in precise multimodal visual editing.

cs.LO [Back]

[114] ZX-Calculus:Trace-Indexed Dependent Types and Epistemic Semantics cs.LO | cs.CLPDF

Peng Chen

TL;DR: 本文提出了ZX-Calculus，作为Martin-Lof依赖类型理论的保守扩展，集成了迹索引类型、预层非单调语义和构造性AGM信念修正。论文包含Coq形式化验证，证明了核心结果。

Details

Motivation: 旨在解决依赖类型理论与动态、事件驱动的计算模型（如执行迹）的集成问题，同时为信念修正提供构造性语义基础，以弥合证明论单调性与语义非单调性之间的差距。

Result: 论文证明了迹-可达性对应、确定性重放和典范性框架等核心定理，并在Coq中验证了除RC-elim外的所有结果；AGM信念修正的八个公理均被证明为定理，并通过构造性部分交收缩算法验证。

Insight: 创新点包括引入迹索引类型以显式处理事件标签，提供更符合人机工程学的归纳接口；利用预层语义区分证明论与语义单调性；揭示路径依赖信念修订与函子一致性之间的根本张力，这是先前未识别的问题。

Abstract: We propose ZX-Calculus (Knowledge Evolution Calculus), a conservative extension of Martin-Lof Dependent Type Theory (MLTT) integrating trace-indexed types, presheaf non-monotone semantics, and constructive AGM belief revision. A Coq mechanisation accompanies the paper (34 complete proofs; zero admits for the two central results). (I) Trace types. FinTrace(s0,sn) is an inductive family of typed execution traces. FinTrace and Star(Step) are isomorphic as path types but not judgementally equal; TraceElim exposes the event label e:Event explicitly, giving a more ergonomic interface for event-driven induction. We prove the Trace-Reachability Correspondence, Deterministic Replay, and a canonicity framework via reducibility candidates with a Transport Lemma (RC-elim deferred; all other Core results are Coq-verified). (II) Sheaf semantics. Trace-indexed propositions are contravariant sheaves over the free trace partial-order category Tf. A Separation Theorem (explicit countermodel) distinguishes proof-theoretic monotonicity from semantic non-monotonicity. The term model is an initial CwF (syntactic universal property, not classical completeness). (III) AGM belief revision. We give an explicit constructive partial meet contraction algorithm verified against (C1)-(C4). All eight AGM postulates (R1)-(R8) are theorems. Proofs of R7 and R8 use the Disjunctive Entrenchment Lemma, given a self-contained constructive derivation. (IV) Integration. B^AGM fails the sheaf composition law BP-comp for sequential revision (explicit countermodel, Coq-verified). We introduce Single-Step Revision Systems (SSRS), prove B^AGM is a valid SSRS (Coq-verified), and show this suffices for trace morphisms, retraction characterisation, and revision witnesses. The BP-comp failure reveals a fundamental tension between path-dependent belief revision and functor consistency, not previously identified.

eess.IV [Back]

[115] SEAOTTER: Sensor Embedded Autoencoding with One-Time Transcode for Efficient Reconstruction eess.IV | cs.CV | cs.LG | cs.ROPDF

Dan Jacobellis, Neeraja J. Yadwadkar

TL;DR: 本文提出了一种名为SEAOTTER的压缩框架，专为云机器人系统设计，旨在解决高分辨率视觉数据传输中的带宽和计算资源限制问题。该方法结合了学习型隐表示的紧凑性和标准JPEG格式的广泛兼容性，通过一次性转码实现高效重建，在保持与JPEG基础设施兼容的同时，显著提升了编码/解码速度和感知任务性能。

Details

Motivation: 动机是解决机器人系统中，低成本硬件捕获的高分辨率视觉数据因带宽和计算资源有限而无法充分利用的问题；现有标准编解码器（如JPEG/MPEG）效率不足，而新型编解码器（如AV1/AVIF）或非对称自编码器又存在编码资源需求高、解码成本大或与标准基础设施不兼容等局限性。

Result: 在200:1的压缩比下，与AVIF相比，SEAOTTER实现了编码速度提升7倍、解码速度提升3.5倍，并在ImageNet top-1准确率上提高了+8%，同时保持了与JPEG基础设施的兼容性。

Insight: 创新点在于提出了一种可学习的JPEG颜色和量化变换，以提升全局、密集和基于视觉-语言的感知任务的准确性；同时，通过将学习型隐表示与标准JPEG文件结合，实现了资源受限场景下的高效压缩与广泛适用性，为云机器人系统提供了兼顾性能与兼容性的解决方案。

Abstract: In robotics systems, vast amounts of visual data are easily captured at high resolution using low-cost, low-power hardware. Yet, limited bandwidth and on-device compute resources prevent full utilization when transmitted via conventional codecs like JPEG/MPEG. Newer codecs, like AV1/AVIF, improve the rate-distortion trade-off, but demand far more resources for encoding, impractical without custom ASICs. Recent asymmetric autoencoders deliver high quality under extreme power and bandwidth constraints, but add prohibitive decoding cost and use bespoke formats that ignore decades of infrastructure built around standards like JPEG. To address these limitations, we introduce a compression framework for cloud robotics based on a Sensor Embedded Autoencoder paired with a One-Time Transcode for Efficient Reconstruction (SEAOTTER). Because the sensor, cloud, and consumer stages face very different power and bandwidth budgets, SEAOTTER combines the compactness of a learned latent with the broad usability of a standard JPEG file. Since naive transcoding degrades performance, we propose a learnable JPEG color and quantization transform that enables increased accuracy for global, dense, and vision-language-based perception. Using SEAOTTER, we train both general-purpose and task-aware transcoding pipelines for a pre-trained, frozen encoder. At a compression ratio of 200:1 and compared to AVIF, we observe 7 times faster encoding, 3.5 times faster decoding, and +8% ImageNet top-1 accuracy, while retaining compatibility with JPEG infrastructure. Our code is available at https://github.com/UT-SysML/seaotter .

cs.IR [Back]

[116] Taiji: Pareto Optimal Policy Optimization with Semantics-IDs Trade-off for Industrial LLM-Enhanced Recommendation cs.IR | cs.AI | cs.CLPDF

Yuecheng Li, Zeyu Song, Jing Yao, Chi Lu, Peng Jiang

TL;DR: 本文提出了Taiji框架，一种用于工业推荐系统的LLM增强方法，通过反向工程推理和开放域拒绝采样生成高质量思维链数据，并利用帕累托最优策略优化自适应调整跨域奖励权重，以平衡LLM语义知识与推荐ID特征。

Details

Motivation: 解决现有LLM推荐范式在SFT阶段难以衡量和改进开放域推荐中思维链质量，以及在RL对齐阶段忽视LLM语义奖励与推荐偏好奖励之间权衡的问题。

Result: 在快手广告平台自2026年5月部署，每日服务超4亿用户，离线评估和在线A/B测试验证了其有效性，并带来显著商业收入，展示了在Web规模环境中的强大可扩展性。

Insight: 创新点包括利用反向工程推理和拒绝采样提升思维链数据质量，以及提出帕累托最优策略优化实现LLM语义空间与推荐ID空间的自适应权衡，为工业级LLM增强推荐提供了可扩展解决方案。

Abstract: Scaling recommender systems via large language models (LLMs) has become a prominent trend in the industry. However, aligning the LLM’s semantic space with the recommender’s ID space via post-training (e.g., SFT and RL) remains challenging. Existing LLM4Rec paradigms are bottlenecked by two main issues: (1) the difficulty of measuring and improving chain-of-thought (CoT) quality in open-domain recommendation during SFT, and (2) the neglect of the trade-off between LLM semantic rewards and recommendation preference rewards during RL alignment. Inspired by these challenges, we present Taiji, a novel LLM-as-Enhancer framework designed for industrial recommender systems. To overcome the SFT bottleneck, we utilize reverse-engineered reasoning and open-ended rejection sampling to generate high-quality, domain-specific CoT data. To resolve the RL alignment issue, we propose Pareto Optimal Policy Optimization (POPO), which adaptively adjusts cross-domain reward weights. Theoretically, it achieves an optimal trade-off between the semantic world knowledge of LLMs and the collaborative ID features representing online user preferences. Extensive offline evaluations and online A/B tests validate the effectiveness of Taiji. Deployed on Kuaishou’s advertising platform since May 2026, Taiji currently serves over 400 million users daily, yielding significant commercial revenue and demonstrating its robust scalability in web-scale environments.

eess.AS [Back]

[117] Wavelet as Tokenizer: Preliminary Results on a Shared Wavelet Token Schema for Natural Signals eess.AS | cs.AI | cs.CV | cs.LG | cs.SDPDF

Shenghao Ding

TL;DR: 本文探讨了音频、图像和视频信号是否可以采用统一的小波标记模式，而非依赖各自模态特定的潜在网格。作者提出了一种基于单层Haar离散小波变换/逆变换前端的连续标记模型，包含共享的系数标记布局、可选的结构元数据、轻量级模态值适配器以及共享的标记级编码器-解码器主干。在Speech Commands、EuroSAT RGB和DAVIS 2017数据集上，密集共享模型分别达到了39.92 dB、29.37 dB和23.93 dB的峰值信噪比。实验表明，视觉性能提升不能仅归因于潜在容量，且元数据嵌入并非普遍有效。固定率能量选择提供了强大的非参数基线，而掩码稀疏训练在50%密集标记下实现了34.45 dB的视频PSNR。

Details

Motivation: 研究动机是探索跨模态（音频、图像、视频）的自然信号是否能够共享统一的小波标记模式，以替代传统上各自独立的模态特定潜在表示方法，旨在实现更通用的信号处理框架。

Result: 在Speech Commands（音频）、EuroSAT RGB（图像）和DAVIS 2017（视频）基准测试中，密集共享模型分别达到39.92 dB、29.37 dB和23.93 dB的PSNR；稀疏训练在50%标记密度下实现34.45 dB视频PSNR，能量选择策略相比均匀选择在压缩保持比下平均提升PSNR约16-17 dB。

Insight: 创新点在于提出了一种基于小波变换的跨模态统一标记模式，通过共享系数布局和轻量适配器处理多模态信号；客观分析显示，该方法为稀疏标记接口和跨模态表示学习提供了新思路，但尚未建立通用的离散词汇表。

Abstract: This paper studies whether audio, images, and video can share a common wavelet token schema rather than relying on separate modality-specific latent grids. It introduces a preliminary continuous-token model built around a one-level Haar DWT/IDWT frontend, a shared coefficient-token layout, optional structural metadata, lightweight modality value adapters, and a shared token-wise encoder-decoder trunk. On Speech Commands, EuroSAT RGB, and DAVIS 2017 data, a dense shared model reaches 39.92 dB audio, 29.37 dB image, and 23.93 dB video PSNR. A matched-rate sweep under continuous latent scalar budgets indicates that the visual gains are not explained solely by latent capacity, while also showing that additive metadata embeddings are not a universal source of improvement. Finally, fixed-rate energy selection provides a strong non-parametric baseline: energy_global improves average PSNR over uniform selection by 16.73 dB for audio, 16.90 dB for images, and 15.86 dB for video under compressed keep ratios. Masked sparse training reaches 34.45 dB video PSNR with 50% of dense tokens. The results support a unified wavelet token schema and sparse token interface, while stopping short of establishing a universal discrete vocabulary.

[118] SVHalluc: Benchmarking Speech-Vision Hallucination in Audio-Visual Large Language Models eess.AS | cs.AI | cs.CV | cs.LG | cs.MM | cs.SDPDF

Chenshuang Zhang, Kyeong Seon Kim, Chengxin Liu, Tae-Hyun Oh

TL;DR: 本文提出了SVHalluc，首个用于评估视听大语言模型中语音-视觉幻觉的综合基准。该基准从语义和时间两个关键互补维度诊断幻觉问题。实验表明，当前顶尖的开源视听大模型在将语音内容与对应视觉信号对齐方面存在困难，而Gemini 2.5 Pro则表现显著更优。

Details

Motivation: 现有基准主要关注环境声音（如狗叫）以指示事件发生，但人类语音承载着根本不同、丰富的语义和时间结构。目前尚不清楚现有模型是否能准确对齐语音内容与视觉信号，因此需要系统研究语音内容是否会导致视听大模型产生幻觉。

Result: 实验结果显示，最先进的开源视听大语言模型在多个任务上的准确率接近随机水平，表明其在语音-视觉对齐方面存在严重困难。相比之下，Gemini 2.5 Pro的表现显著优于开源模型。

Insight: 论文的核心创新点在于首次构建了针对语音-视觉幻觉的综合性诊断基准，并从语义和时间两个维度进行系统评估。分析表明，模型失败的原因在于跨模态理解能力有限，尽管单模态感知能力很强，这揭示了当前视听大模型一个新的根本性局限，并强调了基于语音的视频理解的重要性。

Abstract: Despite the success of audio-visual large-language models (LLMs), they can produce plausible but ungrounded outputs, termed hallucination. Existing benchmarks focus on environmental sounds (e.g., dog barking) to indicate event occurrence. In contrast, human speech carries fundamentally different, rich semantics and temporal structures, yet it remains unexplored whether current models can accurately align speech content with corresponding visual signals. In this work, we show that speech content can induce hallucinations in audio-visual LLMs. To systematically study this, we introduce SVHalluc, the first comprehensive benchmark for evaluating speech-vision hallucination in audio-visual LLMs. Our benchmark diagnoses speech-vision hallucinations from two critical and complementary aspects: semantic and temporal. Experimental results demonstrate that state-of-the-art open-source audio-visual LLMs struggle with aligning speech content with corresponding visual signals, with a near-random accuracy on multiple tasks. In contrast, Gemini 2.5 Pro significantly outperforms the open-source models. Our analysis suggests that their failures stem from limited ability in cross-modality understanding, despite strong performance in single-modality perception. Our work uncovers a new and fundamental limitation of current audio-visual LLMs and highlights the need for speech-grounded video comprehension. Project page: https://chenshuang-zhang.github.io/projects/svhalluc/.

q-bio.NC [Back]

[119] BEAST3D: Animal behavioral analysis and neural encoding from multi-view video via Gaussian splatting q-bio.NC | cs.CVPDF

Yanchen Wang, Lenny Aharon, Wangshu Zhu, Kyle Daruwalla, Linghua Zhang

TL;DR: BEAST3D是一个自监督预训练框架，用于从无标签、已校准的多视角视频中学习3D视觉表征。它使用视觉Transformer预测3D高斯泼溅来重建被遮挡的视角，并同时分割动物与背景。该框架仅需四个视图即可重建3D结构，并生成可用于多种下游任务（如新视角合成、姿态估计和神经编码）的视角不变特征。

Details

Motivation: 解决从实验室环境下的多视角视频中提取丰富3D表征的挑战，因为监督姿态估计需要大量人工标注，而通用3D重建模型在实验室特有的稀疏视图和专用图像上表现不佳。

Result: 在四个物种上的综合评估表明，BEAST3D能生成丰富的视角不变特征，有效迁移到新视角合成、多视角姿态估计和神经编码三个下游任务，验证了其学习到的3D表征质量。

Insight: 创新点在于将已知相机参数直接作为条件输入，使模型能在稀疏视图（如四个）的实验室设置下工作，而无需像通用模型那样从密集重叠视图中估计相机几何；同时通过自监督方式联合学习3D重建和分割，为行为分析提供了一个多功能框架。

Abstract: Multi-view video recordings are increasingly used to capture the 3D movements of animals in experimental settings, yet extracting rich 3D representations from these recordings remains challenging. Supervised pose estimation requires extensive manual annotation, while general-purpose 3D reconstruction models trained on generic scene datasets fail on the specialized imagery and sparse-view setting of laboratory experiments. We address these limitations with BEAST3D, a self-supervised pretraining framework that learns 3D visual representations from unlabeled, calibrated multi-view video. BEAST3D uses a vision transformer to predict 3D Gaussian splats that reconstruct held-out views through differentiable rendering, while simultaneously segmenting the animal from the background. BEAST3D reconstructs 3D structure with as few as four views by conditioning directly on known camera parameters–unlike general-purpose models, which must estimate camera geometry from dense overlapping viewpoints that are seldom available in lab settings. Through comprehensive evaluation across four species, we demonstrate that BEAST3D produces rich, viewpoint-invariant features that transfer effectively to three downstream tasks: novel view synthesis, which validates the quality of the learned 3D representations; multi-view pose estimation, which provides the sparse keypoint trajectories widely used in behavioral analysis; and neural encoding, which relates 3D behavioral features to simultaneously recorded neural activity. BEAST3D thus establishes a versatile framework for behavioral analysis that leverages 3D structure in modern multi-view laboratory recordings.

cs.DB [Back]

[120] CAPER: Clause-Aligned Process Supervision for Text-to-SQL cs.DB | cs.CLPDF

Lujie Ban, Jiasheng Shi, Jinyang Li, Xiaolin Han, Tsz Nam Chan

TL;DR: 本文提出了CAPER，一种用于Text-to-SQL任务的子句对齐过程监督方法。该方法通过对SQL抽象语法树进行反事实干预，自动生成子句级别的监督信号，用于训练一个轻量级的Clause-PRM模型（CAPER-9B）。该模型能为策略优化和候选验证提供子句边界反馈，在BIRD和Spider基准测试中显著提升了执行准确性和故障定位能力。

Details

Motivation: 现有Text-to-SQL系统通常基于查询级别的执行正确性进行评估，这种终端信号无法指导中间SQL决策的成败原因。而令牌级别的密集监督也不适用，因为SQL令牌与完整的语义决策不对齐，可能惩罚执行等效的查询，且难以大规模可靠标注。

Result: 在BIRD和Spider基准上的实验表明，子句对齐的监督不仅提高了执行准确率（相对于GPT-5.4实现了高达15.3%的相对EX提升），还增强了故障定位能力（在保留的故障集上达到84.53%的准确率和90.60%的MRR）。

Insight: 核心创新在于提出了通过反事实干预SQL抽象语法树来自动生成子句级别监督信号的方法，实现了对错误的根本原因定位。这避免了令牌级监督的缺陷，并训练出能提供细粒度反馈的轻量级奖励模型，有效结合了过程监督与策略优化。

Abstract: Text-to-SQL systems are typically evaluated by query-level execution correctness, but this terminal signal provides little guidance about which intermediate SQL decision caused success or failure. Token-level dense supervision is also ill-suited: SQL tokens do not align with complete semantic decisions, can penalize execution-equivalent queries, and are difficult to label reliably at scale. We therefore propose CAPER, which automatically derives clause-level supervision via counterfactual intervention on the SQL abstract syntax tree, enabling root-cause error localization for reward modeling; the resulting data is used to train CAPER-9B, a lightweight Clause-PRM that provides clause-boundary feedback for policy optimization and candidate verification. Experiments on BIRD and Spider show that clause-aligned supervision not only improves execution accuracy, achieving up to a 15.3% relative EX improvement over GPT-5.4, but also strengthens failure-localization capability, reaching 84.53% accuracy and 90.60% MRR on held-out failures. Our project page is at https://github.com/banrichard/RL-NL2SQL.

cs.LG [Back]

[121] Filter, Then Reweight: Rethinking Optimization Granularity in On-Policy Distillation cs.LG | cs.AI | cs.CLPDF

Yuying Li, Leqi Zheng, Yongzi Yu, Wenrui Zhou, Xuchang Zhong

TL;DR: 本文提出了一种名为FiRe-OPD（Filter, then Reweight）的新方法，用于重新思考大语言模型策略内蒸馏（OPD）的优化粒度。该方法通过联合调整轨迹和令牌级别的监督信号，先过滤低质量的轨迹样本，再对保留轨迹中的信息丰富的令牌进行软重加权，以实现更细粒度的优化。

Details

Motivation: 动机源于当前策略内蒸馏（OPD）方法正从全轨迹KL监督转向更选择性的训练范式，关注于选择学习哪些轨迹、哪些令牌信息最丰富以及哪些监督信号最可靠。本文旨在重新思考OPD的优化粒度。

Result: 在强到弱、单教师和多教师设置下验证了FiRe-OPD的有效性，结果表明其优于最近的令牌级OPD方法（例如，在强到弱设置下AIME 2024提升+6.25，在多教师设置下Miner提升+18.81）。

Insight: 创新点在于联合轨迹过滤和令牌软重加权，以细粒度调整监督信号。与硬令牌选择相比，软加权机制能有效减少信息损失并增强优化稳定性，为OPD提供了更精细的优化框架。

Abstract: On-Policy distillation (OPD) in large language models is shifting from full-trace KL supervision toward more selective training paradigms. Recent OPD methods increasingly focus on selecting which trajectories to learn from, which tokens are most informative, and which supervision signals are most reliable. Motivated by this trend, we rethink optimization granularity of OPD and propose \fireicon\ FiRe-OPD (Filter, then Reweight), which jointly adjusts supervision signals at both trajectory and token levels. In details, FiRe-OPD first filters trajectories to remove low-quality rollout samples, and then applies soft reweighting within the retained trajectories to emphasize informative tokens. Compared with hard token selection, FiRe-OPD leverages a soft-weighting mechanism to effectively mitigate information loss and enhance optimization stability, thereby achieving finer-grained OPD optimization. We validate the effectiveness of FiRe-OPD across strong-to-weak, single-teacher, and multi-teacher settings, and demonstrate its superiority over recent token-level OPD methods ( (e.g., +6.25 on AIME 2024 in strong-to-weak, +18.81 on Miner in multi-teacher). Our code is available at https://github.com/YuYingLi0/FiRe-OPD.

[122] When Model Merging Breaks Routing: Training-Free Calibration for MoE cs.LG | cs.AI | cs.CLPDF

Canbin Huang, Tianyuan Shi, Xiaojun Quan, Jingang Wang, Jianfei Zhang

TL;DR: 本文针对混合专家模型合并时出现的路由崩溃问题，提出了一种无需训练的校准框架HARC。该方法利用二阶曲率信息重新对齐合并后的路由器，有效解决了因参数扰动导致的专家分配失效问题。实验表明，HARC在数学推理和代码生成任务上显著提升了合并后MoE模型的性能。

Details

Motivation: 现有模型合并技术主要基于线性参数算术或优化，在应用于混合专家架构时，由于非线性softmax和离散Top-k路由机制对参数扰动敏感，导致合并后的路由器无法将token正确分配给合适的专家，造成性能严重下降。

Result: 在数学推理和代码生成任务上的实验表明，HARC有效缓解了多种MoE合并基线中的路由崩溃问题，并带来了显著的性能提升。

Insight: 创新点在于首次识别了MoE合并中的路由崩溃问题，并提出了基于Hessian感知的无训练路由器校准框架。该方法利用二阶曲率信息进行路由器对齐，提供了闭式解，可通过无矩阵共轭梯度法高效求解，为MoE模型合并提供了新思路。

Abstract: Model merging has emerged as a cost-effective approach for consolidating the capabilities of multiple LLMs without retraining. However, existing merging techniques, largely based on linear parameter arithmetic or optimization, struggle when applied to Mixture-of-Experts (MoE) architectures. We identify a critical failure mode in MoE merging, termed routing breakdown, in which the merged router fails to dispatch tokens to suitable experts. Routing breakdown stems from the sensitivity of the non-linear softmax and discrete Top-k routing mechanisms to parameter perturbations from merging, a sensitivity further amplified by load-balancing constraints imposed during MoE pretraining. Because fine-tuned experts exhibit distinct specializations, even modest misrouting can cause severe performance degradation. To address this issue, we propose Hessian-Aware Router Calibration (HARC), a training-free framework that leverages second-order curvature information to realign the merged router. This approach admits a closed-form solution that can be efficiently solved using a matrix-free conjugate gradient method. Experiments on mathematical reasoning and code generation tasks show that HARC effectively mitigates routing breakdown across diverse MoE merging baselines and leads to substantial performance improvements. Our code is available at https://github.com/huangcb01/HARC.

[123] Value-Aware Stochastic KV Cache Eviction for Reasoning Models cs.LG | cs.CLPDF

Ting-Yun Chang, Harvey Yiyun Fu, Deqing Fu, Chenghao Yang, Jesse Thomason

TL;DR: 本文提出了一种名为VaSE（Value-aware Stochastic KV Cache Eviction）的训练无关方法，用于压缩推理模型中的KV缓存。该方法通过保护大模值的value状态并引入随机性来提升缓存多样性，从而在显著压缩KV缓存的同时，保持了甚至超越了基于选择的稀疏注意力方法的准确率。

Details

Motivation: 推理模型通过长链思维提高准确性，但其长输出会导致内存和计算瓶颈。现有的KV缓存驱逐方法虽然能降低成本，但准确率通常低于保留完整KV缓存的基于选择的稀疏注意力方法。本文旨在识别影响驱逐方法准确性的关键因素，并弥合效率与准确性之间的差距。

Result: 在六个推理任务上，使用VaSE进行4倍KV缓存压缩的Qwen3模型，在相同稀疏度下，其平均准确率超过了最先进的（SOTA）选择方法，同时比最强的驱逐方法高出超过4%。该方法支持FlashAttention2，并为推理模型实现了静态内存占用。

Insight: 创新点在于识别出大模值value状态对模型性能至关重要，驱逐它们会导致灾难性故障（如重复推理循环），并发现引入随机性以增加缓存多样性可以提升准确性。基于此，VaSE通过保护关键value状态和随机化驱逐决策，提供了一种高效且准确的缓存压缩方案。

Abstract: Reasoning models improve accuracy through extended chains of thought, but their long outputs create a memory and compute bottleneck. KV cache eviction methods reduce this cost by evicting unimportant key-value pairs from the cache, yet they often yield worse accuracy than selection-based sparse attention alternatives, which keep the full KV cache. We identify key factors crucial to KV cache eviction accuracy. First, a small fraction of value states have abnormally large magnitudes, and evicting them causes catastrophic failure where models enter repetitive reasoning loops. Second, introducing stochasticity during eviction improves accuracy by increasing cache diversity. Based on these findings, we propose Value-aware Stochastic KV Cache Eviction (VaSE), a training-free recipe that protects large-magnitude value states and promotes diverse eviction decisions. Across six reasoning tasks, Qwen3 models using VaSE with 4x KV cache compression yield higher average accuracies than SOTA selection method at the same sparsity, while outperforming the strongest eviction method by more than 4%. Overall, VaSE bridges the gap between efficiency and accuracy, supporting FlashAttention2 and enabling a static memory footprint for reasoning models.

[124] Skill-RM: Unifying Heterogeneous Evaluation Criteria via Agent Skill cs.LG | cs.CLPDF

Tao Chen, Gangwei Jiang, Pengyu Cheng, Siyuan Huang, Yihao Liu

TL;DR: 本文提出了Skill-RM，一个将奖励建模重新定义为可重复使用的奖励评估技能执行的统一框架。它通过将奖励计算视为结构化智能体任务，提供了一个一致的接口来编排异构评估资源，从而超越静态评估，确保跨任务的一致性和透明度。

Details

Motivation: 当前奖励模型的评估依赖于规则验证器、真实参考、程序检查表和复杂评分标准等异构标准，缺乏一个统一的机制来整合所有类型的证据。

Result: 在奖励基准测试和下游应用（包括最佳N选择（best-of-N selection）和强化学习）上的大量实验表明，Skill-RM始终优于传统的评判基线。

Insight: 核心创新点在于将奖励建模框架化为一个可执行的、结构化的智能体技能，实现了对异构证据的动态选择和聚合，为奖励建模提供了一个统一的解决方案，并通过这种策略性的动态编排实现了更优的性能。

Abstract: Reward models (RMs) provide critical feedback signals for LLM post-training, notably in reinforced fine-tuning (RFT) and reinforcement learning (RL) pipelines. However, current reward evaluation relies on heterogeneous criteria such as rule-based verifiers, ground-truth references, procedural checklists, and complex rubrics, where a unified mechanism to integrate all types of evidence remains unexplored. To this end, we propose Skill Reward Model (Skill-RM), a unified framework that reformulates reward modeling as the execution of a reusable Reward-Evaluation Skill. By treating reward computation as a structured agentic task, Skill-RM provides a consistent interface to orchestrate heterogeneous resources, dynamically selecting and aggregating evidence tailored to the specific requirements of each input. This approach enables the reward model to move beyond static evaluation, ensuring consistency and transparency across diverse tasks. Extensive experiments on reward benchmarks and downstream applications, including best-of-N selection and reinforcement learning, demonstrate that Skill-RM consistently outperforms traditional judge baselines. Our findings suggest that Skill-RM not only provides a unified solution for reward modeling but also achieves superior performance through the strategic and dynamic orchestration of evidence. The code is at https://github.com/Qwen-Applications/Skill-RM.

[125] Neuron Populations Exhibit Divergent Selectivity with Scale cs.LG | cs.CL | cs.CVPDF

Amil Dravid, Yasaman Bahri, Alexei A. Efros, Yossi Gandelsman

TL;DR: 本文研究了神经网络中神经元群体（特别是Rosetta Neurons）如何随模型规模扩展而演变。研究发现，Rosetta Neurons的数量随模型规模呈亚线性幂律增长，但其占总神经元比例下降，同时这些神经元的选择性和单义性增强，与非Rosetta神经元群体出现极化现象。

Details

Motivation: 动机是探究神经网络内部神经元群体（超越宏观损失指标）是否随模型规模扩展呈现可预测的演变规律，特别是针对跨独立训练模型激活模式相似的Rosetta Neurons类神经元。

Result: 在参数规模达300亿的语言模型和50亿的视觉模型上，Rosetta Neurons数量遵循亚线性幂律增长，并表现出神经元极化效应：其选择性和单义性随规模增强，与非Rosetta群体分离。一个平衡特征效用与神经元容量的解析模型解释了该幂律和极化现象。

Insight: 创新点在于揭示了神经元层面可解释的共享结构也存在缩放定律，将模型规模与神经元的普遍性、选择性和专业化程度的变化系统关联起来，并通过数据过滤案例展示了Rosetta Neurons随规模增长更趋领域专业化。

Abstract: We investigate whether neuron populations within neural networks evolve predictably with scale, extending scaling laws beyond macroscopic observables such as loss. To probe this question, we study Rosetta Neurons, a previously characterized class of neurons whose activation patterns are similar across independently trained models (Dravid et al., 2023). In separate analyses of language models up to 30B parameters and vision models up to 5B parameters, we observe that the population of Rosetta Neurons follows a sublinear power law in model size, growing in absolute number but occupying a shrinking fraction of the total neuron count. We further observe a Neuron Polarization Effect: Rosetta Neurons become more selective and increasingly monosemantic with scale, separating from a growing non-Rosetta population that remains less selective. An analytical model balancing feature utility against limited neuron capacity explains the sublinear power-law scaling and this polarization effect. Finally, we find that Rosetta Neurons become more domain-specialized with scale and illustrate their selectivity through a targeted data-filtering case study for continued pretraining. Our results point to a scaling law for interpretable, shared neuron-level structure, linking model size to systematic changes in neuron universality, selectivity, and specialization.

[126] BYORn: Bootstrap Your Own Responses to Defend Large Vision-Language Models Against Backdoor Attacks cs.LG | cs.CVPDF

Ivan Sabolić, Marin Oršić, Josip Šarić, Sven Lončarić

TL;DR: 本文提出BYORn框架，通过引导模型生成自身响应来防御视觉语言模型的后门攻击。该方法基于中毒目标响应与图像-文本输入语义不匹配的观察，动态替换异常响应以破坏触发器与目标输出的关联。实验表明BYORn在保持干净任务性能的同时显著提升后门鲁棒性，并有效抵抗针对性自适应攻击。

Details

Motivation: 针对监督微调范式在开放式生成场景中易受后门攻击且现有防御无效的问题，基于中毒响应与预训练模型语义不一致的观察，提出动态响应替换的防御思路。

Result: 在多个基准测试中，BYORn在保持模型原始泛化能力的同时，显著降低了后门攻击成功率，建立了泛化性能与防御效果的新权衡边界，且对针对性自适应攻击保持有效性。

Insight: 创新点在于利用预训练模型自身生成响应替代可疑中毒样本，通过破坏触发器-输出关联实现防御；其理论贡献在于构建了干净数据分布上经验风险上界的梯度优化目标，为后门防御提供了可解释的理论框架。

Abstract: Supervised fine-tuning is the predominant approach for adapting autoregressive vision-language models to downstream tasks. Recent work has shown that this paradigm is highly vulnerable to backdoor attacks, and that existing defenses are ineffective in open-ended generation settings. In response, we propose BYORn, a backdoor-robust fine-tuning framework motivated by the observation that poisoned target responses are often semantically implausible given the corresponding image-text inputs and a pretrained model. BYORn identifies such misaligned responses and dynamically replaces them with alternative responses generated by the model, thereby breaking the correlation between triggers and target outputs. The resulting objective gradient corresponds to the gradient of the empirical estimate of the population risk upper bound over the clean data distribution. Empirically, BYORn consistently improves robustness to backdoor attacks while preserving clean-task performance, establishing a new trade-off frontier between generalization and attack success rate. Finally, we demonstrate that BYORn remains effective against adaptive attacks specifically designed to circumvent the proposed defense.

[127] Learning to See via Epiretinal Implant Stimulation in silico with Model-Based Deep Reinforcement Learning cs.LG | cs.CV | q-bio.NCPDF

Jacob Lavoie, Marwan Besrour, William Lemaire, Jean Rouat, Réjean Fontaine

TL;DR: 该论文提出了一种基于深度强化学习的方法，用于优化视网膜上植入物（epiretinal implants）的电刺激模式，以在虚拟患者视网膜上生成更易理解的图像。该方法在名为rlretina的强化学习环境中，将刺激任务形式化为基于笔触的渲染任务，利用各向同性和各向异性的光幻视形状来组合图像。

Details

Motivation: 视网膜上植入物电刺激会产生沿视网膜神经节细胞轴突束伸长的各向异性形状，导致图像失真。现有方法通过避免刺激轴突束来获得各向同性的像素状形状，但如何有效利用这些形状组合成可理解的图像仍是一个挑战。

Result: 研究训练了一个深度强化学习智能体，该智能体能够学习组合各向同性和各向异性形状以形成图像。在基于心理物理学验证的轴突图模型渲染的不同虚拟患者图像中，该方法相比朴素方法能生成更易理解的图像。

Insight: 创新点在于将视网膜植入物刺激问题形式化为强化学习环境中的基于笔触的渲染任务，并利用模型驱动的数据生成方式训练智能体。这为利用各向异性光幻视改善人工恢复视力的视觉敏锐度提供了新思路。

Abstract: Objective: Diseases such as age-related macular degeneration and retinitis pigmentosa cause the degradation of the photoreceptor layer. One approach to restore vision is to electrically stimulate the surviving retinal ganglion cells with a microelectrode array such as epiretinal implants. Epiretinal implants are known to generate visible anisotropic shapes elongated along the axon fascicles of neighboring retinal ganglion cells. Recent work has demonstrated that to obtain isotropic pixel-like shapes, it is possible to map axon fascicles and avoid stimulating them by inactivating electrodes or lowering stimulation current levels. Avoiding axon fascicle stimulation aims to remove brushstroke-like shapes in favor of a more reduced set of pixel-like shapes. Approach: In this study, we propose the use of isotropic and anisotropic shapes to render intelligible images on the retina of a virtual patient in a reinforcement learning environment named rlretina. The environment formalizes the task as using brushstrokes in a stroke-based rendering task. Main Results: We train a deep reinforcement learning agent that learns to assemble isotropic and anisotropic shapes to form an image. We investigate which error-based or perception-based metrics is adequate to reward the agent. The agent is trained in a model-based data generation fashion using the psychophysically validated axon map model to render images as perceived by different virtual patients. We show that the agent can generate more intelligible images compared to the naive method in different virtual patients. Significance: This work shares a new way to address epiretinal stimulation that constitutes a first step towards improving visual acuity in artificially-restored vision using anisotropic phosphenes.

cs.AI [Back]

[128] Traj-Evolve: A Self-Evolving Multi-Agent System for Patient Trajectory Modeling in Lung Cancer Early Detection cs.AI | cs.CLPDF

Sihang Zeng, Matthew Thompson, Ruth Etzioni, Meliha Yetisgen

TL;DR: 本文提出了Traj-Evolve，一个用于肺癌早期检测中患者轨迹建模的自进化多智能体系统。该系统通过一个非参数化的经验池（ExPool）存储和检索相似患者案例作为上下文，并结合多智能体强化学习（MARL）来优化智能体间及智能体与记忆的协作。在利用长达五年多模态电子健康记录（EHR）的肺癌预测任务中，该方法在总体人群和具有挑战性的非吸烟者人群中均优于9个强基线模型。

Details

Motivation: 现有基于LLM的多智能体系统虽然能处理长上下文，但孤立地处理患者，未能模拟临床医生利用过往相似病例积累经验进行推理的过程。本文旨在解决从稀疏、噪声大、长上下文的多模态EHR序列中对患者轨迹进行建模的挑战。

Result: 在肺癌预测任务上，Traj-Evolve在整体人群和从未吸烟者人群中都超越了9个强基线模型，达到了最先进的性能。分析表明，ExPool提高了特异性，而MARL提高了敏感性。

Insight: 创新点在于提出了一个结合非参数化记忆（ExPool）与参数化优化（MARL）的自进化框架，通过留一交叉检索策略统一训练与推理行为。系统性地展示了经验池扩展如何改变检索偏好，以及管理者和工作者智能体在训练中的不同收敛模式，揭示了两种进化机制在提升模型敏感性和特异性上的互补性。

Abstract: Modeling patient trajectories from longitudinal electronic health records (EHRs) requires reasoning over sparse, noisy, and long-context multimodal sequences. Existing LLM-based multi-agent systems address context length but process patients in isolation, failing to mirror how clinicians leverage accumulated experience from similar prior cases. We present Traj-Evolve, a self-evolving multi-agent system with two complementary evolving mechanisms. First, an Experience Pool (ExPool) acts as a non-parametric memory, indexing rejection-sampled reasoning traces to retrieve similar patients as few-shot contexts. Second, multi-agent reinforcement learning (MARL) via reward-ranked fine-tuning parametrically optimizes inter-agent and agent-memory collaboration. A leave-one-out cross-retrieval strategy unifies the two, aligning training- and inference-time behavior under retrieval augmentation. On a lung cancer prediction task utilizing up to five years of multimodal EHRs, Traj-Evolve outperforms 9 strong baselines on the overall population and a challenging never-smoker population. Analysis of the evolving dynamics highlights three key findings: (1) expanding the ExPool shifts optimal retrieval from diverse to specific samples; (2) under MARL, the manager agent’s prediction loss converges quickly while the worker agents’ temporal reasoning continues to benefit from more verified patients; and (3) the two mechanisms are complementary on the predicted risk, where ExPool improves specificity while MARL improves sensitivity.

[129] Large AI Models in Dental Healthcare: From General-Purpose Systems to Domain-Specific Foundation Models cs.AI | cs.CLPDF

Sema Helali, Lina Abu Nadab, Sausan Alqawas, Alaa Abd-Alrazaq, Faleh Tamimi

TL;DR: 这篇论文系统综述了大型AI模型在牙科医疗领域的应用，比较了通用模型与牙科专用模型的表现。研究发现，语言生成模型擅长文本任务但图像诊断表现不稳定，而牙科专用模型在复杂多模态任务上表现最佳，集成管道优于单一模型。

Details

Motivation: 全球口腔疾病影响数十亿人，但大型AI模型在牙科领域的临床潜力尚不明确，且缺乏对三类模型（语言生成模型、判别性视觉基础模型、牙科专用基础模型）关系的统一综述，因此需要系统评估它们的优劣和局限性。

Result: 在系统综述97项研究后，牙科专用模型（如DentVFM、DentVLM、OralGPT）在复杂多模态任务上表现最强，集成管道一致优于单一模型；通用模型在文本任务（如临床推理）表现优异，但在图像诊断中性能不一致。

Insight: 论文提出一个二维分类框架，按架构范式和牙科专业化程度组织模型，揭示了数据不对称问题（牙科预训练集中于视觉领域，缺乏大规模文本语料）。创新点在于强调通用与专用模型的互补性，并指出安全部署需解决生成模型幻觉、标注数据有限和标准化评估缺失三大障碍。

Abstract: Background: Oral diseases affect nearly 3.5 billion people worldwide, yet the comparative clinical potential of large-scale AI models in dentistry remains poorly understood. Three distinct model categories have emerged: language-generative models, discriminative vision foundation models, and dental-specific foundation models, with no unified review examining their relationships and collective limitations. Methods: Following PRISMA-ScR guidelines, we systematically searched four databases (PubMed, Google Scholar, Scopus, arXiv), screened independently by two reviewers. After applying inclusion/exclusion criteria, 97 studies (2020-2026) were included. We propose a two-dimensional classification framework organizing models by architectural paradigm and dental specialization degree. Results: Language-generative models excel at text-based tasks (clinical reasoning, licensing exams, patient communication) but show inconsistent performance on image-dependent diagnostics. Adapted SAM and CLIP variants achieve strong tooth segmentation and lesion detection results. Dental-specific models (DentVFM, DentVLM, OralGPT) demonstrate strongest performance on complex multimodal tasks. Integrated pipelines consistently outperform single-model approaches. A data asymmetry is observed: dental-specific pretraining concentrates almost entirely in the vision domain, reflecting scarce large-scale dental text corpora. Conclusions: General-purpose and dental-specific models play complementary roles; the most effective systems combine both within structured pipelines. Safe autonomous deployment requires resolving three persistent barriers: hallucination in generative models, limited annotated dental datasets, and absent standardized clinical evaluation benchmarks.

[130] Inducing Reasoning Primitives from Agent Traces cs.AI | cs.CLPDF

Zhihan Lei, Jiarui Yan, Joshua Momo, William W. Cohen

TL;DR: 本文提出了一种名为’推理原语归纳’的单次学习方法，用于从ReAct式LLM智能体的成功轨迹中挖掘并归纳出可复用的推理原语，构建成一个紧凑的、类型化的伪工具库。在测试时，标准的ReAct循环可以组合调用这些原语。实验表明，使用归纳出的原语库的智能体，其性能显著超越了生成这些轨迹的原始智能体，并在多个推理任务上超越了零样本思维链和专家设计的分解方法。

Details

Motivation: ReAct式LLM智能体在不同问题中经常重复发现相同的推理模式，但这些模式仅存在于临时的草稿中，未被显式地捕获和复用。本文旨在从成功的智能体轨迹中自动归纳出这些可复用的推理原语，以提升智能体的性能和泛化能力。

Result: 在RuleArena NBA、MuSR团队分配和NatPlan会议规划等任务上，使用归纳原语库的智能体性能分别提升了44、30和22个百分点。在涵盖叙事推理、规则应用和约束满足规划的五项可比子任务中，该方法在每项任务上都超越了零样本思维链，达到或超过了专家设计的分解方法，并且以更低的平均推理成本超越了AWM方法。

Insight: 核心创新点在于提出了一种从智能体轨迹中自动归纳可复用推理原语（伪工具）的单次学习方法，将隐式的、临时的推理模式转化为显式的、可组合的库。这提供了一种提升智能体性能的新范式，即通过从自身成功经验中学习并固化有效推理步骤，实现自我超越。

Abstract: ReAct-style LLM agents often rediscover the same reasoning routines across problems, yet leave those routines trapped in transient scratchpads. We introduce Reasoning Primitive Induction, a single-pass method that mines successful ReAct traces, clusters recurrent reasoning moves, and converts the most frequent moves into a compact library of typed pseudo-tools. Each pseudo-tool is specified by a natural-language docstring interpreted by an LLM at invocation time, and a standard ReAct loop composes these primitives at test time. The central result is that induced libraries outperform the very agent that generated their traces: by +44pp on RuleArena NBA (30 -> 74), +30pp on MuSR team allocation (38 -> 68), and +22pp on NatPlan meeting planning (7 -> 29). Across five comparable subtasks spanning narrative deduction, rule application, and constraint-satisfaction planning, a single fixed configuration improves over zero-shot Chain-of-Thought on every subtask, matches or surpasses expert-authored decompositions, and outperforms AWM at lower average inference cost.

[131] Bridging Auxiliary Constraints to Resolve Instruction Following in Large Reasoning Models cs.AI | cs.CLPDF

Zhengyi Zhao, Shubo Zhang, Huimin Wang, Zezhong Wang, Yutian Zhao

TL;DR: 本文针对大型推理模型在遵循多指令时存在的约束遵循问题，提出了一种新颖的框架CRGC。该框架将指令表示为结构化的约束知识图谱，通过建模约束关系、识别遵循难点并发现‘桥接约束’来帮助模型更好地聚焦和协调不同要求。实验表明，该方法在三个常用指令遵循数据集上，相比标准提示方法减少了39%的约束违反，同时保持了模型的推理能力。

Details

Motivation: 大型推理模型在许多任务上表现出色，但在可靠遵循多个指令方面存在困难，要么无法满足单个约束，要么难以同时平衡相互竞争的约束。本文将此挑战形式化为约束遵循问题，旨在解决模型在复杂指令遵循中的可靠性问题。

Result: 在三个流行的指令遵循数据集上的实验表明，该方法相比标准提示方法减少了39%的约束违反，同时保持了大型推理模型的推理能力。

Insight: 核心创新在于将指令遵循问题形式化为约束知识图谱的构建与补全，并引入‘桥接约束’作为辅助指令来增强主要约束的显著性和兼容性。与通过通用训练方法增强指令遵循的现有方法不同，CRGC利用模型自身的知识来创建更好的生成路径，从而专门提升约束满足能力。

Abstract: Large Reasoning Models (LRMs) have demonstrated impressive capabilities in many tasks, yet they struggle with reliably following multiple instructions, either by failing to satisfy individual constraints or by struggling to balance competing constraints simultaneously. We formalize this challenge as the Constraint Adherence Problem (CAP). This paper introduces a novel framework that addresses CAP by representing instructions as a structured knowledge graph of constraints. Our approach, Constraint Relationship Graph Completion (CRGC), explicitly models relationships between constraints, identifies adherence challenges, and discovers ``bridge constraints’’ that help the model better focus on and reconcile requirements. Bridge constraints act as auxiliary instructions that make primary constraints more salient and compatible. Unlike existing approaches that enhance instruction following through general training methods, CRGC specifically improves constraint satisfaction by leveraging the model’s own knowledge to create better pathways for generation. Experiments across three popular instruction following datasets demonstrate that our approach reduces constraint violations by 39% compared to standard prompting while maintaining reasoning abilities of large reasoning models.

cs.MM [Back]

[132] Inference-Time Scaling for Joint Audio-Video Generation cs.MM | cs.CV | cs.SD | eess.ASPDF

Jaemin Jung, Kyeongha Rho, Inkyu Shin, Joon Son Chung

TL;DR: 本文首次系统研究了推理时缩放（ITS）在联合音视频生成中的应用，提出了一种无需训练的多验证器框架，通过自适应奖励加权算法平衡异构目标，显著提升了生成样本的语义对齐、感知质量和音视频同步性。

Details

Motivation: 现有联合音视频生成模型通常需要大量训练资源来提升保真度，而ITS作为一种免训练方法在单模态领域展现出潜力，但扩展到多模态领域面临异构目标平衡的挑战。

Result: 在VGGSound和JavisBench-mini基准测试中，该框架显著提升了生成输出的语义对齐、感知质量和音视频同步性，实现了多维度质量的平衡改进。

Insight: 创新点包括：提出多验证器框架以克服单目标引导的局限性；引入自适应奖励加权算法，将奖励聚合视为在线优化问题，无需先验知识即可校准奖励方差，实现鲁棒的多目标选择。

Abstract: Joint audio-video generation aims to synthesize realistic audio-video pairs that are both semantically aligned with text prompts and precisely synchronized. While existing joint audio-video generation models often require substantial training resources to improve fidelity, Inference-Time Scaling (ITS) has recently emerged as a promising training-free alternative in single-modality domains. However, extending ITS from a single modality to multimodal domains is non-trivial, as it requires balancing multiple heterogeneous objectives. In this paper, we present the first comprehensive study of ITS for joint audio-video generation. We first demonstrate that a multi-verifier framework is essential to address the limitations of single-objective guidance, including asymmetric performance trade-offs and verifier hacking. Through systematic analysis, we then identify an optimal multi-verifier combination that yields balanced improvements across all quality dimensions. Finally, to effectively aggregate diverse reward signals, we propose Adaptive Reward Weighting (ARW), a novel test-time optimization algorithm. ARW treats reward aggregation as an online optimization problem, utilizing learnable parameters to calibrate reward variances without requiring prior knowledge of reward distributions, thereby ensuring robust multi-objective selection. Experimental results on VGGSound and JavisBench-mini benchmarks demonstrate that our framework significantly enhances semantic alignment, perceptual quality, and audio-visual synchronization of generated outputs. Synthesized samples and code are available on the project page: https://jung-jaemin.github.io/ITS-AVGen-Proj.

cs.RO [Back]

[133] SCOPE: Real-Time Natural Language Camera Agent at the Edge cs.RO | cs.AI | cs.CL | cs.CV | cs.HCPDF

Nikolaj Hindsbo, Sina Ehsani, Pragyana Mishra

TL;DR: 本文提出了SCOPE，一个专为边缘部署设计的模块化智能体，用于自然语言驱动的云台变焦（PTZ）相机控制和视觉场景理解。它包含一个基于Blender的仿真环境和物理PTZ相机平台，在边缘本地执行感知、规划与控制，并发布了一个包含536个任务的基准测试，用于评估延迟、准确性和错误模式。

Details

Motivation: 为了解决在机器人领域部署语言驱动智能体时，需要能反映真实世界任务需求（如可复现的自然语言指令）的评估问题，并构建一个连接语言模型与可调用感知、控制工具，且能使用延迟、准确性等关键部署指标进行评估的系统。

Result: 在发布的仿真基准上评估了19种规划-感知模型组合（使用Qwen3小型语言模型与Moondream、Qwen视觉语言模型配对）。结果表明，更强的SLM能显著减少幻觉并改进工具路由，从而带来更可靠的闭环行为；当SLM足够强时，感知成为主要性能瓶颈；混合专家模型在规划与感知侧均能持续匹配或超越密集模型，且在延迟和内存占用上与更小网络相当；量化在精度损失最小的情况下提供了额外的效率增益。

Insight: 创新点在于提出了一个专为边缘部署设计的、仿真与实物结合的模块化语言驱动相机控制与评估框架（SCOPE）及配套基准。客观分析认为，其通过系统性的模型组合评估，明确了规划与感知模块在不同能力模型下的瓶颈转移，并验证了混合专家模型和量化在边缘实时应用中的高效性，为实际部署提供了经过仿真到现实验证的设计方案。

Abstract: Deploying language-driven agents in robotics requires evaluations that reflect real-world task demands: natural-language instructions with reproducible outcomes. Such agents must connect language models to callable perception and control tools, and be assessed using deployment-critical metrics including latency, accuracy, and error modes. We present SCOPE (Simulation and Camera Operations for Perception and Evaluation), a modular agent for natural-language, open-vocabulary pan-tilt-zoom (PTZ) camera control and visual scene understanding, designed explicitly for edge deployment. SCOPE operates both in a Blender-based simulation environment and on a physical PTZ camera, executing all perception, planning, and control locally at the deployment site using edge-accessible compute. We release a 536-task benchmark spanning QA, single- and multi-step commands, counting, spatial reasoning, descriptions, and optical character recognition in a Blender-based simulation environment that exposes realistic PTZ control affordances. Execution traces are combined with an LM-as-Judge to evaluate latency, accuracy, and error modes. We evaluate 19 planner-perception model combinations pairing Qwen3 small language models (SLMs) with Moondream and Qwen vision-language models (VLMs). Stronger SLMs substantially reduce hallucinations and improve tool routing, leading to more reliable closed-loop behavior. Once a sufficiently capable SLM is used, perception becomes the dominant performance bottleneck. Mixture-of-Experts models on both the planning and perception side consistently match or exceed dense alternatives at latencies and memory footprints comparable to much smaller networks. Quantization provides additional efficiency gains with minimal accuracy degradation, identifying a practical, sim-to-real validated design point for real-time, edge-feasible language-driven PTZ control.

[134] MARIO: Motion-Augmented Real-Time Multi-Sensor Inertial Odometry cs.RO | cs.CV | cs.HCPDF

Yiquan Li, Taeyoung Yeon, Chenfeng Gao, Vasco Xu, Xuanyou Liu

TL;DR: 本文提出MARIO，一种结合人体运动学先验与多传感器融合的实时惯性里程计方法，通过学习的IMU推断姿态先验增强物理一致性运动约束，并集成磁力计、气压计等辅助传感器，显著降低了位置漂移。

Details

Motivation: 现有基于学习的惯性里程计方法在泛化性上虽有提升，但仍易受漂移和噪声影响，尤其在日常活动数据集（如Nymeria）上表现不佳，原因在于未显式建模人体运动动力学。

Result: 在挑战性数据集Nymeria（规模为先前工作的5倍）上，位置漂移降低高达36%；结合多传感器融合后，漂移进一步减少42%，实现了准确且鲁棒的无相机人体跟踪新基准。

Insight: 创新点在于将人体运动学先验嵌入惯性里程计架构，并利用AR眼镜已有轻量传感器进行多模态融合，为惯性及轻量里程计提供了统一运动学与多模态感知的新范式。

Abstract: Inertial odometry (IO) using only Inertial Measurement Units (IMUs) provides a lightweight solution for human motion tracking in augmented reality (AR) and wearable devices. Recent learning-based IO methods have improved the generalizability of inertial localization through large-scale pretraining on human motion datasets. However, these approaches remain prone to drift and noise because they do not explicitly capture human motion dynamics, especially on daily activity datasets such as Nymeria. In this work, we propose to ground inertial odometry in human kinematics through a learned IMU-inferred pose prior, which promotes physically consistent motion constraints. We integrate this pose prior into existing IO architectures and reduce positional drift by up to 36% on the challenging Nymeria dataset, which is 5x larger than datasets used in prior work. We further improve long-term performance with a sensor-fusion framework that incorporates auxiliary signals from lightweight sensors already available on commercial AR glasses, including magnetometers, barometers, and secondary IMUs. With this fusion strategy, positional drift is reduced by up to 42%, improving robustness and generalization across diverse motion conditions. Together, our results introduce a new paradigm for inertial and lightweight odometry by unifying human motion kinematics with multimodal sensing, setting a new benchmark for accurate and robust camera-less human tracking. Our website is available at https://spice-lab.org/projects/MARIO/.

[135] Face versus Body Tracking for Human-Robot Interaction: An Egocentric Dataset cs.RO | cs.CV | cs.HCPDF

Jessica Wenninger, Gabriel Skantze

TL;DR: 本文针对人机交互中机器人持续跟踪用户身份的需求，提出了一个从Furhat机器人视角收集的新型自定义标注的自我中心数据集，用于捕捉复杂社交动态。通过系统评估比较了人脸与身体跟踪、扩展空间记忆和外观重识别的影响，发现优化后的流程将身份切换减少了49%。

Details

Motivation: 现有最先进的计算机视觉模型主要针对监控或自动驾驶场景优化，而社交机器人面临独特的自我中心挑战，如人员跳动、相互遮挡或离开画面，导致频繁的身份切换，破坏交互连续性。

Result: 在自定义的自我中心数据集上评估表明，增加空间记忆可缓解长时间遮挡，但对复杂动态事件无效；集成重识别显著提升身体跟踪稳定性，却因侧脸角度敏感性导致人脸身份切换激增。最终优化流程将身份切换减少49%。

Insight: 创新点在于构建了专门针对人机交互社交动态的自我中心数据集，并系统分析了人脸与身体跟踪在复杂遮挡下的不同表现；客观而言，该工作强调了使用原生捕获的社交场景数据来验证HRI感知模型的必要性，而非依赖标准基准。

Abstract: To enable meaningful human-robot interaction (HRI), a robot must continuously assess engagement by consistently tracking users over time. State-of-the-art computer vision models, however, are heavily optimized for surveillance or autonomous driving. A social robot faces distinct egocentric challenges, such as humans bouncing, obstructing each other, or leaving the frame. Frequent identity switches (IDSW) cause the robot to lose its footing mid-conversation. To address this, we introduce a novel, custom-annotated egocentric dataset collected via the Furhat robot to capture complex social dynamics. We present a systematic evaluation isolating detection errors from tracking logic, comparing face versus body tracking, and assessing the impact of extended spatial memory and appearance re-identification (ReID). Results indicate that increasing spatial memory mitigates prolonged occlusions but fails on complex dynamic events. Integrating ReID resolves complex switches but exhibits opposing effects: it substantially improves body tracking stability, yet causes facial IDSW to spike due to profile angle sensitivity. Ultimately, our optimized pipeline reduces IDSW by 49%, mitigating interaction breakdowns. Because standard benchmarks lack dense, close-quarter occlusions, this work highlights the critical need for natively captured social dynamics to truly validate HRI perception models.

[136] PHASER: Phase-Aware and Semantic Experience Replay for Vision-Language-Action Models cs.RO | cs.AI | cs.CVPDF

Ziyang Chen, Shaoguang Wang, Weiyu Guo, Qianyi Cai, He Zhang

TL;DR: 本文提出了PHASER，一个用于视觉-语言-动作模型的持续学习框架，旨在解决在开放环境中部署时出现的灾难性遗忘问题。该框架通过基于相位的容量分配确保所有子技能获得同等记忆支持，并采用多模态干扰路由策略动态优先处理高遗忘风险的历史阶段。此外，还集成了Auto-PC，一个结合无监督动作信号变化点检测和基于VLM的语义验证的轻量级流程，以自动提取时间边界。

Details

Motivation: 动机是解决VLA模型在开放环境中持续学习新技能时，不可避免地触发对先前学习行为的严重灾难性遗忘问题。标准的经验回放方法由于均匀采样，与操作轨迹的时间特性不匹配，导致对关键但短暂的子技能采样不足（相位饥饿），并忽略了历史任务间不同程度的遗忘。

Result: 在LIBERO持续学习套件上对三个VLA骨干网络进行评估，PHASER带来了显著的实证改进，相比匹配预算的经验回放，平均成功率最高提升了31%，并在LIBERO-Goal CL设置上实现了87.8%的最终平均成功率。

Insight: 宣称的创新点包括：1) 相位感知的容量分配，以缓解子技能采样不均；2) 多模态干扰路由，动态管理遗忘风险；3) 集成了Auto-PC实现全自动、无需密集人工监督的终身适应。从客观角度看，其将操作轨迹分解为语义相位并据此进行记忆管理的思路，以及对遗忘风险的动态量化，是解决VLA持续学习问题的有效且新颖的途径。

Abstract: Vision-Language-Action (VLA) models have achieved remarkable success in language-conditioned robotic manipulation. However, deploying these models in open-ended environments requires continuously acquiring novel skills, a process that inevitably triggers severe catastrophic forgetting of previously learned behaviors. While experience replay (ER) serves as a standard mitigating strategy, naive uniform sampling fundamentally misaligns with the temporal characteristics of manipulation trajectories. It systematically under-samples brief but causally critical sub-skills, leading to phase starvation, and completely overlooks the varying degrees of forgetting across historical tasks. To overcome these limitations, we introduce PHASER, an architecture-agnostic continual learning framework. PHASER employs a phase-centric capacity allocation to guarantee equal memory support for all sub-skills, coupled with a multi-modal interference routing strategy that dynamically prioritizes historical phases at high risk of forgetting. Furthermore, to enable fully autonomous lifelong adaptation, we integrate Auto-PC, a lightweight pipeline combining unsupervised action-signal change-point detection with VLM-based semantic verification to extract temporal boundaries without intensive manual supervision. Evaluated across three VLA backbones on LIBERO continual learning suites, PHASER yields substantial empirical improvements, increasing Average Success Rate (ASR) by up to 31% over matched-budget ER and achieving an 87.8% final ASR on the LIBERO-Goal CL setting.

Table of Contents

cs.CL [Back]

[1] IdiomX A Multilingual Benchmark for Idiom Understanding, Retrieval, and Interpretation cs.CL | cs.AI | cs.IRPDF

[2] Economy of Minds: Emerging Multi-Agent Intelligence with Economic Interactions cs.CL | cs.AI | cs.MAPDF

[3] Adaptive Latent Agentic Reasoning cs.CL | cs.AIPDF

[4] Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States cs.CL | cs.AIPDF

[5] WRIT: Write-Read Intensive Trajectory Synthesis for Multi-Turn User-Facing Agents cs.CL | cs.AIPDF

[6] EURO-5K: When Does Domain Pretraining Matter? Benchmarking Transformers for EU Reporting Obligation Extraction cs.CLPDF

[7] Predicting Inference-Time Scaling Gains from Labeled Validation-Set Output Statistics cs.CLPDF

[8] Hint-Guided Diversified Policy Optimization for LLM Reasoning cs.CLPDF

[9] PhotoCraft: Agentic Reasoning with Hierarchical Self-Evolving Memory for Deep Image Search cs.CL | cs.AIPDF

[10] DMT-CBT: Longitudinal Therapeutic State Modeling for CBT Counseling cs.CLPDF

[11] MemTrain: Self-Supervised Context Memory Training cs.CLPDF

[12] HyperPatch: Sequential Knowledge Editing Under n-ary Structural Drift cs.CLPDF

[13] WebRISE: Requirement-Induced State Evaluation for MLLM-Generated Web Artifacts cs.CL | cs.AIPDF

[14] ARBOR: Online Process Rewards via a Reusable Rubric Buffer for Search Agents cs.CLPDF

[15] SEA-NLI: Natural Language Inference as a Lens into Southeast Asian Cultural Understanding cs.CLPDF

[16] SagaQA: A Multi-hop Reasoning Benchmark for Long-form Narrative Understanding in TV Series cs.CL | cs.CVPDF

[17] From Script to Semantics: Prompting Strategies for African NLI cs.CL | cs.LGPDF

[18] Evaluating LLMs’ Effectiveness on Real-World Consumer Device Repair Questions cs.CL | cs.AIPDF

[19] Lingo_Research_Group at SemEval-2026 Task 9: Evaluating Prompt Variants for Polarization Detection cs.CL | cs.LGPDF

[20] The Unsampled Truth: Psychometrics in SLMs Measure Prompt Artifacts, Not Psychological Constructs cs.CL | cs.AIPDF

[21] See, Infer, Intervene: Proactive World Modeling for Goal-Oriented Social Intelligence cs.CLPDF

[22] Beyond the Literal: Decomposing Pragmatic Intent in Multimodal Meme Understanding cs.CLPDF

[23] Does Language Shift Break Medical Vision-Language Models? Indonesian Radiology Visual Question Answering Case Study cs.CL | cs.CVPDF

[24] HybridThinker: Efficient Chain-of-Thought Reasoning via Compressed Memory and Transient Thought Steps cs.CLPDF

[25] Framing Migration News with LLMs: Structured CoT as a Support for Human Interpretation cs.CLPDF

[26] Reasoning over Grammar: Can Synthetic Linguistic Reasoning Traces Enhance Low-Resource Machine Translation? cs.CLPDF

[27] Exploring Adversarial Robustness and Safety Alignment in Multilingual Multi-Modal Large Language Models cs.CL | cs.CVPDF

[28] Agentic Chain-of-Thought Steering for Efficient and Controllable LLM Reasoning cs.CL | cs.AIPDF

[29] Synthesize and Reward – Reinforcement Learning for Multi-Step Tool Use in Live Environments cs.CL | cs.AI | cs.LGPDF

[30] Quantifying Faithful Confidence Expression in Large Reasoning Models cs.CL | cs.AIPDF

[31] QUBRIC: Co-Designing Queries and Rubrics for RL Beyond Verifiable Rewards cs.CL | cs.AIPDF

cs.CV [Back]

[32] AVTrack: Audio-Visual Tracking in Human-centric Complex Scenes cs.CV | cs.AIPDF

[33] Plan2Map: A Multimodal Benchmark for Document-Grounded Geospatial Boundary Reconstruction from Planning Records cs.CV | cs.AIPDF

[34] Consistent Yet Wrong: Evidence Insensitivity in Spatial Vision-Language Models cs.CVPDF

[35] MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Data cs.CV | cs.AIPDF

[36] Diagnosis of Human Object Interaction Detectors for Real World Educational Applications cs.CVPDF

[37] GeoDrive-Bench: Benchmarking Region-Specific Multimodal Reasoning in Autonomous Driving cs.CVPDF

[38] Cosmos 3: Omnimodal World Models for Physical AI cs.CV | cs.AI | cs.LG | cs.MM | cs.ROPDF

[39] Automated Report-Derived Oncology VQA Benchmark for Evaluating Vision-Language Models on 3D Medical Imaging cs.CVPDF

[40] Pathway-Structured Privileged Distillation for Deployable Computational Pathology cs.CVPDF

[41] Tiny Collaborative Inference for Occlusion-Robust Object Detection cs.CVPDF

[42] Pixel Cube: Diffusion-based Portrait Video Relighting Through Realistic Lighting Reproduction cs.CVPDF

[43] Any2Poster: Any-Source Poster Generation Across Modalities and Domains cs.CVPDF

[44] The Road Ahead in Autonomous Driving: The KITScenes Multimodal Dataset cs.CV | cs.LG | cs.ROPDF

[45] Hand Trajectory Fusion for Egocentric Natural Language Query Grounding cs.CV | cs.AI | cs.HC | eess.IVPDF

[46] Towards Compact Autonomous Driving Perception with Balanced Learning and Multi-sensor Fusion cs.CV | cs.AI | cs.ROPDF

[47] MUSE: A Unified Agentic Harness for MLLMs cs.CV | cs.AIPDF

[48] Hierarchical Federated Learning with Dynamic Clustering and Adaptive Regularization for Robust Infrastructure Inspection cs.CVPDF

[49] TGV-KV: Text-Grounded KV Eviction for Vision-Language Models cs.CVPDF

[50] Zero-Shot 3D Question Answering via Hierarchical View-to-Token Transportation cs.CV | cs.LGPDF

[51] KC-3DGS: Kurtosis-Constrained Gaussian Splatting for High-Fidelity View Synthesis cs.CVPDF

[52] FAF-CD: Frequency-Aware Fusion for Change Detection under Imperfect Multimodal Remote Sensing cs.CVPDF

[53] Disentangling Visual and Factual Correctness in LVLMs’ Visualization Literacy cs.CVPDF

[54] JAVEDIT: Joint Audio-Visual Instruction-Guided Video Editing with Agentic Data Curation cs.CVPDF

[55] NVIDIA OmniDreams: Real-Time Generative World Model for Closed-Loop Autonomous Vehicle Simulation cs.CV | cs.AI | cs.ROPDF

[56] GLINT: Sparsely Gated Vision-Language Alignment for Fine-Grained Radiology Representations cs.CV | cs.CL | cs.LGPDF

[57] Reinforcement Learning from Cross-domain Videos with Video Prediction Model cs.CV | cs.AIPDF

[58] Ask When It Pays: Cost-Aware Open-Ended Interaction for Instance Goal Navigation cs.CV | cs.ROPDF

[59] Follow-Your-Preference++: Rethinking Preference Alignment for Image Inpainting cs.CVPDF

[60] PaddleOCR-VL-1.6: Expanding the Frontier of Document Parsing with Under-Optimized Region Refinement and Progressive Post-Training cs.CVPDF

[61] MemoGen: Can Past Experience Improve Future Text-to-Image Generation? cs.CVPDF

[62] VistaHop: Benchmarking Multi-hop Visual Reasoning for Visual DeepSearch cs.CV | cs.AI | cs.CLPDF

[63] Cross-Modality Feature Fusion Based on Structured State Space Duality for Multimodal Image Registration Network cs.CVPDF

[64] Beyond Semantics: Modeling Factual and Affective Perceptual Experiences from Vision-Language Data cs.CV | cs.CL | cs.CYPDF

[65] SynCred-Bench: Benchmarking Synthetic Credibility in AI-Generated Visual Misinformation cs.CV | cs.AIPDF

[66] P\textsuperscript{2}-DPO: Grounding Hallucination in Perceptual Processing via Calibration Direct Preference Optimization cs.CV | cs.AI | cs.CL | cs.LGPDF

[67] Towards Characterizing Scientific Image Utility and Upgradability cs.CVPDF

[68] Mamba-Enhanced Implicit Motion Learning for Audio-Driven Portrait Animation cs.CVPDF

[69] SAMatcher: Co-Visibility Modeling with Segment Anything for Robust Feature Matching cs.CVPDF

[70] A unified multi-task framework enables interpretable chest radiograph analysis cs.CVPDF

[71] Enginuity: A Dataset and Benchmark for Vision-Language Understanding of Engineering Diagrams cs.CVPDF

[72] IDO: Incongruity-aware Distribution Optimization for Multimodal Fake News Detection cs.CVPDF

[73] PRISM: Synergizing Vision Foundation Models via Self-organized Expert Specialization cs.CV | cs.AIPDF

[74] Mixed-Modality Dual Face-Hair Retrieval cs.CVPDF

[75] From 3D Perception to Safety Reasoning: A Graph-Based Framework for Real-Time Underground Mine Monitoring cs.CVPDF

[76] EvoMemNav: Efficient Self-Evolving Fine-Grained Memory for Zero-Shot Embodied Navigation cs.CVPDF

[77] PersistGS: Differentiable Physics for Object Permanence in 4D Gaussian Splatting cs.CV | cs.GRPDF