cs.CL [Total: 19]
cs.CV [Total: 72]
cs.MA [Total: 2]
cs.LG [Total: 4]
cs.IR [Total: 2]
cs.RO [Total: 1]
cs.AI [Total: 7]
cs.SD [Total: 1]
cs.SE [Total: 1]
cs.MM [Total: 2]

cs.CL [Back]

[1] PRO-Step: Step-level Process Reward Optimization for Retrieval-Augmented Generation cs.CL | cs.AIPDF

MinKeon Kim, Namjun Lee, Jaekwang Kim

TL;DR: 该论文提出了PRO-STEP方法，用于优化检索增强生成中的多步推理过程。该方法通过训练一个生成式过程奖励模型来评估每一步的逻辑有效性和证据基础，并利用该模型引导的价值树搜索构建偏好对，最后通过步骤级的直接偏好优化来优化策略。

Details

Motivation: 动机在于解决检索增强生成在多跳推理中存在的错误传播问题，以及现有基于结果的优化方法无法检测中间步骤错误、而现有基于过程的方法又可能奖励虚假成功（即错误的检索巧合产生正确答案）的局限性。

Result: 在单跳和多跳问答数据集上的实验表明，PRO-STEP在五个基准测试中取得了最佳的平均精确匹配率和F1分数。

Insight: 创新点在于提出了一个结合逻辑有效性和证据基础的步骤级监督框架，并利用生成式PRM引导的价值树搜索来构建高质量的偏好数据，从而通过步骤级DPO进行更精细的策略优化，有效缓解了错误传播和虚假奖励问题。

Abstract: Retrieval-Augmented Generation enhances Large Language Models by grounding responses in external knowledge, but multi-hop reasoning remains vulnerable to error propagation, where early retrieval failures confound subsequent steps. Standard outcome-based optimization only rewards the final answer, leaving intermediate retrieval and reasoning errors undetected. While existing process-based methods introduce step-level signals, they still score each step against the final answer, rewarding spurious successes where flawed retrieval coincidentally produces the correct answer. Step-level supervision in RAG requires evaluating both logical validity and evidential grounding at each step. We introduce PRO-STEP: we train a generative PRM that evaluates both dimensions, employ PRM-guided value tree search to construct preference pairs contrasting valid steps against flawed ones, and optimize the policy via step-level Direct Preference Optimization. Experiments on single and multi-hop QA datasets demonstrate that PRO-STEP achieves the best average EM and F1 across five benchmarks. Code, models, and training data are publicly available at https://github.com/keemminnke/PRO-Step.

[2] MemeCULT-1K: Benchmarking South Asian Cultural Context and Humor Understanding of Multimodal Models cs.CL | cs.CVPDF

Tawsif Tashwar Dipto, Mehedi Ahamed, Radib Bin Kabir, Mueeze Al Mushabbir, Mohammed Saidul Islam

TL;DR: 本文提出了MemeCULT-1K基准测试，这是一个包含1000个南亚多语言（孟加拉语、英语、印地语）迷因的数据集，用于评估多模态模型对文化背景和幽默的理解能力。研究发现，为模型提供最小的文化背景信息能显著提升其解释迷因的性能，并揭示了开源与闭源模型在文化理解上的不同失败模式。

Details

Motivation: 当前大多数视觉语言模型在理解迷因时，缺乏对隐含文化知识和语用推理的能力，尤其是在南亚多语言文化背景下。本文旨在通过构建一个专门的基准来量化这一差距。

Result: 在MemeCULT-1K基准上评估了13个流行的视觉语言模型。提供文化背景后，所有模型和语言的平均性能均有显著提升：SBERT相似度从44.6提升至56.4，BLEURT从37.3提升至42.3，LLM-as-a-Judge评分（5分制）从2.57提升至3.43。细粒度错误分析揭示了闭源模型主要失败于实体和指代识别，而开源模型则受限于更广泛的文化知识鸿沟。

Insight: 论文的创新点在于构建了一个专注于南亚文化背景的多语言迷因理解基准，并强调了显式整合文化知识对于提升模型性能的重要性。从客观角度看，其提出的“仅迷因”与“上下文感知”两种评估设置，为分析模型的文化理解瓶颈提供了有效框架。

Abstract: Meme understanding goes beyond recognizing visual content or literal text; it requires implicit cultural knowledge and pragmatic inference that most vision-language models still lack. We introduce MemeCULT-1K, a multilingual benchmark of 1,000 South Asian memes in Bengali, English, and Hindi, where each meme is paired with a cultural context note and three human-written explanations, along with a supplementary set of 54 Bengali regional dialect memes. We evaluate thirteen popular Vision Language Models (VLMs) under two settings: meme-only and context-aware. Providing minimal cultural context yields consistent gains across all models and languages: mean SBERT similarity improves from 44.6 to 56.4 (+11.8), BLEURT from 37.3 to 42.3 (+5.0), and LLM-as-a-Judge scores from 2.57 to 3.43 out of 5 (+0.86). Fine-grained error analysis reveals that closed-source models fail mainly on entity and reference misidentification, while open-source models are bottlenecked by broader cultural knowledge gaps, with linguistic and phonological failures proving the most context-resistant across both. These results highlight the difficulty of culturally grounded meme understanding and motivate future work on explicit cultural knowledge integration. Our dataset and code are publicly available at TawsifDipto17/MemeCULT-1K.

[3] Learning Evidence Sufficiency Boundaries for Selective Answering in Grounded Multi-Hop QA cs.CLPDF

Haruto Sato, Yuki Tanaka, Ren Nakamura, Aoi Kobayashi, Mei Ito

TL;DR: 本文提出了一种名为证据充分性边界训练（Evidence Sufficiency Boundary Training）的生成式训练框架，用于解决多跳问答（Multi-Hop QA）中的选择性回答（Selective Answering）问题。该框架通过构建有序的证据链，并直接监督模型从“拒绝回答”到“给出答案”的转换边界，旨在让模型仅在证据充分时回答，在证据不足或部分支持时拒绝回答。

Details

Motivation: 在基于证据的问答系统中，模型应仅在所提供的证据支持答案时才回答。多跳问答中，部分证据可能使不支持的答案看起来合理，导致模型错误回答。因此，需要一种方法来精确界定证据何时变得充分，以指导模型何时应回答或拒绝。

Result: 在HotpotQA、2WikiMultiHopQA和MuSiQue数据集上构建证据链进行评估。使用Qwen2.5-3B-Instruct模型和LoRA适配，该方法在边界定位上表现最强，翻转准确率达到0.807（基线为0.781）。在外部不可回答集上的不支持答案率最低，为0.095（基线为0.101），同时保持了有竞争力的原始问答F1分数。

Insight: 创新点在于提出了“证据充分性边界”的概念，并设计了专门的训练框架来直接监督这个边界转换。该框架结合了层级监督、边界翻转边际、边界后稳定性和答案召回保护等技术，使模型能更精确地判断证据充分性，从而在保持回答能力的同时，有效降低对不支持问题的错误回答率。

Abstract: Grounded question answering systems should answer only when the supplied evidence supports the answer. In multi-hop QA, this requirement is difficult because partial evidence can make an unsupported answer appear plausible. We study selective answering through evidence sufficiency boundaries: for the same question, a model should abstain under unsupported or partially supported context, answer when the context first becomes sufficient, and keep the answer stable when redundant evidence is added. We introduce Evidence Sufficiency Boundary Training, a generation-native training framework that constructs ordered evidence chains and supervises the abstain-to-answer transition directly. The method combines level supervision, a boundary flip margin, post-boundary stability, and answer recall protection. We build evidence chains from HotpotQA, 2WikiMultiHopQA, and MuSiQue, then evaluate models with chain metrics, raw QA utility, and unsupported-answer rates on external non-answerable sets. With Qwen2.5-3B-Instruct and LoRA adaptation, Evidence Sufficiency Boundary Training gives the strongest boundary localization among the tested systems, with flip accuracy of 0.807 compared with 0.781 for a token-level abstention baseline. It also achieves the lowest overall unsupported-answer rate on external non-answerable evaluation, 0.095 compared with 0.101 for the same baseline, while retaining competitive raw QA F1. The results show that grounded selective answering improves when training marks the evidence level where refusal should give way to answering.

[4] VakyArth: Evaluating Pragmatic Competence in LLMs across Indic Languages cs.CL | cs.AIPDF

Usneek Singh, Poorvaja Veera Balaji Kumar, Parth Nanda, Anand Madhusoodanan, Geyang Guo

TL;DR: 论文提出了VakyArth，这是首个针对印度语言（包括印地语、旁遮普语、泰米尔语和马拉雅拉姆语）的语用能力评估基准，用于评估大型语言模型在现实沟通中所需的语用推理能力，如指示、言语行为、含义、社会语用和连贯性。研究发现，现有多语言LLM在处理根植于印度语言文化惯例的语用含义时普遍存在失败，且不同语言和任务间存在系统性差异。

Details

Motivation: 现有语用评估主要局限于英语和高资源语言，而印度语言尽管具有丰富的语言和文化多样性，却未被探索。论文旨在填补这一空白，评估LLM在印度语言中的语用能力。

Result: 在涵盖不同家族和规模的多语言LLM上，模型在处理印度语言文化惯例的语用含义时表现一致不佳。分析显示，多项选择题（MCQ）准确率在所有模型-语言组合中均超过自然语言推理（NLI）准确率，翻译性能不能可靠地反映语用理解，且印度-雅利安语言在翻译上优于达罗毗荼语言。

Insight: 创新点在于构建了首个针对印度语言的语用基准VakyArth，并揭示了自动翻译指标可能遗漏流畅但语用不忠实的输出，特别是在含义和指示现象上，这强调了评估跨语言语用能力时需考虑文化特异性。

Abstract: Real-world communication often requires pragmatic reasoning: interpreting meanings implied through context and cultural convention rather than stated literally. Existing pragmatic evaluation remains largely limited to English and high-resource languages, leaving Indic languages unexplored despite their linguistic and cultural diversity. We introduce VakyArth, the first pragmatic benchmark for Indic languages, designed as a diagnostic evaluation covering Hindi, Punjabi, Tamil, and Malayalam. VakyArth evaluates models across five phenomena: deixis, speech acts, implicature, social pragmatics, and coherence; through multiple-choice questions, natural language inference, and translation, with all items authored by native speakers. Across multilingual large language models (LLMs) of varying families and sizes, we find consistent failures on pragmatic meanings rooted in Indic linguistic and cultural conventions. Our analysis shows systematic differences across languages and tasks: MCQ accuracy exceeds NLI accuracy in all model-language combinations, translation performance does not reliably track pragmatic understanding, and Indo-Aryan languages show a translation advantage over Dravidian languages. We further show that automatic translation metrics can miss fluent but pragmatically unfaithful outputs, especially for implicature and deixis.

[5] TalkFa: A Unified Benchmark for Farsi Dialogue Generation and Understanding cs.CLPDF

Neda Jamshidi, Kamyar Zeinalipour, Fahimeh Akbari, Monica Bianchini, Marco Maggini

TL;DR: 本文针对波斯语缺乏全面对话基准的问题，提出了TalkFa统一基准，包含三个互补数据集：基于维基百科的知识对话、标注对话行为与情感的日常对话、以及带有情感标签的戏剧对话。通过实验验证了LoRA微调在对话生成任务上的高效性，并展示了不同模型在分类任务上的性能表现，同时指出自动评估指标会高估对话质量。

Details

Motivation: 解决波斯语（使用人数超过1.2亿）缺乏统一的对话生成与理解综合基准的问题，以促进该语言的相关研究。

Result: 在六个LLaMA和Mistral模型上的实验表明，LoRA微调显著提升对话生成性能，仅需25-50%训练数据即可恢复90%以上的最终性能增益；在分类任务中，FABERT在对话行为识别上表现最佳，LoRA-Mistral-7B在情感识别上最优，Mistral-24B在情感分析得分最高；人类评估和外部验证证实了基准的可靠性，且与GPT-4.1对比显示自动指标会高估对话质量。

Insight: 创新点在于构建了首个波斯语多任务对话统一基准，并通过严格的人工审核流程确保数据质量；实验发现LoRA微调在数据效率方面优势明显，同时揭示了自动评估指标在对话质量评估上的局限性，为低资源语言对话研究提供了方法论参考。

Abstract: Farsi, spoken by more than 120 million people, lacks a comprehensive benchmark for dialogue generation and understanding. We introduce TALKFA, a unified benchmark comprising three complementary datasets: (1) WIKI-FADIAL, 4.2K Wikipedia-grounded dialogues for knowledge-grounded generation; (2) DAILYDIALOG-FA, 6.6K dialogues annotated for dialogue acts and emotions; and (3) PLAYDIAL-FA, 2.1K theatrical dialogues with sentiment labels. While LLMs assist data construction, every dialogue undergoes multi-stage review and revision by native Farsi speakers, and only the final human-approved dialogues are released. Experiments with six LLAMA and MISTRAL models show that LoRA substantially improves dialogue generation while requiring only 25-50% of the training data to recover over 90% of the final performance gains. Across classification tasks, FABERT achieves the best dialogue-act performance, LORA-MISTRAL-7B performs best on emotion recognition, and MISTRAL-24B achieves the highest sentiment score. Human evaluation and independent external validation demonstrate the reliability of the benchmark, while comparisons with GPT-4.1 as an LLM judge reveal that automatic metrics substantially overestimate dialogue quality. Zero-shot evaluation with frontier LLMs further shows that TalkFa remains a challenging benchmark. We will release all datasets, annotation guidelines, code, and checkpoints.

[6] Cite or Decline: A Strict Course-Grounded Chatbot for STEM Lecture Videos cs.CL | cs.CY | cs.IRPDF

S M Masrur Ahmed, Jaspal Subhlok

TL;DR: 本文介绍了VideoPoints平台中一个基于检索增强的聊天机器人，该机器人针对STEM课程视频设计，能够从特定课程材料中检索答案并提供带时间戳的引用。在一个学期的部署中，它被学生用于快速查询和考试复习，有效避免了跨课程检索，并在无证据时选择拒绝回答。

Details

Motivation: 解决学生在观看录播课程视频时难以提出课程特定问题或验证答案的问题，旨在提供一个严格基于课程内容的聊天机器人，确保答案的可追溯性和准确性。

Result: 在833条消息中，70.5%包含引用，且未出现跨课程检索；在EduVidQA基准测试中，该设计将正确讲座检索率提升了6.3个百分点。

Insight: 创新点包括课程隔离检索、使用章节摘要指导转录排名以及提供可点击的时间戳引用，这些设计确保了答案的严格课程关联性和可验证性，同时拒绝回答无证据的问题，提升了系统的可靠性。

Abstract: Recorded lecture videos, often enhanced with search and summarization features, are a standard study resource. However, students cannot easily ask course specific questions or verify answers against an instructor’s lecture. We report a semester-long deployment of VideoPoints platform with a retrieval-augmented chatbot that answers from course lecture materials and returns timestamped citations. The chatbot retrieves only from the active course, uses chapter summaries to guide transcript ranking, and returns clickable timestamped citations. Students used it for quick lookups and exam review. Across 833 messages, 70.5% included citations, none crossed a course boundary, and when no lecture evidence matched, the chatbot usually declined rather than answering. Among the users, citations were the most consistently useful feature, while practice-question generation was the strongest unmet request. We also evaluated the design on the real-world test split of EduVidQA, a public multimodal benchmark for lecture-video question answering. Our design improved correct-lecture retrieval by 6.3 percentage points over dense-only retrieval. Together, the results show that effective deployment depends on course isolation, supported citations, and alignment with students’ study practices.

[7] Thinking effort aligns between humans and reasoning models in abductive reasoning cs.CL | cs.AIPDF

Henry Arthur

TL;DR: 本文研究了大型推理模型与人类在溯因推理任务中的认知努力对齐问题，通过比较人类反应时间与模型推理轨迹，发现两者在推理努力和错误模式上存在一致性，并证明多路径探索的解码方法能进一步提升这种对齐。

Details

Motivation: 旨在探究大型推理模型与人类在非语言推理任务中的行为对齐，特别是溯因推理因其难度无法从形式结构推断且无捷径可循，为验证认知努力共享提供了更可靠的基础。

Result: 在溯因推理任务中，发现LRM与人类的推理努力存在对齐证据，且两者倾向于犯类似错误；多路径解码方法在测试的三个模型中均提升了人类与LRM在推理成本上的对齐程度。

Insight: 创新点在于利用溯因推理任务隔离认知努力对齐，避免模型通过结构捷径模仿努力；客观分析表明，强化学习优化的推理模型能更真实反映人类搜索过程，多路径探索策略可增强对齐效果。

Abstract: A major question in cognitive modeling concerns the behavioral alignment between large language models and humans across linguistic and non-linguistic tasks. Unlike standard LLMs, large reasoning models (LRMs) are optimized with reinforcement learning from verifiable rewards, encouraging correct solutions to reasoning tasks rather than preference-aligned responses. Recent work (de Varda et al., 2025) investigates the cost of thinking in humans and LRMs by comparing human reaction times with model reasoning traces across a range of reasoning tasks. We isolate this alignment by turning to abductive reasoning: unlike deductive tasks, its difficulty cannot be inferred from formal structure and offers no shortcuts a model could exploit to mimic effort without genuine search, providing firmer ground for empirical claims of shared effort. We find further evidence of alignment between LRM and human reasoning effort, as well as evidence that models and humans tend to make similar errors. Finally, we show that decoding methods that let models explore multiple reasoning paths increase alignment in reasoning cost between humans and LRMs across the three models tested.

[8] IDEEA: training-free Input-Dependent stEEring via Activation cluster matching cs.CL | cs.LGPDF

Zheng Wang, Muchen Li, Renjie Liao, Yan Leng

TL;DR: 本文提出了IDEEA，一种无需训练、输入依赖的LLM引导框架。它通过聚类激活并构建与集群条件相关的引导方向，在推理时根据输入激活选择最佳方向进行引导，从而更有效地将模型对齐到目标概念。

Details

Motivation: 现有无需训练的引导方法大多是输入无关的，即对所有输入使用单一引导方向，这存在根本性限制，因为不同输入在激活空间中占据不同区域，需要不同的最优引导方向。

Result: 在TruthfulQA基准测试中，IDEEA将truth × info rate指标平均提升了9.9%（最高达23.5%），超越了最佳输入无关基线方法。

Insight: 创新点在于提出了输入依赖的引导框架，通过激活聚类和最优匹配构建集群条件方向，其结果表明编码同一概念的激活占据表示空间的多个不同子区域，而非单一区域。

Abstract: Steering aligns large language models (LLMs) by injecting a bias into selected activations at inference time, offering a far cheaper alternative to weight-update methods such as supervised fine-tuning or reinforcement learning. However, most existing training-free steering methods are input-independent: a single direction is fitted once and shared across all inputs. This is fundamentally limiting as different inputs occupy different regions of the activation space and admit different optimal steering directions toward the same target concept, much as the gradient with respect to a fixed loss varies from input to input. We close this gap with IDEEA (Input-Dependent stEEring via Activation cluster matching), a training-free framework for input-dependent steering. IDEEA clusters the positive and negative activation supports per attention head, and solves an optimal-matching problem to construct a set of cluster-conditional directions, all about the target concept. At inference time, it picks from this pool of directions and uses the one that best matches the input’s own activation for steering. IDEEA aligns the model toward the target concept while preserving the input’s original representation, evidence that activations encoding a concept occupy several distinct sub-regions of the representation space rather than a single one. IDEEA improves the truth $\times$ info rate in TruthfulQA by an average of 9.9% (up to 23.5%) over the best input-independent baseline.

S M Rafiuddin, Atriya Sen

TL;DR: 本文提出C$^{3}$T模型，用于分析社交媒体谣言对话树中的情感转移因果推理。通过将对话行为（如否认、证据、攻击）视为干预，模型联合预测节点情感、情感转移及稀疏祖先归因，并支持反事实查询以估计潜在结果。

Details

Motivation: 研究社交媒体对话树中情感变化的原因，特别是围绕谣言的讨论，旨在理解哪些先前的消息（如否认、证据或攻击性言论）最可能驱动回复的情感。

Result: 在事件级分割下，C$^{3}$T模型在跨事件鲁棒性和归因方面优于仅文本、基于图和时序的基线，并产生可解释的模型效应：否认/纠正和证据减少下游负面情感，而毒性增加负面情感。

Insight: 创新点包括引入CaSiRe因果情感推理层来增强数据集，以及提出C$^{3}$T这一结合线程结构、时序建模和反事实推理的Transformer模型，为社交媒体分析提供了结构感知的因果建模方法。

Abstract: Sentiment in social-media threads does not only vary across posts; it shifts as users react to claims, corrections, evidence, and hostility within a branching reply tree. We study why sentiment changes in rumor-centric conversation trees by treating discourse moves (e.g., denial/correction, evidence/link, toxicity/attack) as candidate interventions and asking (i) what sentiment a reply expresses, (ii) whether the sentiment shifts relative to its parent, and (iii) which prior message most plausibly drove the reply’s sentiment. To support this setting, we introduce CaSiRe, a causal sentiment reasoning layer over public rumor conversation datasets that adds post-level sentiment labels, induced parent-child shift labels, calibrated multi-label intervention tags, and explicitly annotated causal-source labels. We then propose C$^{3}$T (Counterfactual Causal Conversation Transformer), a thread-structured temporal model that jointly predicts node sentiment and shifts, learns sparse ancestor attribution, and supports counterfactual queries by forcing conversational intervention embeddings on or off to estimate potential outcomes. Under an event-level split, C$^{3}$T improves out-of-event robustness and attribution over text-only, graph-based, and temporal baselines, and yields interpretable model-based effects: denials/corrections and evidence reduce downstream negativity, while toxicity increases it. We also benchmark open-weight LLM prompting baselines and find that added conversational context helps, but attribution remains less reliable, motivating structure-aware counterfactual modeling for social-media analysis.

[10] OBJECTION! Lawyer Agents Mitigate Guilty Bias in Legal Judgment Prediction cs.CL | cs.AIPDF

Jaehoon Jeong, Jay-Yoon Lee

TL;DR: 本文提出了OBJECTION框架，一个在推理阶段集成对抗性律师代理的流程，旨在缓解法律判决预测模型中的有罪偏见。该框架通过在罪行、违法性和罪责三个推理步骤中主动注入辩护论点，挑战模型对有罪的预设。研究还引入了一个包含3.4k真实案例的’Natural Innocent’数据集，以克服合成无罪基准的局限性。

Details

Motivation: 现有法律判决预测模型通常在从公诉视角描述事实的文档上训练，且数据集存在严重的标签不平衡（偏向有罪结果），导致模型存在’有罪偏见’，盲目接受公诉叙述为客观事实。先前研究虽通过三步推理结构或合成无罪数据训练提升了整体准确率，但未能有效在推理时缓解偏见。

Result: 测试结果显示，OBJECTION将错误有罪率从SOTA基线的82.93%大幅降低至16.69%，证明了其进行实质性法律推理的能力。评估基于新提出的’Natural Innocent’真实世界数据集进行。

Insight: 创新点在于在推理时引入对抗性律师代理，主动挑战模型的有罪预设，而非仅依赖训练数据调整或通用批评机制。这代表了将法律AI与无罪推定原则对齐的关键进展，并通过真实案例数据集提供了更可靠的评估基准。

Abstract: Legal Judgment Prediction (LJP) models are typically trained on documents that describe facts from a prosecutorial perspective. Existing datasets further exhibit severe label imbalance toward guilty outcomes. Consequently, these models suffer from “Guilty Bias”, blindly accepting the prosecution’s narrative as objective truth. Previous studies employing three-step reasoning structures or training on synthetically generated innocence data improve overall accuracy, but they still fail to mitigate bias at inference time. In this paper, we introduce OBJECTION, an inference-time pipeline that integrates an Adversarial Lawyer Agent into each 3-step reasoning of offense, unlawfulness, and culpability. Unlike generic critics, our agent actively challenges the model’s presumptions of guilt by injecting legal defense arguments at each reasoning stage. To thoroughly evaluate this, we present a new “Natural Innocent” dataset including 3.4k real-world cases, overcoming the limitations of synthetic innocence benchmarks. Test results show that OBJECTION drastically reduces the False Guilty Rate (FGR) from 82.93% (SOTA baseline) to 16.69%, proving its capability to perform substantive legal reasoning. This work denotes a key progress toward aligning Legal AI with the presumption of innocence.

[11] NE-R1: Enhancing Named Entity Recognition Model via Reinforcement Learning cs.CL | cs.AIPDF

Meixuan Chen, Hehan Li, Ruizhi Zhao, Xin Lu, peizhi xu

TL;DR: 本文提出了NE-R1，一种基于强化学习的自适应检索增强命名实体识别框架。该框架通过一个’按需检索’机制，动态决定何时利用外部知识库，并采用两阶段训练方法进行优化。实验表明，NE-R1在多个基准测试中取得了最先进的性能。

Details

Motivation: 动机在于解决大语言模型在识别长尾和领域特定实体时，因参数化知识不足而面临的挑战。同时，传统检索增强生成方法在处理熟悉案例时会引入噪声和不必要的成本。

Result: NE-R1在领域内评估中平均F1分数提升了2.52%，在零样本跨领域评估中提升了1.18%，在多个基准测试上达到了最先进的性能水平。

Insight: 创新点在于设计了’按需检索’机制和两阶段训练方法，并构建了一个综合考虑准确性和检索收益的多维奖励函数，以实现参数化知识与外部知识之间的合理选择。从客观角度看，将强化学习与思维链结合用于优化检索决策是一个值得借鉴的思路。

Abstract: Named Entity Recognition (NER) has achieved substantial progress since the advent of large language models (LLMs). Nevertheless, the recognition of long-tail and domain-specific entities remains challenging due to the deficiency in parametric knowledge. Retrieval-augmented generation (RAG) offers a promising remedy by injecting external knowledge, but it also introduces noise and unnecessary cost when dealing with familiar cases. In this paper, we propose NE-R1, a novel framework for adaptive retrieval-augmented NER. We design a “retrieval-on-demand” mechanism for NER. Then we integrate it into models by a two-stage training method: (1) multi-task instruction tuning initialization; (2) end-to-end RL optimization with CoT. To achieve reasonable selection between parameterized and external knowledge, we design a multi-dimensional reward considering both accuracy and retrieval benefit. NE-R1 achieves state-of-the-art performance on various benchmarks, with an average F1 score gain of 2.52% in in-domain evaluation and 1.18% in zero-shot cross-domain evaluation.

[12] MultiGhostBench: A Multilingual Benchmark for Long-Form LLM-Generated Text Attribution under Distribution Shifts cs.CL | cs.AI | cs.CY | cs.DL | cs.IRPDF

Matteo Greco, Anudeex Shetty, Andrea Tagarelli, Jey Han Lau

TL;DR: 本文提出了MultiGhostBench，一个多语言基准测试，用于评估在分布偏移下长文本的LLM作者归属方法。该基准包含由五个最新LLM生成的928本书，涵盖六种语言和三种文字，平均每本书约5.9万字，支持领域、作者和语言偏移的评估。

Details

Motivation: 现有LLM作者归属基准存在局限，如主要关注英语、受控环境或较旧模型，且多语言研究仅考虑短文本，因此需要构建一个涵盖多语言、长文本且支持分布偏移评估的基准。

Result: 评估代表性作者归属方法发现，没有单一方法在所有设置下表现最佳，且性能在分布偏移下普遍下降；基于Transformer的检测器能跨语言保留生成器相关信息，但迁移效果因语言对而异，而基于统计和指纹的检测器更依赖语言。

Insight: 创新点在于构建了首个支持多语言、长文本和分布偏移评估的LLM生成文本归属基准；客观分析表明，该基准揭示了不同检测方法在跨语言和分布偏移下的鲁棒性差异，为开发稳健的作者归属方法提供了关键资源。

Abstract: While existing work on LLM authorship attribution (AA) has made progress, available benchmarks remain limited, often focusing on English, controlled settings, or relatively outdated models, with the few multilingual studies considering only relatively short texts. We introduce MultiGhostBench, a multilingual benchmark comprising 928 books generated by five recent LLMs across six languages and three scripts, with an average length of approximately 59K words per book. The benchmark supports evaluation under domain, author, and language shifts. Evaluation of representative AA methods shows that no single method consistently performs best across settings, and performance generally degrades under distribution shifts. Transformer-based detectors can retain generator-related information across languages, although transfer effectiveness varies by language pair, whereas statistical and fingerprint-based detectors are more language-dependent. We envision MultiGhostBench as a valuable resource for the development and evaluation of robust AA methods. The dataset and code can be found at https://github.com/GrecoMT/MultiGhostBench.

[13] When Decodability Is Not Enough: Logical Validity Representations, Behavioral Dissociation, and Causal Tests in Language Models cs.CL | cs.LGPDF

Smitha Muthya Sudheendra, Jaideep Srivastava

TL;DR: 该论文研究了五个开源Transformer语言模型在逻辑推理任务中的内部表征与行为表现之间的差异。研究发现，尽管模型在逻辑验证任务上的行为表现接近随机水平，但从隐藏状态中几乎可以完美解码出逻辑有效性信息，且该信息在未见过的模板、语义领域和推理家族中仍保持强可解码性。然而，干预实验表明这种表征与行为输出之间的因果联系较弱。

Details

Motivation: 动机在于探究大语言模型在逻辑推理任务中内部表征的真实性，仅凭正确或错误的答案无法揭示模型内部是否真正理解了逻辑有效性，因此需要更深入地分析隐藏状态中的信息。

Result: 在多个开源Transformer模型上，逻辑有效性在隐藏状态中的解码准确率接近完美，即使在行为表现错误的样本上也能高度解码；但通过留一法测试和干预实验发现，这种表征的泛化能力有限，且对行为输出的因果影响较弱。

Insight: 创新点在于区分了逻辑有效性的内部表征、行为表达和因果使用三个层面，揭示了模型可能具备逻辑有效性的内部知识但无法可靠地体现在输出中，这为评估模型推理能力提供了新视角。

Abstract: Large language models can look capable of logical reasoning, but correct or incorrect answers alone tell us little about what the model represents internally. We study logical verification in five open-weight transformer models using matched valid–invalid premise–claim pairs that vary across inference families, semantic domains, templates, and difficulty levels. Despite near-chance behavioral performance, logical validity is often almost perfectly decodable from hidden states and remains strongly decodable under held-out templates, domains, and inference families. Validity also remains highly decodable on behaviorally incorrect examples in the conditions where correctness-conditioned evaluation is well defined. At the same time, exhaustive leave-one-out tests reveal clear limits to this generalization, and interventions along probe-derived validity directions have only weak, nonspecific effects compared with random controls. Our results suggest that representing validity, expressing it in behavior, and using it causally are distinct. Validity related information can be strongly decodable from a model’s hidden states without being reliably expressed in its output.

[14] Predictors of Loneliness in Older Adults Using Multimodal Analysis of Speech and Language cs.CLPDF

Vinmay Khandode, Sai Karthik Kosuri, Neil K. R. Sehgal, Adam Greene, Elif Alpoge

TL;DR: 本研究通过分析310名老年人的半结构化电话访谈，探索了孤独感在语音和语言中的多模态标记。研究发现，较高的孤独感与否定词、负面语气和冲突相关语言相关，而较低的孤独感则与社会性提及、动机驱动和语音情感丰富度相关。多模态模型在预测孤独感方面优于纯文本或纯音频模型。

Details

Motivation: 孤独感是老年人中一个严重的公共健康问题，但目前缺乏可扩展的、客观的检测方法，尤其是在自然对话环境中。本研究旨在通过分析语音和语言特征，为孤独感的客观评估提供新方法。

Result: 研究在310名老年人的数据集上进行了分析，发现多模态模型（r=0.298）在预测自我报告的孤独感分数上优于纯文本模型和纯音频模型。孤独感与否定词（r=0.11）、负面语气（r=0.12）等特征正相关，与社会性提及（r=-0.18）、语音情感丰富度（r=-0.12）等特征负相关。

Insight: 创新点在于结合了语言学特征（如心理语言学词典、n-gram、主题模型）和声学特征（如音高、音调、响度）的多模态分析框架，揭示了孤独感在语言内容和语音表达中的具体模式。该方法可作为心理评估的辅助工具，而非独立的诊断工具。

Abstract: Loneliness is a critical public health issue among older adults, linked to higher risks of depression, cognitive decline, and mortality. Scalable, objective methods for its detection remain limited, particularly in natural conversational contexts. We analyzed speech and language markers of loneliness in 310 older adults using semi-structured telephone interviews to help understand how they process feeling lonely and how their language differs at different levels of feeling loneliness. Our multimodal framework combined linguistic features (psycholinguistic dictionaries, n-grams, and topic models) with acoustic features (pitch, tone, loudness) to examine associations with self-reported loneliness scores. Both predefined and data-driven methods captured patterns in verbal content and vocal delivery. Higher loneliness was associated with negations(r = 0.11), negative tone(r = 0.12), and conflict-related language. Lower loneliness was linked to social references(r = -0.18), motivational drives(r = -0.11), and emotional richness in speech(r = -0.12). We also found that the multimodal model (r = 0.298) outperforms the text-only and audio-only models. Findings suggest that loneliness manifests through both linguistic and acoustic cues, supporting the potential of speech-based analysis in psychological assessments and as an early indicator of emotional loneliness when used alongside existing assessments, rather than as standalone diagnostic tools.

[15] From Tokens to Semantics: Leveraging Complementary Signals for Hallucination Detection in Black-Box LLMs cs.CL | cs.AIPDF

Urja Pawar, Rajitha Ramanayake, Owen O’Neill, Nabeel Kemal, Abhishek Mandal

TL;DR: 该论文研究黑盒大语言模型中的幻觉检测问题，提出并评估了多种无需参考文档的检测方法，包括基于语义熵和词元不确定性的无监督方法（如TopK和CoCoA），以及两种有监督方法（Gated和Stacked）。

Details

Motivation: 当LLMs应用于面向公众或高风险场景时，漏检的幻觉会损害用户和机构，而误报则会消耗有限的人工审核资源。在没有可信上下文或参考文档的情况下，需要利用黑盒模型API可访问的信号进行检测。

Result: 在七个基准测试（包括五个公共基准和两个构建基准）上使用四个语言模型进行评估，Stacked方法在近半数情况下表现最佳，而TopK和CoCoA在无需监督训练标签的情况下也保持竞争力，但需要仔细校准阈值。

Insight: 创新点在于利用语义熵和词元不确定性的互补性，提出了结合两者的混合方法（如CoCoA），并设计了有监督的集成方法（如Stacked）。客观来看，论文系统地评估了不同方法在不同误报率预算和数据集特性下的表现，强调了方法选择的场景依赖性。

Abstract: When LLMs support public-facing or high-stakes workflows, missed fabrications can harm users and institutions, while false alarms consume limited human-review capacity. When no trusted context or reference document is available, we study two signals accessible through black-box model APIs: semantic entropy, which measures disagreement among sampled response meanings, and uncertainty derived from token log-probabilities. Their failure modes can be complementary: semantic entropy becomes uninformative when responses form one semantic cluster, while token uncertainty can miss consistently confident errors. We extend token-based uncertainty detection by aggregating token-level signals across sampled responses through our TopK method, evaluate the hybrid CoCoA method, which combines target-response uncertainty with semantic dissimilarity, and propose and study two supervised methods: Gated, which routes single-cluster cases to an aggregated-token-feature classifier, and Stacked, which learns jointly from semantic uncertainty and broader token features. We evaluate seven benchmarks, including five public benchmarks (four text datasets and multimodal handwritten-cheque extraction) and two constructed benchmarks (Financial Summaries and Long-Text QA), using four language models. In our evaluation across models and datasets, Stacked gave the best performance in nearly half of the cases, while TopK and CoCoA remain competitive without supervised training labels, although their thresholds require careful calibration. No method is universally strongest. We therefore evaluate performance at false-positive-rate budgets from 1% to 15%, assess their sensitivity to generation and calibration choices, and examine variation across dataset characteristics.

[16] Trace as State: Reasoning Traces as Conditional States for Long-Context Transformers cs.CLPDF

Xu Zou, Jie Tang

TL;DR: 本文提出了一种名为“Trace as State”的方法，用于改进Transformer模型在长上下文推理任务中的性能。该方法将收集到的推理轨迹作为任务状态的文本代理，并将其放置在长上下文块之前，使得先前推导出的信息能够指导模型重新阅读上下文。

Details

Motivation: Transformer模型以因果方式处理信息，但长上下文推理可能依赖于后续才被发现的任务状态，这种不匹配可能导致效率低下。本文通过条件状态更新任务形式化这一问题，并旨在探索如何利用推理轨迹优化长上下文处理。

Result: 在三个模型和三个长上下文数据集上的广泛实验表明，Trace as State在27个报告的组合中，有26个优于对照方法Trace Append。例如，在GraphWalks Parents数据集上，DeepSeek V4 Pro Preview的精确匹配率从初始的29.2%提升到81.8%，GLM-5.2从66.4%提升到100.0%。

Insight: 核心创新在于将推理轨迹作为条件状态前置，而非后置，从而在保留因果Transformer结构的同时，显著提升长上下文推理能力。这为解决因果处理与状态依赖之间的不匹配提供了一种简单而有效的文本代理策略。

Abstract: Transformers process information causally, but long-context reasoning may depend on task state discovered only later. We formalize this mismatch through conditional state update tasks. For causal state update processors, providing the condition first can require exponentially less memory in the worst case than providing it last. Motivated by this principle, we introduce Trace as State. We use collected reasoning traces as a textual proxy for task state and place it before the long-context block on a fresh pass, allowing information derived previously to guide rereading. We conduct extensive experiments on Trace as State and Trace Append, a matched control that uses the same task state proxy but put it after the context. Across three models and three long-context datasets, Trace as State outperforms Trace Append in 26 of 27 reported combinations of model, task, and metric. On GraphWalks Parents, exact match lifts DeepSeek V4 Pro Preview from 29.2% on the initial pass and 43.0% with Trace Appendto 81.8% with Trace as State, and from 66.4% and 83.2% to 100.0% for GLM-5.2. These results show that placing traces before the context can improve long-context reasoning while retaining the causal transformer structure.

[17] Untangling the Mechanisms of Misleading Context in Medical Question Answering cs.CL | cs.AI | cs.LGPDF

Robin Linzmayer, Noémie Elhadad

TL;DR: 该论文研究了大型语言模型在医学问答中面对误导性上下文时的脆弱性，通过在MedMisBench基准上注入伪造证据和武断断言两种误导线索，测试了三种推理模型的反应。研究发现模型对武断断言更为敏感，且误导线索在推理轨迹中的披露率远高于在最终回答中的披露率，而基于LLM的监控器在分析开放模型的推理轨迹时能更有效地检测出被污染的决策。

Details

Motivation: 尽管大语言模型在医学问答中已达到专家水平，但其决策所依赖的上下文可能具有误导性，从而损害模型的医学判断。本研究旨在深入理解误导性上下文如何腐蚀模型的判断机制。

Result: 在MedMisBench的医学推理子集（8,627个问题）上，三种测试模型对武断断言的采纳率比对伪造证据高10到27个百分点。误导线索在推理轨迹中的披露率为81%至98%，但在最终回答中仅为7%至90%。使用LLM监控器并辅以指导，在分析开放模型的推理轨迹时能以5%的误报率捕获78%的被污染决策，而仅分析回答时最高捕获率为32%。

Insight: 论文揭示了不同类型的误导性上下文（如证据与断言）以不同机制腐蚀推理过程：证据在早期进入并累积影响，而断言则在推理末期直接扭转结论。关键创新在于强调了开放推理轨迹对于理解和监控模型决策过程的重要性，而当前前沿模型往往不提供此类轨迹，这构成了安全风险。研究为构建更可靠、可监控的医学AI系统提供了重要见解。

Abstract: Large language models now answer medical questions with expert-level performance. However, the context these systems act on can be misleading, and misleading context can corrupt a model’s medical judgment. To understand how misleading context corrupts this judgment, we examine the model’s susceptibility to the context, disclosure of it, mechanism of corrupted reasoning, and monitorability of the decision. On the medical reasoning subset of MedMisBench, a clinician-reviewed question-answering benchmark of 8,627 questions, we inject two types of misleading context cues, fabricated evidence and a bare assertion. We test three reasoning models, two that expose their full reasoning trace and one frontier model that exposes only its response. All three are more susceptible to the assertion than to the fabricated evidence, adopting the asserted answer 10 to 27 points more often. The misleading cues are disclosed in 81 to 98% of traces but only 7 to 90% of responses, and the assertion is disclosed less often than evidence based cues. Resampling from reasoning traces without disclosure shows the two cues corrupt reasoning differently, evidence entering early and accumulating while the assertion redirects the conclusion near its end. An LLM monitor catches 78% of corrupted decisions at 5% false positives when reading an open model’s trace with guidance, against at most 32% from any response. The misleading context that models are most susceptible to is disclosed least, and was caught reliably only from an open reasoning trace, which frontier providers withhold.

[18] From Reweighting to Rewriting: Unlocking the Intervention Effects of Influential Samples in Training Data Attribution cs.CL | cs.AI | cs.LGPDF

Yuzhang Luo, Chenpeng Wang, Jianhui Chen, Liangming Pan

TL;DR: 本文提出了一种名为‘影响引导的响应重写’的新方法，用于训练数据归因（TDA）中的干预。该方法利用影响函数（IF）识别关键训练样本，并通过重写其响应（而非仅重新加权）来更有效地改变模型行为。实验表明，相比传统的基于重加权的干预，响应重写在多个开源大语言模型上能产生更强、更持久且双向的行为改变。

Details

Motivation: 训练数据归因旨在识别影响模型行为的训练样本，但其干预价值取决于样本选择与修改方式。现有基于影响函数的重加权干预效果有限，作者质疑是‘有影响的样本缺乏干预价值’还是‘重加权未能发挥其行为杠杆作用’，从而探索更有效的干预方法。

Result: 在四个开源大语言模型上，以认知性弃权为主要测试平台，响应重写相比对相同影响函数选择样本进行重加权，能产生更强、更持久和双向的行为偏移，而重加权效果微弱且不一致。影响函数选择的样本在重写中比替代选择器提供更大的杠杆，且改变集中在目标相关行为上，该定性对比也延伸至安全拒绝任务。

Insight: 创新点在于将干预方式从样本重加权扩展到响应内容重写，解锁了影响样本的更大行为杠杆。客观来看，该研究区分了影响函数估计的局部重加权效应与样本本身更广泛的干预潜力，强调了TDA方法需要结合干预感知的评估框架。

Abstract: Training data attribution (TDA) aims to identify training examples that shape model behavior, but its intervention value depends on both which examples are selected and how they are modified. Influence functions (IF) estimate behavioral changes under infinitesimal reweighting, yet IF-selected examples often show limited advantages over random selection under conventional weight-based interventions. This raises the question of whether influential examples lack intervention value or whether reweighting fails to realize their behavioral leverage.We introduce influence-guided response rewriting, which uses IF to identify intervention targets and replaces their responses with behavior-aligned or behavior-opposed supervision while keeping instructions fixed. Across four open-weight LLMs, we compare rewriting and reweighting on the same influence-selected examples using epistemic abstention as our primary testbed. Response rewriting produces stronger, more persistent, and bidirectional behavioral shifts, while reweighting the same examples yields weak and inconsistent effects. Further analyses show that influence-selected examples provide greater rewriting leverage than alternative selectors, with changes remaining concentrated on target-relevant behaviors. The same qualitative contrast extends to safety refusal. These results distinguish the local reweighting effects captured by influence estimates from the broader intervention leverage of the examples they identify, motivating intervention-aware evaluation of TDA methods.

[19] User Feedback Provides a Unique Signal that LLMs Can not Detect cs.CLPDF

Shachar Don-Yehiya, Leshem Choshen, Omri Abend

TL;DR: 这篇论文挑战了用户反馈对大型语言模型（LLM）改进无效的普遍观点，认为其感知上的无效源于当前评估范式的系统性偏差。通过合成数据和自然数据实验，论文证明基于反馈的模型修订能更有效地解决特定问题，并揭示了LLM评估者本身存在偏见，经常无法识别真正被反馈修正的回复。

Details

Motivation: 动机是重新评估用户反馈作为LLM学习信号的价值，反驳其因噪声大而难以有效利用的现有观点，并探究当前评估方法可能存在的偏差。

Result: 在合成和自然数据上的实验结果表明，基于反馈的模型修订在解决目标问题上的成功率显著高于基线修订。然而，LLM评估者存在系统性偏差，经常错误地偏好较差的基线输出，而非真正被反馈修正的回复。

Insight: 创新点在于揭示了用户反馈本身是高度可操作的改进信号，其无效性主要源于LLM评估者的判断偏差，而非信号质量本身。这提示需要重新设计评估协议以准确衡量反馈的价值。

Abstract: Harnessing naturally occurring feedback from user interactions offers a promising learning signal for Large Language Models (LLMs). However, recent studies suggest this feedback is inherently noisy and difficult to leverage effectively. We challenge this conception by demonstrating that user feedback is a highly actionable signal for improvement, and that its perceived ineffectiveness stems from a systematic bias in current evaluation paradigms. To isolate the usefulness of feedback, we construct synthetic data with a definitive ground truth, alongside naturalistic data to validate that our findings hold in real-world scenarios. By comparing model revisions generated with and without access to feedback across both settings, we show that feedback-informed revisions resolve targeted issues at significantly higher rates than baseline revisions. Finally, we expose the root of the evaluation bias: when a model successfully fixes an issue exclusively due to feedback, LLM judges frequently fail to identify the genuinely corrected response, systematically preferring inferior baseline outputs instead.

cs.CV [Back]

[20] Beyond Textual Chain-of-Thought: A Survey on Action-Grounded Reasoning in Autonomous Driving cs.CV | cs.CL | cs.ROPDF

Zhengxu Tang, Xiaozhou Zhang, Guofeng Cui, Ziyu Gong, Zi Wang

TL;DR: 这篇综述论文探讨了自动驾驶领域中从文本链式思维（CoT）推理向动作接地推理的转变，提出了一个以中间状态表示形式为核心的组织分类法，并对130篇方法论文进行了系统梳理。

Details

Motivation: 动机在于认识到自动驾驶的决策输出是连续动作，其推理过程必须与物理世界的时空结构保持一致，因此需要研究如何将传统的文本推理范式转变为与动作和环境紧密关联的推理模式。

Result: 论文通过综述171篇文献，将130种方法系统化为四大类别（语言基、视觉空间、潜在动态、外化推理）和13个子类型，并指出该领域的前沿在于开发能够与现实世界接地、与实时动作耦合、并在安全关键系统中可验证的中间表示。

Insight: 创新点在于提出了一个以中间表示形式为中心的、系统化的分类框架，清晰地勾勒了自动驾驶推理研究从文本抽象到动作接地的范式转变路径及其核心挑战，为未来研究提供了结构化的指导方向。

Abstract: Chain-of-thought (CoT) reasoning powers generative models by eliciting intermediate steps before producing an answer. In autonomous driving, the answer is a continuous action. Thus its reasoning must share the same spatiotemporal structure as the physical world. This survey studies the resulting shift from textual CoT to action-grounded reasoning. Surveying 171 papers, including 130 method papers and 41 benchmarks, datasets, surveys, and analysis papers, we propose a representation-centered taxonomy that treats the form of the intermediate state as the organizing axis. We systematize the 130 methods into four categories: language-based, visual-spatial, latent-dynamic, and externalized reasoning, further divided into 13 subtypes tied to distinct regions of interests. Our synthesis shows that the open frontier of reasoning in driving agents lies in intermediate representations that can be grounded in the real world, coupled to real-time action, and verified under safety-critical systems. Project page: https://github.com/tangzhengxu/awesome-av-cot.

[21] FORGE: Forward-Only Test-Time Adaptation for Integer-Only Vision Models on Microcontrollers cs.CV | cs.AR | cs.LGPDF

Muhammad Rehan, Haider Ali, Muhammad Ali Munir, Moaz Amjad

TL;DR: 本文提出了一种名为FORGE的前向测试时适应方法，专为部署在微控制器上的整数量化视觉模型设计。该方法通过重新归一化已折叠的卷积层输出，仅使用前向传播估计，在无需反向传播的情况下适应分布偏移，并在真实微控制器上验证了其低能耗特性。

Details

Motivation: 解决微控制器上部署的整数量化视觉模型在推理运行时无法使用反向传播进行测试时适应的问题，现有方法要么不适用于整数执行，要么依赖已被融合的批归一化层。

Result: 在三个数据集和两种架构上验证，FORGE恢复了梯度基TENT方法大部分准确率增益（+20.9 vs. +24.9点），匹配前向BN适应，且是唯一能在折叠整数模型上运行的方法；在ESP32-S3上部署时，适应能耗仅占推理总能耗的6.8%。

Insight: 创新点在于通过重新归一化已折叠卷积的每通道输出到其干净训练统计量，实现了对整数量化模型的前向适应，无需批归一化层或反向传播，且仅需适应少数层即可获得大部分收益，适用于资源受限的微控制器部署。

Abstract: Vision models deployed on microcontrollers (MCUs) are quantized to integer-only arithmetic and run in inference-only runtimes that do not carry the machinery backpropagation needs: the standard tool for adapting a model to the distribution shift (sensor noise, blur, lighting) it meets in the field. Existing forward-only test-time adaptation (TTA) methods either run only on server- or edge-GPU-class models (not true microcontroller integer execution), or require the batch-normalization (BN) layers that integer deployment fuses away. We present a forward-only TTA method that operates on deployed, BN-folded, integer-only convolutional networks. The key observation is that fusing BN into the preceding convolution, a mandatory step for integer inference, destroys the statistics that normalization-based adaptation relies on. We restore adaptation by re-normalizing each folded convolution’s per-channel output to its clean training statistics, using only forward-pass estimates. The method (i) recovers most of gradient-based TENT’s accuracy gain (+20.9 vs. +24.9 points) and matches forward-only BN adaptation, while being the only method that runs on a folded integer-only model; (ii) needs to adapt only 3 of 21 layers (selected without seeing the test corruptions) to recover 93% of the benefit; (iii) survives single-sample streaming with a batch-size-scaled momentum; and (iv) generalizes across three datasets (up to 200 classes) and two architectures. We validate bit-exact int8 convolution execution and deploy on an ESP32-S3, where, measured with a Nordic PPK2 power profiler, the forward-only adaptation (a lightweight fp32 recalibration around the int8 convolutions) costs only 8.3 mJ (6.8% of inference energy) and 21.9 ms on the deployed SIMD-optimized model: forward-only adaptation is cheap on a real microcontroller.

[22] FairLens: Benchmarking Fairness in Vision-Language Models for High-Stakes Decision-Making cs.CV | cs.LGPDF

Vahid Reza Khazaie, Ahmed Y. Radwan, Shaina Raza

TL;DR: 本文提出了FAIRLENS基准测试框架，用于评估视觉语言模型在招聘、法律和医疗等高风险决策领域的公平性和有效性。该基准包含超过10万个涵盖性别、种族和年龄的真实人脸图像与问题对，并从四个互补视角评估模型响应。研究发现，模型的主要失败在于进行无根据的推断而非不平等对待。

Details

Motivation: 随着视觉语言模型越来越多地用于基于视觉输入的高风险决策，现有评估缺乏对模型公平性和响应有效性的系统性衡量。本文旨在填补这一空白，重点关注模型在证据不足时是否应避免做出推断。

Result: 在评估的八个VLMs中，最弱的模型在99%无法回答的问题上进行了无根据推断。公平性差距绝对值虽小，但由于基线不良率低，可能导致某一人口群体收到不良标签的频率是另一群体的数倍。自由文本生成中的偏见与多项选择准确性关联性弱。

Insight: 创新点在于提出了一个综合评估框架，将公平性（如人口统计奇偶性）与有效性（如合理性）相结合，并强调在高风险领域，模型拒绝从外貌推断属性与实现群体间公平处理同等重要。该问题集可迁移至任何带人口统计注释的人脸数据集。

Abstract: Vision-language models (VLMs) are increasingly used to make decisions from visual inputs. We introduce FAIRLENS, a benchmark and evaluation framework for measuring both the fairness and the validity of VLM responses in three high-stakes domains: hiring, legal, and healthcare. FAIRLENS pairs real face images spanning gender, race, and age groups with closed- and open-ended questions, giving more than 100K image-question pairs per model, and evaluates responses from four complementary views: demographic parity over adverse outcome rates, soundness, demographic association over unsupported roles and statuses, and bias in free-text generation. Soundness is the central validity criterion: a response is sound when it follows the evidence stated in the question and abstains when the image cannot support an answer. Evaluating eight VLMs, we find that the primary failure is unwarranted inference rather than unequal treatment. Models routinely infer qualifications, threat, illness, or professional role from a face instead of abstaining, and the weakest model does so on 99% of the questions its input cannot answer. These failures are most severe in legal and healthcare, where recognizing insufficient evidence matters most, and disparity metrics alone would miss them: parity gaps are small in absolute terms, yet when baseline adverse rates are low the same gap means one demographic group receives adverse labels several times as often as another, and a small gap can equally reflect a model that treats every group unsafely. Bias in free-text responses is only loosely coupled to multiple-choice accuracy, so correct structured answers do not imply safe generation. FAIRLENS shows that fair high-stakes VLM behavior requires similar treatment across groups and refusal to infer high-stakes attributes from appearance, and its question suite transfers to any face corpus with demographic annotations.

[23] From Visual Cues to Spoken Narration: Rethinking Audio Description cs.CVPDF

Akshita Gupta, Aditya Arora, Federico Tombari, Marcus Rohrbach, Anna Rohrbach

TL;DR: 本文提出了Cue2Narrate，一个用于长视频音频描述（AD）生成的两阶段框架，旨在联合解决’描述什么’和’何时描述’的问题。该方法首先通过一个双头视听定位器预测视觉线索窗口和叙述窗口，然后利用LoRA适配的视觉语言模型生成简洁的AD。作者还为此任务引入了新的基准LongLSMDC。

Details

Motivation: 现有工作将音频描述问题简化为对预分割视频片段的视频字幕生成，忽略了’何时描述’这一关键决策，并且’描述什么’也很大程度上被预先定义。本文旨在重新思考并联合解决长视频中’描述什么’和’何时描述’这两个核心问题，以提升视障用户的体验。

Result: 在提出的LongLSMDC基准（包含平均约6.5分钟的长电影片段）上，Cue2Narrate在平均mAP上比仅使用视频或音频的定位基线高出5-12个点。无论在预测窗口还是真实窗口的评估下，该方法都优于相应的微调基础视觉语言模型，为长视频多片段AD生成了首个基准。

Insight: 论文的核心创新在于将AD视为一个联合的’什么’与’何时’的决策问题，并提出了一个包含视听定位和描述生成的两阶段框架。具体技术亮点包括：用于预测两个不同时间窗口（视觉线索窗和叙述窗）的双头定位器，以及使用描述排序损失来训练LoRA适配的VLM，以区分真实AD和同一帧的普通字幕（负样本）。

Abstract: Audio Description (AD) provides spoken narration of visual events during dialogue gaps, making movies accessible to visually impaired audiences. The problem requires determining both what (which visual event) and when (position for inserting the AD) to narrate, to achieve the best user experience. Prior work has largely reduced the problem to video captioning of pre-segmented video clips, i.e., what is largely predefined and when is ignored entirely. We propose Cue2Narrate, a two-stage pipeline that jointly predicts what and when to narrate in longer untrimmed movie clips. A dual-head audio-visual localizer predicts two temporally distinct windows per AD utterance: a visual cue window and a spoken narration window. A LoRA-adapted VLM then generates concise ADs from the predicted visual evidence, trained with a Description Ranking Loss that ranks captions (negative samples) of the same frames lower than the GT AD. To benchmark this new problem statement, we introduce the LongLSMDC benchmark with up to 8-min movie clips (~6.5min on average). On LongLSMDC, Cue2Narrate outperforms video-only and audio-only localization baselines by 5–12 points in avg. mAP. Under both predicted- and GT-window evaluation, Cue2Narrate improves AD generation over the corresponding fine-tuned base VLM. These results establish the first benchmark for multi-segment AD generation on long-form clips. Data & Code: https://github.com/multimodal-ai-lab/Cue2Narrate

Dmitry Golovchits, Seyed Sahand Mohammadi Ziabari, Ali Mohammed Mansoor Alsahag

TL;DR: 本研究评估了证据深度学习（EDL）在多模态反无人机检测中的应用。通过在三项基准测试（AntiUAV600、TRIDENT、MM-UAV）上进行对照消融实验，发现EDL训练目标能显著提升检测精度和错误排序能力，但其他组件（如Dempster-Shafer证据融合、不确定性驱动的时间门控）未能带来额外收益。

Details

Motivation: 当前反无人机系统虽然融合了多种传感器，但其检测头无法提供每个模态的可靠性信号。本研究旨在探索EDL、证据融合和不确定性门控是否能改善多模态反无人机检测的可靠性和性能。

Result: 在AntiUAV600热成像跟踪基准上，EDL相比重训练的Sigmoid基线将准确率提升了5.9个百分点，并将跟踪器在目标缺失时的运行率提高了三倍；在TRIDENT RGB-音频-RF分类基准上，准确率提升了4.8个百分点（p=0.011），且分类错误排序能力显著优于基线（熵UAUC约0.94 vs. 0.51）。然而，Dempster-Shafer融合未超越简单概率平均，时间门控仅在几乎不激活时保持精度，且未实现实际延迟节省。

Insight: 论文的核心创新点在于系统性地评估了EDL在多模态检测中的组件效用，并发现其收益主要源于训练目标而非不确定性估计本身。一个关键洞察是，在检测层面，由于极端背景不平衡，Dirichlet空度等不确定性度量可能编码类别成员而非错误可能性，这一失败在熵和Sigmoid置信度中也被观察到，这揭示了当前不确定性估计方法在特定任务中的局限性。

Abstract: Anti-UAV systems increasingly fuse multiple sensors, yet their detection heads provide no per-modality reliability signal. This study evaluates whether evidential deep learning (EDL) heads, Dempster-Shafer (DS) evidence fusion, and uncertainty-driven temporal sensor gating improve anti-UAV detection through a controlled ablation on three benchmarks: thermal tracking (AntiUAV600), RGB-audio-RF classification (TRIDENT), and RGB-IR tracking (MM-UAV). The EDL training objective improves accuracy over retrained sigmoid baselines (+5.9 percentage points in accuracy and a tripled tracker-on-absent rate in E1; +4.8 percentage points in classification accuracy in E2, surviving a clip-clustered bootstrap, p = 0.011) and ranks classification errors substantially better (entropy UAUC approximately 0.94 vs. 0.51). The remaining components do not support their respective hypotheses. DS fusion does not outperform simple probability averaging. Dirichlet vacuity adds no ranking power beyond predictive entropy and inverts at the detection level, where extreme background imbalance causes it to encode class membership rather than error likelihood, a failure also observed for entropy and sigmoid confidence. Temporal gating preserves accuracy only when nearly inactive and yields no realised latency saving on shared-backbone hardware. The benefit of evidential learning therefore arises primarily from its training objective rather than its uncertainty estimate; a crop-level control further localises the detection-level breakdown to anchor-level evaluation rather than the learned representation.

[25] SCULPT: Training Edge Vision Models for Post-Training Quantization Readiness cs.CVPDF

Bharadwaj Kavuri, Sourav Babu-PK, Varadhraj Ellapan, Pullarao Maddu, Prasad Deshpande

TL;DR: 本文提出了一种名为SCULPT的训练时方法，旨在通过统计裁剪和均匀损失，在标准的FP32微调过程中提升边缘视觉模型对训练后量化（PTQ）的适应性。该方法结合了拓扑感知的激活正则化器来抑制不利于量化的偏度和峰度，以及一个稳定的基于百分比的裁剪机制来学习部署就绪的激活边界。

Details

Motivation: 边缘视觉模型在资源受限的硬件上部署困难，而低比特训练后量化（PTQ）具有吸引力。然而，标准的FP32训练通常会产生重尾的激活分布，其异常值会破坏激活量化：保留全范围会浪费量化区间在罕见的极端值上，而激进的裁剪则会导致信息丢失。现有解决方案通常依赖于量化感知训练（QAT）或复杂的PTQ后处理程序，前者增加了训练复杂性和比特宽度耦合，后者则在训练后修复模型。

Result: SCULPT方法在训练过程中学习到的裁剪边界可以直接导出到标准的PTQ工作流中，用于低比特部署，包括INT8以及W4A8等更低比特的设置。

Insight: SCULPT的创新点在于，它不同于QAT，在优化过程中不模拟量化；也不同于事后修复异常值的PTQ方法，它不需要运行时激活变换。它通过在训练时直接改善模型对量化的固有适应性，简化了低比特部署流程，是一种新颖的训练时PTQ准备方法。

Abstract: Edge vision models are difficult to deploy on resource-constrained hardware, making low-bit post-training quantization (PTQ) attractive. In practice, standard FP32 training often produces heavy-tailed activation distributions whose outliers destabilize activation quantization: preserving the full range wastes quantization bins on rare extremes, while aggressive clipping causes information loss. Existing solutions typically rely on quantization-aware training (QAT), which adds training complexity and bit-width coupling, or advanced PTQ procedures that repair the model after training. We present SCULPT (Statistical Clipping and Uniform Loss for Post-Training), a training-time method that improves PTQ readiness during ordinary FP32 fine-tuning. SCULPT combines a topology-aware activation regularizer that suppresses quantization-hostile skewness and kurtosis with a stable percentile-based clipping mechanism that learns deployment-ready activation bounds. Unlike QAT, SCULPT does not simulate quantization during optimization; unlike post hoc outlier-repair PTQ methods, it does not require runtime activation transformations. The learned clipping bounds can be exported directly into a standard PTQ workflow for low-bit deployment, including INT8 and lower-bit settings such as W4A8.

[26] Swin Meets EfficientNet: Lightweight Architectures for GAN-Based Face Forensics cs.CV | cs.AIPDF

Sejuti Basu, Ashima Sood, Vijay Kumar, Sahil Sharma

TL;DR: 本文提出了一种轻量级架构，用于检测GAN生成的人脸伪造图像。通过结合Swin Transformer和EfficientNet-B0，构建了一种混合模型，在140K Real and Fake Faces数据集上实现了高效的检测性能。

Details

Motivation: 针对GAN生成的人脸图像难以与真实照片区分的问题，现有检测方法要么依赖CNN（擅长局部纹理特征但缺乏全局上下文理解），要么使用Vision Transformer（能捕获长程结构但计算成本高），需要一种更高效、轻量级的检测方案。

Result: 在140K Real and Fake Faces数据集（包含StyleGAN生成的伪造人脸和真实图像）上评估，提出的EfficientNetB0+Swin混合模型在5000张测试图像上达到99%的准确率和99.44%的召回率，优于纯Swin变体和之前的纯CNN基线，在该数据集上达到SOTA水平。

Insight: 创新点在于将分层CNN特征（EfficientNet-B0）与移位窗口自注意力机制（Swin Transformer）相结合，提供了一种计算轻量且高效的检测方法；从客观角度看，这种混合架构有效融合了局部纹理敏感性和全局结构理解能力，适用于资源受限的伪造检测场景。

Abstract: Modern generative models, such as GANs, diffusion architectures, and autoregressive systems, now produce facial images that are nearly indistinguishable from authentic photographs. This capability makes detecting forged images increasingly difficult, raising serious concerns about identity theft, fraud, and misinformation campaigns. Our research focuses specifically on GAN-generated synthetic faces, which underpin many face-centric deepfakes, and investigates efficient detection approaches using image analysis alone. Existing detection systems rely heavily on either convolutional neural networks (CNNs) or global vision transformers. While CNNs excel at identifying texture-based local features, they struggle with broader contextual understanding. Traditional Vision Transformer (ViT) models can capture long-range structures effectively, but demand substantial computational resources. Our work explores Swin-Transformer-based architectures across three implementations: a compact Swin Transformer trained from the ground up, ImageNet-1K pre-trained Swin-Tiny and Swin-Small models adapted for binary classification, and a novel hybrid combining EfficientNet-B0’s convolutional processing with a Swin Transformer backend. We evaluated all models using the 140K Real and Fake Faces dataset, which includes StyleGAN-generated fake faces alongside authentic images from Flickr and DFDC, with balanced splits for training, validation, and testing. The EfficientNetB0+Swin hybrid achieved 99% accuracy and a 99.44% recall on 5,000 test images, outperforming both pure Swin variants and a previous CNN-only baseline on this dataset. Our results suggest that combining hierarchical CNN features with shifted-window self-attention provides an efficient and computationally lightweight method for detecting GAN-generated synthetic faces.

[27] AlphaRAD: Grounded Zero-Shot Classification in Chest Radiology via $α$-Corrected Binary Cross Entropy and Factorized Latent Supervision cs.CVPDF

Jianzhong You, Yuan Gao, Chris McIntosh

TL;DR: AlphaRAD提出了一种用于胸部放射学零样本分类的视觉语言预训练模型，通过构建结构化医学概念空间和设计因子化潜在监督模块，解决了对比学习中的批内噪声和跨模态融合的复杂性挑战，在多个分类、定位和分割任务上实现了SOTA性能。

Details

Motivation: 针对现有视觉语言预训练模型在胸部放射学理解中，结构化临床语义利用不足导致对比学习噪声大，以及跨模态融合设计复杂且空间定位不准确的问题，旨在提升模型的零样本泛化能力和定位忠实度。

Result: 在16个分类基准测试中取得了SOTA平均性能，并在7个定位/短语定位数据集和3个分割数据集上分别实现了SOTA结果，展示了强大的零样本泛化能力。

Insight: 创新点包括：利用大语言模型解析医学报告构建结构化概念空间，通过α校正二元交叉熵自然训练医学概念判别器；提出因子化潜在监督模块，将表示分解为独立子空间进行对齐监督，增强空间定位表达而不增加参数。

Abstract: Vision-Language Pretrained Models (VLPMs) offer a scalable path to open-vocabulary chest radiology understanding, yet two aspects remain underexplored: how structured clinical semantics extracted from medical reports can reduce in-batch noise during contrastive learning, and how cross-modal fusion can be designed to produce more faithful spatial grounding without added complexity. We introduce AlphaRAD, addressing these opportunities through two contributions. First, we construct a large-scale structured medical concept space from medical reports parsed by a Large Language Model for training, thereby mitigating in-batch learning noise and removing heuristic pair matching in contrastive learning, and thus naturally positioning AlphaRAD as a medical concept discriminator trained via $α$-Corrected Binary Cross-Entropy. Second, we propose FLaS (Factorized Latent Supervision), an extremely simple yet effective cross-modal feature fusion module that factorizes VLPM representations into independent subspaces, using dedicated alignment supervision to enhance the expressiveness of spatial grounding without introducing additional model parameters. Through extensive empirical validation, AlphaRAD shows strong zero-shot generalization across diverse chest radiology tasks. Notably, it establishes state-of-the-art average performance across 16 classification benchmarks, while achieving individual state-of-the-art results via distinct gains on 7 grounding/phrase grounding and 3 segmentation datasets.

[28] Allocate Before You Embed: Adaptive Visual Input Allocation for Video Embeddings cs.CVPDF

Song Jin, Zhongtao Jiang, Chenglei Shen, Huanxuan Liao, Haozhe Chi

TL;DR: 本文提出AllocEmbed框架，通过先分配后嵌入的策略优化视频检索中的视觉输入预算分配。该方法利用轻量级分配器根据预览信息动态分配帧分辨率，在固定视觉输入预算下增加时间覆盖并保持关键帧的空间保真度，从而提升检索性能。

Details

Motivation: 现有视频检索方法通常采样固定数量原始分辨率的帧，限制了时间覆盖并忽略了帧的重要性，无法在有限的视觉输入和推理预算下有效编码长且多样的视频。

Result: 在MMEB-V2 V-QA和V-RET任务以及LongRet基准测试中，AllocEmbed在预算匹配方法中实现了最佳的整体检索性能，并能跨嵌入骨干网络迁移。

Insight: 创新点包括提出分配-嵌入框架，在嵌入骨干前重新分配视觉输入预算以平衡时间覆盖和空间保真度；引入检索驱动的策略优化（RDPO），直接利用检索反馈学习分配器，无需修改现有嵌入模型或下游流程。

Abstract: Large-scale video retrieval requires embedding models to encode long and diverse videos under tight visual-input and inference budgets. Existing methods typically sample a small, fixed set of frames at their original resolution, limiting temporal coverage and ignoring frame importance. Our empirical analysis shows that expanding temporal coverage improves retrieval even under a fixed visual-input budget. Gains are larger when the original per-frame resolution is preserved, highlighting the complementary roles of temporal coverage and spatial fidelity. Motivated by this finding, we propose AllocEmbed, an allocate-then-embed framework that reallocates a fixed visual-input budget across more frames. A lightweight allocator uses low-cost previews to assign frame-wise resolutions before the embedding backbone, preserving more detail where it most benefits retrieval while reducing visual cost elsewhere. We further introduce Retrieval-Driven Policy Optimization (RDPO), which learns the allocator directly from retrieval feedback using a rank-validated similarity gap and a confidence-guided efficiency incentive. Operating entirely before the backbone, AllocEmbed integrates with existing retrieval systems without modifying the embedding model or downstream pipeline. Experiments on the MMEB-V2 V-QA and V-RET tasks and our LongRet benchmark show that AllocEmbed achieves the best overall retrieval performance among the evaluated budget-matched methods and transfers across embedding backbones. Our code is publicly available at https://github.com/jinsong8/AllocEmbed.

[29] Ten Architectures, One Error: Shared Failure Modes in Hyperspectral Classification under Spatially Disjoint Evaluation cs.CV | cs.LGPDF

Ehsan Faghih, Fatemeh Ashrafi, Marguerite Moore, Zahra Saki

TL;DR: 该论文揭示了高光谱图像分类中广泛使用的随机像素划分方法存在数据泄漏问题，导致模型性能被高估。作者提出了一种基于空间分离的无泄漏评估协议，并在十种不同架构上验证了该协议的有效性，发现模型性能显著下降且排名发生变化。

Details

Motivation: 高光谱图像分类领域普遍采用随机像素划分来评估模型，但这种方法导致测试像素与训练像素相邻，造成数据泄漏，从而夸大了模型性能。

Result: 应用无泄漏评估协议后，十种架构的平均Macro-F1分数下降了0.147，模型排名变化多达五位；同时，该协议限制了特定基准测试中可评估的架构范围。

Insight: 创新点在于提出了基于空间分离的无泄漏评估协议，并强调在报告结果时需同时考虑模型感受野和分区半径以确保公平比较；此外，研究发现所有架构都错误分类了相同的像素，表明数据中存在未解决的谱间模糊性问题。

Abstract: Hyperspectral image classification still relies heavily on random pixel splits within a single scene. The Salinas dataset, randomly split, is among the most widely used datasets for comparing different architectures. However, under a random split method, a large fraction of test pixels fall immediately adjacent to a training pixel, which inflates reported accuracy. This work introduces a leakage-free evaluation protocol linking spatial separation to the model’s receptive field. Applying this protocol across ten different architectures, including classical, spectral, spectral-spatial, transformer, vision-backbone, and state-space families, shows that Macro-F1 drops by 0.147 on average and model rankings change by as many as five places. Furthermore, leakage-free evaluation limits which architectures can be tested on a given benchmark. Since each partition supports patches only within a finite radius, reporting this radius alongside the receptive field is essential for fair comparison. In addition, this study reveals that all ten architectures misclassify largely the same pixels, pointing to a spectral ambiguity in the data that none of them resolves.

[30] CoViT: Instance-Correspondence Contrastive Learning for Vision Transformer cs.CVPDF

Yisen Wang, Zhirong Wu, Limin Wang

TL;DR: 本文提出了CoViT，一种自监督学习框架，旨在为Vision Transformer注入实例感知能力。该方法通过几何引导的对比学习，利用ViT自身的注意力图生成实例掩码，并构建包含最难正负样本的三元组进行对比学习，从而压缩类内方差并扩大类间间隔。

Details

Motivation: Vision Transformers在语义理解方面表现出色，但缺乏区分不同物体实例的能力（例如，为两只狗生成相同的嵌入），这限制了其在目标检测和实例分割等实例级任务中的应用。

Result: 在多个实例级感知任务中，使用ViT作为骨干架构时，CoViT持续实现了超过2个AP点的稳定性能提升。

Insight: 创新点在于利用ViT固有的注意力先验，通过自适应阈值和形态学操作精炼注意力图来生成实例掩码，并设计了“最难对比挖掘”策略来构建对比学习三元组。这表明纯ViT无需额外解码器或标签，即可通过针对性对比约束学习实例感知表示。

Abstract: Vision Transformers (ViT) excel in semantic understanding but fail to discriminate between object instances (e.g., identical embeddings for two dogs), limiting their use in instance-level tasks such as object detection and instance segmentation. We propose Contrastive Vision Transformer (CoViT), a self-supervised learning framework that injects instance-awareness into ViT through geometry-guided contrastive learning. CoViT uniquely coordinates ViT’s attention maps and embeddings by constructing triplets: (1) Attention-guided masking: Refine multi-head attention via adaptive thresholding and morphological operations to generate instance masks, identifying foreground anchors; (2) Hardest contrastive mining: For each anchor, computing pairwise embedding similarities to select the intra-instance hardest positive (least similar patch within its mask) and inter-instance hardest negative (most similar patch from other instances), with intra-instance regions masked during negative search. These triplets drive a contrastive loss that simultaneously compresses intra-instance variance and expands inter-instance margins, forcing ViT to discern subtle geometric and appearance differences between instances. CoViT consistently achieves stable performance gains of over 2 AP points across multiple instance-level perception tasks by using ViT as backbone architecture. Notably, CoViT requires no extra decoders or labels, demonstrating that a pure ViT can learn instance-aware representations via inherent attention priors and targeted contrastive constraints. Code and models will be released.

[31] Improved Automatic Target Recognition in Synthetic Aperture Sonar Imagery Using Large Deep Neural Networks cs.CVPDF

C. J. Moore, Alex Hurt, Jordan Malof

TL;DR: 本文系统比较了现代CNN和Transformer架构在合成孔径声纳图像自动目标识别任务中的性能，研究了网络规模、预训练方法、数据增强和正则化等因素的影响，旨在提供训练高性能SAS-ATR模型的路线图。

Details

Motivation: 解决合成孔径声纳自动目标识别中Transformer架构应用不足、训练数据稀缺导致性能受限的问题，并探索不同架构和训练配置对性能的影响。

Result: 研究通过系统实验对比了CNN和Transformer架构的性能，重点关注了不同配置下的识别准确率，旨在确定能实现最高性能的模型训练方案。

Insight: 创新点在于首次系统评估Transformer在SAS-ATR任务中的潜力，并提供了涵盖网络规模、预训练和数据增强的完整性能优化路线图，为领域内模型选择提供了实证依据。

Abstract: Automatic Target Recognition (ATR) in Synthetic Aperture Sonar (SAS) is a task largely dominated by deep neural networks (DNNs). Most SAS-ATR models use convolutional neural network (CNN) architectures whereas transformer-based architectures have had much less representation in the literature despite being state of the art in general computer vision (CV) research. Additionally, researchers have had mixed results in attempting to overcome challenges presented by a scarcity of labeled training data by using methods such as data augmentation and the use of pretrained weights from a variety of imaging modalities. In this work, we compare the performance of modern CNN and transformer-based DNNs to determine which architecture and training configurations elicit the highest performance in SAS-ATR. We investigate how network size, architecture, pretraining method, data augmentation and other forms of regularization affect SAS-ATR performance with a focus on producing the highest-performing model and providing a roadmap for training state-of-the-art SAS-ATR models.

[32] DESA-TTA: Dynamic EMA and Source Anchoring for Test-Time Adaptation cs.CVPDF

Atif Belal, Lilian Hollard, Marco Pedersoli, Eric Granger

TL;DR: 该论文提出了一种名为DESA-TTA的低开销测试时自适应方法，用于提升视觉语言目标检测器在分布偏移下的鲁棒性。该方法通过动态时间平均和源锚定技术，联合调控教师模型的更新和学生模型的漂移，从而克服了传统均值教师方法对固定EMA系数的敏感性以及噪声伪标签导致的累积漂移问题。

Details

Motivation: 视觉语言目标检测器在零样本场景下表现良好，但在部署时对分布偏移敏感。传统的均值教师测试时自适应方法存在两个主要问题：对固定指数移动平均系数高度敏感，以及使用噪声伪标签重复优化会导致学生模型发生累积漂移。

Result: 在多种分布偏移和两种VLOD架构上的实验表明，DESA-TTA相比现有TTA方法取得了持续改进。在VOC-C数据集上，DESA-TTA将AP50比零样本推理提升了14.5个百分点，同时推理吞吐量比YOLO-World的先前SOTA TTA方法高出55%。

Insight: 创新点在于提出了动态时间平均（根据伪标签置信度和边界框密度估计教师不确定性，并据此在教师参数漂移确定的边界内选择样本级EMA系数）和源锚定（根据学生漂移程度，部分地将更新后的学生参数恢复至预训练值）的联合机制，以低开销方式有效调控模型更新和防止漂移。

Abstract: Vision-language object detectors (VLODs) achieve strong zero-shot performance but remain vulnerable to distribution shifts during deployment. Mean-teacher methods for test-time adaptation (TTA) can improve robustness by updating a student model using teacher-generated pseudo-labels. However, mean-teacher TTA is highly sensitive to the choice of a fixed exponential moving average (EMA) coefficient for teacher updates, and repeated optimization with noisy pseudo-labels can cause cumulative student drift. We propose Dynamic EMA and Source Anchoring for TTA (DESA-TTA), a low-overhead method that jointly regulates teacher updates and student drift through dynamic temporal averaging and source anchoring. Dynamic temporal averaging estimates teacher uncertainty from pseudo-label confidence and box density and uses it to select a sample-wise EMA coefficient within bounds determined by teacher parameter drift. Source anchoring partially restores the updated student parameters toward their pretrained values, with the anchoring strength increasing according to student drift. Experiments across diverse distribution shifts and two VLOD architectures show consistent improvements over existing TTA methods. On VOC-C, DESA-TTA improves AP$_{50}$ by 14.5 points over zero-shot inference while achieving 55% higher inference throughput than the previous state-of-the-art TTA method for YOLO-World. Our code: https://github.com/imatif17/DESA-TTA

[33] Integrated Laser Scanning and Image-Based Topology Optimization Techniques for Detection and Quantification of Visible and Subsurface Structural Defects cs.CVPDF

Mehrdad Shafiei Dizaji, Devin Harris

TL;DR: 本研究提出了两种互补的非接触式视觉方法，用于结构构件缺陷的检测与定量表征：一是基于高分辨率激光扫描点云对比处理来定位表面损伤并量化几何损失；二是结合三维数字图像相关法（3D-DIC）全场表面变形测量与有限元模型更新及拓扑优化，通过逆问题框架从表面响应推断亚表面异常。

Details

Motivation: 解决结构缺陷可靠表征的难题，需要能同时检测可见表面损伤和不可见亚表面损伤的方法。

Result: 在包含受控平滑缺陷和随机分布缺陷的钢梁试样上评估，与铣削基准测量对比表明，两种方法均能识别和量化缺陷几何形态，并为可见与亚表面损伤评估提供互补信息。

Insight: 创新点在于将激光扫描点云处理与基于3D-DIC的逆问题拓扑优化相结合，形成互补框架，为实现具有复杂不规则损伤构件的高保真非接触结构状态评估和模型更新提供了途径。

Abstract: Reliable characterization of structural defects requires methods capable of resolving both directly observable surface damage and damage that is not visible from the inspected surface. This study presents two complementary non-contact, vision-based approaches for the detection and quantitative characterization of defects in structural components. The first approach employs high-resolution laser scanning to generate three-dimensional (3D) point clouds of damaged steel specimens. Comparative processing of measured and reference point clouds is used to localize damaged regions, quantify geometric loss, and transfer the measured defect geometry to a finite element representation. The second approach combines full-field surface deformation measurements obtained using three-dimensional digital image correlation (3D-DIC) with finite element model updating and topology optimization. In this inverse framework, measured surface response is used to infer subsurface abnormalities through their influence on the spatial distribution of structural response. Experimental steel-beam specimens containing controlled smooth defects and randomly distributed defects are used to evaluate the approaches. Comparisons with milling-based ground-truth measurements demonstrate that both methods can identify and quantify defect geometry, while providing complementary information for visible and subsurface damage assessment. The combined framework establishes a pathway toward high-fidelity, non-contact structural condition assessment and model updating for components with complex and irregular damage.

[34] Consistency as Regularization for Unsupervised Shadow Removal cs.CVPDF

Anh-Kiet Duong, Petra Gomez-Krämer, Jean-Michel Carozza

TL;DR: 本文提出了一种名为ShadowCLR的无监督阴影去除框架，通过利用阴影图像之间的内在一致性作为正则化，直接从阴影图像中学习去除阴影，无需阴影掩码或无阴影参考图像。该方法基于阴影在观察中变化而场景内容基本一致的观察，通过全局和局部一致性约束，鼓励模型恢复场景一致的外观并抑制阴影特定变化。

Details

Motivation: 现有监督方法需要成对的阴影和无阴影图像，而无监督方法通常仍依赖阴影掩码或无阴影参考，这限制了其应用。本文旨在开发一种完全无监督的阴影去除方法，直接从阴影图像中学习，解决对配对数据或额外信息的依赖问题。

Result: 在多个基准测试上的实验表明，ShadowCLR实现了与最先进的无监督方法相当甚至更优的性能，证明了仅通过一致性正则化即可有效去除阴影，无需阴影掩码或无阴影图像。

Insight: 创新点在于将阴影去除视为一致性正则化问题，利用阴影图像间的全局和局部一致性来学习共享场景信息，从而在无监督设置下实现高效阴影去除。这为无监督视觉任务提供了一种基于内在一致性的通用正则化思路。

Abstract: Shadow removal is an important preprocessing step for many vision tasks, yet existing supervised methods require paired shadow and shadow-free images, while unsupervised approaches often still rely on shadow masks or shadow-free references. We propose ShadowCLR, an unsupervised framework that learns shadow removal directly from shadow images. Our key observation is that shadows vary across observations while the underlying scene content remains largely consistent. We therefore use consistency across shadow observations as regularization, encouraging the model to recover scene-consistent appearance while suppressing shadow-specific variations. Global and local consistency further enable us to explore visually related images, learn from imperfectly aligned observations, and focus the representation on shared scene information. Experiments on multiple benchmarks show that ShadowCLR achieves competitive and often superior performance over state-of-the-art unsupervised methods, demonstrating that consistency can provide regularization for shadow removal without shadow masks or shadow-free images.

[35] Video2Reaction: Training Foundation Video Models to Predict Audience Reaction cs.CVPDF

Sidong Zhang, Trang Nguyen, Shiv Shankar, Gauri Jagatap, Deepak Chandran

TL;DR: 该论文提出了Video2Reaction数据集，该数据集将短视频片段与社交媒体评论中反映的观众情感反应进行映射。研究通过微调视觉语言模型（VLM）来预测主导情感反应，并展示了该数据集在跨领域情感数据集（VCE）上的有效迁移能力。

Details

Motivation: 为了解决从视频内容预测观众在真实世界中情感反应的问题，并捕捉情感反应的主观性和模糊性。

Result: 微调后的视觉语言模型在主导反应预测上优于专门的基线模型。在VCE数据集上，使用Video2Reaction预微调并仅用1%的VCE数据适配的LLaVA-NeXT-Video-7B模型，取得了0.682的top-3准确率，与使用完整数据集训练的最佳报告性能相当。

Insight: 创新点在于构建了一个大规模、从社交媒体评论中聚合情感反应的多模态数据集，并将情感标签建模为分类情感上的分布以反映其主观性。此外，展示了该数据集能有效提升模型在跨领域情感理解任务上的数据效率。

Abstract: We introduce Video2Reaction, a multimodal dataset that maps short movie segments to the induced emotional reactions of viewers in the wild, as expressed through social media comments. Video2Reaction captures the natural diversity of emotional responses by aggregating reactions from online comments at scale, modeling labels as distributions over categorical emotions to better reflect the subjective and ambiguous nature of emotional perception. We benchmark two vision-language models (VLMs) finetuned with LoRA, showing that VLMs learn effectively from Video2Reaction and outperform specialized baselines on dominant reaction prediction. We further demonstrate that VLMs pre-finetuned on Video2Reaction transfer effectively to VCE, another induced emotion dataset with a different taxonomy and video domain. Notably, LLaVA-NeXT-Video-7B pre-finetuned on Video2Reaction and adapted on only 1% of VCE training data achieves a top-3 accuracy of 0.682, on par with the best reported VCE performance trained on the full dataset. The dataset is available at https://huggingface.co/datasets/infofusionlab/Video2Reaction

[36] Kirin: Animal Motion Generation from In-the-Wild Video cs.CVPDF

Brian Nlong Zhao, Zhuoyang Pan, James M. Rehg, Jiajun Wu, Shangzhe Wu

TL;DR: 本文提出了Kirin框架，用于从野外视频中生成动物运动。该框架通过重建视频中的3D运动序列并构建大规模数据集AiM3D，开发了一个基于视觉引导的运动生成模型，能够根据文本和图像输入生成逼真的动物运动，并自动驱动3D网格生成可渲染的动画。

Details

Motivation: 由于高质量动物运动数据的稀缺，动物运动研究远落后于人类运动研究，现有数据集规模小且领域受限，限制了动画等下游应用。本文旨在利用野外视频解决这一数据瓶颈。

Result: 研究构建了首个大规模四足动物视频-文本-运动对齐数据集AiM3D，并基于此开发了视觉引导运动生成模型，实现了文本和图像条件下的逼真动物运动生成，为大规模动物运动生成与动画建立了新基础。

Insight: 创新点在于利用野外视频大规模重建3D运动并构建多模态数据集，以及提出文本和图像双条件引导的运动生成框架，将生成的运动直接应用于3D资产动画，实现了从视频到动画的端到端流程。

Abstract: Understanding animal motion is fundamental to modeling animal behavior and biomechanics, yet progress in this area lags far behind human motion research due to the scarcity of high-quality motion data. While human motion can be captured in controlled environments, it is impractical for most animal species, resulting in small, domain-limited datasets that restrict downstream applications such as animation. To address this challenge, we introduce Kirin, a framework that reconstructs motion from video, learns motion priors at scale, and generates realistic motion that can be directly applied to animated assets. Using large collections of in-the-wild animal videos, we reconstruct 3D motion sequences and pair them with captions to create AiM3D, the first large-scale dataset offering aligned video-text-motion tuples for quadruped animals. Building on this dataset, we develop a visual-guided motion generation model that conditions on both text and image to guide the generation of realistic motion across diverse animal species. Finally, by leveraging an off-the-shelf image-to-3D model, we automatically rig and animate 3D meshes using generated motion, producing ready-to-render animated animals. Together, our dataset and framework establish a new foundation for large-scale, text and image conditioned animal motion generation and animation. Project page: https://kirin-ani.github.io/.

[37] Cross-Model Distillation of a Human-Pose Foundation Model from Unannotated Infant Video for Markerless 3D Pose Estimation cs.CVPDF

R. James Cotton, Divya Joshi, Colleen Peyton

TL;DR: 本文提出了一种跨模型蒸馏方法，将Sapiens 2姿态模型的2D关键点知识迁移到SAM 3D Body模型上，专门用于婴儿的无标记3D姿态估计。该方法仅使用未标注的婴儿视频，通过冻结的教师模型提供密集伪标签，并结合可微分渲染器进行训练，从而在婴儿数据上提升了SAM 3D Body模型的性能。

Details

Motivation: 婴儿的自发运动是评估其神经运动健康的重要窗口，但现有的临床评分方法依赖专业评估者且耗时。虽然基于视频的无标记评估方法具有潜力，但现有基础模型主要基于成人数据训练，在婴儿姿态估计上存在2D关键点准确性与3D身体恢复能力之间的权衡，需要专门优化。

Result: 在先前研究的多视角协议下，对11名婴儿（18个会话，173个记录）的测试表明，微调后模型与Sapiens参考的2D关键点一致性显著提升（身体关键点正确百分比中位数从0.22升至0.42），Procrustes对齐后的3D关节位置平均误差从25.5毫米降低至22.2毫米。

Insight: 创新点在于仅利用未标注婴儿视频进行跨模型蒸馏，通过教师模型生成伪标签和可微分渲染器实现2D-3D对齐，有效解决了婴儿数据标注稀缺的问题，并提升了现有3D姿态模型在婴儿领域的适用性。

Abstract: Spontaneous movement is one of the earliest windows onto an infant’s neuromotor health, and structured clinical instruments that score it are validated early predictors of cerebral-palsy risk. However, they require specially trained raters, are time-consuming, and carry inter-rater variability. This motivates automated, video-based markerless assessment, especially as marker-based motion capture is impractical in infants. Yet the foundation models that make markerless capture possible are trained almost entirely on adults: our recent multi-view infant study found that no single model is jointly best, with strong 2D keypoint accuracy and direct 3D body recovery split across different models. While that study identifies this trade-off, it does not resolve it. Here, we perform cross-model distillation from the Sapiens 2 pose model into the SAM 3D Body model, using unannotated infant video alone. A frozen teacher supplies dense pseudo-labels, and a differentiable renderer aligns the predicted mesh to them in the training loop. On eleven held-out infants (18 sessions, 173 recordings) under our prior study’s multi-view protocol, fine-tuning improves same-view 2D keypoint agreement with the Sapiens reference (median body percentage of correct keypoints @ 10px 0.22 -> 0.42, face 0.22 -> 0.42) and Procrustes-aligned mean per joint 3D position error (25.5 -> 22.2 mm). This demonstrates how cross-model distillation improves SAM 3D Body model performance on infants.

[38] Does Playing it Safe Count as Faithfulness? Reassessing LVLM Hallucination Mitigation Methods cs.CVPDF

Mehrdad Fazli, Sina Mansouri, Mohit Marvania, Ziwei Zhu

TL;DR: 这篇论文重新评估了大型视觉语言模型（LVLM）的幻觉缓解方法，指出当前基于幻觉基准的评估可能高估了进展，因为降低幻觉分数往往伴随着信息量的减少和更保守的生成，而非真正的多模态基础改进。

Details

Motivation: 动机在于澄清幻觉缓解方法是否真正提升了多模态基础能力，还是仅仅导致了更保守的生成，从而解决当前评估协议可能存在的偏差问题。

Result: 在三个LVLM和四个基准（包括幻觉评估和多样化能力基准MMStar）上评估了六种缓解方法，发现幻觉减少常伴随对象召回、视觉覆盖或响应详细度的降低，且幻觉基准的改进不能可靠地转移到细粒度感知和推理任务上。

Insight: 创新点在于提出幻觉缓解应被视为忠实性、信息量和能力之间的权衡，而不仅仅是幻觉分数，这为未来评估提供了更全面的视角。

Abstract: Recent inference-time hallucination mitigation methods for large vision-language models (LVLMs) report strong gains on hallucination benchmarks. However, it remains unclear whether lower hallucination scores reflect improved multimodal grounding or more conservative generation. We evaluate six mitigation methods across three LVLMs and four benchmarks, including hallucination-focused evaluation and the diverse capability benchmark MMStar. Our analysis reveals two consistent patterns. First, hallucination reduction is often coupled with reduced informativeness: methods that lower hallucination rates also reduce object recall, visual coverage, or response detailedness. Second, improvements on hallucination benchmarks do not reliably transfer to broader multimodal capabilities, with methods showing inconsistent or degraded performance on fine-grained perception and reasoning tasks. Our findings suggest that current evaluation protocols may overestimate progress by rewarding conservative generation. We argue that hallucination mitigation should be evaluated as a faithfulness–informativeness–capability trade-off rather than through hallucination scores alone.

[39] SignMatch: Matching Dictionary Signs to Continuous Sign Language Video cs.CVPDF

Ryan Wong, Youngjoon Jang, Liliane Momeni, Gül Varol, Andrew Zisserman

TL;DR: 本文提出SignMatch方法，旨在将孤立词典手语视频与连续手语视频中的对应手势进行匹配，匹配仅基于视觉相似性（手形和相对于身体的运动）。该方法通过从标注有手势的连续视频中学习一个原型结构化的手语嵌入空间，其中每个可学习的原型对应一个手势类别，并将孤立词典视频映射到该空间中，从而实现词典样本与连续手语实例的匹配。该方法支持通过嵌入相似性进行直接词典引导的手势匹配，并可仅使用词典样本自然扩展到未见手势。

Details

Motivation: 解决在连续手语视频中匹配孤立词典手势的问题，仅依赖视觉相似性（如手形和运动），无需语言特定监督，以提升跨数据集、任务和手语语言的泛化能力。

Result: 在ASL-Citizen词典检索、ChaLearn OSLWL词典到连续手语匹配以及BOBSL的CSLR2自动手势标注评估中表现出强泛化性，在美式、英式和西班牙手语上均优于先前方法，在所有三个基准测试中达到SOTA水平。

Insight: 创新点在于原型结构化的手语嵌入空间学习，将孤立词典视频映射到连续手势的嵌入空间，实现跨语言和任务的零样本匹配；客观分析认为，该方法通过视觉相似性驱动，避免了语言特定监督，增强了跨数据集和手语的可迁移性。

Abstract: The objective of this paper is to match dictionary sign videos to corresponding signs in continuous signing videos, where a match is defined by the visual similarity alone - the handshape and motion relative to the body. To achieve this, we learn a prototype-structured sign embedding space from continuous video annotated with signs, where each learnable prototype corresponds to a sign class. Isolated dictionary videos are then mapped into this sign space, enabling the matching between dictionary exemplars and continuous sign instances. This design supports direct dictionary-guided sign matching through embedding similarity and naturally extends to unseen signs using only dictionary exemplars. Experiments on ASL-Citizen dictionary retrieval, ChaLearn OSLWL dictionary-to-continuous sign matching, and using BOBSL’s CSLR2 evaluation for automatic sign annotation demonstrate strong generalisation across datasets, tasks and sign languages. Without benchmark-specific supervision, the learned representation transfers effectively across American, British, and Spanish Sign Languages, outperforming prior methods on all three benchmarks. Project page: https://www.robots.ox.ac.uk/~vgg/research/signmatch/

[40] TAPVid-MV: A Benchmark for Tracking Any Point in 3D Across Multiple Views cs.CVPDF

Skanda Koppula, Frano Rajic, Abdullah Faiz Ur Rahman, Yi Yang, Ignacio Rocco

TL;DR: 该论文提出了TAPVid-MV，这是首个用于评估在相机运动下跨多个同步视图进行长期3D点跟踪的基准测试。它包含284个序列、1,142个校准相机流和109,769个点轨迹，涵盖室内外多个领域。通过评估超过30个基线方法，发现现有方法远未解决该任务，且几何恢复是准确3D点跟踪的主要瓶颈。

Details

Motivation: 现有的点跟踪基准测试主要关注单视频或静态多相机系统，缺乏在相机运动下跨多个视图进行长期3D点跟踪的测试。因此，需要一个新的基准来填补这一空白，以支持机器人、AR/VR和自动驾驶等领域的发展。

Result: 在超过30个基线方法评估中，没有方法能接近解决该任务。令人惊讶的是，现有的多视点跟踪器并不总是优于单目点跟踪器。该基准通过在同一数据集上评估重建和点跟踪，有助于区分几何恢复错误和点对应错误。

Insight: 论文的主要创新点是构建了首个跨多视图的3D点跟踪基准TAPVid-MV，其数据集经过人工视觉验证，并利用特定辅助模态获取轨迹。关键发现是几何恢复是准确3D点跟踪的主要瓶颈，这为未来研究指明了方向。此外，发布的标注还支持单目2D/3D点跟踪、未来轨迹预测和4D重建等任务。

Abstract: Multi-camera systems are increasingly practical for robotics, AR/VR, and autonomous driving because complementary views reduce depth ambiguity and preserve visibility under occlusion. Existing point-tracking benchmarks, however, focus on a single video or static multi-camera rigs. None test long-term 3D point tracking across several synchronized views under camera motion. We introduce TAPVid-MV (Tracking Any Point in Video across Multiple Views), the first benchmark for this setting. It contains a curated set of 284 sequences, 1,142 calibrated camera streams, and 109,769 point tracks across seven subsets spanning indoor and outdoor domains, from robotics and human activity to driving and synthetic procedural scenes. We obtain these trajectories using dataset-specific auxiliary modalities: sensor depth, LiDAR, SLAM and SfM points, human meshes, posed object meshes, and simulation. Every sequence and trajectory is visually verified by human annotators. Across more than 30 baselines, no method comes close to solving the task. Surprisingly, existing multi-view point trackers do not consistently outperform monocular point trackers. By evaluating reconstruction and point tracking on the same datasets, TAPVid-MV helps distinguish errors in recovered geometry from errors in point correspondence. Through this joint analysis, we identify geometry recovery as a major bottleneck for accurate 3D point tracking. Beyond multi-view 3D point tracking, our released annotations support monocular 2D and 3D point tracking, future-trajectory prediction, and 4D reconstruction.

[41] Learning with Volterra Neural Networks: A System Theoretic Perspective cs.CVPDF

Haoyu Yun, Hamid Krim, Yufang Bao

TL;DR: 本文提出了一种可学习的核化Volterra神经算子（kVNN），用于实现紧凑的高阶滤波。该方法将Volterra滤波的阶次结构与可学习的多项式核原子相结合，通过核化技术提升Volterra型神经算子的效率，同时为其高阶分量提供结构化解释。kVNN避免了显式的高阶张量参数化，可作为CNN兼容层实现，在代表性视觉任务中取得了良好的精度-效率权衡。

Details

Motivation: 高阶交互分量对于信号、图像和视频建模至关重要，但显式的高阶算子通常面临参数和计算成本快速增加的问题。本文旨在通过核化技术提升Volterra型神经算子的效率，并为其高阶分量提供结构化解释。

Result: 在代表性视觉任务上的实验表明，kVNN实现了良好的精度-效率权衡（favorable accuracy–efficiency trade-off）。

Insight: 创新点在于将Volterra滤波的阶次结构与可学习的多项式核原子相结合，形成阶次解耦表示，避免了显式的高阶张量参数化，并可实现为CNN兼容层，从而在保持高阶建模能力的同时提升了计算效率。

Abstract: Higher-order interaction components are important for signal, image, and video modeling, but explicit high-order operators often suffer from rapidly increasing parameter and computational costs. This paper presents kVNN, a learnable kernelized Volterra Neural operator for compact higher-order filtering. The motivation is to use kernelization to improve the efficiency of Volterra-type neural operators while providing a structured interpretation of their higher-order components. The proposed formulation combines the order-wise structure of Volterra filtering with learnable polynomial-kernel atoms, allowing different interaction orders to be represented by separate learnable centers and coefficients. This order-decoupled representation avoids explicit high-order tensor parameterization and can be implemented as a CNN-compatible layer. Experiments on representative vision tasks show that kVNN achieves a favorable accuracy–efficiency trade-off.

[42] Automated Maize Ear Phenotyping Using 3D Reconstructions cs.CVPDF

Ritwesh A. Kumar, Som Tripathi, Peja Matthews, Srikar Reddy, Talukder Zaki Jubery

TL;DR: 本文提出了一种全自动的玉米穗表型分析流程，利用3D点云技术自动提取玉米穗的籽粒行数、每行籽粒数和籽粒大小等性状。该方法基于视频到点云平台，通过COLMAP和NeRF处理原始视频帧，使用基于密度的分离和PCA对齐，并结合Cellpose-SAM进行零微调实例分割，实现了高效的性状测量。

Details

Motivation: 玉米籽粒性状（如行数、每行籽粒数和籽粒大小）与产量相关，但人工测量无法满足育种计划中大量玉米穗的处理需求，因此需要自动化解决方案。

Result: 在268个标记数据集的168个玉米穗保留集上，该方法实现了籽粒计数的R^2 = 0.921（MAPE = 10.33%），籽粒行数在95.2%的玉米穗中误差在±2行以内（MAE = 0.75行），达到了高精度水平。

Insight: 创新点包括结合COLMAP和NeRF构建3D点云、使用基于密度的分离和PCA对齐进行预处理、采用Cellpose-SAM进行零微调实例分割，以及三重并置展开策略避免重复计数，为农业表型分析提供了高效自动化工具。

Abstract: Maize kernel traits such as row number, kernels per row, and kernel size vary largely for genetic reasons and are consistently associated with regions of the genome that influence yield. Manual measurement of these traits, however, cannot keep pace with the volume of maize generated in a breeding program. To address this, we developed and validated a fully automated pipeline for extracting these traits from 3D point clouds of corn ears, built on a recently developed video-to-point-cloud platform. Raw video frames are processed through COLMAP and NeRF, the ear is isolated via density-based separation, and the point cloud is distance-calibrated to physical units. The calibrated ear point cloud was Z-axis aligned via PCA and cylindrically unwrapped to a 2D image. We enhanced contrast and performed zero-fine-tuning instance segmentation using Cellpose-SAM. A triple-juxtaposed unwrap strategy was used to prevent double-counting at the seam. The pipeline achieved kernel count R^2 = 0.921 (MAPE = 10.33%) and kernel row number within +-2 rows for 95.2% of ears (MAE = 0.75 rows) on a 168-ear held-out set from the 268-ear labeled dataset. The resulting multi-trait dataset has known genotype identity for each ear, positioning it for phenotype-to-genotype association analyses.

[43] Who Drives the Probability Game of VLMs? A Temporal Causal Drive Evaluation Framework cs.CVPDF

Shuyao Xiao, Shengling Wang, Haoyu Niu, Ke Chao, Changwei Xu

TL;DR: 本文提出了一种因果与时间评估框架，用于分析视觉语言模型（VLMs）在自回归解码过程中视觉输入、问题文本和生成前缀等不同信息源的动态影响。该框架基于结构因果模型，通过干预和后门调整定义了三个因果驱动指标（VCD、QCD、PCD），以量化各信息源对生成过程的贡献，无需参考答案。在多个数据集和模型上的实验表明，VLMs的生成模式存在从早期依赖视觉和问题向后期依赖生成前缀的转变，且因果驱动指标在区分先验驱动与视觉基础生成方面表现优异。

Details

Motivation: 现有评估指标主要关注VLMs最终答案的质量，但无法揭示不同信息源在生成过程中的动态作用，因此需要一种能够追溯和量化各信息源因果贡献的评估框架。

Result: 在Qwen3-VL-8B-Instruct等模型上的实验显示，QCD和PCD指标相比观测性PMI基线分别减少了34.8%和47.1%的恢复误差；在VLMBias数据集上，前缀-视觉不平衡分数的AUROC和AUPRC分别达到0.767和0.873，有效区分了先验驱动与视觉基础生成。

Insight: 创新点在于将因果推断与时间动态分析结合，提出了无需参考答案的因果驱动指标，为多模态生成提供了源级别的诊断工具，揭示了VLMs生成过程中信息依赖的时序演变规律。

Abstract: Vision-language models (VLMs) are increasingly evaluated on complex image and video understanding tasks, yet conventional metrics primarily assess final-answer quality and reveal little about how different information sources shape the generation process. We propose a causal and temporal evaluation framework that traces the evolving roles of visual input, question text, and generated prefixes during autoregressive decoding. Grounded in a Structural Causal Model, we use interventions and backdoor adjustment to derive three step-indexed causal-drive metrics—Visual Causal Drive (VCD), Question Causal Drive (QCD), and Prefix Causal Drive (PCD)—for characterizing source-specific generation patterns without requiring reference answers. Experiments on Qwen3-VL-8B-Instruct across MAVIS, LLaVA-Video-178K, and MiraData, together with cross-model validation on InternVL2-8B, reveal a consistent transition from stronger early question and visual guidance toward increasing reliance on generated prefixes. Randomized-intervention validation shows that QCD and PCD reduce recovery error over observational PMI baselines by 34.8% and 47.1%, respectively. On VLMBias, the prefix–visual imbalance score achieves 0.767 AUROC and 0.873 AUPRC for distinguishing prior-driven from visually grounded generations. These results show that causal-drive trajectories provide complementary source-level diagnostics for multimodal generation.

Xuanbing Wen, Boxu Chen, Le Yang, Jiakai Wang, Zhengyu Zhao

TL;DR: 本文提出CADMP框架，用于检测大型视觉语言模型中的物体幻觉问题。该方法通过分析相邻层间跨模态注意力漂移并结合针对性的视觉掩码验证，来识别模型生成中不依赖视觉依据的虚假预测。

Details

Motivation: 现有检测方法通常只关注单层注意力来表征视觉基础，而忽略了注意力在模型层间的演化过程，这限制了幻觉检测的可靠性。

Result: 在多个基准测试和代表性开源LVLM上的实验表明，CADMP实现了持续有竞争力的检测性能。消融研究证实了相邻层漂移建模和基于掩码的验证具有互补贡献。

Insight: 创新点在于利用跨层注意力分布的突变（注意力漂移）来捕捉视觉基础的不稳定性，并与掩码敏感度验证相结合，为幻觉检测提供了内部表征动态和外部预测依赖的双重证据。

Abstract: Despite recent advances in large vision-language models (LVLMs), object hallucination remains a major barrier to their reliable deployment. Existing detection methods often characterize visual grounding using attention from individual layers, leaving its evolution across layers underexplored. We propose CADMP, a lightweight object hallucination detection framework that combines adjacent-layer cross-modal attention drift with prediction sensitivity to targeted visual masking. During decoding, CADMP quantifies distributional changes between consecutive cross-modal attention maps to capture abrupt transitions in visual grounding. It then selects the transition with the largest drift, locates the corresponding visually relevant regions, and measures the change in prediction probability after masking these regions. These two signals provide complementary evidence: attention drift characterizes the stability of internal visual grounding, while probability variation verifies whether a prediction truly depends on the identified visual evidence. A lightweight detector integrates both signals to identify hallucinated predictions. Experiments on multiple benchmarks and representative open-source LVLMs demonstrate that CADMP achieves consistently competitive detection performance. Ablation studies further confirm the complementary contributions of adjacent-layer drift modeling and mask-based grounding verification.

[45] SelfLift: Accelerating Few-Step Diffusion via Self-Recovering Resolution Transition cs.CVPDF

Tingyan Wen, Chenqian Yan, Xurui Peng, Xiazhang Fang, Shuai Wang

TL;DR: SelfLift是一种自恢复渐进分辨率框架，旨在加速少步扩散模型的推理。它通过延迟分辨率转换（即在低分辨率下执行早期去噪，后期再转换到高分辨率进行细化）来降低空间计算成本，并利用模型自身提供修复信号和轨迹对齐监督，从而在有限的恢复步数内有效减少伪影，显著降低端到端延迟。

Details

Motivation: 少步扩散模型压缩了时间计算，使得每次模型评估的空间成本成为推理延迟的主要来源。现有的渐进分辨率推理方法直接将中间潜在变量提升到高分辨率，依赖后续步骤吸收分布不匹配，但在步数有限的情况下，恢复能力不足会导致可见伪影，限制了转换发生的时机和效率。

Result: 在FLUX.2-Klein和Z-Image-Turbo模型上，SelfLift分别将端到端延迟降低了41.5%和44.1%。结合时间步蒸馏，它相比对应的50步模型实现了29.61倍和19.21倍的整体加速，同时保持了有竞争力的生成质量，为少步扩散模型建立了更强的速度-质量前沿。

Insight: 创新点在于提出了自恢复机制：SelfLift-zero使用无训练的伪影感知一致性提升，利用直接潜在提升与像素VAE重新编码之间的不一致性作为局部伪影风险信号和模型原生校正方向；SelfLift-rich则在学生访问的状态上进行策略内自恢复，从内部自教师转移密集的高分辨率指导，同时与改变的渐进分辨率动态保持一致。这种方法无需外部超分辨率、额外去噪器评估或采样计划修改，实现了可靠且高效的延迟分辨率转换。

Abstract: Few-step diffusion models substantially compress temporal computation, making the spatial cost of each model evaluation an increasingly dominant source of inference latency. Progressive-resolution inference reduces this cost by performing early denoising at low resolution and reserving high-resolution computation for refinement. However, existing methods typically lift intermediate latents directly and rely on subsequent steps to absorb the induced distribution mismatch. In the few-step regime, the limited recovery budget leaves these errors as visible artifacts, constraining how late the transition can occur and, consequently, how efficiently it can be performed. We introduce SelfLift, a self-recovering progressive-resolution framework that derives both transition-repair signals and trajectory-aligned supervision from the generative model itself. SelfLift-zero proposes a training-free Artifact-Aware Consistency Lift, using disagreement between direct latent lifting and pixel-VAE re-encoding as both a localized artifact-risk signal and a model-native correction direction. It enables reliable late transitions without external super-resolution, extra denoiser evaluations, or sampling-schedule modifications. Building on this robust transition, SelfLift-rich performs On-Policy Self Recovery on student-visited states, transferring dense high-resolution guidance from an internal self-teacher while remaining aligned with the altered progressive-resolution dynamics. Across FLUX.2-Klein and Z-Image-Turbo, SelfLift reduces end-to-end latency by 41.5% and 44.1%, respectively. Combined with timestep distillation, it delivers overall speedups of 29.61x and 19.21x over the corresponding 50-step models while preserving competitive generation quality, establishing a stronger speed-quality frontier for few-step diffusion.

[46] Test-Time Logit Prompting for Source-Free Missing Modality Adaptation cs.CVPDF

Taixi Chen, Nancy Guo

TL;DR: 本文提出了一种名为测试时对数提示（TLP）的轻量级源无关测试时适应框架，用于处理视觉语言模型在缺失模态输入时的视觉识别任务。该方法通过不确定性感知调整和模态完整性一致性正则化来优化对数提示，自适应地调整预测置信度并保持语义一致性，从而在无需访问源训练数据的情况下提升模型在缺失模态场景下的性能。

Details

Motivation: 现实世界部署中常遇到模态缺失的输入，导致视觉语言模型性能显著下降。现有方法依赖源训练数据学习模态补偿策略，但在隐私、存储或可访问性限制下（如临床应用和个性化AI服务）难以应用，因此需要研究无需源训练数据、在测试时高效适应缺失模态的视觉识别方法。

Result: 在多个视觉语言基准测试上的广泛实验表明，TLP在缺失模态场景下持续提升识别性能，最高提升达8%，同时仅需数百个可调参数和少量测试时优化步骤。

Insight: 创新点在于提出了一种源无关的测试时适应框架，通过优化对数提示结合不确定性调整和一致性正则化来应对缺失模态引起的预测偏移；客观分析认为，该方法轻量高效，无需原始训练数据，适用于数据受限的实际应用场景，为缺失模态适应提供了新思路。

Abstract: Vision-language models (VLMs) have achieved remarkable performance by leveraging complementary information from large-scale image-text pairs. However, missing-modality inputs are commonly encountered during real-world deployment, often leading to significant performance degradation. Existing methods primarily enhance model robustness by learning modality compensation strategies from source training data. However, their reliance on source training data makes them difficult to apply when original data are unavailable due to privacy, storage, or accessibility constraints, such as clinical applications and personalized AI services. This raises an important yet underexplored question: can VLMs be efficiently adapted at test time for visual recognition with missing modalities without accessing source training data? To this end, we propose Test-Time Logit Prompting (TLP), a lightweight source-free test-time adaptation framework for visual recognition with missing modalities. To address missing-induced prediction shifts, TLP optimizes logit prompts with uncertainty-aware adjustment and modality-complete consistency regularization, adaptively adjusting prediction confidence while preserving semantic consistency. Extensive experiments across diverse vision-language benchmarks demonstrate that TLP consistently enhances recognition performance under missing-modality scenarios, achieving up to 8% improvements while requiring only hundreds of tunable parameters and a few test-time optimization steps.

[47] Rendering-in-the-Loop: An Execution-Driven Agent for Interactive Web Development cs.CVPDF

Yilong Guo, Hanqi Chen, Zixiao Ye, Guanzhong Wang, Chen Yu

TL;DR: 本文提出了RILA，一种执行驱动的智能体，通过将浏览器渲染纳入循环，利用运行时交互反馈迭代编辑生成的代码，以解决现有多模态大语言模型在网页前端开发中忽视交互功能验证的问题。RILA包含一个动作交互验证模块和一个联合评估交互正确性与视觉保真度的执行感知渲染分数，并构建了一个执行验证的数据合成管道来生成高质量训练数据。

Details

Motivation: 现有基于多模态大语言模型的前端网页生成工作过于关注视觉指标（如美观度和布局相似性），而忽视了对交互功能这一更关键方面的验证。

Result: 在IWR-Bench基准测试中，RILA持续提升了交互和视觉保真度。通过其训练管道，RILA将紧凑的Qwen3.5-9B骨干模型的性能从40.40%提升至57.52%，超越了包括1T参数的Kimi-K2.6（55.61%）和专有模型GPT-5.5（55.74%）在内的更大规模单次生成模型。

Insight: 核心创新在于将浏览器渲染和执行反馈纳入代码生成与优化循环，通过动作交互验证模块获取基于执行的观察，并定义了联合评估交互正确性与视觉保真度的新指标。此外，执行验证的数据合成管道为模型训练提供了高质量数据，与推理时优化形成互补增益。

Abstract: Multimodal large language models have achieved remarkable progress in front-end web development, generating interactive webpages from multimodal references such as screenshots and interaction videos. However, existing work largely emphasizes visual metrics such as aesthetics and layout similarity, while overlooking the more critical validation of interactive functionality. We present RILA, an execution-driven agent that puts browser rendering in the loop, iteratively editing generated code from runtime interaction feedback. RILA introduces an Action Interaction Verification (AIV) module that replays the reference interaction trajectory on the generated webpage to collect grounded execution-aware observations, and an Execution-aware Rendering Score (ERS) that jointly measures interaction correctness and visual fidelity to guide iterative optimization. We further build an execution-verified data synthesis pipeline that produces diverse, high-quality training data, offering gains complementary to inference-time optimization. On IWR-Bench, RILA consistently improves both interaction and visual fidelity across foundation models. Notably, with our training pipeline, RILA lifts the compact Qwen3.5-9B backbone from 40.40% to 57.52%, surpassing far larger one-shot generators, including the 1T-parameter Kimi-K2.6 (55.61%) and the proprietary GPT-5.5 (55.74%).

[48] Evidence-Guided Detection, Localization and Explanation for Text-Centric Image Forensics cs.CVPDF

Peifeng Liu, Bin Li, Qingsong Zhang, Yangxin Yu, Leqing Chen

TL;DR: 本文针对以文本为中心的图像伪造检测挑战，提出了一个证据引导的检测-定位-推理系统。该系统包含图像级检测器、专用定位器和基于MLLM的推理器，通过级联证据流协同工作，实现图像真实性检测、篡改区域定位和结构化取证报告生成。

Details

Motivation: AIGC的快速发展使得以文本为中心的图像篡改变得容易，这带来了新的取证挑战，不仅需要真实性检测，还需要空间定位和基于证据的解释。

Result: 在ACM Multimedia 2026 GenText-Forensics挑战赛的官方隐藏测试集上，该系统取得了0.638的最终得分，在挑战赛中排名第二。

Insight: 创新点在于构建了一个级联的证据流系统，并引入了迭代难度感知挖掘来提升定位质量，以及应用报告-掩码一致性后处理来对齐报告定位与预测掩码，从而将检测决策和定位证据整合到最终的取证报告中。

Abstract: The rapid progress of AIGC has made text-centric image manipulation increasingly accessible, creating new forensic challenges that require not only authenticity detection but also spatial grounding and evidence-based explanation. This paper presents our solution to the GenText-Forensics Challenge at ACM Multimedia 2026. We propose an evidence-guided detector-localizer-reasoner system, where an image-level detector provides a global authenticity prior, a dedicated localizer extracts tampered regions as spatial grounding evidence, and an MLLM-based reasoner generates structured forensic reports grounded in this expert forensic evidence. These modules are connected through a cascaded evidence flow: the detector gates the subsequent localization and prompting process, the localizer converts tamper responses into grounding boxes, and the reasoner is trained to synthesize the detector decision and localized evidence into the final report. As a key part of our method, we introduce iterative difficulty-aware mining to improve localization quality and apply report-mask consistency post-processing to align report grounding with predicted masks. On the official hidden test set, our system achieves a final score of 0.638 and ranks second in the challenge, validating the effectiveness of the proposed evidence-guided system. The code is available at https://github.com/peifengLiu42/ACMMM26-evidence-guided-detector-localizer-reasoner-system.

[49] Disease Burden over Skin Tone: Decomposing the Dermatology-AI Generalization Gap cs.CV | cs.AI | cs.LGPDF

Nirajan Kunwor, Sanjaya Poudel, Quoc-Huy Trinh, Jahidul Arafat, Sunil Kumar Gaire

TL;DR: 该论文研究了皮肤病AI模型在泛化到不同肤色和疾病分布人群时的性能差距，发现疾病分布偏移比肤色差异对模型泛化性能的影响更大。通过评估多种模型在肤色分层和疾病偏移数据集上的表现，揭示了模型泛化失败主要源于特征表示的限制，而非仅缺少输出标签。

Details

Motivation: 皮肤病AI模型主要基于浅肤色、癌症集中的图像训练，但常被部署在资源有限、患者肤色和疾病分布与训练数据不同的环境中，因此需要探究泛化差距的主要成因是肤色代表性不足还是疾病分布偏移。

Result: 在评估中，癌症训练基线模型从平衡准确率0.62降至0.21（当转移到不熟悉的临床条件时），而肤色差距较小（0.10-0.18）且不一致；皮肤病预训练模型的特征表示具有更强的可迁移结构（kNN纯度提升+0.23），且轻量适应下每个临床类别约10个标注样本即可恢复大部分性能。

Insight: 创新点在于分解了皮肤病AI泛化差距的成因，强调疾病分布偏移是关键因素，并展示了特征表示质量对轻量适应性能的预测作用；客观分析认为，该方法为可复现的皮肤病AI泛化审计提供了协议和代码支持。

Abstract: Dermatology artificial intelligence (AI) models are predominantly trained on light-skinned, cancer-focused image collections, yet they are increasingly proposed for deployment in resource-constrained settings where patients differ from training populations along two confounded axes: skin tone and disease distribution. We investigate whether poor generalization is primarily caused by skin-tone underrepresentation or disease-distribution shift. We evaluate a cancer-trained baseline (ResNet-50 fine-tuned on HAM10000 and ISIC 2019), two dermatology foundation models (DermLIP and MONET), and a general-purpose vision model (DINOv3) as frozen feature extractors. Models are evaluated on a tone-stratified disease-matched dataset (Diverse Dermatology Images, DDI) and a disease-shifted tone-diverse dataset (Skin Condition Image Network, SCIN). Our results show that disease-distribution shift contributes more than skin tone in the evaluated settings. The cancer baseline decreases from 0.62 to 0.21 balanced accuracy when transferred to unfamiliar clinical conditions, while the within-disease skin-tone gap is smaller (0.10-0.18) and inconsistent. Label-free representation analysis shows that this failure reflects a representational limitation rather than only missing output labels: cancer-specialized features poorly cluster unfamiliar conditions (kNN purity lift +0.06 over chance), whereas dermatology-pretrained features retain stronger transferable structure (+0.23). Finally, we show that representation quality predicts recoverable performance under lightweight adaptation. Starting from dermatology foundation models, approximately ten labeled examples per clinical category recover most attainable performance. We release the evaluation protocol and code to support reproducible auditing of dermatology AI generalization.

[50] World-Coherent Decoding: Self-Verifying Test-Time Planning for World Action Models cs.CVPDF

Chuhan Zhang, Seiji Ito, Kenta Hoshino, Satoshi Ikehata, Ikuro Sato

TL;DR: 本文提出了一种名为World-Coherent-Decoding（WCD）的自验证测试时规划框架，用于改进世界动作模型（WAMs）在机器人控制中的性能。该方法将WAM生成的未来-动作假设视为可证伪的，通过内部生成信号（如基于流的视频惊奇度和动作路径努力）对多个候选进行排序，并在执行后利用观察-想象差异在线训练轻量级预测器，从而将延迟的自我验证转化为执行前的可靠性估计，而无需更新主干模型。

Details

Motivation: 现有世界动作模型（WAMs）通过随机生成视觉未来并解码动作来控制机器人，但实验结果强烈依赖于所选择的未来，导致性能不稳定。本文旨在解决WAMs在测试时因未来选择不可靠而导致的决策质量波动问题。

Result: 在RoboTwin 2.0基准测试中，WCD在有限随机场景监督下将Hard成功率从55.80%提升至60.90%，在Horizon-3任务上获得了+16.43的增益，并在真实Franka视觉偏移测试中表现出定性鲁棒性。

Insight: 创新点在于将WAM的推演视为可证伪假设，并利用内部生成信号进行候选排序，结合执行后的观察-想象差异进行在线学习，从而实现了无需更新主干模型的测试时可靠性选择。核心洞察是：对于WAMs，测试时性能的提升更依赖于选择可靠的未来，而非单纯增加采样数量。

Abstract: World Action Models (WAMs) aim to control robots by stochastically generating visual futures and then decoding actions, but empirical observations indicate that the results can strongly depend on which future is selected. We propose World-Coherent-Decoding (WCD), a self-verifying test-time planning framework that treats WAM rollouts as falsifiable future–action hypotheses. At each decision step, WCD samples multiple candidates from a frozen WAM and ranks them using internal generative signals: flow-based video surprisal for visual plausibility and action path effort for action-generation stability. After execution, the realized observation audits the selected imagination, yielding an imagination–reality mismatch that trains a lightweight online predictor for future candidate selection. Thus, WCD converts delayed self-verification into pre-execution reliability estimation without updating the backbone model. On RoboTwin 2.0, WCD improves Hard success under limited randomized-scene supervision from $55.80%$ to $60.90%$, with a $+16.43$ gains on Horizon-3 tasks, and shows qualitative robustness on real Franka visual-shift tests. These results highlight a simple principle: test-time scaling for WAMs depends less on sampling more futures than on selecting reliable ones.

[51] TAME: Temporal-Aware Mixture-of-Experts for Text-Video Retrieval cs.CVPDF

Uicheol Jung, Juyoung Hong, Hojung Kwon, Yukyung Choi

TL;DR: 本文提出了一种名为TAME的时序感知专家混合模型，用于改进基于CLIP的文本-视频检索任务。该方法通过集成稀疏MoE层、引入帧-时序令牌以及设计跨时序交互聚合模块，联合建模帧级结构和时序关系，从而提升视频检索性能。

Details

Motivation: 现有基于CLIP的图像-文本模型扩展到视频检索时，由于缺乏时序建模能力，难以处理视频帧间的外观和运动异质性，将多帧压缩为单一表示会模糊时序结构和语义转换。

Result: 在MSR-VTT、DiDeMo、MSVD、LSMDC和ActivityNet等标准文本-视频检索基准测试中，TAME均优于基于CLIP的基线模型，例如在MSR-VTT上相比CLIP4Clip将R@1提升了4.0个百分点。

Insight: 创新点包括：在CLIP编码器中集成稀疏MoE层并采用帧一致路由，使专家能根据帧级视觉模式进行专业化处理；引入帧-时序令牌以聚合全局跨帧信息并反馈给各帧，从而捕捉长程时序依赖；设计跨时序交互聚合模块，通过分阶段时序过滤和融合优化帧级句子-视频相似度。

Abstract: Text-Video Retrieval (TVR) retrieves videos that match a natural-language query, but extending image-text models such as CLIP to videos is fundamentally limited by the lack of temporal modeling. Videos exhibit frame-wise heterogeneity in appearance and motion, and compressing all frames into a single representation often obscures temporal structure and semantic transitions. To address this, we propose Temporal-Aware Mixture-of-Experts for Text-Video Retrieval (TAME), a CLIP-based framework that jointly models frame-level structure and temporal relations. First, we integrate sparse Mixture-of-Experts (MoE) layers into both CLIP encoders and apply frame-consistent routing on the vision branch so that experts specialize according to frame-level visual patterns while preserving the original vision-language alignment. Second, we introduce Frame-Temporal (FT) tokens that aggregate global cross-frame information and feed it back to each frame, enabling the visual encoder to capture long-range temporal dependencies without harming local details. Third, we design a Cross-Temporal Interaction and Aggregation (CTIA) module that refines frame-wise sentence-video similarities through staged temporal filtering and fusion. Experiments on standard TVR benchmarks show that TAME consistently improves over CLIP-based baselines. On MSR-VTT, it improves R@1 by 4.0 over CLIP4Clip, and also achieves consistent gains on DiDeMo, MSVD, LSMDC, and ActivityNet. The code is available at https://github.com/sejong-rcv/TAME.

[52] Lightweight Adaptation of General-Purpose VLMs for Multispectral and SAR Image Understanding cs.CVPDF

Shanji Liu, Kelu Yao, Junxiao Xue, Chenghui Lv, Xiangyang Miao

TL;DR: 本文提出了一种轻量级方法，将通用视觉语言模型（VLMs）适配于多光谱和合成孔径雷达（SAR）图像理解。该方法通过将多光谱和SAR观测数据渲染为多个光学视图和SAR视图，并利用LoRA微调语言网络和部分视觉Transformer块，使现有VLM能够处理这些非三通道传感器数据。在BigEarthNet-v2衍生的六类土地覆盖基准测试中，适配后的Qwen3-VL模型达到了0.8275的微平均F1分数，且该方法可推广至其他VLM架构和任务。

Details

Motivation: 通用VLM的视觉编码器通常针对三通道自然图像设计，无法直接处理多光谱或SAR等传感器数据。传统适配方法需要专用编码器和领域预训练，阻碍了利用更强通用模型检查点的效率。本文旨在通过轻量级适配，使现有VLM能够直接复用，避免训练新的基础模型。

Result: 在基于BigEarthNet-v2的平衡六类土地覆盖基准测试中，适配后的Qwen3-VL模型达到0.8275微平均F1分数。该方法在测试的四种VLM架构上均有效，并可迁移至Sen1Floods11洪水验证和BigEarthNet.txt图像描述任务。消融实验表明模型确实利用了提供的传感器观测数据。

Insight: 创新点在于利用VLM的多图像接口，将多光谱和SAR数据渲染为多个命名视图，通过现有视觉界面暴露波段组合、光谱指数和雷达后向散射信息。采用结构化监督将预测类别与传感器证据耦合，并构建偏好对以鼓励完整且与观测一致的预测。核心是仅需LoRA微调和输入渲染，无需训练新基础模型，实现了轻量级领域适配。

Abstract: General-purpose vision-language models (VLMs) now support strong visual recognition, instruction following, and generation. However, most pretrained visual encoders are built around three-channel natural images and do not directly accommodate observations such as native multispectral measurements or synthetic aperture radar (SAR). Adapting VLMs to these sensors typically requires dedicated encoders and domain pretraining, slowing the reuse of stronger general-purpose checkpoints. We show that the multi-image interface of general-purpose VLMs offers a lightweight alternative. Our protocol renders each observation as five optical views and one SAR view, names them in the prompt, and adapts the language network and selected visual transformer blocks with LoRA. This exposes band composites, spectral indices, and radar backscatter through an existing visual interface. For land-cover recognition, structured supervision couples predicted classes with sensor evidence. We further construct preference pairs in which a true label is omitted while its supporting evidence is retained, encouraging complete predictions that remain consistent with the observations. On a balanced six-class land-cover benchmark derived from BigEarthNet-v2, the adapted Qwen3-VL reaches 0.8275 micro F1. The same input and adaptation protocol improves all four tested VLM architectures and transfers to Sen1Floods11 flood verification and BigEarthNet.txt captioning. Image removal and mismatch controls show that the adapted models use the supplied sensor observations. Together, these results demonstrate that VLMs can be repurposed for multispectral and SAR tasks through rendered inputs and compact LoRA adaptation, without training a new foundation model.

[53] LeakageBench: Document-Level Leakage Risk for Redacting Personally Identifiable Information in Document Images cs.CV | cs.CLPDF

Vishnu Prasad Vijaya Kumar, Santhosh Venkatesh, Ivan P. Yamshchikov

TL;DR: 本文提出了LeakageBench，一个用于评估文档图像中个人身份信息（PII）脱敏风险的基准测试集。该基准包含500个文档图像和近1.2万个符合GDPR标准的PII标注，用于衡量现有OCR管道、商业检测器及视觉语言模型在实体级、组级和文档级上的泄漏风险。实验表明，即使使用工具辅助提升了定位精度，文档级别的泄漏风险仍然极高。

Details

Motivation: 现实世界中的PII脱敏通常针对文档图像（如扫描件、截图）进行操作，而现有的PII基准大多以文本为中心，未能衡量文档级别的整体泄漏风险——即只要遗漏一个标识符，整个页面就不安全。

Result: 在LeakageBench上评估了多种方法：通用OCR管道、商业及任务适配的OCR依赖检测器，以及OCR-free的视觉语言模型。使用Code Interpreter将GPT-5.5的定位F1从0.090提升至0.249，但关键的页面级泄漏率仍高达0.968。结果表明，更强的检测和工具辅助能改善定位，但无法使大多数页面达到安全发布的标准。

Insight: 创新点在于首次提出了一个专注于文档图像级别PII泄漏风险的诊断性基准，引入了GDPR对齐的细粒度标注（直接标识符、链接键、上下文重识别表面）和多级评估指标（实体级F1、组级泄漏、文档级泄漏），揭示了当前方法在确保整页安全方面的根本性不足，强调了高召回率和空间定位在文档图像脱敏中的重要性。

Abstract: Real-world personally identifiable information (PII) redaction often operates on document images—scans, screenshots, and PDF renderings—where OCR errors, layout structure, and visual noise determine whether sensitive information is actually removed. Existing PII benchmarks are mostly text-centric and do not measure document-level redaction risk: a page remains unsafe if even one identifier is missed. We introduce LeakageBench, a challenge set of 500 document images with 11,954 GDPR-aligned PII annotations spanning direct identifiers, linkage keys, and contextual re-identification surfaces. We evaluate generic OCR pipelines, commercial and task-adapted OCR-dependent detectors, and OCR-free vision-language models using entity-level F1, group-wise leakage, and document-level leakage metrics. Code Interpreter raises GPT-5.5 localization F1 from 0.090 to 0.249, but critical page-level leakage remains 0.968. These results show that stronger detection and tool assistance improve localization without making most pages safe for release. LeakageBench provides a diagnostic benchmark for high-recall, spatially grounded PII redaction in document images.

[54] Asymmetric Paired-Annotation Learning for Multi-Structure ULF Pediatric Brain MRI Segmentation cs.CVPDF

Ha-Hieu Pham, Dang P. M. Cao, Minh Hoang Pham, Khanh Nguyen Vo Ngoc, Thanh-Huy Nguyen

TL;DR: 该论文提出了AURA，一种基于nnU-Net的非对称监督策略，用于解决便携式超低场（ULF）儿科脑MRI分割的挑战。该方法将LISA 2026挑战赛中提供的高场衍生（HF）和低场编辑（LF）两种非等效标注作为不同的观察结果进行处理，通过一个基于标签不一致性、边界、预测不确定性、类别可靠性和训练阶段的有界可靠性门控机制来整合LF标注。

Details

Motivation: 便携式超低场（0.064 T）MRI可以扩大儿科神经成像的获取途径，但其分割面临解剖边界模糊、小结构部分可见以及高场参考图像局部配准错误等挑战。LISA 2026挑战赛提供了两种反映不同解剖证据来源的非等效标注，需要一种策略来有效利用这些不同质量的标注。

Result: 在16个病例的开发集上，仅使用HF标注监督的基线模型、AURA模型以及它们的集成模型的Dice分数分别为0.7984、0.7950和0.7988。集成模型还取得了HD95为1.8892和ASSD为0.7855的结果。这些结果为AURA在LISA 2026挑战赛中的初步评估提供了依据。

Insight: 创新点在于将两种非等效标注视为不同的观察结果而非可互换的真值，并设计了一个有界可靠性门控机制来动态、有条件地整合质量较低的LF标注。这为利用多源、质量不一的标注进行医学图像分割提供了一种新的监督学习范式。

Abstract: Portable ultra-low-field (ULF) MRI can expand access to pediatric neuroimaging, but segmentation at 0.064 T remains challenging because anatomical boundaries are weakly delineated, small structures may be only partially visible, and high-field references can be locally misregistered. The LISA 2026 Challenge provides two non-equivalent annotations reflecting different sources of anatomical evidence: a highfield-derived (HF) mask defining the scored target and a low-field-edited (LF) mask aligned with visible ULF anatomy. In this challenge report, we describe AURA, an nnU-Net-based asymmetric supervision strategy that treats these annotations as distinct observations rather than interchangeable ground truths. AURA anchors training to the HF mask and incorporates the LF mask through a bounded reliability gate based on label disagreement, boundaries, predictive uncertainty, class reliability, and training stage. On a 16-case development split, the HF-supervised baseline, AURA, and their ensemble achieved Dice scores of 0.7984, 0.7950, and 0.7988, respectively, while the ensemble achieved an HD95 of 1.8892 and an ASSD of 0.7855. These results provide a preliminary evaluation of AURA within the LISA 2026 Challenge and motivate further assessment on the hidden test set and external ULF cohorts. Our code and pretrained models are available at https://github.com/minhdang050806/ A-nnU-Net-based-asymmetric-supervision-strategy.

[55] Signal or Noise? Auditing Rotation-Induced Saliency Drift in Medical and Aerial Imaging cs.CV | cs.AIPDF

Khawaja Murad ul Hassan, Mehran Ebrahimi

TL;DR: 本文研究了在医学影像和航拍图像等无标准方向的领域中，Grad-CAM等事后显著性图在输入旋转时发生漂移的问题。通过分析CAM操作符的每个阶段，发现漂移主要源于空间激活张量的变化，而非通道权重。作者提出了EquiGrad-CAM方法，通过聚合多个旋转视图的显著性图来提高旋转等变性，并在多个数据集和模型上验证了其有效性。

Details

Motivation: 在无标准方向的领域（如组织病理学和航拍图像）中，显著性图作为模型决策证据的可靠性受到挑战，因为输入旋转会导致热图漂移，即使预测结果不变。本文旨在探究这种漂移是真实信号还是CAM操作符引入的噪声。

Result: 在ImageNet-1K验证集上，EquiGrad-CAM相比单视图Grad-CAM将旋转等变性分别提升了36.0%（ResNet-50）、87.5%（VGG-16）和247%（ViT-B/16）。该方法在不重新训练的情况下优于旋转增强训练，并将零样本CLIP的等变性提升了145%，在PatchCamelyon和RESISC45数据集上产生了旋转一致的解释。

Insight: 创新点在于通过测量CAM操作符每个阶段的等变性而非从网络输出推断，揭示了漂移源于空间激活张量而非通道权重。提出的EquiGrad-CAM是一种无需训练的包装器，通过逆旋转和对齐多个视图的显著性图进行平均，显著提高了旋转等变性。其副产品PEUM可免费评估解释的可重复性。

Abstract: Post-hoc saliency maps such as Grad-CAM are increasingly used to audit why a deployed vision model made a decision, yet the heatmap drifts when the input is rotated, even when the prediction is unchanged. In domains with no canonical orientation, such as histopathology and aerial imagery, this undermines using saliency as evidence. We ask whether that drift is faithful signal or noise introduced by the CAM operator, and answer it by measuring equivariance at every stage of the operator rather than inferring it from the network’s output. The instability is not where one would guess: the channel weights are the most rotation-stable stage, and on ResNet-50 exactly stable, because a GAP+linear head makes the class gradient field spatially constant. What moves is the spatial activation tensor, and the classifier’s own pooling discards that movement. A causal test confirms the consequence: occluding the pixels whose saliency drifts costs the model less than occluding random pixels, at either orientation. The drift is carried by degrees of freedom the classifier throws away, which is what makes removing it faithful rather than destructive. EquiGrad-CAM is a training-free wrapper that takes T rotated views, inverse-rotates each view’s saliency into a common canonical frame, and averages. On the full ImageNet-1K validation set it raises equivariance over single-view Grad-CAM by +36.0% (ResNet-50), +87.5% (VGG-16) and +247% (ViT-B/16); a scale-matched ablation isolates alignment before averaging, not the locus of aggregation, as the driver. It beats rotation-augmented training without retraining, lifts zero-shot CLIP by +145%, and yields rotation-consistent explanations on PatchCamelyon and RESISC45. Its by-product PEUM ranks explanations by how reproducible they are, at no cost beyond the views already taken. Code: https://github.com/Khawaja-Murad/EquiGrad-CAM

[56] InfraPatch: Cross-Task Targeted Grayscale Patch Attacks on Infrared-Adapted Vision-Language Models cs.CV | cs.AIPDF

Chengyin Hu, Dingyi Lu, Jiaju Han, Xiang Chen, Weiwen Shi

TL;DR: 本文提出InfraPatch，一种针对红外视觉语言模型（IR-VLM）的白盒、单实例目标性灰度补丁攻击框架。该方法在约5%的局部区域预算内优化单通道补丁，结合代理引导的放置策略和任务自适应的语义目标，在图像分类、图像描述和二元视觉问答任务中成功诱导目标行为。

Details

Motivation: 红外视觉语言模型在低能见度条件下展现出潜力，但其对目标性对抗攻击的鲁棒性尚未被充分研究。现有对抗补丁方法主要针对RGB模型或单一任务，未能探究局部扰动能否在IR-VLM中引发特定语义目标。

Result: 在由DiffV2IR生成的300张合成红外风格图像（基于固定30类COCO子集）上评估了10个红外适应模型变体，使用干净条件下的目标成功率标准。InfraPatch在10个变体上的目标攻击成功率从86.00%到100%不等；在CLIP和BLIP-2上，代理位置搜索相比优化随机放置分别提升了6.67和10.33个百分点。

Insight: 创新点在于针对IR-VLM设计跨任务的目标性灰度补丁攻击框架，结合代理引导放置与任务自适应语义目标；客观分析表明，该方法揭示了不同架构和任务格式在漏洞上的显著差异，强调了红外多模态系统需加强鲁棒性评估。

Abstract: Infrared vision-language models (IR-VLMs) have emerged as a promising paradigm for multimodal perception under low-visibility conditions, yet their robustness to targeted adversarial attacks remains poorly understood. Existing adversarial patch methods mainly study RGB-based models or a single downstream task and do not characterize whether localized perturbations can induce an intended semantic target in IR-VLMs. We propose InfraPatch, a white-box, per-instance framework for targeted digital grayscale patch attacks against IR-VLMs. InfraPatch optimizes a compact single-channel patch within an approximately 5% local-area budget, combines proxy-guided placement with task-adaptive semantic objectives, and induces target behaviors in image classification, image captioning, and binary visual question answering. We evaluate ten infrared-adapted model variants on 300 synthetic infrared-style images generated by applying DiffV2IR to a fixed 30-category COCO subset, using clean-conditioned targeted success criteria. InfraPatch achieves targeted attack success rates from 86.00% to 100% across the ten variants. On CLIP and BLIP-2, proxy location search improves success by 6.67 and 10.33 percentage points over optimized random placement, respectively; LLaVA-1.5 remains saturated near 100% under both settings. Patch-area and objective ablations further expose substantial differences in vulnerability across architectures and task formats. These results show that small grayscale patches can inject chosen target semantics across IR-VLM families under a controlled digital threat model, motivating stronger robustness evaluation for infrared multimodal systems.

[57] RouteGraph-Mona: Confusion-Aware Routing Fine-Tuning for Mineral Image Classification cs.CV | cs.AIPDF

Jierui Li, Zhiyuan Qi, Hao Zhu, Yufan Liu, Jixian Liu

TL;DR: 本文提出RouteGraph-Mona，一种基于Mona视觉适配器的轻量级路由空间正则化方法，用于矿物图像分类。该方法将Mona的静态多尺度聚合替换为样本自适应路由，并通过类别路由锚点和混淆加权边距对路由签名进行正则化，以更好地处理类内外观差异大和类间视觉相似度高的问题。

Details

Motivation: 矿物图像分类因类内差异大和类间视觉相似度高而具有挑战性。现有Mona适配器静态聚合多尺度响应，无法适应样本特定的尺度偏好，也难以处理视觉相似类别间的模型混淆问题。

Result: 在三个公共矿物图像数据集和两种视觉骨干网络上的实验表明，RouteGraph-Mona在平均准确率上持续优于Mona，并与代表性的微调方法和矿物图像分类基线保持竞争力。

Insight: 创新点在于引入样本自适应路由机制构建紧凑的路由空间，并设计基于类别的路由锚点和混淆加权边距进行正则化，从而在参数高效的前提下，动态捕捉样本的尺度偏好并增强视觉相似类别在路由空间中的可分离性。

Abstract: Mineral image classification is important for geological exploration and resource development, but it remains challenging due to substantial intra-class variations in appearance and high inter-class visual similarity. Multi-cognitive Visual Adapter (Mona) is a vision-oriented parameter-efficient adapter that adapts pre-trained visual models by tuning only a few parameters. However, Mona statically aggregates responses from multiple scales, limiting its ability to accommodate sample-specific scale preferences and model confusion among visually similar mineral categories. To address this issue, we propose \textbf{RouteGraph-Mona}, a lightweight route-space regularization method built on Mona. Specifically, we replace Mona’s static multi-scale aggregation with sample-adaptive routing. The resulting branch-selection behavior defines a compact routing space that captures each image’s scale preferences. We then regularize the resulting routing signatures with class-wise route anchors and confusion-weighted margins. The route anchors encourage class-consistent routing patterns, while the margins promote greater separation between visually similar categories in the routing space. Experiments on three public mineral image datasets with two visual backbones show that RouteGraph-Mona consistently outperforms Mona in mean accuracy and remains competitive with representative fine-tuning methods and mineral image classification baselines.

[58] Diffusion-Encoding Gaussian Field for Joint k-q dMRI Reconstruction cs.CVPDF

Zhibo Chen, Yajuan Huang, Yu Guan, Qiuyun Fan, Dong Liang

TL;DR: 本文提出了一种基于高斯场的自监督联合k-q空间扩散MRI重建方法，通过共享的3D高斯基元耦合空间与扩散方向信息，从欠采样的k空间数据中优化重建缺失方向的扩散加权图像。

Details

Motivation: 现有联合k-q重建方法未能充分利用解剖结构的共享性与扩散编码依赖的信号变化之间的互补性，导致空间误差被误认为角度变化并传播到未观测方向。

Result: 在多个加速设置下的三个HCP扩散壳数据上实验表明，该方法在缺失方向DWI重建、张量衍生指标和主扩散方向估计方面均取得一致提升。

Insight: 创新点在于引入主体特定的空间-角度高斯场，通过连续q条件张量残差响应耦合相邻空间区域与扩散方向，实现无全采样目标或保留方向监督的自监督优化。

Abstract: Diffusion MRI requires repeated k-space acquisitions over multiple diffusion-encoding directions, making acquisition time dependent on both spatial and angular sampling. Existing joint k-q methods either associate directional parameters with fixed voxels or separate spatial reconstruction from angular completion. However, diffusion-weighted images acquired under different directions share the same anatomical organization, while their local signal intensities vary with diffusion encoding. Existing formulations do not fully exploit the complementarity between shared anatomy and direction-dependent signal variation. Consequently, residual spatial errors may be misinterpreted as genuine angular variation and propagated to unobserved directions. We propose a subject-specific spatial-angular Gaussian field for self-supervised joint k-q dMRI reconstruction. Shared 3D Gaussian primitives provide local spatial support, with each primitive carrying a continuous q-conditioned tensor-residual response. The signal at each location is synthesized from multiple overlapping primitive responses, coupling neighboring spatial regions and diffusion directions. The field is progressively optimized from undersampled k-space measurements of observed directions, without fully sampled targets or held-out-direction supervision. Experiments on three HCP diffusion shells under multiple acceleration settings demonstrated consistent improvements in missing-direction DWI reconstruction, tensor-derived metrics, and principal diffusion orientation estimation.

[59] T2LSC-Bench: Benchmarking Localized Semantic Control in Text-to-Image Generation cs.CVPDF

Yan Wang, Xinyi Hou, Weiguo Lin, Junjun Si, Siwei Ma

TL;DR: 本文提出了T2LSC-Bench，一个用于评估文本到图像生成模型中局部语义控制能力的诊断性基准。该基准通过因子化设计（如语义关系、场景开放性等）构建了包含50个种子主体和大量提示案例的数据集，并采用结合OCR-VLM文本验证和结构化VLM语义判断的双分支协议来量化文本渲染准确性和语义泄漏问题。实验表明，在压力测试下，语义泄漏率显著上升，而准确的文本渲染并不能保证目标文本语义被局部约束。

Details

Motivation: 现有文本到图像模型在渲染显式文本方面能力增强，但在产品标签、标识和界面设计等应用中，需要在指定文本承载区域内渲染目标文本，同时不改变预定义的主体身份或周围场景语义。现有基准主要评估可读性、拼写准确性和布局，而未能充分评估目标文本语义泄漏到非文本视觉内容中的问题。

Result: 在六个模型上评估了7,160张图像。在压力测试条件下，语义泄漏率（SLR）从1.2%增加到18.1%，条件语义泄漏率（cSLR）从1.3%增加到18.2%，而文本在锚点准确率（TAA）仅从91.4%微降至90.9%。抗泄漏提示技术将SLR从16.6%降低到8.4%，且未降低渲染准确性。在420张图像上的人工验证显示自动标注与人工裁定标注具有强一致性。

Insight: 论文的创新点在于首次系统性地定义和量化了文本到图像生成中的“目标文本关联语义泄漏”问题，并构建了一个专门用于诊断该问题的可控基准（T2LSC-Bench）。其因子化设计和结合OCR与VLM的双分支评估协议为深入理解模型在局部语义控制上的失败模式提供了新工具，揭示了文本渲染准确性与语义局部约束之间的脱节，并为通过提示工程缓解泄漏问题提供了实证依据。

Abstract: Recent text-to-image models have become increasingly capable of rendering explicit text, but reliable localized text control requires more than generating the correct string. In applications such as product labeling, signage, and interface design, target text should be rendered within a designated text-bearing region without altering the predefined subject identity or surrounding scene semantics. We refer to violations of this requirement as target-text-associated semantic leakage, in which target-text semantics are expressed through non-textual visual content beyond the designated anchor. Existing visual-text benchmarks primarily evaluate readability, spelling accuracy, and layout, leaving this form of semantic leakage largely unexamined. We introduce T2LSC-Bench, a controlled diagnostic benchmark comprising 50 seed subjects and 1,200 prompt cases per model, yielding 7,160 evaluated images across six models. Its factorized design varies semantic relation, scene openness, prompt mode, and language. A dual-branch protocol combines OCR-VLM text verification with structured VLM semantic judgments to measure Text-at-Anchor Accuracy (TAA), Semantic Subject Preservation (SSP), Semantic Leakage Rate (SLR), and Conditional Semantic Leakage Rate (cSLR). Under stress-test conditions, SLR increases from 1.2% to 18.1% and cSLR from 1.3% to 18.2%, whereas TAA decreases only from 91.4% to 90.9%. Anti-leakage prompting reduces SLR from 16.6% to 8.4% without degrading rendering accuracy. Human validation on 420 images shows strong agreement between automatic and adjudicated annotations. These results show that accurate text rendering does not guarantee local containment of target-text semantics.

[60] VoRTeC: Taming Foundation Flow for One-step Real time Video Compression cs.CV | cs.AIPDF

Yichong Xia, Qinhong Wu, Qinhong Wu, Jinpeng Wang, Zeyuan Chen

TL;DR: VoRTeC是一个基于基础流模型（Wan2.1）的视频压缩框架，旨在解决超低码率视频压缩中传统神经方法导致的模糊伪影以及基于扩散生成方法存在的高解码延迟和时序不一致性问题。它通过紧凑编码潜在视频表示、预测压缩表示在流轨迹上的位置并整合多尺度先验，实现了无需访问流匹配网络参数或梯度的单步解码，同时通过尾帧重用和先验缓存保持帧组间一致性。

Details

Motivation: 解决超低码率视频压缩中传统神经方法引入模糊伪影，以及基于扩散的生成方法解码延迟高、时序一致性差的问题。

Result: 在实验中，该方法比先前的基于扩散的方法减少了58%的比特消耗，解码速度提升了3到197倍，在720p分辨率下达到13 FPS，480p下达到32 FPS的解码速度。

Insight: 创新点在于利用基础流模型（Wan2.1）的生成视频流先验，通过紧凑编码和流轨迹预测实现高效的单步解码，同时结合尾帧重用和先验缓存机制来保证时序一致性，从而在保持高感知保真度的同时大幅提升解码效率。

Abstract: Ultra-low bitrate video compression still faces critical challenges: traditional neural video compression inevitably introduces blurring artifacts, while diffusion-based generative video compression suffers from excessive decoding latency and poor temporal consistency. To address these issues, we propose $\mathtt{VoRTeC}$, a Video Compression framework built upon a foundational flow model (Wan2.1). By compactly encoding latent video representations, predicting the positions of compressed representations along flow trajectories, and integrating multi-scale priors, $\mathtt{VoRTeC}$ enables the compressor to harness generative video flow priors effectively. Without accessing the parameters or gradients of flow matching networks, our framework achieves one-step decoding and reconstructions with high perceptual fidelity. Meanwhile, we maintain consistency across frame groups via tail-frame reuse and prior caching. Extensive experiments demonstrate that our method reduces bit consumption by 58% compared to prior diffusion-based approaches, with decoding speed boosted by 3 to 197 times: $\mathtt{VoRTeC}$ achieves a decoding speed of 13 FPS at 720p and 32 FPS at 480p.

[61] If It Moves, Radar Knows: A Physics-Aware Radar Transformer for Class-Agnostic Moving-Object Detection cs.CVPDF

Yinghao Sun, Shuguang Li, Jinliang Shao, Tieshan Li

TL;DR: 本文提出了一种名为物理感知雷达变换器（PART）的纯雷达检测器，用于类别无关的移动物体检测。该方法利用雷达的多普勒运动线索，通过多普勒感知查询初始化（DAQI）和物理引导交叉注意力（PGCA）等机制，在稀疏、嘈杂的雷达回波中预测移动物体的存在置信度、代表表面点和二维地面速度。

Details

Motivation: 基于封闭集标注训练的检测器可能遗漏训练分类学之外的罕见移动物体。汽车雷达提供与类别无关的多普勒运动线索，且受恶劣光照和天气影响较小，但其稀疏、嘈杂的回波阻碍了基于类别的3D框检测。当难以恢复完整几何形状时，表面位置和速度对于运动推理和碰撞避免仍然有用。

Result: 在nuScenes数据集上，仅110万参数的PART实现了0.8827的类别无关平均精度（CA-AP）、0.3188米的平均表面平移误差（mASTE）和0.8084米/秒的平均速度误差（mAVE）。在标准评估中排除的罕见且与安全相关的类别上，其召回率达到0.9203，并且在夜间、雨天和严重遮挡下仍保持有效。

Insight: 创新点包括：1）用输入依赖的多普勒感知查询初始化（DAQI）替代场景无关的学习查询，通过位置和速度聚类雷达回波，简化稀疏场景中的查询-对象分配；2）物理引导交叉注意力（PGCA）将径向多普勒一致性和雷达截面积（RCS）纳入查询-点关联；3）不确定性感知监督随机掩码真实对象并为模糊的雷达支持查询分配软存在目标，减少对详尽标注的依赖。这些机制增强了模型在稀疏雷达数据下的物理一致性和鲁棒性。

Abstract: Detectors trained on closed-set annotations can miss rare moving objects outside the training taxonomy. Automotive radar provides category-independent Doppler motion cues and is less affected by adverse illumination and weather, but sparse, noisy returns hinder class-aware 3D box detection. Surface location and velocity remain useful for motion reasoning and collision avoidance when full box geometry is difficult to recover. We present the Physics-Aware Radar Transformer (PART), a fully sparse radar-only detector that predicts existence confidence, a representative surface point, and 2D ground-plane velocity for each moving-object hypothesis. Doppler-Aware Query Initialization (DAQI) replaces scene-independent learned queries with input-dependent proposals by clustering radar returns in position and velocity, easing query-object assignment in sparse scenes. Physics-Guided Cross-Attention (PGCA) incorporates radial-Doppler consistency and radar cross section (RCS) into query-point association. Uncertainty-aware supervision randomly masks ground-truth objects and assigns soft existence targets to ambiguous radar-supported queries, reducing reliance on exhaustive annotations. With only 1.1 million parameters, PART achieves a class-agnostic average precision (CA-AP) of 0.8827, a mean average surface translation error (mASTE) of 0.3188 m, and a mean average velocity error (mAVE) of 0.8084 m/s on nuScenes. It attains 0.9203 recall on rare and safety-relevant categories excluded from the standard evaluation and remains effective at night, in rain, and under severe occlusion. Inspection of apparent false positives shows that some predictions correspond to moving objects absent from the nuScenes annotations. Code and pretrained model weights will be publicly available at https://github.com/sunyinghao-uestc/PART.

[62] YesTrack: Referring Multi-Object Tracking via MLLM-based Yes/No Verification cs.CVPDF

Quansheng Hu, Qin Sun, Qiansen Dai, Jin Ding, Wan Zhang

TL;DR: 本文提出YesTrack，一种基于多模态大语言模型（MLLM）进行是/否验证的两阶段指代多目标跟踪方法。它将指代任务重新定义为判别任务，直接利用MLLM进行验证，避免了显式文本生成，并引入了两种轻量级时序一致性约束来提升验证的可靠性和效率。该方法在Refer-KITTI和Refer-KITTI-V2基准上显著优于现有SOTA方法，且保持了高效率。

Details

Motivation: 现有方法主要将MLLM用作字幕生成器，需要外部模块进行最终决策，这不仅引入了额外延迟，也严重未充分利用MLLM固有的视觉-语言对齐能力。本文旨在解决这些限制，更直接地利用MLLM进行判别。

Result: 在Refer-KITTI和Refer-KITTI-V2基准上的实验表明，YesTrack显著优于现有的最先进方法，同时保持了高效率，即使使用最小的Qwen3-VL变体实现。

Insight: 核心创新在于将指代跟踪重新定义为判别性的是/否验证任务，直接利用MLLM进行决策，避免了显式文本生成。此外，引入了两种轻量级时序一致性约束（TCP和TRP）来增强MLLM验证的可靠性和效率，并展示了该判别范式在通用多目标跟踪（YesTrack-MOT）上的泛化潜力。

Abstract: Referring multi-object tracking (RMOT) aims to track every instance in a video that matches a given language expression. Despite the recent integration of multimodal large language models (MLLMs) to enhance generalization, existing methods predominantly relegate them to the role of caption generators, necessitating external modules for final decision-making. This paradigm not only introduces extra latency but also severely underutilizes the inherent vision-language alignment capabilities of MLLMs. To address these limitations, we propose YesTrack, a novel two-stage RMOT method that reformulates referring as a discriminative task, directly leveraging MLLMs for Yes/No verification without explicit text generation. To further enhance the reliability and efficiency of this MLLM-based verification, we introduce two lightweight temporal consistency constraints: Temporal Confidence Prior (TCP) and Temporal Reference Propagation (TRP). We further validate the generality of this discriminative paradigm by proposing YesTrack-MOT, a straightforward yet highly effective instantiation for generic multi-object tracking (MOT). Experiments on Refer-KITTI and Refer-KITTI-V2 show that YesTrack significantly outperforms existing state-of-the-art methods while maintaining high efficiency, even when implemented with the smallest variant of Qwen3-VL. Code is released at https://github.com/ggbondrighthere24/YesTrack.

[63] Towards Zero-Shot Transfer Across Embodiments For Driving VLAs cs.CVPDF

Caio Azevedo, Stefano Sabatini, Sascha Hornauer, Fabien Moutarde

TL;DR: 本文研究了自动驾驶中视觉-语言-动作模型的多数据集训练问题，提出了BEV-Forcing辅助目标，通过将鸟瞰图模型中的地面物体布局信息注入VLA主干网络，以提升模型在分布内和分布外的性能。研究发现，在训练相机配置较少时该辅助任务效果显著，但随着训练数据多样性增加，其收益会减弱。

Details

Motivation: 当前自动驾驶VLAs主要在单个数据集上训练，缺乏对未见数据集和相机配置的零样本迁移能力，且简单地增加训练数据并不能保证性能提升，因此需要研究能促进跨具身（跨数据集/相机配置）泛化的方法。

Result: 在少量相机配置上训练时，BEV-Forcing能同时提升分布内和分布外性能；但随着训练具身数量增加，该辅助任务的收益减弱。论文强调了在评估新技术时需考虑数据规模扩展的影响。

Insight: 创新点在于引入BEV-Forcing作为辅助任务，通过共享的BEV空间接口统一物体位置表示，以促进跨具身泛化。客观来看，论文揭示了辅助任务的有效性可能随训练数据多样性增加而递减，这为未来研究提供了重要的规模扩展视角。

Abstract: Vision-Language-Action models (VLAs) have shown strong potential in autonomous driving by leveraging multimodal pretraining for instruction following, visual reasoning, and scene-level generalization. In robotic manipulation, scaling VLA fine-tuning across multiple robot setups–especially when unifying representations across embodiments–has been shown to improve in-dataset performance and cross-embodiment generalization; in autonomous driving, however, VLAs remain largely trained on individual datasets and are rarely evaluated for zero-shot transfer to unseen datasets and camera rigs; furthermore naively adding more datasets to the training data does not necessarily lead to better performance within seen embodiments. To address these problems, we study multi-dataset training for the driving task and BEV-Forcing, an auxiliary objective that transfers ground-plane object-layout information from a specialized Bird’s-Eye-View model into the VLA backbone. By encouraging the model to represent object position through a shared BEV spatial interface, we show that an auxiliary task such as BEV-Forcing can improve both in-distribution and out-of-distribution performance when training on a small number of camera rigs. As the number of training embodiments increases, however, the benefits of the auxiliary task are reduced; we present this as evidence that new techniques in the literature may see their benefits diminish when simply scaling up training diversity, which motivates presenting results taking into account data scaling.

Kun-Yang Yu, Yingzhe Li, Hongyu Xu, Shi-Yu Tian, Zhi Zhou

TL;DR: 本文提出LookStep框架，用于高效视觉语言导航（VLN），通过语言中心未来状态建模和事件驱动滚动内存，减少训练数据需求和计算开销，在VLN-CE任务上实现更高的成功率和内存效率。

Details

Motivation: 现有VLN方法依赖大量训练数据，并使用认知地图、历史帧或外部3D工具维持状态，导致计算和内存开销高，本文旨在实现资源高效的导航。

Result: 在VLN-CE任务中，LookStep在相同训练设置下优于现有方法，在R2R-CE Val-Unseen上达到49.7%的成功率，同时内存效率更高、数据使用更少。

Insight: 创新点包括语言中心未来状态建模，用语言标签生成粗粒度导航进度和未来状态，以及事件驱动滚动内存，自主决定是否将观察写入有界内存，减少冗余信息处理。

Abstract: Vision-Language Navigation (VLN) requires an embodied agent to follow natural-language instructions in unseen environments. Recent progress has been largely driven by Multimodal Large Language Models (MLLMs). Existing methods follow a next-step action prediction paradigm, supervising only the expert action, which requires a high quantity of data for training. They also rely on cognitive maps, accumulated historical frames, or external 3D tools to maintain states, leading to high computational and memory overhead. To realize resource efficiency VLN, we propose LookStep, a unified end-to-end framework that combines Language Centric Future State Modeling and Event Driven Rolling Memory that uses language labels to generate coarse-grained navigation progress and future states for each candidate action, while autonomously deciding whether to write each observation into a bounded rolling memory with a semantic role. We validate LookStep empirically. On VLN-CE tasks, LookStep outperforms existing methods under the same training settings, achieving a 49.7% success rate on R2R-CE Val-Unseen with better memory efficiency and less data usage. Code and model is available at https://github.com/kunyang-YU/LookStep.

[65] CA-OPD: Confidence-Aware On-Policy Distillation for Structured Visual Prediction cs.CVPDF

Menghao Li, Linjie Mu, Yin Wang, Haotian Hu, Yannian Gu

TL;DR: 本文提出了CA-OPD（置信度感知的在线策略蒸馏）框架，用于解决自回归视觉语言模型在结构化视觉预测任务中容易产生误差累积的问题。该框架通过教师模型的置信度来选择性纠正学生模型在训练过程中生成的不可靠预测，并采用从严格到宽松的调度策略逐步将生成控制权转移给学生，同时根据干预决策调整知识蒸馏的监督信号。

Details

Motivation: 自回归视觉语言模型统一了异构感知任务，但极易产生误差累积。现有的在线策略蒸馏方法通过让学生模型在自己的生成序列上训练来弥合训练-推理的不匹配，但训练早期学生模型不可靠的预测会破坏生成轨迹并降低教师监督的质量。

Result: 在GUI定位和光学字符识别的多教师设置下进行评估，CA-OPD显著提升了Qwen3.5-0.8B基线模型在所有六个目标基准测试上的性能，例如在ScreenSpot-Pro上提升了9.50分，在OCRBench-v2 English上提升了6.72分。控制研究表明，性能提升依赖于干预位置、渐进式的生成控制以及与干预对齐的监督，而非仅仅干预频率。

Insight: 核心创新在于将可靠的生成序列构建与自适应监督相结合。具体包括：利用教师置信度进行选择性干预，通过严格到宽松的调度实现控制权渐进转移，以及根据干预决策（纠正或保留学生预测）来调整监督信号（分别使用教师预测的直接交叉熵或完整预测分布），从而更精细地引导知识蒸馏过程。

Abstract: Autoregressive vision language models unify heterogeneous perception tasks but are highly susceptible to compounding errors. On-policy distillation (OPD) bridges the training-inference mismatch by training students on their own rollouts. However, unreliable student predictions, especially early in training, can derail the trajectory and degrade the quality of teacher supervision. While recent interleaved distillation methods allow the teacher to verify and replace student tokens, they primarily rely on rigid ranking metrics rather than exact teacher confidence, and they overlook how intervention decisions can inform token-level supervision. To address this, we introduce Confidence-Aware On-Policy Distillation (CA-OPD), a framework that couples reliable rollout construction with adaptive supervision. CA-OPD utilizes teacher confidence to selectively correct unreliable student transitions, gradually transferring rollout control to the student via a strict-to-relaxed schedule. Crucially, CA-OPD aligns knowledge transfer with these intervention decisions: corrected positions receive direct cross-entropy supervision from the teacher’s prediction, while retained positions benefit from the teacher’s full predictive distribution. Evaluated in a multi-teacher setting for GUI grounding and optical character recognition, CA-OPD substantially improves the Qwen3.5-0.8B baseline across all six target benchmarks, including gains of $9.50$ points on ScreenSpot-Pro and $6.72$ points on OCRBench-v2 English. Controlled studies further show that the gains depend on intervention placement, progressive rollout control, and intervention-aligned supervision, rather than intervention frequency alone.

[66] TempoGround: State-Aware Streaming Visual Grounding with Vision-Language Models cs.CVPDF

Leqian Ding, Junning Qiu, Manwen Yang, Yu Guo, Fei Wang

TL;DR: 本文提出了TempoGround，一个基于视觉语言模型（VLM）的原生框架，用于解决流式输入下的视觉定位问题。该框架通过检测跨帧物体对应关系并显式建模物体存在状态，以应对身份漂移、跨帧不一致和部分遮挡下的定位脆弱性。核心在于一个由状态感知跨帧对应关系引导的课程预测机制，以及一个名为流式定位强化（SGR）的优化方法，通过可验证的奖励函数联合强化持久定位和时间一致性预测。

Details

Motivation: 现有方法在单帧和基于视频的视觉定位上取得了进展，但在流式输入下仍存在身份漂移、跨帧不一致和部分遮挡下定位脆弱的问题。本文旨在解决这些挑战，实现流式输入下准确且一致的视觉定位。

Result: 在多个具有挑战性的基准测试上对因果流式输入进行评估，TempoGround在2D定位指标F1_2D@0.5和F1_2D@0.95上平均分别提升了4.4和0.5，在3D定位指标F1_3D@0.25和AP_3D上分别提升了6.2和7.5。

Insight: 创新点在于提出了一个状态感知的跨帧对应关系引导的课程预测机制，将物体存在状态（进入、持续、离开）建模为预测流程的一部分。此外，引入了流式定位强化（SGR），使用可验证的定位、身份和一致性奖励进行优化，弥补了仅靠token级监督的不足，共同强化了持久定位和时间一致性。

Abstract: Visual grounding maps language referents to spatial targets and is central to open-vocabulary perception with vision-language models. Existing methods have made substantial progress on single-frame and video-based visual grounding, yet under streaming inputs they still suffer from identity drift, cross-frame inconsistency, and fragile localization under partial occlusion. To address these issues, we present TempoGround, a VLM-native framework that detects cross-frame object correspondence and explicitly models object presence states, thereby enabling accurate and consistent visual grounding under streaming inputs. The key is a curriculum prediction mechanism guided by state-aware cross-frame correspondence: TempoGround resolves 2D instance association, predicts whether each object newly enters, continues in, or leaves the view, decodes the 2D box, and then lifts it to a camera-frame 3D box. As token-level supervision alone cannot capture the geometric objectives of streaming grounding, we further introduce Streaming Grounding Reinforcement (SGR), which optimizes TempoGround with verifiable Grounding, Identity, and Consistency rewards, jointly reinforcing persistent localization and temporally consistent predictions. We carefully design a three-stage training strategy and train TempoGround on large-scale data. We evaluate visual grounding under causally streaming inputs on multiple challenging benchmarks: TempoGround improves F1_2D@0.5 and F1_2D@0.95 by 4.4 and 0.5 on average, and F1_3D@0.25 and AP_3D by 6.2 and 7.5, respectively. These results demonstrate that TempoGround provides a practical foundation for visual grounding under streaming inputs.

[67] Adapting a Foundation Model for Lunar Surface Height Estimation cs.CVPDF

Patrick Bauer, Marius Schwinning, Melanie Siegel, Andreas Weinmann, Hichem Snoussi

TL;DR: 本文提出了一种基于深度学习的月球表面相对高度估计方法，通过微调预训练的零样本深度估计模型Depth Anything V2 (DAV2)，使其适应月球表面地形，以提供危险地形定位所需的额外高度信息。

Details

Motivation: 为支持未来月球着陆任务，需要精确的高度信息来识别危险地形。传统方法依赖2D图像数据，而现有深度学习方法（如DAV2）未经领域适应，在月球表面可能表现不佳。

Result: 使用公开的立体摄影测量（SPG）生成的DEM数据进行微调后，模型性能相比零样本DAV2有显著提升，使其成为可靠的月球表面相对深度估计器。

Insight: 创新点在于将通用视觉基础模型（DAV2）通过领域特定的微调策略，成功迁移到月球遥感这一特殊场景，证明了领域适应在提升基础模型专业任务性能上的有效性。

Abstract: Digital elevation models (DEMs) can provide accurate height information, making it invaluable for analyzing the lunar surface. As the European Space Agency (ESA) prepares for future lunar missions that aim to land on the Moon, a precise method for height estimation will be essential for hazardous terrain that could endanger the landing approach. Traditional approaches to generate DEMs from imagery, such as shape from shading (SfS) and stereophotogrammetry (SPG) have been proven highly valuable for this task. However, due to advancements in machine learning, especially computer vision, the focus has shifted towards monocular depth estimation via deep learning. The lunar surface is covered by rocks and craters, and classic hazard detection methods rely solely on 2D image data. Our goal is to address this issue by developing a relative lunar surface height estimator that can provide additional information for hazard localization. In this letter, we present a methodology that builds on the well-known zero-shot relative depth estimation model Depth Anything V2 (DAV2). Other works have been using it as a state-of-the-art comparison for their proposed lunar DEM estimation method, but without adaptations to the target domain. Thus, it may underperform. Therefore, we propose a fine-tuning strategy with publicly available SPG-derived DEM data of the lunar surface. Our results demonstrate a significant improvement in performance compared to the zero-shot model, effectively transforming DAV2 into a reliable relative depth estimator of the lunar surface.

[68] UnCapsTSR: An Unsupervised Transformer-based Image Super-Resolution Approach for Capsule Endoscopy Images cs.CVPDF

Anjali Sarvaiya, Shubh Kawa, Lalit Agrawal, Jagrit Joshi, Kishor Upla

TL;DR: 本文提出了一种名为UnCapsTSR的无监督Transformer-GAN框架，用于提升无线胶囊内窥镜（WCE）图像的空间分辨率。该方法无需真实低分辨率-高分辨率图像对，通过引入双边全变分损失确保空间连续性，并在新的Kvasir Capsule数据集上进行训练。实验表明，该方法在多个非参考指标和提出的EndoQM指标上优于现有无监督超分辨率方法。

Details

Motivation: 无线胶囊内窥镜（WCE）因设备尺寸和无线传输限制，生成的图像分辨率较低，影响诊断。现有方法通常需要成对的LR-HR数据或明确的退化模型估计，这在真实WCE场景中难以获取。

Result: 在KID和GIANA数据集上的泛化实验表明，该方法在NIQE、BRISQUE、PIQE等非参考指标以及新提出的EndoQM指标上均优于当前最先进的无监督超分辨率方法。统计评估显示，从LR到SR，EndoQM指标在评估数据集上提升了40%到80%。

Insight: 创新点包括：1) 提出了一种无需真实LR-HR配对和显式退化估计的无监督Transformer-GAN框架；2) 引入了双边全变分（BTV）损失来增强SR图像的空间连续性；3) 构建了新的WCE SR训练数据集并提出了领域专用的无参考评估指标EndoQM。

Abstract: Wireless Capsule Endoscopy (WCE) captures and streams video while passing through a patient’s Gastrointestinal (GI) tract and is used to examine its irregularities. Although advantageous over conventional endoscopy, WCE suffers from limitations related to capsule size and wireless transmission, resulting in images with coarser resolution. This work presents UnCapsTSR, an unsupervised transformer-based Generative Adversarial Network (GAN) framework for improving the spatial resolution of Low-Resolution (LR) WCE images. The proposed method accomplishes SR without explicit degradation estimation of real-world LR data and eliminates the need for true LR-HR pairs. UnCapsTSR employs a Bilateral Total Variation (BTV) loss to ensure spatial continuity in SR images. A newly curated dataset from the Kvasir Capsule dataset is also presented for training WCE SR models. Generalizability is validated on KID and GIANA datasets that are not used during training. A new non-reference metric, Endoscopy Quality Metric (EndoQM), is introduced for quantitative evaluation of domain-specific WCE data. Experiments demonstrate consistent improvement over state-of-the-art unsupervised SR approaches using NIQE, BRISQUE, PIQE, and EndoQM. Statistical evaluation shows 40 to 80 percent improvement in EndoQM from LR to SR across the evaluated datasets.

[69] Blending Concepts: Benchmarking Visual Metaphor Generation in Text-to-Image Models cs.CV | cs.AIPDF

Chuer Chen, Zichen Wang, Yi He, Zhengxi Yu, Nan Cao

TL;DR: 本文提出了首个评估文本到图像（T2I）模型生成视觉隐喻能力的基准测试VMetaphor-Bench。该基准包含1500个从真实创意图像中筛选的视觉隐喻样本，并设计了一个结合多项选择题和维度评分的混合评估框架。通过对11个代表性T2I模型的广泛评估，发现即使是性能最强的专有模型在视觉隐喻生成的关键方面（如组合结构和跨域映射）仍存在困难。

Details

Motivation: 当前T2I模型在忠实渲染指定对象和属性方面已取得显著成功，但其生成视觉隐喻（即通过结合两个不同领域的元素来传达抽象概念的图像）的能力尚未得到充分研究和评估。本文旨在填补这一空白。

Result: 在VMetaphor-Bench基准上对11个代表性T2I模型进行了广泛评估。结果表明，即使是性能最强的专有模型在视觉隐喻生成的关键方面（如组合结构和跨域映射）也表现不佳，凸显了该任务是未来T2I研究的重要前沿。

Insight: 论文的创新点在于首次构建了专门用于评估T2I模型视觉隐喻生成能力的基准数据集和评估框架。客观来看，其提出的混合评估范式（结合MLLM作为评判者的多项选择题协议和基于维度的评分协议）为评估生成模型的复杂语义理解能力提供了新思路，强调了组合性和跨域映射是当前模型能力的薄弱环节。

Abstract: Text-to-image (T2I) models have achieved remarkable success at faithfully rendering specified objects and attributes, yet their ability to produce visual metaphors, images that convey abstract ideas by combining elements from two distinct domains, remains largely unexamined. To bridge this gap, we introduce VMetaphor-Bench, the first benchmark for evaluating visual metaphor generation in T2I models. It comprises 1,500 visual metaphors curated from real-world creative imagery, organized into three levels and ten categories, with each sample paired with two prompts of differing specificity. For evaluation, we develop a hybrid framework within an MLLM-as-judge paradigm, combining a multiple-choice question (MCQ) based protocol of 9,594 questions across four levels of metaphorical fidelity with a dimension-based scoring protocol along three perceptual dimensions. Extensive evaluation of 11 representative T2I models reveals that even the strongest proprietary models struggle with compositional structuring and cross-domain mapping, key aspects of metaphorical expression, highlighting visual metaphor generation as an important frontier for future T2I research.

[70] Beauty is in the AI of the beholder: MLLMs systematically overrate facial attractiveness cs.CV | cs.HCPDF

Santiago Grandas, Juan Sebastian Cely-Acosta, Mohit Mendiratta, Shafee Hassan, Macken Murphy

TL;DR: 本研究比较了四种主流多模态大语言模型（Claude、Gemini、GPT、Grok）与2,513名人类参与者对面部吸引力的评分。研究发现，MLLMs倾向于给出比人类更积极、评分范围更窄的评价，且无法在绝对值上复现人类评分，但能准确追踪人脸吸引力的排序。

Details

Motivation: 随着MLLMs在用户、公司和美容师中越来越广泛地用于评估面部吸引力，本研究旨在探究这些AI模型是否能准确反映人类对吸引力的判断。

Result: 在2,513张人脸的评估中，MLLMs的评分比人类更积极且范围更窄，与人类评分的绝对一致性较低，但在排序上表现出强相关性。Grok模型与人类的一致性最低，且与其他AI模型差异较大。

Insight: 论文揭示了当前商用MLLMs在面部吸引力评估中存在系统性高估的偏差，其判断依据可能与人类不同（仅年龄是共同预测因子）。这提示直接使用现成MLLMs进行此类主观评估需谨慎，但其排序能力或可借鉴。

Abstract: Beauty assessments from Multimodal Large Language Models (MLLMs) are increasingly popular amongst users, companies, and aestheticians. This raises the question of whether these AI models can accurately reflect human judgments of attractiveness. In a pre- registered exploratory study, we compared the attractiveness ratings of 2,513 human participants to four widely used commercial AI models: Claude, Gemini, GPT, and Grok. Results showed that MLLMs systematically rate faces more favourably and within a narrower range than humans and, at the time of study, do not reproduce human ratings in absolute terms. However, MLLMs exhibit strong correlations with human attractiveness judgments, accurately tracking the rank-ordering of faces. MLLMs may judge faces by different cues than humans; only face age was a predictor of facial attractiveness in both humans and MLLMs, with inconsistent patterns across models for ethnicity and gender. AI models strongly agree with one another, except for Grok, which also showed the lowest agreement with humans. Our findings suggest that while they may be able to approximate rank-orderings of human attractiveness, current off-the-shelf commercial MLLMs systematically overrate the beauty of human faces.

[71] Doppio: A Dataset for Contactless Weight Estimation of Falling Particles cs.CVPDF

Simon Kiefhaber, Jan-Martin O. Steitz, Julia Grabinski, Christoph Reich, Paul Wagner

TL;DR: 本文提出了一个名为Doppio的新型视频数据集，用于无接触质量估计任务，特别关注咖啡研磨过程中下落颗粒的重量测量。该数据集包含下落咖啡颗粒的视频序列，并提供了精确的每帧真实重量标注。作者评估了多种基于深度学习的计算机视觉方法，从纯空间前馈网络到循环时空模型，以展示无接触测量的可行性。

Details

Motivation: 在工业应用中，测量粉末（包括下落颗粒）的质量是常见任务。现有无接触传感方案通常成本高、应用特定且技术复杂，因此研究计算机视觉作为一种实用的无接触质量估计替代方案。

Result: 实验表明，基于深度学习的计算机视觉模型能够准确估计下落颗粒的累积重量，为未来基于视觉的无接触测量解决方案奠定了坚实基础。

Insight: 创新点在于引入了首个针对下落颗粒无接触重量估计的公开视频数据集Doppio，并系统评估了不同深度学习架构的预测精度与计算权衡，证明了视觉方法在该任务上的潜力。

Abstract: Measuring the mass of powder, including falling particles, is a common task in industrial applications. While scales are effective for static measurements, many applications require contactless sensing, where existing solutions are often costly, application-specific, and technically complex. In this work, we investigate computer vision as a practical alternative for contactless mass estimation. As an accessible real-world case study, we focus on coffee grinding and introduce \emph{Doppio}, a novel video dataset capturing videos of falling ground coffee, paired with precise, per-frame ground-truth weight measurements. To demonstrate contactless measuring, we evaluate deep learning-based approaches ranging from purely spatial feed-forward networks to recurrent spatio-temporal models. These models are analyzed with respect to their predictive accuracy and computational trade-offs. We demonstrate that deep learning-based computer vision models accurately estimate the cumulative weight of falling particles, establishing a solid foundation for future vision-based contactless measurement solutions.

[72] Spatially Aware World Action Model via Geometric Latent Diffusion cs.CV | cs.ROPDF

Javier Alejandro Lopetegui Gonzalez, Paul Pacaud, Cordelia Schmid

TL;DR: 本文提出了一种空间感知的世界动作模型（SA-WAM），通过几何潜在扩散方法，将预训练的视频扩散模型重新用于联合预测动作、RGB图像和深度信息，从而在单一扩散主干中实现3D感知的世界建模和动作预测。

Details

Motivation: 现有世界动作模型（WAMs）仅基于RGB观测，缺乏对3D信息的利用，限制了其在机器人策略学习中的潜力。本文旨在弥补这一差距，通过引入深度信息来增强模型的3D空间感知能力。

Result: SA-WAM在RoboCasa和LIBERO-Plus基准测试中取得了最先进（SOTA）的结果，同时在真实世界评估中使用UR5机械臂超越了强基线，在随机化环境中表现尤为突出。

Insight: 创新点在于通过非线性编码将无界的深度信号映射到冻结VAE分词器期望的有界输入域，从而无需针对3D进行微调即可重用分词器，在保留预训练先验的同时整合了几何信息。这为提升世界模型预测质量与任务执行成功率之间的相关性提供了新思路。

Abstract: World Action Models (WAMs) leverage the capabilities of large-scale pretrained video diffusion models to jointly predict future observations and actions, inheriting rich visual and physical priors from internet-scale video. This has made them a promising paradigm for robot policy learning, yet the prevailing models operate exclusively on RGB observations and do not leverage 3D information. To bridge this gap, we introduce a Spatially Aware World Action Model (SA-WAM), which repurposes a pretrained video model for joint action, RGB, and depth prediction, enabling 3D-aware world modeling and action prediction within a single diffusion backbone. We use a nonlinear encoding that maps the unbounded depth signal into the bounded input domain expected by the frozen VAE tokenizer. This allows us to reuse the tokenizer without 3D-specific fine-tuning, incorporating geometric information without sacrificing the pretrained priors. SA-WAM achieves state-of-the-art results on the RoboCasa and LIBERO-Plus benchmarks, while simultaneously improving future-state predictions. Furthermore, SA-WAM outperforms strong baselines in real-world evaluation using a UR5 robotic arm, with strong gains in randomized environments. We analyze the correlation between world model prediction quality and rollout success, providing insights into WAM performance and avenues for its improvement.

[73] Fine-Grained Anomaly Perception in Wild UGC-Enhanced Images: A Comprehensive Dataset and Difference-Fusion Framework cs.CV | cs.AIPDF

Yan Zhong, Gefei Chen, Qiufang Ma, Zhen Wang, Zhiwei Fan

TL;DR: 本文针对用户生成内容（UGC）图像增强后产生的视觉异常问题，提出了一个新的任务——UGC图像增强质量异常感知（UEAP），并构建了首个真实业务场景下的精细标注数据集UEAP-4k。同时，作者设计了一种基于差异融合的异常感知方法（DFAP-UGC），该方法结合显式问题-参考差异融合、密集空间查询、区域验证和质量感知排序，以鲁棒地识别异常。此外，还提出了一种局部感知动态任务优先级（LADTP）训练策略，以有效处理子任务间的耦合，实现端到端学习。

Details

Motivation: 现有图像质量评估（IQA）方法主要针对整体质量评估和经典失真，无法有效捕捉真实世界UGC图像经增强算法处理后产生的特定、局部化视觉异常（如人脸、文本和纹理的异常），这些异常会直接影响感知保真度和观看者信任。

Result: 大量实验表明，所提出的DFAP-UGC方法在UEAP任务上优于从经典方法改编的基线模型，验证了UEAP-4k数据集的价值以及DFAP-UGC在鲁棒的UGC增强图像异常感知方面的优越性。

Insight: 创新点在于正式定义了UEAP这一新任务并提供了首个精细标注的基准数据集；提出了DFAP-UGC方法，其核心是利用显式的问题-参考差异融合机制并结合多阶段处理（密集查询、区域验证、质量排序）来定位和评估异常；针对多子任务耦合问题，提出了LADTP训练策略以实现高效的端到端学习，避免了多阶段流程的开销。

Abstract: Image enhancement and restoration have become standard back-end operations on short-video and social media platforms to boost UGC visual experience. Yet these processes inevitably introduce visual anomalies–especially in faces, texts, and textures–that directly undermine perceptual fidelity and viewer trust. While existing IQA methods perform well on classic distortions, they target holistic quality assessment and fail to capture the specific, localized anomalies caused by enhancement algorithms in real-world UGC. To bridge this gap, we formally define a new task-quality Anomaly Perception for UGC image Enhancement (UEAP), and contribute the first UEAP benchmark dataset, named UEAP-4k, curated from the real business scenarios. It provides fine-grained annotations for anomaly categories, localization and severity levels. Furthermore, we propose a Difference-Fusion Anomaly Perception Method (DFAP-UGC) for wild UGC-enhanced images, which leverages explicit problem-reference difference fusion with dense spatial querying, regional verification, and quality-aware ranking, enabling robust anomaly identification in challenging scenarios. To handle the inherent coupling of subtasks in this new task, we propose a Locality-Aware Dynamic Task Prioritization (LADTP) training strategy that enables effective end-to-end learning and eliminates multi-stage overhead. Extensive experiments show that our method outperforms baselines adapted from classical approaches for this task, validating the value of this dataset and the superior of DFAP-UGC for robust UGC-enhanced image anomaly perception. Code and data will be public.

[74] MARS: What Retrieval Signals Are Hidden in Multimodal Large Language Models for Text-Video Retrieval? cs.CVPDF

Uicheol Jung, Juyoung Hong, Geuntaek Lim, Yukyung Choi

TL;DR: 本文提出MARS，一种用于文本-视频检索的多层多槽嵌入框架。该方法通过融合解码器不同层的隐藏状态构建多个自适应表示槽，比较对应的文本和视频槽并聚合相似度进行检索，以解决现有多模态大语言模型将多样线索压缩为单一向量导致的细粒度检索受限问题。

Details

Motivation: 现有基于多模态大语言模型的嵌入方法通常仅使用最终层的单个token表示输入，这会将视频和文本的多样线索压缩到单一向量中，限制了细粒度检索能力。

Result: 在四个文本-视频检索基准测试中，MARS在基于直接相似度的检索和重排序设置下均取得了最先进（SOTA）的结果。

Insight: 创新点在于提出多层多槽嵌入框架，通过融合不同层的隐藏状态构建多个自适应表示槽，并引入难负样本感知的槽专业化目标，使槽能捕获更具判别性的匹配线索。多层融合、多槽机制和难负样本感知专业化三者具有互补增益。

Abstract: Text-video retrieval requires representations that can distinguish videos with similar scenes, actions, and temporal patterns. Recent multimodal large language models have been adapted as embedding models, but they often represent each input using a single token from the final layer. This can compress diverse video-text cues into a single vector and limit fine-grained retrieval. To address this limitation, we propose MARS, a multi-layer and multi-slot embedding framework for text-video retrieval. MARS constructs multiple adaptive representation slots by combining hidden states from different decoder layers, compares corresponding text and video slots, and aggregates their similarities for retrieval. To better handle confusing candidates, we further introduce a hard-negative-aware slot specialization objective that encourages the slots to capture discriminative matching cues. Experiments on four text-video retrieval benchmarks show that MARS achieves state-of-the-art results in both direct similarity-based retrieval and reranking settings. Ablation studies and analyses demonstrate that multi-layer fusion, multiple slots, and hard-negative-aware slot specialization provide complementary gains. Code is available at https://github.com/sejong-rcv/MARS.

[75] RGB-to-IR image translation for infrared vehicle detection in unseen UAV domains cs.CVPDF

Thijs A. Eker, Ella P. Fokkinga, Jan Erik van Woerden, Elfi I. S. Hofmeijer, Sebastiaan P. Snel

TL;DR: 本文研究利用生成式模型将RGB图像转换为红外（IR）图像，以解决无人机（UAV）领域红外车辆检测中真实数据稀缺的问题。通过在源数据集上训练多种翻译模型（包括GAN、基于ControlNet的扩散模型和基于LoRA的基础模型编辑），并将生成的合成IR数据用于训练RF-DETR检测器，在未见过的目标域数据集上进行评估。

Details

Motivation: 在热红外（IR）空中车辆检测等真实数据稀缺的场景中，合成训练数据对开发视觉AI至关重要。虽然无人机RGB图像丰富，可用于通过RGB-to-IR转换进行数据增强，但不可观测的热特征（如发动机热量）使得学习可迁移的映射具有挑战性。

Result: 在五个航空数据集上评估，以Kust4K和VTUAV作为目标域。合成IR数据一致优于RGB和灰度基线。使用ControlNet的Stable Diffusion 3.5取得了最佳结果，在Kust4K上mAP从50.8提升至60.1，在VTUAV上从25.6提升至38.4（与仅在源域IR数据上训练的模型相比）。通过多种种子和提示词变化增加输出多样性，在VTUAV上分别带来了+1.1和+3.3 mAP的额外提升。

Insight: 论文的创新点在于系统评估了现代生成式翻译模型（包括GAN、扩散模型和基础模型编辑）在克服RGB到IR的跨模态差距、为未见目标域生成有效合成数据方面的能力。客观来看，其核心洞察是：尽管与真实目标IR数据仍存在性能差距，但生成式RGB-to-IR转换能有效缓解IR数据稀缺性，并显著提升跨域空中车辆检测性能，其中基于扩散模型的方法结合输出多样性策略表现最佳。

Abstract: Synthetic training data is crucial for developing vision AI when real-world data is scarce, as in thermal infrared (IR) aerial vehicle detection. While abundant UAV RGB imagery motivates RGB-to-IR translation for data augmentation, unobservable thermal traits (e.g., engine heat) make learning transferable mappings challenging. This work investigates whether modern generative translators can overcome this cross-modal gap to improve infrared vehicle detection on unseen UAV target domains. Translators are trained on paired RGB-IR source datasets and applied to RGB training images from held-out target datasets to generate synthetic IR data. Evaluated methods include supervised GANs, ControlNet-based diffusion models, and foundation-model editing via LoRA. The resulting synthetic IR imagery is used to train RF-DETR vehicle detectors, which are evaluated on unseen IR target test splits across five aerial datasets, with Kust4K and VTUAV serving as target domains. Synthetic IR consistently outperforms RGB and grayscale baselines. Stable Diffusion 3.5 with ControlNet yields the best results, improving mAP from 50.8 to 60.1 on Kust4K and from 25.6 to 38.4 on VTUAV compared to models trained only on source-domain IR data. Increasing output diversity via multiple seeds (+1.1 mAP) and prompt variations (+3.3 mAP) provides additional gains on VTUAV. Although a performance gap to real target IR data remains, generative RGB-to-IR translation effectively mitigates IR data scarcity and improves cross-domain aerial vehicle detection.

[76] Deeply Interleaved Text-Image Contexts for Multimodal LLMs Assessment cs.CVPDF

Zihao Wang, Xi Xiang, Yuwen Sun, Yingyu Li, Yabo Zhang

TL;DR: 该论文提出了一个名为TIC-Bench的新基准，用于评估多模态大语言模型在深度交错文本-图像上下文中的理解能力。该基准包含逻辑、时间和空间关联三个核心领域，共2280个问题，旨在弥补当前评估主要关注多图像任务而忽视文本与图像深度交互的不足。

Details

Motivation: 当前多模态模型的评估和训练主要集中在多图像任务上，文本通常仅作为任务指令，缺乏与视觉内容的深度语义交互。然而，现实应用（如文本-图像协同创作、角色追踪和空间重建）需要文本和图像之间的持续交互，因此模型必须具备对深度交错上下文的理解能力。

Result: 论文评估了10个最先进的多模态大语言模型，发现它们与人类专家之间存在显著的性能差距，并且在整合交错分布的视觉和文本证据方面存在持续困难。

Insight: 论文的创新点在于构建了一个专注于深度交错文本-图像上下文的评估基准，强调了模型整合跨模态线索以恢复事实的能力，这为分析和提升多模态模型在复杂真实场景中的信息融合能力提供了有价值的工具。

Abstract: Current evaluations and training of multimodal models predominantly focus on multi-image tasks, largely overlooking interleaved text-image scenarios. In such multi-image tasks, text typically serves merely as task instructions, lacking deep semantic interaction with the visual content. In contrast, realworld applications like text-image co-creation, character tracking, and spatial reconstruction require constant interaction between text and images. Consequently, models must possess a deep understanding of these interleaved contexts. To bridge this gap, we introduce a novel benchmark, TIC-Bench (deeply interleaved Text-Image Contexts), designed to evaluate the capability of models to integrate text-image clues and recover the ground truth facts within deeply interleaved contexts. This benchmark encompasses three core domains: Logical, Temporal, and Spatial Association, which are further categorized into eight specific types, comprising a total of 2,280 questions. We evaluated 10 state-of-the-art MLLMs and observed a substantial performance gap compared to human experts, together with persistent difficulties in integrating evidence distributed across interleaved visual and textual inputs. Ultimately, this benchmark provides a valuable analytical tool for assessing and advancing the ability of multimodal models to effectively integrate text and image information in deeply interleaved contexts. TIC-Bench is publicly available at https://huggingface.co/datasets/pino10010/TIC-Bench

[77] ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding cs.CV | cs.CLPDF

Jitai Hao, Ke Yang, Qiang Huang, Jun Yu

TL;DR: ShallowStream是一个用于流式视频理解的新框架，通过利用多模态大语言模型（MLLM）的浅层来同时进行帧编码和检索索引构建，从而显著降低计算开销。它使用浅层KV缓存维护轻量级索引，并在查询时基于注意力分数和多样性感知策略检索证据，实现了与现有最强方法相当的性能，同时大幅减少了延迟。

Details

Motivation: 解决流式视频理解中，使用MLLM处理连续视频流计算成本高昂的问题，特别是现有方法忽略了模型深度维度，导致重复执行全深度预填充带来巨大开销和KV缓存增长。

Result: 在流式视频理解任务中，ShallowStream实现了与现有最强方法相当的性能，同时将每帧预填充延迟和10秒端到端延迟分别降低了高达52.1倍和11.9倍。

Insight: 创新点在于利用MLLM浅层进行帧编码和索引构建，避免了全深度预填充；通过注意力分数和多样性感知策略进行检索，平衡了精确性和全面性；从模型深度维度优化流式处理，为高效视频理解提供了新思路。

Abstract: Streaming video understanding is a critical capability for real-world applications, including embodied intelligence, autonomous driving, industrial monitoring, surveillance and early warning, and wearable assistants. However, processing continuous video streams with multimodal large language models (MLLMs) is computationally expensive. Existing efforts have explored reducing streaming overhead through visual token pruning, token merging, quantization, on-demand frame retrieval, and context offloading. However, most existing methods overlook the dimension of model depth. Repeatedly executing full-depth MLLM prefill over incoming frames is prohibitively expensive, incurring substantial computational overhead and causing the KV cache to grow at a rate directly proportional to the prefill depth. To address these challenges, we propose ShallowStream, a novel framework that leverages the shallow layers of an MLLM to simultaneously perform frame encoding and retrieval index building. During stream processing, ShallowStream maintains an always-on lightweight index using the KV cache of shallow layers. During query-time answering, we leverage the attention scores generated by the shallow layers to score context frames and employ a diversity-aware selection strategy to retrieve precise and comprehensive evidence. ShallowStream achieves performance on par with the strongest existing streaming methods, while reducing per-frame prefill latency and 10-second end-to-end latency by up to 52.1x and 11.9x, respectively. Our code is available at https://github.com/CURRENTF/ShallowStream.

[78] Characterizing Text Branch Sensitivity in Medical Vision-Language Segmentation via Evidence Decoupling cs.CVPDF

Ziquan Liu, Zhewei Zhu, Xuyang Shi

TL;DR: 本文系统研究了文本在多模态医学图像分割中的作用，发现分割性能对融合模块选择不敏感，并提出了基于证据深度学习的证据解耦解码器（EDD）来分析模态交互。实验表明，文本敏感性在不同数据集间差异显著，文本主要通过全局语义调制而非独立空间定位影响预测。

Details

Motivation: 尽管预训练视觉语言模型在医学图像分割中表现出色，但文本信息在像素级预测中的实际贡献尚不明确，本文旨在量化分析文本在多模态分割中的角色。

Result: 在BUSI和BTMRI数据集上，移除文本会导致性能灾难性下降，表明模型对文本输入依赖性强；而在ISIC和Kvasir-SEG数据集上，文本影响相对边缘。EDD在保持竞争力的分割性能的同时，实现了对图像证据和文本调制证据的解耦分析。

Insight: 创新点在于提出EDD作为内部表示分析工具，揭示了文本敏感性随数据集变化的特性，并发现文本主要通过全局语义调制发挥作用，这为未来多模态模型设计提供了实用见解。

Abstract: Pretrained vision-language models (VLMs) have shown promising performance in medical image segmentation by incorporating clinical text. However, it remains unclear how much textual information actually contributes to pixel-level predictions. In this work, we systematically investigate the role of text in multimodal medical image segmentation. We first analyze several commonly used fusion strategies and find that segmentation performance is largely insensitive to the choice of fusion module. To further understand modality interactions, we propose an Evidence Decoupling Decoder (EDD) based on evidential deep learning and deep supervision. EDD serves as an internal representation analysis tool that decomposes image evidence and text-modulated evidence throughout the decoding process while maintaining competitive segmentation performance. Experimental results show that the sensitivity to text perturbation varies substantially across datasets. On BUSI and BTMRI, removing text causes catastrophic performance drops, indicating strong model reliance on textual input. On ISIC and Kvasir-SEG, text exerts relatively marginal influence. We further find that text affects predictions mainly through global semantic modulation rather than independent spatial localization, and that the specific semantic components driving text sensitivity differ across datasets. These findings provide a deeper understanding of modality interaction in multimodal medical image segmentation and offer practical insights for future model design.

[79] Query Rewriting for Complex Object Segmentation in 4D Gaussian Representations cs.CVPDF

Thanh-Khoi Nguyen, Thien-Phuc Tran, Minh-Triet Tran

TL;DR: 本文研究了查询改写对4D高斯表示中复杂物体分割的影响，提出了一种无需训练的重解释策略，将冗长的描述性查询转化为简洁的关键词形式，从而显著提升了时间定位和空间分割性能。

Details

Motivation: 现有4D高斯表示框架对包含噪声上下文信息的冗长叙述式查询高度敏感，影响了语言引导的动态场景理解效果，因此需要解决查询噪声问题以提升分割准确性。

Result: 在HyperNeRF和Neu3D基准测试中，该方法将平均时间准确率从60.92%提升至92.21%，平均vIoU从20.08%提升至76.94%，无需额外微调即达到显著性能提升。

Insight: 创新点在于借鉴检索增强语言模型和关键词引导查询重构的思想，通过训练无关的查询重解释策略减少语言噪声，同时保留与物体中心表示相关的语义锚点，实现更稳定的视频特征相似性分布和对齐。

Abstract: Recent 4D Gaussian representation frameworks have demonstrated strong performance in language-guided dynamic scene understanding. However, these methods remain highly sensitive to verbose and narrative-style queries that contain noisy contextual information. In this paper, we investigate the impact of query rewriting for complex object segmentation in 4D Gaussian representations. Inspired by recent findings in retrieval-augmented language models and keyword-guided query reformulation, we propose a training-free reinterpretation strategy that transforms long descriptive queries into concise keyword-grounded forms. Our approach progressively reduces linguistic noise while preserving semantic anchors relevant to object-centric representations. Experiments on HyperNeRF and Neu3D demonstrate that concise rewritten queries significantly improve both temporal localization and spatial segmentation performance. In particular, our method improves average temporal accuracy from 60.92% to 92.21% and average vIoU from 20.08% to 76.94% without any additional fine-tuning. Extensive ablation studies further reveal that shorter, keyword-focused queries consistently yield stable video-feature similarity distributions and better alignment with object-centric Gaussian representations

[80] RVSD: Retrieval Vision Sparse Decoding for Mitigating Visual Hallucinations in Large Vision-Language Models cs.CV | cs.AIPDF

Canjie Liu, Jiawen Kang, Jinbo Wen, Zishao Zhong

TL;DR: 该论文提出了一种名为RVSD（检索视觉稀疏解码）的训练无关即插即用解码框架，旨在缓解大型视觉语言模型中的视觉幻觉问题。该方法首次在单次解码过程中统一了令牌稀疏化和语义空间视觉检索，通过语义导向的令牌选择策略保留关键视觉信息，并通过跨模态检索进行视觉补偿。

Details

Motivation: 大型视觉语言模型在视觉语言任务上取得了显著成功，但仍容易产生视觉幻觉，这削弱了其在现实应用中的可靠性。现有解决方案通常需要精心策划的数据集、额外训练或多轮解码，导致计算开销巨大。

Result: 大量实验表明，RVSD在缓解视觉幻觉方面实现了最先进的性能，并在长上下文生成设置下保持了强大的抑制能力。

Insight: 论文的创新点在于首次将令牌稀疏化与语义空间视觉检索统一在单次解码过程中，提出了语义导向的令牌选择策略和按需跨模态检索机制，这是一种无需训练、即插即用的高效方法，可有效减少冗余计算并提升模型可靠性。

Abstract: Large vision-language models have achieved remarkable success in vision-language tasks. However, they remain prone to Visual Hallucinations (VHs), undermining their reliability in real-world applications. Existing solutions typically require curated datasets, additional training, or multi-round decoding, resulting in considerable computational overhead. In this paper, we propose \textbf{RVSD} (\underline{R}etrieval \underline{V}ision \underline{S}parse \underline{D}ecoding), a training-free and plug-and-play decoding framework that, for the first time, unifies token sparsification and \textbf{Semantic-Space Visual Retrieval} (SSVR) within a single decoding pass. Within RVSD, we introduce a \textbf{semantics-directed token selection} strategy that selectively sparsifies redundant tokens while preserving critical visual information. We further propose the SSVR mechanism, which reformulates visual compensation as an on-demand cross-modal retrieval process within a shared semantic space. Extensive experiments demonstrate that RVSD achieves state-of-the-art performance in mitigating VHs while maintaining robust suppression capabilities under long-context generation settings. Our code is available here.\footnote{https://github.com/canjie-liu/RVSD}

[81] InceptionGS: Generative Bootstrapping for Large-Scale Gaussian Splatting under Unstructured View Sampling cs.CVPDF

Tianheng Lu, Guangyu Wang, Ruqi Huang, Lu Fang

TL;DR: 本文提出InceptionGS方法，通过平衡重建与生成来提升大规模场景下高斯溅射（Gaussian Splatting）的渲染质量，特别针对非结构化视角采样导致的视角稀缺问题。该方法从初始高斯溅射出发，利用场景和视角自适应的生成先验，修复视角不足区域，同时保持其他区域的质量。

Details

Motivation: 解决大规模场景数字化中，由于采集成本、场景复杂性等因素导致的多视角图像非结构化采样问题，现有重建方法对视角稀缺敏感，而生成方法存在泛化性、可控性和3D一致性问题。

Result: 在真实世界大规模场景上的大量实验表明，该方法在处理非结构化图像和提升高保真高斯溅射方面具有优越性和广泛适用性，但摘要未提及具体基准测试或定量结果。

Insight: 创新点在于通过引导式生成（generative bootstrapping）巧妙结合重建与生成，利用场景和视角自适应的先验来修复视角稀缺区域，从而提升3D重建的鲁棒性和视觉一致性。

Abstract: Achieving truly immersive large-scale scene digitization necessitates consistent and visually pleasing rendering across all possible viewing perspectives. However, collecting multi-view images covering every fine detail of a large-scale scene is prohibitive due to scene complexity, capture cost, negligence, or accessibility constraints. As a result, the sampled views tend to be highly unstructured – the majority of the scene is well covered yet certain regions inevitably lack sufficient observations. Existing reconstruction based methods are vulnerable to view scarcity while generation based approaches suffer from generalization, controllability, and 3D consistency issues. To address this challenge, we propose InceptionGS, which bootstraps Gaussian splatting by subtly balancing reconstruction and generation. Starting from an initial Gaussian splatting, InceptionGS reasonably rethinks and repairs problematic regions caused by view scarcity while preserving the quality elsewhere, by softly incorporating scene- and view-adaptive generative priors. Extensive experiments on real-world large-scale scenes demonstrate the superiority and broad applicability of our approach in handling unstructured imagery and boosting high-fidelity Gaussian splatting. Please refer to the supplementary video for better visual demonstrations.

[82] Balancing Frequencies and Pixels in Flow Matching cs.CVPDF

Lucas Degeorge, Paul Couairon, Arijit Ghosh, Alexei A. Efros, David Picard

TL;DR: 本文提出了一种名为Focal Log-Frequency Loss（f-loss）的频谱平衡目标函数，用于解决像素空间流匹配模型训练中因频谱不平衡（低频主导）导致的学习效率低下问题。该方法结合了频域和像素空间的监督，先强调频域学习以捕获所有频率，再转向像素空间进行空间细化，从而加速收敛并提升生成质量。

Details

Motivation: 自然图像遵循1/f²频谱分布，但像素空间重建目标对所有空间误差一视同仁，导致低频主导优化信号，延迟了细粒度细节的学习。本文旨在解决像素空间流模型训练中这种目标层面的频谱不平衡问题。

Result: 该方法在多个模型规模上，将收敛速度提升了高达40%，并持续改善了FID分数和感知保真度。

Insight: 核心创新点是提出了频谱平衡的目标函数（f-loss），并通过结合频域与像素监督的两阶段训练策略，有效缓解了像素损失的固有低频偏差，无需改变模型架构即可作为流匹配损失的即插即用替代方案。

Abstract: Natural images follow a $1/f^2$ spectral distribution: most signal energy lies in the low spatial frequencies, while the perceptually important structures such as textures and edges occupy sparse high-frequency bands. Pixel-space reconstruction objectives, however, treat all spatial errors uniformly, causing low frequencies to dominate the optimization signal and delaying the learning of fine-scale details. In this work, we identify this objective-level spectral imbalance as a key inefficiency in training pixel-space flow models. To address it, we propose a Focal Log-Frequency Loss (f-loss), a spectrally balanced objective that equalizes the learning signal across frequencies, emphasizing high-frequency components that are otherwise underrepresented in pixel-space objectives. Building on this, we introduce a simple training strategy that combines frequency and pixel supervision: we first emphasize frequency-domain learning early to capture all frequencies, and then transition to standard pixel-space v-loss for spatial refinement. This balancing mitigates the low-frequency bias of pixel losses and aligns the training signal with the evolving needs of the model. Our approach is conceptually simple, requires no architectural changes, and acts as a drop-in replacement for flow matching losses. Across multiple model scales, it accelerates convergence by up to 40% while consistently improving FID and perceptual fidelity. We will release code and models.

[83] GDB-Reward: From Evaluation Metrics to Training Rewards for Graphic Design cs.CVPDF

Adrienne Deganutti, Purvanshi Mehta, Simon Hadfield, Andrew Gilbert

TL;DR: 本文提出GDB-Reward框架，将非可微的平面设计评估指标转化为统一的强化学习奖励，用于优化冻结文本到图像模型的提示，从而提升生成图像在排版、布局、颜色和视觉传达方面对设计规范的遵循度。

Details

Motivation: 解决文本到图像模型在平面设计任务中难以满足精确约束的问题，并探索如何利用非可微的评估指标作为强化学习的奖励，以替代昂贵的模型微调。

Result: 实验表明，GDB-Reward能有效优化提示，在感知质量、渲染保真度和空间准确性方面显著提升对设计规范的遵循，同时保持图像生成器完全冻结。

Insight: 创新点在于将异构的非可微评估指标系统性地转化为强化学习奖励，使这些指标从被动评估工具变为主动优化目标，为缺乏可微监督的领域提供了新的训练范式。

Abstract: Text-to-image models excel at natural image synthesis but struggle with graphic design, where success depends on satisfying precise constraints on typography, layout, color, and visual communication. While prompt optimization offers an attractive alternative to expensive diffusion model fine-tuning, learning prompts for frozen image generators requires informative reward functions despite the entirely non-differentiable generation process. Reinforcement learning does not require differentiable objectives; it requires only scalar rewards capable of ranking candidate outputs. This raises a simple question: can design evaluation metrics themselves become reinforcement learning rewards? Our central contribution is GDB-Reward, a framework that systematically transforms heterogeneous graphic design evaluation metrics into a unified reinforcement learning reward. Experiments demonstrate that GDB-Reward provides an effective optimization objective, substantially improving adherence to the design specification in perceptual quality, rendering fidelity, and spatial accuracy while keeping the image generator entirely frozen. More broadly, our results demonstrate that heterogeneous, non-differentiable evaluation metrics can move beyond passive benchmarking to become effective optimization objectives for reinforcement learning in domains where differentiable supervision is unavailable.

[84] AutoCompass: Accurate Visual Localization on Public Maps by Learning from Weak Labels cs.CVPDF

Javier Tirado-Garín, Alan Savio Paul, Shuai Chen, Axel Barroso-Laguna, Tommaso Cavallari

TL;DR: AutoCompass是一种用于训练神经地图匹配器的监督方法，旨在解决地理参考图像数据集中位置和朝向标签噪声问题。该方法通过利用原始GPS标签、定义容忍区域以及结合SLAM/SfM获取的相对位姿，减少了对精确绝对位姿标签的依赖，从而提升了模型在驾驶和第一人称视角基准测试中的定位精度。

Details

Motivation: 神经地图匹配器在训练时依赖大规模地理参考图像数据集，但这些数据集的位姿标签（位置和朝向）通常包含噪声，影响了训练模型的准确性。

Result: 在驾驶和第一人称视角基准测试中，AutoCompass方法在定位精度上持续优于依赖精确绝对位姿标签的对比方法。

Insight: 创新点在于证明了朝向标签并非必需，模型能从原始GPS标签自动学习准确朝向；通过定义GPS容忍区域提升位置精度；并利用SLAM/SfM的相对位姿作为更准确的训练信号，减少了对噪声绝对标签的依赖。

Abstract: Neural map matchers estimate an image’s 3-DoF pose relative to a 2D map. These models are trained on large-scale datasets of geo-referenced images, whose position and heading labels often contain noise that affects the trained models. To address this, we present AutoCompass, a supervision approach for training neural map matchers from inaccurate absolute pose labels. First, we show that heading labels are unnecessary: trained from raw GPS labels, models learn to predict accurate headings, automatically. Second, defining a tolerance region around raw GPS improves positional accuracy. Third, if available, our supervision uses relative poses between training images, obtained via SLAM or SfM, which provide a more accurate training signal. Across driving and egocentric benchmarks, AutoCompass consistently outperforms counterparts trained with the usual strong reliance on absolute pose labels.

[85] Video-Based Palm-Vein Authentication under Challenging Conditions cs.CVPDF

Xiaofeng Yan, Kechen Liu, Abhilash Venkatesh, Cathy Zhang, Xia Zhou

TL;DR: 该论文针对现实世界中手掌静脉生物识别技术面临的表面噪声、光照变化、运动及温度变化等挑战，引入了首个公开的视频手掌静脉数据集CUP，并评估了多种识别器。研究发现，现有模型在脏污手掌上性能大幅下降，而通过时空融合策略（如帧间共识和基于显著性的区域级最优传输匹配）可显著提升鲁棒性，在CUP数据集上实现了SOTA性能，且能有效提升现有SOTA骨干网络的识别准确率。

Details

Motivation: 解决手掌静脉生物识别在现实部署中因表面噪声、光照变化、运动及温度变化导致的血管可见性变化问题，这些挑战因缺乏相关数据而未被充分研究。

Result: 在CUP数据集上，提出的方法在EER、TAR@FAR=0.01和Rank-1指标上均达到最优，参数和计算成本远低于视频模型；当与四个冻结的SOTA骨干网络结合时，平均EER降低了29-37%，且在四个公开单图像数据集上区域匹配策略也有效。

Insight: 创新点包括引入首个公开视频手掌静脉数据集CUP以研究挑战性条件，以及提出无需额外学习参数的测试时匹配器，通过时空融合（帧间共识和基于显著性的区域级最优传输）来绕过损坏区域，提升模型鲁棒性。

Abstract: Palm-vein biometrics are increasingly used for secure, contactless authentication. Yet real-world deployment exposes them to surface noise (sweat, dirt), illumination and motion variation, and temperature-driven changes in vascular visibility, which remain underexplored for lack of data captured under such conditions. To study these effects, we introduce the Columbia University Palm-vein (CUP) dataset, to our knowledge the first public video-based palm-vein dataset. CUP records every palm under four surface conditions (a clean baseline, warm, wet, and dirty) and pairs each subject with physiological and demographic metadata. On it we benchmark twenty-one recognizers spanning static, video, and multi-frame aggregation architectures. Models that verify reliably on clean palms lose most of their accuracy on dirty ones, and the mean equal error rate (EER) roughly quadruples. We recover much of that robustness along both axes of the capture. Temporally, a consensus over the few frames the sensor already returns cancels transient corruption; spatially, a test-time matcher that adds no learned parameters fuses the global cosine with a saliency-steered region-level optimal transport that routes the comparison around corrupted regions. The full design leads on every surface of CUP in EER, TAR@FAR=0.01, and Rank-1, at 4.3M parameters and 3.1 GFLOPs, a fraction of the video models’ cost. Attached to four frozen state-of-the-art backbones it cuts their mean EER by 29-37% without retraining, and on four public single-image datasets the regional matching alone still helps. A preliminary audit across ten demographic and physiological traits finds two warm-condition gaps, along body water and gender, that survive multiple-comparison correction. CUP will be released for non-commercial research use at https://github.com/MobileX-CU/CUP_v1 upon publication.

[86] Benchmarking RAW and RGB Restoration in Image Signal Processors cs.CVPDF

Zihao Lu, Radu Timofte, Marcos V. Conde

TL;DR: 本文对图像信号处理器（ISP）前后的盲图像恢复进行了基准测试，比较了RAW域（ISP前）和sRGB域（ISP后）两种恢复策略。研究覆盖了多种智能手机设备、学习型ISP、退化类型（噪声、模糊及联合退化）以及代表性的恢复模型。结果表明，恢复性能不仅取决于放置位置，还与恢复模型和成像流程的匹配度密切相关。

Details

Motivation: 现代相机通过ISP将RAW传感器数据转换为sRGB图像，但图像恢复任务（如去噪、去模糊）在ISP前（RAW域）还是ISP后（RGB域）进行更优，缺乏系统的基准研究。本文旨在通过一个全面的基准测试来回答这个问题。

Result: 基准测试显示，RAW恢复策略优于通用的RGB恢复模型。然而，在训练时考虑了ISP变换的RGB恢复模型取得了最佳的整体性能。这表明性能强烈依赖于恢复模型与目标成像流程的对齐程度。

Insight: 论文的创新点在于构建了一个全面的、考虑多种设备、ISP和退化类型的基准测试框架。核心洞察是，图像重建性能的关键并非简单的“前处理”或“后处理”位置选择，而是恢复模型与特定成像流程（ISP）的协同设计（ISP-aware supervision）。这为未来的图像恢复研究提供了关键的实验设计建议。

Abstract: Modern cameras transform RAW sensor measurements into sRGB images through an image signal processor (ISP). We benchmark two placements for blind restoration around a fixed ISP: (A) pre-ISP restoration in the RAW domain and (B) post-ISP restoration in the sRGB domain. The benchmark covers four smartphone device groups, two learned ISPs, three degradation regimes–noise, blur, and joint noise and blur–, and several representative RAW and RGB restoration models. Our results show that placement alone does not determine performance. The RAW restoration strategy outperforms the best generic RGB restoration models. However, RGB restoration models trained considering the ISP transformations, achieve the best overall performance. Our novel benchmark demonstrates that the image reconstruction performance strongly depends on the alignment between the restoration model and the target imaging pipeline. We consequently recommend reporting restoration placement and ISP-aware supervision as key experimental factors. Our code is available at https://github.com/mv-lab/AISP

[87] RoGe: Novel View Synthesis via End-to-End Implicit Reconstruction and Generation cs.CVPDF

Xiaolei Lang, Ze Kang, Zehao Huang, Naiyan Wang

TL;DR: 本文提出RoGe，一种端到端的隐式重建与生成统一框架，用于解决稀疏输入下的新视角合成问题。该方法通过前馈重建模型构建隐式场景表示，并利用射线查询获取几何特征，直接注入视频扩散模型作为条件，无需3D中间表示。实验表明RoGe在DL3DV数据集上超越了基于重建、生成及混合基线的方法，在图像质量和视频时序一致性方面表现优异。

Details

Motivation: 现有混合方法通过渲染图像或显式3D表示（如点云或3D高斯）桥接重建与生成，导致生成过程受限于有损且不完美的场景投影误差，且重建无法从生成中获得纠正信号。RoGe旨在消除这种显式桥接，实现端到端的统一优化。

Result: 在DL3DV数据集上，RoGe在图像级指标和视频级时序一致性方面均优于基于重建、生成及混合的基线方法。消融实验证实，射线查询的隐式特征作为条件优于原始重建token和渲染RGB，且联合训练带来进一步增益。

Insight: 创新点在于去除显式3D中间表示，通过端到端联合训练隐式重建与视频生成模型，使生成目标直接优化几何条件特征；从客观角度看，该方法实现了重建与生成的双向信号流动，提升了稀疏视角下场景漫游视频合成的几何一致性与生成质量。

Abstract: Novel view synthesis from sparse inputs requires both geometric grounding from the observed views and generative priors of unobserved regions, motivating recent hybrid methods that combine reconstruction and generation. However, existing methods bridge the two with rendered images or explicit 3D representations such as point maps or 3D Gaussians. Generation is thus conditioned on a lossy and imperfect projection of the scene, inheriting its errors, and reconstruction receives no signal from generation to correct them. We present RoGe, an end-to-end unified reconstruction and generation framework that removes this explicit bridge. It targets roaming within a scene anchored by sparse views: given a few posed images and a camera trajectory, it synthesizes a temporally coherent video along that trajectory. From the sparse input views, RoGe builds an implicit scene representation with a feed-forward reconstruction model, and queries it with target camera rays to obtain per-view geometric features. These features are injected into a video diffusion model as conditioning, without any 3D intermediate. Both modules are trained jointly, so the generation objective directly shapes its own geometric conditioning. We conduct experiments on DL3DV, where RoGe outperforms reconstruction-based, generation-based, and hybrid baselines on image-level metrics and video-level temporal consistency. Ablations confirm that ray-queried implicit features outperform both raw reconstruction tokens and rendered RGB as conditioning, and that joint training brings further gains.

[88] MuyBridge: Mobile Human Center-of-Mass Estimation from Monocular Video via Sparse Fusion cs.CVPDF

Aidan Bradshaw, Marco Giordano, David Rode, Andreas Habersack, Elif Basokur

TL;DR: 本文提出了MuyBridge，一种基于单目手机视频流、在设备端实时估计运动员分段质心轨迹的系统。该系统通过稀疏融合一个紧凑的2D姿态网络和一个蒸馏的单步单目深度网络，利用解剖学和物理先验进行度量融合，无需3D或任务特定监督。

Details

Motivation: 现有3D姿态跟踪、网格恢复或多视角三角测量方法要么缺乏解剖学约束，要么计算和捕获基础设施过于笨重，难以在运动员训练和比赛的单一摄像头场景中部署，导致教练和运动分析师难以从单摄像头测量度量化的3D质心。

Result: 在AthletePose3D数据集（包含跑步、田径、花样滑冰等运动）上评估，MuyBridge在一次性校准后实现了33-41毫米的垂直质心误差和2.3-6.6%的绝对相对范围误差（AbsRel），并在iPhone 15上以63 FPS的姿态估计速率和异步2.86 Hz的深度更新频率生成质心估计。

Insight: 创新点在于提出了一种轻量级的、无需3D监督的度量质心估计方法，通过结合解剖学和物理先验的稀疏融合策略，实现了在移动设备上的实时、高精度性能。从客观角度看，其将2D姿态与单目深度进行高效融合的框架，为移动端生物力学分析提供了实用且可部署的解决方案。

Abstract: The 3D center of mass (CoM) is a primary quantity in the biomechanical analysis of sport, rehabilitation, and clinical movement, yet existing 3D pose tracking, mesh recovery, and multi-view triangulation methods either optimize 3D keypoint accuracy without anatomical constraints or carry compute and capture infrastructure too heavy to deploy where CoM tracking is most useful. As a result, the metric CoM remains difficult for coaches and movement analysts to measure from a single camera where athletes train and compete. In this work, we introduce MuyBridge, an on-device system that estimates the athlete’s segmental center of mass trajectory from a single phone camera video stream. MuyBridge couples a compact 2D pose network and a distilled single-step monocular depth network through an analytic metric fusion that uses anatomical and physical priors to anchor the metric CoM, requiring no 3D or task-specific supervision. Evaluated on the athletic movements of AthletePose3D (running, track and field, and figure skating), MuyBridge achieves 33-41 mm vertical CoM error and 2.3-6.6% absolute-relative range error (AbsRel) under a one-time calibration, and produces CoM estimates at the 63 FPS pose-estimation rate using asynchronous 2.86 Hz depth updates on iPhone 15. Code is available at: https://github.com/Abradshaw1/Muybridge

[89] PlantC2USeg: Cross-Scale Consistent Pre-Training for Few-Shot Unified Plant Point Cloud Segmentation cs.CVPDF

Yu Tian, Xintong Jiang, Jan Franklin Adamowski, Shiv O. Prasher, Shangpeng Sun

TL;DR: 本文提出了PlantC2USeg，一个用于植物点云分割（PPCS）的深度迁移学习框架。该框架通过跨尺度一致性学习和信息受限解码策略进行预训练，旨在实现从少量标注样本中稳定地泛化到不同物种和感知条件。该方法在多个数据集上实现了优异的少样本分割性能，并展示了在农业领域之外处理多样化形状变化的能力。

Details

Motivation: 现代作物育种需要对器官进行精确分析以实现性状量化，这使得植物点云分割（PPCS）日益重要。然而，传统深度学习方法严重依赖密集标注的数据集，获取成本高昂。在分布偏移的情况下，利用极少额外训练实现统一的PPCS适应仍然具有挑战性。

Result: 在Soybean3D数据集上，在完全监督下，PlantC2USeg取得了最高的语义IoU（91.91%）和实例mWCov（94.62%）。在20个标注样本下，两项指标分别为89.78%和90.27%；在仅10个样本下，仍保持最高的mWCov（83.23%）和83.19%的IoU。在HR3D数据集上，10样本迁移到烟草、番茄和高粱平均获得78.41% IoU和79.42% mWCov；22样本迁移到SYAU-Maize获得最高的IoU（92.75%）和mRec（93.51%）。在ShapeNet Part数据集上，取得了领先的类别平均mIoU（85.0%），证明了其跨领域泛化能力。

Insight: 论文宣称的创新点在于提出了一个结合跨尺度一致性学习和信息受限解码策略的预训练框架，以显式对齐跨空间尺度的特征并防止解码捷径，从而促进鲁棒的少样本适应。从客观角度看，其核心创新在于通过统一的预训练和微调机制，显著降低了在分布偏移下适应新物种或条件所需的标注成本，实现了可扩展的植物表型分析和可迁移的3D表示学习。

Abstract: Modern crop breeding demands precise organ-level analysis for trait quantification, making plant point cloud segmentation (PPCS) increasingly important. However, conventional deep learning approaches rely heavily on densely annotated datasets that are labor-intensive to acquire. Unified PPCS adaptation from distribution-shifted examples with minimal additional training remains challenging. To address this, we propose PlantC2USeg, a deep transfer learning framework featuring cross-scale consistency learning to explicitly align features across spatial scales and an information-restricted decoding strategy that prevents reconstruction shortcuts and promotes robust adaptation. The resulting pre-training enables stable few-shot generalization across species and sensing conditions, while unified fine-tuning with inherited thresholds further reduces adaptation overhead. Under full supervision on Soybean3D, PlantC2USeg achieves the highest semantic IoU and instance mWCov among compared methods, at 91.91% and 94.62%. With 20 labeled samples, it leads both metrics at 89.78% and 90.27%; with only 10 samples, it retains the highest mWCov of 83.23% while achieving 83.19% IoU. Across HR3D, 10-shot transfer to tobacco, tomato, and sorghum averages 78.41% IoU and 79.42% mWCov, while 22-shot transfer to SYAU-Maize achieves the highest IoU and mRec at 92.75% and 93.51%. Furthermore, a leading category-averaged mIoU of 85.0% on ShapeNet Part demonstrates the framework’s capability to handle diverse shape variations beyond agricultural domains. These results demonstrate that PlantC2USeg reduces overall adaptation effort under distribution shifts, enabling scalable plant phenotyping and transferable 3D representation learning beyond agriculture.

[90] Thinking in Pictures: A Systematic Benchmark for Reasoning-driven Image Generation cs.CVPDF

Yutong Liu, Nan Huang, Xu Cao, James M. Rehg

TL;DR: 本文提出了一个名为RIG-BENCH的新型综合基准，用于系统评估推理驱动的图像生成能力。该基准涵盖概念、变换、模式与结构以及场景四大认知领域，包含2000个精选样本，旨在测试模型从视觉输入中推断潜在规则并生成逻辑约束图像的能力。

Details

Motivation: 当前统一生成模型和世界模拟器主要依赖表层事件对齐，其高级视觉推理能力未被充分探索。真正的视觉生成智能需要‘推理到生成’的能力，即从视觉输入推断潜在规则，并通过精确、逻辑约束的视觉结果展现解决方案。

Result: 对最先进的统一生成模型和图像/视频生成模型进行的广泛评估揭示了一个显著的推理-生成鸿沟：模型经常产生局部合理但全局不合逻辑的输出。RIG-BENCH为评估现有模型提供了诊断框架。

Insight: 论文的创新点在于首次系统性地构建了一个专注于评估高级视觉推理生成能力的基准。从客观角度看，其将视觉推理任务结构化分类为四个认知领域，为诊断和推动下一代具有逻辑基础的生成模型发展提供了关键工具。

Abstract: Recent advancements in unified generative models (UGMs) and world simulators have achieved unprecedented results in visual perception and synthesis. However, these models primarily rely on surface-level event alignment, leaving the capacity for high-level visual reasoning underexplored. True visual generative intelligence demands “Reasoning-to-Generation”, an ability to infer latent rules from visual inputs and manifest solutions through precise, logically constrained visual outcomes. We introduce RIG-BENCH, a novel comprehensive benchmark that systematically evaluates Reasoning-driven Image Generation (RIG) across four cognitively demanding domains: Concept-based, Transformation-based, Pattern & Structure, and Scenario-based. Featuring 2000 curated samples, RIG-BENCH serves as a rigorous stress test for RIG. Our extensive evaluations of state-of-the-art UGMs and image/video generation models reveal a significant reasoning-generation gap, wherein models frequently produce locally plausible but globally illogical outputs. RIG-BENCH provides a vital diagnostic framework to guide the development of next-generation, logically grounded UGMs and world simulators.

[91] SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models cs.CVPDF

Junchao Huang, Guian Fang, Shengju Qian, Xianghao Kong, Zhuoran Zhao

TL;DR: SolarWM是一个完全开放的基础框架，用于构建交互式视频世界模型，涵盖从数据准备到长时程推理的全流程。它通过可重构的多源数据引擎和骨干网络原生适配框架，解决了异构数据源和视频骨干网络训练中的不一致性问题，并基于统一的三阶段训练方法实例化了多个大模型。

Details

Motivation: 解决从异构数据源（如时间尺度、相机几何、视觉质量、运动模式和标注风格不同）训练视频世界模型时，因数据混合和模型特定实现导致监督信号不一致、结果难以复现和比较的问题。

Result: 基于Wan2.2、LTX-2.5和MiniMax-H3骨干网络，实例化了四个参数量为5B到33B的因果模型，这些模型在仅训练5秒序列后，能够在分钟到小时级别的推演中实现实时交互。

Insight: 创新点在于提出了解耦数据源处理与混合构建的统一数据引擎，以及保持骨干网络原生表示和目标的适配框架，结合双向适应、教师强制自回归初始化和分布匹配蒸馏的三阶段训练方法，为交互式世界模型研究提供了可复现和可扩展的基础。

Abstract: We introduce SolarWM, a fully open foundation for building interactive video world models from data preparation through long-horizon inference. Training across heterogeneous data sources and video backbones is challenging: datasets differ in temporal scale, camera geometry, visual quality, motion, and captioning styles, while video generators use distinct representations and architectures. Naive data mixing and model-specific implementations therefore produce inconsistent supervision and make results difficult to reproduce and compare. SolarWM addresses this coupling with a reconfigurable multi-source data engine and a backbone-native adaptation framework. The engine converts 1.43 million canonical clips from 10 datasets into a unified, frame-aligned contract covering visual observations, metric camera geometry, captions, quality metadata, selection decisions, and provenance, while decoupling source processing from mixture construction. Under shared camera-conditioning, training, and inference interfaces, we instantiate four 5B–33B models based on Wan2.2, LTX-2.5, and MiniMax-H3 while preserving their native representations and objectives. A unified three-stage recipe combines bidirectional adaptation, teacher-forced autoregressive initialization, and distribution matching distillation. The resulting causal models enable real-time interaction over rollouts ranging from minutes to hours after being trained on only 5s sequences. By releasing the resulting data, pipeline, recipes, weights, and framework, SolarWM provides a reproducible and extensible foundation for interactive world-model research.

cs.MA [Back]

Zijian Zhao, Sen Li, Xialiang Tong, Mingxuan Yuan

TL;DR: 本文提出RideSkill，一种用于广义拼车（ride-sharing）的分层算法，利用LLM驱动的自动进化方法进行算法设计。该方法包含组合器（combiner）和重定位器（repositioner），通过从学习的技能库中分配技能来适应不同场景和目标，并避免车辆冲突，且无需在部署时调用LLM，保证了实时性能。

Details

Motivation: 现有拼车问题解决方案（如多智能体强化学习MARL）存在泛化性差、可迁移性低、训练困难等问题；而引入LLM的方法要么不支持车辆共享，要么在推理时需要频繁调用LLM，难以实时部署。本文旨在解决这些问题。

Result: 论文未在摘要中提及具体定量结果或基准测试，但声称所提方法能适应不同环境场景和平台目标，并确保高实时性能。

Insight: 创新点在于提出了一种分层架构（技能库、组合器、重定位器）并结合LLM驱动的自动进化进行训练，从而在支持车辆共享的同时避免了部署时的LLM调用，实现了可泛化、可迁移且高效的实时拼车调度。

Abstract: Ride-sharing, which allows multiple passengers with different origin-destination (OD) pairs to share a single vehicle, is a challenging operational problem, as it requires orders with different OD pairs to be efficiently bundled and assigned to vehicles under uncertain and varying scenarios. Although multi-agent reinforcement learning (MARL) solutions have achieved promising performance, they suffer from limited generalization (adapting to different environmental scenarios), low transferability (adapting to different platform objectives), and training difficulties in large-scale systems, such as the curse of dimensionality. Recently, motivated by the scaling of large language models (LLMs), several works have incorporated LLMs into ride-hailing systems, either by employing LLMs directly as decision-making agents or using them for automatic algorithm design. However, none of these approaches support vehicle sharing, which complicates the problem by expanding both the state and action spaces exponentially. Moreover, most of them require frequent LLM calls at inference time, making them infeasible for real-time deployment. To address these issues, we propose RideSkill, a hierarchical method for ride-sharing that leverages LLM-assisted automatic algorithmic design. RideSkill consists of a combiner that assigns appropriate skills to each vehicle from a learned skill repository, enabling adaptive dispatch under varying scenarios and objectives, and a repositioner that sequentially relocates idle vehicles to emerging regions, avoiding conflicts among vehicles. Crucially, the skill repository, combiner, and repositioner are all trained by an LLM-based automatic evolutionary method, eliminating the need for LLM calls during deployment and thus ensuring high real-time performance.

[93] PRISM: An Agentic Multi-Model Architecture for Proactive Safety in Autonomous Transportation Systems cs.MA | cs.CV | cs.ET | cs.LG | physics.soc-phPDF

Joyjit Roy, Samaresh Kumar Singh, Sushanta Das

TL;DR: 该研究提出了PRISM（主动风险智能与安全管理）架构，这是一种用于自动驾驶系统的智能多模型安全架构。它通过将传统的被动碰撞避免转变为主动、持续的风险管理，利用逆向碰撞概率建模将二元碰撞分类器转换为动态可解释的安全评分，并通过包含强化学习、上下文记忆和特征级归因的推理层协调三个专门模型（轨迹运动学、环境风险和弱势道路使用者交互）来实现分级安全干预。

Details

Motivation: 当前先进的驾驶辅助系统（ADAS）主要采用被动机制，仅在危险出现后才激活，这在美国弱势道路使用者（VRU）伤亡率上升的背景下是一个关键局限。研究旨在解决复杂城市环境中，自动驾驶系统安全依赖于车辆行为、环境条件与VRU之间交互的挑战，从而从被动避免转向主动风险管理。

Result: 在无需针对数据集重新训练的情况下，使用来自三个自然驾驶数据集的1296个场景进行验证，系统平均安全评分为68分（满分100），将77.6%的场景分类为建议级别，标记出3.8%的险肇事件率，并有11%的场景升级为干预或紧急响应。特征归因一致地将轨迹风险和VRU接近度识别为主要安全因素。

Insight: 论文的创新点在于提出了一个统一的、可解释的主动安全框架，其核心是从被动到主动范式的转变，以及通过逆向概率建模和智能多模型协调（结合强化学习与上下文记忆）实现动态、实时的安全参数调整，特别强调在密集城市环境中减少VRU风险。

Abstract: Autonomous and intelligent transportation systems operate in complex urban environments where safety depends on interactions among vehicle behavior, environmental conditions, and vulnerable road users (VRUs) such as pedestrians and cyclists. Most advanced driver assistance systems (ADAS) employ reactive mechanisms that activate only after hazards have emerged, a critical limitation underscored by rising VRU fatalities in the United States. This study introduces PRISM (Proactive Risk Intelligence and Safety Management), an agentic multi-model safety architecture that transitions from reactive crash avoidance to proactive, continuous risk management. PRISM employs inverse crash-probability modeling to convert binary crash classifiers into dynamic, interpretable safety scores. Three specialized models addressing trajectory kinematics, environmental risk, and VRU interaction operate concurrently, coordinated by a reasoning layer incorporating reinforcement learning, contextual memory, and feature-level attribution. The system provides graduated safety interventions across four tiers, from silent monitoring to emergency alerts. Unlike rule-based systems with static thresholds, PRISM dynamically adjusts safety parameters in real time. Validated across 1,296 scenarios from three naturalistic driving datasets without dataset-specific retraining, the system yielded a mean safety score of 68 out of 100, classified 77.6% of scenarios as advisory, and flagged a near-miss rate of 3.8%, with 11% of scenarios escalating to intervention or emergency response. Feature attribution consistently identified trajectory risk and VRU proximity as primary safety factors. PRISM provides a unified, interpretable framework for proactive transportation safety with emphasis on VRU risk reduction in dense urban environments.

cs.LG [Back]

[94] The Dynamics of Continuous Mixture Collapse in Language Models cs.LG | cs.CLPDF

Ali Backour

TL;DR: 本文研究了语言模型中连续混合状态（如词嵌入的加权混合）在推理过程中崩溃的现象，通过理论分析和实证研究识别了三个独立的失效原因：Transformer架构本身会扭曲混合几何结构且训练会放大此效应；softmax读出和自回归反馈构成的动力系统会放大微小差异导致单一成分主导或收缩不同混合使其无法区分；精确保持混合状态通常需要依赖上下文的高维校正。

Details

Motivation: 为了解决预训练语言模型在采用连续状态（如词嵌入混合）进行潜在状态推理时，常常无法保持这些混合状态，导致多路径推理失败的问题。

Result: 理论预测的收缩与放大之间的转变阈值在实证中得到验证，且预训练模型的展开轨迹主要位于放大一侧。

Insight: 揭示了Transformer架构、softmax读出与自回归反馈构成的动力系统是导致混合崩溃的关键机制，并指出精确保持混合需要高维的上下文依赖校正，这对设计更鲁棒的连续状态推理方法具有重要启示。

Abstract: LLMs latent-state reasoning methods replace discrete intermediate tokens with continuous states, such as weighted mixtures of token embeddings, to retain multiple possible reasoning directions rather than committing to one. Yet pretrained language models often fail to preserve these mixtures. We study why through a combination of theoretical analysis and controlled empirical investigations on a variety of models. We identify three independent, distinct sources of failure. First, transformer architectures already distort mixture geometry, and training substantially amplifies this effect. Moreover, the failure can occur even if the model transports mixtures perfectly linearly: the softmax readout and autoregressive feedback form a dynamical system that either amplifies small differences until one component of the mixture dominates or contracts different mixtures until they become indistinguishable. We verify this theoretical prediction empirically: the observed transition between contraction and amplification occurs near the theoretical threshold derived by our analysis, and pretrained-model rollouts lie predominantly on the amplifying side. Finally, we generalize to mixtures of many components and show that exact preservation generally requires context-dependent correction, whose required dimensionality can grow with the number of components.

[95] Post-Training Language Models for Gold-Medal Performance in Coding Competitions cs.LG | cs.AI | cs.CL | cs.MA | cs.SEPDF

Aleksander Ficek, Sean Narenthiran, Mehrzad Samadi, Somshubra Majumdar, Boris Ginsburg

TL;DR: 本文提出了一种针对编程竞赛的端到端大语言模型后训练方法，包括大规模问题收集、合成推理轨迹、监督微调（SFT）、强化学习（RL）以及一种名为GenCorrect的测试时迭代修正策略。该方法训练了Nano-CC和Ultra-CC两个模型，并在IOI竞赛中取得了超越人类金牌选手的成绩。

Details

Motivation: 解决大语言模型在最具挑战性的国际编程竞赛（如IOI、ICPC）中推理能力不足的问题，旨在让AI模型达到甚至超越顶尖人类选手的水平。

Result: 在IOI 2025基准上，Nano-CC模型后训练后分数从130提升至291，结合GenCorrect后达到468分，超过了金牌线（438.3）；Ultra-CC模型达到502分。在IOI 2026前瞻性评估中，Ultra-CC系统在与人相同的约束条件下获得535.4分，超过了金牌线（361.12）和人类最高分（498.27）。

Insight: 创新点在于整合了大规模竞赛问题数据集、合成推理轨迹进行监督微调、强化学习以及反馈驱动的测试时计算策略（GenCorrect）。客观来看，其核心是将传统模型训练与针对竞赛特点的、迭代式的测试时推理优化相结合，构建了一个完整的竞赛专用AI系统，并首次在IOI问题上超越了人类最高分。

Abstract: Competitive programming has become a key test of large language model reasoning, with international competitions such as IOI and ICPC representing its most challenging settings. We present an end-to-end specialization pipeline combining large-scale problem curation, synthetic reasoning traces, supervised fine-tuning (SFT), and reinforcement learning (RL). Using 22,000 curated problems, we train Nemotron-3-Nano-CC (30B-A3B) with SFT and RL and Nemotron-3-Ultra-CC (550B-A55B) with SFT alone. We further introduce GenCorrect, a feedback-driven test-time compute strategy that iteratively generates, evaluates, and refines diverse solutions. On IOI 2025, Nano-CC improves from 130 points to 291 after post-training and to 468 with GenCorrect, exceeding the gold threshold of 438.3 while Ultra-CC reaches 502. Guided by these results, we develop a competition-specific Ultra-CC system and evaluate it prospectively during IOI 2026. Under the same time, internet-access, and submission constraints as human contestants, it scores 535.4 out of 600, exceeding both the gold threshold of 361.12 and the top human score of 498.27. To our knowledge, this is the first AI system to outscore the highest-scoring human contestant on an IOI problem set.

[96] TC-Next: Zero-Shot Multimodal Cyclone Forecasting cs.LG | cs.CVPDF

Zhe Wang, Sijie Chen, Yiming Luo, Daehyun Kim, Chien-Yi Chang

TL;DR: TC-Next是一个零样本多模态深度学习模型，用于热带气旋的轨迹和强度预报。它利用基础模型（如GraphCast）生成的大气运动与热力学场预报数据以及GridSat红外卫星图像，在6至24小时预报时效内，无需针对特定预报系统重新训练，即可实现优于传统规则追踪器和专门化直接追踪器的性能。

Details

Motivation: 旨在解决热带气旋预报中，传统方法（如基于规则的追踪器）精度有限，以及现有深度学习模型通常需要针对特定预报系统进行专门训练的问题，目标是开发一个通用、可迁移且性能更强的多模态预报模型。

Result: 在GraphCast预报场上，相比传统规则追踪器TempestExtremes，轨迹误差降低15%-44%，强度误差降低3-6倍；在Pangu-Weather和IFS HRES预报场上零样本应用，性能依然优于TempestExtremes；在WeatherNext Cyclones的2025年西太平洋季节数据上，其强度误差在所有预报时效均低于该模型的专门直接追踪器，轨迹误差相当或更低。

Insight: 创新点在于构建了一个多模态（大气场+卫星图像）且与基础预报模型解耦的架构，实现了强大的零样本跨模型迁移能力；消融研究表明，多模态融合有效提升了轨迹预报和长时效强度预报的精度，这为构建通用、可迁移的极端天气AI预报系统提供了新思路。

Abstract: We present TropicalCycloneNext (TC-Next), a multimodal deep learning model that forecasts tropical cyclone track and intensity at $6$-$24$ h leads by leveraging a foundation model’s forecast fields of atmospheric kinematic and thermodynamic fields and GridSat infrared satellite imagery. Trained only on GraphCast forecasts over the Western Pacific (WP), yet reliant only on generic atmospheric variables, TC-Next on GraphCast lowers track error by $15$-$44%$ and intensity error by a factor of $3$-$6$ relative to a conventional, rule-based tracker, TempestExtremes; applied without retraining to the forecast fields of Pangu-Weather and IFS HRES, it stays ahead of TempestExtremes on both. Applied zero-shot to the generic weather fields of WeatherNext Cyclones on the 2025 WP season, TC-Next attains lower intensity error at every lead time, and lower or comparable track error, compared to that model’s specialized direct tracker in a deterministic comparison. Our ablation studies show that our multimodal model is able to utilize the additional modality to improve performance in tracking errors at every lead time and in intensity prediction at longer lead times.

[97] Federated LoRA Adaptation of BiomedCLIP Across Four International Chest X-Ray Cohorts cs.LG | cs.AI | cs.CVPDF

Sanjaya Poudel, Nirajan Kunwor, Manish Dhakal, Debesh Jha, Sunil Kumar Gaire

TL;DR: 本文研究了在四个国际胸部X光数据集上，使用联邦学习（FL）结合低秩适应（LoRA）对BiomedCLIP模型进行参数高效微调（PEFT）。结果表明，联邦LoRA适应显著提升了共享类别的AUC性能，接近集中式训练的效果，且FlexLoRA提出的基于SVD的聚合方法至关重要。

Details

Motivation: 解决生物医学影像数据因隐私法规难以集中、且机构间存在扫描仪、协议和计算资源异构性的问题，探索如何在联邦学习框架下高效适应多模态视觉语言模型。

Result: 在四个公共胸部X光队列（美国、越南、西班牙）上的分类任务中，联邦LoRA适应将平均AUC从未适应的0.687提升至0.802，接近集中式参考的0.812。FlexLoRA的SVD聚合方法相比朴素平均带来显著增益（AUC提升0.097），而FedProx优化器未显示出优于FedAvg的额外收益。

Insight: 创新点在于将联邦学习与LoRA结合应用于异构生物医学影像数据，并验证了基于SVD的LoRA更新聚合（FlexLoRA）对性能提升的关键作用，证明了在保护隐私的前提下跨机构协作适应大型视觉语言模型的可行性。

Abstract: Federated learning (FL) lets institutions train a shared model without exchanging data, and Low-Rank Adaptation (LoRA) makes this practical at scale by communicating only compact low-rank updates. Biomedical imaging is a compelling setting for this combination: patient data are archived behind privacy regulations, and institutions differ widely in scanners, protocols, and compute. Such heterogeneity raises the question of how federated LoRA updates should be aggregated, increasingly pressing as multimodal vision-language models become central to medical image analysis. We benchmark federated Parameter-efficient fine-tuning (PEFT) of BiomedCLIP for chest radiograph classification across four public cohorts on three continents (USA, Vietnam, Spain). Federated LoRA adaptation improves shared-class AUC on all four cohorts over the unadapted BiomedCLIP backbone (mean 0.687 to 0.802), showing that the gains come from federated adaptation rather than from the pretrained model’s zero-shot ability. Relative to isolated single-cohort training, federation improves the weaker cohorts while largely preserving the strongest and approaches a centralized reference (0.812) that pools all data. The singular value decomposition (SVD)-based product-space aggregation introduced by FlexLoRA is essential to this gain (naive factor averaging drops mean AUC by 0.097), whereas a drift-correcting optimizer (FedProx) shows no benefit over FedAvg in our single-seed runs, consistent with LoRA’s low-rank updates already limiting client drift. Biomedical vision-language models can thus be adapted collaboratively across heterogeneous, geographically distributed institutions without centralizing data.

cs.IR [Back]

[98] ViSAR: Training-Free Adaptive-$k$ Retrieval for Visual Document Question Answering cs.IR | cs.AI | cs.CL | cs.CVPDF

Adrien Mialland, Marc Plantevit, Julien Gallois, Céline Robardet

TL;DR: 本文提出了ViSAR，一种无需训练的自适应k值检索方法，用于视觉文档问答中的文档检索。该方法在嵌入空间中构建查询条件化的页面级相似度矩阵，动态确定检索页面数量，从而减少检索增强生成的延迟并保持或提高答案准确性。

Details

Motivation: 现有文档视觉问答方法通常固定检索top-k个页面，无论查询复杂度如何，这会增加大型视觉语言模型的延迟并可能降低答案准确性。

Result: 在多个编码器和大型视觉语言模型上，ViSAR将检索增强生成的延迟降低高达58.7%，同时相比固定top-k和自适应检索启发式方法，保持或提高了答案准确性。

Insight: 创新点在于无需训练的自适应k值检索机制，通过查询条件化相似度矩阵动态调整检索数量；相似度矩阵结构与答案准确性相关，为未来检索质量感知的文档理解提供了方向。

Abstract: Document Visual Question Answering (DocVQA) often leverages Retrieval-Augmented Generation (RAG), where late-interaction encoders are commonly used to identify document pages relevant to a user query, before answer generation by a Large Vision-Language Model (LVLM). Existing approaches typically retrieve a fixed top-$k$ number of pages regardless of query complexity, which increases LVLM latency and may degrade answer accuracy. We introduce ViSAR (Visual Semantic Activation Retrieval), a training-free adaptive-$k$ retrieval method for late-interaction visual document retrieval. ViSAR operates directly in the embedding space to construct a query-conditioned page-level similarity matrix that highlights query-relevant semantics and dynamically determines the number of pages to retrieve. Across multiple encoders and LVLMs, ViSAR retrieves compact, query-adapted page sets that reduce RAG latency by up to 58.7%, while maintaining or improving answer accuracy compared with fixed top-$k$ and adaptive retrieval heuristics. Furthermore, we show that the similarity matrix structure correlates with answer accuracy, suggesting future directions for retrieval quality-aware document understanding.

[99] NeoMME: A Single-Tower Multimodal-Native Multilingual Foundation Encoder for Efficient Fine-Tuning and Inference cs.IR | cs.AI | cs.CVPDF

Aurélien Lac, Tony Wu

TL;DR: NeoMME是一种单塔多模态原生多语言基础编码器，支持高效微调和推理。它采用单一双向Transformer编码器处理多语言文本和原始图像块，通过掩码离散扩散文本目标进行预训练，并支持16,384个令牌的上下文。在ViDoRe v3基准测试中，NeoMME-Retriever 260M和800M参数模型分别达到0.523和0.556的nDCG@10分数，优于同规模模型，并通过分层令牌池化和非对称量化实现高效嵌入压缩。

Details

Motivation: 现有多模态模型通常基于生成式视觉语言建模架构，将预训练的视觉编码器与因果语言模型结合，导致在非生成任务（如视觉文档检索）中参数和计算开销过大。NeoMME旨在设计一种高效的单塔编码器，直接处理多模态输入，减少冗余。

Result: 在ViDoRe v3基准测试中，NeoMME-Retriever 260M（0.523 nDCG@10）优于所有低于800M参数的评估模型，而800M参数模型达到0.556 nDCG@10。在NVIDIA L40S上，NeoMME-260M的编码吞吐量约为ColModernVBERT的2倍。通过嵌入压缩技术，模型在保持95%以上基线性能的同时，将嵌入大小压缩了255倍。

Insight: 创新点包括：采用单塔双向Transformer编码器统一处理文本和图像，避免了传统多模态模型的架构冗余；使用掩码离散扩散文本目标进行预训练，增强多模态对齐；支持长上下文（16,384令牌）以处理高分辨率图像；结合分层令牌池化和非对称量化，显著提升推理效率和存储效率。这些设计为多模态编码任务提供了更轻量、高效的解决方案。

Abstract: Multimodal models often build on architectures designed for generative vision-language modeling, typically combining separately pretrained vision encoders with causal language models. Visual document retrievers such as ColPali repurpose these models as encoders, carrying over the parameter and compute overhead of a VLM for a non-generative task. We introduce NeoMME, a family of 260M and 800M-parameter Multimodal and Multilingual bidirectional Encoders that process multilingual text and raw image patches in a single bidirectional Transformer encoder. Both models are pretrained from scratch with a masked discrete-diffusion text objective, conditioned on visible image patches for multimodal examples. Both support a 16,384-token context, enough to encode up to two standard 4K UHD images. To demonstrate its downstream capabilities, we fine-tune NeoMME with jointly trained dense and late-interaction heads. On the ViDoRe v3 benchmark, the resulting NeoMME-Retriever 260M outperforms all evaluated models strictly below 800M parameters with 0.523 nDCG@10, while NeoMME-Retriever 800M reaches 0.556. At a matched 2048x2048 image input size on an NVIDIA L40S, NeoMME-260M encodes pages with about 2x the throughput of ColModernVBERT. Hierarchical token pooling and asymmetric quantization compress late-interaction multimodal document embeddings by 255x while preserving over 95% of baseline nDCG@10. We contribute NeoMME to Hugging Face Transformers and release the pretrained backbone and retrieval-compatible checkpoints under Apache 2.0 at https://hf.co/collections/Hcompany/neomme.

cs.RO [Back]

[100] Designing Versatile Samples for Learned Trajectory Scoring cs.RO | cs.CVPDF

Yaguang Li, Jiaru Zhang, Chuheng Wei, Can Cui, Ziran Wang

TL;DR: 该论文提出了一种为学习型轨迹评分器设计多样化训练样本的方法，以解决现有端到端驾驶策略中轨迹选择模块因规划器候选轨迹过于集中在安全模式而缺乏决策边界附近监督信息的问题。作者通过沿横向（向可行驶边界）和纵向（向前车）扰动记录的人类轨迹来构建生成器，从而创建更具信息量的正负样本数据集。

Details

Motivation: 当前许多端到端驾驶策略会生成候选轨迹池并选择其一，这使得评分器可独立训练，但强规划器生成的提案多集中于安全模式，导致在决策边界附近缺乏足够的监督信息。

Result: 在NAVSIM navtrain数据集上，将基于Transformer的评分器连接到两个冻结的生成式规划器（DiffusionDrive和MeanFuser）进行训练。使用ResNet-34时，通过所设计的数据集，分别在DiffusionDrive上达到90.1 EPDMS（提升0.4 EPDMS），在MeanFuser上达到90.4 EPDMS（提升0.3 EPDMS）。

Insight: 创新点在于设计了一种通过扰动人类轨迹来生成多样化训练样本的方法，专门针对车辆横向和纵向位移轴，从而为评分器提供更丰富的决策边界监督。这提供了一种提升轨迹选择模块性能的数据增强策略，而不需修改规划器本身。

Abstract: Many current end-to-end driving policies emit a pool of candidate trajectories and select one, which makes selection a separable component: a scorer can be retrained while the planner, its backbone, and its trajectory generator all stay frozen. However, many strong planners concentrate their proposals around safe mode, providing limited supervision near decision boundaries. In this work, we design a training dataset that provides more informative supervision for the scorer. In particular, we construct two generators that perturb the logged human trajectory along the two axes a vehicle can be displaced: laterally toward the drivable boundary and longitudinally toward a leading vehicle. The designed dataset produces more informative positive and negative samples than the base planner’s proposal pool. We attach a transformer-based scorer to two frozen generative planners, DiffusionDrive and MeanFuser, and train it on the NAVSIM navtrain dataset. The results of the experiments show that we achieve 90.1 EPDMS on DiffusionDrive and 90.4 EPDMS on MeanFuser when using ResNet-34, with 0.4 and 0.3 EPDMS respectively, from the designed training dataset.

cs.AI [Back]

[101] EmoStance: Response-Side Affective-Orientation Control for Empathetic Response Generation via Emoji Weak Supervision cs.AI | cs.CLPDF

Ziyuan Jin, Yuxuan Ge, Zheng Tian

TL;DR: 本文提出EmoStance模型，用于实现共情对话生成中的响应侧情感导向控制。该方法利用多标注者表情符号分布作为弱监督信号，构建了一个潜在控制空间来近似听者立场，并通过连续前缀嵌入引导冻结的指令调优大语言模型生成回复。

Details

Motivation: 共情回复生成不仅需要决定说什么，还需要决定如何回应当前说话者的情感状态。本文旨在解决响应侧情感导向控制问题，以生成更具情境特异性和感知响应性的回复。

Result: 在包含20名标注者和800个判断的盲测成对评估中，EmoStance模型取得了62.2%的决定性胜率，在上下文特异性和感知响应性方面提升最为明显，并且与外部知识方法保持互补。

Insight: 创新点在于将多标注者表情符号分布作为弱情感-态度证据，用于诱导潜在控制空间，而非作为输出符号或黄金标签；同时，该方法通过建模源侧情感表达、从对话上下文和说话者角色预测软响应侧导向，并利用连续前缀嵌入引导冻结的LLM，实现了有效的响应侧控制。

Abstract: Empathetic response generation requires models to decide not only what to say, but also how to respond to the previous speaker’s affective situation. We formulate this as response-side affective-orientation control and use multi-annotator emoji distributions as weak affective–attitudinal evidence, rather than as output symbols or gold labels, to induce a latent control space that operationally approximates listener stance. We construct EmojiDialogue, an utterance-level extension of EmpatheticDialogues with emoji votes and confidence scores, and propose EmoStance, which models source-side affective expression, predicts a soft response-side orientation from dialogue context and speaker roles, and steers a frozen instruction-tuned LLM through continuous prefix embeddings. In blind pairwise evaluation with 20 annotators and 800 judgments, EmoStance achieves a 62.2% decisive win rate, with the clearest gains in contextual specificity and perceived responsiveness, while remaining complementary to external-knowledge methods. Code, annotation metadata, and reconstruction scripts are available in our GitHub repository: https://github.com/18277390221/EmoStance.

[102] APEx: Distillation of Agent Procedural Experience for Adaptive Deep Research Question Answering cs.AI | cs.CLPDF

Jie Ding, Rui Sun, Xinyuan Zhang, Zeyu Zhang, Xin Liu

TL;DR: 本文提出了APEx框架，用于提升深度研究智能体在复杂、长程问题上的回答能力。该框架通过分层组织交互历史为实例级轨迹记忆和类别级程序技能，并利用执行器、蒸馏器和规划器三个模块的闭环架构进行耦合优化。在测试时，通过技能引导的测试时强化学习实现无需真实标签的自适应改进。

Details

Motivation: 现有方法在利用历史经验进行持续改进时存在不足：要么检索冗长的任务特定轨迹加重决策负担，要么蒸馏出的程序技能与下游策略适应解耦。APEx旨在解决这一问题，实现高效、自适应的经验利用。

Result: 在7个基准测试上的实验表明，APEx达到了最先进的性能，超越了GPT-5.4模型14.7个百分点，并比最强的记忆增强基线高出3.0个百分点。

Insight: 核心创新在于提出了一个分层经验利用框架，将实例记忆与程序技能耦合，并通过三阶段交替GRPO训练范式实现奖励引导的技能蒸馏。其技能引导的测试时强化学习机制允许智能体在无真实标签情况下进行自我改进，并通过技能对齐正则化防止策略漂移。

Abstract: Deep research agents augment large language models with external tools to answer complex, long-horizon questions through multi-turn reasoning. Learning from prior experience is crucial for continual improvement, yet existing methods either retrieve verbose task-specific traces that burden decision-making, or distill procedural skills that remain decoupled from downstream policy adaptation. We propose APEx, a hierarchical experience utilization framework that organizes interaction history into instance-level trajectory memories and category-level procedural skills, and couples them through a closed-loop architecture of Executor, Distiller, and Planner. The three modules are optimized via a three-stage alternating GRPO training paradigm, enabling reward-guided skill distillation rather than fixed-prompt generation. At test time, distilled skills serve as procedural priors for online Planner adaptation through skill-guided test-time reinforcement learning, allowing ground-truth-free self-improvement with skill-alignment regularization to prevent policy drift. Experiments on 7 benchmarks demonstrate that APEx achieves state-of-the-art performance, surpassing GPT-5.4 by 14.7 points and the strongest memory-augmented baseline by 3.0 points.

[103] SCX Router: Streaming Zero-Shot Model Selection with a Decoder-KV Classifier and a Real-World Task Ontology cs.AI | cs.CLPDF

Ihor Stepanov, Aleksandr Smechov, Mykhailo Shtopko, Dmytro Vodianytskyi, Oleksandr Lukashov

TL;DR: 本文介绍了SCX Router，一个轻量级的零样本模型选择路由器，用于在推理时根据任务特性自动选择最合适的LLM。它基于Qwen3解码器和浅层双向评分器构建，通过解码器-KV执行路径高效评估候选模型标签，无需自回归生成。该方法还利用一个包含23个家族、115个任务类型的真实世界任务本体进行训练，并在LiveBench基准上验证了其有效性。

Details

Motivation: 动机在于解决LLM应用场景多样化带来的模型选择难题，即如何在推理时根据任务特性（如速度、成本、质量）自动选择最合适的模型，以克服手动启发式方法难以维护且无法持续优化权衡的局限性。

Result: 在六个LiveBench子集上，该路由器优于候选模型的平均性能；在选定的1000个任务子集上，其聚合top-1得分为0.707，而最强固定模型为0.696，具体增益因基准而异。

Insight: 创新点包括：1) 提出解码器-KV执行路径，在会话中维护纯文本键值缓存，仅编码新对话轮次，高效评估瞬态候选标签；2) 构建大规模真实世界任务本体，用于生成多样化训练数据；3) 将学习到的请求预测与任务特定策略（如资格、成本、缓存重用）显式分离，实现灵活的路由决策。

Abstract: The rapid proliferation of large language models (LLMs) and the growing diversity of their applications presents a unique optimization opportunity: selecting the right model for the task, while optimizing for speed, cost, and quality at a per-task level. However, inference endpoints can vary widely in quality, price, latency, context support, tool use, domain expertise, and reasoning behavior. This heterogeneity makes manual heuristics difficult to maintain and unlikely to achieve consistently favorable speed–cost–quality trade-offs on their own. We introduce \router{}, a lightweight GLiClass-based router that assigns a suitability score to each inference-time model label without autoregressive generation. The released 0.6B-parameter checkpoint combines a Qwen3 decoder with a shallow bidirectional scorer. Its decoder-KV execution path preserves a text-only key–value cache across a session, encodes only new dialogue turns, and evaluates transient candidate-label tokens without adding them to the persistent cache. The same checkpoint also predicts task type, difficulty, reasoning mode, and expected output length, and supports custom zero-shot labels. For task generation, we construct a task ontology with 23 families, 115 task types, 345 routable subtypes, 1,173 synthetic examples, and an orthogonal axis of 30 domains. Using this structure, we generate 150,000 verifier-scored tasks and 15,000 open-ended tasks. We then train the Qwen3 decoder on these tasks, while explicitly separating learned request prediction from per-task policies for attributes such as eligibility, cost, cache reuse, safety, and sovereignty. Across six LiveBench subsets, the router outperforms the mean candidate; on the selected 1,000-task subset, it achieves an aggregate top-1 score of 0.707 versus 0.696 for the strongest fixed model, with benchmark-dependent gains.

[104] SALA: Semantic-Aware Logical Alignment for Complex Reasoning in In-Context Learning cs.AI | cs.CLPDF

Zhao Ji, Wenqing Chen, Zhixuan Chu, Jianxing Yu, Jingping Liu

TL;DR: 本文提出了SALA（语义感知逻辑对齐）框架，用于改进上下文学习（ICL）中复杂推理任务的示例选择。传统方法基于表面相似性或预定义逻辑步骤匹配，而SALA能自动学习任务特定的推理操作，将其嵌入连续语义空间，并利用动态时间规整（DTW）进行软对齐，从而更灵活地匹配多样化的推理逻辑。

Details

Motivation: 解决复杂推理任务中上下文学习的示例选择问题，传统基于表面相似性的检索方法无法捕捉底层解题逻辑，而现有基于逻辑的方法依赖预定义步骤和严格匹配，难以处理灵活多变的推理过程。

Result: 在四个推理基准测试和三种大语言模型（LLMs）上的实验表明，SALA在性能上超越了现有的示例选择方法。

Insight: 创新点在于自动归纳任务特定的推理操作而非依赖固定规则库，并通过连续语义嵌入与DTW实现推理序列的软对齐，兼顾了灵活性与可解释性。从客观角度看，该方法将离散的逻辑步骤匹配转化为连续空间中的相似性度量，是提升复杂推理任务示例选择效果的有效途径。

Abstract: Effective in-context learning (ICL) for complex reasoning relies on selecting the right demonstrations. Traditional retrieval methods based on surface similarity fail to capture the underlying problem-solving logic. Recent logic-based methods address this by matching predefined reasoning steps, but the rigid rules and exact-match criteria is improper to handle flexible or diverse reasoning processes. To address the problem, we propose SALA, a Semantic-Aware Logical Alignment framework. Instead of relying on a fixed inventory, SALA automatically learns task-specific reasoning operations. It then embeds these operations into a continuous semantic space and uses dynamic time warping (DTW) to align the reasoning sequences. This approach allows for soft, flexible matching of reasoning logic while remaining highly interpretable. Experiments across four reasoning benchmarks and three LLMs demonstrate that SALA outperforms existing demonstration selection methods. Further analysis confirms the roles of the operation induction and the logical semantic alignment.

[105] UTP-Bench: Uncertainty-aware Travel Planning Benchmark cs.AI | cs.CLPDF

Etcharla Revanth Rao, Priyanshu Karmakar, Shubhojit Mallick, Manish Gupta, Shreya Ghosh

TL;DR: 本文提出了UTP-Bench，一个用于不确定性感知旅行规划的大规模基准测试。该基准整合了印度504个城市的真实旅行数据，并引入了基于经验延迟分布和人群密度模式的随机扰动来模拟现实世界的不确定性。论文还提出了三个新的评估指标来量化行程在交通延误和人群变化下的鲁棒性。

Details

Motivation: 现有旅行规划基准（如TravelPlanner和TripCraft）假设确定性环境，仅评估静态约束满足，忽略了现实旅行中普遍存在的交通延误、人群波动等不确定性因素，无法评估生成计划的鲁棒性。

Result: 在UTP-Bench上对GPT-5、Qwen3、Mistral和Phi-4等先进大语言模型的实验表明，模型生成的计划与人工编写的计划在时间缓冲、延迟感知的交通调度和人群敏感规划方面存在显著差距。

Insight: 主要创新点在于构建了首个整合真实世界不确定性的旅行规划基准，并提出了三个针对鲁棒性（Buffer Adequacy Score, Crowd-Aware Timing Score, Transport Delay Absorption Score）的量化评估指标，为评估和提升LLMs在动态、随机环境下的规划能力提供了新方向。

Abstract: Large Language Models (LLMs) have recently demonstrated strong capabilities in automated travel itinerary generation. However, real- world travel planning is inherently uncertain: transportation delays, crowd fluctuations, and unexpected stochastic delays frequently inval- idate otherwise feasible schedules. Existing benchmarks like TravelPlanner and TripCraft assume deterministic environments, evaluating only static constraint satisfaction and ignoring whether generated plans remain robust when such uncertainties arise. To address this limitation, we introduce UTP-Bench1 , a large-scale benchmark for uncertainty-aware travel planning. The dataset integrates real-world travel data spanning 504 cities of India, including attractions, restau- rants, accommodations, and multi-modal trans- portation networks. To model realistic disrup- tions, UTP-Bench incorporates empirical delay distributions and crowd-density patterns col- lected from major cities, enabling evaluation of travel plans under stochastic conditions. We further propose three evaluation metrics, namely Buffer Adequacy Score (BAS), Crowd- Aware Timing Score (CATS), and Transport Delay Absorption Score (TDAS), which quan- tify the ability of generated itineraries to main- tain robustness against transit delays and crowd variability. Experiments with state-of-the-art LLMs like GPT-5, Qwen3, Mistral and Phi-4 re- veal substantial gaps between model-generated and human-authored plans, particularly in tem- poral buffering, delay-aware transportation scheduling, and crowd-sensitive planning.

[106] Loom: Weaving Diagnostic Strands into Free-Text Consensus via Embedding-Space Reweighting cs.AI | cs.CL | cs.LGPDF

Ron Begleiter, Katya Egert Berg, Gilad Saban, Gil Shabat

TL;DR: Loom是一个用于根因分析（RCA）的生成式共识框架，通过将模块化启发式方法产生的开放式假设投影到连续嵌入空间中，并采用基于质心的迭代重加权算法解决冲突信号，最终利用轻量级LLM合成步骤生成可靠共识。

Details

Motivation: 解决在工业部署中聚合嘈杂、冲突的文本假设以形成可靠共识的挑战，克服大型语言模型（LLM）代理在上下文限制、幻觉累积和推理延迟方面的不足，同时弥补传统弱监督方法仅限于离散类别的数学限制。

Result: 在OpenRCA基准测试中，Loom在准确性与效率的帕累托前沿上表现优异：在Bank和Market-2数据集上匹配最先进的自主代理性能，在Market-1和Telecom上略有落后，同时在所有四个数据集中每个事件仅需一次LLM调用（速度提升约26倍，使用8B参数合成器时提升约33倍）。

Insight: 创新点在于将开放式假设投影到连续嵌入空间进行聚合，并采用迭代质心重加权算法解决冲突，实现了生成式共识与统计严谨性的结合；客观分析认为其通过轻量级LLM合成步骤平衡了表达性与效率，为工业部署提供了可信任的确定性共识生成方法。

Abstract: Aggregating noisy, conflicting textual hypotheses into a reliable consensus is a fundamental challenge when deploying NLP systems in real-world industrial settings. While monolithic Large Language Model (LLM) agents offer unbounded expressivity for tasks like Root Cause Analysis (RCA), they suffer from context limits, compounding hallucinations, and prohibitive inference latency. Traditional weak supervision offers statistical rigor but is mathematically restricted to discrete classes. We present Loom, a generative consensus framework deployed for real-world RCA that bridges these paradigms. Loom aggregates open-form hypotheses emitted by modular heuristics (diagnostic templates dynamically populated with episode-specific entities, times, and metrics) by projecting them into a continuous embedding space, and resolves conflicting signals with an iterative centroid-based reweighting algorithm. The resulting consensus weights ground a single lightweight LLM synthesis step. Evaluated on the OpenRCA benchmark, Loom occupies the accuracy–efficiency Pareto frontier: it matches a state-of-the-art autonomous agent on Bank and Market-2 and trails on Market-1 and Telecom, while using a single LLM call per incident on all four datasets ($\sim$26$\times$ faster; $\sim$33$\times$ with an 8B-parameter synthesizer). We discuss our deployment experience, highlighting lessons learned regarding the trade-offs between agentic depth and inference latency, negative results in redundancy detection, and how deterministic consensus fosters trust among Subject Matter Experts~(SMEs).

[107] DocHop: Benchmarking Out-of-domain Multi-hop Reasoning in Information-Dense Documents cs.AI | cs.CV | cs.LGPDF

Zhuoran Yu, Le Thien Phuc Nguyen, Jaden Park, Xinyi Gu, Zexue He

TL;DR: 本文提出了DocHop基准测试，用于评估多模态大语言模型在信息密集文档中进行跨领域多跳推理的能力。该基准通过逻辑优先的生成流程构建了2074个样本，涵盖六个任务类别，要求模型结合文档叙述和图表数据进行多步推理。实验表明，现有模型与人类性能存在显著差距，推理增强模型虽有提升，但性能随复杂度增加而下降。

Details

Motivation: 现有基准通常孤立评估图表和文档问答任务，未能充分探索模型利用文本上下文选择、解释和聚合图表证据的关键能力，因此需要构建一个集成图表与上下文推理的基准。

Result: 在DocHop基准上，人类标注者准确率超过90%，而最佳模型仅达到62.83%；推理增强模型性能有所提升，但随着推理复杂度增加而下降。

Insight: 创新点在于通过逻辑优先的生成流程构建可控推理深度和视觉密度的基准，强调模型需先解析文本中的语义参考标签再跨图表聚合证据，为挑战性多跳文档推理提供了受控测试平台。

Abstract: Multimodal Large Language Models (MLLMs) have achieved strong performance on structured visual understanding tasks such as chart and document question answering. However, existing benchmarks typically evaluate these domains in isolation, leaving underexplored a key capability: whether models can use textual context to determine how chart evidence should be selected, interpreted, and aggregated. We introduce DocHop, a benchmark for integrated chart–context reasoning in document-style images. In DocHop, the document narrative specifies multi-step compositional constraints, while charts provide the corresponding data values. Questions are grounded on a semantic reference label defined in the narrative, requiring models to resolve target entities from context before aggregating evidence across multiple charts. To enable systematic evaluation, we construct DocHop via a stochastic logic-first generation pipeline with controllable reasoning depth and visual density, covering 2,074 examples across six task categories. Experiments on a wide range of proprietary and open-source MLLMs show a substantial gap to human performance: annotators achieve over 90% accuracy, while the best model reaches only 62.83%. Reasoning-enhanced models consistently show improved results, but performance degrades as reasoning complexity increases. Overall, DocHop provides a controlled testbed for challenging multi-hop document reasoning.

cs.SD [Back]

[108] SonicCaps: Large-Scale Diverse and Fine-Grained Captioning for Improved Audio-Retrieval cs.SD | cs.CL | cs.MM | eess.ASPDF

Zineb Lahrichi, Marc Ferras, Gaël Richard, Geoffroy Peeters

TL;DR: 本文介绍了SonicCaps，一个大规模、多样化和细粒度的音频描述数据集，包含约70万音频片段和1500万条描述。该数据集通过多模态大语言模型生成，旨在解决现有音频描述数据集语义多样性低、描述缺乏声学细节以及一对一映射无法反映听觉感知模糊性的问题。实验表明，基于SonicCaps训练的CLAP模型在音频检索和零样本分类任务上取得了更好的性能。

Details

Motivation: 现有音频-语言建模的大规模音频描述数据集存在语义多样性有限、描述过于通用且缺乏声学细节、以及一对一音频-描述映射无法充分反映听觉感知固有模糊性的问题。

Result: 人类评估显示，SonicCaps在质量上显著优于现有数据集，其描述被认为更具描述性和精确性。在SonicCaps上使用多描述采样策略训练的CLAP模型，在音频检索和零样本分类任务上取得了持续改进，并在公共和商业基准测试中表现出更强的泛化能力。

Insight: 主要创新点在于利用多模态大语言模型（Qwen3-Omni）结合音频和文本条件，通过结构化提示工程和少样本生成，为每个音频生成约24条多样化的描述（包括主要描述、改写变体和语义标签），从而构建了一个大规模、高多样性和细粒度的音频描述数据集。这种多描述生成策略有效捕捉了听觉感知的模糊性，并为音频-语言模型训练提供了更丰富的信息。

Abstract: Recent advances in audio-language modeling have been driven by large-scale audio captioning datasets. However, existing datasets remain limited by low semantic diversity, generic descriptions lacking acoustic details, and one-to-one audio-caption mappings that poorly reflect the inherent ambiguity of auditory perception. We introduce SonicCaps, a large-scale audio captioning dataset comprising ~15M captions paired with ~700k audio clips, generated using a multi-modal large language model (Qwen3-Omni) conditioned on both audio and text. To explicitly promote diversity, we generate around 24 captions per audio via structured prompt engineering and few- shot generation, spanning main descriptions, rephrased variants (verbosity, style) and semantic tags. Human evaluation shows that SonicCaps is rated significantly higher than existing captioning datasets, with fine-grained analyses indicating that our captions are perceived as more descriptive and precise, which strongly correlates with quality judgments. Finally, training CLAP models on SonicCaps with a multi-caption sampling strategy consistently improves audio retrieval and zero-shot classification, with stronger generalization across public and commercial benchmarks. We release both SonicCaps and two specialized CLAP models on hugging face: https://huggingface.co/datasets/Zineb/SonicCaps.

cs.SE [Back]

[109] Harness Engineering in LLM Tool Use via Agent-Native Reusable Tool Primitives cs.SE | cs.AI | cs.CL | cs.LG | cs.MAPDF

Haibo Jin, Suijin Wang, Xucheng Yu, Haojing Luo, Haohan Wang

TL;DR: 该论文提出了一种名为HEART的框架，通过引入’工具原语’和’ToolFace’中央仓库，解决了LLM工具调用中因API模式不兼容和大规模工具目录导致的性能下降问题。HEART框架包含规划器、路由器和验证器，支持动态工具调用规划、多步执行和反馈驱动的恢复。

Details

Motivation: 现有LLM工具调用方法存在两个关键挑战：一是由于工具输出类型和API模式不兼容导致的多步骤、多轮次推理脆弱性；二是在大规模工具目录下性能下降的问题。

Result: 在五个基准测试中，HEART平均比基于SFT的模型性能提升10%，平均超越GPT-5.4、Claude-4.6-Sonnet和Gemini-3.1-Pro 6%，同时API成本降低高达85%。在50个真实世界任务中，HEART实现了84%的任务完成率，是三个前沿商业模型平均完成率（22%）的3.8倍。

Insight: 创新点包括：1）用自然语言接口替代刚性API模式调用的’工具原语’设计；2）包含25,519个函数的中央仓库ToolFace，支持动态工具检索；3）HEART框架通过规划器、路由器和验证器的协同工作，实现了可靠的复杂工具调用编排。

Abstract: Large language models (LLMs) augmented with external tools have demonstrated remarkable capability in solving complex real-world tasks. However, existing approaches suffer from two key challenges: brittle multi-step and multi-turn reasoning caused by incompatible tool output types and API schemas, and performance degradation under large tool catalogues. To address these, we introduce \textbf{Tool Primitives}, a design that replaces rigid API schema-based invocation with natural language as the interface for tool calling, where each tool is wrapped with an LLM interface that handles schema resolution and execution internally, enabling natural inter-tool communication for nested and multi-turn tool calling. Building on Tool Primitives, we host \textbf{ToolFace}, a centralized repository of 25,519 functions from which LLMs dynamically retrieve only the relevant tools at inference time, eliminating the need to enumerate raw API schemas in context. To orchestrate Tool Primitives and ToolFace reliably in complex settings, we further propose \textbf{HEART}, a \textbf{H}arness \textbf{E}ngineering framework via \textbf{A}gent-native, \textbf{R}eusable \textbf{T}ool Primitives, comprising a Planner, Router, and Verifier that jointly support dynamic tool invocation planning, multi-step execution, and feedback-driven recovery. Experiments on five benchmarks demonstrate that HEART outperforms SFT-based models by $10%$ on average and surpasses GPT-5.4, Claude-4.6-Sonnet, and Gemini-3.1-Pro by $6%$ on average while reducing API cost by up to $85%$. On 50 real-world tasks, HEART achieves $84%$ task completion, $3.8\times$ the average of three frontier commercial models ($22%$).

cs.MM [Back]

Tianqi Xiao, Shiyao Cui, Minghao Zhang, Junxiao Yang, Renmiao Chen

TL;DR: 本文研究了多模态大语言模型（MLLMs）中的跨模态安全漂移问题，即良性的文本查询在结合视觉图像时可能传达有害意图。作者提出了一种轻量级的安全感知表示转移（SRT）方法，通过微调方向向量来缓解此问题，实验表明SRT能在保持模型实用性的同时有效提升安全性。

Details

Motivation: 视觉模态增强了MLLMs的能力，但也引入了安全风险：当文本查询与视觉图像结合时，原本良性的查询可能传达有害意图，导致模型的安全响应率显著降低。本文旨在系统研究这种跨模态安全漂移问题。

Result: 在多个基准测试和模型上的实验表明，所提出的SRT方法能有效提升多种跨模态场景下的安全性，同时保持模型的实用性。

Insight: 创新点在于发现不安全文本处理中的安全信号可以转移，并据此提出了轻量级的方向精炼方法SRT，该方法无需更新整个MLLM主干网络，即可增强模型对视觉风险线索的注意和拒绝触发能力。

Abstract: Visual modality enhances the capabilities of multimodal large language models (MLLMs) but also introduces a safety concern: a benign textual query may convey harmful intent when grounded in a visual image. We term this cross-modal safety drift and our pilot studies show that the safety response rate for such requests is substantially lower than that for requests containing explicitly unsafe text. This paper aims to systematically study this issue. First, we conduct an empirical analysis to identify representative unsafe response patterns. Building on these, we interpret model representations and attentions, revealing that visually risky cues receive limited attention and weakly trigger refusal. Motivated by the observation that safety signals from unsafe text processing can be transferred, we propose safety-awareness representation transfer (SRT), a lightweight direction-refinement method that mitigates cross-modal safety drift with a frozen MLLM backbone. Experiments across multiple benchmarks and models show that SRT effectively improves safety in diverse cross-modal settings while preserving utility. Code is available at https://github.com/cucu220123/safety-awareness.

[111] The Missing Temporal Link: Temporal Context Routing for Script-Driven Audio-Video Generation cs.MM | cs.CVPDF

Yichen Liu, Quanwei Zhang, Haozhe Wang, Donghao Zhou, Xiaojie Li

TL;DR: 本文提出了一种名为Temporal Context Routing（TCR）的新方法，用于解决脚本驱动的音视频生成中时序控制不足的问题。该方法通过将脚本中的精确时序（如镜头切换和对话时间点）映射到音视频生成的共享时间轴上，从而确保生成的音视频内容严格遵循脚本的时序要求。

Details

Motivation: 现有的联合音视频生成模型虽然在视觉质量和音画同步方面取得了进展，但难以精确控制镜头切换和对话发生的具体时间点，这限制了其在脚本驱动内容创作中的应用，因为时序错误会破坏叙事连贯性和观看体验。

Result: 在200个测试脚本上，与基线模型相比，TCR将镜头边界平均绝对误差（Shot Boundary MAE）降低了96%（从1.11秒降至0.042秒），并将对话在0.5秒容差内的准确率（Dialogue Acc@0.5 s）从28.3%提升至84.1%，同时保持了与基线相当的视觉质量和音画同步水平。用户研究也表明参与者在所有五个评估维度上都更偏好TCR。

Insight: 论文的核心创新点是将时序对齐从音视频之间扩展到包含结构化脚本，提出了TCR机制来显式地将脚本时序路由到音视频生成过程中。这为解决多模态生成中细粒度时序控制问题提供了一个新的思路，即通过引入额外的时序上下文信息来精确引导不同模态的生成过程。

Abstract: Joint audio-video generation models have made substantial progress in visual quality and audio-visual synchronization. However, they still provide limited control over when shot transitions occur and dialogue is spoken. This limitation constrains their application in script-driven content creation, where timing errors can undermine narrative coherence and the viewing experience. Current joint generators align video and audio representations on a shared temporal axis, yet the precise timing of shots and dialogue specified in a structured prompt is encoded only in the prompt’s text representation and remains unaligned with the temporal coordinates of either modality. Consequently, video and audio may remain synchronized with each other while both fail to follow the script timeline. This mismatch motivates us to extend temporal alignment beyond video and audio to include the structured script. We therefore introduce Temporal Context Routing (TCR), which maps the script timing onto the shared temporal axis of video and audio generation and routes each prompt’s guidance to the corresponding positions in both modalities. Compared with the baseline on 200 test scripts, TCR reduces Shot Boundary MAE by 96%, from 1.11 s to 0.042 s, and raises Dialogue Acc@0.5 s from 28.3% to 84.1%. TCR achieves these improvements while maintaining visual quality and audio-visual synchronization comparable to those of the baselines. A user study further shows that participants prefer TCR on all five evaluated dimensions.

Table of Contents

cs.CL [Back]

[1] PRO-Step: Step-level Process Reward Optimization for Retrieval-Augmented Generation cs.CL | cs.AIPDF

[2] MemeCULT-1K: Benchmarking South Asian Cultural Context and Humor Understanding of Multimodal Models cs.CL | cs.CVPDF

[3] Learning Evidence Sufficiency Boundaries for Selective Answering in Grounded Multi-Hop QA cs.CLPDF

[4] VakyArth: Evaluating Pragmatic Competence in LLMs across Indic Languages cs.CL | cs.AIPDF

[5] TalkFa: A Unified Benchmark for Farsi Dialogue Generation and Understanding cs.CLPDF

[6] Cite or Decline: A Strict Course-Grounded Chatbot for STEM Lecture Videos cs.CL | cs.CY | cs.IRPDF

[7] Thinking effort aligns between humans and reasoning models in abductive reasoning cs.CL | cs.AIPDF

[8] IDEEA: training-free Input-Dependent stEEring via Activation cluster matching cs.CL | cs.LGPDF

[9] C$^{3}$T: Counterfactual Causal Reasoning for Sentiment Shifts in Social-Media Conversation Trees cs.CL | cs.AIPDF

[10] OBJECTION! Lawyer Agents Mitigate Guilty Bias in Legal Judgment Prediction cs.CL | cs.AIPDF

[11] NE-R1: Enhancing Named Entity Recognition Model via Reinforcement Learning cs.CL | cs.AIPDF

[12] MultiGhostBench: A Multilingual Benchmark for Long-Form LLM-Generated Text Attribution under Distribution Shifts cs.CL | cs.AI | cs.CY | cs.DL | cs.IRPDF

[13] When Decodability Is Not Enough: Logical Validity Representations, Behavioral Dissociation, and Causal Tests in Language Models cs.CL | cs.LGPDF

[14] Predictors of Loneliness in Older Adults Using Multimodal Analysis of Speech and Language cs.CLPDF

[15] From Tokens to Semantics: Leveraging Complementary Signals for Hallucination Detection in Black-Box LLMs cs.CL | cs.AIPDF

[16] Trace as State: Reasoning Traces as Conditional States for Long-Context Transformers cs.CLPDF

[17] Untangling the Mechanisms of Misleading Context in Medical Question Answering cs.CL | cs.AI | cs.LGPDF

[18] From Reweighting to Rewriting: Unlocking the Intervention Effects of Influential Samples in Training Data Attribution cs.CL | cs.AI | cs.LGPDF

[19] User Feedback Provides a Unique Signal that LLMs Can not Detect cs.CLPDF

cs.CV [Back]

[20] Beyond Textual Chain-of-Thought: A Survey on Action-Grounded Reasoning in Autonomous Driving cs.CV | cs.CL | cs.ROPDF

[21] FORGE: Forward-Only Test-Time Adaptation for Integer-Only Vision Models on Microcontrollers cs.CV | cs.AR | cs.LGPDF

[22] FairLens: Benchmarking Fairness in Vision-Language Models for High-Stakes Decision-Making cs.CV | cs.LGPDF

[23] From Visual Cues to Spoken Narration: Rethinking Audio Description cs.CVPDF

[24] Evidential Deep Learning for Multi-Modal Anti-UAV Detection cs.CVPDF

[25] SCULPT: Training Edge Vision Models for Post-Training Quantization Readiness cs.CVPDF

[26] Swin Meets EfficientNet: Lightweight Architectures for GAN-Based Face Forensics cs.CV | cs.AIPDF

[27] AlphaRAD: Grounded Zero-Shot Classification in Chest Radiology via $α$-Corrected Binary Cross Entropy and Factorized Latent Supervision cs.CVPDF

[28] Allocate Before You Embed: Adaptive Visual Input Allocation for Video Embeddings cs.CVPDF

[29] Ten Architectures, One Error: Shared Failure Modes in Hyperspectral Classification under Spatially Disjoint Evaluation cs.CV | cs.LGPDF

[30] CoViT: Instance-Correspondence Contrastive Learning for Vision Transformer cs.CVPDF

[31] Improved Automatic Target Recognition in Synthetic Aperture Sonar Imagery Using Large Deep Neural Networks cs.CVPDF

[32] DESA-TTA: Dynamic EMA and Source Anchoring for Test-Time Adaptation cs.CVPDF

[33] Integrated Laser Scanning and Image-Based Topology Optimization Techniques for Detection and Quantification of Visible and Subsurface Structural Defects cs.CVPDF

[34] Consistency as Regularization for Unsupervised Shadow Removal cs.CVPDF

[35] Video2Reaction: Training Foundation Video Models to Predict Audience Reaction cs.CVPDF

[36] Kirin: Animal Motion Generation from In-the-Wild Video cs.CVPDF

[37] Cross-Model Distillation of a Human-Pose Foundation Model from Unannotated Infant Video for Markerless 3D Pose Estimation cs.CVPDF

[38] Does Playing it Safe Count as Faithfulness? Reassessing LVLM Hallucination Mitigation Methods cs.CVPDF

[39] SignMatch: Matching Dictionary Signs to Continuous Sign Language Video cs.CVPDF

[40] TAPVid-MV: A Benchmark for Tracking Any Point in 3D Across Multiple Views cs.CVPDF

[41] Learning with Volterra Neural Networks: A System Theoretic Perspective cs.CVPDF

[42] Automated Maize Ear Phenotyping Using 3D Reconstructions cs.CVPDF

[43] Who Drives the Probability Game of VLMs? A Temporal Causal Drive Evaluation Framework cs.CVPDF

[44] Detecting Object Hallucinations in Large Vision-Language Models via Cross-Modal Attention Drifts and Mask-Based Verification cs.CVPDF

[45] SelfLift: Accelerating Few-Step Diffusion via Self-Recovering Resolution Transition cs.CVPDF

[46] Test-Time Logit Prompting for Source-Free Missing Modality Adaptation cs.CVPDF

[47] Rendering-in-the-Loop: An Execution-Driven Agent for Interactive Web Development cs.CVPDF

[48] Evidence-Guided Detection, Localization and Explanation for Text-Centric Image Forensics cs.CVPDF

[49] Disease Burden over Skin Tone: Decomposing the Dermatology-AI Generalization Gap cs.CV | cs.AI | cs.LGPDF

[50] World-Coherent Decoding: Self-Verifying Test-Time Planning for World Action Models cs.CVPDF

[51] TAME: Temporal-Aware Mixture-of-Experts for Text-Video Retrieval cs.CVPDF

[52] Lightweight Adaptation of General-Purpose VLMs for Multispectral and SAR Image Understanding cs.CVPDF

[53] LeakageBench: Document-Level Leakage Risk for Redacting Personally Identifiable Information in Document Images cs.CV | cs.CLPDF

[54] Asymmetric Paired-Annotation Learning for Multi-Structure ULF Pediatric Brain MRI Segmentation cs.CVPDF

[55] Signal or Noise? Auditing Rotation-Induced Saliency Drift in Medical and Aerial Imaging cs.CV | cs.AIPDF

[56] InfraPatch: Cross-Task Targeted Grayscale Patch Attacks on Infrared-Adapted Vision-Language Models cs.CV | cs.AIPDF

[57] RouteGraph-Mona: Confusion-Aware Routing Fine-Tuning for Mineral Image Classification cs.CV | cs.AIPDF

[58] Diffusion-Encoding Gaussian Field for Joint k-q dMRI Reconstruction cs.CVPDF

[59] T2LSC-Bench: Benchmarking Localized Semantic Control in Text-to-Image Generation cs.CVPDF

[60] VoRTeC: Taming Foundation Flow for One-step Real time Video Compression cs.CV | cs.AIPDF

[61] If It Moves, Radar Knows: A Physics-Aware Radar Transformer for Class-Agnostic Moving-Object Detection cs.CVPDF

[62] YesTrack: Referring Multi-Object Tracking via MLLM-based Yes/No Verification cs.CVPDF

[63] Towards Zero-Shot Transfer Across Embodiments For Driving VLAs cs.CVPDF

[64] LookStep: Efficient Vision-Language Navigation with Linguistic Foresight and Event Driven Memory cs.CV | cs.ROPDF

[65] CA-OPD: Confidence-Aware On-Policy Distillation for Structured Visual Prediction cs.CVPDF

[66] TempoGround: State-Aware Streaming Visual Grounding with Vision-Language Models cs.CVPDF

[67] Adapting a Foundation Model for Lunar Surface Height Estimation cs.CVPDF

[68] UnCapsTSR: An Unsupervised Transformer-based Image Super-Resolution Approach for Capsule Endoscopy Images cs.CVPDF

[69] Blending Concepts: Benchmarking Visual Metaphor Generation in Text-to-Image Models cs.CV | cs.AIPDF

[70] Beauty is in the AI of the beholder: MLLMs systematically overrate facial attractiveness cs.CV | cs.HCPDF

[71] Doppio: A Dataset for Contactless Weight Estimation of Falling Particles cs.CVPDF

[72] Spatially Aware World Action Model via Geometric Latent Diffusion cs.CV | cs.ROPDF

[73] Fine-Grained Anomaly Perception in Wild UGC-Enhanced Images: A Comprehensive Dataset and Difference-Fusion Framework cs.CV | cs.AIPDF

[74] MARS: What Retrieval Signals Are Hidden in Multimodal Large Language Models for Text-Video Retrieval? cs.CVPDF

[75] RGB-to-IR image translation for infrared vehicle detection in unseen UAV domains cs.CVPDF

[76] Deeply Interleaved Text-Image Contexts for Multimodal LLMs Assessment cs.CVPDF

[77] ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding cs.CV | cs.CLPDF