Table of Contents
- cs.CL [Total: 21]
- cs.CV [Total: 78]
- eess.IV [Total: 2]
- cs.CR [Total: 1]
- cs.AI [Total: 6]
- eess.AS [Total: 1]
- cs.LG [Total: 5]
- cs.RO [Total: 4]
cs.CL [Back]
[1] Mapping Text to Multiplex Graph: Prompt Compression as Lévy Walk-Guided Graph Pruning cs.CL | cs.AIPDF
Yaxin Gao, Yao Lu, Jinhong Deng, Jiaqi Nie, Zhe Tang
TL;DR: 本文提出了一种名为RAGP的新方法，将提示压缩重新定义为在多重图上的冗余感知图剪枝问题。该方法通过构建一个同时建模细粒度注意力依赖和粗粒度语义关系的多重图来捕捉文本中的分布式信息，并利用Lévy游走（一种具有重尾步长分布的随机游走）来高效识别图中的非冗余节点，从而在保持关键信息的同时实现高压缩比。
Details
Motivation: 现有提示压缩方法将文本视为扁平的标记序列，无法捕捉重要信息在多个位置分布并通过局部句法依赖和全局语义关系连接的本质。为了解决这个问题，需要一种能够建模这种关系结构的表示方法。
Result: 在LongBench基准测试上，RAGP在4倍压缩比下取得了49.3的平均分，优于现有的LLM-based压缩方法（如LongLLMLingua在3倍压缩比下为48.8）。此外，RAGP在多项任务上也超越了最先进的基于视觉的文本压缩方法。
Insight: 核心创新点在于将提示压缩问题形式化为图剪枝，并引入了多重图来联合建模不同粒度的关系。Lévy游走的应用是一个巧妙的算法设计，其重尾步长分布能自然地在局部利用和全局探索之间取得平衡，从而高效处理异构的图结构（密集的局部子图和稀疏的全局连接）。
Abstract: Existing prompt compression methods treat text as flat token sequences, failing to capture the distributed nature of important information, which is often spread across multiple locations and connected through both local syntactic dependencies and global semantic relations. Such relational structure is naturally represented as a graph, where tokens or sentences become nodes and their dependencies become edges. To this end, we propose RAGP, which formulates prompt compression as Redundancy-Aware Graph Pruning on a multiplex graph that jointly models fine-grained attention-based dependencies and coarse-grained semantic relations. To efficiently identify non-redundant nodes in this heterogeneous structure (dense local subgraphs and sparse global connections), we employ Levy walks whose heavy-tailed step distribution naturally balances local exploitation with global exploration. Experiments on LongBench show that RAGP achieves an average score of 49.3 under a 4x compression ratio, outperforming existing LLM-based compression methods, such as LongLLMLingua, which attains 48.8 at a 3x compression ratio. Besides, RAGP also surpasses state-of-the-art vision-based text compression paradigms on multiple tasks. The code is available at https://anonymous.4open.science/r/RAGP-B0CB.
[2] Safeguarding LLM Agents from Misalignment through Provenance Analysis cs.CL | cs.AIPDF
Yining She, Yiliang Liang, Eunsuk Kang
TL;DR: 本文提出了一种基于溯源分析的概念框架和名为ProvenanceGuard的多阶段流水线，用于在LLM代理执行工具调用前检测其行为是否与用户意图对齐，以防止因未对齐（misalignment）导致的有害后果。该方法通过分析代理上下文中的可追溯证据来形式化未对齐检测，并在两个基准测试上显著降低了错误率和干预负担。
Details
Motivation: 随着LLM代理获得越来越多强大工具的访问权限，确保其行为与用户意图对齐变得至关重要。现有的运行时护栏依赖于LLM-as-a-judge范式，缺乏系统化的对齐推理框架，常产生不一致或难以审计的判断。
Result: 在Agent-SafetyBench和WorkBench两个基准测试上，使用10个骨干LLM进行评估。与LLM-as-a-judge基线相比，ProvenanceGuard将未对齐轨迹的错误率从42.9%降至1.8%（Agent-SafetyBench）和从32.1%降至17.3%（WorkBench），同时将任务成功轨迹的干预负担从30.5%降至12.8%，且未对齐轨迹上未引入统计显著的额外干预。
Insight: 核心创新在于将溯源分析形式化为一个概念框架，用于系统化地推理LLM代理行为的对齐性。ProvenanceGuard通过多阶段流水线检查三种未对齐类型，提供了结构化、可审计的推理基础，相比依赖单一LLM判断的基线方法更有效、更实用。
Abstract: As LLM agents gain increasing access to powerful tools, ensuring that their actions are aligned with the user’s intent becomes critical. When an agent’s proposed tool invocation deviates from the user’s intent – a phenomenon called misalignment – it may lead to harmful consequences that are difficult to undo. Existing runtime guardrails rely on an LLM-as-a-judge paradigm that lacks a systematic framework for reasoning about alignment, often producing judgments that are inconsistent or difficult to audit. Motivated by provenance analysis, we propose a provenance-based conceptual framework that formalizes misalignment detection as determining whether a proposed tool call is supported by traceable evidence in the agent’s context. Building on this framework, we propose ProvenanceGuard, a multi-stage pipeline that analyzes the agent’s action for three types of misalignment before the selected tool is executed and only allows the action to take place when it is considered aligned with the user’s input query. We evaluated our proposed approach on two different benchmarks, Agent-SafetyBench and WorkBench, across 10 backbone LLMs. Compared to the LLM-as-a-judge baseline, ProvenanceGuard reduces error rate on misaligned traces from 42.9% to 1.8% on Agent-SafetyBench and from 32.1% to 17.3% on WorkBench, while reducing intervention burden on task-successful traces from 30.5% to 12.8% and introducing no statistically significant increase in unnecessary interventions on aligned traces. These results demonstrate that structured, provenance-based reasoning provides an effective and practical foundation for safeguarding LLM agents from misalignment.
[3] Kara: Efficient Reasoning LLM Serving via Sliding-Window KV Cache Compression cs.CL | cs.AIPDF
Shen Han, Yuyang Wu
TL;DR: 本文提出了一种名为Kara的滑动窗口KV缓存压缩方法，旨在解决推理语言模型生成长思维链时KV缓存占用内存大、解码延迟高和吞吐量受限的问题。Kara通过仅对最近生成的上下文进行解码时压缩，利用双向注意力对窗口内的KV对进行评分和选择，并通过Token2Chunk模块将选中的KV对扩展为块以灵活保留重要语义信息。此外，作者将Kara适配到PagedAttention并开发了基于vLLM的推理框架KvLLM，有效减少了KV缓存内存使用并提高了输出吞吐量。
Details
Motivation: 现有KV缓存压缩方法存在两个关键局限：一是其基于阈值的压缩策略可能对吞吐量提升有限甚至降低吞吐量，并可能完全消除序列中某些块的KV对，加剧信息损失；二是它们通常保留孤立的KV对或具有固定边界的块，无法在任意令牌位置保留重要且灵活大小的块。
Result: 广泛的实验表明，提出的Kara和KvLLM实现了一致的性能提升，有效减少了KV缓存内存使用并提高了输出吞吐量。
Insight: 主要创新点包括：1) 滑动窗口压缩策略，仅操作最近生成的上下文以进行高效解码时压缩；2) 利用双向注意力对窗口内KV对进行评分和选择；3) 设计Token2Chunk模块，将选中的KV对扩展为块以灵活保留重要语义信息；4) 将方法适配到PagedAttention并集成到vLLM框架中，形成实用的推理系统KvLLM。
Abstract: Reasoning language models often generate long chain-of-thought (CoT), which accumulates a massive KV cache during the decoding phase and incurs high decoding latency and limited throughput. To address these issues, KV cache compression has emerged as a promising technique for reducing memory overhead by selectively removing unimportant KV pairs while preserving useful ones for subsequent decoding. Nevertheless, we identify two key limitations in existing KV cache compression methods: 1) their threshold-triggered compression policy may provide limited throughput improvement or even reduce throughput, and may fully eliminate KV pairs from certain blocks of the sequence, potentially worsening information loss. 2) they typically retain either isolated KV pairs or fixed-size chunks with rigid boundaries, failing to preserve important flexible-sized chunks at arbitrary token positions. To overcome these limitations, we propose Kara, a sliding-window KV cache compression method that performs decoding-time compression by operating only on the recently generated context. Kara leverages bidirectional attention to score and select informative KV pairs in the window. To enable flexible preservation of important semantic information, we design a Token2Chunk module to expand a subset of selected KV pairs into chunks. Furthermore, we adapt Kara to PagedAttention and develop KvLLM, an inference framework built upon vLLM, which reduces KV cache memory usage and effectively improves output throughput. Extensive experiments demonstrate consistent performance improvements of proposed Kara and KvLLM.
[4] Office Comprehension Benchmark cs.CL | cs.AI | cs.CY | cs.IR | cs.LGPDF
Firoz Shaik, Mateus Picanço Lima Gomes, Tanvir Aumi, Jingci Wang, Milos Milunovic
TL;DR: 本文介绍了Office Comprehension Bench (OCB)，这是首个针对Word、Excel和PowerPoint原生文件格式（.docx、.xlsx、.pptx）及其变体进行LLM系统综合理解的公开基准。OCB包含两个赛道：文件保真度问答测试对办公文档结构（如表格、图表、公式等）的感知能力，而领域问答则基于12个专业领域的真实行业文档测试专家级推理能力。评估采用基于原子声明的LLM法官集成评分方法，结果显示即使最强前沿系统在默认推理模式下领域问答准确率仅约59.3%。
Details
Motivation: 现有基准缺乏对办公文档原生格式的联合评估能力，需要建立能同时测试LLM对办公软件文件结构感知和跨文档专业推理能力的标准化基准。
Result: 在OCB基准测试中，最强前沿LLM系统在领域问答赛道仅达到59.3%准确率；增加思考深度对性能提升有限，而升级到更高产品层级仅带来小幅改进。
Insight: 创新点在于首次构建覆盖三大办公软件原生格式的联合评估基准，采用原子声明分解的客观评估框架，并揭示了当前LLM在专业文档推理任务上的显著局限性。
Abstract: We introduce Office Comprehension Bench (OCB), the first public benchmark to jointly evaluate LLM systems on Word, Excel, and PowerPoint comprehension over native file formats (.docx, .xlsx, .pptx) and their variants. OCB consists of two tracks. File Fidelity Q&A tests structural and visual perception of office artifacts - tables, charts, embedded images, formulas, and app-specific elements such as headers, speaker notes, and named ranges. Domain Q&A tests expert-level reasoning grounded in real-world industry documents across 12 professional domains, with queries requiring multi-step analysis and synthesis across documents. Each reference answer is decomposed into atomic, binary-gradable claims, and an ensemble of LLM judges scores responses against each claim independently. Even the strongest frontier system in its default reasoning mode reaches only about 59.3% on Domain Q&A; increasing thinking depth within a tier does not move performance materially, while moving to a higher product tier yields modest gains. We release the dataset, evaluation tooling, judge prompt, and a public leaderboard.
[5] RusFinChain: A Russian Benchmark for Verifiable Chain-of-Thought Reasoning in Finance with Fuzzy-Aligned Evaluation cs.CLPDF
M. K. Arabov
TL;DR: 本文提出了RusFinChain，首个俄语金融领域可验证思维链推理基准，涵盖17个领域和172个主题，包含5,280个参数化示例。通过引入模糊数值对齐和软注意力对齐等增强指标，评估了8个开源大语言模型，揭示了模型推理步骤对齐与最终答案正确性之间的显著差距。
Details
Motivation: 现有金融推理基准（如FINCHAIN）仅限于英语，而俄语基准（如FINESSE-Bench）缺乏步骤级监督，因此需要构建一个支持可验证思维链推理的俄语金融基准。
Result: 在分层样本上评估了8个开源LLM，生成8,100个响应：模型在步骤对齐上的Hard F1约为0.65，但仅约29%的最终答案正确；新提出的模糊对齐指标与最终答案正确性的相关性（Spearman rho约0.48）优于原始ChainEval（rho约0.38-0.46）。
Insight: 创新点包括：1）首个俄语可验证思维链金融基准，基于可执行Python模板生成防污染数据；2）引入模糊数值对齐和软注意力对齐指标，提升对推理过程的诊断能力；3）公开数据集、代码和评估框架，促进俄语社区可验证金融AI发展。
Abstract: Multi-step symbolic reasoning is essential for robust financial analysis, yet most benchmarks neglect intermediate reasoning steps. FINCHAIN introduced verifiable Chain-of-Thought (CoT) evaluation but is limited to English. FINESSE-Bench includes a Russian block but relies on multiple-choice questions without step-level supervision. We present RusFinChain, the first Russian-language symbolic benchmark for verifiable CoT reasoning in finance. It spans 17 domains, 172 topics, and comprises 5,280 parameterized examples from executable Python templates, ensuring contamination-free evaluation. Each example includes a gold-standard reasoning chain with intermediate numeric values for automatic verification. We also introduce enhanced metrics: Fuzzy Numeric Alignment and Soft-Attention Alignment. We evaluate 8 open-weight LLMs on a stratified sample, generating 8,100 responses. Results reveal a substantial reasoning gap: models achieve Hard F1 of ~0.65 for step alignment, but only ~29% of final answers are correct. Our fuzzy and soft metrics show stronger correlation with final-answer correctness (Spearman rho approx 0.48) than the original ChainEval (rho approx 0.38-0.46), demonstrating superior diagnostic power. We release dataset, code, and evaluation framework to foster verifiable financial AI for the Russian-speaking community.
[6] IsoSci: A Benchmark of Isomorphic Cross-Domain Science Problems for Evaluating Reasoning versus Knowledge Retrieval in LLMs cs.CL | cs.AIPDF
Samir Abdaljalil, Erchin Serpedin, Hasan Kurban
TL;DR: 论文提出了ISOSCI基准，这是一个包含同构跨领域科学问题对的基准测试，旨在分离LLM评估中的推理能力与领域知识检索。研究发现，在五个模型对中，91.3%的推理模式增益依赖于知识而非结构不变性，直接挑战了思维链推理能改进短视程程序性科学问题解决的假设。
Details
Motivation: 为了解决LLM评估中难以区分推理能力和知识检索的问题，作者创建了ISOSCI基准，通过同构问题对来控制逻辑结构，从而更精确地评估推理的效用。
Result: 实验结果显示，推理模式在高度能力模型上仅带来不到5个百分点的准确率增益；在GPQA Diamond上表现优异的推理专用模型o3-mini在ISOSCI上反而表现不佳（下降24.7个百分点），表明基准选择决定了关于推理效用的结论。
Insight: 创新点在于设计了同构跨领域问题对来分离推理和知识，客观分析揭示了当前LLM推理增益主要依赖知识而非纯逻辑结构，这挑战了现有推理评估的假设，并强调了基准设计对结论的重要性。
Abstract: We introduce ISOSCI, a benchmark of isomorphic cross-domain science problem pairs that separates reasoning ability from domain knowledge retrieval in LLM evaluation. Each pair shares identical logical structure but requires different domain-specific knowledge, enabling controlled attribution of reasoning-mode gains. Across five model pairs spanning four model families, we find that 91.3% of reasoning-mode gains are knowledge-dependent rather than structure-invariant (63/69 gains; Wilson 95% CI [82.3%, 96.0%]), directly challenging the assumption that chain-of-thought reasoning improves short-horizon procedural scientific problem-solving. Reasoning toggles on highly capable models provide less than 5 percentage points accuracy gain across all domains, and a reasoning-specialized model (o3-mini) that outperforms its standard counterpart on GPQA Diamond (+19.2 percentage points) underperforms on ISOSCI (-24.7 percentage points), showing that benchmark choice determines conclusions about reasoning utility. We release ISOSCI at https://huggingface.co/datasets/isosci/isosci
[7] MultAttnAttrib: Training-Free Multimodal Attribution in Long Document Question Answering cs.CL | cs.AI | cs.CVPDF
Dang Quang Thien Tran, Quang V. Dang, Vinamra Tyagi, Sai Soorya Rao Veeravalli, Trang Nguyen
TL;DR: 本文提出了MultAttnAttrib，一种无需训练的多模态归因方法，用于长文档问答中精确定位答案的证据来源。该方法利用模型的前馈过程、选定的注意力头和校准阈值来识别文档中的证据。同时，作者还引入了MultAttrEval基准数据集，这是首个专门为长文档多模态归因设计的评估数据集。实验表明，该方法在归因准确性和推理延迟方面均优于现有方法。
Details
Motivation: 随着基于证据的问答系统在AI助手中的应用日益广泛，准确地将生成的答案归因于证据来源对于用户信任和模型安全至关重要。目前，单模态归因已有深入研究，但多模态归因领域仍相对缺乏探索，因此需要开发专门的方法和评估基准。
Result: 实验结果表明，MultAttnAttrib在归因准确性上持续优于多种归因生成方法，包括几种基于提示的强基线方法，并与GPT-5.4等前沿模型性能相当。该方法不仅显著提高了单模态和多模态归因类型的准确性，而且与在同一基础模型上使用提示方法相比，其归因生成延迟最多可减少至七分之一。
Insight: 论文的创新点在于首次提出了无需训练的多模态归因方法，通过利用模型内部注意力机制和预填充过程实现高效证据定位；同时，创建了首个专门针对长文档多模态归因的基准数据集MultAttrEval，为后续研究提供了评估基础。从客观角度看，该方法在保持高性能的同时大幅降低了计算开销，具有实际部署价值。
Abstract: As grounded QA systems are increasingly deployed in AI assistants, accurately attributing generated answers to evidence is critical for user trust and model safety. While unimodal attributions have been explored in depth, the multimodal setting remains relatively under-researched. As a result, we introduce MultAttnAttrib, a training-free attribution-generation method that leverages a model’s prefill pass, selected attention heads, and calibrated thresholds to locate source evidence within a document. To establish baseline results for the method, we introduce MultAttrEval, a complementary benchmark dataset annotated with fine-grained, ground-truth attributions for answer components grounded in multimodal source documents. To our knowledge, this is the first evaluation dataset designed specifically for multimodal attribution in long-form documents. Experimental results show that MultAttnAttrib consistently outperforms a variety of attribution-generation methods, including several strong prompting-based approaches and matches the latest frontier models such as GPT 5.4. Our method not only substantially improves attribution accuracy for both unimodal and multimodal attribution types, but also produces attributions at up to one-seventh of the direct inference latency compared to prompting on the same base model.
[8] FaithMed: Training LLMs For Faithful Evidence-Based Medical Reasoning cs.CLPDF
Zhiyun Zhang, Liwen Sun, Xiang Qian, Chenyan Xiong
TL;DR: 本文提出FaithMed框架，通过将循证医学原则形式化为过程级标准，结合临床医生设计的自动优化评估准则与强化学习，提升大型语言模型在医学推理中的忠实性。该方法在七个医学基准测试中优于基于搜索的基线模型和仅优化结果的强化学习方法，显著提高了任务成功率和推理过程的忠实度。
Details
Motivation: 当前医学大语言模型在推理时要么缺乏主动获取证据的能力，要么使用检索到的证据但未监督其在推理过程中如何评估和应用，无法满足临床决策需要透明、基于可靠证据的忠实推理要求。
Result: 在七个医学基准测试中，FaithMed相比基于搜索的基线模型平均提升9%，相比仅优化结果的强化学习方法提升5.8%，并将循证医学评估准则分数相对于基于搜索的Qwen3基线平均提高了15.5%。
Insight: 创新点在于将循证医学原则形式化为过程级监督标准，并引入结合自动优化评估准则与强化学习的框架，通过步骤级过程奖励分配和优势分组技术，实现对推理过程忠实性的显式监督，这为提升专业领域大模型的可解释性和可靠性提供了新思路。
Abstract: Faithful reasoning is essential in medicine, where clinical decisions require transparent justification grounded in reliable evidence. Current medical LLMs either lack active access to evidence or use retrieved evidence without supervising how it should be appraised and applied during reasoning. To address this, we formalize evidence-based medicine principles as process-level criteria and introduce FaithMed, a framework that combines clinician-designed, automatically refined rubrics with reinforcement learning using step-level process reward assignment and advantage grouping. Across seven medical benchmarks, FaithMed improves over agentic-search baselines (+9% on average) and outcome-only RL (+5.8%), while raising average evidence-based medicine rubric scores over agentic-search Qwen3 baselines (+15.5%). This work demonstrates that explicit step-level supervision can improve both task success and the faithfulness of the reasoning process. Code is available at https://github.com/cxcscmu/FaithMed.
[9] Beyond Skepticism: Evaluating LLMs Pedagogical Intent Reasoning with the Adaptive Pedagogical Vigilance Framework cs.CLPDF
Minghao Chen, Ruihan Zhou, Jiayi Tang, Zihan Xu, Bowen Huang
TL;DR: 本文提出了自适应教学警觉性（APV）框架，用于评估和增强大型语言模型（LLM）对教学交流中潜在教学意图的推理能力。该框架通过一个贝叶斯教学意图推理引擎（PIIE）将教学警觉性形式化为一种自适应机制，并在三个层次上（区分教学体裁、推理结构化教学设置、泛化至真实教育话语）对领先的LLM（如GPT-4o、Claude 3.5）进行评估。
Details
Motivation: 大型语言模型在教学交流（如翻译教学）中，对教学意图的推理能力尚未得到充分探索，这限制了其在构建可靠AI辅助学习系统中的应用。
Result: 实验表明，APV框架显著提升了模型的警觉性：它在区分教学性内容和展示性内容上达到了最强的判别能力，与人类判断高度相关（r=0.958），并且在自然数据上保持了稳健性能，而基线方法则表现下降。
Insight: 创新点在于将教学警觉性形式化为一个基于贝叶斯推理的自适应优化问题，为评估和增强LLM对复杂教学动机的理解提供了一个统一的计算框架，这有助于开发更可靠的AI教育助手。
Abstract: The capacity of Large Language Models (LLMs) to reason about pedagogical intent within instructional communication remains underexplored, particularly in educational domains such as translation pedagogy. To address this, we propose the \textbf{Adaptive Pedagogical Vigilance (APV)} framework, a novel computational formalism that reframes communicative vigilance as an adaptive mechanism for optimizing learning through intent inference. APV formalizes the problem via a Bayesian Pedagogical Intent Inference Engine (PIIE), which models how instructors select content to maximize pedagogical utility and how vigilant learners should inversely reason about latent instructional configurations – encompassing genre, stance, and incentives. We evaluate APV through a three-tier hierarchy: distinguishing instructional genre, reasoning about structured pedagogical setups, and generalizing to authentic educational discourse. Experiments on leading LLMs (e.g., GPT-4o, Claude 3.5) show that APV substantially improves model vigilance. It achieves the strongest discrimination between pedagogical and exposure-based content, correlates highly with human judgments ($r=0.958$), and maintains robust performance on naturalistic data where baseline methods degrade. This work establishes a unified framework for assessing and enhancing LLMs’ understanding of pedagogical motives, advancing the development of more reliable AI-assisted learning systems.
[10] PARTREP: Learning What to Repeat for Decoder-only LLMs cs.CL | cs.LGPDF
Andikawati P Widjaja, Yongjun Kim, Hyounghun Kim, Jaeho Lee
TL;DR: 论文提出了一种名为PartRep的选择性增强方法，用于解决仅解码器LLMs中因果注意力导致的信息流不对称问题。该方法通过仅重复提示中最具信息量的标记，而非整个提示，来重新分配上下文基础并提升推理性能，同时显著降低KV缓存和预填充计算成本。
Details
Motivation: 仅解码器LLMs因因果注意力导致信息流不对称，后期标记比早期标记具有更丰富的上下文基础，而完全重复提示虽能改善性能但会大幅增加KV缓存和注意力计算开销，尤其在长上下文场景中不实用。
Result: 在八个基准测试（包括MMLU、GSM8K和RULER）和三个模型系列（Qwen2.5、Llama3.2、Gemma4）上，PartRep保留了完全重复的大部分性能增益，同时仅使用其59.4%的KV缓存和79.0%的预填充FLOPs。
Insight: 创新点在于使用标记级负对数似然作为选择信号来识别信息量大的标记，并训练轻量级门控网络基于早期层隐藏状态预测这些标记，实现预填充中途的早期退出选择，从而在降低计算开销的同时保持性能。
Abstract: While decoder-only LLMs excel at a vast array of natural language tasks, it suffers from an asymmetric information flow induced by causal attention: later tokens are richer in contextual grounding than earlier ones. A simple and effective remedy is prompt repetition – just appending a second copy of prompt before generation can redistribute grounding across positions and improve reasoning performance. However, full repetition of the original prompt doubles the KV cache footprint and quadruples attention cost during prefill, making it impractical for long-context settings. We propose PartRep, a selective augmentation method that appends only the most informative tokens – rather than the entire prompt. We use token-wise negative log-likelihood (NLL) as a selection signal, motivated by the hypothesis that less predictable tokens are less recoverable from surrounding context and therefore benefit more from late-position repetition. To avoid the heavy cost of a full forward pass for scoring, we train a lightweight gate that predicts high-NLL tokens from early-layer hidden states, enabling token selection during mid-prefill via early exit. Across eight benchmarks (including MMLU, GSM8K, and RULER) and three model families (Qwen2.5, Llama3.2, Gemma4), PartRep retains most of the gains of full repetition while using only 59.4% of its KV cache and 79.0% of its prefill FLOPs.
[11] PairCoder++: Pair Programming as a Universal Paradigm for Verified Code-Driven Multimodal and Structured-Artifact Generation cs.CLPDF
Junhao Chen, Xiang Li, Mingjin Chen, Boran Zhang, Henghaofan Zhang
TL;DR: 本文提出了PairCoder++，一种将结对编程作为通用范式，用于通过代码生成经过验证的多模态和结构化工件（如图表、3D场景、硬件设计等）的方法。该方法通过两个智能体（Driver和Navigator）协作，Driver编写程序，Navigator根据验证证据（如诊断信息、执行结果、当前与目标工件的渲染对比）进行审查，并在错误持续时交换角色，从而显著提升了生成工件的可靠性和质量。
Details
Motivation: 当前大型语言模型通过编写代码生成结构化工件时，单次推理过程是脆弱的，因为决定工件是否存在的编译器、渲染器或模拟器对模型是不可见的，导致生成结果不可靠。
Result: 在17个公共基准测试和来自三个供应商的七个模型上，PairCoder++在几乎所有可验证工件的基准测试上都取得了显著改进，使用了完整的官方指标套件（例如，Blender场景可执行性从0.20提升到0.78；TikZ编译率在每个模型上提升了10到30个百分点），成本是单模型的2.9到9.2倍（总体约7倍）。
Insight: 创新点在于将结对编程范式形式化为一个可验证的代码驱动生成框架，通过两个智能体协作并利用工具链提供的验证证据进行迭代审查，从而在工具链提供有效信息且基线模型有提升空间的任务上实现可靠改进。该方法为代码驱动的多模态生成提供了一个通用且可复现的解决方案。
Abstract: Code is the medium through which large language models generate structured artifacts: charts, scientific figures, vector graphics, CAD models, 3D scenes, and hardware designs are all produced by writing programs. In this regime single pass inference is brittle, because the compiler, renderer, or simulator that decides whether the artifact exists is invisible to the model. We present PairCoder, which grounds review in the toolchain and realizes it as two agent pair programming: a Driver agent writes the program, a Navigator agent reviews it against verification evidence (diagnostics, execution results, and renderings of the current artifact beside the target), and the two switch roles when errors persist. Across 17 public benchmarks and seven models from three vendors, PairCoder improves essentially every benchmark whose artifact is verifiable, on full official metric suites rather than execution alone (for example, Blender scene executability 0.20 to 0.78; TikZ compile rate up 10 to 30 points on every model), at 2.9 to 9.2 times single model cost (about 7 times overall). The improvements concentrate where the toolchain provides an informative oracle and the baseline leaves headroom, and the method ties or mildly regresses where the oracle is weak; we frame pair programming as a reliable recipe for verified code driven generation.
[12] TUDUM: A Turkish-Thinking Reasoning Pipeline for Qwen3.5-27B cs.CL | cs.AIPDF
Baran Bingol, Bahaeddin Turkoglu
TL;DR: 本文介绍了TUDUM项目，这是一个针对Qwen3.5-27B模型的土耳其语思维推理适配管道。其核心目标不仅是让模型用土耳其语回答土耳其语提示，更重要的是使其显式的推理过程（即“思考痕迹”）本身也使用土耳其语。管道从基础检查点出发，通过监督微调和强化学习进行训练，最终发布了公开可用的模型。
Details
Motivation: 解决现有思维模型在处理土耳其语推理任务时，倾向于将土耳其语提示翻译成英语进行内部思考，仅将最终答案本地化为土耳其语的问题。目标是让模型的整个推理过程（包括显式的思考痕迹）都使用土耳其语。
Result: 结果好坏参半。监督微调使模型的推理行为更简短、更一致地使用土耳其语，显著降低了平均响应长度和思考消耗，但降低了基准测试的准确率。强化学习部分恢复了部分数学性能（尤其在AIME24基准上），但并未在所有基准上取得一致提升，也未在报告的Macro-6平均分数上超越基础模型。
Insight: 主要创新点在于将模型生成的“思考痕迹”视为可训练的行为，并构建了一个完整的技术管道（结合SFT和GRPO强化学习）来专门适配土耳其语推理。贡献在于提供了一个技术上诚实的土耳其语思维推理管道和评估框架，而非宣称达到最先进水平。
Abstract: This paper presents TUDUM (Türkçe Düşünen Üretken Model), a project pipeline for adapting a Qwen-family 27B thinking model toward Turkish reasoning. The central problem is not only to answer Turkish prompts in Turkish, but to make the explicit reasoning trace itself Turkish. A thinking model may translate a Turkish prompt into an English-centered internal or visible scratchpad, solve the problem mostly in English, and only localize the final answer. TUDUM instead treats the generated
[13] Beyond Supervised Clarification: Input Rewriting with LLMs for Dialogue Discourse Parsing cs.CLPDF
Yiming Liu, Ziyue Zhang, Zhichao Xu, Xin Yu, Yingheng Tang
TL;DR: 本文研究在无监督条件下利用大语言模型（LLM）重写对话输入以提升冻结的对话语篇解析器（DDP）性能。研究发现，在真实部署场景中，零样本提示或基于解析器反馈的重写策略效果有限，常导致性能下降而非提升，揭示了输入重写作为选择性干预问题的本质。
Details
Motivation: 在现实部署中，缺乏监督澄清数据，需要探索如何利用零样本提示或冻结解析器的反馈来重写片段化或未充分说明的话语（如解决省略或指代），以提升对话语篇解析的准确性。
Result: 在三个SDRT数据集和多个解析器上的实验表明，最后话语澄清的可靠性远低于监督设置下的预期；解析器无关的重写常导致更多性能退化；使用GRPO训练的解析器感知澄清器虽能减少37%的退化，但仍无法持续改善解析性能。
Insight: 创新点在于将澄清重新定义为选择性干预问题，并指出‘可重写性预测’（即干预前判断话语是否可修复）是优化冻结语篇解析器的关键缺失能力，也是改进更广泛智能体流程的重要方向。
Abstract: Rewriting inputs to improve frozen downstream models has become a common strategy in modern NLP pipelines. Prior work on incremental dialogue discourse parsing (DDP) shows that supervised clarification models can rewrite fragmentary or underspecified utterances, such as resolving ellipsis or references, to improve parsing accuracy. In this work, we revisit this idea under realistic deployment conditions, where no clarification supervision is available and the clarifier must rely on zero-shot prompting or feedback from a frozen parser. Across three Segmented Discourse Representation Theory (SDRT) datasets and multiple parsers, we find that last-utterance clarification is far less reliable than suggested by supervised settings. Parser-agnostic rewriting often introduces more regressions than repairs, as edits that enable fixes also disrupt discourse cues relied upon by the parser. A best-of-8 rewriting analysis further reveals a practical ceiling: a large fraction of errors are not repairable through input rewriting alone. A parser-aware clarifier trained with GRPO reduces regressions by up to 37% by learning conservative abstention, yet still fails to produce selectivity-aware clarifications that consistently improve parsing. Together, these findings recast clarification as a selective intervention problem. We identify rewritability prediction, deciding whether an utterance is repairable before intervention, as the key missing capability for input-side optimization of frozen discourse parsers, and a critical direction for improving agentic pipelines more broadly.
[14] EduArt: An educational-level benchmark for evaluating art history knowledge in large language models cs.CL | cs.CVPDF
Gianmarco Spinaci, Lukas Klic, Giovanni Colavizza
TL;DR: 本文提出了EduArt基准测试，这是一个用于评估多模态大语言模型在艺术史知识和视觉推理方面能力的教育级基准。该基准包含871个人工编写的问题，涵盖两种语言和七种题型，评估了12个模型在默认答案模式和需要书面论证的动机模式下的表现。研究发现，多项选择题的准确率接近天花板，但开放题型准确率显著下降，表明单一题型基准会高估模型的实际能力。
Details
Motivation: 当前大语言模型在通用基准测试上表现接近天花板，但无法反映其在特定学科（如艺术史）内的真实能力。现有艺术相关评估多依赖合成问题，且缺乏题目层面的属性分析。
Result: 在EduArt基准上，六个模型在多项选择题上的准确率接近饱和（接近天花板），但在开放完成题型上最高准确率降至23.9%（Claude Opus 4.6），在错误识别题型上降至6.2%（Claude Sonnet 4.6）。基准显示出良好的心理测量学特性（平均区分度0.514）。
Insight: 创新点在于构建了一个高质量、多题型、多语言的教育级艺术史基准，揭示了模型的艺术史知识与应用该知识的能力是两种不同的能力。客观来看，该研究强调了单一题型（尤其是选择题）基准的局限性，并指出在艺术史学术等需要生成和操作内容的任务中，全面评估模型能力是负责任使用的前提。
Abstract: Large language models now score near ceiling on general benchmarks, but these aggregate measures reveal little about how models behave within single disciplines. Existing art-focused evaluations rely on synthetic questions and rarely report item-level properties. This paper introduces EduArt, an educational-level benchmark for art-historical knowledge and visual reasoning in multimodal LLMs. EduArt comprises 871 human-authored questions from Italian secondary-school exercises and US Advanced Placement Art History exams, spanning two languages and seven formats from multiple choice to in-text word placement and error identification. Twelve models from six provider families were evaluated under a default answer-only condition and a motivation condition requiring written justification, and characterized using Classical Test Theory and a logistic regression isolating the effects of format, language, image presence, and model. The benchmark showed strong psychometric properties (mean discrimination 0.514, 82.3 percent good discriminators), while multiple-choice accuracy saturated near ceiling for six models, showing recognition formats alone cannot distinguish frontier models. Format was a strong independent predictor of accuracy: models exceeding 94 percent on multiple choice fell to 23.9 percent on open completion (Claude Opus 4.6) and 6.2 percent on error identification (Claude Sonnet 4.6). The motivation condition changed accuracy in a predominantly negative, family-dependent direction. These dissociations indicate that art-historical knowledge and the ability to deploy it are distinct capabilities, and that single-format benchmarks overestimate what models can reliably do. Mapping this capability profile is a precondition for responsible use of multimodal LLMs in art-historical scholarship, where tasks demand producing and manipulating content rather than selecting from fixed options.
[15] CheckRLM: Effective Knowledge-Thought Coherence Checking in Retrieval-Augmented Reasoning cs.CLPDF
Dingling Xu, Ruobing Wang, Qingfei Zhao, Yukun Yan, Zhichun Wang
TL;DR: 本文提出了CheckRLM框架，旨在通过检索增强生成（RAG）技术，在推理过程中及时检查和纠正事实错误，以提高推理语言模型（RLMs）在知识密集型任务中的可靠性。该框架从推理链中提取事实主张以识别知识不一致性，并利用外部知识进行低成本精确修正。
Details
Motivation: 动机是解决推理语言模型在复杂任务中推理链容易包含事实错误的问题，特别是在知识密集型任务中，以提高推理过程的可靠性。
Result: 大量实验表明，CheckRLM显著优于现有基线，在长视野推理中表现出强大的缓解错误累积能力，且成本更低。
Insight: 创新点在于提出了一个及时检查和修正事实错误的框架，通过提取推理链中的事实主张来定位知识不一致性，并利用外部知识进行最小成本但精确的修正，确保推理链与正确知识的一致性。
Abstract: Reasoning Language Models (RLMs) have significantly improved performance on complex tasks by extending the reasoning chain. However, these chains are prone to containing factual errors, particularly in knowledge-intensive tasks. To address this issue, we propose CheckRLM, a framework that improves the reliability of the reasoning process through Retrieval-Augmented Generation (RAG) by timely checking and correcting factual errors. Specifically, CheckRLM extracts factual claims from the reasoning chain to identify and localize subtle knowledge inconsistencies during inference. Upon detection of errors, a refinement mechanism performs minimal-cost yet precise corrections by leveraging external knowledge, ensuring coherence between the reasoning chain and correct knowledge. Extensive experiments demonstrate that CheckRLM substantially outperforms existing baselines, exhibiting a strong capability to mitigate error accumulation in long-horizon reasoning with lower costs. The code and data are available at https://github.com/AI9Stars/CheckRLM.
[16] Know Your Source: A Public Knowledge Store for Media Background Checks cs.CLPDF
Benjamin Nichols, Michael Schlichtkrull, Nedjma Ousidhoum
TL;DR: 本文介绍了MEDIAREF，一个公开可用的基于网络文档的知识库，旨在支持媒体背景检查（MBC）的生成与评估。该工作解决了现有基于检索增强生成（RAG）的自动事实核查系统因依赖可能不可靠的专有搜索API而导致的成本高、可复现性差的问题。
Details
Motivation: 现有基于RAG的自动事实核查系统常假设检索到的证据是可靠的，但现实信息可能相互冲突、过时或源自不可靠/有偏见的来源。媒体背景检查（MBC）通过评估证据来源的可信度来应对这一挑战，但其生成依赖昂贵且专有的搜索API，限制了研究的可复现性。
Result: 论文构建了包含200个媒体来源的MEDIAREF知识库，并评估了广泛使用的LLM在MBC生成任务上的表现。通过自动和定性评估，证明了MEDIAREF能够支持更高质量的MBC生成。
Insight: 核心创新在于构建了一个公开、可复现、低成本的知识库（MEDIAREF），专门用于媒体背景检查任务的评估，从而降低了该领域研究的门槛并提升了可复现性。这为解决RAG系统中证据来源可靠性评估的依赖问题提供了一个实用的基础设施解决方案。
Abstract: LLM-based retrieval-augmented generation (RAG) is increasingly used for automated fact-checking (AFC) and related tasks. By grounding LLM outputs in retrieved evidence, RAG-based systems provide transparent justifications while allowing external information to be updated independently of the underlying model. However, existing approaches often assume retrieved evidence is reliable, although real-world information may be conflicting, outdated, and can originate from unreliable or biased sources. Recent work on source-critical reasoning addresses this challenge through media background checks (MBCs) (Schlichtkrull, 2024), which assess the credibility of evidence sources to support downstream fact verification. However, generating MBCs relies on costly proprietary search APIs, limiting reproducibility. To mitigate this issue, we introduce MEDIAREF, a publicly available knowledge store of web-sourced documents that enables reproducible, low-cost evaluation of MBC generation across 200 media sources. We describe a reproducible methodology for constructing and updating the collection, assess widely used LLMs on the MBC generation task, and demonstrate that MEDIAREF supports higher-quality MBC generation through both automatic and qualitative evaluation.
[17] Language Models as Measurement Apparatus for Culture cs.CLPDF
Kent K. Chang
TL;DR: 本文探讨了语言模型作为文化测量工具的本质，指出其并非被动记录文化现象，而是通过模型、数据、标注和评估等构成的‘装置’主动参与并构建了所测量的文化现实。作者借鉴Karen Barad的‘能动性切割’概念，分析了模型设计选择如何划定现象与工具的边界，并通过影视对话等案例研究，论证了语言模型已内化了其所测量的文化材料，因此边界从一开始就是纠缠的。
Details
Motivation: 动机在于厘清语言模型在量化文化现象时的独特作用，即它们如何作为一种物质-话语实践，主动构成而非被动反映文化现实，以解决当前NLP文化研究中工具与对象边界模糊的问题。
Result: 论文通过三个关于电视和电影对话的案例研究（测量结构、互动和偏差）以及三个对测量装置本身的考察（文化标记的擦除、对历史材料的调谐、在能动性工作流程中的代理），进行了定性的理论分析和实证说明，但未提及具体的基准测试或定量SOTA结果。
Insight: 创新点在于将Barad的‘能动性切割’理论引入NLP文化测量领域，强调测量装置的设计选择是一种同时具有方法论和伦理意义的自觉承诺，倡导一种理论驱动、实证严谨且文化偶然的研究范式，以正视模型内化文化材料所带来的边界纠缠问题。
Abstract: Language models are increasingly used to quantify cultural phenomena, but what makes such measurement distinctively cultural? This paper argues that NLP work on culture is a material-discursive practice: the apparatus – model, data, annotation, evaluation – participates in constituting the cultural reality it measures, rather than passively recording it. Drawing on Karen Barad’s concept of the agential cut – the contingent boundary between phenomenon and instrument – I show that the apparatus’s substantive design choices draw such boundaries, and that the boundary is entangled from the start because language models have already internalized much of the cultural material they measure. I illustrate this through three case studies on television and film dialogue (measuring structure, interaction, and deviation) and three examinations of the apparatus itself (erasure of cultural markers, attunement to historical material, and agency in an agentic workflow). This big picture analysis proposes a research program that is theory-driven, empirically rigorous, and culturally contingent, treating each agential cut as a conscious commitment, at once methodological and ethical.
[18] Will Scaling Improve Social Simulation with LLMs? cs.CLPDF
Caleb Ziems, William Held, Su Doga Karaca, David Grusky, Tatsunori Hashimoto
TL;DR: 本文研究了大型语言模型（LLM）在社交模拟中的表现，探讨了模型规模扩展（scaling）是否能够提升模拟的保真度（fidelity）。研究发现，在大多数社交模拟子领域（如意见建模和行为模拟）中，模型规模的扩大确实能带来显著改进，尤其是在训练数据中代表性较好的群体上。然而，在纵向预测、代表性不足的观点以及涉及人类认知偏差（如风险规避）的任务上，规模扩展带来的改进有限或不稳定。
Details
Motivation: 当前基于LLM的社交模拟方法虽然前景广阔，但其保真度尚不足以被广泛采用。本研究旨在探究语言模型的规模扩展范式是否能有效缩小这一保真度差距，还是说模拟保真度与模型的通用能力正交，从而需要独立的研究关注。
Result: 研究使用了从10^18到10^20 FLOPs固定计算预算下预训练的85个Qwen3架构的Transformer LLM，以及35个参数高达700亿的更大、能力更强的开源模型进行评估。结果表明，大多数行为和意见模拟任务会随着模型规模迅速提升性能，尤其是在英语网络语料库中代表性好的群体上。然而，纵向预测、代表性不足的观点以及与通用知识推理基准（如MMLU）相关性低的任务，其性能随规模提升较慢。在行为模拟中，规模扩展未能改善模型对人类认知偏差（如风险规避）和启发式（如从相关任务中学习关联奖励）的校准能力。
Insight: 论文的核心创新点在于系统性地将规模扩展定律（scaling laws）应用于评估LLM在社交模拟多个子领域的保真度，并揭示了模拟性能提升与模型规模、训练数据代表性以及任务与通用能力基准的相关性之间的复杂关系。从客观角度看，该研究的重要洞见是：虽然规模扩展在多数情况下能提升社交模拟质量，但它并非万能药；对于涉及特定人类心理机制或数据稀缺领域的模拟任务，可能需要专门的研究和优化方法，而非单纯依赖模型规模的扩大。
Abstract: Large Language Model (LLM) social simulations are a promising research method, but they are not yet faithful enough to be adopted widely. In this work, we investigate whether the current scaling paradigm in language modeling is likely to close these gaps, or whether simulation fidelity is orthogonal to general capabilities and therefore deserving of more research attention. We use scaling laws to study the relationship between LLMs’ compute scale, general capability benchmarks, and the fidelity of social simulation in three representative sub-domains: opinion modeling, behavioral simulation, and longitudinal forecasting. Surprisingly, we discover strong compute scaling in all three settings, using a suite of 85 transformer LLMs with the Qwen3 architecture pre-trained on the DCLM web text corpus under fixed-compute budgets from $10^{18}$ to $10^{20}$ FLOPs. Then we evaluate 35 larger and more capable open-weight models up to 70B parameters, allowing us to predict downstream accuracy from loss. This reveals that the majority of behavioral and opinion simulation tasks will rapidly improve with scale, particularly when they involve populations that are well-represented in English web corpora. Longitudinal forecasting and underrepresented opinions scale more slowly, especially when they are less correlated with general knowledge and reasoning benchmarks like MMLU. In behavior simulation, scaling fails to improve model calibration with human cognitive biases like risk aversion, as well as human heuristics like learning correlated rewards from related tasks. On these tasks, even fine-tuned models fail to noticeably scale up performance from 0.5B to 8B parameters. Taken together, we conclude that scale will improve social simulations in most settings, but outliers exist, and improvements will be less reliable in low-resource domains.
[19] Visually Grounded Self-Reflection for Vision-Language Models via Reinforcement Learning cs.CL | cs.CVPDF
Liyan Tang, Fangcong Yin, Greg Durrett
TL;DR: 本文提出了一种名为VRRL的强化学习训练框架，旨在提升大型视觉语言模型在视觉基础任务中的自我反思能力。该框架通过随机掩码轨迹前缀和引入经验回放缓冲区的缓冲滚入，使模型能够从错误中间预测中恢复并学习纠正多样化的失败状态。在涉及表格、图表的视觉基础任务以及空间导航基准测试中，该方法显著提高了分布外数据的平均准确率，优于标准强化学习和面向反思的微调基线。
Details
Motivation: 现有大型视觉语言模型在自我反思过程中往往未能充分关注视觉输入，导致其难以将反馈转化为基于视觉的修正，尤其是在处理分布外图像时。
Result: 在视觉基础任务（涉及表格和图表）和空间导航基准测试上，该方法显著提升了分布外数据的平均准确率，优于标准强化学习和面向反思的微调基线。
Insight: 创新点在于通过强化学习框架显式地引导视觉基础的自我反思，具体通过随机掩码轨迹前缀来强调从错误中恢复，以及利用经验回放缓冲区引入多样失败状态供模型学习纠正，从而有效提升了模型在分布偏移下的鲁棒性。
Abstract: Large vision-language models can reason over multimodal inputs by generating textual chains of thought (CoT). A key capability exhibited in CoT reasoning is self-reflection: revisiting earlier decisions and correcting previous errors. However, existing LVLMs often fail to properly attend to visual inputs during reflection, limiting their ability to translate feedback into grounded corrections, especially for out-of-distribution images. To address this issue, we propose a novel reinforcement learning training framework VRRL, with two components explicitly designed to elicit visually grounded self-reflection. First, we randomly mask trajectory prefixes during training to emphasize recovery from incorrect intermediate predictions rather than making early mistakes. Second, we introduce buffered roll-ins from an experience replay buffer to expose the model to diverse failure states that it must learn to correct. We evaluate our approach on visual grounding tasks involving tables and charts, as well as spatial navigation benchmarks. While off-the-shelf and conventionally fine-tuned models degrade substantially under distribution shift, our method substantially improves average out-of-distribution accuracy over standard RL and reflection-oriented fine-tuning baselines by using self-reflection effectively.
[20] Reasoning LLM Improves Speaker Recognition in Long-form TV Dramas cs.CL | cs.AI | cs.CVPDF
Yuxuan Li, Lingxi Xie, Xinyue Huo, Jihao Qiu, Jiacheng Shao
TL;DR: 本文针对长篇幅电视剧中的说话人识别任务，提出了一个大规模基准数据集DramaSR-532K和一个基于大型推理模型（LRM）的鲁棒方法DramaSR-LRM。该方法通过多模态工具自主聚合上下文证据，显著提升了识别性能，尤其是在声学特征不可靠的短话语上。
Details
Motivation: 长篇幅电视剧的视频理解依赖于准确的说话人识别，但现有方法在处理复杂剧情时面临挑战，需要整合听觉、语言和视觉线索。
Result: 实验结果表明，DramaSR-LRM在DramaSR-532K基准上显著优于现有基线方法，特别是在声学生物特征不可靠的短话语上取得了优异性能。
Insight: 创新点在于构建了大规模多模态说话人识别基准，并利用大型推理模型进行上下文证据的自主聚合，这为复杂场景下的多模态识别任务提供了新的思路。
Abstract: Long-form TV dramas present a formidable challenge for comprehensive video understanding, where deciphering complex storyline often relies on \textbf{speaker recognition}, the task of accurately attributing each spoken utterance to its respective character. In this paper, we advance this field through two primary contributions. (1) We introduce \textbf{DramaSR-532K}, a large-scale benchmark comprising 532K annotated dialogue lines across more than 900 unique characters, necessitating the integration of auditory, linguistic, and visual cues for speaker recognition. (2) We propose \textbf{DramaSR-LRM}, a robust approach built upon a large reasoning model (LRM). DramaSR-LRM is designed to autonomously aggregate contextual evidence via multimodal tool-use, synthesizing diverse inputs to achieve high-fidelity attribution. Experimental results demonstrate that DramaSR-LRM significantly outperforms existing baselines, particularly on short utterances where acoustic biometrics are inherently unreliable. \textit{All the data and code will be made publicly available at the project page: https://www.github.com/198808xc/DramaSR-LRM.}
[21] LACUNA: A Testbed for Evaluating Localization Precision for LLM Unlearning cs.CL | cs.AI | cs.LGPDF
Matteo Boglioni, Thibault Rousset, Siva Reddy, Marius Mosbach, Verna Dankers
TL;DR: 本文介绍了LACUNA，这是一个用于评估大语言模型（LLM）遗忘方法中参数定位精度的首个测试平台。它通过将合成个人的PII注入到预定义的模型参数中，为评估遗忘方法是否真正从存储知识的权重中擦除信息提供了真实参数级定位基准。研究发现，尽管现有SOTA遗忘方法在输出层面表现良好，但参数定位不精确且易受知识重现攻击，而成功的定位能使简单的基于梯度的遗忘方法实现强擦除和鲁棒性。
Details
Motivation: 现有遗忘基准仅评估输出层面，无法确定遗忘是真正从模型参数中擦除知识还是仅仅混淆输出，这因知识重现攻击的成功而引发担忧。需要一种能直接评估参数级定位精度的测试平台来弥补这一差距。
Result: 在基于1B和7B OLMo模型的LACUNA测试平台上对当前SOTA遗忘方法进行基准测试，结果显示这些方法在输出层面性能虽强，但参数定位高度不精确且易受知识重现攻击。当定位成功时，即使简单的基于梯度的遗忘方法也能实现强擦除和对重现攻击的鲁棒性。
Insight: 论文的核心创新点是构建了首个具有真实参数级定位的遗忘测试平台LACUNA，它通过可控的知识注入实现了对遗忘方法定位精度的直接评估。客观来看，这为评估遗忘方法的内部机制而非仅输出行为提供了关键工具，强调了精确参数定位对于实现稳健遗忘的重要性。
Abstract: LLMs memorize sensitive training data, including personally identifiable information (PII), creating a pressing need for reliable post hoc removal methods. Unlearning has emerged as a promising solution, with state-of-the-art(SOTA) methods often following a localize-first, unlearn-second paradigm that targets specific model parameters. However, existing benchmarks evaluate unlearning solely at the output level, leaving open the question of whether unlearning truly erases knowledge from a model’s parameters or merely obfuscates it, a concern reinforced by the success of resurfacing attacks. To bridge this gap, we introduce LACUNA: the first unlearning testbed with ground-truth parameter-level localization. LACUNA injects PII of synthetic individuals into predefined parameters of 1B and 7B OLMo-based models via masked continual pretraining, enabling direct evaluation of whether unlearning targets the weights responsible for knowledge storage. We use LACUNA to benchmark current SOTA unlearning methods and find that, despite strong output-level performance, existing methods are highly imprecise and susceptible to resurfacing attacks. We further show that when localization is successful, even a simple gradient-based unlearning method achieves strong erasure and robustness to resurfacing attacks, highlighting the importance of precise unlearning. We release LACUNA to complement behavioral evaluations and drive further advances in robust, localization-based unlearning.
cs.CV [Back]
[22] CPG-PAD: Concept-Informed Prompts Guided Presentation Attack Detection cs.CV | cs.AIPDF
Haoyuan Zhang, Xiangyu Zhu, Li Gao, Ajian Liu, Siran Peng
TL;DR: 本文提出了CPG-PAD框架，用于提升人脸呈现攻击检测（PAD）的跨域泛化能力。该框架通过引入模型级概念引导，利用可解释AI技术自动发现与攻击相关的视觉概念，并生成热图来指导提示学习，使学习到的表示能捕获可迁移的攻击线索。
Details
Motivation: 现有PAD模型因传感器、光照和攻击材料的变化，在未见过的域上泛化能力不足。尽管视觉语言模型（VLMs）展现出强大的泛化能力，但其在PAD中的应用受限，因为通常基于类别标签监督优化的学习提示未能与细粒度的攻击相关视觉语义明确对齐，导致表示过拟合于特定域的伪影。
Result: 在九个基准数据集上的大量实验表明，CPG-PAD在多源、有限源和单源设置下，均能一致地实现最先进的跨域性能。
Insight: 创新点在于将模型级概念引导引入提示学习过程，通过视觉概念驱动增强模块自动发现PAD相关概念并生成热图，再通过基于提示的概念注入机制将这些概念整合到提示空间中。这使模型能够捕获可泛化且域不变的攻击线索，同时有效抑制数据集特定的偏差。
Abstract: Presentation Attack Detection (PAD) serves as a crucial safeguard for face recognition systems against presentation attacks such as printed photos, replayed videos, and 3D masks. Despite significant progress, existing PAD models still struggle to generalize across unseen domains due to variations in sensors, lighting, and attack materials. Recent Vision-Language Models (VLMs) have shown strong generalization ability, yet their applications in PAD remain limited because learned prompts, typically optimized under class-label supervision, fail to explicitly align with fine-grained attack-relevant visual semantics. As a result, the learned representations often overfit domain-specific artifacts instead of capturing transferable attack cues. To address this, we propose Concept-Informed Prompts Guided Presentation Attack Detection (CPG-PAD), a framework that introduces model-level concept guidance into the prompt learning process. Specifically, we design a Visual Concept-driven Enhancement (VCE) module that employs eXplainable AI (XAI) techniques to automatically discover PAD-relevant visual concepts and generate concept-associated heatmaps providing localized fine-grained guidance. Guided by these heatmaps, a Prompt-based Concept Injection (PCI) mechanism integrates these concepts into the prompt space through a Visual-Prompt Decoder (VPD) and a concept-mapping loss, enabling prompts to align with the model’s internal concept space. This design enables CPG-PAD to capture generalizable and domain-invariant attack cues while effectively suppressing dataset-specific biases. Extensive experiments across nine benchmark datasets demonstrate that CPG-PAD consistently achieves state-of-the-art cross-domain performance under multi-source, limited-source, and single-source settings.
[23] Spatial-Temporal Expert Learning for Video-based Person Re-identification cs.CVPDF
Xiaofei Hui, Pengfei Wang, Evan Ling, Dezhao Huang, Keng Teck Ma
TL;DR: 本文提出了一种新颖的输入感知可扩展专家模块，用于增强视频行人重识别任务中探索细粒度信息的能力。该方法通过输入感知专家选择机制和时空选择机制，动态激活专家处理相似样本子集，并提升其对时空细粒度差异的敏感性。
Details
Motivation: 视频行人重识别旨在从图库视频片段中检索查询视频片段中的同一身份，当外观相似的身份难以区分时，利用细粒度特征至关重要。
Result: 该方法在两个大规模数据集上取得了出色的性能。
Insight: 创新点在于提出了输入感知可扩展专家模块，通过动态激活专家处理相似样本子集来挖掘细粒度差异，并设计了可扩展方案以灵活增加新专家。
Abstract: Video-based person re-identification (Re-ID) aims to retrieve the same identity in the query video clips from the gallery video clips. To solve this problem, exploiting fine-grained features is of great importance, especially when discriminating identities that are similar in appearance. In this paper, we propose to enhance the ability to explore fine-grained information with a novel input-aware extendable expert module. Instead of updating the network parameters with every sample in the dataset, we aim to train the experts within specific subsets that only contain similar samples and promote their ability to exploit fine-grained information within these similar samples. To achieve this goal, we incorporate two mechanisms in this module: input-aware expert selection mechanism and spatial-temporal selection mechanism. The first mechanism dynamically activates a set of experts on subsets of similar samples, pushing the experts to exploit subtle differences between these similar samples, while the second one further increases their sensitivity to the fine-grained differences in spatial and temporal aspects and allows the experts to dynamically utilize them for different input samples. In addition, to facilitate the expert module, we design an extendable scheme that allows the module to flexibly add new experts when necessary. As a result, our method achieves outstanding performance on two large-scale datasets.
[24] Rethinking Generic Object Tracking Toward Human-Level Perceptual Intelligence cs.CV | cs.AI | cs.LG | cs.MM | eess.IVPDF
Shih-Fang Chen
TL;DR: 该论文旨在通过提升跟踪模型的目标判别、鲁棒适应和几何推理能力，缩小机器视觉跟踪系统与人类视觉感知之间的差距。论文针对通用目标跟踪任务，提出一系列方法以应对目标剧烈变形、复杂干扰、环境变化和未见类别等挑战。
Details
Motivation: 人类视觉系统能够整合先验知识、空间几何和语义上下文来理解复杂场景及其变化，而当前通用目标跟踪模型在泛化和在线适应能力上存在瓶颈，无法可靠处理不可预测的未来事件和现实变化。
Result: 论文未在摘要中提及具体的定量实验结果或基准测试表现。
Insight: 创新点在于系统性地增强跟踪模型的目标判别、鲁棒适应和几何推理能力，以模拟人类视觉的连续性和适应性，从而提升在动态场景中的跟踪可靠性。
Abstract: At the heart of human visual perception lies the ability to maintain a continuous and coherent understanding of the external world. By integrating observations with accumulated experience, the human visual system can continuously adapt to variations in both the target and its surrounding environment, while preserving robust visual continuity as scene dynamics evolve. Human vision can therefore integrate prior knowledge, spatial geometry, and semantic context to understand complex scenes and their changes. As a core problem in computer vision, visual object tracking aims to bring machine perception closer to human visual perception. These capabilities are central to the task of Generic Object Tracking (GOT). In this task, a visual tracker is initialized only with the bounding box of an arbitrarily specified target in the first frame, and must continuously localize the target in subsequent dynamic visual streams. However, future events, observations, and real-world variations are inherently unpredictable; therefore, the model’s generalization and online adaptation capabilities remain bottlenecks. Tracking reliability can deteriorate when the target undergoes severe deformation, is affected by complex distractors, encounters significant environmental changes, or belongs to a category unseen during training. This dissertation aims to narrow the gap between machine visual tracking systems and human visual perception by proposing a series of methods that systematically enhance the target discrimination, robust adaptation, and geometric reasoning capabilities of tracking models.
[25] MIBE: Multi-subject Interaction Benchmark and Evaluator for Personalized Image Generation cs.CVPDF
Zhihan Chen, Yuhuan Zhao, Yijie Zhu, Xinyu Yao, Mengcong Ren
TL;DR: 本文提出了MIBE框架，用于评估多主体个性化图像生成任务。该框架包含一个多主体交互基准（MIB）和一个多主体交互评估器（MIE）。MIB通过解耦的数据方案，系统地覆盖了多种关系类型和场景复杂度，包括一个用于可扩展指标训练的60K对VLM标注的Silver Set和一个覆盖多种SOTA生成器的4K对双盲人工评估Gold Set。MIE是一个轻量级的、基于参考条件的评估器，仅在Silver Set上训练，在Gold Set上表现出强大的跨生成器泛化能力。
Details
Motivation: 当前的多主体个性化图像生成模型在精确渲染所有参考身份及其指定交互方面存在困难，经常遗漏主体、无法保持参考外观或错误分配交互。同时，现有的、主要为单主体保真度设计的指标无法可靠地捕捉这些错误，随着主体数量增加，其排序可分离性和与人类偏好的对齐性严重下降。
Result: 在构建的基准上，Silver Set达到了95.1%的跨VLM偏好一致性。训练得到的评估器MIE在Gold Set上实现了0.922的总体成对准确率（相对于人类偏好），其中在见过的生成器上为0.982，在未见过的生成器上为0.884。MIE的表现优于包括CLIP和DINO变体在内的一系列基线指标。
Insight: 论文的主要创新点是构建了一个专门针对多主体交互生成任务的统一评估框架（基准+评估器）。其核心洞察在于，通过解耦的数据方案（Silver Set用于训练，Gold Set用于验证）和诊断性监督的评估器设计，可以有效解决现有指标在多主体场景下排序可分离性和人类对齐性崩溃的问题，实现了强大的跨模型泛化能力。
Abstract: Multi-subject personalized image generation requires the precise rendering of all requested reference identities and their specified interactions based on a guiding prompt. However, state-of-the-art models still struggle with this process, frequently omitting subjects, failing to preserve reference appearances, or misattributing interactions. Furthermore, existing metrics designed primarily for single-subject fidelity cannot reliably capture these errors, suffering severe degradation in ranking separability and failing to align with human preference as the subject count increases. To address this gap, we introduce Multi-subject Interaction Benchmark and Evaluator (MIBE), a unified framework comprising a Multi-subject Interaction Benchmark (MIB) and a Multi-subject Interaction Evaluator (MIE). MIB systematically covers diverse relation types and scene complexities through a decoupled data regime. This consists of a 60K-pair VLM-labeled Silver Set for scalable metric training and a 4K-pair double-blind Human Evaluation Gold Set covering a diverse range of state-of-the-art generators, with the Silver Set reaching 95.1% cross-VLM preference agreement. To demonstrate the utility of this benchmark, we present MIE, a lightweight, reference-conditioned evaluator trained exclusively on the Silver Set with a dual-head ranking and diagnosis objective. MIE exhibits strong cross-generator generalization on the Gold Set, achieving 0.922 overall pairwise accuracy against human preference, including 0.982 on seen generators and 0.884 on unseen generators. By outperforming a broad spectrum of baseline metrics, including CLIP and DINO variants, MIE demonstrates that diagnostic supervision can preserve ranking separability and human alignment where traditional evaluators collapse.
[26] Computer Vision for Wildlife Monitoring: Detecting Brown Howler Monkeys using YOLO cs.CV | cs.GRPDF
Gabriel Ferri Schneider, Guido Luis Glufke Mainardi, Paulo Ricardo Knob, Patrícia Dias, Márcia Jardim
TL;DR: 本研究探索了利用YOLOv10框架自动检测摄像机陷阱视频中的褐吼猴，以优化森林冠层桥梁使用监测。通过引入辅助数据并调整其比例进行微调，提升了目标物种检测模型的性能，从而支持生物多样性保护工作。
Details
Motivation: 城市化导致森林栖息地碎片化，威胁树栖物种生存；冠层桥梁作为保护策略需持续监测，但传统摄像机陷阱方法产生大量误报图像，耗费保护人员时间，因此需要计算机视觉算法来自动化检测目标物种。
Result: 论文通过微调YOLOv10框架，结合不同比例的辅助数据来改进检测模型，但摘要未提及具体定量结果或基准测试，仅强调技术改进有助于保护工作。
Insight: 创新点在于将辅助数据整合到YOLOv10训练中，以解决野生动物监测中标注数据不足的问题；客观分析表明，该方法可推广到其他树栖物种检测，为计算机视觉在生态保护中的应用提供实践参考。
Abstract: Urban expansion threatens global biodiversity, especially affecting arboreal species due to the fragmentation of forest habitats. The movement of arboreal species across disjointed forest patches increases mortality risk and, thus, compromises their conservation. In this context, the installation of canopy bridges can be a viable strategy; yet continuous monitoring of their use by arboreal species is essential for ensuring their effectiveness, typically carried out with the aid of camera traps. However, this method often produces false-positive images that demand time from conservationists for review. In this context, computer vision algorithms can optimize the task of detecting target species using the canopy bridges. In this study, we explored the automatic detection of brown howler monkeys (Alouatta guariba) in videos obtained by camera traps. Given the need for a large number of annotated images of the target animals to train the algorithms, we tested the incorporation of auxiliary data to improve detection models, fine-tuning the YOLOv10 framework using varying proportions of them. The improvement of these automatic detection techniques contributes to conservation efforts, by providing automatic tools to monitor solutions that minimize the impact of human interference in animals habitats.
[27] Beyond Heatmaps: Unsupervised Concept-Graph Reasoning for Interpretable Visual Explanation cs.CVPDF
Md Mohasin Hossain, Anar Amirli, Robert Leist, Md Abdul Kadir, Daniel Sonntag
TL;DR: 本文提出了一种基于图的概念瓶颈模型（G-CBM），这是一种本质可解释的视觉解释框架。它通过非负矩阵分解（NMF）进行无监督概念发现，将概念表示为每张图像的概念图节点，并使用图注意力网络（GAT）对概念间的非线性依赖关系进行推理。该方法在多个数据集上超越了基线模型，并在皮肤镜基准测试中与需要外部标注的监督方法性能相当。
Details
Motivation: 现有概念瓶颈模型（CBMs）通常依赖预定义的概念词汇或监督标注，缺乏明确的概念定位，并且用单一图像级分数总结每个概念，忽略了空间重复性和概念间的依赖关系。本文旨在解决这些问题，提供一种更精细、无监督且可解释的视觉解释方法。
Result: 在ImageNet、HAM10000、PH2和Derm7pt数据集上，G-CBM相比ResNet-50基线实现了平均相对AUC提升3.7%。在PH2数据集上，仅使用10个概念中的2个就达到了0.96的峰值AUC；在HAM10000上，使用9个概念中的3.8个达到了0.92的AUC。在皮肤镜基准测试中，其性能与需要外部标注的监督方法相当。删除/插入分析表明，学习到的概念排序忠实地反映了模型预测。
Insight: 创新点在于将无监督概念发现（通过NMF）与图表示学习（通过GAT）相结合，构建了每张图像的概念图，实现了区域级概念定位并建模了概念间的非线性依赖。此外，引入可调概念过滤阈值来抑制弱区域特征，这既能选择性使用概念，又经常能提升预测性能，为模型可解释性提供了新的结构化和可操作的视角。
Abstract: Concept Bottleneck Models (CBMs) provide an intrinsically interpretable alternative to post-hoc explanations. However, existing CBMs often rely on predefined concept vocabularies or supervised annotations, lack explicit concept grounding, and summarize each concept with a single image-level score – discarding spatial recurrence and inter-concept dependencies. We propose a Graph-based Concept Bottleneck Model (G-CBM), an intrinsically interpretable framework that performs unsupervised concept discovery via Non-negative Matrix Factorization (NMF) and represents the discovered concepts as nodes in a per-image concept-graph representation. G-CBM matches region-level features to these concept nodes – providing concept grounding and capturing concept recurrence across the image – and applies a \emph{tunable concept filtering threshold} $τ$ to suppress weak region-level features. A Graph Attention Network (GAT) then performs concept-level reasoning by modeling nonlinear dependencies across nodes. Across ImageNet, HAM10000, PH2, and Derm7pt, G-CBM achieves an average relative AUC improvement of 3.7% over a ResNet-50 baseline. Concept filtering frequently improves predictive performance while inducing selective concept use, achieving peak AUC of $0.96$ on PH2 with only 2 of 10 concepts and 0.92 on HAM10000 with 3.8 of 9 concepts. On dermoscopy benchmarks, G-CBM is competitive with supervised approaches requiring external annotations. Deletion/insertion analyses with random ablation controls show that the learned concept ranking faithfully reflects model predictions.
[28] A Cost-Aware, Paired Protocol for Auditing Dynamic Tool Synthesis in Agentic Video Question Answering cs.CVPDF
Aseel Mohamed, Rama AlHamidi, Mohamed Rayan Barhdadi, Rasul Khanbayov, Erchin Serpedin
TL;DR: 本文提出了一种成本感知的配对审计协议，用于评估动态工具合成在代理式视频问答系统中的效果。该协议通过配对比较两个完整系统在相同输入上的表现，联合报告准确性和成本的变化。作者将协议应用于Dynamic-SAGE框架，发现其在提升准确性的同时改变了推理成本结构。
Details
Motivation: 现有代理式视频问答系统在推理时调用固定工具库，导致重复过程需每次从基础工具重建，合成复合工具可能消除此开销，但标准准确率指标无法反映系统如何转移成本，因此需要一种能联合评估准确性与成本的审计方法。
Result: 在SAGE-Bench基准测试中，Dynamic-SAGE相比SAGE基线准确率提升7.5个百分点（p < 0.001），推理轮次和可见工具调用减少约28%，但令牌使用量增加34%、成本上升26%，在视觉和开放式问题上收益最大，在语言和多模态问题上表现中性。
Insight: 创新点包括提出联合评估准确性与成本的配对审计协议，以及实现动态合成、验证和持久注册可执行复合工具的VideoQA框架；客观来看，该研究强调了在评估AI系统时需超越单一准确率指标，全面考量效率与资源消耗的权衡。
Abstract: Agentic Video Question Answering (VideoQA) systems invoke tools during inference, but their tool libraries are fixed, so recurring procedures are rebuilt from primitives on every question. Synthesizing composite tools could remove this overhead, but whether such expansion helps is hard to assess: final-answer accuracy, the standard metric, ignores inference effort, so it cannot reveal how a system shifts cost. We propose a cost-aware, paired protocol for auditing tool-augmented video agents. The protocol pairs two complete systems on the same input for each question and reports their net difference across accuracy and cost jointly. For each question, it sorts the paired outcome into one of six groups defined by joint correctness and by the change in visible tool calls, separating accuracy-preserving efficiency gains from harmful regressions. Significance is reported with McNemar’s test and paired bootstrap confidence intervals. We instantiate the protocol on Dynamic-SAGE, an agentic VideoQA framework that synthesizes, validates, and persistently registers executable composite tools for reuse on unseen questions, and evaluate it against the SAGE baseline on SAGE-Bench. The audit reveals a multi-axis profile that a scalar accuracy comparison would miss: Dynamic-SAGE improves accuracy by 7.5 points (p < 0.001) and reduces reasoning turns and visible tool calls by roughly 28%, while shifting rather than reducing inference cost, as token usage rises 34% and cost 26%. Gains are largest on visual and open-ended questions and neutral on verbal and multimodal ones, and residual failures concentrate on hard, open-ended questions where the pipeline does the most work. By measuring accuracy and cost jointly, the protocol shows where the pipeline-level difference is reliable and where it is not. The code is available at https://github.com/KurbanIntelligenceLab/Dynamic-SAGE.
[29] How Much Future Helps? A Controlled Study of Future-Privileged Supervision for Causal Egocentric Gaze Estimation cs.CVPDF
Jia Li, Wenjie Zhao, Fnu Atisri, Sanskriti Aripineni, Shijian Deng
TL;DR: 本文研究了在因果性（在线）自我中心注视估计任务中，未来帧信息在训练期间作为特权监督的作用。作者提出了一个可控框架，在训练时引入一个可调未来视野的辅助分支，但在推理时丢弃该分支以保持严格因果性。实验发现，未来特权监督能持续提升因果注视预测性能，但性能增益并非随未来视野长度单调增加，而是在一个有限的时间窗口内达到峰值。
Details
Motivation: 自我中心注视估计的现有模型通常在训练和推理时都能访问整个视频（包括未来帧），而实际应用需要严格的因果性在线预测。这种差异引发了两个关键问题：未来上下文是否对视注视估计提供了有价值的信号？如果是，在训练期间，多长的未来视野能最优地监督一个因果模型？
Result: 在EGTEA Gaze+和Ego4D数据集上的实验表明，未来特权监督能持续提升因果注视预测性能。性能增益在有限的未来视野内达到峰值：在EGTEA Gaze+上约为1.7-3.3秒（对应H∈[5, 10]），在Ego4D上约为2.7秒（H=10）。
Insight: 论文的核心创新点在于提出了一个可控的框架，将未来信息作为训练时的特权监督，从而在保持推理严格因果性的前提下，量化并利用了未来上下文的价值。研究发现，轻量级因果模型能够有效吸收未来感知信号，且存在一个最优的未来视野长度，这为实时自我中心注视建模提供了实用指导。
Abstract: Egocentric gaze estimation is commonly studied using models that process the full video with access to future frames, while real-world applications require strictly causal, online prediction. This discrepancy raises key questions: Does future context inherently provide valuable signals for gaze estimation? If so, how much future look-ahead optimally supervises a causal model during training? To investigate, we propose a controlled framework featuring a future-aware branch that accesses a tunable look-ahead horizon during training but is discarded at inference. This design isolates the impact of future context while keeping the inference architecture fixed and strictly causal. Across EGTEA Gaze+ and Ego4D, we find that future-privileged supervision consistently improves causal gaze prediction, confirming its utility. However, performance gains do not increase monotonically with longer look-ahead, but rather peak within a bounded temporal regime. Specifically, optimal performance corresponds to roughly 1.7–3.3 seconds of future context ($H{\in}[5, 10]$) on EGTEA Gaze+ and 2.7 seconds ($H{=}10$) on Ego4D. Our results demonstrate that lightweight causal models can effectively absorb future-aware signals, providing practical guidance for real-time egocentric gaze modeling.
[30] Anti-Prompt: Image Protection against Text-Guided Image-to-Video Generation cs.CVPDF
Yeonghwan Song, Chanhui Lee, Jinsoo Park, Jeany Son
TL;DR: 本文提出了一种名为Anti-Prompt的图像保护方法，通过向图像中注入人眼难以察觉的扰动，来抵御基于文本引导的图像到视频生成模型可能带来的版权和隐私风险。该方法利用模型对文本引导的依赖性，在去噪过程中削弱文本条件交互并增强纯视觉通路，从而在生成的视频中引发明显的视觉不一致和结构错误。
Details
Motivation: 针对近期图像到视频生成技术（在文本引导下将单张图像动画化为逼真视频）带来的严重版权和隐私风险，旨在开发一种保护图像免受此类滥用的方法。
Result: 在两个代表性的I2V架构上的实验表明，该方法实现了强大的保护性能，同时提高了效率和跨模型的可迁移性。
Insight: 核心创新点在于利用了一个经验观察：当移除文本引导时，现代I2V模型的生成质量会显著下降。基于此，Anti-Prompt通过策略性地操纵去噪过程中的条件交互来攻击模型的这一依赖性。此外，论文还引入了一个由视频大语言模型辅助的评估协议，用于系统性地评估保护效果，提供可解释的、基于帧的生成伪影和不一致性分析。
Abstract: Recent advances in Image-to-Video generation allow a single image to be animated into a convincing video under text guidance, raising serious copyright and privacy risks. We propose Anti-Prompt, an image protection approach that injects imperceptible perturbations into an image, inducing visible inconsistencies and structural failures in text-guided I2V generation. Our method is motivated by a simple empirical observation. When text guidance is removed from modern I2V models, generation quality degrades markedly, not only in motion realism but also in subject preservation, structural coherence, and temporal consistency. Building on this insight, Anti-Prompt exploits the model reliance on textual guidance by attenuating text-conditioned interactions during denoising while strengthening visual-only pathways. To further systematically evaluate protection effectiveness, we introduce a Video-LLM-assisted evaluation protocol that provides interpretable, frame-grounded analyses of generation artifacts and inconsistencies. Experiments on two representative I2V architectures demonstrate that our method achieves strong protection performance while improving efficiency and cross-model transferability.
[31] Disentangling Pictorial Cue Understanding from Language Bias in VLMs via Depth Ordering Task cs.CVPDF
Yiqian Liu, Iuliia Kotseruba, John K. Tsotsos
TL;DR: 该论文通过深度排序任务研究视觉语言模型（VLMs）的深度感知能力，旨在分离图像深度线索和语言偏差对模型性能的影响。作者创建了Odd-One-Out Depth（O3-D）数据集，包含3.7万张真实和合成图像及14.7万个图像-问题对，用于评估12个开源和商业模型。结果显示模型深度排序准确率在47%至56%之间，均未超过随机水平，且存在显著语言偏差，表明静态图像数据可能不足以支持深度理解。
Details
Motivation: 研究动机是分离视觉语言模型中图像深度线索和语言偏差的影响，以更精确地评估模型在深度感知任务上的视觉理解能力。
Result: 在O3-D数据集上评估12个模型，深度排序准确率介于47%到56%，均未超过随机水平（50%），且未达到SOTA；同时，新提出的度量指标揭示了模型答案中存在强烈的语言偏差。
Insight: 创新点包括结合深度排序和odd-one-out心理物理任务来分离深度线索，以及引入量化视觉与语言敏感性的新度量方法；客观分析表明，该方法为评估VLMs的视觉理解提供了细粒度分析框架，但结果暗示静态图像训练可能限制深度感知能力。
Abstract: In this paper, we study depth perception of vision-language models (VLMs) to isolate the effects of pictorial depth cues and disentangle vision and language influences on model performance. To this end, we combine depth-ordering and odd-one-out psychophysical tasks: the VLMs are presented with images where one object is at different depth relative to other, otherwise identical, objects, and must determine whether the odd-one-out target is closer or farther to the observer. To create stimuli, we generate 2D views from simulated and real 3D scenes while controlling the presence of individual pictorial depth cues, enabling a fine-grained analysis of cue-level contributions. Language effects are examined by varying referring expression clarity. We also introduce a novel metric to quantify vision-vs-language sensitivities. Applying this methodology, we create the Odd-One-Out Depth (O3-D) dataset with 37K real and synthetic images and 147K image-question pairs. Evaluation of 12 open-source and commercial models on O3-D shows under-utilization of depth cues and depth-ordering accuracies between 47% and 56%, with no model above chance level. At the same time, our metric reveals strong linguistic bias in the answers. Neither chain-of-thought (CoT) nor in-context learning (ICL) significantly improves performance, suggesting that static image data alone may be insufficient for depth understanding. All code, the image generation pipeline, and the O3-D dataset are publicly released at https://github.com/lyiqian/o3-d.
[32] Boosting Infrared Small Target Detection via Logit-Domain Contrast and Adaptive Shape Refinement cs.CVPDF
Handong Zeng, Zhengeng Yang, Shuai Zhang, Shikai Chen, Hongshan Yu
TL;DR: 本文提出了一种名为自适应对比SLSIoU（AC-SLSIoU）的即插即用损失函数，用于解决红外小目标检测中因目标尺寸小、信噪比低、前景背景不平衡和边界模糊带来的挑战。该方法通过引入对数域边际约束来增强目标与困难负样本之间的响应差异，并利用自适应边界抑制来细化目标轮廓，同时结合误报焦点损失来惩罚高置信度虚警。
Details
Motivation: 现有红外小目标检测方法通常依赖激活后概率域的监督，导致弱目标与强杂波产生饱和且接近的概率，限制了弱目标的区分能力；同时，边界模糊和光晕状预测主要源于热扩散、目标尺度小、边界不确定性以及缺乏显式轮廓约束。
Result: 大量实验和跨骨干网络评估表明，该方法在不引入额外推理开销的情况下，能够无缝集成到现有检测器中，并持续提升检测精度和形状质量，证明了其有效性、鲁棒性和泛化能力。
Insight: 创新点在于将对数域（logit-domain）的对比学习思想引入损失函数设计，通过LDMC增强判别能力，并结合尺度感知的自适应边界抑制（ABS）来显式优化目标形状，同时使用误报焦点损失针对性地处理高置信度虚警，为红外小目标检测提供了一种通用的损失函数改进方案。
Abstract: Infrared small target detection (IRSTD) remains challenging due to tiny target size, low signal-to-noise ratio, severe foreground-background imbalance, and blurred boundaries in complex scenes. Existing methods usually rely on post-activation probability-domain supervision for discrimination, where weak targets and strong clutter may produce saturated and close probabilities, limiting weak-target discrimination. Meanwhile, blurred boundaries and halo-like predictions mainly stem from thermal diffusion, tiny target scale, boundary uncertainty, and insufficient explicit contour constraints. To address these issues, we propose Adaptive-Contrastive SLSIoU (AC-SLSIoU), a plug-and-play discriminative and shape-aware loss for IRSTD. Specifically, a Logit-Domain Margin Constraint (LDMC) is introduced to enlarge the response gap between targets and informative hard negatives in the logit space, thereby enhancing weak-target discrimination. Adaptive Boundary Suppression (ABS) applies scale-aware annular penalties to refine target contours and suppress halo-like overflow responses. In addition, False-Alarm Focal Loss assigns larger weights to high-probability negative samples, further penalizing persistent high-confidence false alarms. Without introducing extra inference overhead, the proposed method can be seamlessly integrated into existing detectors and consistently improves both detection accuracy and shape quality. Extensive experiments and cross-backbone evaluations demonstrate the effectiveness, robustness, and generalization ability of the proposed method for infrared small target detection.
[33] Hidden-Shot: Towards One-Shot Task Generalization for Low-Level Vision Generalist Models cs.CVPDF
Shao-Jun Xia, Xianzheng Ma, Zichong Meng
TL;DR: 本文提出了Hidden-Shot，一种用于低级视觉通用模型的隐式提示机制，旨在探索模型在少量样本下对新任务的适应能力。该方法通过提取隐式视觉任务信息、利用全局任务感知的文本提示，并选择性地与任务内处理信息融合，以增强模型在新任务上的单样本泛化能力。同时，论文引入了一个名为C/U评估的数据驱动评估框架（包括3C4U和3C7U两种场景），用于系统性地测试低级通用模型的泛化性能。
Details
Motivation: 当前低级视觉通用模型在零样本/少样本场景下对新任务的泛化能力尚未得到充分验证，其核心挑战在于如何实现从未见任务的有效泛化，并匹配定量评估标准。现有方法在提示工程上虽有进展，但未能在广泛的低级视觉任务中系统性地探索这一差距。
Result: 在七个和十个数据集上的实验分别通过3C4U和3C7U评估框架验证，性能超越了当前最先进的视觉通用模型。Hidden-Shot方法在单样本新任务上表现出优越性能，同时在现有任务上保持了一致的性能。
Insight: 创新点在于提出了一个隐式的、基于视觉任务的提示机制（Hidden-Shot），能够以低成本、最小化修改原模型架构的方式直接注入任务信息，有效提升单样本泛化能力。同时，论文设计了一个系统性的数据驱动评估框架（C/U评估），为低级通用模型的泛化能力提供了更全面的定量评估基准。
Abstract: Despite the intense engagement surrounding low-level vision generalist models, their effectiveness in zero/few-shot scenarios beyond learned tasks remains unverified. The primary challenge of developing an ideal generalist lies in achieving the ability to generalize from new unseen tasks, which also can be assessed by matched quantitative criteria. Existing methods have made some progress in prompt engineering but have not systematically explored this gap across a wide range of low-level visual tasks. Stimulated by the problem, we propose Hidden-Shot, an implicit prompt mechanism aimed at exploring low-level task adaptation in a vision generalist model. Specifically, the method extracts implicit visual task-based information, utilizes a global task-aware textural prompt, and selectively merges implicit information with in-task processing information to enhance one-shot capabilities in new tasks. The overall design performs direct injection in a cost-effective manner, while minimally altering the architecture of the original generalist model. Additionally, we introduce a data-driven evaluation framework termed C/U assessment to cover two basic scenarios, 3C4U (3 conventional and 4 unconventional tasks) for retraining existing models and 3C7U (3 conventional and 7 unconventional tasks) for training from scratch, as a comprehensive assessment to systematically test the generalization ability of low-level generalist models. Experiments on seven and ten datasets outperform the state-of-the-art vision generalist model, respectively verified by 3C4U and 3C7U framework. Our presented Hidden-Shot approach demonstrates superior performance on one-shot new tasks while maintaining consistent performance on existing tasks.
[34] VLAFlow: A Unified Training Framework for Vision-Language-Action Models via Co-training and Future Latent Alignment cs.CV | cs.AI | cs.ROPDF
Guoyang Xia, Fengfa Li, Hongjin Ji, Lei Ren, Fangxiang Feng
TL;DR: 本文提出了VLAFlow，一个用于统一比较视觉-语言-动作模型不同训练范式的流程匹配框架。研究基于包含约5000小时数据的异构机器人数据集OXEMix，在相同的架构和动作空间下，评估了四种训练范式：仅动作建模、语言监督协同训练、未来潜在对齐以及它们的组合。实验表明，组合方法MindLWPI在多个基准测试中实现了最稳定和可迁移的性能。
Details
Motivation: 现有视觉-语言-动作模型在架构、数据、动作空间和评估协议上差异很大，导致不同机器人数据预训练范式的效果难以公平比较。本文旨在提供一个统一的框架，以可控的方式比较VLA训练目标。
Result: 在LIBERO、LIBERO-Plus和SimplerEnv基准测试上的实验表明，仅动作预训练对异构数据敏感，语言监督有助于保持视觉-语言泛化能力，未来潜在对齐改进了状态转移和动作结果建模。组合方法MindLWPI实现了最稳定的整体迁移性能。
Insight: 论文的核心创新在于提出了一个统一的比较框架VLAFlow，并系统性地评估了不同训练信号的作用。从客观角度看，其关键见解是提出了’元动作空间’的观点，即语言和未来潜在表征提供了互补的中间约束，使得异构动作监督更平滑且更具可迁移性。
Abstract: Vision-language-action models (VLAs) have recently advanced robotic manipulation, yet the effects of different robot-data pre-training paradigms remain difficult to compare because existing models often differ in architecture, data, action space, and evaluation protocol. We present VLAFlow (Vision-Language-Action Flow), a unified flow-matching framework for controlled comparison of VLA training objectives. Using a heterogeneous robot corpus, OXEMix, containing approximately 5,000 hours of data from DROID, OpenX-Embodiment, OpenX-Augmented, and RoboCOIN, we evaluate four paradigms under the same pi0-style architecture, shared VLM backbone, action expert, and 14-dimensional action space: action-only modeling (MindPI), language-supervised co-training (MindLPI), future latent alignment (MindWPI), and their combination (MindLWPI). Experiments on LIBERO, LIBERO-Plus, and SimplerEnv show that action-only pre-training is sensitive to heterogeneous data. In contrast, language supervision helps preserve vision-language generalization, while future latent alignment improves state-transition and action-outcome modeling. By combining both signals, MindLWPI achieves the most stable overall transfer performance across benchmarks. These results suggest a meta-action space view: language and future latent representations provide complementary intermediate constraints that make heterogeneous action supervision smoother and more transferable.
[35] Multi-THuMBS: Multi-person Tracking of 3D Human Meshes Beyond Video Shots cs.CVPDF
Jeongwan On, Muhammad Salman Ali, Muneeb A. Khan, Sunwoo Park, Inwoong Moon
TL;DR: 本文提出了Multi-THuMBS方法，旨在解决非受控视频中跨镜头（shot changes）的多人物3D人体网格跟踪问题。该方法利用先进的3D场景先验，在共享的3D空间中重建镜头边界帧，并在该空间内注册人体网格，从而在镜头切换时保持人物身份和运动的一致性。
Details
Motivation: 现有方法在跟踪非受控视频中的多人物3D人体网格时，主要关注复杂交互、遮挡和截断等问题，但普遍忽略了现实视频中频繁的镜头切换。镜头切换导致现有方法丢失人物身份标识，无法重建时间上连贯的轨迹，且现有跨镜头跟踪工作仅限于单人场景，无法处理多人同时出现和交互的真实世界视频。
Result: 大量实验表明，与之前的最先进方法相比，该方法在3D人体网格恢复、相机姿态估计和身份跟踪方面取得了显著改进，确保了跨镜头的高保真运动重建和一致的身份保持。
Insight: 核心创新点在于将跨镜头跟踪问题扩展到多人场景，并利用3D场景先验在共享的3D空间中统一处理镜头边界帧，从而在3D空间而非2D图像空间实现身份和运动的跨镜头关联。这为解决真实世界视频中因视角突变导致的跟踪中断问题提供了新思路。
Abstract: Tracking multi-person 3D human meshes from in-the-wild videos is a highly challenging problem due to complex interactions, frequent occlusions, and severe truncation inherent in unconstrained environments. While recent approaches have improved robustness against these issues, they largely overlook the critical challenge prevalent in real-world footage: frequent shot changes. These abrupt transitions in camera viewpoints often cause existing methods to lose track of human identities and fail in reconstructing temporally coherent trajectories. Although several recent works have explored 3D human mesh tracking under shot changes, they are still limited to single-person scenarios, making them inadequate for real-world videos where multiple people interact and appear simultaneously. To address this limitation, we propose Multi-THuMBS (Multi-person Tracking of 3D Human Meshes Beyond Video Shots) that leverages a state-of-the-art 3D scene prior to reconstruct the two boundary frames in a single shared 3D space. Human meshes are then registered within the shared 3D space, maintaining per-person identity and motion consistency across shot changes. Extensive experiments demonstrate that our approach yields significant improvements in 3D human mesh recovery, camera pose estimation, and identity tracking, thereby ensuring high-fidelity motion reconstruction with consistent identity preservation across shots compared to previous state-of-the-art methods.
[36] DRDN: Decoupled Representation Dynamic Network for From-Scratch ViT Class-Incremental Learning cs.CVPDF
Bingchen Huang, Yifu Chen, Zhiling Wang, Yuanchao Du
TL;DR: 本文提出了一种名为解耦表示动态网络（DRDN）的方法，用于解决ViT模型在从头开始的类增量学习（CIL）中面临的挑战。该方法通过两个正交机制：在共享主干网络上持续应用掩码图像建模（MIM）以保留通用视觉结构，以及采用分层任务令牌扩展和修改的注意力规则来减少任务间干扰，从而提升长期增量序列中的表示质量和分类性能。
Details
Motivation: 解决动态扩展方法在类增量学习中，仅依赖分类监督无法在长增量序列中充分保留任务无关的共享主干表示的问题，具体挑战包括顺序训练导致的跨任务混淆（决策边界偏向近期任务）以及主干中共享表示优化不足导致的长期可区分性下降。
Result: 在从头开始的ViT CIL设置（无外部预训练）下，DRDN在CIFAR100-B0（10步）上达到77.19%的平均准确率，优于DKT 1.36个百分点和DyTox 3.53个百分点，且在更长增量序列中优势更明显；多种子验证显示稳定性高（+/-0.31%），且MIM解码器仅在训练时激活，不增加推理时的参数或计算开销。
Insight: 创新点在于将表示学习与分类任务解耦：通过持续MIM训练强化主干网络的通用视觉表示，结合分层任务令牌扩展和修改的注意力机制来管理任务特异性，从而在长期增量学习中有效平衡共享知识与任务区分，提升模型鲁棒性和性能。
Abstract: Dynamic expansion methods for class-incremental learning (CIL) protect task-specific knowledge by growing dedicated tokens or subnetworks, yet our analyses suggest that classification supervision alone does not sufficiently preserve task-agnostic shared backbone representations over long incremental sequences. We identify two intertwined challenges: cross-task confusion from sequential training on predominantly current-task data, which biases decision boundaries toward recent tasks; and under-optimized shared representations in the backbone that cap long-term discriminability as tasks accumulate. We propose the Decoupled Representation Dynamic Network (DRDN), which addresses these challenges via two orthogonal mechanisms. For shared backbone representations, DRDN continuously applies masked image modeling (MIM) at every incremental step, with reconstruction gradients routed exclusively through the backbone, encouraging it to retain general visual structure beyond class-discriminative cues. For task-specific discrimination, DRDN employs hierarchical task token expansion across all transformer layers, with a modified per-task attention rule that reduces inter-task interference. We support this design with accuracy degradation analysis and cross-task confusion rate measurements. In the from-scratch ViT CIL setting (no external pretraining), DRDN consistently improves over strong token-expansion baselines with comparable backbone scale. On CIFAR100-B0 (10 steps), DRDN achieves 77.19% average accuracy, outperforming DKT by 1.36 points and DyTox by 3.53 points, with an advantage that grows at longer incremental sequences. Multi-seed validation confirms stability (+/-0.31%). The MIM decoder is active only during training, adding no inference-time parameters or computation.
[37] Bridging 3D Gaussians and Semantic Occupancy for Comprehensive Open-Vocabulary Scene Understanding from Unposed Images cs.CVPDF
Hu Zhu, Bohan Li, Xianda Guo, Yanlun Peng, Zheng Zhu
TL;DR: 本文提出了COVScene，一种无需相机姿态的语义高斯框架，通过可微体积提升将可渲染的高斯基元与密集语义占据场耦合。该框架结合了语义感知的几何变换器、多任务高斯解码、几何基础蒸馏和占据熵正则化，支持单一表示下的新视角合成、开放词汇语义查询和语义占据预测。在ScanNet和ScanNet++数据集上的实验表明，COVScene保持了有竞争力的渲染质量，提升了开放词汇分割能力，并在无直接体素级监督的情况下实现了比自监督基线更强的语义占据预测。
Details
Motivation: 从稀疏、无姿态图像中实现全面的3D场景理解需要模型恢复可渲染几何、开放词汇语义和自由/占据3D空间，而不依赖外部相机标定。现有前馈高斯方法在姿态无关重建和语义渲染方面有所改进，但其高斯基元主要通过图像空间目标优化，在未观测区域约束较弱。
Result: 在ScanNet和ScanNet++数据集上，COVScene保持了有竞争力的渲染质量，提升了开放词汇分割能力，并在无直接体素级监督的情况下实现了比自监督基线更强的语义占据预测。
Insight: 通过可微体积提升将高斯基元与语义占据场耦合，使体积正则化能为高斯不透明度、几何和语义特征提供梯度，从而在单一表示中统一了渲染、语义查询和占据预测。创新点包括语义感知几何变换器、多任务高斯解码和占据熵正则化，增强了未观测区域的约束和语义一致性。
Abstract: Comprehensive 3D scene understanding from sparse, unposed images requires a model to recover renderable geometry, open-vocabulary semantics, and free/occupied 3D space without relying on external camera calibration. Recent feed-forward Gaussian methods improve pose-free reconstruction and semantic rendering, but their Gaussian primitives are mainly optimized through image-space objectives and remain weakly constrained in unobserved regions. We propose \textit{COVScene}, a pose-free semantic Gaussian framework that couples renderable Gaussian primitives with a dense semantic occupancy field through differentiable volumetric lifting. Instead of converting Gaussians to voxels only at evaluation time, COVScene lifts the predicted semantic Gaussians inside the training computation graph, so volumetric regularization provides gradients to Gaussian opacity, geometry, and semantic features. The framework combines a semantic-aware Geometry Transformer, multi-task Gaussian decoding, geometric foundation distillation, and occupancy entropy regularization to support novel view synthesis, open-vocabulary semantic querying, and semantic occupancy prediction within a single representation. Experiments on ScanNet and ScanNet++ show that COVScene maintains competitive rendering quality, improves open-vocabulary segmentation, and achieves stronger semantic occupancy prediction than the self-supervised baseline without direct voxel-level supervision.
[38] Domain Generalization via Text-Anchored Information Bottleneck cs.CVPDF
Eunyi Lyou, Yunjeong Choi, Junho Lee, Joonseok Lee
TL;DR: 本文提出了一种基于文本锚定信息瓶颈的领域泛化方法，通过摒弃视觉引导，将语言嵌入空间作为领域不变性的主要来源，以抑制领域特定变化并保留核心语义，从而提升模型在新环境中的鲁棒性。
Details
Motivation: 针对视觉识别模型在新环境中部署时因依赖视觉表达的虚假线索而泛化失败的问题，研究旨在探索如何通过语言监督而非视觉引导来强制学习领域不变表示。
Result: 在多种骨干网络上的广泛实验表明，该方法实现了最先进的性能，并进一步分析了有效监督对鲁棒泛化的关键作用。
Insight: 创新点在于将领域泛化的焦点从改进表示转向设计强制不变性的监督，利用语言嵌入作为信息瓶颈来抑制领域特异性变化，这为基于视觉-语言模型的泛化方法提供了新视角。
Abstract: Visual recognition models often fail when deployed in new environments. Domain Generalization (DG) addresses this by learning representations that remain invariant to environment-specific variations. Recent approaches increasingly rely on large vision-language models, assuming that preserving their expressive visual representations improves robustness. However, we show that such visual expressiveness can instead propagate spurious cues that tie representations to the training environments, hindering invariant learning. We therefore discard visual guidance and instead treat the language embedding space as the primary source of domain invariance, naturally acting as an information bottleneck that preserves core semantics while suppressing domain-specific variations. Extensive experiments across diverse backbones exhibit state-of-the-art performance and further analyze what makes guidance effective for robust generalization. These findings shift the focus of DG from improving representations to designing supervision that enforces invariance.
[39] Teaching Vision-Language-Action Models What to See and Where to Look cs.CVPDF
Yuguang Yang, Canyu Chen, Zhewen Tan, Yizhi Wang, Zichao Feng
TL;DR: 本文提出DriveTeach-VLA框架，旨在解决现有视觉-语言-动作模型在自动驾驶中因依赖文本中心数据而缺乏空间依赖性的问题。该框架通过驾驶感知视觉蒸馏和2D轨迹引导提示，明确教导模型’看什么’和’看哪里’，从而提升轨迹预测的可靠性。
Details
Motivation: 现有VLA模型训练过度依赖以文本为中心的视觉问答和思维链推理数据，导致学习到的表征缺乏对可靠轨迹预测至关重要的空间依赖性，无法有效支持基于动作的规划。
Result: DriveTeach-VLA在NAVSIM和nuScenes基准测试上取得了最先进的性能。
Insight: 创新点在于提出了一个明确的视觉引导学习流程：通过驾驶感知视觉蒸馏注入驾驶特异性感知先验（看什么），再通过2D轨迹引导提示进行空间条件化（看哪里），最后进行动作学习。这为VLA模型提供了更紧密对齐驾驶任务的空间-语义联合表征。
Abstract: Vision-Language-Action (VLA) models have emerged as a promising paradigm for end-to-end autonomous driving. However, existing VLAs’ training relies heavily on text-centric visual question answering and chain-of-thought reasoning data, which emphasizes linguistic reasoning rather than action-grounded planning. As a result, the learned representations capture semantic knowledge but lack spatial dependencies crucial for reliable trajectory prediction. We propose DriveTeach-VLA, a framework that explicitly teaches VLAs what to see and where to look. Driving-aware Vision Distillation (DVD) injects driving-specific perceptual priors into the vision encoder, while 2D Trajectory-Guided Prompts (2D-TGP) provide spatial conditioning aligned with feasible driving trajectories. Together, they form a vision-guided learning pipeline: what to see (DVD pretraining) - where to look (TGP-guided SFT) - how to act (TGP-guided GRPO). DriveTeach-VLA achieves the state-of-the-art performance on NAVSIM and nuScenes. Our code is available at: https://github.com/ShivaTeam/DriveTeach-VLA.
[40] Unified Panoramic-Gaussian Representation for Monocular 4D Scene Synthesis cs.CVPDF
Yuankun Yang, Yi Wei, Wenyang Zhou, Li Zhang
TL;DR: 本文提出了一种名为PanoGaussian的统一全景-高斯表示方法，用于从单目视频进行4D场景合成。该方法通过全景轨迹引导构建统一的训练和推理框架，并将全景表示蒸馏为显式的动态高斯表示，以解决传统方法在视角外推和动态内容建模方面的局限性。
Details
Motivation: 现有基于单目视频的4D场景合成方法通常局限于视图插值，难以推断观测视角之外的未观察区域；而基于相机条件的视频生成方法虽能合成未观察区域，但缺乏显式3D先验，在大轨迹偏差下会导致严重的不一致性。
Result: 实验表明，PanoGaussian即使在大的视点变化下也能实现一致的4D场景合成，在视角外推任务上超越了传统插值方法的限制。
Insight: 核心创新在于提出了一个统一的全景轨迹引导框架，并引入了全景-高斯表示，将隐式的全景表示蒸馏为显式的动态高斯表示，从而有效结合了跨视图一致性与动态物理先验的建模能力。
Abstract: 4D scene synthesis from monocular videos has made significant progress in recent years. However, existing methods are typically constrained by view interpolation. As a result, they struggle to infer unseen regions beyond the observed views. In this paper, we reformulate the task as 4D scene synthesis with unseen regions, which extends beyond traditional interpolation settings. Camera-conditioned video generation enables unseen region synthesis by guiding generation along specified cameras. However, these methods lack explicit 3D priors and are optimized with random camera trajectories. This design leads to severe inconsistencies under large trajectory deviations. To address this limitation, we build a unified training and inference framework with panoramic trajectory guidance. While this design improves cross-view consistency, the panoramic representation alone fails to model dynamic content effectively. Object motion in panoramic space introduces scale and shape distortions. To address this, we propose PanoGaussian, a unified Panoramic-Gaussian representation that distills the panoramic representation into an explicit dynamic Gaussian representation to capture dynamic physical priors of the 4D scene. Experiments demonstrate that PanoGaussian achieves consistent 4D scene synthesis even under large viewpoint variations.
[41] Temporal and Cross-Modal Alignment for Enhanced Audiovisual Video Captioning cs.CVPDF
Chen Zhao, Jiajun Ma, Qilong Huang, Tiehan Fan, Hongyu Li
TL;DR: 本文提出TCA-Captioner框架，旨在解决视听视频描述中模态分离和时间不连贯的问题。通过引入Observer-Checker-Corrector迭代优化策略生成高质量训练数据，并构建TCA-Bench诊断基准进行解耦评估。实验表明该框架在时间一致和同步的视听叙事方面达到了新水平。
Details
Motivation: 现有多模态大语言模型在视听视频描述中存在模态分离和时间不连贯的挑战，难以准确绑定听觉事件与视觉实体或捕捉复杂的因果动态。
Result: 在TCA-Bench基准上通过解耦评估协议进行广泛实验，TCA-Captioner在视听绑定和时间关系推理方面表现出色，为时间一致和同步的视听叙事设立了新标准。
Insight: 创新点包括Observer-Checker-Corrector迭代优化框架用于生成高质量训练数据，以及TCA-Bench诊断基准通过解耦评估协议量化模型在视听绑定和时间推理方面的能力。
Abstract: While Multimodal Large Language Models (MLLMs) have advanced video understanding, achieving precise temporal and cross-modal alignment in audiovisual video captioning remains a formidable challenge. Most existing approaches suffer from modality detachment and temporal incoherence, failing to accurately bind auditory events to visual entities or capture complex causal dynamics. To address these deficiencies, we propose TCA-Captioner, a framework specifically engineered to enhance Temporal and Cross-Modal Alignment for audiovisual video captioning. We first introduce the Observer-Checker-Corrector (OCC) framework, an iterative refinement strategy that generates high-fidelity, meticulously grounded training data. Leveraging a curated high-density human interaction dataset, TCA-Captioner is optimized to model sophisticated audiovisual interactions. Furthermore, we present TCA-Bench, a diagnostic benchmark utilizing a Decoupled Evaluation Protocol to isolate and quantify model proficiency in audiovisual binding and temporal relational reasoning. Extensive experiments demonstrate that TCA-Captioner sets a new standard for temporally-coherent and synchronized audiovisual narratives.
[42] Structure-Aware Gaussian Splatting for Large-Scale Scene Reconstruction cs.CVPDF
Weiyi Xue, Fan Lu, Chi Zhang, Tianhang Wang, Sanqing Qu
TL;DR: 本文提出了一种名为SIG（Structure-Aware Gaussian Splatting）的新框架，用于解决大规模场景重建中，3D高斯溅射在稀疏观测区域因高频图像监督导致的密度失控和冗余基元问题。该方法从信号结构恢复的视角重构问题，通过同步图像监督与高斯频率，并引入球面约束高斯来利用点云空间先验，从而实现频率一致、几何感知且无漂浮物的训练。
Details
Motivation: 大规模场景中不可避免地存在稀疏观测区域，从低频稀疏点初始化高斯并采用高频图像监督会导致不受控的致密化和冗余基元，从而降低重建效率和质量。现有调度策略多为硬编码，无法感知场景频率的收敛行为。
Result: 该方法在大型场景的重建中，在效率和渲染质量方面均取得了显著的性能提升，达到了最先进的水平。
Insight: 核心创新点在于将场景重建问题重新定义为信号结构恢复，并提出了一个能感知场景频率收敛的自适应调度器（SIG），它同步调节训练图像分辨率和高斯致密化过程。同时，引入的球面约束高斯利用了点云初始化的空间先验，有助于控制高斯优化过程，减少伪影。从客观角度看，这是一种将信号处理理论与3D重建相结合的、新颖的频率感知训练范式。
Abstract: 3D Gaussian Splatting has demonstrated remarkable potential in novel view synthesis. In contrast to small-scale scenes, large-scale scenes inevitably contain sparsely observed regions with excessively sparse initial points. In this case, supervising Gaussians initialized from low-frequency sparse points with high-frequency images often induces uncontrolled densification and redundant primitives, degrading both efficiency and quality. Intuitively, this issue can be mitigated with scheduling strategies, which can be categorized into two paradigms: modulating target signal frequency via densification and modulating sampling frequency via image resolution. However, previous scheduling strategies are primarily hardcoded, failing to perceive the convergence behavior of scene frequency. To address this, we reframe the scene reconstruction problem from the perspective of signal structure recovery and propose SIG, a novel scheduler that synchronizes image supervision with Gaussian frequencies. Specifically, we derive the average sampling frequency and bandwidth of 3D representations, and then regulate the training image resolution and the Gaussian densification process based on scene frequency convergence. Furthermore, we introduce Sphere-Constrained Gaussians, which leverage the spatial prior of initialized point clouds to control Gaussian optimization. Our framework enables frequency-consistent, geometry-aware, and floater-free training, achieving state-of-the-art performance by a substantial margin in both efficiency and rendering quality in large-scale scenes. The code is available at: https://github.com/weiyixue999/Signal_Structure_Aware_Gaussian
[43] LASER: A Corrective Lens for LVLMs via Visual Attention Preservation and Sink Suppression cs.CVPDF
Bowen Yuan, Zijian Wang, Yadan Luo, Shijie Wang, Zi Huang
TL;DR: 本文提出LASER框架，通过视觉注意力保持和汇点抑制来解决大型视觉语言模型（LVLM）在长序列解码中的视觉遗忘问题。该框架引入视觉基础奖励和汇点抑制奖励，在推理过程中调节视觉注意力轨迹和内部视觉令牌注意力分布，从而提升模型性能。
Details
Motivation: LVLM在长序列解码中会出现视觉遗忘现象，即注意力逐渐偏离视觉证据。现有方法多将其视为后期注意力衰减问题，或通过启发式提醒或事后注意力提升来缓解。本文通过系统实证分析发现，性能下降主要由两个被忽视的因素驱动：早期注意力衰减破坏证据获取，以及注意力过度集中在任务无关的视觉汇点令牌上。
Result: 在八个基准数据集上的广泛实验表明，LASER持续优于强基线模型，验证了注意力感知训练作为视觉遗忘有效补救措施的有效性。
Insight: 创新点在于识别出视觉遗忘的两个关键因素（早期注意力衰减和汇点令牌过度集中），并设计互补奖励机制（视觉基础奖励和汇点抑制奖励）进行联合调控。从客观角度看，该方法将注意力轨迹和分布纳入训练目标，为缓解LVLM的视觉遗忘提供了系统化解决方案。
Abstract: Large vision-language models (LVLMs) exhibit strong reasoning ability but suffer from visual forgetting during long-horizon decoding, where attention progressively drifts away from visual evidence. Existing methods largely treat this issue as a late-stage attention decay problem or attempt to mitigate it through heuristic reminders or post-hoc attention lifting. Through systematic empirical analysis, we find that performance degradation under visual forgetting is largely driven by two overlooked factors: early-stage attention decay disrupts evidence acquisition, and attention concentration on a subset of task-irrelevant visual sink tokens. Motivated by these insights, we propose LASER, a post-training framework that regulates both the visual attention trajectory and intra-visual token attention distribution during reasoning. Technically, LASER introduces two complementary rewards: a Visual Grounding Reward, which encourages the model to maintain attention on semantically salient visual tokens throughout decoding, and a Sink Suppression Reward, which penalizes excessive attention concentration on visual sink tokens. Together, these rewards preserve early-stage grounding while preventing attention collapse onto uninformative regions. Extensive experiments on eight benchmark datasets demonstrate that LASER consistently outperforms strong baselines, validating attention-aware training as an effective remedy for visual forgetting.
[44] ICDepth: Taming Video Diffusion Models for Video Depth Estimation via In-Context Conditioning cs.CVPDF
Xuanhua He, Jiaxin Xie, Mingzhe Zheng, Qifeng Chen
TL;DR: 本文提出ICDepth框架，通过In-Context Conditioning（ICC）将预训练的文生视频扩散模型适配于视频深度估计任务，以同时实现时间一致性、几何精度和跨场景泛化。为解决从生成任务迁移到密集预测任务的关键挑战，提出了SAND-Attention机制确保时空对齐并防止噪声污染，以及SRFM模块注入DINOv2语义和分辨率先验以提升几何精度。
Details
Motivation: 现有单目视频深度估计方法难以同时保证时间一致性、几何精度和泛化能力：判别式模型帧间精度高但受限于上下文窗口导致时间漂移，生成式方法虽改善一致性和泛化但需要海量训练数据且几何精度不足。
Result: ICDepth在多个基准测试上取得了最先进（SOTA）的结果，且数据效率显著，仅使用0.8M帧进行训练（比同类生成方法少6-13倍），并在多样领域展现出强大的零样本泛化能力。
Insight: 创新点在于通过In-Context Conditioning利用预训练视频扩散模型的丰富时空先验，并设计了SAND-Attention和SRFM两个关键模块来解决任务迁移中的对齐与精度问题，实现了数据高效且性能优越的视频深度估计。
Abstract: Monocular video depth estimation requires temporal consistency, geometric accuracy, and generalization across diverse scenarios, yet existing methods struggle to achieve all three simultaneously. Discriminative models excel at per-frame accuracy but suffer from temporal drift due to limited context windows, while generative methods improve consistency and generalization at the cost of extensive training data (10M+ samples) and lack of geometric precision. In response to these issues, we introduce \textbf{ICDepth}, a framework that adapts pre-trained text-to-video diffusion transformers for video depth estimation via In-Context Conditioning (ICC), leveraging their rich spatial-temporal priors. To address key challenges in transferring ICC from generation to dense prediction, we propose: (1)\textbf{SAND-Attention}, which ensures precise spatial-temporal alignment via shared RoPE and enforces unidirectional attention to prevent noise contamination; (2)\textbf{SRFM}, which injects DINOv2 semantic and resolution priors to enhance geometric precision. ICDepth achieves state-of-the-art results on multiple benchmarks with remarkable data efficiency, trained on only 0.8M frames ($6$–$13\times$ less than competing generative methods), while demonstrating strong zero-shot generalization to diverse domains.
[45] ReQuest: Rethinking-based Question-Aware Frame Selection for Long-Form Video QA cs.CVPDF
Minkuk Kim, Suyong Yun, Young Tae Kim, Jinyoung Moon, Jinwoo Choi
TL;DR: 本文提出ReQuest方法，通过基于不确定性的问题自适应关键帧选择管道，解决长视频问答中固定输入令牌预算下证据定位效率低下的问题。该方法包含轻量级问题感知选择器、再思考路由机制和不确定性引导的自适应非极大值抑制，无需修改或微调底层多模态大语言模型即可提升长视频QA性能。
Details
Motivation: 针对现有多模态大语言模型在固定输入令牌限制下处理长视频问答时，均匀采样方法在证据定位上效率不足的问题，旨在通过选择性计算使问题意图与相关视频内容对齐。
Result: 在Video-MME、MLVU和LongVideoBench基准测试中，ReQuest以有竞争力的计算成本实现了准确率的持续提升，在中长视频场景下表现尤为突出。
Insight: 创新点包括：从MLLM生成监督中蒸馏出的轻量级问题感知选择器、基于模型不确定性的长度自适应再思考路由触发机制，以及根据问题难度调整间距的不确定性引导自适应非极大值抑制帧选择策略，可作为即插即用模块提升现有模型效率。
Abstract: Recent multimodal large language models (MLLMs) have substantially advanced video understanding, yet long-form video QA remains challenging under fixed input token budgets, where uniform sampling can be inefficient for evidence localization. We propose ReQuest , an uncertainty-driven, question-adaptive keyframe selection pipeline that aligns question intent with relevant video content through selective computation. ReQuest integrates (i) a lightweight question-aware selector distilled from MLLM-generated supervision, (ii) Re-thinking Routing that triggers additional inference only when the model is uncertain with a length-adaptive criterion, and (iii) uncertainty-guided adaptive non-maximum suppression that selects temporally diverse frames while adjusting spacing based on question difficulty. As a plug-andplay method, ReQuest improves long-video QA without modifying or fine-tuning the underlying MLLM. Experiments on Video-MME, MLVU, and LongVideoBench demonstrate consistent accuracy gains with competitive computational cost, with particularly strong improvements in medium and long video regimes.
[46] MedStreamBench: A Time-Aware Benchmark for Streaming and Proactive Medical Video Understanding cs.CV | cs.AIPDF
Yuan Wang, Shujian Gao, Songtao Jiang, Zhengyu Hu, Zuozhu Liu
TL;DR: 本文提出了MedStreamBench，一个时间感知的医学视频理解基准测试，旨在评估模型在临床流式场景中‘何时回答’的能力。该基准整合了22个医学数据集和5419个问答实例，覆盖回顾性、当前、未来和主动监测四种时间设置，并引入了响应性和证据后稳定性等时间行为评估指标。
Details
Motivation: 现有医学视频基准主要评估模型答案的正确性，而忽略了其回答的时机是否恰当，这与临床部署中AI系统需要实时决策、延迟判断或主动预警的实际需求存在关键差距。
Result: 在领先的通用及医学视觉语言模型上的实验表明，离线识别性能与基于时间的决策能力之间存在显著差距，模型在流式和主动监测设置下的表现明显下降。
Insight: 创新点在于将时间维度（何时回答）作为核心评估标准，并设计了支持流式评估和主动监测的基准框架，强调了临床AI决策中时机判断的重要性，为模型开发提供了更贴近实际应用的评估体系。
Abstract: Existing medical video benchmarks primarily evaluate whether a model produces the correct answer, but rarely assess whether it answers at the right time. In real clinical settings, AI systems must decide not only what to predict, but also when to answer, defer judgment, or proactively raise alerts. This creates a critical gap between benchmark evaluation and deployment requirements. We present MedStreamBench, a benchmark for time-aware medical video understanding. MedStreamBench integrates 22 medical datasets and 5,419 QA instances across four temporal settings: retrospective, present, future, and proactive. Unlike conventional benchmarks that assume full-video access, MedStreamBench restricts models to temporally bounded evidence windows and supports both single-turn and streaming evaluation. We further introduce a proactive monitoring setting that requires models to determine whether and when clinically relevant alerts should be triggered. Beyond answer correctness, MedStreamBench evaluates temporal behavior through responsiveness and post-evidence stability. Experiments on leading general-purpose and medical vision-language models reveal a substantial gap between offline recognition and temporally grounded decision-making, with performance dropping markedly in streaming and proactive settings. Our benchmark is available at https://huggingface.co/datasets/Venn2024/MedStreamBench.
[47] RTE-FM-Dehazer: Radiative Transfer Equation Inspired Flow Matching for Real-World Image Dehazing cs.CVPDF
Chenfeng Wei, Chun Wang, Boyang Zhao, Si Zuo, Shenhong Wang
TL;DR: 本文提出了一种名为RTE-FM-Dehazer的新型图像去雾方法，该方法受辐射传输方程（RTE）启发，并结合流匹配（Flow Matching）技术。为了解决传统大气散射模型（ASM）的局限性，作者引入了一个基于简化RTE的扩散-吸收正则化器来引导去雾过程。同时，利用现代视觉-语言模型构建了一个自动化数据管道，并发布了包含5万对真实感雾霾/清晰图像的数据集P-HAZE。
Details
Motivation: 现有单图像去雾方法严重依赖于大气散射模型（ASM）的先验，但ASM的单次散射和均匀介质假设在真实复杂雾霾场景中常不成立，导致残留雾霾和颜色失真。此外，大规模真实雾霾/清晰图像对难以获取，现有合成方法无法复现自然雾霾的全部复杂性。
Result: 在仅使用P-HAZE数据集训练的情况下，RTE-FM-Dehazer在五个真实世界去雾基准测试上取得了领先的结果，有效消除了残留雾霾和颜色失真等伪影，并展现出强大的跨域泛化能力。
Insight: 主要创新点在于将更符合物理现实的辐射传输方程（RTE）与流匹配框架相结合，通过从简化RTE推导出的扩散-吸收正则化器来约束去雾轨迹。此外，利用视觉-语言模型自动化构建大规模、高质量的真实感雾霾数据集（P-HAZE）也是一个重要的数据层面的贡献。
Abstract: Single-image dehazing aims to recover a clear scene from a hazy image and is generally formulated as an image-to-image translation task; however, it faces two limitations. Its performance depends heavily on the haze-formation priors embedded in the model. Prevailing methods adopt the Atmospheric Scattering Model (ASM), whose assumptions of single scattering and homogeneous media are often violated, leading to residual haze and color drift. Moreover, large-scale real hazy/clear pairs are impractical to collect, and existing synthesis approaches fail to reproduce the full complexity of natural haze. To address these issues, we present RTE-FM-Dehazer, a novel dehazing approach, together with a scalable data pipeline. Unlike the ASM, the Radiative Transfer Equation (RTE) jointly accounts for both scattering and absorption, naturally accommodating the non-homogeneous, multiple-scattering media that characterize real hazy scenes. Motivated by the structural similarity between the RTE diffusion-absorption term and the ODE in flow matching, we introduce a diffusion-absorption regularizer derived from a reduced RTE, to steer the flow matching trajectory at each step. Next, leveraging modern vision-language models, we build an automated pipeline and release P-HAZE, a dataset of 50000 realistic hazy/clear pairs. Extensive evaluations demonstrate that RTE-FM-Dehazer, trained solely on P-HAZE, effectively eliminates artifacts like residual haze and color drift, exhibits strong cross-domain generalization, and achieves leading results on five real-world dehazing benchmarks.
[48] ProCal: Inference-Time Proposal Calibration for Open-Vocabulary Object Detection cs.CV | cs.AIPDF
Jae-Ryung Hong, Ho-Joong Kim, Seong-Whan Lee
TL;DR: 本文提出了一种名为ProCal的推理时提案校准方法，用于提升开放词汇目标检测中的定位质量。该方法通过结合定位感知的前景得分和背景感知的抑制得分，计算提案先验，以校准分类分数，从而抑制背景区域的误激活并提升新类别目标的排名。
Details
Motivation: 开放词汇目标检测旨在定位和分类训练时未见过的类别，现有方法利用冻结的视觉语言模型作为检测器主干，但其分类分数缺乏对图像中物体位置和尺度的识别能力。
Result: 在OV-LVIS基准测试中，将ProCal应用于CLIPSelf ViT-L/14模型，使APr指标提升了+2.5，有效缓解了新物体在排名上的校准错误。
Insight: 创新点在于利用预训练视觉语言模型区分前景和背景区域的能力，设计了一个简单高效的推理时校准机制，通过结合定位和背景信息来改进提案排序，无需重新训练模型即可提升新类别的检测性能。
Abstract: Open-vocabulary object detection aims to localize and classify objects beyond the fixed set of categories seen dur ing training. Recent open-vocabulary object detection methods improve localization and classification for unseen categories by leveraging a frozen VLM as a detector backbone. However, VLM classification score lacks recognizing position and scale of the object in an image. We observe that pretrained VLMs en able to classify foreground and background regions. According to this observation, we propose a simple inference-time Pro posal Calibration (ProCal) that improves localization quality of the classification score. ProCal computes a proposal prior by combining two scores: localization-aware foreground score and background-aware suppression score. Localization-aware foreground score captures whether a proposal contains an object area. Background-aware suppression score measures the extent to which the proposal resembles background. We analyze that ProCal suppresses false novel activation on background proposals and consistently ranks true novel proposals above background and partial novel proposals. Applied to CLIPSelf ViT-L/14, ProCal improves APr +2.5 on OV-LVIS. The analyses show that proposal-level localization-aware reranking effects to mitigate ranking miscalibration for novel objects.
[49] The Turning Point of 3D Plant Phenotyping: 3D Foundation Models Enable Minute-to-Second Cross-Crop Reconstruction and Beyond cs.CV | q-bio.QMPDF
Hanyue Jia, Wei Zhou, Wenbo Zhou, Yanan Li, Hao Lu
TL;DR: 本文提出了一种基于3D基础模型（3DFMs）的跨作物3D植物表型分析框架，旨在解决传统方法流程复杂、效率低下的问题。该框架利用前馈几何恢复、几何约束的3D高斯溅射、迭代视图合成与细化以及2D到3D语义转移等技术，实现了从低成本图像采集到快速重建、感知和表型测量的全流程加速。
Details
Motivation: 传统3D植物表型分析流程因多视角成像复杂、重建管道脆弱且从重建几何到表型提取成本高昂，导致效率低下，尤其在低成本数据采集（如智能手机视频或稀疏多视图图像）中，视图重叠有限和自遮挡问题进一步加剧了这些限制。
Result: 在包含26个植物序列的实验中，3D基础模型将平均重建时间从6.52分钟大幅缩短至1.58秒，同时保持了高重建质量和表型分析精度，为高通量3D植物表型分析提供了新的技术路线。
Insight: 创新点在于首次将3D基础模型应用于跨作物3D表型分析，通过前馈几何恢复替代传统稀疏初始化，结合几何约束的3D高斯溅射进行密集重建，并利用迭代视图合成与细化实现少视图重建，从而显著提升效率；客观来看，该研究展示了3D基础模型在简化复杂视觉任务流程方面的潜力，为低成本、高通量农业应用提供了可行方案。
Abstract: 3D plant phenotyping is notoriously known to be procedure-complicated and of low throughput due to the extensive multi-view imaging, the fragile 3D reconstruction pipeline, and the additional cost from reconstructed geometry to phenotypic extraction. These limitations are further amplified in low-cost data acquisition, where smartphone videos or sparsely sampled multi-view images provide limited view overlap and self-occlusion. In this work, we show that the conventional 3D plant phenotyping pipeline could be streamlined and significantly accelerated with 3D Foundation Models (3DFMs), and particularly, present one of the first cross-crop 3D phenotyping frameworks powered by 3DFMs. The framework replaces COLMAP-style sparse initialization with 3DFM-based feed-forward geometric recovery, combines geometry-constrained 3D Gaussian Splatting for dense reconstruction, enables few-view reconstruction through iterative view synthesis and refinement, and converts reconstructed geometry into measurable organs through 2D-to-3D semantic transfer, metric scale recovery, and organ instance separation. We further construct a cross-crop dataset with smartphone-based image acquisition, diverse plant morphologies, and manual annotations for segmentation and phenotypic evaluation. Experiments across 26 plant sequences show that 3D Foundation Models reduce the average reconstruction time from 6.52 minutes to 1.58 seconds while maintaining high reconstruction quality and phenotyping accuracy. These results suggest a fresh technical route for high-throughput 3D plant phenotyping, from low-cost image acquisition to fast reconstruction, perception, scale recovery, and phenotypic measurement.
[50] LLM-Empowered Multimodal Fusion Framework for Autonomous Driving: Semantic Enhancement and Channel-Adaptive Design cs.CV | eess.SPPDF
Wen Wang, Yaping Sun, Yejun He, Hao Chen, Zhiyong Chen
TL;DR: 本文提出了一种基于大语言模型（LLM）的多模态融合框架LM-SCIP，用于自动驾驶感知。该框架将LLM作为核心推理单元，通过通道自适应语义模块（CASM）动态融合质量变化的视觉和雷达流，以应对遮挡、恶劣天气等导致的输入质量动态变化问题，并输出定位、轨迹预测和图像重建等多任务结果。
Details
Motivation: 自动驾驶中视觉-雷达融合的鲁棒性受到输入质量动态变化（如遮挡、恶劣天气、通道噪声）的根本性挑战，传统静态数据融合方法难以应对。因此，论文旨在将问题重新定义为通道感知的语义推理，以提升在真实复杂环境下的感知可靠性。
Result: 在nuScenes数据集上，通过控制雷达输入开关，LM-SCIP相比纯视觉基线将定位RMSE降低了40.0%。在VIRAT数据集上，模型取得了0.214米的定位RMSE和0.179米的最小最终位移误差（minFDE, k=1）。结果表明，LM-SCIP在低信噪比下实现了鲁棒的视觉主导回退，在高信噪比下实现了协同融合。
Insight: 创新点在于将LLM作为多模态融合的中心推理核心，并设计了通道自适应语义模块（CASM）来动态门控外部雷达特征，这实现了从静态数据融合到通道感知语义推理的范式转变。同时，采用参数高效的LoRA微调LLM并结合异构专家混合（H-MoE）进行仲裁，是一种高效利用大模型进行多模态决策的可借鉴方法。
Abstract: Vision-radar fusion is central to robust autonomous driving, combining dense visual semantics with precise range and velocity measurements from radar. However, real-world fusion quality is fundamentally challenged by dynamically varying input quality, stemming from occlusion, adverse weather, and channel noise. To address this, we re-frame the problem from static data fusion to channel-aware semantic reasoning and propose a Large Language Model-centric Semantic-layer Channel-aware Integrated Perception (LM-SCIP) framework. It places a Large Language Model (LLM) as a central reasoning core to fuse a local visual stream with a quality-varying external radar stream used to cover perception-blind spots. Concretely, LM-SCIP couples a hierarchical radar-vision encoder with a Channel-Adaptive Semantic Module (CASM) that maps link indicators into a “Channel Prompt” to dynamically gate external radar features. A parameter-efficient, LoRA-tuned LLM, in conjunction with a heterogeneous Mixture-of-Experts (H-MoE), then arbitrates between local visual cues and the channel-conditioned radar context. Finally, a decoupled multi-task decoder outputs localization, trajectory forecasting, and image reconstruction. Experiments on nuScenes and VIRAT validate our approach. On nuScenes, under a controlled toggle of radar input, LM-SCIP reduces localization RMSE by 40.0% versus a vision-only baseline. On VIRAT, the model attains a 0.214m localization RMSE and 0.179m minFDE (k=1). These results reveal that the proposed LM-SCIP enables a robust vision-dominant fallback at low SNR and synergistic fusion at high SNR.
[51] SpaceEra++: A Unified Framework Towards 3D Spatial Reasoning in Video cs.CVPDF
Weili Guan, Haoyu Zhang, Meng Liu, Qianlong Xiang, Yaowei Wang
TL;DR: 本文提出了SpaceEra++框架，旨在解决视频中3D空间推理的挑战。该框架通过ScenePick策略优化视频帧采样以增强输入信息，并利用SpaceAlign模块结合绝对坐标与相对空间关系来强化推理约束，从而提升模型的空间理解能力。
Details
Motivation: 现有预训练视觉语言模型（VLMs）在3D空间理解上存在局限性，主要源于其基于2D观测的空间不确定性以及相关数据的稀缺性。尽管先前提出的SpaceEra框架取得了进展，但其仍受限于视频扫描输入不足和推理约束较弱的问题。
Result: 在多个基准测试上的广泛实验表明，SpaceEra++相比强基线模型取得了持续的性能提升。消融研究验证了各个组件（ScenePick和SpaceAlign）的独立及联合贡献。
Insight: 创新点在于提出了一个涵盖数据构建、模型设计、训练优化和提示推理的完整系统。具体而言，ScenePick通过平衡空间覆盖与对象语义来生成紧凑而全面的场景表示；SpaceAlign则通过联合利用绝对坐标和相对空间关系来强制执行对象间的成对约束，从而将优化与空间准确性对齐。
Abstract: Visual-spatial understanding, defined as the ability to infer object relationships and scene layouts from visual inputs, is fundamental to downstream tasks such as robotic navigation and embodied interaction. However, pre-trained vision-language models (VLMs) remain constrained by spatial uncertainty stemming from inherently 2D observations and by the scarcity of data for 3D spatial understanding. To address these limitations, we proposed a novel framework, SpaceEra, in the NeurIPS 2025 Spotlight paper. Although it achieved significant performance gains, we further observed that its effectiveness is hindered by insufficient input from scanning videos and weak reasoning constraints. To tackle these newly emerged challenges, we extend the original framework into a comprehensive system, termed SpaceEra++, which spans data construction, model design, training optimization, and prompting inference. Specifically, to alleviate input insufficiency, we introduce ScenePick, a frame sampling strategy that balances spatial coverage with object semantics to produce compact yet comprehensive scene representations. In addition, to enhance spatial reasoning, we develop SpaceAlign, which enforces pairwise object constraints by jointly exploiting absolute coordinates and relative spatial relations, thereby aligning optimization with spatial accuracy. Extensive experiments across multiple benchmarks demonstrate consistent improvements over strong baselines, while ablation studies validate both the individual and joint contributions of each component, and further analyses provide guidance for future research.
[52] PixGS: Pixel-Space Diffusion for Direct 3D Gaussian Splat Generation cs.CV | cs.GR | cs.ROPDF
Duy Cao, Phong Nguyen-Ha
TL;DR: 本文提出PixGS，一种单阶段直接生成高质量3D高斯泼溅（3DGS）的管道。该方法利用像素空间扩散模型，绕过有损的潜在压缩，直接对3D高斯属性进行去噪，并结合了表面法线、深度和高频结构信息的全面监督策略。实验表明，PixGS在保持快速推理速度的同时，性能超越了当前最先进的方法。
Details
Motivation: 现有方法通常将大规模预训练的文本到图像潜在扩散模型适配用于生成3DGS，但依赖于训练计算成本高、可扩展性有限的复杂级联管道，且生成质量受限于组件能力和压缩潜在空间，导致解码伪影和累积误差。
Result: 实验证明，PixGS在性能上超越了当前最先进的方法，同时保持了快速的推理速度（在单个A100 GPU上为1秒）。
Insight: 创新点在于提出单阶段直接生成3DGS的管道，利用像素空间扩散绕过潜在压缩，并引入包含表面法线、深度和高频结构信息的全面监督策略，实现了对外观和几何的精确泼溅级正则化。
Abstract: Recent advances in 3D content generation from text or images have achieved impressive results, yet view inconsistency from 2D generators and the scarcity of high-quality 3D data remain significant bottlenecks. Existing solutions typically adapt large-scale pre-trained text-to-image latent diffusion models to generate 3D Gaussian Splats (3DGS). However, these approaches often rely on training complex cascade pipelines that are computationally expensive and scalability-limited. Most critically, the quality of generated 3D assets is inherently constrained by each component capacity and compressed latent space, leading to decoding artifacts and accumulated errors. To address these limitations, we propose PixGS, a single-stage pipeline for direct high-quality 3DGS generation, which leverages recent advances in pixel-space diffusion to bypass lossy latent compression while still benefiting from the vast 2D generative priors. By directly denoising 3D Gaussian attributes at each timestep, our method enables precise, splat-level regularization of both appearance and geometry. Furthermore, we introduce a comprehensive supervision strategy that incorporates surface normals, depth, and high-frequency structural information, which is often overlooked in prior works. Experiments demonstrate that PixGS outperforms current state-of-the-art methods while maintaining a fast inference speed (1s on a single A100 GPU), offering a robust and efficient alternative to multi-stage generation pipelines.
[53] MMBench-Live: A Continuously Evolving Benchmark for Multimodal Models cs.CV | cs.AIPDF
Yuanzhi Liu, Shousheng Zhao, Bo Zhou, Kongming Liang, Zhanyu Ma
TL;DR: 本文提出了MMBench-Live，一个通过多智能体驱动的自动化流程构建的、持续演进的多模态基准测试。该框架将基准演进视为任务导向的数据集构建，集成了结构化基准规范、反馈控制的实时数据采集以及带有可执行推理的可验证问答生成。为了保持跨版本可比性，引入了分布一致更新策略，从原始基准中提取任务相关的视觉模式来指导数据收集和过滤。
Details
Motivation: 当前大多数多模态基准测试是静态的，容易受到时间过时、数据污染和维护成本高昂等问题的影响，因此需要一种可持续、可扩展的基准演进范式。
Result: 基于MMBench实例化的MMBench-Live包含了5.9K个新生成的评估实例，答案正确率高。每次更新成本约30美元，耗时1-2小时。广泛评估表明，MMBench-Live保持了稳定的模型排名，与原始基准保持语义对齐，并表现出更弱的与污染相关的记忆信号。
Insight: 创新点在于将基准演进视为一个自动化、任务驱动的数据集构建流程，并提出了分布一致更新策略来确保跨版本可比性。这为可持续的多模态基准测试演进提供了一个实用且可扩展的范式，通过低成本、高效率的自动化流程解决了静态基准的固有问题。
Abstract: Evaluation benchmarks are essential for assessing vision-language models (VLMs), but most multimodal benchmarks are static, making them vulnerable to temporal staleness, data contamination, and costly maintenance. We present MMBench-Live, a continuously evolving multimodal benchmark built by a multi-agent-driven automated pipeline. Our framework treats benchmark evolution as task-guided dataset construction, integrating structured benchmark specification, feedback-controlled real-time data acquisition, and verifiable QA generation with executable reasoning. To maintain cross-version comparability, we introduce a distribution-consistent update strategy that extracts task-related visual patterns from the original benchmark to guide data collection and filtering. Instantiated from MMBench, MMBench-Live contains 5.9K newly generated evaluation instances with a high answer correctness rate, while each update costs about USD 30 and takes 1-2 hours. Extensive evaluations show that MMBench-Live preserves stable model rankings, maintains semantic alignment with the original benchmark, and exhibits weaker contamination-related memorization signals, suggesting a practical and scalable paradigm for sustainable multimodal benchmark evolution. The project is available at https://github.com/PRIS-CV/MMBench-Live.
[54] Geometric Foundation Model Distillation for Efficient Lunar 3D Reconstruction cs.CVPDF
Clémentine Grethen, Florient Chouteau, Géraldine Morin, Simone Gasparini
TL;DR: 本文研究了如何通过知识蒸馏压缩大型3D基础模型MASt3R，以月球立体重建为案例，提出了一种基于结构化SVD的初始化方法，成功将模型尺寸压缩至多7倍，同时保持了大部分重建精度。
Details
Motivation: 解决MASt3R等大型3D基础模型在行星探索等硬件严格受限场景下计算需求过高、难以部署的问题。
Result: 在月球数据上，蒸馏后的学生模型在保持教师模型大部分重建精度的同时，模型尺寸最多减小7倍，甚至优于使用稀疏真实标注直接训练的基线。
Insight: 创新点包括提出结构化SVD初始化方法以弥合师生模型维度差异，并总结出几何基础模型蒸馏的实用原则，如基于Transformer的编码器优于CNN、保持编码器容量比大解码器更重要、特征级蒸馏优于仅输出监督等。
Abstract: Large 3D foundation models such as MASt3R achieve state-of-the-art stereo reconstruction but are computationally demanding for deployment under strict hardware constraints – a critical limitation in domains such as planetary exploration, where onboard computing is severely restricted. We study how far such models can be compressed through knowledge distillation, using lunar stereo reconstruction as a challenging and practically relevant case study. Starting from a 688M-parameter MASt3R teacher fine-tuned on lunar imagery, we distill its dense geometric predictions into a family of lightweight students spanning different encoder types (CNN vs ViT), decoder widths and depths, and training strategies. To bridge the dimensional mismatch between teacher and student, we propose a structured SVD-based initialization that projects the teacher’s decoder weights into the student’s smaller latent space, yielding a warm start that significantly improves convergence and final performance. Based on our results on lunar data, we can obtain a distilled student that retains most of teacher’s reconstruction accuracy while reducing the model size up to 7 times, and even outperforms a baseline trained directly with sparse ground-truth annotations. Beyond compression, our study highlights both principles and practical insights for distilling geometric foundation models: a convolutional encoder underperforms transformer-based alternatives (though pretraining availability remains a confounding factor), preserving encoder capacity is more critical than maintaining a large decoder, feature-level distillation consistently outperforms output-only supervision, and SVD-based initialization improves optimisation stability. These findings provide practical guidelines for deploying 3D reconstruction models in resource-constrained environments.
[55] QWERTY: Training-Free Motion Control via Query-Warped Video Diffusion Transformers cs.CVPDF
Kyobin Choo, Youngmin Kim, Hyunkyung Han, Geunrip Park, Chanyoung Kim
TL;DR: 论文提出了一种名为QWERTY的训练免费框架，用于在预训练的图像到视频扩散变换器（DiT）中实现灵活的运动控制。该方法通过用户定义的对象扭曲和光流来操纵DiT的3D全注意力机制，无需额外训练即可引导视频生成朝向期望的运动轨迹。
Details
Motivation: 现有视频扩散变换器（DiT）主要依赖文本提示进行隐式运动控制，导致需要大量提示工程和重复采样；而基于空间提示（如边界框或点轨迹）的微调方法虽能实现显式控制，但需要大量数据准备和计算，且可能损害预训练模型的生成能力。因此，作者旨在探索一种无需训练的运动控制方法，以弥补DiT在这一领域的空白。
Result: 实验表明，在最新的图像到视频DiT上，QWERTY在现有训练免费方法中实现了最有效的运动控制，其性能与基于微调的方法相当。
Insight: 创新点在于通过扭曲查询的帧不变语义子空间来操纵DiT的3D全注意力，并利用预测噪声作为自引导进行潜在优化，从而提升控制稳定性和视觉质量。该方法无需训练即可实现显式运动控制，保持了预训练模型的生成能力。
Abstract: Video diffusion transformers (DiTs) generate high-fidelity and temporally coherent videos, yet motion control remains implicit, primarily relying on text prompts. As a result, achieving desired motion often requires extensive prompt engineering and repeated resampling. While fine-tuning models with additional spatial prompts (e.g., bounding boxes or point trajectories) enables explicit control, it demands substantial data curation and computation, and may compromise the generative capabilities of pretrained models. Consequently, training-free motion control using such spatial prompts has been explored in U-Net-based video diffusion models, but remains largely unexplored for DiTs. We introduce QWERTY, a training-free framework that enables flexible motion control in pretrained image-to-video DiTs via user-defined object warping and optical flow. We carefully manipulate the 3D full attention of DiTs by warping the frame-invariant semantic subspace of queries. We find that the noise predicted by the query-warped DiT naturally guides the diffusion trajectory toward the desired motion, and further show that leveraging this noise as self-guidance for latent optimization improves control stability and visual quality. Experiments show that QWERTY achieves the most effective motion control among existing training-free approaches on a recent image-to-video DiT, with performance comparable to fine-tuning-based methods.
[56] Descriptor: LYNRED Mobility Dataset Multimodal Detection Subset (LYNRED-MDS) cs.CVPDF
Loïc Arbez, Jessy Matias, Xavier Brenière, Jocelyn Chanussot, Ronald Phlypo
TL;DR: 本文介绍了LYNRED-MDS数据集，这是一个包含4000对RGB-红外图像的多模态检测子集，采集自法国格勒诺布尔地区多种天气、光照和道路条件，旨在提升低能见度环境下的行人检测性能。
Details
Motivation: 现有RGB-红外数据集（如FLIR ADAS和LLVIP）多局限于清晰天气和简单场景，无法充分支持早期碰撞预测，尤其是在夜间或雾天等低能见度条件下，因此需要更具多样性的数据集来开发可靠的驾驶辅助系统。
Result: 使用YOLOv8n基线进行的跨数据集评估表明，该数据集在驾驶场景的行人检测任务中展现出良好的泛化潜力。
Insight: 创新点在于提供了覆盖多样化驾驶环境（城市、乡村、山区等）和西欧标准车辆的多模态数据，专注于关键边缘案例，有助于推动更可靠、可部署的视觉系统发展。
Abstract: Current road safety systems primarily focus on minimizing post-collision damage. However, advances in algorithmic perception are shifting focus toward early collision prediction, especially in lowvisibility conditions like nighttime or fog, where thermal infrared sensing outperforms both human vision and RGB imaging. While available RGB-infrared datasets such as FLIR ADAS and LLVIP are good benchmarks, they mostly consist of clear weather and overly simple scenarios. In this article, we introduce the LYNRED-MDS: Multimodal Detection Subset, a subset of the LYNRED Mobility Dataset, comprised of 4000 RGB-infrared image pairs captured under diverse weather, lighting, and road conditions around Grenoble, France. Our dataset spans varied driving contexts (urban, rural, mountainous, etc.) and a vehicle fleet compliant with Western European standards. Thermal cross-dataset evaluation using a YOLOv8n baseline suggests that our dataset offers strong generalization potential for pedestrian detection in driving scenarios. By covering critical edge cases, our dataset supports the development of more reliable and deployable vision systems for advanced driver-assistance systems.
[57] SAB-LVLM: Significance-Aware Binarization for Large Vision-Language Models cs.CV | cs.AIPDF
Qi Lyu, Jiahua Dong, Baichen Liu, Xudong Wang, Mingfei Han
TL;DR: 本文提出了一种针对大型视觉语言模型（LVLM）的显著性感知二值化方法（SAB-LVLM），旨在解决现有二值化方法因忽略不同层和模态间权重重要性差异而导致的性能显著下降问题。该方法通过构建Hessian矩阵、生成空间显著性图并设计模态引导的集成策略，来识别和加权关键权重，从而在约1比特的压缩约束下，有效减少模型存储和计算开销。
Details
Motivation: 大型视觉语言模型参数量巨大且涉及跨模态计算，导致内存和延迟开销高，难以在资源受限设备上部署。现有二值化方法未能考虑不同层和模态间权重的重要性差异，导致不相关参数被保留而关键权重未被充分优化，造成性能显著下降。
Result: 大量实验表明，在约1比特的压缩约束下，SAB-LVLM方法优于现有的二值化后训练量化（PTQ）方法，实现了更优的性能表现。
Insight: 创新点在于提出了一个显著性感知的二值化框架，通过构建Hessian矩阵和空间显著性图来量化权重在不同模态和层中的重要性，并设计了模态引导的集成策略和交替的显著性加权更新方案，从而在极端压缩下更好地保留模型性能。
Abstract: Large Vision-Language Models (LVLMs) have achieved remarkable progress in multimodal understanding, yet their enormous parameter scale and cross-modal computation incur substantial memory and latency overhead, severely limiting real-world deployment on resource-constrained devices. Binarization offers an attractive solution by drastically reducing storage and computational costs. However, existing binarization methods neglect the varying importance of weights across different layers and modalities. This causes parameters irrelevant to downstream tasks to be unnecessarily retained, whereas modality-critical weights may not be adequately optimized, resulting in significant performance degradation. To address these challenges, we develop a novel \underline{S}ignificance-\underline{A}ware \underline{B}inarization for \underline{L}arge \underline{V}ision-\underline{L}anguage \underline{M}odels (SAB-LVLM). Specifically, after constructing Hessian matrices for textual and visual inputs, we propose a spatial significance map to distinguish full-precision weights activated under a single modality from those activated across modalities. We then devise a modality-guided integration strategy to obtain the significance-aware binarization map, which measures weight significance across layers and modalities. Subsequently, this binarization map is incorporated into the binarization objective as an error reweighting term, and binarization fitting is performed through an alternating significance-weighted update scheme. Extensive experiments illustrate the superiority of our SAB-LVLM over existing binary PTQ methods under an approximately 1-bit compression constraint. Our code is accessible at https://github.com/LyuQi127/SAB_LVLM.
[58] FoundDP: Revisiting Weak Disparity Observability in Dual-Pixel Depth Estimation cs.CVPDF
Fengchen He, Hao Xu, Dayang Zhao, Tingwei Quan, Shaoqun Zeng
TL;DR: 本文提出FoundDP框架，通过结合双像素（DP）成像的度量深度与单目深度基础模型的全局结构先验，解决了DP深度估计中因有效基线极小导致的弱视差可观测性问题，从而在纹理稀疏、低对比度或下采样区域提升深度估计的结构保真度和度量精度。
Details
Motivation: 双像素成像利用亚孔径视差实现单相机度量深度估计，但极小的有效基线限制了视差可观测性，导致在纹理稀疏、低对比度或下采样区域出现结构退化和深度估计失败；现有方法主要依赖局部视差线索，在视差信号弱或模糊时不可靠。
Result: 在合成和真实世界DP基准测试上的大量实验表明，FoundDP实现了优越的性能，在结构保真度和度量精度上均取得一致提升，尤其在视差可观测性降低的情况下表现突出。
Insight: 创新点在于将DP度量深度与单目深度基础模型的全局结构先验统一整合，并通过ViT特征对齐缓解DP散焦模糊引起的ViT表示退化，从而在弱视差区域实现稳定的度量引导深度估计；客观分析认为其核心贡献是首次系统性地利用基础模型增强DP深度估计的鲁棒性。
Abstract: Dual-pixel (DP) imaging enables metric depth estimation from a single camera using sub-aperture disparity. However, the extremely small effective baseline limits disparity observability, leading to structural degradation and depth failure in textureless, low-contrast, or downsampled regions. Existing DP-based methods rely primarily on local disparity cues and therefore become unreliable when disparity signals are weak or ambiguous. To address this limitation, we propose \emph{FoundDP}, a unified framework that integrates metric DP depth with global structural priors from a monocular depth foundation model. Our method preserves metric scale through DP-derived depth and leverages Vision Transformer (ViT) features to restore structural consistency in weak-disparity regions. To ensure reliable metric guidance under DP imaging conditions, we identify and mitigate ViT representation degradation induced by DP defocus blur via ViT feature alignment, enabling stable metric-guided depth estimation. Extensive experiments on synthetic and real-world DP benchmarks show that FoundDP delivers superior performance, with consistent gains in structural fidelity and metric accuracy, especially under reduced disparity observability. Code will be available at: https://github.com/EchoLighting/FoundDP
[59] Diversity-aware View Partitioning for Scalable VGGT cs.CVPDF
Jinsoo Park, Donggyu Choi, Ahyun Seo, Minsu cho, Jeany Son
TL;DR: 本文提出了一种无需训练、即插即用的VGGT推理框架，通过基于视觉差异性和空间分散性的组合图划分，将多视图组织成多样性感知的平衡块，以解决VGGT等几何变换器在扩展到大型视图集合时面临的注意力二次方成本问题，并缓解因视点分布不均导致的性能下降。
Details
Motivation: 动机在于VGGT等几何变换器虽通过全局注意力在多个视图上联合推理取得强性能，但扩展到大型视图集合时面临注意力二次方成本的挑战，且实证分析发现重建质量对视点分布敏感，缺乏足够视点多样性的冗余视图会引入高度相似的token，稀释注意力机制中的几何信息信号。
Result: 大量实验表明，该框架在相机姿态估计、多视图深度预测和3D重建任务中提升了性能，同时减少了内存使用和推理延迟，并能与现有VGGT变体互补，在不牺牲几何保真度的前提下实现可扩展的多视图重建。
Insight: 创新点在于提出了一种基于视觉差异性和空间分散性的组合图划分方法，将视图组织成多样性感知的平衡块，使变换器能专注于几何信息丰富的视图，减少冗余注意力交互；同时，通过基于视觉相似性的软姿态传播策略从小规模种子帧近似空间关系，避免了完整的姿态估计，实现了无需训练和即插即用的高效推理。
Abstract: Geometry transformers such as VGGT achieve strong performance by jointly reasoning over multiple views with global attention. However, scaling them to large view collections remains challenging due to the quadratic cost of attention. Moreover, our empirical analysis reveals that the reconstruction quality in VGGT is sensitive to the distribution of viewpoints. Simply increasing the number of views without sufficient viewpoint diversity can even degrade performance, as redundant views introduce highly similar tokens that dilute informative geometric signals in the attention mechanism. Motivated by this observation, we propose a training-free and plug-and-play VGGT inference framework that organizes views into diversity-aware balanced chunks. The chunks are constructed through combinatorial graph partitioning over visual dissimilarity and spatial dispersion. This view organization allows the transformer to focus attention on geometrically informative views while reducing redundant attention interactions. To estimate spatial dispersion without full pose estimation, we approximate spatial relationships via a soft pose propagation strategy based on visual similarity from a small set of seed frames. Extensive experiments demonstrate improved performance in camera pose estimation, multi-view depth prediction, and 3D reconstruction while reducing memory usage and inference latency. Our framework also complements existing VGGT variants, enabling scalable multi-view reconstruction without sacrificing geometric fidelity.
[60] Towards Real-World Ultrasound Understanding: Large Vision-Language Models from Multi-Image Examinations with Long-Form Reports cs.CVPDF
Bingcong Yan, Chunlei Li, Jingliang Hu, Yilei Shi, Xiao Xiang Zhu
TL;DR: 该论文通过构建一个包含150万次真实世界超声检查、1770万张图像的大规模数据集，并以检查级别组织数据，将多张图像与对应的长格式临床报告对齐，从而训练大型视觉语言模型（LVLM）。研究表明，数据规模和临床对齐是关键因素，无需复杂架构或训练策略，仅使用低秩适应（LoRA）微调标准LVLM即可在多种超声理解任务上取得优异性能。
Details
Motivation: 解决超声图像理解中因复杂性和变异性导致的现有大型视觉语言模型应用受限问题，探索实现真实世界超声理解所需的关键因素。
Result: 在多种超声理解任务上，该方法超越了先前采用更复杂流程设计的模型，表现出强大的性能。
Insight: 创新点在于强调数据规模（大规模真实世界超声检查数据集）和临床对齐（以检查级别组织多图像-报告对）的重要性，而非模型架构本身；通过简单的LoRA微调标准LVLM即可实现SOTA水平，为医学影像理解提供了可扩展且高效的范式。
Abstract: Large vision-language models (LVLMs) have achieved strong performance across many medical imaging tasks, yet their application to ultrasound remains limited due to its inherent complexity and variability. In this work, we revisit what is truly needed to enable real-world ultrasound understanding. Instead of introducing complex architectures or elaborate training strategies, we show that data scale and clinically faithful data alignment are the key factors. We construct a large-scale dataset of 1.5M real-world ultrasound examinations, containing 17.7M images, multi-organ coverage, and paired uncurated clinical reports. Crucially, we organize the data at the examination level, aligning multiple images with their corresponding reports to reflect real clinical workflows. We then fine-tune a standard LVLM using low-rank adaptation (LoRA) on this dataset without task-specific modifications. Surprisingly, this simple recipe already leads to strong performance across diverse ultrasound understanding tasks, outperforming prior methods designed with more complex pipelines. Beyond these results, we present model and data scaling analyses that provide insights into the role of scale in ultrasound LVLMs.
[61] Personalized 4D Whole-Heart Mesh Reconstruction from Cine MRI via Multi-Scale Temporal Modeling and Differentiable Contour Rendering cs.CVPDF
Xiaoyue Liu, Dongcheng Cang, Xiaohan Yuan, Mark YY Chan, Ching-Hui Sia
TL;DR: 本文提出了一种新颖的端到端框架，用于从多视角2D电影磁共振成像序列中重建具有时间分辨率的完整心脏网格。该方法通过结合可微分轮廓渲染器和多尺度时序建模模块，直接学习从图像到网格的映射，旨在解决稀疏MRI数据下心脏形状与运动复杂耦合的挑战。
Details
Motivation: 从稀疏的电影MRI数据中准确重建4D完整心脏网格对于创建心脏数字孪生体至关重要。现有方法通常依赖中间轮廓拟合，且只能重建静态、单相或部分心脏几何结构，难以捕捉完整心腔的动态变化。
Result: 该方法在完整心脏重建上取得了1.68 ± 0.31 mm的平均绝对误差和0.77 ± 0.17 mm/帧³的运动抖动，优于现有方法，具有更低的重建误差和显著改善的运动平滑度。它还在多视角MRI轮廓对齐方面表现更优，并支持了下游概念验证的电生理模拟。
Insight: 创新点在于引入了受比尔-朗伯衰减定律启发的可微分轮廓渲染器，实现了基于轮廓投影损失的解剖感知监督；以及一个整合全局周期级动态与局部帧间一致性的多尺度时序建模模块，以生成平滑且生理上合理的网格轨迹。这为从稀疏2D图像直接重建动态3D几何体提供了一种新的监督和建模思路。
Abstract: Accurate 4D whole-heart mesh reconstruction from sparse cine MRI is critical for creating cardiac digital twins, but remains challenging due to limited 2D slice coverage and the complex coupling between cardiac shape and motion. Existing methods often rely on intermediate contour fitting and typically reconstruct static, single-phase, or partial cardiac geometries, limiting their ability to capture full-chamber dynamics. We propose a novel end-to-end framework for reconstructing temporally resolved whole-heart meshes from multi-view 2D cine MRI sequences by learning an image-to-mesh mapping. The framework incorporates a differentiable contour renderer inspired by the Beer-Lambert attenuation principle, enabling anatomy-aware supervision of 3D+t mesh deformation through contour-based projection losses. To improve temporal consistency across the cardiac cycle, we further introduce a multi-scale temporal modeling module that integrates global cycle-level dynamics with local inter-frame coherence to generate smooth and physiologically plausible mesh trajectories. The proposed method achieved a whole-heart mean absolute error of 1.68 $\pm$ 0.31 mm and a motion jitter of 0.77 $\pm$ 0.17 $\mathrm{mm}/\mathrm{frame}^{3}$, outperforming existing methods with lower reconstruction error and substantially improved motion smoothness. It also improved 2D contour alignment across multiple cine MRI views and supported downstream proof-of-concept electrophysiological simulation. The code will be released publicly upon acceptance of the manuscript for publication.
[62] NeoMap: Training-free Novel-View Synthesis from Single Images and Videos cs.CV | cs.AI | cs.GR | cs.ROPDF
Jinxi Li, Tianyi Zhang, Yafei Yang, Zihui Zhang, Peng Huang
TL;DR: NeoMap是一种无需训练的新视角视频合成框架，能够从单张图像或单目视频生成高质量、视角一致的新视角视频。该方法基于预训练视频模型，通过收敛流形交替投影迭代优化初始噪声，从而在预训练模型学习的自然视频数据流形中定位最优的新视角解。
Details
Motivation: 现有方法通常假设预训练视频模型缺乏原生新视角合成能力，并通过相机条件、任务特定微调或逐步硬去噪引导来强制视角对齐，这往往导致伪影和全局场景一致性受损。NeoMap旨在解决这些问题，直接利用预训练模型中的隐含新视角信息。
Result: 在Tanks-and-Temples、LLFF和DAVIS三个标准新视角合成基准测试中，NeoMap显著优于所有现有方法，实现了最先进的生成保真度和顶级的视角一致性。
Insight: 核心创新在于认识到预训练视频模型学习的自然视频数据流形中已编码了高质量的新视角解，关键在于定位这一最优解；通过收敛流形交替投影迭代优化初始噪声，无需额外训练即可实现高效合成，避免了微调或硬引导带来的问题。
Abstract: We study the challenging problem of novel view video synthesis from single images or monocular videos. Existing methods, which operate under the assumption that pre-trained video models lack native novel view synthesis capability and enforce view alignment via camera conditioning, task-specific fine-tuning, or stepwise hard denoising guidance, often suffer from artifacts and compromised global scene consistency. In this paper, we introduce NeoMap, a novel training-free framework designed to locate high-fidelity, view-consistent novel view solutions from general pre-trained video models. The key to our approach is the core insight that promising novel view solutions are inherently encoded within the natural video data manifold learned by pre-trained models, and the core challenge is simply to locate this optimal solution. We solve this via our core mechanism: convergent manifold alternating projection iterations that optimize the initial noise. Extensive experiments demonstrate that NeoMap significantly outperforms all existing methods across 3 standard novel view synthesis benchmarks, including the challenging Tanks-and-Temples, LLFF and DAVIS datasets, achieving state-of-the-art generation fidelity and top-tier view consistency.
[63] Assessing VLM Reliability for Medical Image Quality Evaluation Under Corruption and Bias cs.CV | cs.AI | cs.LGPDF
Sofiane Ouaari, Kevin Vorwalder, Nico Pfeifer
TL;DR: 本文评估了视觉语言模型（VLMs）在医学图像质量评估（MIQA）任务中的可靠性，特别是在图像损坏和文本上下文偏见影响下的表现。研究使用MediMeta-C数据集，在零样本设置下测试了16个VLMs对七种损坏类型和五种严重程度的敏感性，并分析了文本属性（如人口统计、机构声望）对评分的影响。
Details
Motivation: VLMs在医学任务中的应用日益增多，自动化MIQA有望减轻工作负担，但模型在真实世界条件下（如图像退化或文本上下文影响判断）的行为仍需深入探索，以确保临床部署的可靠性与安全性。
Result: 在多个模态上，像素化损坏导致评分最大下降（平均-20.58%，OCT图像最高达-34.4%），而亮度变化影响较小（-0.81%）。文本属性如机构声望可使评分提升+17.15%，设备老旧则降低-14.7%，其中InternVL-8B和MedGemma模型变化最大（分别+95.62%和-37.7%）。同系列模型间评分相关性为0.67-0.83。
Insight: 研究揭示了VLMs在MIQA中的局限性：像素化（一种隐私保护处理）会显著降低性能，表明患者隐私与模型可靠性之间存在权衡；对上下文元数据的敏感性则反映了模型客观性不足，且元数据可能成为隐私和偏见的来源，这提示隐私保护与客观质量评估是实际应用中的相关需求。
Abstract: Vision-Language Models (VLMs) are increasingly applied in medical tasks such as pathology description, report generation, and visual question answering. Medical Image Quality Assessment (MIQA) supports diagnostic accuracy and patient safety by determining whether images meet the standards required for clinical decision-making. Automating MIQA with VLMs may reduce workload, but their behavior under real-world conditions, where images may be degraded or textual context may affect judgments, should be further explored before deployment. We benchmark VLMs on medical image quality using the MediMeta-C dataset zero-shot across seven corruption types and five severity levels. We evaluate sensitivity to degradation patterns, the effect of corruptions on embedding geometry, and whether textual attributes (demographics, expertise, infrastructure, institution) alter scores. Across 16 VLMs and seven modalities, pixelation produced the largest score reductions (mean -20.58%, up to -34.4% for OCT), whereas brightness had limited effect (-0.81%). Embedding displacement was associated with score changes. Same-family models showed correlations of 0.67-0.83; some produced increases up to +31% for corrupted mammography. Textual attributes affected scores: institutional prestige raised them +17.15%, and equipment age lowered them -14.7%. The largest changes were +95.62% (InternVL-8B) and -37.7% (MedGemma). Current VLMs show limitations for medical image quality assessment. Pixelation, a privacy-preserving transformation, reduces performance, indicating a trade-off between patient privacy and reliability. Sensitivity to contextual metadata indicates limited objectivity and marks metadata as a privacy and bias source. Privacy protection and objective quality assessment are related requirements for use.
[64] MolSight: A Graph-Aware Vision-Language Model for Unified Chemical Image Understanding cs.CV | cs.AI | q-bio.BMPDF
Wenda Wang, Yihan Tong, Yuwei Hu, Zhewei Wei
TL;DR: MolSight是一个图感知的视觉语言模型框架，旨在通过整合分子拓扑信息和对齐视觉与符号语义，来增强对分子图像的理解。它解决了现有分子LLM和VLM在捕捉结构表示和拓扑建模方面的不足。
Details
Motivation: 现有分子大语言模型和视觉语言模型难以充分捕捉分子结构的视觉表示，且在结构对齐和拓扑建模方面存在挑战，限制了其在分子理解和药物发现等任务中的潜力。
Result: 实验表明，MolSight在多个化学视觉理解任务上显著优于现有的VLM、分子LLM和专用工具，实现了分子图像推理的新水平。
Insight: 创新点在于引入了分子拓扑模块来将化学键邻接信息注入视觉令牌，以及分子接地模块来对齐视觉特征与化学符号语义，从而实现了图感知的跨模态对齐，提升了模型对分子结构的理解能力。
Abstract: Using molecular large language models (LLMs) as a unified framework for understanding molecular structures and functions is emerging as a new trend in tasks such as molecular design and drug discovery. However, these models struggle to fully capture the visual representation of molecular structures, limiting their potential. While existing molecular vision-language models (VLMs) show promise, they still face challenges in structural alignment and lack the necessary topological modeling for accurate molecular understanding. To address this, we propose MolSight, a graph-aware vision-language model framework designed to enhance the understanding of molecular images by VLMs. MolSight integrates a Molecular Topology Module to inject chemical-bond adjacency information into vision tokens, and a Molecular Grounding Module to align visual features with chemical symbolic semantics. Our experiments demonstrate that MolSight significantly outperforms existing VLMs, molecular LLMs, and specialized tools across multiple chemical visual understanding tasks, achieving a new level of molecular image reasoning.
[65] Understanding Geometric Representations in Self-Supervised Vision Transformers via Subspace Intervention cs.CVPDF
Weichen Zhou, Yawen Zou, Chunzhi Gu, Ran Dong, Haoran Xie
TL;DR: 本文提出了一种受控子空间干预框架，用于研究自监督视觉变换器（ViTs）如何编码密集的几何信息。通过将收敛线性探针的权重分解，利用奇异值分解（SVD）来隔离包含显式几何信号的低秩子空间。研究发现，预训练目标决定了特征的编码方式，几何表示具有高度可压缩性，且几何精度在中间层达到峰值。
Details
Motivation: 线性探针被广泛用于评估几何表示，但将特征视为黑箱，无法解耦底层拓扑结构。为了解决这个问题，需要一种方法来探究自监督ViTs内部如何编码几何信息。
Result: 研究提供了三个关键发现：DINOv2能对齐空间特征以进行高效线性提取，而MAE倾向于分散这些信号；显式几何表示高度可压缩，可约束在低秩子空间中；几何精度在中间层达到峰值，然后在最终层让位于语义抽象。
Insight: 创新点在于通过子空间干预框架连接内部编码机制与下游性能，为有效的特征选择和轻量级解码器设计提供了基础。从客观角度看，该方法提供了一种解耦和可视化自监督模型中几何信号的新视角。
Abstract: We introduce a controlled subspace intervention framework to investigate how self-supervised Vision Transformers (ViTs) encode dense geometric information. While linear probing is widely used to assess geometric representations, it treats features as a black box, failing to disentangle the underlying topology. To address this issue, we decompose the weights of converged linear probes to isolate the low-rank subspaces containing explicit geometric signals using Singular Value Decomposition (SVD). Our perspective yields three key insights: (1) Pre-training objectives determine how features are encoded. DINOv2 aligns spatial features for efficient linear extraction, while Masked Autoencoders (MAE) tend to disperse these signals, requiring a broader spatial context. (2) Explicit geometric representations are highly compressible, suggesting dense predictive heads could potentially be constrained to low-rank subspaces with minimal performance loss. (3) The layer-wise task affinity suggests that geometric precision peaks at intermediate layers before yielding to semantic abstraction in the final layers. By connecting internal encoding mechanics with downstream performance, these findings provide a basis for effective feature selection and lightweight decoder design. The source code is available at https://github.com/Zhou-Weichen/Geosubprobe.
[66] Spatio-Temporal and Clinical Conditioning for Fine-Grained Radiology Report Retrieval cs.CVPDF
P. Sloan, E. Simpson, M. Mirmehdi
TL;DR: 本文提出了STAR3，一种用于放射学报告生成的多模态时空注意力检索框架，通过结合区域级解剖信息、临床指征和纵向变化，实现了更符合临床实践的自动化报告生成。
Details
Motivation: 针对现有基于检索的放射学报告生成方法缺乏明确的解剖学基础、未考虑纵向疾病进展和临床上下文的问题，旨在提升自动化报告生成的临床适用性。
Result: 在MIMIC-CXR数据集上的实验表明，STAR3在检索、自然语言处理和临床指标上均优于现有检索方法，验证了其有效性。
Insight: 创新点在于将对象检测用于解剖区域识别，并融合当前临床上下文及前后检查的纵向变化来检索报告句子，实现了解剖学、时间维度和临床信息的联合条件化检索。
Abstract: Radiology is vital to modern healthcare, but rising imaging demand and persistent workforce shortages strain reporting capacity and clinical workflows. Automated radiology report generation has the potential to support radiologists and help alleviate this burden; however, existing retrieval-based methods remain rigid, lack explicit anatomical grounding, and do not account for longitudinal disease progression or available clinical context. In this work, we introduce STAR3, a multimodal, spatio-temporal, attentive retrieval framework for radiology report generation that aligns region-level anatomical information with clinical indications and longitudinal changes across chest X-ray studies. Our framework employs an object detector to identify anatomically meaningful regions and retrieves semantically relevant report sentences conditioned on both current clinical context and changes observed between prior and current examinations. This design enables anatomically and temporally grounded report generation that better reflects clinical reporting practice. Experiments on the MIMIC-CXR dataset demonstrate that STAR3 outperforms current retrieval-based approaches on retrieval, NLP and clinical metrics, highlighting the value of conditioning retrieval anatomically, temporally and clinically for advancing automated radiology report generation.
[67] Evaluating Vision-Language Models as a Zero-Shot Learning Alternative to You Only Look Once and Optical Character Recognition for Nigerian License Plate Recognition cs.CVPDF
Ismail Ismail Tijjani, Ahmad Abubakar Mustapaha, Sunusi Ibrahim Muhammad, Muhammad Bashir Aliyu
TL;DR: 本文评估了五种视觉语言模型（VLMs）作为零样本学习方案，用于尼日利亚车牌识别任务，以替代传统的YOLO+OCR多阶段流程。研究使用包含88张具有挑战性的真实世界图像的数据集，发现Gemini和Qwen模型在准确性和鲁棒性上显著优于其他模型。
Details
Motivation: 传统车牌识别系统依赖YOLO和OCR的多阶段流程，存在资源需求高、非结构化环境下性能差、需要大量标注数据等局限性。本研究旨在探索VLMs作为统一的零样本学习解决方案，以克服这些限制。
Result: 基于字符错误率（CER）的评估结果显示，在具有挑战性的图像场景中，Gemini和Qwen模型在准确性和鲁棒性上显著优于GPT-4o、Claude 4 Sonnet和Llama 3.2 Vision 90b等其他模型。
Insight: 论文的创新点在于将VLMs作为零样本学习方案应用于特定区域（尼日利亚）的车牌识别，避免了传统方法对大量标注数据和复杂流程的依赖，并展示了VLMs在真实世界挑战性场景下的实用优势，对模型提供商的宣传声明提出了质疑。
Abstract: License Plate Recognition (LPR) systems are critical tools in traffic monitoring, security enforcement, and urban mobility management. Traditional LPR systems often rely on a multi-stage pipeline involving object detection using You Only Look Once (YOLO) and Optical Character Recognition (OCR), which suffer from limitations such as high resource demands, poor performance in unstructured environments, and the need for large annotated datasets. This study explores the potential of Vision-Language Models (VLMs) as a unified, zeroshot learning solution for Nigerian license plate recognition. Using a curated dataset of 88 challenging real-world images collected in Nigeria, we evaluate five selected VLMs: Gemini 2.0 Flash Exp (Google DeepMind), Qwen2.5-VL-7B-Instruct (Alibaba), GPT-4o (OpenAI), Claude 4 Sonnet (Anthropic), and Llama 3.2 Vision 90b (Meta). Results based on Character Error Rate (CER) reveal that Gemini and Qwen significantly outperform other models in both accuracy and robustness, on the challenging image scenarios. This work highlights the practical advantages of VLMs over YOLO+OCR, questions the claims by model providers, and compares the performances of the VLMs.
[68] ComplexMimic: Human-Scene Interaction Imitation in Complex 3D Environments cs.CVPDF
Lu Pan, Hongwei Zhao
TL;DR: 本文提出了ComplexMimic框架，用于在复杂3D环境中进行基于物理的人-场景交互模仿学习。该框架通过双专家策略和难度感知蒸馏方法，解决了在复杂场景中同时实现精确动作跟踪和物理合理交互的权衡问题。
Details
Motivation: 现有的人-场景交互模仿学习方法大多关注简化场景，在复杂环境中的研究不足，这限制了其在真实世界中的应用。本文旨在解决在复杂3D环境中进行交互模仿时，在成功执行交互与保持自然、物理合理的运动之间存在的固有权衡问题。
Result: 在三个基准数据集上进行的大量实验表明，该方法优于当前最先进的方法。
Insight: 主要创新点包括：1）双流策略，通过模仿专家和交互专家分别处理运动跟踪和碰撞感知适应；2）难度感知蒸馏策略，利用失败统计和学习进度信号自适应地加权监督，优先处理困难但可学习的轨迹，以解决多专家蒸馏中挑战性行为采样不足的问题。
Abstract: Physics-based Human-Scene Interaction (HSI) imitation learning is crucial for embodied intelligence as it bridges the gap between kinematic 3D motions and real-world dynamics. However, most existing methods focus on simplified scene settings, leaving complex environments largely unexplored, which limits their applicability in real-world scenarios. In this paper, we focus on HSI mimicry in complex environments. Under this complex setting, we observe an inherent trade-off between successfully performing interaction and maintaining natural, physically plausible motions. To address this challenge, we propose ComplexMimic, a framework that reconstructs diverse HSI by interpreting imperfect MoCap data. First, we introduce a Dual Flow Strategy, which learns two complementary experts: an imitation expert for accurate motion tracking and an interaction expert for collision-aware adaptation in complex scenes. Second, naive multi-expert distillation, which treats all experts equally, often under-samples challenging behaviors, limiting effective learning. To mitigate this issue, we propose a difficulty-aware distillation strategy that adaptively weights supervision and prioritizes hard-yet-learnable trajectories guided by failure statistics and learning progress signals. Extensive experiments on three benchmark datasets demonstrate that our approach outperforms current state-of-the-art methods. Our implementation is available at https://github.com/LuPan23/ComplexMimic.
[69] ESC: Emotional Self-Correction for Reliable Vision-Language Models cs.CV | cs.AI | cs.CL | cs.LG | cs.MMPDF
Tien-Huy Nguyen, Minh-Nhat Nguyen, Nguyen Nhat Huy, Hung Viet Nguyen, Huy Nguyen Minh Nhat
TL;DR: 本文提出ESC（Emotional Self-Correction）框架，一种无需额外训练、利用情感信号触发视觉语言模型（VLMs）进行自我修正的方法。该方法通过外部验证器检测潜在错误初始响应，并注入情感反馈以鼓励模型反思并生成修订后的答案，从而提升模型的可靠性。
Details
Motivation: 现有视觉语言模型在推理中存在不可靠性问题，而传统的自我修正方法通常依赖后训练或精心设计的反馈，计算成本高昂。本文旨在探索是否可以利用情感线索作为触发器，在不增加训练开销的情况下激活模型潜在的自我修正行为。
Result: 在安全性、幻觉、以视觉为中心的感知以及多模态推理等多个基准测试上的广泛实验表明，ESC能持续提升模型的可靠性，同时保持模型的整体效用。
Insight: 创新点在于将情感信号作为一种无需训练的控制信号，用于触发和引导视觉语言模型的自我修正过程。这揭示了情感不仅可作为模型的识别对象，还能作为一种实用的、可扩展的机制来增强模型推理的可靠性和类人化。
Abstract: Vision-language models (VLMs) have achieved strong performance across diverse multimodal tasks, yet they remain vulnerable to unreliable reasoning. Existing self-correction methods mitigate these issues but typically rely on post-training or carefully engineered feedback, incurring high computational cost. In this work, we revisit this challenge through the lens of emotional cues, asking whether they can activate latent self-correction behaviors in VLMs without additional training. \textbf{We find that emotional signals serve as an effective trigger for self-correction, encouraging more cautious and reflective reasoning}. Motivated by this finding, we propose \escabstract (\textbf{\underline{E}}motional \textbf{\underline{S}}elf-\textbf{\underline{C}}orrection), a training-free self-correction framework. ESC introduces an external verifier that detects potentially incorrect initial responses and injects emotional feedback to encourage model to reflect, and produce a better revised response without additional training. Extensive experiments across safety, hallucination, vision-centric perception, and multimodal reasoning benchmarks show that ESC consistently improves reliability while preserving overall model utility. These results suggest that emotion can function not only as an ability to be recognized, but also as a practical control signal for scalable self-correction in VLMs. \textbf{We therefore believe that ESC provides a strong foundation for a new reliable human-like, emotion-integrated research direction.} Our project is publicly available at \textcolor{red}{https://genai4e.github.io/ESC/}.
[70] HandsOnWorld: Unconstrained Egocentric Video Generation with Camera-Disentangled Hand Control cs.CVPDF
Yushuo Chen, Xiaoyu Shi, Xiaoshi Wu, Xintao Wang, Pengfei Wan
TL;DR: 本文提出了HandsOnWorld框架，用于生成以手部动作为控制信号的第一人称视角视频，无需多视角或基于标记的运动捕捉数据，仅从单目视频中学习。该方法通过单目重建在野外第一人称视频上直接标注3D手部数据，构建了EgoVid-Pro数据集，并提出了Plücker Hand Map表示来解耦相机和手部运动。实验表明，该方法在重建保真度和控制准确性上优于现有方法，并能泛化到实验室数据集之外的日常场景。
Details
Motivation: 解决现有手部控制视频生成方法受限于可扩展的3D手部标注数据的问题，这些方法通常依赖受限的场景分布或仪器化设置，无法处理大规模、无约束的日常第一人称视频。
Result: 在重建保真度和控制准确性上超越了先前的手部控制生成方法，并能够泛化到分布外的日常场景，而先前方法依赖于实验室数据集。
Insight: 创新点包括：通过单目重建和基于动作语义、图像质量及3D几何的过滤流程构建大规模、高质量的第一人称手部轨迹数据集EgoVid-Pro；提出Plücker Hand Map这一3D感知控制信号，在表示层面解耦相机和手部运动，以处理大尺度自我运动带来的纠缠问题。
Abstract: We present HandsOnWorld, a framework for hand-controlled egocentric video generation that forgoes multi-view and marker-based motion capture, learning instead from unconstrained monocular video. Such generality is bottlenecked by the scarcity of scalable 3D hand annotations: large egocentric corpora lack finger-level labels, whereas precise hand datasets are confined to narrow, instrumented settings, limiting prior hand-controlled generators to restricted scene distributions. We instead annotate 3D hands directly on in-the-wild egocentric video through monocular reconstruction, introducing a protagonist-centered annotation pipeline that filters the reconstructions at the action-semantic, image-quality, and 3D-geometric levels to build EgoVid-Pro, a dataset of clean, protagonist-only hand trajectories spanning 103K clips and roughly 12M frames across diverse everyday scenes. To resolve the camera-hand entanglement induced by large ego-motion, we further propose the Plücker Hand Map, a 3D-aware control signal that extends Plücker-ray representations from camera rays to the hand surface, disentangling camera and hand motion at the representation level. Experiments show that \method surpasses prior hand-controlled generators in reconstruction fidelity and control accuracy, and generalizes to out-of-distribution everyday scenes beyond the laboratory datasets on which prior methods rely.
[71] Multimodal Fusion for Fine-Grained Classification of Breast Fibroadenoma and Phyllodes Tumors cs.CVPDF
Chuxi Nan, Di Wu, Hongming Guo, Ning Cao, Xiaohui Zhu
TL;DR: 本文提出了一种用于细粒度分类乳腺纤维腺瘤和叶状肿瘤的临床引导多模态融合框架。该方法构建了包含超声图像、结构化临床属性和超声诊断描述的多模态数据集FAPT-M，并设计了结合视觉、文本和临床编码的网络，通过自适应调制和跨模态融合提升分类性能。
Details
Motivation: 解决B超图像中乳腺纤维腺瘤和叶状肿瘤形态高度重叠导致的误诊问题，现有方法多依赖单一模态图像，未能充分利用临床和文本信息的互补性。
Result: 在患者级别的五折交叉验证中，该方法取得了77.64%的准确率、73.38%的F1分数和89.74%的AUC，优于代表性的CNN、Transformer和视觉语言基线模型。
Insight: 创新点在于构建了高质量的多模态乳腺超声数据集，并提出了临床条件自适应调制、跨模态Transformer融合和双路径表示学习机制，有效整合了图像、文本和临床信息进行细粒度分类。
Abstract: Breast fibroadenoma (FA) and phyllodes tumor (PT) are fibroepithelial breast lesions with highly overlapping appearances on B-mode ultrasound, making benign and borderline PT prone to being misclassified as FA and complicating preoperative decision-making. Existing computer-aided diagnosis methods commonly rely on single-modal imaging features and insufficiently exploit complementary clinical and textual information. To address this limitation, we construct the FAPT-M Dataset, a pathology-confirmed multimodal dataset comprising 910 patients with strictly reviewed ultrasound images, structured clinical attributes, and ultrasound diagnostic descriptions. Based on this dataset, we propose a clinically guided multimodal framework that integrates DenseNet-based visual encoding, CLIP-inspired text encoding, and lightweight clinical encoding, and further introduces clinical-conditioned adaptive modulation, cross-modal Transformer fusion, and dual-path representation learning to improve feature alignment and multimodal interaction. Under patient-level five-fold cross-validation, the proposed method achieves an accuracy of 77.64%, F1-score of 73.38%, and AUC of 89.74%, outperforming representative CNN-, Transformer-, and vision-language-based baselines. Ablation studies and class-balanced evaluations further confirm the contribution of three-modality fusion and the key architectural components. Overall, this work provides an effective multimodal approach for fine-grained FA-PT classification and establishes a high-quality benchmark for multimodal breast ultrasound analysis.
[72] LongEgoRefer: A Benchmark for Long-Form Egocentric Video Referring Expression Comprehension cs.CVPDF
Shunya Kato, Taiki Miyanishi, Shuhei Kurita, Mahiro Ukai, Nakamasa Inoue
TL;DR: 本文提出了LongEgoRefer，一个基于Ego4D数据集构建的、用于长时第一人称视频指代表达理解（Video REC）的新基准。该基准包含平均时长45分钟的长视频和1498个指代表达，特点是目标对象稀疏、语言描述详细且交互复杂，旨在评估模型在长时、未修剪视频中进行时空定位的能力。实验表明，现有先进方法在该基准上表现不佳，凸显了任务的挑战性。
Details
Motivation: 现有第一人称Video REC基准主要关注目标对象密集出现的短视频片段，无法反映真实世界中长时、未修剪、对象稀疏且活动转换复杂的第一人称视频特性。
Result: 在LongEgoRefer基准上，评估了包括基于视觉语言模型和Grounded SAM2的免训练基线在内的现有Video REC方法。广泛实验表明，即使是先进的基线和当前最先进的模型在该基准上也表现显著困难。
Insight: 创新点在于构建了一个更贴近真实场景的长时、稀疏目标、复杂交互的第一人称Video REC基准，将任务定义为一个要求高的时空定位问题。这揭示了现有模型在长时视频理解上的局限性，并强调了开发更鲁棒模型的必要性。
Abstract: Egocentric videos capture rich and diverse human-object interactions and have emerged as a fundamental resource for understanding human activities related to objects. In this context, Video Referring Expression Comprehension (Video REC), the task of localizing the temporal and spatial extent of a referred object in video frames given a natural language query, plays a key role in linking textual descriptions to observed objects in untrimmed egocentric recordings. However, existing egocentric Video REC benchmarks primarily focus on short video clips, where some target object appears densely within frames. Such settings do not reflect real-world egocentric recordings, which are long-form, untrimmed, and characterized by sparse object occurrences and complex activity transitions. To address this limitation, we introduce LongEgoRefer, a novel and challenging benchmark constructed from long-form videos in the Ego4D dataset. LongEgoRefer contains 1,498 referring expressions with an average video duration of 45 minutes. The benchmark exhibits extreme target sparsity, detailed linguistic descriptions, and complex human-object interactions embedded in long, dynamic egocentric narratives. Consequently, it defines a demanding spatio-temporal grounding problem that requires models to identify both when an event occurs and where the referred object appears within extended video sequences. We evaluate existing Video REC approaches, including training-free baselines based on vision-language models combined with Grounded SAM2. Extensive experiments show that even advanced baselines and current state-of-the-art models struggle significantly on LongEgoRefer. These results highlight the intrinsic difficulty of long-form egocentric spatio-temporal grounding and emphasize the need for more robust video understanding models.
[73] DeepGaze3.5-VL: Modeling Scanpaths via Autoregressive Token Prediction cs.CVPDF
Susmit Agrawal, Matthias Bethge, Matthias Kümmerer
TL;DR: 该论文提出了一种名为DeepGaze3.5-VL的模型，将视觉扫描路径预测纯粹视为离散序列建模任务。通过将坐标映射到文本词汇，并利用预训练视觉语言模型的表征，该模型能够灵活地整合多种条件信息，如观看者身份或任务目标，并在多个数据集上实现了新的最优性能。
Details
Motivation: 传统扫描路径建模方法依赖具有手工先验的专用架构，其僵化的结构偏差限制了可扩展性和灵活的条件控制。本文旨在解决这一问题，提出一个能够灵活整合多种变化因素（如个性化偏差、任务目标）的统一框架。
Result: 该模型在多个数据集上建立了新的最优性能，在MIT1003数据集上实现了2.18比特的信息增益，比DeepGaze III提高了46%。即使基线模型使用相同的高容量视觉编码器，该优势依然存在。
Insight: 核心创新点在于将扫描路径预测重新定义为离散序列建模任务，并利用预训练视觉语言模型的表征能力。这允许通过简单提示实现全局条件控制，并能整合每注视点的属性。此外，其自回归对齐方式实现了可扩展的、精确的每注视点对数似然计算，可直接等同于常用的信息增益指标，并可作为强大的计算工具进行受控的行为干预模拟。
Abstract: Understanding human visual attention on a scene over time has applications in domains such as interface design and inferring cognitive states. Modeling visual scanpaths has historically relied on specialized architectures with hand-crafted priors. While these architectures can model fixation sequences, their rigid structural biases restrict easy extendability and flexible conditioning. For instance, integrating task-specific instructions or adapting to distinct viewer identities requires custom, disjoint architectural additions. We frame scanpath prediction purely as a discrete sequence modeling task. By mapping coordinates into a text vocabulary, we leverage the pretrained representations of Vision-Language Models. This framing absorbs diverse factors of variation: simple prompting allows for global conditioning, such as providing viewer identities to capture personalized biases, or task-specific objectives like visual search. The framework can also integrate per-fixation attributes, such as individual fixation durations, alongside spatial locations. The autoregressive alignment enables the scalable, exact computation of per-fixation log-likelihoods, directly equivalent to the commonly used Information Gain (IG) metric. Our model, DeepGaze3.5-VL, establishes a new state-of-the-art across multiple datasets, achieving 2.18 bits of IG on MIT1003, a 46% improvement over DeepGaze III. This advantage persists even when baselines use identical high-capacity vision encoders. Beyond predictive performance, our generative framework serves as a powerful computational tool for direct behavioral interventions, allowing for controlled in-silico simulations that would be experimentally difficult or impossible to conduct in vivo. We demonstrate this ability by performing controlled interventions on the durations of pre-saccadic fixations, recovering known oculomotor phenomena purely from data.
[74] X-Splat: Gaussian Splatting for 3D CBCT Generation from Single Panoramic Radiograph cs.CVPDF
Tomasz Szczepański, Szymon Płotka, Michal K. Grzeszczyk, Tomasz Trzciński, Arkadiusz Sitek
TL;DR: 本文提出了X-Splat，首个基于高斯溅射（Gaussian Splatting）的框架，用于从单张全景X光片（PXR）生成类似锥形束计算机断层扫描（CBCT）的3D牙科体积。该方法利用已知的X射线采集几何作为生成支架，通过可学习的各向异性高斯基元沿X射线路径初始化，并在前馈过程中进行调整，同时结合了Beer-Lambert重投影和多视角X光训练监督。
Details
Motivation: 从单张全景X光片生成3D牙科体积可以作为一种低辐射的CBCT替代方案，但该问题高度欠定，现有隐式或生成方法常产生过度平滑的几何结构或解剖学不一致的幻觉，缺乏几何驱动的监督。
Result: 在合成PXR-CBCT配对数据上训练，X-Splat在基于分割的几何感知指标上超越了基于NeRF和GAN的基线方法，能够重建单个牙齿、皮质边界和牙槽结构（包括下颌管），而先前方法无法重建下颌管。
Insight: 创新点在于首次将高斯溅射框架应用于单视图3D牙科体积生成，并利用已知的X射线采集几何作为生成约束；同时，引入轻量级残差细化器在不覆盖已解析几何的情况下融入数据集级解剖先验，并提出了基于分割的几何感知评估指标。
Abstract: Generating a 3D dental volume from a single panoramic radiograph (PXR) could provide a low-radiation alternative to Cone-Beam Computed Tomography (CBCT), but the problem is highly underdetermined: panoramic acquisition integrates 3D attenuation along curved X-ray paths into a 2D image, leaving depth-resolved anatomy unobserved. Existing implicit and generative approaches often produce oversmoothed geometry or anatomically inconsistent hallucinations, lacking geometry-driven supervision and relying on smooth representations unable to precisely localize sharp anatomical boundaries. We propose X-Splat, the first Gaussian Splatting framework for generating CBCT-like 3D dental volumes from a single PXR. X-Splat uses the known panoramic acquisition geometry as a generation scaffold: learnable anisotropic Gaussian primitives are initialized along the X-ray paths that formed the input image and adjusted in a single feed-forward pass, constrained by Beer-Lambert reprojection and multi-view radiographic training supervision. A lightweight residual refiner adds dataset-level anatomical priors without overriding the geometry already resolved by the Gaussians. We train on synthetic PXR-CBCT pairs, enabling direct volumetric supervision without paired real scans. We further introduce segmentation-based geometry-aware metrics, providing the first evaluation of PXR-based generation over maxillofacial anatomy. X-Splat outperforms NeRF- and GAN-based baselines, recovering individual teeth, cortical boundaries, and alveolar structure, including the mandibular canal which prior methods fail to reconstruct. Code will be available at https://github.com/tomek1911/X-Splat
[75] Towards Robustness against Typographic Attack with Training-free Concept Localization cs.CV | cs.CLPDF
Bohan Liu, Wenqian Ye, Guangzhi Xiong, Zhenghao He, Sanchit Sinha
TL;DR: 本文针对CLIP模型在Typographic Attack（TA）下的脆弱性，提出了一种无需训练的机制可解释性方法，通过分析ViT隐藏状态表示，定位编码词汇信息的特定注意力头，并实施简单干预以提升模型对TA的鲁棒性。该方法在目标分类和LVLMs的视觉问答任务中显著提高了准确性。
Details
Motivation: CLIP模型作为现代大型视觉语言模型的基础视觉编码器，存在一个关键但未被充分探索的失效模式：图像中出现的无关文本会混淆视觉表示，使其偏向词汇含义而非真实视觉语义，这种Typographic Attack对自动驾驶等安全关键应用构成重大风险。
Result: 在RIO-Bench上的实验表明，该方法在Typographic Attack干扰下，显著提升了多个SOTA LVLMs的视觉问答准确率，且干预效果优于有监督和无训练的防御方法。
Insight: 创新点在于提出了一种基于采样的机制可解释性方法，无需训练即可定量归因语义与词汇焦点，并通过电路挖掘定位TA的机制根源；客观来看，其通过选择性调整注意力权重等简单干预直接作用于已识别电路，有效且通用地提升了鲁棒性。
Abstract: Models trained via Contrastive Language-Image Pretraining (CLIP) serve as the foundational vision encoders for most modern Large Vision Language Models (LVLMs). Despite their widespread adoption, CLIP models exhibit a critical yet underexplored failure mode: irrelevant text appearing within images confounds visual representations, biasing them toward lexical meaning rather than true visual semantics. This robustness issue, commonly described as a Typographic Attack (TA), exposes a vulnerability that poses a significant risk to safety-critical applications such as autonomous driving. To achieve interpretable and effective robustness against TA, we propose a novel, training-free mechanistic interpretability method. Our method provides sampling-based interpretations of hidden state representations and quantitatively attributes semantic versus lexical focus to individual attention heads. Through probabilistic analysis and circuit mining, we isolate specific Vision Transformer (ViT) components that disproportionately encode lexical information, thereby identifying the mechanistic source of TA. We further show that simple interventions applied directly to the identified circuits, without any additional training, can substantially improve robustness against Typographic Attacks in object classification. These interventions, such as selective adjustment of attention weights, also outperform both supervised and training-free defense methods. Our experiments demonstrate that applying the proposed intervention to the vision encoders of several state-of-the-art LVLMs yields substantial gains in Visual Question Answering accuracy under Typographic Attack interference on RIO-Bench. These results confirm both the efficacy and the generalizability of our mechanistic approach. Code is released at https://github.com/Liu-524/SamplingTAR.
[76] SAMoR: Motion Modelling for Articulated Objects of Any Skeleton and Topology cs.CVPDF
Yuhao Zhang, Gerard Pons-Moll, Tolga Birdal
TL;DR: 论文提出了SAMoR（Skeleton-Aware Motion Representation for Articulated Objects），一种用于任意骨架拓扑的铰接物体的跨拓扑运动表示方法。该方法通过将每个运动片段编码为少量（K=8）跨骨架共享的部件令牌，解决了现有运动生成器局限于固定人体骨架、跨拓扑适应方法无法共享词汇或丢失运动细节的问题。SAMoR支持精确重建、跨拓扑运动迁移，并能通过文本条件生成和部件级编辑。
Details
Motivation: 现有运动生成模型主要针对固定的人体骨架，而现有适应方法在处理任意骨架拓扑的铰接物体（如不同动物）时，要么无法在骨架间共享运动词汇，要么通过全局池化丢失了运动细节。
Result: 在由HumanML3D、Truebones Zoo和Objaverse-XL动画资产组成的异构数据集上评估，SAMoR在未见过的骨架上的跨拓扑重建任务中达到了2.75e-2的归一化MPJPE，比最强的可变关节数（variable-J）分词器基线低5.8倍。在HumanML3D数据集上，其性能与固定骨架的专家模型保持竞争力。
Insight: 核心创新在于观察到功能关节组（如人类手臂、狼前腿、鸟翅膀）的运动结构具有跨物种的对应性，即使关节数量和连接方式不同。因此，SAMoR利用关节名称嵌入和拓扑无关的注意力监督损失，将关节级运动特征压缩为部件级离散令牌，构建了一个跨骨架共享的离散运动码本，实现了对运动细节的保留和跨拓扑的泛化。
Abstract: Modeling motion for articulated objects of arbitrary skeleton topology remains difficult: existing motion generators target a fixed human skeleton, and prior adaptations either fail to share a vocabulary across rigs or discard motion detail through global pooling. Our key observation is that while joint-level motion does not correspond cleanly across species, motion of functional joint groups does: a human arm, a wolf foreleg, and a bird wing share motion structure despite differing joint counts and connectivity, a correspondence that joint names (e.g., “forearm”, “wing_L1”) partially expose even when topology does not. We introduce SAMoR (Skeleton-Aware Motion Representation for Articulated Objects), a cross-topology motion representation that encodes each motion segment as a small fixed number ($K=8$) of part tokens shared across arbitrary skeletons. A graph-transformer encoder consumes per-joint motion features, kinematic graph structure, and joint-name embeddings, then compresses them into part-level tokens via cross-attention pooling and residual vector quantization, yielding a discrete motion codebook shared across rigs. To keep the part queries from collapsing into redundant global representations, we introduce a topology-agnostic attention supervision loss, with joint-name dropout to reduce over-reliance on text labels. We curate a heterogeneous corpus from HumanML3D, Truebones Zoo, and animated Objaverse-XL assets, and evaluate SAMoR on held-out characters with unseen skeletons. It supports accurate reconstruction and cross-topology transfer, and enables text-conditioned generation and part-wise editing via a MaskGIT token generator. SAMoR reaches $2.75 \times 10^{-2}$ normalized MPJPE on cross-topology reconstruction, $5.8\times$ below the strongest adapted variable-$J$ tokenizer baseline, while remaining competitive with fixed-skeleton specialists on HumanML3D.
[77] DetailAnywhere: Fashion Detail Generation via Cross-Modal Feature Alignment Distillation cs.CVPDF
Zijun Li, Yimin Zhou, Jia Sun, Honglie Wang, Pengcheng Wei
TL;DR: 本文提出DetailAnywhere方法，用于解决时尚细节生成任务，即根据参考图像上的焦点标记生成指定区域（如衣领、袖口）的逼真特写视图。该方法通过跨模态特征对齐蒸馏（CFAD）和一致性奖励模型，在首个基准FDBench上显著优于现有开源方法。
Details
Motivation: 现有扩散生成模型在电商应用中未明确研究用户希望检查服装特定细节区域（如纹理、局部结构）的需求，因此本文形式化了一个新的、非模板化的时尚细节生成任务。
Result: 在包含4万+人类验证参考-细节对的FDBench基准上，DetailAnywhere在所有指标和人工评估中均显著优于所有最先进的开源方法，达到了SOTA水平。
Insight: 创新点包括：形式化了焦点条件化的时尚细节生成新任务并构建首个基准；提出跨模态特征对齐蒸馏（CFAD）来弥合参考图像与细节区域之间的语义鸿沟；引入联合评分三个质量轴的一致性奖励模型，通过强化学习优化生成一致性。
Abstract: Diffusion-based generative AI has achieved remarkable success in e-commerce applications such as virtual try-on, poster generation, and product background synthesis. However, when making online purchasing decisions for apparel, consumers also desire the freedom to examine specific detail regions of interest, such as collars, cuffs, and fabric textures, yet existing methods have not explicitly studied this setting. We therefore formalize a new, non-template task: Fashion Detail Generation with focus conditioning, and release FDBench, the first benchmark comprising 40K+ human-verified reference-detail pairs across 41 different categories. This task poses a unique semantic gap challenge: the model must bridge the correspondence between a focus marker on a product reference image and a photorealistic close-up view of the indicated region, while faithfully preserving the garment’s identity, without any precise prompt. To bridge this gap, we propose Cross-modal Feature Alignment Distillation (CFAD), which leverages a fine-tuned DINOv3 teacher to align both branches of a Multimodal Diffusion Transformer in a shared semantic space via dual-branch distillation. To further improve consistency between generated details and reference images, we introduce a consistency reward model that jointly scores image pairs along three quality axes and optimizes generation via reinforcement learning. Experiments show that our model DetailAnywhere significantly outperforms all state-of-the-art opensource methods across all metrics and human evaluations.
[78] When Token Compression Breaks: Structural Pruning vs. Token Reduction for Robust ViT Segmentation under High Compression cs.CVPDF
Tien-Phat Nguyen, Ngai-Man Cheung
TL;DR: 本文系统比较了视觉Transformer（ViT）语义分割中两种效率优化方法——令牌压缩与结构剪枝——在高压缩率及输入损坏场景下的性能。研究发现，令牌压缩在温和压缩时有效，但在高压缩下因信息丢失而性能急剧下降；结构剪枝则表现更稳定。基于此，作者提出了一种先剪枝后合并的流水线，在高压缩下实现了更好的精度-鲁棒性权衡。
Details
Motivation: 现有研究主要关注低到中等压缩率下的令牌压缩方法，缺乏对高压缩和输入损坏情况下性能的系统评估，且令牌压缩与结构剪枝这两种正交的效率优化路径缺乏统一协议下的公平比较。
Result: 在ADE20K、Cityscapes及其损坏变体ADE20K-C、Cityscapes-C上，在匹配FLOPs条件下进行基准测试。结果显示，令牌压缩在严重压缩时性能急剧下降，而结构剪枝退化曲线更平滑；提出的先剪枝后合并策略在高压缩下取得了更优的精度-鲁棒性权衡。
Insight: 论文的创新点在于首次系统评估并揭示了令牌压缩在高压缩下的脆弱性，以及结构剪枝的相对稳定性；提出的组合策略（先适度剪枝再令牌压缩）为部署导向的ViT分割提供了一个实用的高效方案，平衡了效率与鲁棒性。
Abstract: Vision Transformers (ViTs) are strong backbones for semantic segmentation, but their computational cost limits deployment. Recent token compression methods for efficient transformer-based segmentation reduce this cost by decreasing the number of tokens. However, existing evaluations primarily focus on low-to-moderate compression, leaving their behavior under aggressive compression and corrupted inputs unclear. Meanwhile, structural pruning provides an orthogonal route to efficiency by removing redundant components in the ViT architecture, but is rarely compared to token compression under a unified protocol. To bridge this gap, we benchmark representative token compression and structural pruning methods for ViT-based semantic segmentation under matched FLOPs on ADE20K and Cityscapes, together with their common-corruption variants ADE20K-C and Cityscapes-C. Our results reveal a consistent trend on both clean and corrupted inputs: token compression is highly effective at mild reductions but degrades sharply when compression becomes severe, consistent with substantial information loss from overly aggressive token reduction. In contrast, structural pruning exhibits a smoother degradation curve and is more stable at high compression. Motivated by these findings, we study a prune-then-merge pipeline that applies moderate token compression on top of a moderately pruned backbone. At comparable FLOPs, this combined strategy consistently achieves a better accuracy-robustness trade-off at high compression, offering a practical recipe for deployment-oriented ViT segmentation. Code is available at https://github.com/phatnguyencs/vit-seg-compression.
[79] Efficient PEFT Methods with Adaptive Checkpointing for Vision Models and VLMs on Resource Constrained Consumer-GPUs cs.CVPDF
Altay Toktassyn, Jurn-Gyu Park
TL;DR: 本文在资源受限的消费级GPU上，系统比较了五种参数高效微调方法（Full FT、LoRA、AdaLoRA、QLoRA、BitFit）和三种梯度检查点策略（包括提出的自适应算法）在Transformer和Mamba视觉骨干网络上的性能。实验在CIFAR-100和DTD数据集上进行，评估了准确性、训练时间、能耗以及扩展的多目标NetScore指标。结果表明，QLoRA和BitFit能以1-2%的准确率代价降低20-30%的能耗，自适应算法能减少43-79%的峰值内存但增加9-30%的能耗；同时发现DINOv2在CIFAR-100上以更低的能耗超越了微调模型的准确率。
Details
Motivation: 现代预训练视觉模型虽然准确率高，但全参数微调需要大量GPU内存，难以在边缘设备上部署。本文旨在在严格的设备VRAM预算下，评估和比较不同的参数高效微调方法和内存优化策略，以实现资源受限环境下的高效模型适配。
Result: 在CIFAR-100数据集上，DINOv2（使用轻量级评估协议）的准确率达到0.917，超越了微调模型的0.897，且能耗显著更低。QLoRA和BitFit在牺牲1-2%准确率的情况下，降低了20-30%的能耗；提出的自适应梯度检查点算法将峰值内存降低了43-79%，但带来了9-30%的能耗开销。小型自回归视觉语言模型在实验中表现不佳。
Insight: 论文的创新点在于提出了一个内存预算感知的自适应梯度检查点算法，并首次在资源受限设置下系统性地比较了多种PEFT方法、梯度检查点策略以及不同类型的视觉基础模型。从客观角度看，其将多目标评估指标NetScore扩展为包含部署感知变体，为边缘设备上的模型微调提供了实用的综合评估框架和内存优化方案。
Abstract: Modern pretrained vision models achieve strong accuracy but demand substantial GPU memory for fine-tuning, making edge deployment impractical. This paper compares five parameter-efficient fine-tuning (PEFT) methods (Full FT, LoRA, AdaLoRA, QLoRA, BitFit) on Transformers- (ViT-Small, TinyViT) and Mamba-based vision backbones (Vim-Small, MambaVision-T) under an on-device VRAM budget (e.g., 2 GB), together with three gradient-checkpointing strategies (none, static, and a proposed memory-budget-aware adaptive algorithm); and we evaluate three families of foundation-model baselines: zero-shot contrastive vision language models (OpenCLIP, SigLIP), self-supervised vision backbones with lightweight evaluation protocols (DINOv2), and autoregressive VLMs for prompt-based classification (PaliGemma, MobileVLM, SmolVLM). Experiments on CIFAR-100 and DTD report accuracy, training time, energy, and the NetScore family of multi-objective metrics, which we extend with two deployment-aware variants. QLoRA and BitFit cut energy 20-30% at a 1-2% accuracy cost; the adaptive algorithm reduces peak memory 43-79% with 9-30% energy overhead. DINOv2 surpasses fine-tuned models on CIFAR-100 (0.917 vs. 0.897) at a fraction of the energy, while small autoregressive VLMs remain uncompetitive.
[80] AnyGroundBench: A Specialized-Domain Benchmark for Video Grounding in Vision-Language Models cs.CV | cs.AIPDF
Rintaro Otsubo, Ryo Fujii, Reina Ishikawa, Taiki Kanaya, Kanta Sawafuji
TL;DR: 本文介绍了AnyGroundBench，这是一个针对视觉语言模型在时空视频定位任务中的领域适应基准。该基准专注于五个专业领域（动物、工业、体育、手术和公共安全），旨在将评估范式从静态的零样本测试转向严格的领域适应评估。
Details
Motivation: 当前视觉语言模型在时空视频定位的评估主要局限于通用日常场景的零样本测试，这与需要处理罕见视觉概念和复杂时空动态的专业领域实际应用存在脱节。由于无法对所有数据分布进行穷举预训练，模型适应新领域的能力至关重要。
Result: 研究对15个最先进的视觉语言模型进行了广泛评估，测试了它们在零样本泛化和实际计算约束下的上下文学习能力。结果表明，当前模型在面对专业领域时，在零样本和基于上下文学习的适应方面均告失败，暴露了时空推理的关键缺陷。
Insight: 论文的创新点在于构建了一个专门用于评估领域适应能力的时空视频定位基准，通过整合新采集视频与现有数据集并提供专用训练子集，系统地衡量模型的专业领域适应性，揭示了当前模型在专业领域泛化上的不足。
Abstract: Vision-Language Models (VLMs) have demonstrated immense promise in Spatio-Temporal Video Grounding (STVG). However, current evaluation protocols are largely confined to zero-shot assessments on general, daily-life benchmarks. This creates a critical disconnect from real-world applications in specialized fields, where models inevitably encounter rare visual concepts and complex spatio-temporal dynamics. Since exhaustive pre-training across infinite data distributions is infeasible, the ability to adapt to novel domains is essential. To bridge this gap, we introduce AnyGroundBench, a domain-adaptation benchmark designed to shift the STVG evaluation paradigm from static zero-shot testing to rigorous domain adaptation. Targeting five specialized domains (animal, industry, sports, surgery, and public security), AnyGroundBench pairs newly captured videos such as expert-annotated mouse behaviors with established datasets, unifying them through dense, high-fidelity spatio-temporal annotations. Crucially, the benchmark provides dedicated training subsets to systematically measure domain adaptability. We extensively evaluate 15 state-of-the-art VLMs, assessing their zero-shot generalization and In-Context Learning (ICL) capabilities under practical computational constraints. Ultimately, our findings reveal that current models fail in both zero-shot and ICL-based adaptation when confronted with specialized domains, exposing critical flaws in spatio-temporal reasoning that future research must address.
[81] AGVBench: A Reliability-Oriented Benchmark of Data Augmentation for Vein Recognition cs.CVPDF
Haiyang Li, Yuming Fu, Qun Song, Hongchao Liao, Jing Chen
TL;DR: 本文提出了AGVBench，一个面向可靠性的静脉识别数据增强基准测试，系统评估了30种代表性增强策略在五个公开手掌和手指静脉数据集、七种骨干网络上的表现，发现多图像混合方法在识别性能上表现最佳，但在对抗性攻击下鲁棒性较差，揭示了准确性与安全性之间的不一致性。
Details
Motivation: 静脉识别作为一种安全的生物识别技术，常受限于标注数据不足和成像变化，而现有面向自然图像的数据增强策略可能破坏静脉的细粒度拓扑结构和纹理特征，因此需要建立一个专门针对静脉识别的可靠性导向的增强基准。
Result: 实验结果表明，多图像混合方法（如MixUp、PuzzleMix、StarMixup）在干净数据上识别性能最强，但校准性差且易受对抗性扰动攻击；严重的几何变换会降低识别性能，且增强效果在手掌和手指静脉数据集间存在差异。
Insight: 创新点在于首次构建了面向可靠性的静脉识别数据增强基准，揭示了仅以准确性为中心的评价不足，强调了增强策略在对抗性安全性和校准性方面的重要性，为设计可靠、安全、鲁棒的静脉识别系统提供了标准化协议和指导。
Abstract: Vein recognition is a secure biometric technology often constrained by limited annotated data and imaging variations. While data augmentation mitigates this, strategies designed for natural images may disrupt the fine-grained topology and textures essential for identity discrimination. We present AGVBench, which evaluates 30 representative augmentation strategies on five public palm- and finger-vein datasets with seven backbone architectures, covering classic CNNs, vision transformers, and vein-specific recognition models. Our results show that multi-image mixing methods (e.g., MixUp, PuzzleMix, StarMixup) generally provide the strongest recognition performance. However, they are often poorly calibrated and vulnerable to adversarial perturbations, revealing a clear inconsistency between clean accuracy and adversarial security. We also find that severe geometric transformations frequently degrade recognition, which is potentially due to feature misalignment or spatial cropping, and that augmentation effectiveness varies across palm and finger vein datasets. These findings prove that accuracy-centric evaluation is insufficient for biometric augmentation. AGVBench provides standardized protocols to support reproducible research and guide the design of reliable, secure, and robust vein recognition systems. Our codebase is available at https://github.com/Advance-VeinTech-Innovators/AGVBench.
[82] Search-based Testing of Vision Language Models for In-Car Scene Understanding cs.CV | cs.SEPDF
Lev Sorokin, Chen Yang, Ken E. Friedl, Andrea Stocco
TL;DR: 本文提出了一种名为ISU-Test的自动化测试方法，用于评估车载场景理解（ISU）系统中的视觉语言模型（VLMs）。该方法结合了基于渲染的场景生成和基于搜索的测试，将测试构建为优化问题，通过系统性地修改场景参数来生成多样化的车载场景。
Details
Motivation: 车载场景理解对于检测安全关键事件（如驾驶员分心）至关重要，行业正越来越多地使用VLMs来解析车载摄像头场景。然而，VLMs可能产生不完整、错误或误导性的描述，且收集真实车载数据成本高、难以扩展，因此需要一种系统性的自动化测试方法。
Result: 在两个案例研究（问答和图像描述）中，ISU-Test在工业原型和开源VLMs上的评估结果表明，其显著优于随机场景生成的基线方法，实现了高达10倍的故障发现率和高达3.6倍的故障覆盖率。
Insight: 创新点在于将基于搜索的测试（视为优化问题）与基于渲染的场景生成相结合，以自动化方式系统地探索和生成多样化的、可能触发模型故障的车载场景配置，从而高效地评估VLMs在安全关键领域的可靠性。
Abstract: In the automotive domain, in-car scene understanding (ISU) enables the detection of safety-critical events, such as driver distraction, and supports drivers or passengers by analyzing the in-car scene and adapting the environment (e.g., ambient lighting). The industry is increasingly exploring vision-language models (VLMs) to interpret camera-recorded in-car scenes and extract information for downstream reasoning tasks. However, VLMs may generate incomplete, erroneous, or misleading scene descriptions, highlighting the need for systematic testing. Collecting real in-vehicle data is costly, difficult to scale, and often infeasible, particularly in early design stages. In this paper, we present ISU-Test, an automated testing approach that combines rendering-based scene generation with search-based testing to evaluate ISU systems. By framing testing as an optimization problem and systematically modifying scene parameters, our method generates diverse in-car scenarios and explores a wide range of configurations. We evaluate ISU-Test on both an industrial prototype and open-source VLMs across two case studies: question answering and captioning, comparing against randomized scenario generation. Results show that ISU-Test significantly outperforms the baseline, achieving up to 10 times higher failure rates and up to 3.6 times higher failure coverage.
[83] FlowCIR: Semantic Transport via Flow Matching for Zero-Shot Composed Image Retrieval cs.CVPDF
Zhenqi He, Ziqi Jiang, Yuanpei Liu, Yanghao Wang, Teng Wang
TL;DR: 本文提出FlowCIR，一种用于零样本组合图像检索（ZS-CIR）的新范式。该方法将ZS-CIR视为参考图像与目标图像嵌入之间的条件语义传输问题，通过条件流匹配学习一个轻量级的传输场来生成目标对齐的查询嵌入。相比基于文本反转的现有方法，FlowCIR在预提取的视觉语言模型（VLM）嵌入上操作，仅训练小型传输模块，显著提升了训练效率，并针对VLM在否定指令上的弱点提出了推理阶段的多重负向引导策略以增强鲁棒性。
Details
Motivation: 现有ZS-CIR方法大多依赖文本反转将参考图像转换为伪文本标记，并通过文本空间简单拼接与指令组合，这种方法对于细粒度语义可能是有损且脆弱的。本文旨在提出一种更高效、更鲁棒的新范式来改进语义组合过程。
Result: 在标准CIR基准测试上的广泛实验表明，FlowCIR与最近的ZS-CIR方法相比，实现了强大且有竞争力的性能。
Insight: 主要创新点在于将ZS-CIR重新定义为条件语义传输问题，并利用条件流匹配学习轻量级传输场，从而避免了基于文本反转方法的局限性。此外，提出的推理阶段多重负向引导策略有效缓解了VLM处理否定语义的不足，提升了模型在否定密集型查询上的鲁棒性。从客观角度看，该工作通过解耦训练与预训练编码器，实现了约10倍的训练资源节省，为高效ZS-CIR提供了一种有前景的架构思路。
Abstract: Zero-shot composed image retrieval (ZS-CIR) aims to retrieve a target image by editing a reference image with a natural-language instruction, without relying on domain-specific annotated triplets. Most existing ZS-CIR methods rely on textual inversion to translate the reference image into pseudo-text tokens and then compose them with the instruction via simple concatenation in the text space, which can be lossy and brittle for fine-grained semantics. In this work, we propose a new paradigm, namely FlowCIR, that casts ZS-CIR as conditional semantic transport between reference and target embeddings. Leveraging \emph{conditional flow matching}, our model learns a lightweight transport field that maps the instruction representation toward a target-aligned query embedding conditioned on the reference image. Since FlowCIR operates on pre-extracted VLM embeddings and trains only a small transport module without updating the image or text encoder, it offers a computationally efficient training protocol compared with prior textual-inversion-based approaches. The resulting framework is training-efficient, requiring roughly $10\times$ fewer training resources than prior textual-inversion-based approaches. We further identify negation and removal as a major failure mode of VLM-based composition. To address this, we propose an inference-only Multi-Negative Steering strategy that steers a negation-containing relative instruction away from its negated semantics, mitigating the limited negation handling of VLMs and improving robustness on negation-heavy queries. Extensive experiments on standard CIR benchmarks demonstrate that FlowCIR achieves strong and competitive performance compared with recent ZS-CIR methods.
[84] DisciplineGen-1M: A Large-Scale Dataset for Multidisciplinary Visual Generation and Editing cs.CVPDF
Zhaokai Wang, Mingxin Liu, Zirun Zhu, Ziqian Fan, Yiguo He
TL;DR: 本文介绍了DisciplineGen-1M，一个包含120万个样本的大规模多学科视觉生成与编辑数据集，涵盖数学、物理、化学、生物、地理、计算机科学、经济学、历史、音乐和体育等多个领域。论文还提出了一种基于该数据集的学科感知推理生成模型，用于文本到图像生成和图像编辑任务。实验表明，该模型在学科相关基准测试（如GenExam和GRADE）上相比开源基线有显著提升，并在通用推理基准（如WISE和RISE）上展现出良好的泛化能力。
Details
Motivation: 当前图像生成和编辑模型在处理需要依赖学科概念、符号结构和精确空间关系的知识密集型图表时，其可靠性和准确性仍然不足。为了解决这一问题，作者旨在构建一个大规模、结构化的多学科视觉数据集，以推动图像生成从美学合理性向可验证的知识基础视觉创作发展。
Result: 在学科相关基准测试GenExam和GRADE上，提出的学科感知推理生成模型相比开源基线取得了显著改进。在通用推理基准WISE和RISE上的评估也显示出更广泛的迁移能力，表明模型具有良好的泛化性。
Insight: 论文的核心创新在于构建了一个大规模、高质量的多学科视觉数据集DisciplineGen-1M，其数据构建框架结合了矢量图形渲染、基于OCR的编辑、精心策划的程序化合成和大规模文本到图像过滤，能够生成带有可控语义差异的配对图像和结构化标注。这为知识密集型视觉内容的生成和编辑提供了关键的数据基础，并展示了结构化学术视觉数据对于提升模型在知识密集型任务上性能的重要性。
Abstract: Recent image generation and editing models can produce visually appealing natural images, yet they remain unreliable when the target image is a knowledge-intensive diagram whose correctness depends on disciplinary concepts, symbolic structure, and precise spatial relations. We introduce DisciplineGen-1M, a million-scale multidisciplinary dataset that supports text-to-image generation and image editing. It contains 1.2M samples spanning mathematics, physics, chemistry, biology, geography, computer science, economics, history, music, and sports. To construct the dataset, we design a scalable framework that combines vector-graphics rendering, OCR-based editing, curated programmatic synthesis, and large-scale text-to-image filtering. These pipelines produce captions, editing instructions, structured annotations, and paired images with controllable semantic differences. Building on DisciplineGen-1M, we further introduce a discipline-informed reasoning-generation model for both text-to-image generation and image editing. Experiments on discipline-related benchmarks, GenExam and GRADE, show substantial improvements over open-source baselines, while evaluations on general reasoning-informed benchmarks, WISE and RISE, further indicate broader transfer. The results suggest that large-scale structured academic visual data is a key ingredient for moving image generation from aesthetic plausibility toward verifiable knowledge-grounded visual creation. We will publicly release our dataset, model, and source code of the data curation pipeline to ensure reproducibility and benefit future research.
[85] NEvo: Neural-Guided Evolutionary Video Synthesis for Dynamic Visual Selectivity cs.CVPDF
Yingtian Tang, Sogand Salehi, Ming Zhou, Amir Zamir, Leyla Isik
TL;DR: 本文提出了一种名为NEvo的神经引导进化视频合成框架，用于探索大脑动态视觉选择性。该方法通过进化搜索结构化提示空间，并利用动态编码模型预测视频输入引发的体素级响应，从而生成针对目标大脑区域优化的刺激视频。合成的视频不仅超越了手工定位视频的激活效果，还揭示了腹侧、背侧和外侧通路对时间动态敏感性的系统差异，并为外侧流中复杂社会动态特征的渐进处理提供了新见解。
Details
Motivation: 现有的大脑编码模型主要局限于静态图像刺激，无法充分探索动态视觉处理机制。本文旨在开发一种能够合成优化动态刺激以探测视觉皮层不同脑区选择性的计算框架。
Result: 该方法生成的动态刺激在激活目标脑区方面持续超越手工制作的定位视频。通过搜索分析，该框架揭示了沿外侧流对复杂社会动态特征敏感性逐渐增加的进展，并通过合成的抽象、非自然刺激得到了进一步验证。
Insight: 创新点在于将进化搜索与神经动态编码模型结合，构建了一个结构化提示空间来高效合成超激活动态视频。这为在计算层面系统探索动态视觉选择性提供了新工具，并能生成可用于体内实验验证的新预测。
Abstract: The human brain processes dynamic visual input through hierarchically organized, functionally specialized regions. While recent in silico brain encoding models can synthesize optimal stimuli to probe selectivity in different brain regions, prior work has been largely limited to static images, leaving dynamic visual processing underexplored. We introduce a novel neural-guided video synthesis framework that generates stimuli optimized for target brain regions across visual cortex. Our method performs evolutionary search over a structured prompt space, guided by a dynamic encoding model that predicts voxel-level responses to video inputs. By maximizing predicted activity for a target ROI, the framework efficiently discovers hyper-activating dynamic stimuli that consistently surpass handcrafted localizer videos. The synthesized videos recover known selectivities across ventral, dorsal, and lateral pathways, and further reveal systematic differences in sensitivity to temporal dynamics. A searchlight analysis provides new insight into the progression toward increasingly complex social-dynamic features along the lateral stream, further supported by probing with synthesized abstract, non-naturalistic stimuli. Taken together, our framework enables in silico exploration of dynamic visual selectivity, with new predictions for in vivo experiments
[86] GAP-GDRNet: Geometry-Aware Monocular Visual Pose Sensing on a Single-Target Synthetic Spacecraft Dataset cs.CV | cs.AIPDF
Yonglong Zhang, Yang Liu
TL;DR: 本文提出GAP-GDRNet，一种几何感知注意力增强的单目RGB 6D位姿估计框架，用于解决航天器非合作交会中的相对位姿感知问题。该方法在GDR-Net的几何引导直接回归范式基础上，引入了注意力特征精炼模块和补丁级几何自注意力模块，以增强对弱纹理、光照变化和部分遮挡等挑战的鲁棒性。
Details
Motivation: 针对航天器图像中表面纹理弱、细长附件、光照变化和部分遮挡导致几何证据稀疏且不稳定的问题，需要提升单目视觉位姿估计的鲁棒性和准确性。
Result: 论文未在摘要中提供具体的定量实验结果或基准测试比较，但通过基于Blender的标注流程提供了用于监督训练的目标掩码、可见区域掩码、密集模型坐标图、相机内参和6D位姿标签。
Insight: 创新点在于将注意力机制与几何引导相结合，通过AFR模块强化全局结构和局部弱纹理线索，PGSA模块在补丁级建立几何关联，提升了在稀疏几何证据下的位姿回归性能。
Abstract: Monocular relative pose sensing is a central perception problem in non-cooperative rendezvous and on-orbit servicing. In spacecraft images, however, weak surface texture, thin appendages, illumination changes, and partial occlusion often leave only sparse and unstable geometric evidence. This article presents GAP-GDRNet, a geometry-aware attention-enhanced framework for monocular RGB-based 6D pose sensing. The method follows the geometry-guided direct regression paradigm of GDR-Net and modifies two points in the pipeline: an attention-based feature refinement (AFR) module is placed before dense geometric prediction, and a patch-level geometric self-attention (PGSA) module is inserted into Patch-PnP. AFR reinforces global spacecraft structure together with local weak-texture cues; PGSA then relates downsampled geometric patches before final pose regression. A Blender-based annotation process supplies target masks, visible-region masks, dense model-coordinate maps, camera intrinsics, and 6D pose labels for supervised training.
[87] Learning Spectral and Polarimetric Clues for One-to-Multimodal Novel View Synthesis cs.CVPDF
Federico Lincetto, Gianluca Agresti, Mattia Rossi, Piergiorgio Sartor, Pietro Zanuttigh
TL;DR: 本文提出了一种名为SPoILeR的新方法，用于从RGB图像或极少量多模态数据中，合成多视角一致的非传统模态（如红外、偏振、多光谱）渲染。该方法通过多模态预训练学习不同模态间的相互关联，从而在仅由RGB图像监督的微调阶段，能够准确预测其他模态的视图。
Details
Motivation: 现有神经渲染方法扩展到多模态（如多光谱、红外、偏振）时，通常需要昂贵的传感器和校准设置来为每个新场景捕获多模态数据，这限制了其应用。本文旨在解决仅使用RGB图像或极少多模态样本，即可合成其他模态新视图的问题。
Result: 实验结果表明，该方法能够在未提供红外、偏振或多光谱传感器输入样本的场景中，准确渲染出这些模态的帧。虽然没有明确提及具体基准和SOTA比较，但结果证明了其有效性。
Insight: 核心创新点在于通过多模态预训练学习模态间的内在关联，从而在推理时仅需RGB监督即可实现一对多的跨模态视图合成。这为数据获取成本高昂的多模态应用提供了一种高效的解决方案。
Abstract: Neural rendering techniques allow for accurate reconstruction of the geometry and color appearance of 3D scenes. Some methods have extended their use to additional imaging modalities, such as multispectral, infrared, or polarimetric data. However, all of these approaches require expensive sensors and calibrated setups to capture new multimodal frames for each new scene. We propose Spectral and Polarimetric Implicit Learned Representation (SPoILeR), a novel method to obtain multi-view consistent renderings of unconventional modalities for scenes where either only RGB frames or very few of the additional modalities are available. Thanks to a multimodal pre-training phase, the model learns the mutual correlation between different modalities. This step allows predicting accurate renderings of unconventional modalities during a fine-tuning phase supervised only by RGB images. Experimental results show that the approach can accurately render infrared, polarimetric, and multispectral frames for scenes where no input sample captured by these types of sensors is provided.
[88] VisionAId: An Offline-First Multimodal Android Assistant for People with Visual Impairment, Featuring Personalized Object Retrieval cs.CV | cs.AIPDF
Cristian-Gabriel Florea, Stelian Spînu
TL;DR: VisionAId是一款面向视障人士的离线优先多模态Android助手，它将普通智能手机转化为实时视觉辅助工具。该系统集成了六个在设备端运行的深度学习模型，用于深度估计、实例分割、人脸检测和钞票识别等任务，并引入了一个用于检索个性化物品的少样本流程。
Details
Motivation: 全球有超过2.85亿视障人士，他们在避障、寻找个人物品、识别人脸或处理现金等日常任务上面临持续挑战。现有辅助应用通常局限于识别预定义类别、严重依赖云连接或需要专用硬件。
Result: 在三星Galaxy S21 Ultra参考设备上，INT8量化将深度估计延迟从约1200毫秒降至约491毫秒，定制的钞票检测器mAP@50达到0.986，且在3米范围内度量深度校准误差小于1厘米。
Insight: 主要创新点在于一个用于检索用户特定物品实例的少样本流程，以及一个以设备端模型为主、云LLM为辅的混合架构，实现了离线优先、低延迟的多模态交互（包括AR标记、空间音频和触觉反馈）。
Abstract: Over 285 million people worldwide live with a visual impairment, for whom everyday tasks such as avoiding obstacles, locating personal belongings, recognizing familiar faces, or handling cash remain persistent obstacles to personal autonomy. Existing assistive applications are typically limited to recognizing predefined categories, depend heavily on cloud connectivity, or require dedicated hardware. We present VisionAId, an Android application that turns a commodity smartphone into a real-time visual assistant. The system integrates six on-device deep learning models (metric monocular depth estimation, instance segmentation, visual and facial embeddings, face detection, and a custom banknote detector) running entirely through ONNX Runtime, with an optional cloud large language model (Google Gemini Flash) used only for narrative scene description and automatic object labeling. A distinctive contribution is a few-shot pipeline for personal objects: the user photographs an object from several angles, and the system later locates that specific instance in the environment, guiding the user toward it with augmented-reality markers, spatial audio, and distance-proportional haptics. All feedback is multimodal (Romanian speech synthesis, voice commands, vibration). On a reference device (Samsung Galaxy S21 Ultra), INT8 quantization reduces depth latency from ~1200 ms to ~491 ms, the custom banknote detector reaches an mAP@50 of 0.986, and metric depth is calibrated to below 1 cm of error within 3 m.
[89] Transformer Geometry Observatory TGO-II: Representational Similarity Observatory cs.CV | cs.LGPDF
Kaustubh Kapil, Kishor P. Upla
TL;DR: 本文提出了Transformer Geometry Observatory-II（TGO-II）框架，用于分析Vision Transformer在监督训练过程中内部表示几何结构的演变。研究发现，表示相似性（CKA/SVCCA）随训练下降，内在维度先增后稳，且token间强交互结构持续存在，表明表示复杂性和层特异性同步增长，且流形扩展并非源于token解耦。
Details
Motivation: 现有研究主要关注注意力机制和下游性能，对Transformer内部表示几何结构在训练过程中的演变缺乏深入理解。本文旨在填补这一空白，探究Vision Transformer表示几何的演化规律。
Result: 在ViT-Small/16模型上的实验表明：CKA和SVCCA值在整个训练过程中持续下降；内在维度先增加后稳定；token协方差分析显示强token交互结构始终存在。这些结果为理解表示演化提供了定量依据。
Insight: 创新点在于提出了一个综合表示几何分析框架（TGO-II），并发现表示复杂性的增加与层特异性的增强同步发生，且流形扩展过程并未破坏token间的强耦合结构，这挑战了‘复杂性源于token独立性增强’的假设。
Abstract: While Vision Transformers have achieved remarkable success across computer vision and language applications, the geometric evolution of their internal representations throughout training remains insufficiently understood. Existing analyses primarily focus on attention mechanisms and downstream performance, leaving the evolution of representation geometry largely unexplored. In this work, we present Transformer Geometry Observatory-II (TGO-II), a representation geometry analysis framework designed to investigate how Transformer representations evolve during supervised training. TGO-II analyzes Vision Transformer (ViT-Small/16) representations using Centered Kernel Alignment (CKA), Singular Vector Canonical Correlation Analysis (SVCCA), Two-Nearest Neighbor Intrinsic Dimensionality (TwoNN-ID), and token covariance analysis. Our experiments reveal three key observations. First, both CKA and SVCCA progressively decrease throughout training, indicating increasing representational specialization across Transformer layers. Second, intrinsic dimensionality consistently increases before stabilizing, suggesting progressive expansion of the representation manifold into a larger set of locally accessible degrees of freedom. Third, token covariance and coupling analyses demonstrate that strong token interaction structure persists throughout training, challenging the hypothesis that increasing representational complexity arises primarily from progressive token independence. These findings suggest that representation complexity and layer specialization emerge simultaneously during training. Manifold expansion appears to occur without token decoupling. Together, these observations motivate a new hypothesis in which Vision Transformers increase representational complexity through progressively richer transformations while preserving strong token interaction structure during learning.
[90] Show Me Examples: Inferring Visual Concepts from Image Sets cs.CVPDF
Nick Stracke, Kolja Bauer, Stefan Andreas Baumann, Miguel Angel Bautista, Josh Susskind
TL;DR: 本文提出了视觉概念集合推理（VICIS）任务，旨在评估模型从一组共享概念的示例图像中推断视觉概念并应用于新图像的能力。针对当前视觉语言模型（VLMs）在此任务上的不足，作者提出了一个训练框架和架构，能够从图像集合中学习推断视觉概念，并从查询图像中提取概念特定嵌入。实验表明，该模型在合成数据和ImageNet/WordNet数据上生成更准确、更多样化的输出，并能泛化到未见过的概念和草图等模态。
Details
Motivation: 当前视觉语言模型虽能遵循复杂文本指令，但难以从纯视觉上下文中进行推理，特别是无法从示例图像集合中推断共享概念并将其应用于新输入。
Result: 在合成数据和ImageNet/WordNet数据上的实验表明，所提模型比现有SOTA VLMs生成更准确、更多样化的输出，并能泛化到未见概念和草图等模态。
Insight: 创新点在于提出了VICIS任务以评估视觉概念推理能力，并设计了相应的训练框架和架构，通过从图像集合学习概念推断和从查询提取概念嵌入来提升性能，这为纯视觉上下文推理提供了新思路。
Abstract: Vision-language models (VLMs) can follow complex textual instructions, yet they struggle to reason from purely visual context. In particular, current models fail to infer shared concepts from sets of example images and apply them to new inputs. We introduce Visual Concept Inference from Sets (VICIS), a task that evaluates this capability. Given a small context set of images sharing a concept and a query image, the model must generate new images that preserve the context-defined concept while remaining consistent with the query. We show that state-of-the-art VLMs perform poorly on this task, often ignoring the visual context or defaulting to biased generations. To address this gap, we propose a training framework and architecture that learn to infer visual concepts from image sets and extract concept-specific embeddings from queries. Experiments on synthetic data and large-scale ImageNet/WordNet data show that our model generates more accurate and diverse outputs and generalizes to unseen concepts and modalities such as sketches.
[91] Learning to Evolve Scenes: Reasoning about Human Activities with Scene Graphs cs.CVPDF
Francesca Pistilli, Simone Alberto Peirone, Giuseppe Averta
TL;DR: 本文提出了SG-Ego数据集和GLEN模型，用于通过时空场景图来理解和推理人类活动如何随时间改变场景。SG-Ego扩展了Ego4D数据集，提供了大规模的场景图标注；GLEN是一个基于图的模型，用于对齐文本动作并建模场景图的时序演化。
Details
Motivation: 现有方法依赖隐式的视觉或语言对齐表示，忽略了场景动态的结构化推理。本文认为，明确、可组合且可编辑的人与环境交互表征对于实现丰富的、基于实地的活动理解至关重要。
Result: GLEN在多个下游任务上验证有效，包括EgoMCQ、EgoCVR检索基准，以及EXPLORE-Bench和新的A-GEF长时程推理基准。与原始视频基线相比，GLEN取得了强劲结果，并在通常仅由MLLM处理的推理场景中表现出色。
Insight: 创新点在于引入了大规模的时空场景图标注集（SG-Ego），提出了基于图的模型（GLEN）进行时序推理，并形式化了活动驱动的图编辑预测（A-GEF）这一新任务，将场景动态建模为受持续动作影响的结构化变换序列，实现了可控且结构化的场景动态预测。
Abstract: Understanding human behavior while interacting with the surrounding world is crucial for many applications of embodied AI. First-person videos are particularly informative for this problem, as they well capture how activities reshape the scene over time. However, existing approaches often rely on implicit visual or language-aligned representations, disregarding structured reasoning over the scene dynamic. We argue that explicit, compositional and editable representations of human-environment interactions can play a crucial role for rich grounded activity understanding. To this end, we introduce SG-Ego, a large scale annotation set extending Ego4D with spatio-temporal scene graphs, where relations triplets are consolidated over time into explicit time-evolving descriptions of the scene state. To reason over this representation, we propose GLEN, a graph-based model that operates over scene graph sequences to both align them with textual actions and model their temporal evolution. In addition, we formulate the activity-driven graph-edit forecasting (A-GEF) problem, a novel task that casts scene dynamics as a sequence of structured transformations conditioned on ongoing actions, enabling explicit reasoning about how scenes change over time. We validate our approach across multiple downstream tasks, spanning retrieval benchmarks as EgoMCQ and EgoCVR, as well as long-horizon reasoning benchmarks as EXPLORE-Bench and the newly introduced A-GEF. GLEN achieves strong results compared to raw video baselines and it excels in reasoning settings, typically addressed only with MLLMs, while enabling controllable and structured predictions of scene dynamics driven by human activities. We believe our results establish spatio-temporal scene graphs, together with models that reason over them, as strong compositional and interpretable representations for video understanding and potentially beyond.
[92] OrbitQuant: Data-Agnostic Quantization for Image and Video Diffusion Transformers cs.CV | cs.AI | cs.LGPDF
Donghyun Lee, Jitesh Chavan, Duy Nguyen, Sam Huang, Liming Jiang
TL;DR: 本文提出了OrbitQuant，一种数据无关的权重量化方法，用于图像和视频扩散变换器（DiTs）。该方法通过在归一化、旋转的基上进行量化，绕过了对输入数据范围估计的需求，从而能够使用单一的量化码本处理不同时间步、提示词和层的激活值。该方法无需针对不同模态进行调优，并在多个低比特设置下实现了后训练量化（PTQ）的SOTA性能。
Details
Motivation: 扩散变换器（DiTs）在图像和视频生成上达到了SOTA，但其多步采样和庞大的参数量使得推理成本高昂。现有的后训练量化（PTQ）方法因DiT的激活值会随时间步、提示词和引导分支的变化而发生偏移，需要为每个新检查点或模态重新拟合校准数据，这带来了不便。
Result: 在FLUX.1、Z-Image-Turbo、Wan 2.1和CogVideoX等多个模型上，OrbitQuant在多个低比特设置下（如W2A4）实现了后训练量化的SOTA性能，并将图像扩散变换器的PTQ推进到W2A4比特位宽且保持可用生成质量。
Insight: 核心创新在于提出了一种数据无关的量化框架，通过随机置换块哈达玛（RPBH）旋转将激活值集中在固定的已知边缘分布上，从而允许使用单一的量化码本。该方法还将旋转操作吸收到权重中，使得运行时仅需对激活值进行一次前向旋转，实现了从图像到视频模型的零调优迁移。
Abstract: Diffusion transformers (DiTs) achieve state-of-the-art image and video generation, but their multi-step sampling and growing parameter count make inference expensive. Post-training quantization (PTQ) is the natural remedy, yet DiT activations shift across timesteps, prompts, and guidance branches, forcing prior methods to re-fit calibration data for every new checkpoint or modality. We present OrbitQuant, a data-agnostic weight-activation quantizer that bypasses range estimation by quantizing in a normalized, rotated basis. In this basis, a randomized permuted block-Hadamard (RPBH) rotation concentrates each coordinate around one fixed, known marginal regardless of the input, so a single Lloyd-Max codebook serves all timesteps, prompts, and layers of a given input dimension. We extend the same quantizer to weight rows offline, absorbing the rotation into the weights so that it cancels inside each linear layer and only a forward rotation on the activations remains at runtime. The same recipe transfers from image to video with no per-modality tuning. Across FLUX.1, Z-Image-Turbo, Wan 2.1, and CogVideoX, it sets the state of the art for PTQ at several low-bit settings. It also pushes PTQ of image diffusion transformers to W2A4 with usable generation quality.
[93] EAGLE-360: Embodied Active Global-to-Local Exploration in 360$^\circ$ cs.CVPDF
Jingtao Xu, Zizhuo Lin, Jianwen Sun, Yi Yang, Yawei Luo
TL;DR: 论文提出了EAGLE-360框架，旨在解决多模态大语言模型在360°全景环境中进行主动视觉搜索时存在的局限性。该框架通过利用全局先验建立初始整体视角，并迭代推理逐步缩小搜索空间，同时采用RoPE Rolling位置编码机制来建模全景的连续拓扑结构。
Details
Motivation: 现有全景搜索方法严重依赖碎片化的局部视点，缺乏全局全景先验，导致探索效率低下且难以进行稳健的错误恢复。标准MLLMs难以有效建模全景的固有特性（如严重的极坐标畸变和连续圆柱拓扑），这显著降低了目标检测精度。
Result: 在360°视觉搜索任务上，EAGLE-360建立了新的最先进水平（SOTA），其准确率相比基础模型提升了近8倍，同时显著提高了探索效率。
Insight: 创新点在于提出了一个从全局到局部的具身主动探索框架，并采用了适应全景连续拓扑的RoPE Rolling位置编码机制。此外，构建的大规模EAGLE-360数据集以及结合SFT与GRPO的训练流程，有效激发了复杂的空间推理和工具调用能力。
Abstract: While Multimodal Large Language Models (MLLMs) have demonstrated exceptional capabilities in standard visual understanding, adapting them for active visual search in 360$^\circ$ panoramic environments exposes fundamental limitations. Specifically, standard MLLMs struggle to effectively model inherent panoramic properties, such as severe polar distortion and continuous cylindrical topologies, which significantly degrades target detection accuracy. Consequently, existing panoramic search methods attempt to compensate by relying heavily on fragmented local viewpoints. Burdened by rigid initialization and a lack of global panoramic priors, these approaches suffer from myopic, inefficient exploration and struggle with robust error recovery when targets are out of view. To overcome these challenges, we propose EAGLE-360, a novel Embodied Active Global-to-Local Exploration framework. Rather than performing exhaustive local searches, EAGLE-360 leverages global priors to establish an initial holistic perspective, iteratively reasoning and progressively narrowing the search space. Architecturally, we adapt RoPE Rolling, a coordinate-shifting positional encoding mechanism, to seamlessly model the continuous topologies of panoramas. To facilitate this paradigm, we construct the large-scale EAGLE-360 dataset, comprising 14,000+ 4K panoramas and 70,000+ rounds of high-quality VQA dialogues. By employing a training pipeline that integrates Supervised Fine-Tuning (SFT) with Group Relative Policy Optimization (GRPO), we effectively elicit complex spatial reasoning and tool-calling capabilities. Extensive experiments demonstrate that EAGLE-360 establishes a new state-of-the-art for 360$^\circ$ visual search, achieving nearly an 8-fold increase in accuracy over the base model while significantly enhancing exploration efficiency.
[94] Combating Textual Noise and Redundancy: Entropy-Aware Dense Visual Token Pruning cs.CV | cs.AIPDF
Xuehui Wang, Xuankun Yang, Wei Shen
TL;DR: 本文提出了一种名为熵感知密集剪枝（EADP）的框架，用于解决视觉语言模型（VLM）中视觉令牌剪枝在密集指令和细粒度查询下失效的问题。该方法通过熵量化过滤文本噪声，并将令牌选择建模为带空间先验的子模最大化问题，以实现结构化压缩。
Details
Motivation: 现有视觉令牌剪枝方法在压缩冗余图像块时，面对密集指令和细粒度查询，往往无法保留关键视觉线索。作者发现其根本瓶颈在于文本噪声的广泛分散破坏了跨模态评分，以及标准令牌选择固有的特征碎片化问题。
Result: 大量实验表明，EADP改善了VLM的精度-效率权衡，在严格的令牌预算下鲁棒地保留了细粒度视觉线索，并在具有挑战性的多模态基准测试中达到了最先进（SOTA）性能。
Insight: 核心创新点在于将剪枝重新定义为结构化压缩问题，并引入两个关键机制：利用统计熵量化并过滤文本噪声以获得鲁棒的细粒度指令相关性分数；将令牌选择从简单的Top-K选择转变为带空间先验的子模最大化问题，以显式确保视觉表征的整体性和非冗余性。
Abstract: Visual token pruning is a crucial strategy for accelerating VLMs by compressing redundant image patches, yet existing methods often fail to preserve critical cues under dense instructions and fine-grained queries. In this paper, we investigate this failure and identify two underlying bottlenecks: the widespread dispersion of textual noise that corrupts dense cross-modal scoring, and the feature fragmentation inherent to standard token selection. To address these issues, we propose Entropy-Aware Dense Pruning (EADP), a framework that reformulates pruning as a structured compression problem. EADP first leverages statistical entropy to quantify and filter out textual noise, yielding a robust, fine-grained instruction relevance score. Subsequently, instead of naive Top-K selection, EADP casts token selection as a submodular maximization problem with a spatial prior, explicitly ensuring a holistic and non-redundant visual representation. Extensive experiments demonstrate that EADP improves the accuracy-efficiency trade-off of VLMs, robustly preserving fine-grained visual cues under strict token budgets while achieving SoTA performance on challenging multimodal benchmarks.
[95] Seek to Segment: Active Perception for Panoramic Referring Segmentation cs.CVPDF
Song Tang, Shuming Hu, Xincheng Shuai, Henghui Ding, Yu-Gang Jiang
TL;DR: 本文提出了一项新任务：主动全景参考分割（APRS），要求智能体在360度环境中主动调整视角以寻找用户指令指定的目标并进行分割。为应对此任务，作者提出了PanoSeeker，一个结合视觉语言模型与显式空间视觉记忆（EgoSphere）的智能体，通过逐步整合局部观察来规划高效搜索轨迹，并采用监督微调和强化学习进行优化。
Details
Motivation: 现有参考分割模型被动处理固定视角的静态图像，限制了其在具身AI中的应用，因为智能体需要在连续360度环境中进行主动感知。
Result: 在新建立的APRS基准测试上进行的广泛实验表明，PanoSeeker在搜索效率和分割准确性方面均优于经过调整的现有最先进基线方法，达到了SOTA水平。
Insight: 创新点在于将主动感知引入参考分割任务，通过EgoSphere显式空间记忆整合序列观察以实现高效非冗余搜索，并结合监督微调与强化学习优化探索效率，为具身AI中的视觉任务提供了新思路。
Abstract: Existing referring segmentation models passively process static images captured from fixed perspectives, limiting their applicability in Embodied AI, where agents must perform active perception in the continuous 360$^\circ$ environments. To bridge this gap, we introduce a novel task: Active Panoramic Referring Segmentation (APRS). In this setting, an agent is required to adjust its viewing direction ($Δθ, Δφ$) to explore the 360$^\circ$ environment, seeking the object specified by a user instruction for segmentation. To tackle this challenging task, we propose PanoSeeker, a memory-augmented agent for efficient APRS. Rather than relying on heuristic scanning, PanoSeeker integrates a Vision-Language Model (VLM) with EgoSphere, an explicit spatial visual memory. By progressively integrating sequential local observations into a unified 360$^\circ$ representation, EgoSphere enables the agent to plan efficient and non-redundant search trajectories. Once the target is found, the agent performs active viewpoint alignment and outputs the segmentation mask. Furthermore, we curate an expert-annotated search trajectory dataset with memory timelines for Supervised Fine-Tuning, followed by Reinforcement Learning post-training to explicitly optimize PanoSeeker’s exploration efficiency. Extensive experiments on our newly established APRS benchmark demonstrate that PanoSeeker achieves superior search efficiency and segmentation accuracy, significantly outperforming adapted state-of-the-art baselines.
[96] Interpretation-Oriented Cloud Removal via Observation-Anchored Residual Flow with Geo-Contextual Alignment cs.CVPDF
Ziyao Wang, Maonan Wang, Yucheng He, Xianping Ma, Ziyi Wang
TL;DR: 本文提出了一种面向解释的云去除方法Geo-Anchored Cloud Removal (GACR)，该框架通过观测锚定残差流（OAR-Flow）和地理上下文先验对齐（GCPA）联合确保重建的保真度和下游任务（如语义分割）的鲁棒可解释性。
Details
Motivation: 现有云去除方法过于关注视觉真实性，而忽略了其对下游分析任务（如语义分割、变化检测）的影响，导致语义漂移和性能下降。本文旨在解决这一问题，确保重建结果对后续解释任务友好。
Result: 在六个云去除数据集和十二个下游任务上的大量实验表明，GACR不仅产生了更优的重建质量，而且持续提高了下游任务的准确性。
Insight: 核心创新点在于将云去除重新定义为基于物理的残差反演过程（OAR-Flow），并将生成轨迹锚定在有云观测而非纯噪声上，实现了快速、稳定且忠实重建。同时，利用视觉基础模型（VFM）诱导的语义流形进行地理上下文先验对齐（GCPA），严格保持了复杂景观的空间语义完整性。
Abstract: Cloud removal (CR) is essential for optical remote sensing, serving as a prerequisite for reliable downstream interpretation, such as semantic segmentation and change detection. However, existing CR approaches often prioritize visual realism while overlooking their impact on subsequent analytical tasks, leading to semantic drift and degraded downstream performance. To address this issue, we propose Geo-Anchored Cloud Removal (GACR), a unified framework that jointly ensures faithful reconstruction and robust interpretability. At its core, GACR incorporates Observation-Anchored Residual Flow (OAR-Flow), which reformulates CR as a physically grounded residual inversion process. By anchoring the generative trajectory to the cloudy observation rather than pure noise, OAR-Flow enables fast, stable, and faithful reconstruction. To further preserve semantic structures critical for downstream interpretation, GACR integrates Geo-Contextual Prior Alignment (GCPA) to constrain the reconstruction within a semantic manifold induced by a Vision Foundation Model (VFM). Consequently, GACR strictly maintains the spatial-semantic integrity of complex landscapes. Extensive experiments across six CR datasets and twelve downstream tasks demonstrate that GACR produces superior reconstruction quality while consistently improving downstream task accuracy. The code is available at https://github.com/wzy6055/GACR.
[97] From SRA to Self-Flow: Data Augmentation or Self-Supervision? cs.CVPDF
Dengyang Jiang, Mengmeng Wang, Harry Yang, Jingdong Wang
TL;DR: 本文重新审视了从SRA到Self-Flow的改进机制，挑战了其归因于不同噪声水平token间交互的解释。通过引入注意力分离方法，实验表明性能提升主要源于噪声维度的数据增强，而非token交互。研究进一步结合自表示对齐与双重时间步及注意力分离增强，在ImageNet上验证了其有效性。
Details
Motivation: 旨在探究扩散模型中自对齐方法（如SRA和Self-Flow）性能提升的根本原因，澄清其机制是源于数据增强还是自监督的token交互。
Result: 在ImageNet基准上，注意力分离方法在保持双重时间步输入的同时，移除不同噪声水平token间的注意力交互，不仅未降低性能，甚至有所提升，表明Self-Flow的增益主要来自数据增强。
Insight: 创新点在于揭示了自对齐方法性能提升的关键是噪声维度的数据增强，而非先前认为的token交互；提出的注意力分离技术本身通过将单张图像分割为多个有效训练部分，提供了额外的数据增强效果，为扩散模型训练提供了新的设计思路。
Abstract: Representation alignment has become an effective way to accelerate diffusion transformer training and improve generation quality. Recent self-alignment methods, such as SRA and Self-Flow, further remove the dependency on external pretrained encoders by constructing alignment within the diffusion model itself. However, the mechanism behind the improvement from SRA to Self-Flow, dual-time scheduling, remains under-examined: Self-Flow attributes its gain to interactions between tokens at different noise levels, where cleaner tokens help infer noisier ones. In this work, we revisit this explanation and ask whether the gain instead comes from data augmentation along the noise dimension. To disentangle these factors, we introduce Attention Separation, which preserves the same dual-timestep input as Self-Flow while blocking attention between tokens assigned to different noise levels. Surprisingly, removing such interaction does not degrade performance and can even improve it, suggesting that the improvement from SRA to Self-Flow mainly comes from data augmentation. Furthermore,We show that Attention Separation itself provides an augmentation effect by splitting a single image into multiple effective training parts to expand the training data. Based on these observations, we combine self-representation alignment with dual-timestep and attention-separation augmentation, and demonstrate the effectiveness of this design on ImageNet.
[98] WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory cs.CVPDF
Hanlin Wang, Hao Ouyang, Qiuyu Wang, Wen Wang, Qingyan Bai
TL;DR: 本文提出了WorldDirector，一个高度可控的视频世界模型框架，旨在实现持久的动态物体记忆和无限制的视点探索。该框架通过将语义运动编排与视觉生成解耦，利用大语言模型协调3D轨迹和摄像机运动，再将这些编排好的轨迹作为视频生成的控制信号，从而确保了严格的物理逻辑和外观稳定性。
Details
Motivation: 现有世界模型通常将物理动力学与像素渲染纠缠在一起，并依赖连续的视觉观察来维持运动，这限制了可控性和物体记忆的持久性。本文旨在解决这些问题，构建一个能精确控制动态物体并保持其视觉身份一致性的世界模拟器。
Result: 实验结果表明，该方法支持合成复杂且长时间的事件，具有前所未有的可控性和持久的动态物体记忆能力。
Insight: 核心创新点在于将语义运动编排（由LLM协调）与视觉生成明确解耦，并使用编排的3D轨迹作为控制信号，这确保了物理逻辑和外观稳定性，特别是解决了动态物体长时间离开视野后重新进入时视觉身份保持的难题。
Abstract: We present WorldDirector, a highly controllable video world model framework designed for persistent dynamic object memory and unrestricted viewpoint exploration. Unlike existing world models that entangle physical dynamics with pixel rendering and rely on continuous visual observation to sustain motion, our framework explicitly decouples semantic motion orchestration from visual generation. By leveraging an LLM to coordinate 3D trajectories with camera movements and subsequently employing these orchestrated trajectories as control signals for video generation, our approach ensures strict physical logic and appearance stability, successfully preserving the exact visual identities of dynamic entities even when they re-enter the scene after prolonged periods out of view. Experimental results demonstrate that our method supports the synthesis of complex and extended events with unprecedented controllability and persistent dynamic object memory. Project Page: https://worlddirector.github.io/
[99] Alignment Is All You Need For X-to-4D Generation cs.CVPDF
Qiaowei Miao, Kehan Li, Yawei Luo, Yi Yang
TL;DR: 本文提出了Align4D框架，用于解决任意模态输入（如文本、图像、视频、3D）生成动态4D内容（即随时间变化的3D几何）的挑战。该框架通过视频引导4D运动、3D数据引导4D几何，并引入了对象距离对齐、运动-几何联合对齐和异步优化三项关键技术，在自建的X4D数据集和Consistent4D基准上实现了最先进的生成质量和一致性。
Details
Motivation: 现有生成模型在图像、视频、3D内容合成上表现出色，但任意用户定义模态到4D（X-to-4D）的生成仍面临挑战，主要由于构建多样化数据集的成本高昂以及现有方法可扩展性有限。
Result: 在提出的X4D数据集和Consistent4D基准上进行实验，结果表明Align4D在X-to-4D生成任务中实现了最先进（SOTA）的生成质量和一致性。
Insight: 创新点包括：1）对象距离对齐技术（VAOD/MAOD），协调4D渲染与视频及多视图扩散模型先验；2）运动-几何联合对齐，通过同步的视频和3D输入约束已知与未知视图；3）异步优化策略，解耦高斯属性与变形网络训练以提升保真度。这些技术为多模态可控的4D生成提供了灵活且可扩展的解决方案。
Abstract: Generative diffusion models excel at synthesizing high-quality images, videos, and 3D content under multimodal control. However, arbitrary user-defined modality-to-4D (X-to-4D) generation remains challenging due to the high cost of constructing diverse datasets and the limited scalability of existing methods. This paper presents Align4D, a flexible framework that translates any-modal input into coherent video-3D pairs, using video to guide 4D motion and 3D data to shape 4D geometry. Align4D introduces three key techniques: (1) Object Distance Alignment, which searches Video-Aligned and Multiview-Aligned Object Distances (VAOD/MAOD), respectively, to reconcile 4D renderings with video and the priors of multiview diffusion models; (2) Motion-Geometry Joint Alignment, which constrains known and unknown views through synchronized video and 3D inputs, ensuring consistent 4D generation; and (3) Asynchronous Optimization, which decouples Gaussian attribute and deformation network training to enhance motion and geometry fidelity. We further propose the X4D dataset, which integrates prompt, image, video, and 3D data for benchmarking. Experiments on X4D and Consistent4D demonstrate that Align4D achieves state-of-the-art quality and consistency in X-to-4D generation. Project page: https://miaoqiaowei.github.io/Align4D/.
eess.IV [Back]
[100] Quantum-Inspired Vision: Leveraging Wave-Particle Duality for Low-Illumination Enhancement eess.IV | cs.CV | cs.LG | math.OC | quant-phPDF
Yiquan Gao
TL;DR: 本研究对数据相对论不确定性（DRU）框架进行了理论扩展，提出了一种物理到AI的范式，用于图像增强。该范式将图像建模为概率波函数而非确定性状态，并整合波粒二象性来解释DRU如何利用光的内在物理不确定性，从而提供了一种可解释的AI方法，增强了DRU在缓解光照偏差和保持对数据噪声鲁棒性方面的可解释性。
Details
Motivation: 解决低光照图像增强中光照偏差和数据噪声的问题，通过理论扩展DRU框架，将物理原理（如波粒二象性）整合到AI模型中，以提高方法的可解释性和鲁棒性。
Result: 摘要中未提及具体的定量实验结果或基准测试，但强调该范式提供了严格的解释性AI方法，增强了DRU在低光照增强中的理论解释和鲁棒性。
Insight: 创新点在于将量子物理中的波粒二象性概念引入图像增强，将图像建模为概率波函数，从而提供了一种新颖的物理到AI范式，增强了模型的可解释性，这可能为计算机视觉任务带来新的理论视角。
Abstract: This study provides a theoretical expansion of the recent Data Relativistic Uncertainty (DRU) framework by formalizing a physics-to-AI paradigm for image enhancement. By modeling images as probabilistic wave functions rather than deterministic states, the paradigm explicitly integrates wave-particle duality to illustrate the system flow of how DRU leverages the intrinsic physical uncertainty of light, a dimension requiring further theoretical discussion. Consequently, this paradigm provides a rigorous Explainable AI (XAI) approach that enhances the interpretability of how DRU mitigates illumination bias and maintains robustness against data noise.
[101] Self-Auditing Residual Drifting for Pathology-Preserving Accelerated Knee MRI eess.IV | cs.CV | physics.med-phPDF
Qing Lyu, Jianxu Wang, Mohammad Kawas, Ge Wang, Christopher T. Whitlow
TL;DR: 本文提出了一种名为SA-RDM-DC的自审计残差漂移模型，用于加速膝关节MRI重建。该方法通过训练一个物理条件化的漂移场，从零填充重建向全采样残差校正进行生成，同时预测图像和缺失k空间的残差校正，并强制执行数据一致性。它在保持亚秒级每切片推理速度的同时，在多个加速因子下实现了最高的结构相似性指数（SSIM），并提供了密集误差图和切片级风险评分以评估重建可靠性。
Details
Motivation: 加速磁共振成像（MRI）可以减少采集时间，但从欠采样k空间重建可能会模糊诊断相关的结构或引入全局图像指标无法捕捉的伪影。本文旨在解决这一问题，提出一种能够保留病理细节并评估自身重建可靠性的方法。
Result: 在fastMRI膝关节数据集上，针对4、8和12倍加速因子，SA-RDM-DC在SSIM指标上超越了零填充重建、UNet-image-SENSE、DC-UNet、Score-Diffusion、ELF-Diff、SENSE-VarNet和MoDL等基线方法，达到了最高水平。在病理感知分析中，它有效保留了病变区域的结构保真度，并减少了半月板预测的不稳定性。其自审计评分在fastMRI上能强有力地识别高误差重建，并在SKM-TEA协议偏移下部分转移为选择性审查信号。
Insight: 创新点在于将生成漂移范式适应于加速MRI，通过物理条件化的漂移场联合学习图像和k空间残差校正，并引入频率感知和残差漂移监督以恢复细节。该方法在单次推理中同时输出重建图像、密集误差图和切片级风险评分，实现了重建质量、病理保留、运行时间和病例特定可靠性的联合评估，避免了迭代扩散基线方法的长采样时间。
Abstract: Accelerated magnetic resonance imaging reduces acquisition time, but reconstruction from undersampled k-space can blur diagnostically relevant structures or introduce failures that are not captured by global image metrics. We propose SA-RDM-DC, a Self-Auditing Residual generative Drifting Model with Data Consistency for accelerated knee MRI. The method adapts the newly proposed generative drifting paradigm to accelerated MRI by training a physics-conditioned drift field from the zero-filled reconstruction toward the fully sampled residual correction. It predicts image- and missing-k-space residual corrections, enforces data consistency with acquired k-space, uses frequency-aware and residual drifting supervision to recover fine detail, and produces dense error maps and slice-level risk scores in the same inference pass. We evaluate SA-RDM-DC on multi-coil fastMRI knee data at acceleration factors of 4, 8, and 12, with fastMRI+ pathology annotations for region-level and classifier-based task preservation, and on SKM-TEA for zero-shot and fine-tuned protocol-shift evaluation. Compared with zero-filled reconstruction, UNet-image-SENSE, DC-UNet, Score-Diffusion, ELF-Diff, SENSE-VarNet, and MoDL baselines, SA-RDM-DC achieves the highest SSIM across fastMRI acceleration factors while retaining subsecond per-slice inference and avoiding the long sampling time of iterative diffusion baselines. In pathology-aware analysis, SA-RDM-DC preserves lesion-region structural fidelity and reduces meniscus prediction instability. Its self-auditing scores strongly identify high-error reconstructions on fastMRI and partially transfer as a selective-review signal under SKM-TEA protocol shift. These results support reconstruction evaluation that jointly considers image fidelity, pathology preservation, runtime, and case-specific reliability.
cs.CR [Back]
[102] From Forgeries to Foundation Models: A Systematic Survey of Identity Document Attack and Detection cs.CR | cs.CVPDF
Gourab Das, Pavan Kumar C, Raghavendra Ramachandra
TL;DR: 这篇论文系统性地调查了身份文档伪造攻击与检测方法，指出生成式AI工具已显著提升伪造能力，而检测方法仍受限于与现实脱节的基准测试。论文首次将呈现攻击、数字注入攻击和生成式AI驱动的合成攻击统一到一个威胁模型中，并分析了从启发式规则到基础模型等多种检测方法。
Details
Motivation: 动机在于生成式AI工具使得身份文档伪造变得高保真且易于操作，而现有的检测方法及其基准测试未能充分反映这一新兴威胁，导致攻击面扩大和法证失效模式，因此需要一个统一的威胁模型和评估框架来应对。
Result: 通过对2019-2025年公共数据集的系统审计，揭示了基准条件与操作部署之间存在持续的‘现实鸿沟’；在未见过的合成身份证上进行零样本基准测试显示，即使在安全导向的操作条件下，最强的公开模型APCER值也超过25%，表明跨领域泛化能力存在显著局限。
Insight: 创新点包括首次统一处理多种身份验证威胁模型，并识别出非拉丁文字修复中反复出现的‘脚本依赖生成不稳定性’这一排版失败模式；从客观角度看，论文强调了构建基于法证、保护隐私且法律可问责的身份验证系统的未来方向，对基准测试的批判性分析也具有借鉴意义。
Abstract: Identity document forgery has undergone a fundamental capability shift: generative AI tools now enable high-fidelity document synthesis and field-level manipulation with minimal technical expertise, while detection methods remain constrained by benchmarks that do not reflect this threat. The resulting attack surface spans physical presentation, digital injection, and fully generative synthesis, introducing distinct forensic failure modes that require a unified threat model and evaluation framework. This survey provides, to our knowledge, the first unified treatment of Presentation Attacks, Digital Injection Attacks, and GenAI-driven synthesis within a single identity verification threat model. We trace detection methodologies from rule-based heuristics through forensic localisation, injection-aware pipelines, foundation models, and few-shot frameworks. A systematic audit of public datasets from 2019–2025 exposes a persistent Reality Gap between benchmark conditions and operational deployment. We further analyse large multimodal models for identity document manipulation, identifying Script-Dependent Generative Instability (SDGI) as a recurring typographic failure mode in non-Latin script inpainting. Finally, zero-shot benchmarking on unseen synthesised ID cards shows that even the strongest publicly available models achieve APCER values above 25% under security-oriented operating conditions, highlighting substantial limits in cross-domain generalisation. We conclude by outlining future directions toward forensically grounded, privacy-preserving, and legally accountable identity verification systems.
cs.AI [Back]
[103] Safe and Adaptive Cloud Healing: Verifying LLM-Generated Recovery Plans with a Neural-Symbolic World Model cs.AI | cs.CLPDF
Junyan Tan, Haoran Lin, Siyuan Guo, Yichen Fang, Xinyue Luo
TL;DR: 本文提出了一种名为PASE的新型云故障自愈框架，它将恢复过程重新定义为神经符号程序合成任务。该框架利用大语言模型作为核心计划合成引擎来生成结构化恢复计划，并通过神经符号世界模型进行验证，同时使用基于深度强化学习的元提示优化器来指导LLM的规划过程。
Details
Motivation: 随着基于云的AI系统规模和复杂性不断增加，通过快速故障检测和自适应恢复来确保服务可靠性已成为关键挑战。现有方法通常采用顺序、松散耦合的架构，未能充分利用LLM的生成和推理能力。
Result: 在真实世界的云故障注入数据集上的实验表明，PASE显著优于最先进的方法，将平均系统恢复时间减少了40%以上，并在未知故障场景中提高了故障检测准确率。
Insight: 论文的创新点在于将LLM的推理能力与模型辅助验证和元学习指导相统一，通过紧密的“推理-规划-验证-适应”循环，实现了超越预定义动作空间的动态、上下文感知的恢复策略生成。
Abstract: As the scale and complexity of cloud-based AI systems continue to escalate, ensuring service reliability through rapid fault detection and adaptive recovery has become a critical challenge. While existing approaches integrate Large Language Models (LLMs) for semantic understanding and Deep Reinforcement Learning (DRL) for policy optimization, they often rely on sequential, loosely coupled architectures that underutilize the generative and reasoning capabilities of LLMs. In this paper, we propose a paradigm shift with PASE, a Planning-Aware Semantic self-healing engine, a novel fault self-healing framework that reconceptualizes recovery as a neuro-symbolic program synthesis task. PASE employs an LLM as a core Plan Synthesis Engine to generate structured recovery plans from a library of semantic primitives. A Neural-Symbolic World Model verifies plan feasibility through simulation, while a Meta-Prompt Optimizer, trained via DRL, learns to generate optimal prompts that guide the LLM’s planning process. This tight reason-plan-verify-adapt loop enables dynamic, context-aware recovery strategy generation beyond predefined action spaces. Experiments on a real-world cloud fault injection dataset demonstrate that PASE significantly outperforms state-of-the-art methods, reducing average system recovery time by over 40% and improving fault detection accuracy in unknown fault scenarios. Our framework advances autonomous system management by unifying LLM-based reasoning with model-assisted verification and meta-learned guidance.
[104] Spec-AUF: Accept-Until-Fail Training under Train-Inference Misalignment for Masked Block Drafters cs.AI | cs.CLPDF
Tianjian Yang, Meng Li
TL;DR: 本文提出了一种名为Spec-AUF（Accept-Until-Fail）的训练方法，用于解决推测解码中块状草稿器（block drafter）训练与推理目标不一致的问题。该方法通过修改交叉熵损失的支持范围，仅在草稿器预测的第一个失败位置之前提供监督，从而更紧密地模拟推理时仅接受最长正确前缀的行为。实验表明，该方法在多个基准测试上提升了草稿器的平均输出长度，且无需改变推理流程或增加辅助目标。
Details
Motivation: 动机在于解决块状草稿器（如DLM风格）训练与推理之间的错位问题：训练时使用全块交叉熵损失监督所有位置，而推理时一旦出现首个预测错误，后续所有令牌都会被丢弃。这种不一致性可能导致训练效率低下。
Result: 在Qwen3-8B模型上的实验结果显示，该方法将DFlash草稿器的平均发射长度τ在六个基准测试上从2.40提升至2.61，且在每一个基准上都有增益；该方法也能迁移到Domino的双分支头部结构上（从2.56提升至2.68）。
Insight: 核心创新点在于提出了一种简单、解耦的训练目标修改（AUF），它通过截断交叉熵损失的支持范围来近似模拟推理时的前缀敏感监督，无需额外目标、验证器展开或改变推理流程。一个有趣的发现是，仅使用指数位置衰减的基线方法虽然提高了令牌准确率，但解码效果更差，而AUF方法在截断支持后，这种位置衰减加权变得无效，凸显了损失支持范围调整的关键作用。
Abstract: Speculative decoding accelerates autoregressive generation by drafting a block of tokens that the target model verifies left-to-right, committing only the longest accepted prefix. Block (DLM-style) drafters predict the whole block in parallel, which is fast but trained with a full-block cross-entropy that supervises every position against the gold continuation – even though inference discards every token after the first rejection. Recent acceptance-aware objectives patch this by reweighting the full-block loss; we instead use teacher-forced learning as a motivation for how supervision should concentrate on the accepted prefix. A mask-only block drafter has no input-side channel for gold-prefix conditioning, so AUF approximates that prefix-sensitive supervision on the loss side by keeping the cross-entropy support only through the drafter’s first predicted failure. AUF is a single, detached change to the CE support – no auxiliary objective, no verifier rollouts, and no change to the inference pipeline or the exactness contract. Within fixed drafter backbones and serving settings on Qwen3-8B, AUF raises the DFlash drafter’s average emitted length $τ$, averaged over six benchmarks, from 2.40 to 2.61, with a gain on every benchmark, and transfers to Domino’s two-branch head (2.56 to 2.68). Two findings sharpen the picture: the decay-only baseline reaches higher token accuracy on the shared block mask yet decodes worse, and on DFlash, once AUF truncates the support, the standard exponential position-decay weighting becomes empirically inert.
[105] SkillCoach: Self-Evolving Rubrics for Evaluating and Enhancing Agentic Skill-Use cs.AI | cs.CLPDF
Jiayin Zhu, Kelong Mao, Yudong Guo, Dengbo He, Sulong Xu
TL;DR: 论文提出SkillCoach框架，通过自演化评分标准来评估和增强LLM代理的技能使用。该框架从实际运行轨迹中提取基于技能的过程评分标准，从技能选择、执行、组合和反思四个维度评估代理行为，并将外部验证器作为独立结果信号，从而区分过程质量与偶然任务成功。演化后的评分标准进一步作为过程监督，用于筛选高质量训练轨迹。
Details
Motivation: 现有技能库中技能重叠导致代理可靠使用技能困难，仅依赖最终验证成功率过于粗糙，无法评估技能选择错误、步骤跳过、工作流组合错误或最终检查遗漏等过程问题。
Result: 实验表明，演化评分标准显著提升评估质量，暴露了被最终准确率掩盖的失败案例，并为增强代理技能使用提供了比仅基于结果过滤更强的监督信号。
Insight: 创新点在于将过程评估与结果验证解耦，构建多维度的技能使用评分体系，并通过自演化机制将评估标准转化为过程监督信号，为代理技能优化提供细粒度反馈。
Abstract: Skills are becoming a reusable operational layer for LLM agents, encoding SOPs, domain rules, tool workflows, scripts, and validation routines. In realistic skill repositories, overlapping skills make reliable skill-use difficult. Final verifier success is too coarse for both evaluation and training, since an agent may pass through trial and error while selecting distractor skills, skipping required steps, composing workflows incorrectly or omitting final checks. We introduce SkillCoach, a self-evolving rubric framework for evaluating and enhancing agentic skill-use. SkillCoach derives skill-grounded process rubrics from real rollouts and evaluates trajectories along four dimensions: skill selection, skill following, skill composition, and skill-grounded reflection. It keeps the external verifier as a separate outcome signal, allowing process quality to be distinguished from accidental task success. The evolved rubrics further serve as process supervision for selecting high-quality training trajectories. Experiments show that evolved rubrics substantially improve evaluation quality, expose failures hidden by final accuracy, and provide stronger supervision signals than outcome-only filtering for enhancing agentic skill-use.
[106] PACE: A Proxy for Agentic Capability Evaluation cs.AI | cs.CLPDF
Yueqi Song, Lintang Sutawika, Jiarui Liu, Lindia Tjuatja, Jiayi Geng
TL;DR: 本文提出了PACE框架，通过从现有的非智能体基准测试中精心挑选一个小子集的原子评估实例，构建代理基准来预测模型在昂贵且耗时的智能体基准（如SWE-Bench和GAIA）上的性能。PACE-Bench作为具体实现，能以极低成本（不到完整评估成本的1%）准确预测智能体得分。
Details
Motivation: 当前在SWE-Bench和GAIA等智能体基准上评估LLM智能体成本高昂、耗时且基础设施复杂，而测试原子能力（如推理、代码生成）的非智能体基准则快速廉价。本文旨在探究是否能用少量精选的原子评估实例性能来准确预测昂贵的智能体基准性能。
Result: 在14个模型、4个智能体基准和19个非智能体基准上的实验表明，PACE-Bench在留一法交叉验证（LOOCV）下，预测智能体得分的平均绝对误差（MAE）低于4%，斯皮尔曼相关系数高于0.80，模型对排序准确率约85%，且成本不到完整智能体评估的1%。
Insight: 创新点在于结合目标相关性的局部选择和全局信息性的全局选择两种互补策略，从原子能力池中精选实例构建代理基准，实现了低成本、高精度的智能体性能预测，为模型开发、选择和路由提供了实用工具，并揭示了各智能体基准所需的独特技能。
Abstract: Evaluating LLM agents on benchmarks like SWE-Bench and GAIA can be expensive, time-consuming, and requires complex infrastructure. A single evaluation can cost thousands of dollars and take days to complete. In contrast, non-agentic LLM benchmarks that test individual capabilities (e.g., reasoning, code generation) are fast and cheap to run. In this paper, we investigate whether performance on expensive agentic benchmarks can be accurately predicted by the performance on a small, carefully selected subset of atomic evaluation instances. We introduce PACE, a framework that constructs proxy benchmarks by selecting instances from existing non-agentic evaluations whose aggregate scores most reliably predict model performances on agentic benchmarks. Given a pool of candidate instances spanning atomic capabilities, PACE fits a regression that maps a model’s scores on a compact subset of source instances to its score on the target agentic benchmark. The subset itself is curated by combining two complementary instance-selection strategies, target-relevance local selection and globally informative global selection. We apply PACE to the 4 target agentic benchmarks in this paper, which yields PACE-Bench, the concrete proxy benchmark that we evaluate in the paper. Experiments across 14 models, 4 agentic benchmarks, and 19 non-agentic benchmarks show that PACE-Bench predicts agentic scores with leave-one-out cross-validation (LOOCV) mean absolute error (MAE) under 4%, Spearman correlation above 0.80, and pairwise model-ranking accuracy around 85%, all at much less than 1% of the full agentic evaluation cost. We further analyze the selected proxy instances, revealing which skills each agentic benchmark uniquely demands. PACE enables practitioners to obtain reliable estimates of agentic performance during model development, selection, and routing, without the overhead of full agent evaluation.
[107] Multimodal Knowledge Edit-Scoped Generalization for Online Recursive MLLM Editing cs.AI | cs.CL | cs.CVPDF
Siyuan Li, Youyuan Zhang, Ruitong Liu, Junxi Wang, Jing Li
TL;DR: 本文提出了一种用于在线多模态大语言模型（MLLM）知识编辑的新框架ScopeEdit，旨在将每个编辑的语义传播范围控制在特定边界内，以解决现有方法在跨模态泛化和无关输入泄漏方面的问题。该方法通过模态局部吸收分支和证据门控共享泛化分支分解更新，在正交低秩空间中进行范围分离的写入，实现了恒定开销的在线递归编辑。
Details
Motivation: 现有的在线多模态知识编辑方法主要关注编辑的可靠性和长期稳定性，但很少控制每个编辑的语义边界。初步分析发现，实例级的编辑成功既不能保证向有效的跨模态变体迁移，也无法防止向无关输入泄漏，且编辑相关的跨模态响应集中在更深层的语义层中。因此，需要重新定义在线MLLM编辑，从单纯纠正实例转向控制每个编辑的传播边界。
Result: 在多种基准测试、长时程编辑流、不同MLLM主干网络、真实世界VLKEB场景以及复杂视觉语言架构上的广泛实验表明，ScopeEdit在保持编辑可靠性、稳定性和在线效率的同时，持续改善了在范围（in-scope）内跨模态迁移与范围外（out-of-scope）局部性之间的权衡。
Insight: 论文的创新点在于提出了“编辑范围泛化”这一新问题表述，并将在线编辑重构为控制编辑传播边界。ScopeEdit方法的核心是范围感知的分解式更新机制：局部分支稳定吸收编辑，而共享分支仅在视觉和文本证据充分对齐时才允许跨模态传播。其实用创新在于利用正交低秩空间进行范围分离的写入，并通过Sherman-Morrison递归维护分支预条件子，实现了恒定开销，这对在线递归编辑的部署至关重要。
Abstract: Online multimodal knowledge editing requires injecting a continual stream of visual-textual corrections into multimodal large language models (MLLMs) with bounded overhead and minimal disruption to unrelated behaviors. Existing editors mainly emphasize edit reliability and long-horizon stability, but rarely control the semantic boundary of each edit. Our pilot analyses of post-edit behaviors and internal neuronal activities reveal a scope gap behind reliable edits: instance-level success neither guarantees transfer to valid cross-modal variants nor prevents leakage to unrelated inputs, while edit-related cross-modal responses concentrate in deeper semantic layers. Therefore, we formulate Edit-Scoped Generalization, reframing online MLLM editing from merely correcting an instance to controlling the propagation boundary of each edit. To this end, we propose ScopeEdit, a scope-aware online editor that decomposes each update into a modality-local absorption branch and an evidence-gated shared generalization branch. The local branch supports stable edit absorption, whereas the shared branch enables cross-modal propagation only when visual and textual evidence are sufficiently aligned. Both branches perform scope-separated write geometries in orthogonal low-rank spaces and maintain branch-wise preconditioners via Sherman–Morrison recursions, yielding constant per-edit overhead. Extensive experiments across diverse benchmarks, long-horizon edit streams, MLLM backbones, real-world VLKEB scenarios, and complex vision-language architectures show that ScopeEdit consistently improves the trade-off between in-scope cross-modal transfer and out-of-scope locality, while preserving edit reliability, stability and online efficiency. Our code is available at https://github.com/lab-klc/ScopeEdit.
[108] Online Safety Monitoring for LLMs cs.AI | cs.CL | cs.LG | stat.AP | stat.MLPDF
Mona Schirmer, Metod Jazbec, Alexander Timans, Christian Naesseth, Maja Waldron
TL;DR: 该论文研究了一种用于大型语言模型的简单在线安全监控方法。该方法通过对外部模型提供的验证信号进行阈值化处理来触发警报，并使用风险控制来校准阈值。在数学推理和红队测试数据集上的实验表明，这种简单设计与基于序列假设检验的更先进监控器性能相当。
Details
Motivation: 尽管经过对齐训练，大型语言模型在部署时仍可能生成不安全的输出。因此，在线监控输出并在无法保证安全时发出警报至关重要。
Result: 在数学推理和红队测试数据集上的实验表明，这种基于风险控制校准阈值的简单监控器，与基于序列假设检验的更先进监控器性能相当。
Insight: 论文的创新点在于提出了一种简单、实时且基于风险控制进行阈值校准的监控框架。从客观角度看，其核心是将外部验证信号与统计风险控制理论相结合，提供了一种计算高效且理论上有保障的监控方案，而非依赖于复杂的序列决策模型。
Abstract: Despite alignment training, LLMs remain prone to generating unsafe outputs at deployment time. Monitoring outputs online and raising an alarm when safety can no longer be assumed is therefore critical. We study a simple real-time monitor that turns a verifier signal from an external model into an alarm decision by thresholding, with the threshold calibrated via risk control. In experiments on mathematical reasoning and red teaming datasets, we show that this simple design is competitive with more advanced monitors based on sequential hypothesis testing.
eess.AS [Back]
[109] Self-Supervised Test-Time Tuning for Packet Loss Concealment eess.AS | cs.CLPDF
Yehoshua Dissen, Joseph Keshet
TL;DR: 本文提出了一种名为TTT-PLC的自监督测试时调优框架，用于改进现有的丢包隐藏（PLC）模型。该方法利用接收到的音频包本身，通过合成掩码创建自监督信号来微调模型，无需额外数据或架构改动。研究在非因果和因果两种部署场景下验证了其有效性。
Details
Motivation: 传统PLC模型在部署时参数固定，忽略了每次通话或录音中已接收包所包含的信号特定信息，无法动态适应具体信号。
Result: 在两个公开PLC骨干网络（FRN和PARCnet）上的实验表明，该方法能有效利用有损信号中仍可观测的部分作为训练信号，提升同一信号上的隐藏性能。
Insight: 核心创新在于将自监督学习和测试时适应引入PLC任务，证明了预训练模型在推理时无需保持固定，可利用当前信号本身进行在线优化，且无需干净参考信号或外部数据。
Abstract: Packet loss concealment (PLC) reconstructs audio packets that are missing at the receiver, usually with a trained model whose parameters remain fixed at deployment time. This treats the PLC model as static, even though each call or recording exposes signal-specific information through the packets that did arrive. We present TTT-PLC, a self-supervised test-time tuning framework that adapts existing PLC models using only those received packets. The method creates supervision by synthetically masking portions of the available signal, training the model to conceal them with its native PLC objective, and then using the adapted model to reconstruct the true packet losses. No clean reference signal, external adaptation data, or architectural modification is required. We study TTT-PLC in two deployment settings. In the non-causal setting, the received file is available before reconstruction, allowing repeated self-supervised adaptation passes and providing a per-file adaptation ceiling. In the causal setting, audio is streamed without revising emitted samples; adaptation is performed only on completed past blocks, and updated parameters affect only future audio. We instantiate the framework on two public PLC backbones, FRN, a recurrent full-band speech PLC model, and PARCnet, a hybrid autoregressive-neural model for networked music. Across these settings, the results show that pretrained PLC systems do not need to be treated as fixed at inference time, the still-observed portions of a lossy signal can provide an effective training signal for improving concealment on that same signal.
cs.LG [Back]
[110] Black-Box Inference of LLM Architectural Properties with Restrictive API Access cs.LG | cs.AI | cs.CL | cs.CRPDF
Christopher Ellis, Shreyas Chaudhari, Mei-Yu Wang, Leighton Barnes, Giulia Fanti
TL;DR: 本文提出了一种名为NightVision的攻击方法，能够在当前商业LLM提供商仅提供单token对数概率的严格API访问限制下，推断出LLM的关键架构参数，包括隐藏层维度、深度和参数量。该方法通过新颖的’公共集合提示’技术和首次token生成时间测量来实现推断，并在32个开源LLM上进行了实证评估。
Details
Motivation: 商业LLM提供商通常不公开模型架构细节，且为应对早期攻击，已进一步限制API访问（仅提供单token对数概率，取消对数偏置功能）。本文旨在探究即使在当前严格的API限制下，是否仍能恢复模型的架构参数。
Result: 在32个开源LLM上的实验表明，NightVision能恢复隐藏层维度，平均相对误差为23%（在MoE模型上为9%）；对于参数量超过30亿的模型，深度和参数量的恢复平均相对误差在53%以内。
Insight: 创新点在于提出了’公共集合提示’技术，通过多个提示暴露同一输出token集合的对数概率，并对其进行谱分析以推断隐藏维度；同时结合TTFT测量进行联合推断。客观来看，该方法揭示了当前LLM API的限制仍不足以完全隐藏模型架构，为模型安全和隐私评估提供了新视角。
Abstract: In practice, most commercial LLM providers do not publicly release details of underlying LLM architectures. However, prior work has shown that given limited API access to an LLM (namely, top-$k$ logits and/or a logit bias function), one can recover certain architectural details of an LLM, such as the hidden dimension of the feed-forward network. Perhaps in response to these results, most commercial LLM providers have restricted their APIs to expose only the single logit for each decoded token, and they no longer give users the ability to bias logits. We show that even under current restrictive APIs, several architectural parameters are still recoverable. We present NightVision, an attack that uses restrictive black-box API access to estimate the hidden dimension, depth, and parameter count of an LLM. Algorithmically, NightVision relies on a novel common set prompting technique in which multiple prompts expose log probabilities for the same set of output tokens; a spectral analysis of these results is used to infer hidden dimension. NightVision additionally uses end-to-end time to first token (TTFT) measurements and the estimated hidden dimension to estimate depth and parameter count. We empirically evaluate NightVision on 32 open-source LLMs, recovering hidden dimension to within 23% average relative error across all models (9% on MoE models), and depth and parameter count to within 53% for models exceeding three billion parameters. We run extensive ablations to demonstrate how these accuracies scale with token budget and model properties. Overall, our results suggest that current LLM APIs are not sufficiently restricted to fully obfuscate the architectural details of their underlying models.
[111] Denser $\neq$ Better: Limits of On-Policy Self-Distillation for Continual Post-Training cs.LG | cs.CLPDF
Meng Wang, Haohan Zhao, Wenzhuo Liu, Lu Yang, Geng Liu
TL;DR: 本文通过自蒸馏策略优化（SDPO）重新评估了持续后训练中策略自蒸馏的有效性。研究发现，当教师信号稳定且对齐良好时，SDPO能加速领域内专业化，但在分布外场景泛化能力差，且在持续训练中会导致更强的遗忘甚至崩溃，而策略强化学习方法（如GRPO）则能更保守地适应并更好地保留先前能力。
Details
Motivation: 研究动机是探讨持续后训练中策略自蒸馏（特别是SDPO）是否真能有效缓解遗忘问题，并分析其在实际应用中的局限性。
Result: 实验表明，SDPO在教师信号稳定时加速专业化，但在持续训练中导致更强遗忘和崩溃，而GRPO等方法表现更优；分析揭示密集自蒸馏会引发参数和响应空间的较大漂移，并放大高频格式伪影。
Insight: 创新点在于指出密集自蒸馏并非持续后训练的默认稳定器，策略数据本身不足以保证持续学习，需谨慎使用；客观分析认为，该研究揭示了自蒸馏在动态环境中的脆弱性，为设计更鲁棒的持续学习算法提供了重要见解。
Abstract: Continual post-training enables foundation models to acquire new knowledge while preserving existing capabilities. Recent work suggests that on-policy learning can mitigate forgetting, with on-policy self-distillation emerging as a particularly attractive approach. In this work, we revisit this optimistic view through self-distillation policy optimization (SDPO). Our experiments show that SDPO can accelerate in-domain specialization when teacher signals are stable and well aligned, but it struggles to generalize to out-of-distribution scenarios. In continual post-training, SDPO exhibits stronger forgetting and can even collapse, whereas on-policy reinforcement learning methods such as GRPO adapt more conservatively and better preserve prior capabilities. Further analyses reveal that denser self-distillation induces larger drift in both parameter space and response space, and can amplify high-frequency formatting artifacts through a self-reinforcing teacher–student loop. These findings suggest that on-policy data alone is insufficient for continual learning. Dense self-distillation can accelerate specialization when teacher targets are stable and token-level supervision is reliable, but it should not be treated as a default stabilizer for continual post-training. Our code is available at https://github.com/Moenupa/SDPO-CL.
[112] Bayesian Sparse Low-Rank Adaptation for Large Language Model Uncertainty Estimation cs.LG | cs.CLPDF
Jijie Zhang, Zhe Ren, Quan Zhang, Dandan Guo
TL;DR: 本文提出了一种名为DALorRA的变分贝叶斯稀疏框架，用于解决大语言模型（LLM）在任务特定微调中存在的过度自信问题。该方法将不确定性量化从密集参数空间转移到低秩适应（LoRA）的轻量级秩级别，通过对秩维度施加随机掩码，实现了训练时的贝叶斯正则化和推理时的集成式校准。
Details
Motivation: 大语言模型虽然展现出卓越的推理能力，但其任务特定微调过程普遍存在过度自信问题，这严重阻碍了其可信部署。
Result: 大量实验表明，DALorRA能够在不损害LLM推理准确性的前提下，实现出色的模型校准效果。
Insight: 核心创新在于认识到LoRA本质上是聚合了多个可能提供冗余模型容量的秩为一的分量，因此通过随机掩码秩维度，在训练时进行贝叶斯正则化，在推理时实现类似集成的校准，从而将不确定性量化范式从密集参数空间转移到轻量级的LoRA秩级别。
Abstract: Large language models (LLMs) exhibit remarkable reasoning capabilities, but their task-specific fine-tuning is notoriously plagued by overconfidence, severely hindering trustworthy deployment. We propose Data-Adaptive Lower-Rank Adaptation (DALorRA), a simple and effective variational Bayesian sparse framework that shifts the paradigm of uncertainty quantification from the dense parameter space to the lightweight rank level of low-rank adaptation (LoRA). With the insight that LoRA essentially aggregates multiple rank-one components that may provide superfluous model capacity, DALorRA imposes stochastic masking on rank dimensions, enabling Bayesian regularization of model capacity during training and ensemble-like calibration during inference. Extensive experiments demonstrate DALorRA’s excellent calibration of LLMs without compromising reasoning accuracy.
[113] Multi-modal Rail Crossing Safety Analysis cs.LG | cs.AI | cs.CVPDF
Paimon Goulart, Chansong Lim, Nícolas Roque dos Santos, Yue Dong, Sheldon Peterson
TL;DR: 这篇论文提出了一种多模态铁路道口安全分析系统，旨在通过结合视觉图像和结构化事故历史数据，自动评估铁路道口的安全风险。论文构建了一个概念验证流程，能够对道口进行高风险/低风险分类并估算联邦铁路管理局（FRA）风格的安全评分。
Details
Motivation: 论文旨在解决如何利用视觉图像和结构化数据（如官方事故报告）来稳健地评估铁路道口的安全水平，目标是构建一个与专家意见和FRA安全评分标准一致的多模态AI评估系统。
Result: 在提出的系统中，使用基于路由的微调紧凑视觉语言模型（VLM）流程，对高风险和低风险道口的分类宏F1分数达到0.757，估算FRA安全分数的RMSE为0.078，相关性为0.492，并产生了与领域专家评估一致的定性结果。
Insight: 论文的创新点在于将多模态数据（图像与结构化报告）融合用于铁路安全评估，并探索了从数据准备到不同学习范式的全流程关键挑战。客观来看，其提出的基于路由的微调紧凑VLM流程，为资源受限环境下的多模态安全分析提供了一个有效的概念验证方案。
Abstract: Given one or more images of a railway crossing, can we leverage visual cues that allow us to robustly estimate how safe it is? Can we improve our ability to do so by introducing structured data (such as official accident reports) about the accident history of that crossing into our models? In this work, we explore how to best answer those questions towards building an AI system that can ingest multi-modal data for railway crossings and provide safety assessment and scores that align with expert opinion and with safety scoring used by the Federal Railroad Administration (FRA). To that end, we propose a proof-of-concept pipeline that delivers on that goal, while at the same time exploring and tackling a number of critical research challenges that pertain to different parts of the pipeline, from data preparation to different learning paradigms that can allow us to realize such a system. Indicatively, our proposed system identifies HIGH-RISK and LOW-RISK crossings with a macro F1 score of 0.757 and estimates FRA-based safety scores with an RMSE of 0.078 and correlation of 0.492 using a routed fine-tuned compact VLM pipeline, while producing qualitative results that align with domain-expert assessment.
[114] Optimizing Visual Generative Models via Distribution-wise Rewards cs.LG | cs.CVPDF
Ruihang Li, Mengde Xu, Shuyang Gu, Leigang Qu, Fuli Feng
TL;DR: 本文提出了一种基于分布级奖励的视觉生成模型优化框架，以解决传统强化学习中样本级奖励导致的奖励黑客攻击、图像多样性下降和视觉异常问题。该方法通过评估样本的数据分布而非单个样本，缓解了模式崩溃，并引入了子集替换策略来高效计算奖励。此外，还应用强化学习优化后验模型合并系数，以减少训练-推理不一致性。实验表明，该方法在多个基准模型上显著提升了FID-50K分数，并改善了感知质量与样本多样性。
Details
Motivation: 传统强化学习在视觉生成中采用样本级奖励函数，常导致奖励黑客攻击，损害图像多样性和引入视觉异常，因此需要一种更有效的奖励机制来更好地对齐真实数据分布。
Result: 在SiT模型上，FID-50K从8.30提升至5.77；在EDM2模型上，从3.74提升至3.52，实现了显著的定量改进。定性评估也证实了该方法在保持样本多样性的同时增强了感知质量。
Insight: 创新点包括引入分布级奖励来缓解模式崩溃，提出子集替换策略以降低计算成本，以及应用强化学习优化后验模型合并系数来减少训练-推理不一致性，这些方法可借鉴于提升生成模型的稳定性和多样性。
Abstract: Conventional reinforcement learning strategies for visual generation typically employ sample-wise reward functions, yet this practice frequently results in reward hacking that degrades image diversity and introduces visual anomalies. To address these limitations, we present a novel framework that finetunes generative models using distribution-wise rewards, ensuring better alignment with real-world data distributions. Unlike rewards that evaluate samples individually, distribution-wise reward accounts for the data distribution of the samples, mitigating the mode collapse problem that occurs when all samples optimize towards the same direction independently. To overcome the prohibitive computational cost of estimating these rewards, we introduce a subset-replace strategy that efficiently provides reward signals by updating only a small subset of a generated reference set. Additionally, we apply RL to optimize post-hoc model merging coefficients, potentially mitigating the train-inference inconsistency caused by introducing stochastic differential equation (SDE) in regular RL practices. Extensive experiments show our approach significantly improves FID-50K across various base models, from 8.30 to 5.77 for SiT and from 3.74 to 3.52 for EDM2. Qualitative evaluation also confirms that our method enhances perceptual quality while preserving sample diversity.
cs.RO [Back]
[115] Real-Time Visual Intelligence on Low-Cost UAVs: A Modular Approach for Tracking, Scanning, and Navigation cs.RO | cs.CVPDF
Andrei-Marian Ungureanu, Stelian Spînu
TL;DR: 本文提出了一种基于低成本DJI Tello无人机的集成智能系统，采用模块化架构实现了人脸检测、人脸识别和单目深度估计三大核心AI功能。系统通过Web界面进行控制和实时监控，并由运行轻量级神经网络的Python服务器处理视觉数据，在现实场景中实现了人员跟踪、室内扫描和自主循线等功能。
Details
Motivation: 旨在开发一个低成本、易获取的智能无人机个人助理系统，以验证先进AI技术在资源受限的嵌入式硬件上实时运行的可行性，区别于现有商业解决方案。
Result: 系统在真实世界条件下（如人员跟踪、室内扫描、自主循线）表现出鲁棒性能，验证了在受限硬件上部署实时AI的可行性，为军事、救援和监视等任务的自主无人机研究奠定了基础。
Insight: 创新点在于采用模块化、开源技术栈整合多种轻量级AI功能于低成本消费级无人机平台，强调了系统的可访问性和硬件成本效益，为资源受限场景下的实时视觉智能应用提供了实用范例。
Abstract: Autonomous drones are rapidly transforming modern warfare and civil applications alike. This paper presents the development of an integrated intelligent drone system designed to serve as a personal assistant. Leveraging the DJI Tello drone platform, we implemented a modular architecture that integrates three core artificial intelligence functionalities: facial detection, facial recognition, and depth estimation from monocular vision. A web-based interface enables seamless drone control and real-time video monitoring, while a Python-based server processes visual data and executes inference pipelines using lightweight neural models optimized for embedded systems. Unlike existing commercial solutions, this system emphasizes accessibility, low-cost hardware, and open-source technologies. The system demonstrates robust performance in real-world conditions, including person tracking, indoor scanning, and autonomous line following using virtual sensors. This project validates the applicability of advanced AI techniques in real-time robotic systems and illustrates the feasibility of deploying them on constrained hardware, providing a foundation for future research in autonomous UAVs for military, rescue, and surveillance missions.
[116] LIME: Learning Intent-aware Camera Motion from Egocentric Video cs.RO | cs.CV | cs.LGPDF
Boyang Sun, Jiajie Li, Yung-Hsu Yang, Chenyangguang Zhang, Tim Engelbracht
TL;DR: 本文提出了LIME模型，用于根据自然语言意图和当前RGB观测生成目标相机位姿。该模型从第一人称视频中挖掘多意图相机运动监督，结合自回归观测增益输出和连续流匹配位姿头，能够预测下一视图应揭示的内容并表征多假设目标视图。
Details
Motivation: 解决自主机器人根据语言指令主动调整相机位姿的问题，将语言条件相机运动生成作为一类独立动作进行探索，以支持意图感知的主动感知。
Result: 在实验和下游机器人任务中，LIME能够从被动的人类视频中学习主动选择相机位姿，将普通的第一人称录像转化为意图感知主动感知的监督信号。
Insight: 创新点在于将语言条件相机运动生成任务形式化，并设计结合自回归观测增益和连续流匹配的模型架构，能够处理不同语义粒度的意图，实现从被动视频数据中学习主动感知策略。
Abstract: Autonomous robots often need to move their camera before they can act: to inspect an object, reveal an occluded region, or obtain a view that responds to a user’s intent. While vision-language navigation translates instructions to base motion and vision-language-action policies map instructions to manipulation actions, language-conditioned camera motion remains comparatively underexplored as a first-class action. We formulate language-conditioned camera motion generation: given a current RGB observation and a free-form natural-language intent, predict a relative target camera pose for the next observation. This task is inherently non-trivial: viewpoint changes are driven by latent perceptual intentions, and a valid motion may operate at different semantic granularity, from entering a room to looking around a corner, inspecting a visible object, or revealing an occluded detail. To model this structure, we mine multi-intention camera-motion supervision from egocentric video, pairing plausible intents and observation-gain descriptions with relative SE(3) target poses. We propose LIME, a vision-language camera-motion generator that combines an auto-regressive observation-gain output with a continuous flow-matching pose head. This design lets the model jointly predict what the next view should reveal while representing multi-hypothesis target views. Across experiments and downstream robotic tasks, we show that LIME can learn to actively choose camera poses from passive human video, turning ordinary egocentric recordings into supervision for intent-aware active perception.
[117] The Moving Eye: Enhancing VLA Spatial Generalization via Hybrid Dynamic Data Collection cs.RO | cs.CVPDF
Jincheng Tang, Yilong Zhu, Zhengyuan Xie, Jiang-Jiang Liu, Jiaxing Zhang
TL;DR: 本文针对视觉-语言-动作模型在机器人操作任务中空间泛化能力脆弱的问题，提出了一种以数据为中心的解决方案。通过使用双机械臂设置，其中一个作为移动环境摄像头，系统评估了固定、多固定和移动视角三种数据分布模式。研究发现，结合连续相机运动和多样化静态视角的混合策略能最有效地减少虚假相关性，从而显著提升多种VLA模型在未见过的相机位姿和物体配置下的泛化能力。
Details
Motivation: 当前VLA模型在机器人操作中展现出泛化潜力，但其空间泛化能力仍然脆弱。模型容易陷入捷径学习的陷阱，即依赖虚假相关性（如物体间或相机与机器人基座间的固定相对位姿），而非学习真实的空间关系。
Result: 实验表明，提出的混合数据策略（结合移动视角和多样静态视角）能有效缓解虚假相关性，使模型在未见过的相机位姿和物体配置上实现泛化，而仅增加静态视角则失败。该策略显著提升了所有评估模型（包括ACT、Diffusion以及Pi0和Gr00t等VLA模型）的性能。
Insight: 论文的核心创新点在于揭示了VLA模型空间泛化脆弱的普遍性（跨多种架构），并提出了一种数据中心的混合动态数据收集策略来应对。从客观角度看，利用移动摄像头主动改变观察视角以打破数据中的虚假模式，是一种有效且可借鉴的增强模型鲁棒性的方法。
Abstract: Vision-Language-Action (VLA) models have shown remarkable promise in generalized robotic manipulation. However, their spatial generalization remains fragile. We argue that simply increasing the number of viewpoints is insufficient. Models often fall into the trap of Shortcut Learning, latching onto spurious correlations (e.g., fixed relative poses between objects or between the camera and robot base) rather than learning true spatial relationships. In this work, we propose a data-centric solution to enhance VLA spatial generalization. We utilize a dual-arm setup where one arm performs manipulation while the other serves as a mobile environmental camera. We systematically evaluate three data distribution patterns: Fixed, Multi-Fixed, and Moving Views. Our findings reveal that a hybrid strategy, combining continuous camera motion with diverse static viewpoints, yields the best performance by substantially reducing spurious correlations while maintaining training stability. Our experiments demonstrate that this strategy mitigates spurious correlations, enabling VLAs to generalize to unseen camera poses and object configurations where simply adding more static viewpoints fails. Crucially, we reveal that the susceptibility to shortcut learning and the struggle with spatial generalization are universal characteristics shared across diverse architectures. Consequently, all evaluated models (ACT, Diffusion, and VLA models including Pi0 and Gr00t) benefit significantly from our mixed data strategy.
[118] Embodied.cpp: A Portable Inference Runtime of Embodied AI Models on Heterogeneous Robots cs.RO | cs.CV | cs.OSPDF
Ling Xu, Chuyu Han, Borui Li, Hao Wu, Shiqi Jiang
TL;DR: 本文提出了Embodied.cpp，一个用于具身AI模型（包括视觉-语言-动作模型和世界-动作模型）的便携式C++推理运行时。它旨在解决现有推理运行时在异构机器人边缘设备上部署时存在的碎片化、延迟和接口不匹配问题，通过统一的架构实现了跨设备、机器人和模拟器的高效部署。
Details
Motivation: 现有具身AI模型的部署依赖于模型特定的Python栈、不同的后端假设和机器人端胶水代码，导致在异构边缘设备上部署碎片化且低效。现有推理运行时主要为请求-响应服务设计，无法满足具身部署所需的运行时契约，如闭环控制内的多速率执行、异构硬件上的低延迟单批次推理以及超越固定令牌I/O的可扩展接口。
Result: 在HY-VLA和pi0.5两个VLA模型上的部署实现了成功的闭环执行，任务成功率分别为100.0%和91.0%。在一个初步的WAM基准测试（使用LingBot-VA Transformer块）中，块内存从312.2 MiB减少到88.1 MiB。结果表明，Embodied.cpp在保持高精度的同时提高了部署效率。
Insight: 创新点在于通过架构分析提取了具身模型的共享执行路径，并将其组织为五个层次化模块（输入适配器、序列构建器、骨干执行、头部插件、部署适配器），从而实现了模块化的多速率执行、延迟优先的融合推理以及可扩展的算子和I/O支持。这提供了一个统一的后端抽象，简化了跨异构环境的部署流程。
Abstract: Embodied AI models now span vision-language-action (VLA) models and world-action models (WAMs), but practical deployment remains fragmented across model-specific Python stacks, backend assumptions, and robot-side glue code, especially on heterogeneous edge devices. Existing inference runtimes are designed mainly for request-response serving and therefore do not satisfy the runtime contract of embodied deployment: multi-rate execution inside closed-loop control, latency-first batch-1 inference on heterogeneous hardware, and extensible embodied interfaces beyond fixed token I/O. We present Embodied.cpp, a portable C++ inference runtime for embodied models. Based on an architectural analysis of representative VLA models and WAMs, Embodied.cpp captures a shared execution path and organizes it into five layers: input adapters, sequence builders, backbone execution, head plugins, and deployment adapters. The runtime provides modular multi-rate execution, latency-first fused inference, and extensible operator and I/O support, enabling deployment across heterogeneous devices, robots, and simulators through one backend abstraction. We evaluate Embodied.cpp on two VLA models, HY-VLA and pi0.5, and on a preliminary WAM benchmark using a LingBot-VA Transformer block. The VLA deployments achieve successful closed-loop execution with 100.0% and 91.0% task success rates, respectively. The WAM benchmark reduces block memory from 312.2 MiB to 88.1 MiB. These results show that Embodied.cpp improves deployment efficiency while preserving high accuracy across diverse embodied model architectures.