cs.CV [Total: 54]
cs.CL [Total: 26]
cs.AI [Total: 4]
cs.LG [Total: 4]
cs.MM [Total: 1]
cs.RO [Total: 4]
cs.SD [Total: 1]
cs.SE [Total: 1]
eess.AS [Total: 1]
eess.IV [Total: 1]
quant-ph [Total: 1]

cs.CV [Back]

[1] Adaptive Interpolatory Curve Subdivision with Learned Local Angles cs.CVPDF

Hassan Ugail, Newton Howard

TL;DR: 该研究提出自适应局部角插值细分，在保留控制点精确保留性质的同时，为每条边学习顶点插入角度，并在平面、球面及庞加莱圆盘使用对应测地几何原语；匹配点密度下最近邻误差较调优固定张力基线降低5–17倍。

Details

Motivation: 插值曲线细分可从控制多边形生成平滑曲线，并保证曲线经过设计者的原始控制点。经典四点和六点方法通过单个全局张力参数调节，难以针对平坦区域、尖锐转角及不同局部几何自适应。

Result: 方法提出自适应局部角参数化，在保持插值结构的同时学习每个新顶点的插入方式。紧凑逐边预测器为每条边预测一个插入角，而每轮细分均原样复制原始顶点，因此插值性是算子结构保证的，不取决于训练权重。相同预测器可配合针对欧氏平面、二维球面和庞加莱圆盘的几何特定测地原语。匹配密度评测中，最近邻误差较验证集调优的最佳固定张力基线降低5至17倍；欧氏场景下较向心Catmull–Rom低约1.8倍。方法还显著降低弯曲能量和切线粗糙度，并与分别针对各几何训练的模型竞争。

Insight: 将可学习参数局部化到每条边，同时把插值性质编码进细分算子，可获得自适应性而不失去控制点约束；同一预测器也能配合不同空间的测地几何。摘要未给出数据规模、各几何的单独数字及训练开销。

Abstract: Curve subdivision is pivotal in computer graphics for generating smooth geometric objects from control polygons. Interpolatory subdivision is especially attractive because the refined curve is guaranteed to pass through the designer’s control points. Classical four-point and six-point schemes preserve this property, but their behaviour is governed by a single global tension parameter, limiting their ability to adapt across flat regions, sharp turns and varying local geometries. We introduce an adaptive local-angle formulation that keeps the interpolatory structure intact while learning how each new vertex should be inserted. A compact edge-wise predictor assigns one insertion angle per edge, while the original vertices are copied exactly at every refinement level. Interpolation is therefore a structural property of the operator and does not depend on the trained weights. The same predictor is used with geometry-specific geodesic primitives on the Euclidean plane, the two-sphere and the Poincaré disk. Under a matched-density evaluation protocol, the method reduces nearest-neighbour error by factors of five to seventeen over the best validation-tuned fixed-tension baseline, and by about 1.8 over centripetal Catmull-Rom in the Euclidean case. It also substantially reduces bending energy and tangent roughness, while remaining competitive with separately trained per-geometry models.

[2] A Heisenberg Lift Descriptor for Order Sensitive Online Handwriting Recognition cs.CVPDF

Hassan Ugail, Newton Howard

TL;DR: 该工作提出Heisenberg提升的在线笔迹特征，以签名面积等方向敏感统计补充只表示笔画空间轮廓、忽略绘制顺序的欧氏描述符；在两个标准基准上提升识别率，字母o/y最难区分对仅用一个有符号面积特征即可完美分离，且更能抵抗坐标噪声。

Details

Motivation: 在线手写识别常用固定长度欧氏形状描述符表示笔迹空间轮廓，却忽略轮廓的生成顺序。相反方向沿同一区域绘制的笔画会被这类顺序无关表征视为相同，但在依赖环方向和笔顺的字符中，绘制方向可能决定类别。

Result: 作者提出Heisenberg提升框架，作为紧凑、可解释、对顺序敏感的在线笔迹特征扩展。最简单形式是终点有符号面积，一个无需学习的标量，直接拼接到既有欧氏描述符，计算成本可忽略。在两个标准在线手写基准上，单一面积特征稳定提高分类准确率；最难的字母o与y区分中，仅该面积即可完美分离，而欧氏基线无法做到。加入坐标噪声后优势进一步扩大。更丰富的15维扩展基于非交换Heisenberg群细分方案，在噪声和含环结构条件下再带来收益。维度匹配统计控制确认改善来自几何信息，而非仅增加特征数。

Insight: 笔迹不仅是几何路径，还包含沿路径的遍历方向和顺序；利用非交换几何特征可补足传统欧氏轮廓表征。该特征闭式、轻量且可解释，适合需要笔画执行方向信息的在线手写和轨迹分类；摘要未提供两个基准名称及总体增幅。

Abstract: Online handwriting recognition systems typically represent pen trajectories through fixed-length Euclidean shape descriptors that capture the spatial outline of each stroke, but are insensitive to the order in which that outline is produced. Two strokes that trace the same region of the plane in opposite directions are indistinguishable to any such order-blind representation, yet their traversal directions may carry decisive class information in characters where loop orientation and stroke sequencing matter. This paper introduces a Heisenberg-lift framework that addresses this gap through a compact, interpretable, order-sensitive augmentation for online pen-trajectory features. The simplest instance is the terminal signed area, a single parameter-free scalar appended to an existing Euclidean descriptor at negligible computational cost. Evaluated on two standard online handwriting benchmarks, this one-scalar addition, consistently raises classifier accuracy over the Euclidean baseline. On the hardest character pair in our study, the letters o and y, the signed area alone achieves perfect separation while the Euclidean baseline falls short. The advantage grows further under additive coordinate noise, a practically relevant degradation in pen-trajectory data. A richer fifteen-dimensional extension, derived from a noncommutative Heisenberg-group subdivision scheme, provides additional gains in noisy and loop-structured conditions. Dimension-matched statistical controls confirm that all improvements reflect geometric information rather than feature-count inflation. The resulting descriptor is lightweight, closed-form, and directly interpretable, making it a practical augmentation for online handwriting and related document-trajectory classification pipelines in which the direction of stroke execution carries discriminative information.

[3] Not All Patches Are Equally Forgettable: Spatially Localized Domain Unlearning in Vision-Language Models cs.CVPDF

Akanksha Singh, Vinod K. Kurmi

TL;DR: 该研究针对视觉语言模型的选择性领域遗忘提出两阶段patch级方法：估计每个ViT图像区域对遗忘领域预测的敏感度，再抑制领域贡献大于语义效用的patch；在Office-Home、Mini DomainNet和DomainNet上改善遗忘—保留折衷，保留领域识别最高提升3.8%。

Details

Motivation: 预训练VLM无需额外训练即可具备跨域识别能力，但这也可能保留不希望出现的领域特定行为。领域与语义信息在表征空间中纠缠，使选择性遗忘困难；现有方法多在潜空间解耦或干预提示/特征，未直接分析和削弱各图像patch的贡献。

Result: 作者提出两阶段patch选择框架：先估计每个patch的领域敏感度，再选择性削弱对遗忘领域预测贡献高于其语义效用的区域。方法利用视觉Transformer的空间结构，避免均匀压制整个表征。评估覆盖Office-Home、Mini DomainNet和DomainNet，显示相较既有方法改善遗忘与保留的权衡，保留领域识别最高提升3.8%。在视觉相互重叠和未见领域设置下的额外实验也显示分布偏移鲁棒性提高。

Insight: 遗忘目标可能集中在少数对特定领域预测贡献大的局部视觉区域，逐patch处理可比整体抑制更好地保留有用语义。摘要未给出遗忘准确率、各数据集结果或隐私/记忆删除的验证，结论聚焦领域行为遗忘。

Abstract: Pre-trained vision-language models (VLMs) exhibit strong cross-domain recognition performance even without additional training. However, this robustness can also preserve undesirable domain-specific behavior, as domain-related and semantic information often remain entangled within the learned representation space, making selective domain unlearning challenging. Existing approaches typically address this problem through latent-space disentanglement and prompt- or feature-level interventions, without directly attributing and attenuating individual patch-token contributions. However, here we suggest that rather than uniformly suppressing the full representation, it may be more effective to exploit the spatial structure of vision transformers to localize and suppress patch regions that contribute disproportionately to forget-domain prediction. Patches that strongly influence forget-domain prediction may not be equally important for semantic recognition, suggesting that forgetting should be guided according to the domain contribution of different visual regions. Specifically, we propose a two-stage patch-selective framework that first estimates patch-level domain sensitivity and then selectively attenuates patches whose contribution to forget-domain prediction is stronger than their semantic utility. We evaluate our framework on Office-Home, Mini DomainNet, and DomainNet. Experimental results demonstrate improved forgetting-retention tradeoffs compared to prior methods while improving retained-domain recognition by up to 3.8%. Additional evaluations under visually overlapping and unseen-domain settings further demonstrate improved robustness under distribution shift.

[4] AgenTeeth: A Model-Agnostic Framework for Suppressing Hallucination in Frozen Vision-Language Models on Dental X-Rays via Tool Evidence Injection cs.CVPDF

Ahmed Rafid, Fariya Ahmed, Rumman Adib, Mehedi Ahamed, Ajwad Abrar

TL;DR: AgenTeeth无需微调冻结VLM，而是由问题感知编排器选择七个牙科视觉专家，将检测映射到FDI牙位或解剖区域并返回标注图，再让VLM基于证据作答；MMOral-OPG-Bench上四个骨干提升12.9–23.0个百分点，最佳开放式VQA为65.66%。

Details

Motivation: VLM在全景牙片上仍不可靠，常依赖学到的解剖先验而非图像证据；牙齿定位和空间推理尤为困难，牙科领域微调模型也可能保留相同空间偏差。

Result: AgenTeeth是模型无关的工具增强框架，使用七个专用牙科视觉专家为冻结VLM提供接地证据。问题感知编排器按问题选择工具，将检测结果映射到FDI牙位编号或解剖区域，并以结构化发现和带标注影像形式返回；随后发起新的综合调用，让VLM依据这些证据回答，无需微调底层模型。在MMOral-OPG-Bench上，四个骨干VLM相较各自基线提高12.9–23.0个百分点。最佳配置开放式VQA为65.66%，高于OralGPT-Plus的45.35%。在匹配骨干时，冻结Qwen2.5-VL-7B-Instruct加AgenTeeth为48.11%，也超过同骨干经监督微调和工具使用强化学习的OralGPT-Plus。作者发布框架、七个专家模型及牙槽骨流失检测牙医标注数据集。

Insight: 牙科视觉幻觉可通过工具产生的定位和结构证据来约束回答，而不必重新训练VLM；七专家与问题路由将领域能力模块化。结果来自专门基准，摘要未提供不同工具错误传播、真实临床使用或诊断工作流评估。

Abstract: Vision-language models (VLMs) remain largely unreliable on panoramic dental radiographs and can rely on learned anatomical priors rather than evidence in the image. This is particularly problematic for tooth localization and spatial reasoning, and fine-tuned dental VLMs can retain the same spatial biases. We present AgenTeeth, a model-agnostic, tool-augmented framework that grounds frozen VLMs using seven specialized dental vision experts. A question-aware orchestrator selects the relevant tools, whose detections are mapped to FDI tooth numbers or anatomical regions and returned as structured findings together with annotated image overlays. A fresh synthesis call then answers the question using this evidence, without fine-tuning the underlying VLM. On MMOral-OPG-Bench, AgenTeeth improves four backbone VLMs by 12.9-23.0 percentage points over their baselines. Our strongest configuration reaches 65.66% on open-ended VQA, compared with 45.35% for OralGPT-Plus. The advantage also holds at matched scale: a frozen Qwen2.5-VL-7B-Instruct with AgenTeeth reaches 48.11%, exceeding OralGPT-Plus built on the same backbone after supervised fine-tuning and reinforcement learning for tool use. We release the framework, all seven expert models, and a dentist-annotated dataset for alveolar bone-loss detection in panoramic radiographs.

[5] CALIPER: Metric-Grounded Model-Free Recognition of Visually Similar Industrial Parts cs.CVPDF

Alankrit Gupta, Chenxi Tao, Seung-Kyum Choi

TL;DR: CALIPER是无需逐类大规模训练数据的RGB-D工业零件识别系统，以少量样本建立外观支持和公制尺寸档案，在外观模糊时融合尺寸证据；18种相似零件闭集准确率88.2%，两种未见螺钉十样本注册后总体85.7%，机械臂上18种识别17种。

Details

Motivation: 工业零件类别可能主要由实际尺寸区分，将检测裁剪缩放至固定输入大小会抹去绝对尺度；快速变化的库存也可能缺少CAD模型和大型类别专用数据集，因此需要易扩展、无需重新训练的精细识别方案。

Result: CALIPER将支持样本外观匹配与公制尺寸证据结合。每个训练类只需一段转台RGB-D视频和一至两张标注实拍图；3D重建生成新视角外观支持，配准深度形成类别尺寸档案。推理时由粗粒度YOLOv8n-seg定位零件，冻结DINOv2骨干和情景训练嵌入头执行细粒度支持匹配；仅在外观相似、置信边界较小时，按边界条件激活概率尺寸证据。新类别可用少量RGB-D样本注册而不更新参数。18种相似零件实验中，16类训练、两种螺钉留作免训练注册；闭集准确率88.2%，定位召回率99.8%，两种新螺钉十样本注册后的整体准确率85.7%。尺寸融合使未见类别准确率最高提升37.4个百分点，且未显著损害原类别。机械臂部署无需专门重训即可识别18种中的17种。

Insight: 在外观证据不确定时，绝对尺寸可以提供独立互补的判别线索；只对歧义样本融合尺寸还可避免其损害已知类别。实验覆盖18种零件和两种新类，摘要未报告不同照明、遮挡及更大规模库存的泛化。

Abstract: Fine-grained recognition of visually similar industrial parts is challenging when classes differ primarily in physical dimensions. Normalizing detected object crops to a fixed input size suppresses absolute scale, while CAD models and large class-specific datasets may be unavailable in evolving industrial inventories. We present CALIPER, a model-free RGB-D framework that couples support-based appearance matching with metric size evidence. Each training class is onboarded from a single turntable RGB-D video and one to two labeled real images; 3D reconstruction provides novel-view appearance support, while aligned depth yields a class-specific metric size profile. At inference, a coarse YOLOv8n-seg model localizes parts, and a frozen DINOv2 backbone with an episodically trained embedding head performs fine-grained support matching. Margin-conditioned metric fusion activates probabilistic size evidence only for appearance-ambiguous decisions. New classes are enrolled from a small RGB-D support set without updating network parameters. We evaluate CALIPER on 18 visually similar industrial parts: 16 classes are used for training, while two screws are reserved for training-free enrollment. CALIPER achieves 88.2% closed-set accuracy with 99.8% localization recall and 85.7% overall accuracy after 10-shot enrollment of the two unseen screws. Metric fusion improves unseen-class accuracy by up to 37.4 percentage points without statistically significant degradation of the original inventory. Robot-arm deployment identifies 17/18 parts without deployment-specific retraining.

[6] RoboVAD: A Large Cross-Domain Evaluation Benchmark for Anomaly Detection in Robotic Arm Manipulation Videos cs.CV | cs.AI | cs.LG | cs.ROPDF

Alexandru-Bogdan Dura, Sebastian Balmus, Radu Tudor Ionescu

TL;DR: RoboVAD是面向机械臂操作视频异常检测的大规模跨域基准，训练时将特定任务和异常类型留出以模拟真实新任务；新方法超过不少基线，但最难设置下所有方法的微平均帧级AUC均低于70%，显示跨域异常识别仍困难。

Details

Motivation: 视频异常检测也可用于机器人交互与操作，例如改进学习、发现操作失误并触发恢复程序，但机械臂操作视频可用资源有限，阻碍了该方向研究。

Result: 作者发布RoboVAD，设计具有挑战性的跨域评测场景：部分机械臂执行任务及其异常类型在训练时均未出现，模拟机器人遇到新任务和新错误的真实情况。研究训练并评估多种先进视频异常检测方法，还提出专门针对机械臂操作改造的新方法。新方法超过多种先进竞争方法，但在最困难设置下所有方法的微平均帧级AUC都低于70%，表明该基准具有较高难度。数据集和代码公开。

Insight: 机械臂异常检测需要在未见任务及未见错误类型上泛化，随机拆分同一任务的视频可能高估部署能力；RoboVAD通过留出任务和异常类别刻画这一挑战。摘要未报告具体数据规模、方法分项AUC或异常定义。

Abstract: Video anomaly detection (VAD) is an actively studied task, having wide applications in typical scenarios such as public surveillance and road traffic safety. The task is also relevant for robotic arm interactions, where it has several downstream applications, including learning better interaction and manipulation abilities, triggering recovery procedures when anomalies occur, etc. Despite its relevance, the exploration of anomaly detection in robotic arm manipulation videos is limited by the low number of available resources. To this end, we introduce RoboVAD, a large-scale benchmark for video anomaly detection that comprises challenging cross-domain evaluation scenarios, where certain actions (tasks executed by a robotic arm) and anomaly types (mistakes that occur while performing certain tasks) remain unseen during training. RoboVAD is designed to benchmark VAD methods in realistic scenarios, where robotic arms can perform unforeseen tasks, and thereby encounter new anomaly types. We train and evaluate several state-of-the-art VAD methods, including a novel method specifically adapted for robotic arm manipulation. While the proposed method outperforms many state-of-the-art competitors, all methods remain below a micro-averaged frame-level AUC threshold of 70% in the most challenging evaluation setup, confirming the difficulty of the proposed benchmark. We publicly release our dataset and code at https://zenodo.org/records/22754659.

[7] Can VLMs Reliably Assess Sidewalk Accessibility Attributes from Pedestrian-Level Imagery? cs.CV | cs.CY | cs.LGPDF

Seung Jae Lieu, Diego Morra, Chiara Cadoni, Wonseop Song, Martina Mazzarello

TL;DR: 该研究以首尔514张人行道实测图像测试四个VLM能否估计无障碍属性，并首次应用抽样保形预测校准区间；有效宽度估计最有信息量（最佳模型平均半宽约1米），纵坡略有信息、横坡与路面状况难以判断，所有属性尚未达到通用合规评估精度。

Details

Motivation: 人行道是否满足可测量无障碍要求，对轮椅使用者和行动不便者尤为重要。作者研究VLM能否仅凭行人视角图像可靠估计有效宽度、纵坡、横坡和铺面状况，并强调不确定性必须通过现场真值校准。

Result: 研究使用首尔514张人行道图像及现场测量真值评估四种VLM，并首次将抽样式保形预测用于VLM无障碍属性估计。校准后所有模型和属性均达到名义90%覆盖率，但区间信息量差异很大：有效宽度最有信息，最佳模型平均区间半宽约1米。所有模型都高估宽度，因此非对称校准可在覆盖率不变时将区间缩短最多33%。纵向坡度略有信息；横坡区间太宽，无法区分法规阈值；四个模型中三个的路面状况集合退化为全部五级A–E。未经校准的原始采样离散度在名义90%覆盖率下，实际只覆盖17%–47%的实测值；即使挑选回答最一致的图像，区间也会在最多96%的案例中漏掉实测值。没有任何属性达到通用合规评估所需精度，但保形预测可仅凭校准数据识别哪些属性适合筛查远离阈值的路段。

Insight: 回答自洽并不代表估计正确，模型采样分散度也不能直接当成不确定性；只有对照现场测量进行保形校准后，覆盖保证才有意义。当前模型适合筛查明显远离法规阈值的部分属性，不适用于普遍合规判定。作者公开图像及现场属性数据。

Abstract: An important component of urban accessibility, particularly for wheelchair users and people with reduced mobility, is sidewalk compliance with measurable requirements. We test whether effective width, longitudinal slope, cross slope, and pavement condition can be assessed reliably from pedestrian-level imagery using vision-language models (VLMs). We present the first application of sampling-based conformal prediction (CP) for VLM-based accessibility assessment. We evaluate four VLMs on 514 sidewalk images from Seoul, South Korea, with field-measured ground truth. Conformal calibration attains the nominal 90% coverage for all models and attributes, but the calibrated regions differ in informativeness. Effective width yields the most informative estimates, with a mean interval half-width of about 1.0 m for the best model. Since every model overestimates width, asymmetric calibration shortens the intervals by up to 33% at unchanged coverage. Longitudinal slope is marginally informative, cross-slope intervals are too wide to resolve regulatory thresholds, and pavement-condition sets degenerate to all five grades (A-E) for three of the four models. Uncalibrated intervals from raw sampling dispersion cover only 17-47% of field-measured values at a nominal 90% level. Among the images with the most self-consistent responses, these intervals miss the field-measured value in up to 96% of cases. Response self-consistency is therefore not evidence of accuracy, and sampling dispersion cannot be interpreted as uncertainty until it has been calibrated against field-measured ground truth. No quantitative attribute reaches the precision required for general compliance assessment, but CP identifies from calibration data alone which attributes can support screening of segments far from the thresholds. We release the annotated pedestrian-level images and their corresponding field-measured attribute values.

[8] Lumen: Parameter-Efficient Alignment of Pretrained Vision and Language Encoders for Zero-Shot Computational Pathology cs.CVPDF

Kiarash Tajbakhsh, Abdelrahman Faqieh, Michael Jopiti, Javier Garcia-Baroja, Philipp Zens

TL;DR: Lumen用rank-4适配器和投影头对齐冻结的Virchow2与BioMedBERT，仅训练总参数的0.40%；九个病理零样本切片基准的平均机会校正平衡准确率为0.546，高于最强基线0.461，并在内外部淋巴结转移检测切片上AUROC分别达0.964和0.955。

Details

Motivation: 病理视觉语言模型通常需在图像—文本配对数据上预训练或微调大型编码器。作者研究能否冻结单模态基础模型的预训练表示，只进行参数高效对齐，组建可用于零样本计算病理的视觉语言模型。

Result: Lumen使用rank-4适配器及投影头对齐冻结的Virchow2和BioMedBERT，并在公开QUILT-1M数据集上仅训练总参数的0.40%。九个公开零样本切片基准上，平均机会校正平衡准确率最高，为0.546；最强基线为0.461，配对差0.086，95% CI [0.042,0.136]。淋巴结转移检测中，4,214张内部留出切片AUROC为0.964（95% CI [0.956,0.971]），九个外部队列、六个器官共2,368张切片AUROC为0.955（[0.942,0.966]）。使用内部校准阈值时，平衡准确率内测0.909、外测0.915，均超过视觉语言基线。跨模态检索是例外，Lumen排名第三，落后CONCH和PathGen-L/14。完全微调两个编码器相比低秩适配无稳定收益，虽改善检索。

Insight: 冻结两个专用单模态基础模型并进行极少参数对齐，即可取得强零样本病理表现和跨医院泛化；完全微调并非普遍有益。不同任务有取舍，跨模态检索落后，说明分类/切片识别优势不应外推到所有多模态任务。

Abstract: Pathology vision-language models are commonly built by pretraining or fine-tuning large encoders on paired image-caption data. We asked whether a pathology vision-language model can instead be assembled by parameter-efficient alignment of frozen unimodal foundation models, leaving their pretrained representations untouched. Here we present Lumen, which aligns frozen Virchow2 and BioMedBERT backbones using rank-4 adapters and projection heads, training only 0.40% of the total parameters on the public QUILT-1M corpus. Across nine public zero-shot patch benchmarks, Lumen achieved the highest mean chance-corrected balanced accuracy, 0.546 versus 0.461 for the strongest baseline (paired difference 0.086, 95% CI 0.042-0.136). On lymph-node metastasis detection, Lumen reached an AUROC of 0.964 (95% CI 0.956-0.971) on 4,214 held-out internal slides and 0.955 (95% CI 0.942-0.966) on 2,368 slides across nine external cohorts and six organs. At the internally calibrated threshold, it outperformed all vision-language baselines, with a balanced accuracy of 0.909 (95% CI 0.896-0.923) internally and 0.915 (95% CI 0.902-0.929) externally. Lumen performed competitively across the evaluations, with the exception of cross-modal retrieval, where it ranked third behind CONCH and PathGen-L/14. Fully fine-tuning both encoders gave Lumen no consistent benefit over low-rank adaptation, although it improved retrieval. Aligning frozen unimodal foundation models therefore yields strong and transferable performance at patch and slide level while training only a small fraction of the parameters.

[9] Zing-0.5: Toward Playable Worlds with Real-Time Joint Action and Text Control cs.CV | cs.LGPDF

Mingyang Chen, Shengdong Chen, Xiaoxiao Fu, Bosheng Gong, Haoyuan Guo

TL;DR: Zing-0.5是5B可玩世界自回归模型，支持用户同时用键盘和在线文本控制生成世界；它联合动作/文本条件、用事件级教师蒸馏分块学生，并以四步生成流式推理，在832×480下达24 FPS，WBench Navigation总体分81.0。

Details

Motivation: 可玩生成世界要求用户能探索场景、影响事件发展并对反馈作出响应；单纯从提示生成视频无法支持生成中途持续交互，联合动作和文本控制及实时推理仍是关键挑战。

Result: Zing-0.5是5B自回归世界模型，支持键盘与在线文本共同控制。模型通过幅度感知键盘输入、时间对齐文本指令和联合标注视频进行统一条件化，在同一序列中学习导航和事件控制；事件级分段教师由多提示连贯视频训练，再以分布匹配蒸馏指导块级因果学生；实时交互采用四步生成和保持上下文的流式机制。系统在832×480分辨率下达到24 FPS，估算服务器租赁成本约每条流每分钟0.009美元。158个WBench Navigation案例上总分81.0、一致性分88.5。联合控制演示显示，用户可在继续导航中用文字指令改变事件而不重启生成。模型权重、推理代码和Zing-SGLang服务实现公开。

Insight: 可玩世界需把导航动作与事件语言指令统一在连续生成上下文中，同时控制延迟；事件级教师分块蒸馏为实时自回归学生提供监督。帧率和成本是报告配置下的估算，摘要未给基准对比基线分数。

Abstract: We introduce Zing-0.5, a 5B autoregressive world model designed for playability: users can explore generated worlds, influence unfolding events, and respond to the resulting feedback through joint keyboard and online text control. Our approach brings together three technical contributions: (1) Unified action and text conditioning, combining magnitude-aware keyboard inputs with temporally aligned text instructions and jointly annotated videos to learn navigation and event control within the same sequence; (2) Event-scale supervision for incremental generation, using a segment-level teacher trained on connected multi-prompt videos to supervise a block-level causal student through distribution-matching distillation; and (3) Low-cost real-time interaction, combining four-step generation with context-preserving streaming to support 832 x 480 inference at 24 FPS at an estimated server rental cost of approximately USD 0.009 per stream-minute. Zing-0.5 achieves an overall score of 81.0 and a consistency score of 88.5 across 158 WBench Navigation cases. A joint-control demonstration shows a text-directed event change during continued navigation without restarting generation. We release the model weights, inference code, and Zing-SGLang serving implementation to support further work on playable generated worlds.

[10] Audio for Sports Highlight Detection: A Comparative Empirical Study cs.CVPDF

Hao Xu, Meenakshi Sarkar, Vishnu Raj, David Gunawan

TL;DR: 该研究在SV-Highlights上比较体育高光检测中的音频、视觉及融合方法，发现轻量音频GRU单独即可超过若干既有音视频方法，简单音视频融合在所有指标上最好；解说语音比纯背景音更有信息量，但音量特征单独不足以识别高光。

Details

Motivation: 体育高光检测要从长视频中找到最激动人心、最有意义的时刻。既有方法多强调视觉或视觉语言表征，但体育视频包含解说、观众反应、哨声、球声和裁判口令等丰富音频信息，其独立贡献尚未充分研究。

Result: 作者构建轻量音频单模态基线，利用预训练音频表征，并在SV-Highlights上与视觉单模态和音视频方法比较。音频GRU基线表现强劲，在监督评测设置下超过若干既有音视频方法；简单音视频融合在全部指标上表现最佳，说明两种模态互补。音源分离分析发现，解说/人声比仅背景音更有信息，二者结合最佳。可解释音频分析显示，高光片段RMS响度、峰值响度和中频能量均更高，但与非高光分布仍大量重叠，仅靠响度无法判定。

Insight: 体育视频高光的情绪和事件线索常由声音直接表达，音频可作为主要信号，而非视觉系统的附属输入；最佳表现来自视听融合。响度差异虽相关但不足以单独分类，摘要未列各方法的具体指标值。

Abstract: Sports highlight detection aims to identify the most exciting and meaningful moments from long sports videos. While existing methods often emphasize visual or visual-language representations, sports videos contain rich audio cues, including commentator speech, crowd reactions, whistles, ball impacts, and referee calls. In this work, we revisit the role of audio in sports highlight detection and ask a simple question: how far can audio alone go? We construct lightweight audio-only baselines using pretrained audio representations and compare them with visual-only and audio-visual methods on the SV-Highlights benchmark. Surprisingly, our audio-only GRU baseline achieves strong performance and outperforms several existing audio-visual methods under our supervised evaluation setting. Furthermore, a simple audio-visual fusion baseline achieves the best performance across all metrics, indicating that audio and visual cues provide complementary information. To better understand the contribution of audio, we conduct source-separated analysis and show that vocal/commentary audio is more informative than background-only audio, while their combination performs best. We also analyze interpretable audio cues and find that highlight clips exhibit higher RMS loudness, peak loudness, and mid-frequency energy than non-highlight clips, although substantial distribution overlap indicates that loudness alone is insufficient. Our findings suggest that audio is an underexplored but highly informative modality for sports highlight detection and should be treated as a primary signal rather than merely an auxiliary cue.

[11] EDCT-Bench: Uncovering Faithfulness Gaps in VLMs via Explanation-Driven Counterfactual Testing cs.CV | cs.AIPDF

Sihao Ding, Santosh Vasa, Aditi Ramadwar, Thomas Monninger

TL;DR: EDCT-Bench用解释驱动的反事实干预评测VLM回答是否忠实于其引用的视觉证据：从解释中抽取视觉概念、对图像做验证过的最小编辑，再检查答案和解释是否随图像变化保持一致；基准覆盖OK-VQA、DriveLM和3DSRBench，微调研究显示生成反事实可提供高价值训练信号。

Details

Motivation: VLM生成的自然语言解释可能听起来合理，却与其引用的图像证据不一致。仅评答案正确性或解释流畅度难以发现这种忠实性缺口，需要直接干预图像并检验模型主张是否相应变化。

Result: 解释驱动反事实测试EDCT先从模型解释中提取被引用的视觉概念，对这些概念对应内容做经验证的最小图像编辑，再检查编辑后答案和解释是否与新图像保持一致。基于此构建EDCT-Bench，覆盖知识密集型视觉问答OK-VQA、安全关键驾驶DriveLM和3D空间推理3DSRBench三个领域。受测VLM普遍存在显著忠实性差距，常输出与经过验证的视觉变化不一致的结果。微调实验提示EDCT生成的反事实样本可提供高影响训练信号。摘要未给出各模型不一致比例或基准规模。

Insight: 若解释声称依赖某个视觉概念，针对该概念做最小图像干预后，答案应随证据变化；这种因果一致性测试能比静态答案评分更直接揭露解释是否忠实。反事实编辑需要经过验证以确保只改变目标证据，摘要未报告编辑误差。

Abstract: Vision-Language Models (VLMs) can produce Natural Language Explanations (NLEs) that sound plausible yet remain inconsistent with the visual evidence they cite. We present Explanation-Driven Counterfactual Testing (EDCT), an intervention-based protocol that extracts visual concepts cited in a model’s explanation, applies verified minimal edits to them, and tests whether the resulting answer and explanation remain consistent with the edited image. Using this protocol, we create EDCT-Bench, a comprehensive benchmark spanning three complementary domains: knowledge-intensive visual question answering (OK-VQA), safety-critical driving (DriveLM), and 3D spatial reasoning (3DSRBench). Across the evaluated VLMs, EDCT reveals substantial faithfulness gaps, with models frequently producing responses inconsistent with verified visual changes. Finally, our fine-tuning study suggests that EDCT-generated counterfactuals provide high-impact training signals.

[12] IRIS: Implicit Rendering Matters for Pose-Free Novel View Synthesis cs.CVPDF

Wenyu Li, Sidun Liu, Peng Qiao, Yong Dou, Tongrui Hu

TL;DR: IRIS是无位姿多视图新视角合成的完全自监督框架，在隐式潜空间渲染与显式3D表示间折衷：以潜在神经场表示场景，在自预测相机下查询渲染，并沿目标射线组合参考视图投影特征；实验显示合成质量强且位姿精度有竞争力。

Details

Motivation: 从无位姿多视图图像进行新视角合成需同时学习场景表征和相机参数，且没有位姿监督。隐式潜空间渲染灵活、易优化但相机估计几何依据弱；显式3D表示几何约束更强，却参数化重、优化更脆弱。

Result: IRIS提出完全自监督的折衷方案：不直接解码自由潜token，也不重建完全显式3D基元，而将场景表示为潜在神经场，并在模型自预测的相机位姿条件下查询该场。具体将参考视图投影特征聚合到采样3D点形成点级潜特征，再沿目标视线组合进行渲染。该设计保留隐式建模的灵活性和优化稳定性，同时比不受约束的潜变量渲染具有更强几何结构。大量实验显示，IRIS在完全自监督训练下取得较强新视角合成质量和有竞争力的位姿精度。摘要未给出数据集、指标值或基线分数。

Insight: 位姿估计和场景建模需要隐式灵活性与几何约束兼具；潜在神经场通过3D点聚合和自预测相机提供中间折衷。摘要只定性报告质量和位姿结果，不能量化优势。

Abstract: Novel view synthesis from unposed multi-view images remains challenging, as the model must jointly learn scene representations and camera parameters without pose supervision. Existing approaches largely fall into two extremes: implicit latent-space rendering is flexible and easy to optimize, but often yields weakly grounded camera estimation; explicit 3D representations provide stronger geometric grounding, but introduce heavier parameterization and more fragile optimization. In this paper, we present IRIS, a fully self-supervised framework that provides a practical middle ground between these two paradigms. Instead of decoding free latent tokens or reconstructing fully explicit 3D primitives, IRIS represents the scene as a latent neural field and renders novel views by querying this field under self-predicted cameras. Specifically, projected features from reference views are aggregated at sampled 3D points to form point-wise latent features, which are then composed along target rays for rendering. This design preserves the flexibility and optimization stability of implicit modeling, while introducing stronger geometric structure than unconstrained latent rendering. Extensive experiments show that IRIS achieves strong novel view synthesis quality with competitive pose accuracy under fully self-supervised learning. Our project page: https://leo-frank.github.io/IRIS

[13] vidax: A Unified JAX Framework for Video Generative Models on Accelerator Meshes cs.CV | cs.DC | cs.LGPDF

Congyue Deng

TL;DR: vidax是统一的视频生成JAX/Flax推理框架，提供PyTorch到JAX零拷贝权重转换，支持DiT、多模态MoT、3D VAE、文本编码器及采样器；在单一JAX分片网格整合张量和序列并行、TPU Flash Attention及逐层权重卸载，并在TPU v4-8评测延迟、编译和峰值内存。

Details

Motivation: 开源视频生成模型几乎都以PyTorch/CUDA参考实现发布，云端TPU因而缺少可用于生产的推理路径，尽管TPU的大型且较低成本加速器内存池适合长序列时空注意力。

Result: vidax是开源JAX/Flax推理引擎及PyTorch到JAX零拷贝权重转换器，支持扩散Transformer、多模态混合Transformer、3D VAE、文本编码器和原生采样器等时空模型。执行路径不依赖PyTorch。框架在单个JAX分片网格上统一一维张量并行与DeepSpeed-Ulysses序列并行，集成TPU Flash Attention内核，并实现逐层权重卸载，以支持超过单设备内存的参考分辨率。作者在TPU v4-8上评测编译时间、推理延迟和峰值内存，并记录检查点转换中发现的实际数值错误。项目开源作为JAX和TPU视频生成研究基线。

Insight: 给TPU提供覆盖模型、权重转换和并行策略的一体化推理路径，可降低视频生成从CUDA迁移到JAX的工程门槛；检查点迁移数值错误也说明执行框架兼容性需要实际验证。摘要未报告性能数值或相对PyTorch速度。

Abstract: Open-source video generative models ship almost exclusively as PyTorch/CUDA reference implementations. This leaves Cloud TPU pods without a production-ready inference path, despite offering large, cost-effective accelerator memory pools ideal for long-sequence spatiotemporal attention. We present vidax, an open-source JAX/Flax inference engine and zero-copy PyTorch-to-JAX weight translator for modern video generation architectures. vidax covers a diverse set of spatiotemporal models — including Diffusion Transformers, omnimodal Mixture-of-Transformers, 3D VAEs, text encoders, and native samplers — with zero PyTorch dependency in the execution path. The framework unifies 1D tensor parallelism with DeepSpeed-Ulysses sequence parallelism on a single JAX sharding mesh, integrates TPU flash-attention kernels, and implements per-layer weight offloading to support reference resolutions that exceed single-device memory. We benchmark compile times, latency, and peak memory utilization on TPU v4-8 hardware, and document real-world numerical bugs surfaced during checkpoint translation. vidax is released open-source as a baseline for JAX and TPU video generation research.

[14] Mask 2D-3D: Adaptive Dual-Masked Autoencoder Network for Image-to-Point Cloud Registration cs.CV | cs.AIPDF

Zhixin Cheng, Jiacheng Deng, Xiaotian Yin, Baoqun Yin, Richang Hong

TL;DR: ID-MAE针对图像—点云配准提出跨模态双掩码自编码器，并用相似度驱动的强化学习掩码策略优先遮蔽信息位置，以缩小模态差距、改善二维三维对应；在RGB-D Scenes v2和7-Scenes上取得先进配准表现。

Details

Motivation: 无检测器图像—点云配准容易因视觉域与模态差异、特征提取敏感度不足和非重叠区域而产生错误对应。标准MAE使用随机遮蔽，在有限相机视角下可能漏掉关键区域，降低跨模态配准效果。

Result: 作者提出跨模态双MAE框架ID-MAE及相似度驱动的强化学习遮蔽策略SRLM。方法利用跨模态相似度和强化学习自适应选择信息位置进行遮蔽，以缩小模态差异；训练还通过约束特征提取过程中的表征一致性，增强跨模态表示并改善可靠2D—3D对应估计。在RGB-D Scenes v2和7-Scenes基准上的实验显示，ID-MAE达到图像—点云配准先进水平。摘要未给出具体误差或相较基线提升数值。

Insight: 跨模态配准的掩码位置应利用图像与点云的相似性自适应选择，而非独立随机遮蔽；保留对应区域可改善共享特征空间。摘要未说明强化学习奖励、监督方式及数据集上的数值结果。

Abstract: Detection-free methods for image-to-point cloud registration are prone to erroneous correspondences caused by domain and modality discrepancies, limited sensitivity of feature extractors, and the presence of non-overlapping regions. The Masked Autoencoder (MAE) has shown strong performance in visual representation for images and point clouds. It may be helpful to apply this approach to image-to-point cloud registration, a task that requires unified feature extraction and accurate cross-modal correspondences. Standard MAE’s random masking may overlook key regions due to limited camera views, reducing registration effectiveness. To address this, we propose the Intermodal Dual-MAE Framework (ID-MAE) with a Similarity-based RL Masking Strategy (SRLM), which adaptively masks informative positions by leveraging cross-modal similarity and reinforcement learning, thus narrowing the modality gap. Our method enhances cross-modal representation learning by enforcing representation consistency during feature extraction, thereby enabling more reliable 2D-3D correspondence estimation. Experiments on RGB-D Scenes v2 and 7-Scenes benchmarks show that our method achieves state-of-the-art performance in image-to-point cloud registration.

[15] Aligned Consensus Teaching for Label-Efficient Oriented Object Detection in Weakly-Aligned Visible-Infrared Imagery cs.CVPDF

Qi Ming, Xiaxin Yuan, Jiahuan Zhou, Jiangmeng Li, Xudong Zhao

TL;DR: ACT面向弱对齐可见光—红外旋转框检测，以循环一致区域匹配对齐双模态、跨模态均值教师恢复分支漏检伪标签，并用VLM场景先验增强尾部类别；DroneVehicle仅用10%标注图像对就达到全监督mAP的94.3%，在多个标注比例上持续增益。

Details

Motivation: 可见光—红外目标检测需在成对图像中预测旋转框，传统方案依赖昂贵的双模态标注。半监督学习可减少标注量，但图像对级别仅少量双模态标注、其余完全无标注时，会遇到跨模态对齐不足、自训练中分支漏检累积及低资源下尾部类别标注极少等问题。

Result: 作者提出Aligned Consensus Teacher（ACT）。循环一致区域对齐CRA将循环一致性与稀疏锚点结合，并用可靠性加权区域匹配；跨模态共识均值教师CMC-MT在保持图像对关系的视图下形成共识伪标签，恢复单分支漏检并监督无标注图像对；文本引导跨模态实例增强TG-CMIA则用视觉语言场景先验合成尾部类别实例对，同时保留RGB—红外偏移。作者称ACT是首个研究该图像对级弱标注设定的半监督VIOD框架。在DroneVehicle和VEDAI上，不同标注比例均持续增益；DroneVehicle仅使用10%标注图像对时，mAP达到同一检测器全监督结果的94.3%。

Insight: 半监督可见光—红外检测需要联合解决几何配准、互补伪标签和尾部类别数据稀缺；将视觉语言先验用于实例合成补充了纯自训练。94.3%是相对同一检测器全监督mAP的比例，摘要未给绝对mAP或不同标注预算结果。

Abstract: Visible-infrared object detection (VIOD) detects objects with oriented bounding boxes from paired visible and infrared images. Existing methods depend on costly dual-modality annotations. Semi-supervised learning can reduce this burden, but extending it from single-modal detection to VIOD is challenging. In the practical image-pair-level setting considered here, only a few pairs are labeled in both modalities, while the rest are completely unlabeled. This limited supervision creates three challenges: (i) too few labeled boxes for robust cross-modal alignment; (ii) pseudo-label errors caused by branch-wise misses accumulate during self-training; and (iii) tail-class annotations become critically scarce as the labeling budget decreases. We propose Aligned Consensus Teacher (ACT) for label-efficient VIOD in this setting. Its Cycle-Consistent Region Alignment (CRA) combines cycle consistency and sparse anchors with reliability-weighted regional matching. Cross-Modal Consensus Mean-Teacher (CMC-MT) forms consensus pseudo labels under pair-preserving views to recover branch-wise misses and supervise unlabeled pairs. Text-Guided Cross-Modal Instance Augmentation (TG-CMIA) uses a vision-language scene prior to compose tail-class instance pairs while preserving RGB–IR offsets. To the best of our knowledge, ACT is the first framework to study semi-supervised VIOD under this image-pair-level setting. Experiments on DroneVehicle and VEDAI show consistent gains across annotation ratios. With 10% labeled pairs on DroneVehicle, ACT reaches 94.3% of the mAP obtained by the same detector under full supervision. Code and models will be available on GitHub to facilitate future work.

[16] MCLC-NET: Multimodal Continual Learning for Leaf Counting cs.CV | cs.LGPDF

Ruchi Bhatt, Pratibha Kumari, Shreya Bansal, Vedant Agnihotri, Dwarikanath Mahapatra

TL;DR: MCLC-NET是用于叶片计数的多模态持续学习框架，利用记忆缓冲区保留既往任务样本，并发布含RGB、深度及热成像的真实MMLC数据集，覆盖按作物、时间和混合排序的域增量场景；三种顺序下平均AMSE分别为0.675、0.542和0.745。

Details

Motivation: 叶片计数可用于作物表型监测和产量估计，但只依赖RGB图像的方法容易受遮挡、光照变化等现实因素影响；深度和热成像可提供互补信息。此外，农业数据随时间和来源逐步收集，假设所有训练数据同时可用并不符合实际。

Result: 作者提出多模态持续学习框架MCLC-NET，以记忆缓冲策略顺序学习不同任务并保留以往重要样本。论文还建立真实多模态叶片计数数据集MMLC，用于域增量持续学习，含RGB、深度和热成像，覆盖不同作物与环境条件，并按作物顺序、时间顺序及混合顺序组织任务。三个随机种子平均实验显示，MCLC-NET在三种顺序下均优于既有方法，AMSE分别为0.675±0.027、0.542±0.069和0.745±0.057。

Insight: 将多模态互补线索与记忆回放结合，更符合农业观测数据随时间累积的部署方式；不同任务顺序的误差差异也说明持续学习评估需考虑数据到达顺序。摘要未给出基线AMSE、数据集规模和各模态单独贡献。

Abstract: Leaf counting is an important task in plant phenotyping for monitoring plant growth and estimating crop yield. Most existing methods rely on RGB images, but their performance is often affected by occlusion, lighting variations, and other real-world challenges. Additional modalities, such as depth and thermal images, can provide useful complementary information. However, multimodal leaf counting remains underexplored. Also, many existing methods assume that all training data are available simultaneously, which is impractical in real agricultural settings, where data is collected over time from multiple sources. To address these challenges, we propose MCLC-NET, a multimodal continual learning framework for leaf counting. It learns tasks sequentially using a memory-based strategy with a memory buffer to retain important samples from previous tasks. We also introduce MMLC, a real-world multimodal leaf-counting dataset designed for a domain incremental scenario (DIS) in CL. It contains RGB, depth, and thermal images collected across different crop types under varying environmental conditions, arranged in three orderings: crop-wise, time-wise, and mixed. Experimental results, averaged over three random seeds, demonstrate that MCLC-NET consistently outperforms existing methods across all three task orderings, achieving the lowest AMSE of 0.675$\pm$0.027, 0.542$\pm$0.069, and 0.745$\pm$0.057, respectively.

[17] Multi-View Mixture-of-Experts with Vision-Language Reranking for Cross-View Object Geo-Localization cs.CVPDF

Xuyu Fan, Qi Ming, Zhu Han, Liuqian Wang, Siyuan Cao

TL;DR: MVLGeo统一处理无人机、街景等不同查询视角下的卫星目标定位，以共享编码器配合视角专用专家减少冗余，用视觉语言重排区分相似候选，并以自适应椭圆先验编码各向异性几何；在跨视角定位基准上取得先进表现。

Details

Motivation: 跨视角目标地理定位需根据无人机或街景查询在卫星图像中找到目标。现有方法为不同视角训练独立检测器，导致参数冗余并阻碍知识共享；卫星候选间视觉外观相似，仅靠类别标签和外观难以消除歧义。

Result: MVLGeo首先将查询视角的环境上下文文本作为线索，通过视觉语言重排区分相似卫星候选；其次采用多视角混合专家结构，共享编码器并为各查询视角配置专用专家，以减少冗余、促进知识共享，再用跨视角对比学习对齐表征；第三加入自适应椭圆先验作为辅助位置编码，建模各向异性几何。多个跨视角目标定位基准实验显示，MVLGeo作为统一多视角模型取得先进表现，并对输入退化具有鲁棒性、能跨视角泛化。摘要未列具体基准分数。

Insight: 共享专家结构提升跨视角知识利用，文本场景上下文则帮助区分视觉相似候选；几何先验补充方向相关的位置线索。三种机制分别处理模型冗余、候选语义歧义和各向异性几何，摘要未提供组件消融或具体增益。

Abstract: Cross-view object geo-localization (CVOGL) locates a target in satellite imagery using drone or street-view queries. Existing methods train separate detectors for each viewpoint, leading to parameter redundancy and impeding cross-view knowledge sharing. Moreover, top-ranked satellite candidates are often visually similar, so visual appearance and categorical labels alone are insufficient to resolve such ambiguity. To address these, we propose MVLGeo, an efficient framework designed to unify multiple viewpoints and reduce model redundancy. First, we introduce environmental contextual text from the query view as cues to distinguish visually similar candidates via Vision-Language Reranking (VL-Rerank). Second, we design a multi-view Mixture-of-Experts architecture (MV-MoE) with a shared encoder and view-specific experts to reduce redundancy and promote knowledge sharing, while cross-view contrastive learning aligns their representations for consistency. Third, we introduce an adaptive elliptical prior (ESAM-Prior) as auxiliary positional encoding for anisotropic geometric perception. Extensive experiments on the CVOGL benchmarks confirm that MVLGeo, as a unified model for multiple query viewpoints, achieves state-of-the-art performance, demonstrating robustness to input degradation and generalization across viewpoints. Code and models will be available on GitHub to facilitate future work.

[18] CapMap-MS-TTA: 3rd Place Solution for the MUMU Track of the 8th LSVOS Challenge at ECCV 2026 cs.CV | cs.AIPDF

Chengfeng Qiu, Kaifeng Wei

TL;DR: CapMap-MS-TTA是ECCV 2026大规模视频目标分割挑战MUMU赛道第三名的免训练方案，以约2.31亿参数Florence-2-base统一完成图像标签、开放词汇检测和英文描述；通过字幕关键词映射及多尺度翻转测试时增强，将复现基线公开分数由15.16提高到16.4815。

Details

Motivation: MUMU赛道要求单一多模态模型在严格资源限制（参数不超过0.5B、峰值GPU显存不超过8GB）下同时完成图像标签、开放词汇目标检测和英文图像描述。多任务统一和资源效率是实际挑战。

Result: CapMap-MS-TTA以约2.31亿参数的Microsoft Florence-2-base为基础，无需训练。描述任务使用原生详细描述路径并进行长度和token清理；图像标签任务将同一详细描述映射至官方质量、场景和事件词表，借助扩展关键词词典、全词匹配和轻量提示扩展；开放词汇检测使用原生检测路径，结合多尺度、水平翻转测试时增强，再进行标签感知非极大值抑制。系统无需微调，将作者复现的Florence-2基线分数15.16提高至最佳公开分数16.4815，并在最终MUMU排行榜获第3名。

Insight: 在资源受限的统一视频分割赛道中，原生模型能力配合任务映射和测试时增强即可带来竞争力，无需额外训练；但三项任务依赖不同输出后处理。摘要未给各任务单项分数和增强带来的计算代价。

Abstract: The MUMU track of the 8th Large-scale Video Object Segmentation (LSVOS) Challenge requires a single unified multimodal model to jointly solve image tagging (Task A), open-vocabulary object detection (Task B), and English captioning (Task C) under strict resource constraints (<=0.5B parameters and <=8 GB peak GPU memory). We present CapMap-MS-TTA, a training-free submission built on Microsoft Florence-2-base (~231M parameters), combining caption keyword mapping with multi-scale flip test-time augmentation. Task C uses the native pathway with length/token sanitization. Task A maps the same detailed caption into the official quality/scene/event vocabularies via an expanded keyword lexicon with whole-word matching and a lightweight expand-hints stage. Task B runs Florence-2 open detection () with multi-scale and horizontal-flip test-time augmentation (TTA), followed by label-aware non-maximum suppression (NMS). Without fine-tuning, the system improves our reproduced Florence-2 baseline from 15.16 to a best public score of 16.4815, and ranks 3rd on the final MUMU leaderboard.

[19] WISE: A Lightweight, Weakly-Supervised Model for Onboard Fire Smoke Detection and Localization cs.CVPDF

Sha Lu, Yu Sun, Liang Zhao, Jixue Liu, Lin Liu

TL;DR: WISE是面向卫星在轨运行的弱监督火烟检测与定位框架，用离线教师的空间软监督蒸馏轻量学生，单次前向同时预测瓦片级烟雾和像素概率图；已在ISS搭载IMAGIN-e载荷执行，每瓦片耗时0.10–0.26秒，最佳模型仅0.12M参数且瓦片/像素F1分别0.964/0.750。

Details

Motivation: 卫星影像野火烟雾检测对早期预警和应急响应重要，但在轨系统受内存和延迟严格约束。瓦片级分类计算高效却无法定位烟雾，像素分割虽提供详细掩码但通常难以实时在轨运行。

Result: WISE（弱监督高效烟雾提取）仅使用瓦片级标签，通过离线教师向轻量学生进行蒸馏，提供空间软监督。学生在一次前向中联合预测瓦片级烟雾存在与烟雾概率图，兼顾部署效率和空间定位。在ISS搭载的IMAGIN-e载荷上进行了在轨执行评测，三个模型版本每瓦片平均推理时间分别为0.10、0.14和0.26秒。Landsat 5和Landsat 8地面实验显示检测有效且定位信息较强。最佳版本仅0.12M参数、约3 GFLOPs，在10次运行中的平均瓦片级F1为0.964、像素级F1为0.750。

Insight: 教师提供的离线空间软标签可将瓦片级弱标注转化为定位能力，使轻量学生在保持低算力的同时输出像素热图；在轨运行提供实际部署证据。摘要未报告在轨数据规模、每种变体准确率以及不同传感器条件下的泛化。

Abstract: Wildfire smoke detection from satellite imagery is critical for early warning and rapid response. For onboard satellite deployment, detection systems must operate under strict memory and latency constraints while providing spatially informative outputs for downstream decision-making. Existing tile-level classification methods are computationally efficient but lack spatial localization, whereas pixel-level segmentation approaches provide detailed masks yet are typically too computationally demanding for real-time onboard execution. To address this gap, we propose WISE (Weakly-supervised Inference-efficient Smoke Extraction), a deployment-oriented framework for onboard fire smoke detection and localization. WISE leverages only tile-level annotations through a teacher-student distillation strategy, where an offline teacher provides soft spatial supervision to a lightweight WISE-Student optimized for efficient onboard inference. The student jointly predicts tile-level smoke presence and smoke probability maps within a single forward pass, enabling spatially informative detection under strict computational constraints. WISE was evaluated through in-orbit execution aboard the ISS-mounted IMAGIN-e payload. Three model variants achieve average inference times of 0.10 s, 0.14 s, and 0.26 s per tile, indicating near-real-time per-tile inference within onboard resource limits. Ground-based experiments on Landsat 5 and Landsat 8 imagery further indicate effective detection and spatially informative localization. The best-performing variant achieves a mean tile-level F1 score of 0.964 and a mean pixel-level F1 score of 0.750 across 10 runs, while containing only 0.12M parameters and requiring approximately 3 GFLOPs. Together, these results indicate that WISE is a practical candidate for low-latency wildfire smoke monitoring from space under onboard resource constraints.

[20] Understanding Dynamic Scenes at Gigapixel Scale: Wide-Area Spatio-Temporal Perception from UAVs cs.CVPDF

Yuhang Zhu, Meiyi Zhu, Yunkai Dang, Zhangnan Li, Yuxuan Wang

TL;DR: HARD是面向无人机千兆像素动态场景理解的数据集，使用12,768×9,564高分辨率航拍图像并标注检测、跟踪和场景级VQA；作者提出延迟感知跟踪指标s-HOTA，发现高延迟会更重地惩罚关联环节，且VLM难以跨帧绑定目标身份。

Details

Motivation: 无人机成像从百万像素迈向千兆像素，使空中感知从识别单个目标转向理解整个动态场景。宽域覆盖、目标级分辨率和时间连续性需同时满足，但现有数据集缺少对这类能力的支持；超高分辨率还会使单帧处理耗时达到数秒，评估不能忽略延迟。

Result: 作者将任务定义为广域时空场景理解WSTU，并发布超高分辨率航空遥感数据集HARD，图像尺寸为12,768×9,564，包含物体检测、多目标跟踪及场景级视觉问答三层标注。针对高分辨率下延迟不可忽视的问题，提出多目标跟踪延迟感知指标streaming-HOTA（s-HOTA）。基线实验发现，检测任务的端到端管线对速度和准确率的影响可与检测器本身相当；跟踪中高延迟对关联轴的惩罚远高于检测轴，离线最佳管线在s-HOTA下可能失去领先。VQA方面，VLM在跨帧身份绑定上较弱，无法将单帧能力增益迁移至跨帧任务。所评基线整体尚未达到WSTU要求。

Insight: 千兆像素视频理解不能只看离线识别准确率，完整管线延迟会改变跟踪关联质量及方法排名；跨帧身份保持也是VLM重要短板。HARD提供数据与系统基线，摘要未报告数据集规模、各任务具体指标和VQA分数。

Abstract: UAV-borne imaging has advanced from megapixel to gigapixel sensors, shifting aerial perception from recognizing individual targets to understanding entire dynamic scenes. We characterize this demand as Wide-area Spatio-temporal Scene Understanding (WSTU), which requires wide-area coverage, per-target resolution, and temporal continuity at once, a combination existing datasets lack. To fill this gap, we introduce an ultra-High-resolution (12768x9564) Airborne Remote-sensing Dataset (HARD) annotated at three levels for object detection, multi-object tracking, and scene-level visual question answering. Ultra-high-resolution imagery raises per-frame processing time to seconds. At that scale latency can no longer be ignored in evaluation. Thus, we propose a latency-aware metric for multi-object tracking called streaming-HOTA (s-HOTA). Extensive baseline experiments show how ultra-high-resolution processing reshapes each task. For detection, the end-to-end pipeline affects accuracy and speed as much as the detector itself does. For tracking, high latency charges the association axis far more unevenly than the detection axis, and association is where pipelines diverge. As a result, the pipeline that performs best offline can lose its lead under s-HOTA. For VQA, vision-language models remain weak at cross-frame identity binding and cannot transfer their single-frame gains to it. Together these findings show that the baselines we evaluate fall short of WSTU. HARD provides the data and the systematic baselines to advance it.

[21] MS-RFD: Multi-Signal Release Frame Detection in Hammer Throw from Reconstructed 3D Trajectories cs.CVPDF

Ahmed Endris Hasen, Nikolaos Passalis, Tomi Vanttinen, Jenni Raitoharju

TL;DR: MS-RFD利用重建的3D链球轨迹，融合速度变化、角速度转折、相对旋转中心的径向距离及释放后轨迹线性度，自动检测链球释放帧；投掷距离估计和消融分析显示速度动态与径向扩张贡献最大。

Details

Motivation: 链球运动表现高度依赖释放瞬间的速度、角度和高度，但从视频中确定释放帧通常需人工逐帧检查，主观且繁琐，不适合真实训练场景中的自动分析。

Result: 作者提出多信号释放帧检测方法MS-RFD，输入为重建的三维链球轨迹。方法综合四类运动学线索：速度动态、角速度转换、相对旋转中心的径向距离，以及释放后轨迹线性度；这些线索用于为候选释放帧打分和验证。评估依据检测帧估计的释放参数所产生的投掷距离误差，并通过消融分析各信号和不同候选选择策略。结果显示，速度变化和径向扩张是最强的释放线索，角速度及释放后轨迹线性度提供进一步但较小的改进。摘要未报告具体距离误差或检测精度。

Insight: 释放帧可由多种互补的三维运动变化共同判定，不必依赖单一阈值；速度和径向扩张在此任务中最具判别性。性能评估以最终投掷距离误差为依据，但摘要缺少误差数值及视频来源。

Abstract: Recent advances in artificial intelligence and computer vision are reshaping sports performance analysis by enabling automated detection, tracking, and performance analysis. In hammer throw, performance is strongly determined by the kinematic conditions at release, particularly release speed, release angle, and release height. However, identifying the release instant from video typically requires manual frame-by-frame inspection, which is subjective and cumbersome in real-world training scenarios. In this paper, we present a fully automatic multi-signal release frame detection (MS-RFD) method for hammer throw using reconstructed 3D hammer trajectories. The proposed method integrates four complementary kinematic signals: speed dynamics, angular velocity transition, radial distance relative to the rotation center, and post-release trajectory linearity. These signals are fused to score and verify candidate release frames. MS-RFD is evaluated through the throwing-distance estimation error obtained from the release parameters estimated at the detected frame. An ablation study analyzes the contribution of each signal and compares alternative candidate selection strategies. The results show that speed dynamics and radial expansion provide the strongest signals for release frame detection, while angular velocity and post-release linearity provide smaller refinements.

[22] Decoder-Agnostic Token Merging for Vision Transformers: A Systematic Study of G2TM cs.CVPDF

Victor Bercy, Martyna Poreba, Michal Szczepanski, Samia Bouchafa

TL;DR: 该研究系统考察图引导token合并G2TM在不同ViT解码器与分类任务中的通用性，发现其效果主要由编码器决定、而非配套解码器；ADE20K分割上计算量降低22%–47%、吞吐最高提升74%，分类任务准确率退化更小。

Details

Motivation: 视觉Transformer的自注意力复杂度随token数平方增长，是效率和规模化部署的主要瓶颈。token合并可聚合冗余视觉token，但既有评估多绑定单一架构，无法判断收益来自合并机制本身还是特定解码器。

Result: 研究将图引导token合并G2TM作为单一模块插入ViT早期层，比较Segmenter、SETR和EoMT三种语义分割框架及线性、Transformer、卷积三类解码器，并评估标准ViT图像分类。对固定骨干规模，G2TM在所测架构中的行为及准确率—效率折衷一致，说明其有效性主要是编码器属性而非解码器特性；迁移到分类时准确率退化比语义分割更小。最佳超参数主要取决于骨干预训练方案和目标数据集，而不是解码器；ADE20K上分割GFLOPs降低22%–47%，吞吐最高提高74%。

Insight: token合并模块可跨不同解码器复用，效果主要由编码器和预训练特性决定；跨架构系统评估能排除单一搭配偶然奏效的可能。效率收益是计算量与吞吐提升，摘要未给出各配置的精度损失数值。

Abstract: Vision Transformers (ViTs) have achieved state-of-the-art performance across a range of computer vision tasks, mainly thanks to the self-attention mechanism. However, its complexity, increasing quadratically with the number of tokens, remains the major obstacle to ViT efficiency and deployment at scale. Token merging reduces this cost by aggregating redundant tokens. Yet existing methods are typically evaluated within a single architecture, leaving open whether their effectiveness stems from the merging mechanism itself or from the specific decoder they are paired with. We extend Graph-Guided Token Merging (G2TM), a single module inserted early in a ViT-based network, beyond its original Segmenter setting. We evaluate G2TM across three semantic segmentation frameworks (Segmenter, SETR, EoMT) and three decoder families (Linear, Transformer-, convolution-based), as well as standard ViT image classification. Our results show that G2TM’s behavior and accuracy-efficiency trade-off are consistent across every tested architecture for a given backbone size, indicating that its effectiveness is a property of the encoder rather than the decoder. G2TM also generalizes well to image classification, achieving an even smaller degradation in accuracy compared to semantic segmentation. We further find that G2TM’s optimal hyperparameters, resulting in a consistent drop in GFLOPs of 22-47% and an increase in throughput by up to 74% for segmentation models on ADE20K dataset, depend primarily on the backbone’s pre-training recipe and on the target dataset, rather than on the decoder choice.

Haoyu Zhao, Zihao Zhao, Tianyu Deng, Ziqin Xu, Zihao Zhang

TL;DR: 该研究用517个实例评估全模态生成模型MiniMax-H3的物理世界推理，设计隐式提示配多帧、音频—图像、前缀视频及音频—视频四类互补输入；整体成功率41.97%，视频决策推理最好为56.00%，音频消歧最弱为27.40%。

Details

Motivation: 全模态模型统一生成文本、图像、视频和音频，但多模态对齐是否改善物理世界推理、以及如何利用全模态输入进行评测仍不清楚。现有视频生成和世界模型评估输入模态有限，提示常与目标视频内容高度匹配，无法检验模型整合互补线索的能力。

Result: 论文围绕物理世界推理建立四个互补维度的评估框架，设计需要跨模态整合的任务：隐式提示与多帧组合、音频—图像、前缀视频及音频—视频。每种模态单独只提供关于底层事件的部分证据，模型需结合互补语义线索推断潜在事件状态和未来动态。517个评测实例上，MiniMax-H3总体成功率为41.97%；视频决策推理最高为56.00%，音频消歧最低为27.40%。结果显示，充分整合多模态输入仍是发挥全模态模型优势的关键。

Insight: 全模态评测应确保单个模态信息不足，必须融合音频、图像或视频线索才能推断事件；当前MiniMax-H3在此条件下仍有明显困难，尤其是音频消歧。摘要未列各任务样本量和其他全模态模型对比。

Abstract: Recent Omni-Modal Generative Models (Omni-Models) have advanced content generation toward unified modeling of text, images, video, and audio. MiniMax-H3 exemplifies this transition by combining multimodal context understanding with joint audio-visual generation in a shared latent framework. Its unified architecture raises a fundamental question: Can multimodal alignment improve the model’s world reasoning, and what new evaluation paradigms do omni-modal inputs enable? To investigate this question, this work introduces a comprehensive evaluation framework organized around four complementary dimensions of physical world reasoning. Unlike existing evaluation frameworks for video generation and world models, which are often constrained by limited input modalities and evaluation settings where prompts closely match the target video content, our evaluation is specifically designed to exploit the multimodal inputs of Omni-Model. We construct a diverse set of novel tasks that require models to integrate complementary information across modalities. Specifically, we consider four scenarios, including implicit prompts paired with multiple frames, audio-image, prefix-videos, and audio-video inputs. Every single modality provides only partial evidence about the underlying event, requiring the model to jointly reason over the complementary semantic cues to infer latent event states and future dynamics. Across 517 evaluation instances, MiniMax-H3 achieves an overall success rate of 41.97%. Video-based Decision Reasoning yields the highest success rate at 56.00%, while Audio-based Disambiguation Reasoning is the weakest, reaching only 27.40%. These results indicate that effective multimodal integration remains key to fully exploiting the benefits of diverse input modalities. The project is available at https://github.com/gulucaptain/MiniMax-H3-Reason.

[24] Visual Input and Its Framing Affect Attribute-based Descriptions Produced by Large Vision-Language Models cs.CVPDF

Xiaomeng Wang, Martha Larson, Zhengyu Zhao

TL;DR: 该研究发现LVLM即使只被要求描述某一概念类别的属性，输入图像中具体实例的存在及其取景方式仍会改变回答；受试者聚焦构图使物理属性词占比从纯文本18%升至45%，情境构图为40%，揭示图像线索及框架对属性描述鲁棒性的影响。

Details

Motivation: 大型视觉语言模型通常只接收文本提示，或文本加图像。作者研究当图像存在、但提示只询问图像中所属类别的概念层面属性而非特定实例时，视觉内容是否仍会改变回答，以及实例在画面中的构图如何影响描述。

Result: 研究以狗品种属性描述为例：即使提示仅询问某个犬种的一般属性，输入图像中的具体该品种犬只仍会改变回答；图像中的实例如何取景，则决定回答偏移方向。细分分析显示，物理属性词在纯文本回答中占18%，在主体聚焦构图下升至45%，在主体置于情境构图下为40%。作者据此指出LVLM对视觉线索存在意外依赖，评估模型鲁棒性时需考虑图像是否存在及实例取景方式。

Insight: 图像可能在模型回答概念层面的通用问题时悄然引入实例级属性偏差，画面突出对象还是将其置于环境中也会改变描述内容；评估时需控制图像构图。摘要未说明数据规模、品种数量及统计显著性。

Abstract: Large vision-language models (LVLMs) are commonly used with only a single text prompt as the input, or plus an image. In this paper, we demonstrate that when the image exists, even if the text prompt is not about the specific instance (but only the concept it belongs to) in that image, the response would still be affected. For example, when the text prompt only asks for the attribute descriptions of a dog breed, an image depicting a specific dog from that breed would shift the response. Further, how the specific instance is framed in that image would determine towards which the response shifts. Detailed analyses also reveal that in the response, physical terms increase from 18% for text-only to 45% (40%) for subject-focused (subject-in-situation) framings. Overall, the unexpected effects of visual cues on LVLMs highlight the need to understand the presence of an image and its framing when evaluating the robustness of LVLMs.

[25] MSR: Multiple Subject Reference for Video Generation cs.CVPDF

Guannan Li, Jiaji Chen, Jingyuan Liao, Yu Geng, Baolan Qiu

TL;DR: MSR为LTX视频生成引入具备槽位意识的多主体参考条件：每张参考图单独编码为静态片段和潜token组，并以槽位嵌入及时间偏移保持对象身份；通过LoRA实现，定性展示多角色与参考场景组合，但相似服饰、复杂衣物和视角变化仍具挑战。

Details

Motivation: 多图像条件视频生成需要保留每个参考主体的外观，同时确保每张参考图对应其预期角色；若参考对象混淆，会破坏多主体视频中的身份和场景一致性。

Result: MSR（多主体参考）为基于LTX的视频生成提供槽位条件机制。每张参考图独立编码为静态片段并对应独立潜token组；紧凑傅里叶特征MLP加入数字槽位嵌入，槽位相关时间偏移再改变该组旋转位置坐标。参考组前置到带噪目标token之前，在仅对目标内容进行流匹配训练时作为干净上下文。作者通过低秩适配实现该机制，并发布权重和推理工作流。定性案例展示真实与风格化场景中的不同角色及参考环境组合；开发观察显示相较早期连续参考基线，参考混淆减少，但相似服饰、复杂服装和视角变化仍困难。补充音频参考实验在冻结视觉参数的条件下加入声音条件。

Insight: 为每个参考图像分配独立潜槽及时间坐标，可显式保留参考对象角色，减少多主体生成中的身份混淆。摘要主要提供定性证据，没有定量质量指标或相似服饰场景的错误率，能力范围仍有限。

Abstract: Conditioning a video generator on multiple images requires preserving appearance while associating each reference with its intended role. We present MSR (Multiple Subject Reference), a slot-aware conditioning scheme for LTX-based video generation. Each reference image is independently encoded as a static clip and represented by a separate latent-token group. A compact Fourier-feature multilayer perceptron adds a numeric slot embedding, while slot-dependent temporal offsets modify the group’s rotary coordinates. The reference groups are prepended to noisy target tokens and serve as clean context during target-only flow-matching training. We implement this scheme through low-rank adaptation and release the resulting weights and inference workflows. Qualitative examples demonstrate compositions containing distinct characters and referenced environments in realistic and stylized scenes. Development observations suggest reduced reference confusion relative to an earlier continuous-reference baseline, while similar clothing, complex garments, and viewpoint changes remain challenging. We describe the conditioning mechanism, the retained training configuration, and the observed strengths and limitations of the released system. A supplementary audio-reference experiment adds voice conditioning while keeping the visual parameters frozen.

[26] Prosthesis-Aware 3D Human Pose Estimation: A Dataset and Benchmark for RSP Users cs.CVPDF

Yilin Wen, Kechuan Dong, Fumiya Suginaka, Ken Endo, Yusuke Sugano

TL;DR: 该研究发布首个跑步专用假肢使用者三维数据集RSP3D，覆盖不同截肢条件下的日常与运动动作，并提出假肢感知3D姿态估计任务及混合基线：人体关节用模型式方法估计，假肢形状用无模型方法恢复。

Details

Motivation: 从视频恢复人体三维动作可支持康复评估和运动表现分析，但假肢使用者还需同时恢复自然身体关节和假肢几何。传统人体模型基于非截肢者训练，无法表示假肢形状；无模型方法又缺乏人体运动学先验，在遮挡下不可靠。跑步专用假肢曲线复杂且运动显著，尤其难处理。

Result: 作者收集RSP3D，称其为首个跑步专用假肢使用者3D数据集，涵盖不同截肢条件参与者的日常生活和运动动作，使用多摄像机、标记点动作捕捉系统采集。论文形式化定义假肢感知3D姿态估计任务，并在零样本设置下评估代表性方法，确认各自局限。作者另提出混合基线：人体关节通过基于身体模型的方法估计，假肢形状则由无模型方法恢复，为后续研究提供起点。摘要未给数据集规模和精度结果。

Insight: 假肢感知姿态估计不能只套用健康人体骨架模型，也不能仅靠无模型几何；将人体运动学与设备形状恢复分开处理提供了互补路线。论文主要贡献是新数据集、任务定义和初始基线，摘要没有量化性能。

Abstract: Recovering 3D human body motion from video is important for applications such as rehabilitation assessment and sports performance evaluation. For prosthesis users, this requires capturing both natural body joints and the geometry of the prosthetic device, a challenge that existing methods are not designed to address. Model-based estimators rely on body models trained on non-amputee individuals and cannot represent prosthesis geometry, while model-free methods lack body kinematic priors and are unreliable under occlusion. This challenge is particularly prominent for users of running-specific prostheses (RSPs), where the RSP has a complex curved geometry and moves dynamically during exercise. To fill this gap, we collect RSP3D, the first 3D dataset of RSP users, covering essential daily-life and exercise actions from participants with varied amputation conditions, using a multi-camera marker-based motion capture setup. We formally define the task of prosthesis-aware 3D pose estimation, evaluate representative methods in a zero-shot setting, and confirm their individual limitations. We further propose a hybrid baseline combining model-based body joint estimation with model-free RSP shape recovery, establishing a starting point for future research.

[27] Vocabulary-Guided Gait Recognition cs.CVPDF

Panjian Huang, Saihui Hou, Chunshui Cao, Xu Liu, Yongzhen Huang

TL;DR: Gait-World探索用VLM词汇知识辅助步态识别：以步态周期动作词汇桥接人体运动和语言表征，并由α-Gait中的词汇关系映射器与细粒度检测器完成跨空间对齐；在CASIA-B、CCPG、SUSTech1K、Gait3D和GREW上评估，显示词汇信息具有潜力。

Details

Motivation: 外观式步态网络通常将步态视为图像中的人体形状和运动，模型式网络则视为点云中的人体内在结构，但这些表述对人类理解仍较模糊。仅凭有限步态数据学习普适特征也较困难；作者探索视觉语言模型所含词汇知识能否提升可解释和可泛化的步态表征。

Result: Gait-World提出词汇引导步态识别范式，通过精心选择的步态周期动作作为词汇基础，桥接步态和语言特征空间。具体模型α-Gait用VLM中的词汇知识引导步态网络学习。由于词汇和步态表征处于不同嵌入空间，方法设计词汇关系映射器和步态细粒度检测器，在步态空间中建立词汇关系并检测对应运动特征。在CASIA-B、CCPG、SUSTech1K、Gait3D和GREW五个数据集上的实验显示，VLM词汇信息在步态领域具有潜在价值，并为后续研究提供方向。摘要未列准确率和基线分数。

Insight: 将可解释的动作词汇映射到步态特征空间，可为以往较难理解的运动表示提供语义锚点；不过不同模态嵌入差异需专门适配。摘要仅报告多数据集评估及潜力，没有给出定量增益，不能判断提升幅度。

Abstract: What is a gait? Appearance-based gait networks consider a gait as the human shape and motion information from images. Model-based gait networks treat a gait as the human inherent structure from points. However, the considerations remain vague for humans to comprehend truly. In this work, we introduce a novel paradigm Vocabulary-Guided Gait Recognition, dubbed Gait-World, which attempts to explore gait concepts through human vocabularies with Vision-Language Models (VLMs). Although VLMs have achieved the remarkable progress in various vision tasks, the cognitive capability regarding gait modalities remains limited. The success element in Gait-World is the proper vocabulary prompt where this paradigm carefully selects gait cycle actions as Vocabulary Base, bridging the gait and vocabulary feature spaces and further promoting human understanding for the gait. How to extract gait features? Although previous gait networks have made significant progress, learning solely from gait modalities on limited gait databases makes it difficult to learn universal gait features for practicality. Therefore, we propose the first Gait-World model, dubbed α-Gait, which guides the gait network learning with vocabulary knowledge from VLMs. However, due to the heterogeneity of the modalities, directly integrating vocabulary and gait features is highly challenging as they reside in different embedding spaces. To address the issues, α-Gait designs Vocabulary Relation Mapper and Gait Fine grained Detector to map and establish vocabulary relations in the gait space for detecting corresponding gait features. Extensive experiments on CASIA-B, CCPG, SUSTech1K, Gait3D and GREW reveal the potential value and research directions of vocabulary information from VLMs in the gait field.

[28] DR.WILSS: Diffusion-Based Replay for Weakly Supervised Continual Semantic Segmentation cs.CVPDF

Leon Arthur Marx, Francesco Barbato, Matteo Caligiuri, Pietro Zanuttigh

TL;DR: DR.WILSS用语言线索引导扩散模型生成回放数据，通过自修复和正则化保持已学类别，从而缓解弱监督类增量语义分割的灾难性遗忘；多个基准和生成架构上达到先进表现，训练时无需保存原训练数据或增加高成本工具。

Details

Motivation: 弱监督类增量语义分割要求模型在多个阶段持续学习新类别，每阶段只有图像级标签；模型易遗忘先前类别，而保存旧数据又受存储和隐私限制。

Result: DR.WILSS提出基于扩散生成回放的持续学习方法。框架使用语言线索引导扩散过程，并采用自修复及正则化技术，高效生成旧类别回放样本，以在增量更新中保留既有类别知识。为使回放图像分布更接近训练样本，作者对生成模型使用LoRA。多个基准和生成架构实验显示，该方法达到先进表现，且无需存储原训练数据，也不需训练时额外使用资源密集型工具。作者称方法在训练复杂度与推理准确率间取得良好折衷；摘要未提供分数及遗忘率。

Insight: 扩散生成回放可在不保存旧图像的情况下为后续类别学习提供历史视觉样本，语言条件和LoRA有助于对齐生成数据分布。具体效果取决于合成样本质量；摘要没有列出基准、类别顺序及相较回放基线的增益。

Abstract: Weakly supervised class-incremental semantic segmentation (WILSS) aims to train a segmentation model over multiple steps, each introducing new concepts to be learned with only image-level supervision. We introduce DR$.$WILSS, an innovative approach to address catastrophic forgetting in continual learning using diffusion-based generative replay. Our framework leverages language clues to guide the diffusion process, employing self-inpainting and regularization techniques to efficiently produce replay data, aiding the learning process. By generating high-quality replay data, the information from previously learned classes can be preserved during continual updates, a critical challenge in incremental learning scenarios. To further align the statistics of replay data with those of training samples, we apply LoRAs to the generative model. Experimental results demonstrate state-of-the-art performance across multiple benchmarks and generative architectures, while avoiding storage of training data and the use of additional resource-demanding tools during training. The proposed technique enables an optimal tradeoff between training complexity and inference-time accuracy, making DR$.$WILSS a promising solution for real-world applications.

[29] StrucPhysVideo: Learning Physical Dynamics from Structured Captions and Robot Actions cs.CVPDF

Awomo-WM Team, :, Enhui Ma, Kaiwen Guo, Tingrui Zhang

TL;DR: StrucPhysVideo通过运动感知分割、物理相关过滤和结构化对象/材料/交互标注构建物理视频数据，训练文本—图像到视频模型并扩展为动作条件世界模型；Physics-IQ Verified得分45.5%，较Cosmos3-Super-Image2Video高2.8个百分点，四步去噪支持机器人动作逐步rollout。

Details

Motivation: 具身AI世界模型需要理解物体如何运动、交互和改变状态。普通视频生成数据不一定明确标注接触、形变、状态转移，也容易把相机运动和物体动态混淆，限制物理推理及动作条件生成。

Result: StrucPhysVideo构建一套物理视频世界模型及数据管线，使用运动感知视频分割、质量与内容筛选和物理相关性验证，并对对象、材料和带时间定位的交互作结构化标注。流程将相机运动与对象行为分离，明确描述接触、形变和状态变化。基于该数据，StrucPhysVideo-TI2V是稀疏混合专家文本—图像到视频模型，训练课程逐步强调物理动态，同时保留通用视频数据。在Physics-IQ Verified上得分45.5%，较Cosmos3-Super-Image2Video高2.8个百分点；跨骨干的字幕消融也支持物理监督有效。进一步扩展为StrucPhysVideo-IA2V交互式图像—动作到视频世界模型，以机器人末端执行器命令预测视觉结果；结合动作条件、因果自回归生成和少步蒸馏，仅用四个去噪步骤即可逐步生成机器人rollout。

Insight: 结构化描述可将视频中的可观测物理事件转为模型监督，并通过动作条件生成从被动视频预测延伸到交互式机器人世界模型。Physics-IQ上的领先幅度有限；摘要未说明动作任务实测部署或跨机器人泛化。

Abstract: Modeling physical dynamics, including how objects move, interact, and change state, is central to video world models for embodied AI. We present StrucPhysVideo, a family of video world models that bridges physics-focused data curation with language- and action-conditioned prediction of scene evolution. Our data pipeline combines motion-aware video segmentation, quality and content filtering, and physical relevance verification with structured annotations of objects, materials, and temporally localized interactions. By disentangling camera motion from object behavior and explicitly describing contact, deformation, and state transitions, the pipeline provides supervision grounded in observable physical events. Building on these data, we introduce StrucPhysVideo-TI2V, a sparse Mixture-of-Experts (MoE) text-image-to-video model trained with a curriculum that progressively emphasizes physical dynamics while retaining general-domain video data. StrucPhysVideo-TI2V achieves state-of-the-art performance on Physics-IQ Verified, scoring 45.5% and outperforming Cosmos3-Super-Image2Video by 2.8 percentage points. Caption ablations across backbones further demonstrate the effectiveness of physics-focused supervision. We further extend StrucPhysVideo-TI2V to StrucPhysVideo-IA2V, an interactive image-action-to-video world model that predicts visual outcomes from robot end-effector commands. Action conditioning, causal autoregressive generation, and few-step distillation enable incremental robot rollouts with only four denoising steps. Together, StrucPhysVideo advances physical dynamics modeling from image- and language-conditioned video prediction toward action-driven interaction.

[30] CSWAM: Better Causal Semantic Representations for Out-of-Distribution Generalization in World Action Models cs.CV | cs.AIPDF

Tianbin Liu, Jian Zhu, Taiyi Su, Jianjun Zhang, Chong Ma

TL;DR: CSWAM在FastWAM上增加基于V-JEPA 2.1的因果语义专家，从稀疏观察历史学习语义状态变化与运动，并将上下文提供给视频和动作分支；RoboTwin 2.0从干净到随机场景成功率由10.16%升至45.18%，真实机器人两项任务OOD平均成功率由27.5%升至70.0%。

Details

Motivation: FastWAM式世界—动作模型推理时只需生成动作，但在视觉分布变化下泛化较差；重建型表征过度关注外观细节，难迁移到新场景和物体。没有观察历史时，模型也缺乏判断陌生视觉条件下任务状态变化和运动所需的时间证据。

Result: 因果语义世界—动作模型CSWAM在FastWAM上增加基于V-JEPA 2.1的因果语义专家。该专家用较少依赖外观细节、具有时间接地的表征描述语义状态变化和运动，并从当前及过去稀疏观察中学习未来演化；通过因果注意力，将历史上下文共享给视频和动作分支。推理时动作去噪以当前视频状态及观察到的语义历史为条件，保留高效动作单独推理。仿真和实机OOD实验中，具身预训练后RoboTwin 2.0从Clean到Randomized迁移的Randomized成功率由FastWAM的10.16%升至45.18%，提高35.02个百分点；两项真实机器人任务、三个OOD难度下平均成功率由27.5%升至70.0%，提高42.5个百分点。

Insight: 在动作模型中加入时间语义状态变化和稀疏历史，可缓解外观分布偏移下的动作规划退化，而无需放弃动作单独推理的效率。实验报告仿真和真实机器人收益，但摘要未列各实机任务、样本数和不确定区间。

Abstract: FastWAM-style world action models enable efficient action-only inference, but generalize poorly under visual distribution shifts. Their reconstruction-oriented representations emphasize appearance-specific details, limiting generalization to unseen scenes and objects. Without observation history, the model also lacks temporal evidence for robustly identifying task-relevant state changes and motion in unfamiliar visual conditions. To address these limitations, we present the Causal Semantic World Action Model (CSWAM), which augments FastWAM with a causal semantic expert built on V-JEPA 2.1. V-JEPA provides temporally grounded representations of semantic state changes and motion with less dependence on appearance-specific details. The expert learns their future evolution from a sparse history of current and past observations and shares the history-derived context with both the video and action streams through causal attention. At inference, CSWAM conditions action denoising on the current video state and observed semantic history, retaining efficient action-only inference. We conduct simulation and real-robot experiments to evaluate generalization under distribution shifts. With embodied pretraining, CSWAM raises Randomized success on RoboTwin 2.0 Clean-to-Randomized transfer from 10.16% to 45.18%, a gain of 35.02 percentage points over FastWAM. Across two real-robot tasks and three OOD difficulty levels, CSWAM improves average success over FastWAM by 42.5 percentage points, from 27.5% to 70.0%.

[31] Learning from Distributed Eyes: Leveraging Collaborative Perception for Automated Model Adaptation cs.CV | cs.LGPDF

Yanan Ma, Yihang Tao, Zhengru Fang, Zihan Fang, Yiqin Deng

TL;DR: LDE将自动驾驶中的协同感知转作无监督域适配监督，以其他车辆的协同感知伪标签改善自车检测器；通过适配导向特征共享、视野过滤和课程学习应对带宽限制、视角错配与伪标签噪声，在多个3D检测实验中超过预训练模型及先进无监督适配方法。

Details

Motivation: 自动驾驶感知模型易受新环境域偏移影响。无监督模型适配无需昂贵人工标注，但只使用自车数据的现有方法伪标签质量较差；协同感知可汇集多车信息，却面临通信、视角和标签可靠性问题。

Result: LDE（从分布式视角学习）将协同感知视为高质量模型适配监督来源，利用协同感知生成的伪标签训练目标模型。该策略假设多车协同感知通常优于单车感知，并设计三项处理机制：选择最关键适配信息进行特征共享以缓解通信带宽限制；视野过滤清除协同视图与学习者视野不一致的标签；课程学习逐步利用伪标签，降低不可靠监督影响。在多项3D目标检测实验中，LDE持续超过预训练模型及先进无监督适配方法。摘要未列数据集和量化指标。

Insight: 协同车辆的观测可为目标车辆提供跨视角伪监督，提升域适配，但必须先筛选通信内容、剔除视野不匹配标签并控制伪标签误差。实验只定性报告多项3D检测提升，摘要未给具体增益和通信成本。

Abstract: In autonomous driving, perception models often struggle to generalize to new environments due to domain shifts. While unsupervised model adaptation offers a feasible solution without labor-intensive manual labeling, existing methods that rely solely on the ego-vehicle’s data often lead to inferior pseudo-labeling performance. To address this critical issue, we propose LDE, Learning from Distributed ``Eyes”, a novel framework that transforms collaborative perception (CP) into a source of high-quality supervision for model adaptation. This pseudo-labeling approach is hyperparameter-insensitive and relatively reliable, assuming CP often outperforms single-agent’s perception. However, naively implementing this approach encounters (1) the communication bottleneck of sharing rich features under time and bandwidth constraints, (2) the view discrepancy between the CP view and the learner’s Field of View (FoV), and (3) the unreliability even in CP-generated labels. To address these issues, we design an adaptation-oriented feature sharing mechanism that selectively transmits the most critical information for adaptation, an FoV filtering method that meticulously eliminates mismatched labels, and a curriculum learning strategy to progressively exploit pseudo labels. Extensive experiments on 3D object detection tasks demonstrate that LDE consistently outperforms both the pre-trained models and state-of-the-art unsupervised adaptation methods.

[32] SVMemAgent: A Streaming Video Memory Agent for Query-Agnostic Online Frame Selection cs.CVPDF

Dohwan Ko, Ji Soo Lee, Pierce Chuang, Debojeet Chatterjee, Ashish Shenoy

TL;DR: SVMemAgent面向未知时长、无查询预告的流式长视频，逐帧决定替换现有记忆帧或丢弃新帧；通过多样问答任务奖励的GRPO训练，学习保留通用信息帧，在在线与离线视频基准上超过在线选择基线并接近可访问完整视频和查询的离线方法。

Details

Motivation: 多数关键帧选择研究假设事先可获得完整视频及查询；真实流式场景中视频时长未知、选择时无法看到查询或未来帧，因此需要在线维护能服务未来未知问题的紧凑视频记忆。

Result: SVMem将已观察内容压缩为持续更新的紧凑代表性记忆。SVMemAgent在每个时间步决定用新帧替换已有记忆帧或直接丢弃新帧。训练使用GRPO，并通过多种问答任务产生任务驱动奖励；训练时接触不同查询分布，使推理阶段即使没有查询，也能保留一般信息量较高的帧。在在线和离线视频基准上，该方法持续超过在线关键帧选择基线，并达到假设能访问完整视频和查询的离线方法的竞争水平。任务奖励还使策略涌现偏好含文字信息帧的选择行为，可能有利于下游视频问答。

Insight: 在无查询条件下学习通用帧价值，可将未来多种问答任务的需求隐式折叠进流式记忆策略；多任务奖励使文字等信息性线索成为重要选择依据。摘要未给基准名称、记忆预算和定量结果。

Abstract: Most keyframe selection studies focus on offline settings, assuming access to the full video and query in advance. In contrast, real-world streaming scenarios require online frame selection under unknown video duration, without access to either the query or future frames during selection. To address this, we introduce Streaming Video Memory (SVMem), a compact and representative memory of previously observed content, updated continuously as the video stream unfolds. Building on this setting, we propose the Streaming Video Memory Agent (SVMemAgent), which dynamically maintains a memory by deciding at each timestep whether to replace an existing memory frame with the incoming frame or discard it. SVMemAgent is trained using Group Relative Policy Optimization (GRPO) with task-driven rewards derived from diverse question-answer pairs, implicitly exposing the policy to a distribution of queries during training so that SVMem retains generally informative frames at inference, when queries are unavailable. Experiments on both online and offline video benchmarks show that SVMemAgent consistently outperforms online frame selection baselines and achieves competitive performance with offline methods that assume access to the full video and query. Through task-driven rewards, SVMemAgent learns an emergent keyframe selection policy that prefers frames containing textual information, which may benefit downstream VideoQA tasks.

[33] STUNet-Fusion: Spatiotemporal Needle-Tip Localization in Ultrasound Video via Multi-Channel Motion Fusion cs.CVPDF

Chia-Chi Hsu, Chia-Hsuan Hsu, Che-Chou Shen

TL;DR: STUNet-Fusion将超声帧灰度外观、网格运动特征和原始帧差融合成三通道时空张量，以ResNet-34、ConvLSTM和U-Net生成针尖概率热图，再用soft-argmax实现亚像素定位；实验显示定位鲁棒性显著优于常规基线。

Details

Motivation: 超声图像中针体可能很弱、不连续或部分不可见，成像伪影和解剖结构又可能产生相似响应，使针尖定位困难。仅依赖单帧外观难以充分利用针和组织之间的时间运动线索。

Result: STUNet-Fusion是一种面向超声视频针尖定位的时空框架，将输入构造为三通道融合张量：灰度外观、网格化运动特征和原始帧差。共享ResNet-34编码器提取空间特征，ConvLSTM整合时间依赖，U-Net解码器恢复稠密概率热图，最后通过soft-argmax提取坐标，实现亚像素级定位。实验显示，多通道时空融合相较传统基线显著提高定位鲁棒性。摘要未给出具体基线、误差值或样本规模。

Insight: 超声针尖即使单帧不可见，也可能在运动和帧间变化中留下线索；将外观、网格运动及原始差分互补融合可提高鲁棒性。摘要未提供针尖误差单位或临床穿刺验证结果。

Abstract: Needle-tip localization in ultrasound remains challenging because the needle may appear weak, discontinuous, or partially invisible, while imaging artifacts and anatomical structures can produce similar responses. To address this problem, we propose STUNet-Fusion, a spatiotemporal framework for needle-tip localization in ultrasound videos. The proposed method formulates the input as a tri-channel spatio-temporal fusion tensor, comprising grayscale appearance, grid-based motion feature, and raw frame difference. A shared ResNet-34 encoder extracts spatial features, ConvLSTM integrates temporal dependencies, and a U-Net decoder reconstructs a dense probability heatmap. The final coordinates are extracted via a soft-argmax operation to achieve sub-pixel localization accuracy. Experimental results demonstrate that this spatiotemporal fusion strategy significantly improves localization robustness compared to conventional baselines.

[34] HAP: A Hand-Driven Active Perception Framework for Egocentric Head Motion Prediction cs.CV | cs.ROPDF

Yunji Feng, Junyi Ma, Guanzhong Sun, Chenyang Xu, Hesheng Wang

TL;DR: HAP 利用手部运动、目标置信度和动态遮挡图预测第一人称场景中的头部六自由度运动；在公开数据集和新建的 Bottle 数据集上优于代表性基线。

Details

Motivation: 现有自我中心运动预测多关注手和物体，较少建模头部如何随操作目标转移视线、获取证据并与身体动作协调。头部运动又受目标可见性和遮挡变化影响，因此仅根据历史轨迹难以准确预测。

Result: 作者提出 HAP：从手部运动和物体几何估计候选目标置信度，构建预测目标中心的非模态遮挡图，以有向图和因果时序推理表示感知状态，再与手部及头部历史融合；按预测时域用门控机制混合学习轨迹和匀速先验。另构建包含目标操作、头手协调运动及目标可见性变化的 RGB-D 数据集 Bottle。实验显示，HAP 在公开数据和 Bottle 上的头部运动预测误差低于代表性方法。

Insight: 关键是把头部运动看作主动感知行为：手部动作提供意图线索，动态遮挡关系刻画目标可见性，两者共同约束未来头部朝向。

Abstract: Egocentric motion forecasting has primarily focused on hands and manipulated objects, leaving future human head motion comparatively underexplored. During manipulation, the head both redirects perception toward the target to acquire task-relevant evidence and coordinates with body and hand motion. We therefore formulate future six Degree of Freedom (6-DoF) head-motion prediction conditioned on observed hand motion and inferred target context, and propose HAP, a Hand-Driven Active Perception framework. HAP infers confidence for each target object from observed hand motion and object geometry. Then constructs a dynamic Predictive Target-Centric Amodal Occlusion Graph (P-TAOG) representing current and potential occlusion among candidate objects. Directed graph and causal temporal reasoning encode the evolving target conditioned perceptual state, which is fused with hand and head motion history. A horizon-wise gate then blends the learned trajectory with a constant velocity prior. We further introduce Bottle, an egocentric RGB-D dataset of object manipulation toward specified targets, with coordinated head and hand motion under changing target visibility. Experiments on the public dataset and Bottle show that HAP achieves lower head motion prediction errors than representative baselines, supporting the value of hand driven intention and dynamic occlusion reasoning for anticipating human head motion. Code will be released at https://HAP-ego.github.io/HAP.

[35] Sim-to-Real Traffic Scene Understanding by Decoupling Semantics from Caption Generation with V-JEPA cs.CVPDF

Nguyen Hoai Thuong Bui, Thanh Nguyen Vo, Trinh Tra Giang Nguyen, Ha Duc Bui

TL;DR: 该 AI City Challenge 系统将交通语义判断与文字生成解耦，以 V-JEPA 表征、结构化事实修正和 Qwen3-VL 描述生成取得赛道第一名。

Details

Motivation: 合成到真实的交通场景迁移会放大视觉语言模型把语义理解和语言生成混在一起所带来的幻觉及事件阶段间推理不一致，影响问答准确性与事件描述可靠性。

Result: 框架先回答预定义交通问题并得到结构化语义事实，再据此生成描述。冻结的 V-JEPA 编码场景表征，轻量 Llama 预测器回答 VQA；无需训练的修正模块利用统计先验、问题间关系及时间一致性校正事实，随后交给 Qwen3-VL-8B 生成行人和车辆描述。在 2026 AI City Challenge Track 2 官方基准上，VQA 准确率为 87.09%，总分 S2 为 60.0853，排名第一。

Insight: 先把可核验的语义事实与自然语言表达分开，再用跨问题和时间约束修正事实，可以让生成器专注于表述，降低域偏移下的幻觉与不一致。

Abstract: Track 2 of the AI City Challenge 2026 requires both visual question answering (VQA) and traffic event description generation under a challenging synthetic-to real domain shift. Existing vision-language approaches often entangle semantic understanding with language generation, making them susceptible to hallucination and inconsistent reasoning across event phases. In this work, we propose a decoupled semantic understanding framework that first resolves predefined traffic questions into structured semantic facts and subsequently uses these facts to guide caption generation. A frozen V-JEPA encoder extracts predictive scene representations, while a lightweight Llama-based predictor produces answers for VQA queries. To improve reliability, we introduce a training-free structured refinement mechanism that exploits statistical priors, inter-question relationships, and temporal event consistency to correct prediction errors. The refined semantic facts are then provided to Qwen3-VL-8B to generate pedestrian and vehicle descriptions for each traffic event. Experimental results on the official 2026 AI City Challenge Track 2 benchmark show that the proposed method achieves 87.09% VQA accuracy and an overall S2 score of 60.0853, ranking first among all participating teams. These results demonstrate that predictive world representations combined with structured semantic refinement enable more accurate and reliable traffic understanding, leading to higher-quality lan guage generation.

[36] Learning Where to Focus: Self-Supervised Multi-Scale ViTs for Histopathology cs.CVPDF

Anabel Stammer, Valay Bundele, Mehran Hosseinzadeh, Hendrik P. A. Lensch

TL;DR: CRAFT 在病理图像自监督学习中按图像内容自适应分配粗细尺度 token，并以跨尺度正则化提升多个下游任务表现，同时降低推理计算量。

Details

Motivation: 病理诊断通常先定位可疑区域，再放大检查；但常规视觉 Transformer 对各区域采用统一分辨率，难以兼顾稀疏病灶证据与跨生物尺度上下文。现有病理基础模型虽扩大数据和模型规模，仍大多沿用均匀 token 化。

Result: 作者提出基于 DINO 的 CRAFT：通过自监督注意力选择性细化信息区域，同时保留粗尺度上下文，并使用对称跨尺度正则化促使粗、细表征互补。在 CAMELYON16、TCGA-Lung 亚型分类和 TCGA-LUAD 生存预测上，CRAFT 一贯优于规模相近的自监督方法且推理计算更低。仅用 22M 参数骨干和相对较小的病理数据训练，也能与更大的病理基础模型竞争，并在不少设置中超过它们。

Insight: 病理表征的改进可以来自学习“看哪里、看多细”，而不只是扩大模型；按内容分配分辨率有望同时保留组织上下文并集中计算于诊断相关区域。

Abstract: Pathologists diagnose diseases by first locating suspicious tissue and then examining it at higher magnification, whereas self-supervised vision transformers (ViTs) allocate the same spatial resolution to every image region despite diagnostic evidence being sparse and spanning multiple biological scales. Recent pathology foundation models have substantially improved representation quality by scaling training data and model capacity, but largely retain uniform tokenization. We instead investigate whether pathology representations can be improved by learning where to allocate spatial resolution during self-supervised learning. To this end, we propose CRAFT (Coarse-to-fine Region-Adaptive Feature Tokenization), a DINO-based framework that learns image-dependent mixed-scale representations by using self-supervised attention to selectively refine informative regions while preserving coarse context, together with a symmetric cross-scale regularization objective that encourages complementary coarse and fine representations. Across CAMELYON16, TCGA-Lung subtype classification, and TCGA-LUAD survival prediction, CRAFT consistently outperforms comparable-scale self-supervised methods while requiring lower inference computation. Despite using only a compact 22M parameter backbone trained on comparatively small pathology datasets, CRAFT remains competitive with, and often surpasses, substantially larger pathology foundation models.

[37] On-the-Fly Homographies Calibration for Multi-Camera Tracking cs.CV | cs.AIPDF

David Voihanski, Mor Sinai, Ben Zion Bobrovsky

TL;DR: 该系统利用实时检测元数据持续优化多摄像头单应性，再以鸟瞰视图跟踪融合跨区域轨迹，目标是在不录制视频和不增加跟踪延迟的条件下适应场景变化。

Details

Motivation: 大规模多摄像头跟踪通常依赖严格的三维现场标定，但隐私限制、带宽不足和布设标定靶的成本使离线视频标定在许多部署环境中不可行；摄像头移动或环境改变也会使固定单应性失准。

Result: 系统从粗略人工单应性开始，使用基于质心投影优化的方法，通过在线检测流持续对齐地平面几何。优化异步处理已传输的轻量元数据，因此声称不增加实时跟踪计算延迟；优化后的几何用于鸟瞰视图跟踪器，在不同区域间融合检测并统一轨迹。作者强调该流程仅使用实时匿名元数据，不需要保存视频。

Insight: 将标定更新放到匿名检测元数据上异步执行，可把几何自适应与主跟踪延迟解耦；摘要主要说明系统设计和部署动机，未给出量化精度结果。

Abstract: Precise multi-camera tracking traditionally relies on rigorous 3D site calibration, yet this requirement is often operationally impossible in large-scale deployments. Privacy regulations frequently prohibit recording video for offline calibration; limited bandwidth precludes synchronizing high-resolution streams from hundreds of cameras; and covering immense physical sites with calibration targets is logistically infeasible. We present a multi-camera homography calibration system designed to overcome these barriers through “on-the-fly” geometric refinement. Starting from coarse manual homographies, we introduce a centroid-based projection optimization (PO) that continuously aligns the ground-plane geometry using live detection streams. Because PO operates asynchronously on already-transmitted, lightweight metadata, it adds zero computational latency to the real-time tracker. This allows the system to adapt automatically to camera movements or environmental changes without human intervention. This optimized geometry feeds a multi-camera bird’s-eye-view (BEV) tracker that fuses detections and unifies trajectories across zones. Crucially, by operating strictly on live anonymous metadata, our solution ensures a privacy-safe, zero-overhead, and resilient tracking pipeline that maintains global consistency in dynamic environments where static, recorded-video calibration is impossible.

[38] FIVE-VLA: Fast and EffectIVE Autonomous Driving with Recurrent Action Memory cs.CV | cs.ROPDF

Kemal Oksuz, Alexandru Buburuzan, Yuhan Yao, Puneet K. Dokania

TL;DR: FIVE-VLA 以高效视觉编码和循环动作记忆改进自动驾驶视觉语言动作模型，在闭环路线完成率、碰撞违规率和推理速度上取得较好结果。

Details

Motivation: 现有自动驾驶 VLA 参数规模大、高分辨率图像处理成本高且缺少时序记忆，影响复杂驾驶决策效率，也不利于在边缘设备部署。

Result: FIVE-VLA 用高效编码器处理 448×896 图像并只生成 98 个 token，直接单次预测轨迹而不生成文本；轻量循环动作记忆模块让预测利用此前动作 token。模型共 641M 参数，在 Bench2Drive 闭环基准上比先前最优 VLA 完成约多 10% 路线且无交通违规；在 NVIDIA Physical AI 自动驾驶数据集的非反应式开环仿真中，单视角和四视角的碰撞违规率分别比 SimLingo 低 10.2% 和 7.7%。A100 上约 30 fps，T4 上约 4 fps。

Insight: 让模型直接回归动作轨迹并显式保留历史动作，可同时压缩文本生成开销并补上驾驶决策所需的时间上下文。

Abstract: State-of-the-art vision-language-action models (VLA) for autonomous driving face critical limitations: excessive parameter counts, inefficient high-resolution image processing, and lack of temporal memory. We introduce Fast and EffectIVE VLA (FIVE-VLA) to address these through two key contributions. First, we employ an efficient vision encoder that processes high-resolution ($448 \times 896$) images while generating only 98 tokens, over $5\times$ fewer than existing approaches, and bypass text generation entirely for single-pass trajectory prediction. Second, we propose Recurrent Action Memory (RAM), a lightweight module that conditions action prediction on previous action tokens, providing temporal context critical for manoeuvres such as overtaking and emergency braking. With only 641M parameters, FIVE-VLA completes $\sim$10% more routes without traffic rule infractions than the previous state-of-the-art VLA on the challenging Bench2Drive closed-loop driving benchmark. Non-reactive open-loop simulation on the large-scale real-world NVIDIA Physical AI AV dataset shows 10.2% and 7.7% lower collision-violation rates than SimLingo in single- and four-view settings, respectively. Additionally, FIVE-VLA runs at $\sim$30 fps on an A100 and $\sim$4 fps on a T4 GPU (proxy to an edge device), representing an 8-30$\times$ speedup over previous methods.

[39] VibeAvatar: Aligning Phonetic Kinematics and Human Aesthetics for High-Fidelity Talking Avatar Synthesis cs.CVPDF

Qilin Wang, Mingyu Li, Hao Tang

TL;DR: VibeAvatar 将语音中的音素运动条件与人类偏好的动作美感分开处理，通过流式运动生成和 GRPO 后训练提升口型、动作审美及生成效率。

Details

Motivation: 扩散式说话头像虽进展迅速，但仍难同时保证口型与语音准确同步、动作符合人类审美以及推理高效；这几项目标来源不同，单一生成器难以隐式兼顾。

Result: VibeAvatar 在条件阶段用音素运动适配器把面向识别的语音特征转换为发音运动条件，在后训练阶段用审美运动策略通过组相对策略优化调整流一致的随机采样策略。轻量流式生成器在紧凑的一维 warp 潜在运动空间工作。作者报告其在客观指标和用户研究中的发音、审美与效率达到先进水平；约 3GB 显存下生成 512 像素、10 秒视频用时不足 10 秒。

Insight: 将发音运动准确性作为输入条件、将动作审美作为采样策略优化目标，按阶段处理不同来源的要求，可能比让同一生成目标兼顾两者更有效。

Abstract: Multi-modal talking avatar synthesis aims to generate realistic talking videos from a reference portrait and speech. Despite rapid progress in diffusion-based methods, existing approaches still struggle to jointly achieve accurate lip articulation, human-preferred motion aesthetics, and efficient inference. We observe that phonetic accuracy and motion aesthetics arise from fundamentally different sources and should be addressed at complementary stages rather than learned implicitly by a single generator. Based on this insight, we propose VibeAvatar, which disentangles these two objectives through a Phonetic Kinematics Adapter (PKA) that converts recognition-oriented speech features into phonetic-kinematic conditions at the conditioning stage, and an Aesthetic Motion Policy (AMP) that optimizes a flow-consistent stochastic sampling policy via Group Relative Policy Optimization (GRPO) at the post-training stage. With a lightweight flow-based motion generator operating in a compact 1D warp-based latent motion space, VibeAvatar achieves state-of-the-art results in articulation, aesthetics, and efficiency on both objective metrics and user studies, while generating a 10-second 512px video in under 10 seconds with only $\sim$3GB VRAM.

[40] GenStream: Semantic Streaming Framework for Generative Reconstruction of Human-centric Media cs.CV | cs.AI | cs.MMPDF

Emanuele Artioli, Daniele Lorenzi, Shivi Vats, Farzad Tashtarian, Christian Timmerer

TL;DR: GenStream 传输人体骨架、相机参数和静态背景模型等结构化信息，再由客户端生成式重建画面；花样滑冰案例展示了相较 HEVC 连续数据流超过 99.9% 的带宽压缩潜力。

Details

Motivation: 传统视频编码对前景人物和静态背景统一逐帧编码，在运动受限且背景固定的人体中心内容中会重复传输大量冗余像素，浪费带宽。

Result: GenStream 用骨架关键点、相机视角参数和静态三维背景模型替代稠密视频帧传输；客户端生成逼真人物并按原视角合成到场景中。作者在奥运花样滑冰片段上进行了初步验证，报告连续数据流带宽相较 HEVC 降低超过 99.9%，并展示在低传输量下实现较高感知保真度的潜力。其代价是把大量计算转移到客户端，且泛化仍有挑战。

Insight: 若内容具有可分离的人体动作与稳定场景结构，传输场景语义并在端侧重建可能远比逐像素压缩省带宽；能否普适取决于重建质量、客户端算力和跨场景泛化。

Abstract: Video streaming dominates global internet traffic, yet conventional pipelines remain inefficient for structured, human-centric content such as sports, performance, or interactive media. Standard codecs re-encode entire frames, foreground and background alike, treating all pixels uniformly and ignoring the semantic structure of the scene. This leads to significant bandwidth waste, particularly in scenarios where backgrounds are static and motion is constrained to a few salient actors. We introduce GenStream, a semantic streaming framework that replaces dense video frames with compact, structured metadata. Instead of transmitting pixels, GenStream encodes each scene as a combination of skeletal keypoints, camera viewpoint parameters, and a static 3D background model. These elements are transmitted to the client, where a generative model reconstructs photorealistic human figures and composites them into the 3D scene from the original viewpoint. This paradigm enables extreme compression, achieving over 99.9% bandwidth reduction compared to HEVC for the continuous data stream. We partially validate GenStream on Olympic figure skating footage and demonstrate potential for high perceptual fidelity under minimal data. While acknowledging the significant computational costs shifted to the client and challenges in generalization, GenStream opens new directions in volumetric avatar synthesis, canonical 3D actor fusion across views, and personalized viewing experiences, laying the groundwork for scalable, intelligent streaming in the post-codec era.

[41] Video-Based Markerless Motion Capture for Clinical and Rehabilitation Biomechanics: A PRISMA-ScR Scoping Review of Validated Architectures, Clinical Readiness, and Emerging Methods cs.CV | physics.med-phPDF

Florian Delaplace, Elodie Piche, Frédéric Chorin, Raphael Zory

TL;DR: 这篇范围综述梳理视频无标记动作捕捉在临床与康复生物力学中的验证现状：当前证据尚不足以证明其可替代标记系统，尤其缺少病理人群、离平面运动和动力学验证。

Details

Motivation: 视频无标记动作捕捉有望降低光学标记系统的成本、空间和贴标负担，但其关节运动估计是否达到临床可接受标准，以及计算机视觉领域的新方法是否进入经验证的临床流程，仍不清楚。

Result: 作者按 PRISMA-ScR 对 2015 年 1 月至 2026 年 2 月的生物力学研究进行范围综述，并补充至 2026 年 7 月的计算机视觉工作扫描，共纳入 117 项研究。研究多在实验室对健康成人步行进行，流程归为五类架构；大多直接报告未经生物力学细化的关节角。矢状面下肢角度一致性多集中于约 5–6°，通常未达临床可接受水平；离平面运动学、动力学、老年及病理人群验证稀少。基础模型网格恢复、可微逆运动学和视频动力学等新方向几乎未进入已验证流程。

Insight: 现阶段不能把视频无标记捕捉视为临床标记系统的等价替代品。综述提出的是面向后续研究的假设性设计建议，而非经过验证的新方法；临床转化的关键缺口集中在真实康复人群和完整生物力学量验证。

Abstract: Background.. Video-based markerless motion capture promises movement analysis without the cost, space and skin-marker constraints of optoelectronic systems, with particular potential for clinical and rehabilitation settings. Whether validated pipelines yet deliver clinically acceptable biomechanics, and how they relate to the underlying computer-vision research, remains unclear. Methods. We conducted a scoping review following the PRISMA extension for Scoping Reviews, with a registered protocol and searches of PubMed, Scopus and IEEE Xplore (January 2015 to February 2026; the computer-vision scan was updated to July 2026). A dual-tier design paired a primary corpus of validated biomechanical studies with a complementary, curated and deliberately non-exhaustive corpus of emerging computer-vision work, used qualitatively. We charted study characteristics, pipeline architecture, validation methods and joint-angle accuracy. Results. We included 117 studies, most published from 2024 onward and conducted on healthy adults walking in a laboratory. Pipelines formed five architectural families across monocular and multi-camera modalities; most reported raw joint angles without biomechanical refinement. Sagittal lower-limb agreement clustered around 5 to 6{\textdegree}, generally short of clinical acceptability, while out-of-plane kinematics, kinetics, and pathological or older populations were rarely validated. Emerging computer-vision building blocks (foundation-model mesh recovery, differentiable inverse kinematics, video-based kinetics) were almost absent from validated studies. Conclusions. Video-based markerless capture is not yet interchangeable with marker-based systems for clinical joint kinematics, and it remains barely validated where rehabilitation needs it most: older and pathological populations, out-of-plane kinematics, and kinetics. Mapping this evidence gap onto emerging computer-vision advances, we propose hypothesis-generating design guidelines, not a validated method, to steer the next generation of pipelines toward accessible, clinically meaningful movement analysis.

[42] Generalist-Specialist Mixture-of-Experts for Rare Pathology Detection in Multimodal Imaging cs.CV | cs.AIPDF

Johannes Kaiser, Florian Braunmiller, Daniel Rückert, Georgios Kaissis

TL;DR: GS-MoE 结合跨模态通才分支与各模态专才专家，在 RadImageNet 多模态影像上改善六种稀有病理检出，并减少推理时激活参数。

Details

Motivation: 多模态医学影像模型需要同时保留各模态特有信息和跨模态共享表征。纯专家混合架构虽便于模态专门化，却可能丢失对低患病率病理尤为重要的跨模态信号。

Result: 作者提出两分支 GS-MoE：一个跨模态通才模型与多个模态专属专家通过受域约束的特征融合协同。在含 135 万张图像、165 种病理和三种模态的 RadImageNet 上，该方法恢复了所有基线 F1 均为 0 的六种低患病率病理检测，每类 F1 最高提升 0.60；总体 MCC 达 0.770，略高于稠密模型及仅专才 MoE 基线。与最强被测稠密模型相比，推理时激活参数约少 53%。

Insight: 稀有类别可能更依赖跨模态共享线索，因此混合专家模型需要让通用表征与模态专长互补，而非仅按模态分割专家。

Abstract: AI models for multimodal medical imaging must balance modality-specific specialization with cross-modal shared representations, a trade-off that pure Mixture-of-Experts (MoE) architectures currently fail to satisfy. Expert-based routing improves in-domain learning but may sacrifice cross-modal signals, which appear particularly important for rare (low-prevalence) pathologies in our experiments. To resolve this, we introduce Generalist-Specialist-MoE (GS-MoE), a two-branch (MoE) architecture that couples a cross-modal generalist model with distinct modality-specific specialists (experts) via domain-constrained feature fusion. On RadImageNet (1.35M images, 165 pathologies, three modalities), GS-MoE recovers detection of six low-prevalence pathologies on which every baseline scores F1 $=$ 0, with per-class gains up to +0.60 F1. It attains this while even slightly exceeding dense and specialist-only MoE aggregate baselines (MCC 0.770), while using ${\sim}53%$ fewer active parameters at inference than the strongest investigated dense model.

[43] RankGround: Efficient High-Resolution GUI Grounding via Lightweight Reranker-Guided Crop Selection cs.CV | cs.CL | cs.HCPDF

Liyang Fan, Xinping Bi, Yitai Li, Shuaimin Li, Hui Li

TL;DR: RankGround 用轻量多模态重排器从密集候选裁剪中挑选最有希望的区域，使 GUI 指令定位只需一次视觉语言模型调用，并提升定位精度和速度。

Details

Motivation: 直接对整张高分辨率界面推理容易漏掉小型或外观相似控件；多裁剪可改善定位，却通常需要针对每个请求多次调用昂贵的视觉语言模型。

Result: RankGround 分两阶段处理：先构造密集候选裁剪并用 GroundRanker 重排，选出最可能包含目标的裁剪，再以单次 VLM 调用完成定位。作者从既有 grounding 数据集构建排序监督数据，采用严格包含标准和边界感知正样本增强空间覆盖，并先用逐点目标学习粗粒度包含关系，再以列表目标区分相似裁剪。实验中，该方法持续优于强基线；相对第二名，跨骨干和屏幕尺度平均定位准确率提升 5.5%，推理速度快 1.4 倍。

Insight: 用便宜的候选重排把高分辨率搜索压缩到一次 VLM 精判，能够缓解 GUI 感知中精度与调用成本的矛盾；排序监督的边界覆盖和难负例区分是关键。

Abstract: Graphical User Interface (GUI) grounding is a fundamental perception task for multimodal agents, enabling them to interpret natural language instructions and interact with digital interfaces. Existing methods face a fundamental trade-off between accuracy and efficiency: direct full-image inference often fails to capture small or visually similar UI elements, while multi-crop strategies improve localization at the cost of multiple expensive Vision-Language Model (VLM) calls per query. To address this challenge, we propose RankGround, a two-stage framework that achieves accurate GUI grounding with a single VLM call per query. Central to our approach is GroundRanker, a lightweight multimodal reranker that identifies the most promising crop from a dense candidate set. Because no off-the-shelf ranking dataset is available, we construct ranking supervision data from existing grounding datasets. A strict containment criterion and boundary-aware positive augmentation improve alignment and spatial coverage in cluttered layouts. GroundRanker is then trained with a two-stage curriculum: a pointwise objective first learns coarse containment, and a listwise objective refines subtle semantic and spatial distinctions among visually similar crops. Experimental results show that RankGround consistently outperforms strong baselines while reducing computational cost. It achieves 1.4 times faster inference and improves localization accuracy by 5.5% on average over the second-best method across all backbones and screen scales, establishing a new state of the art in both efficiency and precision for GUI grounding.

[44] Toward Markerless Video-based Tremor Analysis: Objective Quantification of Pathological Tremor in Mouse Preclinical Models cs.CVPDF

Yota Koshimoto, Akihiro Tsukahara, Yasuhiro Moriwaki, Mariko Isogawa

TL;DR: 该方法仅用两台普通 RGB 摄像机，通过分割和细微振颤评分模块估计自由活动小鼠的病理性震颤强度，并与加速度计测量呈强相关。

Details

Motivation: 小鼠震颤研究常使用肌电或加速度计，但设备成本、复杂安装以及侵入式植入手术会增加实验负担并使动物承受压力。普通 RGB 视频更易部署，却常难检测微弱震颤，尤其小鼠自身移动时。

Result: 流程先用分割预处理提取小鼠区域，再通过震颤评分估计模块分离并量化细微振动。在两台标准摄像机拍摄的非束缚小鼠实验中，估计分数与加速度计测量呈强相关，并能反映随震颤强度变化的特征。

Insight: 通过先隔离动物区域、再针对微小周期性运动设计敏感评分，可在不植入传感器的情况下获得客观震颤量化；摘要支持的是小鼠前临床场景，不能直接外推为人体临床验证。

Abstract: Tremor is a movement disorder characterized by involuntary, rhythmic oscillations of body parts and is a hallmark of several neurological conditions, including Parkinson’s disease and essential tremor. Elucidating its underlying mechanisms relies heavily on mouse models, which offer genetic manipulability and translational relevance to human neural circuitry. Accordingly, these models are indispensable for studying tremor pathophysiology. So far, electromyography and accelerometers have been used as methods to quantitatively observe tremors in mice. However, these methods have several drawbacks, such as high costs and complex setups. In particular, the invasive surgical implantation of devices causes significant stress to the animals. Although RGB-based methods offer non-invasive and cost-effective alternatives, they often lack the sensitivity required to detect subtle tremors. Therefore, this paper addresses these challenges by achieving mouse tremor severity estimation using conventional RGB cameras only. To address the challenging task of isolating tremor-related vibrations while the mouse itself is also in motion, our pipeline incorporates segmentation-based pre-processing to extract the mouse region and a Tremor Score Estimation Module that captures subtle tremors with high sensitivity. In the experiments, we assessed tremors in unrestrained mice using a non-invasive method with two standard cameras. The results demonstrated a strong correlation with accelerometer measurements and confirmed that the method accurately captured the intensity-dependent characteristics of tremors. The project page is available at https://isogawalab.github.io/Video-based-Tremor-Analysis-Project/.

[45] DISTA-Net++: Rethinking Infrared Small Target Unmixing Beyond Sub-Pixel Separation cs.CVPDF

Mengze Xu, Zhu Liu, Weidong Sheng, Boyang Li, Yimian Dai

TL;DR: DISTA-Net++ 将密集红外小目标分离表述为受目标总数约束的连续坐标估计，并构建 CSIST-100K、指标套件和工具；在更粗网格下仍超过更细网格方法。

Details

Motivation: 远距离红外成像中的密集小目标经衍射后会融合为单个光斑，传统无语义约束的离散亚像素分离存在多解、漏检和误检，网格化坐标也把精度绑定到昂贵的网格细化。

Result: 作者建立首个面向密集近邻红外小目标分离的开源生态，包括 CSIST-100K、专用评测指标和 GrokCSO 工具包；提出以深度展开为骨干的 DISTA-Net++，引入显式目标数量先验以约束解空间，并回归连续偏移来校正网格外坐标。即使仅采用 3 倍划分，相较 7 倍划分的先进方法，CSO-mAP 高 16.15%，计数准确率高 62.96%，计算量仅为其六分之一。

Insight: 显式使用全局计数语义并对坐标作连续回归，可减少离散网格造成的歧义和精度成本；该结果说明更精细的离散划分并非提升密集目标分离精度的唯一途径。

Abstract: Long-range infrared imaging frequently confronts dense target clusters whose diffraction-limited signatures merge into a single indistinguishable blob, concealing the number, sub-pixel positions, and radiant intensities of the underlying sources. While deep learning has advanced general object detection, resolving such Closely-Spaced Infrared Small Targets (CSIST) remains largely unexplored, owing to a systemic infrastructure void and a fundamental paradigm mismatch. The dominant formulation, which reduces unmixing to a blind, discrete sub-pixel separation, is inherently insufficient: without semantic guidance, the ill-posed inverse problem admits ambiguous solutions plagued by false and missed detections, while grid-based discretization locks predictions onto fixed lattice centers, chaining precision to prohibitively expensive grid refinement. We argue that CSIST unmixing should instead be informed and continuous. To ground this paradigm shift, we establish the first comprehensive open-source ecosystem for the field, comprising the large-scale CSIST-100K benchmark, a tailored metric suite, and the GrokCSO toolkit. Upon this foundation, we propose DISTA-Net++, which anchors a dynamic deep unfolding backbone with two synergistic mechanisms: a Count-Guided Prior that injects the global target count as an explicit semantic constraint to regularize the solution space, and a Continuous Coordinate Rectification that regresses off-grid offsets to decouple localization accuracy from grid resolution. Extensive experiments validate our paradigm: even under the most economical 3x division, DISTA-Net++ surpasses 7x-division state-of-the-art methods by 16.15% in CSO-mAP and 62.96% in count accuracy at merely one-sixth of their computation, demonstrating that unmixing precision need not be purchased with finer discretization. The complete ecosystem is available at https://github.com/GrokCV/GrokDet.

[46] Using OCR Heads to Verbalize Image Semantics cs.CV | cs.AI | cs.CLPDF

Sheridan Feucht, Benno Krojer, Sarah Wang, Henry Abrahamsen, Byron C. Wallace

TL;DR: 研究发现对视觉语言模型 OCR 有因果作用的注意力头也承载通用图像语义，并据此构造可解释的 verbalization lens，在早期层读出和编辑图像概念。

Details

Motivation: 视觉语言模型如何把像素映射为语义尚不清楚。作者以 OCR 为切入口，试图识别模型中因果上必要的机制，并检验这些机制是否仅负责读字，还是也编码更广泛的视觉概念。

Result: 在四种模型中，作者识别出对 OCR 因果必要的注意力头，发现它们是通用头，能从图像 token 输出可解释语义：例如指向“自行车”可读出 bike，指向鸟翼可读出 feathers。将这些头的注意力权重合并成 verbalization lens 后，可在各层隐藏状态中读出语义特征；投影到词表空间时，甚至能从第 0 层得到标签，显示图像表征早期已与语言对齐。该变换的逆操作还可编辑非文字概念，例如将自然图像中的拖拉机替换为左轮手枪。

Insight: OCR 机制可能暴露更一般的视觉语义子空间，而不只是文字识别路径；概念替换实验提供了该子空间参与图像语义的因果证据。

Abstract: How do VLMs map from pixels to semantics? To understand this general question, we focus on a narrow one: studying how VLMs perform optical character recognition (OCR). Across four models, we identify attention heads causally necessary for OCR, and discover that these are in fact general-purpose heads that output interpretable semantic features across all image tokens. For example, pointing these heads at an image token containing the word “bike” causes Qwen3-VL-8B to output “bike,” but pointing them at a bird wing causes the model to output the token “feathers.” We collapse these heads’ attention weights into a single verbalization lens transformation that reveals interpretable semantic features in hidden states across all layers. When combined with projection to vocabulary space, we can obtain interpretable labels starting from layer 0, showing that image representations are in fact aligned with language in early layers. We find that we can also use the inverse of this transformation to edit non-word concepts, e.g., replacing a tractor with a revolver in a naturalistic image, providing causal evidence that this subspace is useful for more than just OCR. Our results are an example of how the study of specific mechanisms can shed light on broader interpretability problems.

[47] Copy What Is Seen, Generate What Is Not: Training-Free Anomaly-Aware Video Restoration cs.CVPDF

Zhida Qu, Shengchao Chen

TL;DR: AVR 将异常检测与无训练视频修复衔接起来，优先复制视频中已有背景证据，仅对全片都未展示的区域调用生成模型合成。

Details

Motivation: 监控系统发现异常后往往还需修复画面，但无训练异常检测通常只输出分数或标签，视频编辑模型则依赖用户提示，两类工作相互割裂。直接扩散生成还可能带来闪烁或凭空改变有证据的画面内容。

Result: AVR 先用运动证据将开放词表候选转为时空异常掩码，再用整段视频的背景先验填补异常曾遮挡的像素，让扩散模型只生成任何帧中都无从复制的内容；冻结验证器按片段选择经典修复、先验锚定或背景条件修复器。在三个监控数据集、合成异常注入和真实异常上，oracle 掩码设置下全帧保真度领先，在编辑区域与三种训练式视频修复模型相当；对自身预测掩码的端到端流程优于检测后直接生成，并能抑制残留异常和扩散闪烁。

Insight: 视频自身的未遮挡帧可作为修复证据，先复制可见内容、再生成不可见部分，能限制生成模型自由发挥；该方法表现仍受到异常掩码质量和背景先验适用性的影响。

Abstract: A surveillance system that detects an anomaly often has to repair the footage as well, yet the two tasks are studied in isolation: training-free anomaly detectors stop at a score or a label, while training-free video editing answers to a user prompt rather than to a detector. This paper proposes AVR (Anomaly-aware Video Restoration), which closes that gap with frozen pretrained models alone and generates content only where the clip offers no evidence to copy. Motion evidence first gates open-vocabulary proposals into spatio-temporal masks. A background prior computed from the clip then fills every pixel the anomaly ever uncovers, leaving diffusion to synthesize only what no frame showed, and a frozen verifier decides per clip whether to trust a classical, a prior-anchored, or a background-conditioned restorer. Extensive experiments on three surveillance datasets, under both full-reference anomaly injection and real anomalies, show that AVR leads full-frame fidelity under oracle masks, matches three trained video inpainters inside the edited region, and outperforms a detect-then-generate pipeline on the masks it produces itself, while suppressing both the residual anomaly and the flicker of free diffusion.

[48] Decodable but Misrouted: Sparse Features Uncover a Readout Gap in Vision-Language Models for Harmful Meme Detection cs.CV | cs.AI | cs.CL | cs.LGPDF

Girish A. Koushik, Diptesh Kanojia, Helen Treharne

TL;DR: 这项研究用稀疏特征探针、因果干预和恢复实验分析有害梗图分类，发现视觉语言模型内部证据可能已可解码，却未被正确路由到输出。

Details

Motivation: 视觉语言模型误判有害梗图，可能是内部缺少相关视觉证据，也可能是证据已表征但未能进入决策输出。区分这两种原因有助于针对性改进多模态安全分类。

Result: 作者在 Gemma-3 和 Qwen3.5 上结合稀疏自编码器、按角色条件化的探针、因果干预和恢复实验，覆盖六个有害内容基准及西班牙语和印地语-英语混码评测。稀疏读出在六项主二分类任务上均超过模型原生预测；例如 Qwen 平均宏 F1 为 0.740，而原生为 0.432，Gemma 从 0.532 提升到 0.714。校准式路由恢复了平均差距的 93.3%；探针蒸馏 LoRA 能改善原生预测，但多任务联合适配出现负迁移。消融和配对视觉证据控制表明，信号不只来自英语或 OCR。

Insight: 这些实验支持“读出路由”是有害梗图分类的反复瓶颈之一，但探针可解码性本身不等于模型原本已有可直接使用的决策规则；不同任务中起作用的 token 角色也会变化。

Abstract: When a large vision-language model misclassifies a harmful meme, the failure may reflect missing internal evidence or an inability to route represented evidence to its output. We distinguish these cases in Gemma-3 and Qwen3.5 using sparse autoencoders, role-conditioned probes, causal interventions, and recovery experiments across six harmful content benchmarks, with additional Spanish and Hindi-English code-mixed evaluations. Sparse readouts outperform native prediction on all six primary binary tasks: Qwen averages $0.740$ versus $0.432$ native macro-F1, while residual reconstruction reaches $0.486$, whereas Gemma improves from $0.532$ to $0.714$. These differences reflect supervised accessibility rather than a pre-existing, native decision rule, and the most influential token role depends on the task. Under the evaluated score scales, Qwen silent-feature ablation is $24-63$ times more probe-sensitive, whereas routed-feature patching on literal yes/no tasks is $16-140$ times more output-sensitive. Calibration-only routing recovers $93.3$% of the mean gap, and probe-distilled LoRA improves native predictions, although shared multi-task adaptation causes negative transfer. A case study of Gemma-3-12B on Facebook Hateful Memes finds a distributed rank-32 image-prompt interaction, reaching $0.756$ versus $0.685$ native macro-F1. Robustness controls show that the signal extends beyond English, is not explained solely by accompanying OCR, and depends on paired visual evidence. Thus, routing, rather than representation alone, is a recurring bottleneck in harmful meme classification.

[49] PhysVGGT: Feed-Forward Dense Physical Property Estimation from A Single Image cs.CVPDF

Sneha Paul, Guile Wu, Bingbing Liu, Dongfeng Bai

TL;DR: PhysVGGT 从单张 RGB 图像一次前向预测稠密摩擦、硬度、刚度和密度图及物体质量，无需逐物体重建或测试时优化。

Details

Motivation: 物体的摩擦、硬度、刚度和密度决定机器人如何抓取和操作，但仅从 RGB 图像估计这些物理属性仍困难。逐物体重建或测试时查询视觉语言模型会带来较高计算开销。

Result: PhysVGGT 把物理属性估计转成稠密逐像素预测：视觉几何 Transformer 提取几何感知 token，局部分支预测物理属性图，全局分支估计物体质量。作者还提出可扩展的伪标签生成流程，以弱监督降低昂贵直接测量需求。在 ABO-500 上达到先进表现，并能泛化到分布外 NeRF2Physics；单图推理延迟为 0.13 秒，比此前方法快 27 倍。

Insight: 把局部材质属性和整体质量分成稠密与全局预测路径，使几何感知特征可服务于实体交互；其规模化优势依赖伪标签质量和跨数据集泛化。

Abstract: Physical properties, such as friction, hardness, stiffness, and density, govern how robots should grasp, manipulate and interact with objects, yet estimating these properties from RGB images remains challenging. Existing methods typically employ per-object reconstruction augmented with physical properties or directly query vision-language models at test time, which results in substantial computational overhead that limits their applicability. In this work, we present PhysVGGT, a feed-forward model that predicts dense maps of friction coefficient, Shore hardness, Young’s modulus, and density, together with object-level mass, from a single RGB image in one forward pass. The key idea of PhysVGGT is to formulate physical property estimation as a dense per-pixel prediction problem and employ a visual geometry transformer to extract geometry-aware tokens from the input image followed by a dense prediction branch for estimating local physical properties and a global prediction branch for estimating object-level mass. In addition, we introduce a scalable pseudo-label generation pipeline that enables large-scale weakly supervised training for dense physical property prediction, substantially reducing the need for expensive direct physical measurements. Extensive experiments show that PhysVGGT achieves state-of-the-art performance on the ABO-500 dataset and generalizes effectively to the out-of-distribution NeRF2Physics dataset. Moreover, PhysVGGT eliminates the need for per-object reconstruction and test-time optimization, achieving an inference latency of only 0.13s per image, making it $27\times$ faster than the previous state of the art.

[50] LaSeD: Label-Semantic Self-Distillation for Visual-Only Surgical Phase Recognition cs.CVPDF

Ye Tao, Claudia Scherl, Sara Monji-Azad

TL;DR: LaSeD 在训练时把手术阶段名称作为特权提示蒸馏给学生模型，部署时仅输入视频图像，实现无需阶段文本提示的视觉手术阶段识别。

Details

Motivation: 手术阶段识别对每帧预测临床流程阶段，可支持辅助、记录和术后分析。常见方法只把阶段标注作为类别 ID，而视觉语言方法又常依赖额外视频文本数据、字幕或指令微调。

Result: LaSeD 从同一预训练 VLM 初始化冻结教师和学生。训练时教师接收图像、固定任务提示和真实阶段名称，学生不接收阶段名；只优化学生视觉编码器，并结合阶段 token 硬监督与缓存教师表示的特征蒸馏。推理时移除教师和阶段提示，以受约束的数字 token logits 预测 Cholec80 七个阶段。在评测划分上准确率 86.20%、宏召回 77.75%、宏精度 78.13%、宏 Jaccard 64.15%；相较同协议的纯视觉 Qwen3-VL-4B，四项分别提高 9.45、9.42、11.93 和 12.22 个百分点。

Insight: 阶段名称可作为低成本的训练期语义监督，帮助视觉编码器形成工作流表征，而不必在部署时依赖文本提示；文中“纯视觉”指样本特定输入只有图像，固定任务提示仍保留。

Abstract: Surgical phase recognition maps each video frame to a clinically meaningful workflow phase, supporting context-aware assistance, documentation, and postoperative analysis. Most methods treat phase annotations only as class IDs, whereas recent surgical vision-language models often require additional video–text data, captions, or instruction tuning. We propose \emph{LaSeD}, a label-semantic self-distillation framework that uses phase names as privileged training-time context while retaining visual-only deployment without a ground-truth phase-name hint. LaSeD initializes a frozen teacher and a student from the same pretrained VLM checkpoint. The teacher receives the frame, a fixed task prompt, and the ground-truth phase-name hint; the student receives the same frame and prompt without the hint, and only its visual encoder is optimized. Training combines hard phase-token supervision with feature-level distillation from cached teacher representations. At inference, the teacher and hint are removed, and the student predicts one of the seven Cholec80 phases through constrained digit-token logits without an additional classifier head. On the Cholec80 evaluation split, LaSeD achieves 86.20% accuracy, 77.75% macro recall, 78.13% macro precision, and 64.15% macro Jaccard. Under the identical protocol, it improves a visual-only Qwen3-VL-4B baseline by 9.45, 9.42, 11.93, and 12.22 percentage points, respectively. Visual-only means that the image is the only sample-specific inference input, while all frames share the same fixed task prompt. These results suggest that phase names provide a useful low-cost signal for adapting VLMs to surgical workflow analysis. (The code will be published soon.)

[51] Track, Articulate, Act: Generating Articulation from Casual Human Videos cs.CVPDF

Jiaming Zhang, Homanga Bharadhwaj

TL;DR: 该 real-to-sim 框架从单目日常视频的三维点轨迹推断关节与运动状态，重建可在 MuJoCo 中回放的人手与铰接物体交互。

Details

Motivation: 门、抽屉、柜子等铰接物体不能用单一刚体位姿描述，其运动依赖部件和关节。获取可供机器人使用的交互模型，通常需要深度、多视角扫描、人工标注关节或机器人示范。

Result: 作者提出仅从普通单目 RGB 视频重建可供仿真的铰接物体和手物交互的方法。核心线索是稠密三维点轨迹：固定部件上的点近似静止，运动部件上的点呈一致的转动或滑动。系统分割物体部件，估计关节类型及状态轨迹，重建铰接资产，并将恢复的人手三维运动与物体对齐；结合预训练单图三维重建、网格分割和三维场景流模型，通过显式几何推理串接其输出。最后在 MuJoCo 中通过接触回放交互。

Insight: 无需机器人动作标签时，人类手物交互的点轨迹可提供与具体机器人身体无关的关节运动线索，推动日常视频转成可用于具身任务的仿真资产。

Abstract: Human videos contain rich causal evidence for robot manipulation: they reveal how hand motion induces object motion and produces task-relevant changes in object state. In this work, we study articulated objects such as doors, drawers, cabinets, laptops, ovens, and hinged containers that are ubiquitous in daily life and present unique challenges for embodied interaction. These objects cannot be represented by a single pose; their motion depends on the underlying parts and joints. We introduce a real-to-sim framework that reconstructs a simulation-ready articulated object and hand-object interaction from a casual monocular RGB video, without RGB-D or multi-view input, prior scans, manually specified joints, or robot demonstrations. Our key insight is that dense 3D point tracks provide an embodiment-agnostic articulation cue: points on the fixed link remain approximately stationary, while points on the moving link follow coherent revolute or prismatic motion. Our method segments the links, estimates the joint and its state trajectory, reconstructs an articulated asset, and aligns the recovered 3D hand motion with the object. Central to our approach is a modular recipe that repurposes powerful pretrained models for single-image 3D reconstruction, mesh segmentation, and 3D scene flow, connecting their predictions through explicit geometric reasoning to infer articulation. We use the reconstructed articulated object and the human hand trajectory to replay interactions through contact in MuJoCo. The framework shows how pretrained vision models and explicit motion reasoning can turn casual human videos into articulated object models suitable for downstream embodied interactions. https://track-articulate-act.github.io/

[52] In-Context Robot Learning with VLM Agents cs.CV | cs.ROPDF

Dongzhou Cheng, Taoran Yi, Ye Fang, Xingwu Zhang, Fan Feng

TL;DR: GPT-Policy 将视觉语言模型的上下文学习接入机器人执行：编译任务相关视觉转变、提出工具动作，并由受约束控制器验证执行和反馈结果。

Details

Motivation: 有限机器人示范不可能覆盖所有任务与环境，因此机器人需要部署时从上下文示范和交互反馈适应新任务。现有机器人策略通常难以做到这种无需梯度更新、无需改变任务参数的上下文内学习。

Result: GPT-Policy 包含保留任务相关视觉转变的上下文编译器、提出机器人工具动作的 VLM 智能体，以及约束动作并验证执行、回报结果的控制器。论文通过任务成功率、效率、模型配对比较和上下文消融评估可靠性与局限。真实机器人实验显示，即使没有机器人动作标签，人类视频示范也能提高任务完成率；对接触敏感任务，提供对齐的动作参考还会带来额外收益。

Insight: 示范视频能在没有机器人动作标注时提供任务层面的可迁移信息，但物理执行仍需要控制器做约束和结果验证；接触敏感场景可能需要更明确的动作参考。

Abstract: Enabling robots to adapt to unfamiliar environments as readily as humans remains a moonshot goal of embodied AI. No finite collection of demonstrations can cover every task and situation a robot will encounter, making the ability to learn from context at deployment essential for generalization. Such in-context learning (ICL), however, remains largely beyond the reach of existing robotic policies. The broad agentic capabilities of commercial vision-language models (VLMs), such as GPT-6 Astra, raise a compelling question: can these models learn from demonstrations, examples, and interaction feedback, then translate that information into executable and verifiable robot behavior from a new initial state without gradient updates or persistent changes to task-specific parameters? We introduce GPT-Policy, a general-agent framework for in-context robot learning. GPT-Policy integrates a context compiler that preserves task-relevant visual transitions, a VLM that proposes robot-tool actions, and a constrained controller that verifies and executes each action and reports its outcome. We evaluate its reliability and limitations through task success and efficiency metrics, matched comparisons across models, and controlled context ablations. In real-robot trials, human video demonstrations improve task completion even without robot action labels, while aligned action references yield further gains on contact-sensitive tasks. These findings position GPT-Policy as a step toward robot adaptation through in-context learning, providing an empirical foundation for translating the general-purpose capabilities of VLMs into physical behavior and clarifying the challenges that must be overcome for reliable deployment.

[53] PointZero: 3D Point Track Completion for Learning Transferable 3D Dynamics cs.CV | cs.ROPDF

Bardienus P. Duisterhof, Kaifeng Zhang, Adam Hung, Bowen Wen, Stan Birchfield

TL;DR: PointZero 用三维点轨迹补全作为预训练目标，在无需机器人动作标签的情况下学习可迁移三维动力学，并在动作条件预测和操作模仿任务上验证迁移价值。

Details

Motivation: 世界模型可预测交互下场景演化，但许多三维动作条件动力学方法依赖机器人动作标签，因而难以利用大量网络视频数据学习通用动力学先验。

Result: 给定单帧 RGB-D 观测和稀疏部分三维轨迹，PointZero 预测所有可见点的未来轨迹。作者构建涵盖可变形、铰接和刚体物体的 290 万合成帧数据集，并训练 Transformer 模型。相同数据条件下，PointZero 优于既有方法；后训练用于动作条件三维动力学预测和模仿学习时，在 PGND 基准上超过基线，并在七项模拟及真实机器人操作任务中的六项达到超过或持平基线的表现。作者还通过从头训练的对照实验区分架构与预训练目标/数据集的作用。

Insight: 预测点如何随时间运动可以作为不依赖机器人动作标签的三维动力学预训练任务，使多样化非机器人视频更可能用于后续机器人控制迁移。

Abstract: World models endow perceptual systems with the ability to predict how scenes evolve under interaction. They are most beneficial when trained on diverse volumes of data, to instill a rich prior into downstream applications. Existing methods typically require robot action labels to learn action-conditioned 3D dynamics, which excludes web video data from the training pool. We study 3D point track completion as a pre-training objective for learning transferable 3D dynamics without robot data. Given a single RGB-D observation and sparse partial 3D trajectories (tracks), we predict future 3D tracks of all observed points. We show this objective produces a rich 3D dynamics prior, without requiring robot action labels. We contribute a diverse dataset of 2.9 million synthetic frames spanning deformable, articulated, and rigid objects, and use it to train PointZero. We show that a flexible and expressive transformer, PointZero, outperforms prior methods on the same data. We demonstrate the utility of our pre-training objective by post-training PointZero for two downstream applications: (1) action-conditioned 3D dynamics prediction and (2) imitation learning. When fine-tuned to condition on end-effector pose, PointZero outperforms the baselines on the recent PGND 3D dynamics benchmark. When fine-tuned to predict robot actions and 3D tracks, PointZero outperforms or matches the baselines on 6/7 simulated and real-world robot manipulation tasks. We furthermore evaluate training PointZero from scratch to isolate the benefits of our proposed architecture from those of our proposed pre-training objective and dataset. We release the dataset, checkpoints, and full training recipe.

[54] PANORAMA: Panoptic Grounded Captioning via Mask Proposal Selection cs.CV | cs.CLPDF

Sara Pieri, Evangelos Kazakos, Shizhe Chen, Josef Sivic, Cordelia Schmid

TL;DR: PANORAMA 为全景图文描述提供实体级像素掩码，通过 phrase 条件候选掩码选择把文本生成与前景、背景区域定位联合起来。

Details

Motivation: 视觉语言模型能生成流畅详细的图像描述，却难把每个短语稳定地对应到图像像素；现有稠密描述加像素定位方案往往在描述完整性和掩码精度之间失衡。

Result: 论文提出全景接地描述任务，要求描述前景物体与背景区域，并为每个指代表达预测像素掩码。作者构建人工标注基准 PanoCaps，提供近乎完整的像素覆盖和实体级图文对齐；提出短语-掩码匹配协议及联合评估文本与掩码一致性的广义 Panoptic Quality 指标。PANORAMA 将短语接地表述为从短语条件掩码候选集中选择，并以短语上下文表示条件化预训练分割器，联合训练该接口和描述生成；一个短语可对应单一区域或多个实例。该方法在 PanoCaps 上整体接地表现最佳，并在多项像素级接地任务上达到或超过专用模型。

Insight: 先生成候选掩码再针对短语选择，使开放词汇描述与成熟分割器的空间能力相结合；完整像素覆盖基准和联合文本-掩码指标也能更全面检查描述是否真正落到图像内容上。

Abstract: Intelligent systems that act in the world require image understanding that is both comprehensive and spatially grounded. Current vision-language models (VLMs) can generate fluent and detailed image captions, but reliably associating them with image pixels remains challenging. Existing methods that combine dense captioning with pixel-level grounding often produce either incomplete descriptions or inaccurate segmentation masks. We study this problem through panoptic grounded captioning, a task that requires a VLM to describe both foreground objects and background regions while grounding each referring phrase with pixel-level masks. We make three contributions. First, we introduce PanoCaps, a human-annotated benchmark constructed from panoptic segmentation datasets. It provides dense captions with near-complete pixel coverage and image-text alignments at the entity level, supporting both training and evaluation. We further propose a phrase-mask matching protocol and a generalized Panoptic Quality (gPQ) metric that jointly evaluates textual and mask agreement. Second, we formulate phrase grounding as selection from a phrase-conditioned pool of mask proposals and introduce PANORAMA, a VLM that conditions a pretrained segmenter on contextualized phrase representations to obtain candidate masks and learns to select those corresponding to each phrase. Training this interface jointly with caption generation enables PANORAMA to produce high-quality masks while allowing each phrase to refer to a single region or multiple instances. Third, PANORAMA achieves the best overall grounding on PanoCaps and matches or exceeds specialized models across several pixel-level grounding tasks. Experiments show that our method produces precise entity-level segmentations while maintaining detailed, mask-consistent captions. Code, data and models are available at https://www.di.ens.fr/willow/research/panorama/.

cs.CL [Back]

[55] From Pixels to Pairs: A Comprehensive Benchmark of LLM-Based Key-Value Extraction in Noisy Document Settings cs.CL | cs.CVPDF

Zahra Anvari, Vassilis Athitsos

TL;DR: 该研究系统评测开源LLM在干净文本与OCR噪声条件下的键值抽取，覆盖FUNSD、CORD、SROIE及PaddleOCR、EasyOCR、Tesseract输出；模型在高质量文本上具备强语义抽取能力，但OCR损坏会显著降性能并缩小模型差距，主要错误包括键值错配、幻觉和数字损坏。

Details

Motivation: LLM越来越多用于文档结构化信息提取，但在真实OCR噪声下的行为尚不清楚。仅用干净文本评估可能高估实际部署效果，也无法区分输入质量、模型设计和提示策略各自的影响。

Result: 作者统一评估Gemma、Mistral、Qwen2.5、LLaMA 3和DeepSeek等指令微调解码器模型，在FUNSD、CORD和SROIE上进行键值对抽取；输入包括Gold人工文本，以及PaddleOCR、EasyOCR和Tesseract生成的OCR结果。统一协议用于隔离输入质量、模型设计和提示影响。结果显示，高质量文本下现代LLM是强语义抽取器，部分情况接近有监督布局感知系统；OCR噪声会显著降低性能，且输入损坏越大模型间差距越小。整体表现由文本语义推理和OCR文本保真度共同决定：大模型在干净文本上优势明显，但噪声下收益缩小，OCR质量成为主导因素。常见失败包括键值错配、幻觉及数字损坏。

Insight: 文档抽取管线的瓶颈会随输入质量变化：文本清洁时模型推理能力重要，OCR破坏时保真度成为主因；仅比较干净Gold文本会高估部署能力。摘要没有列模型分项分数或错误率，未量化不同OCR引擎的相对影响。

Abstract: Large language models (LLMs) are increasingly used for structured information extraction from documents, yet their behavior under realistic OCR noise remains poorly understood. We present a systematic benchmark of open-source instruction-tuned LLMs for key-value pair (KVP) extraction under both clean-text and noisy OCR conditions. We evaluate representative decoder-only models (Gemma, Mistral, Qwen2.5, LLaMA 3, and DeepSeek) on the FUNSD, CORD, and SROIE benchmarks using both Gold-text annotations and OCR outputs from PaddleOCR, EasyOCR, and Tesseract. A unified evaluation protocol isolates the effects of input quality, model design, and prompting under consistent conditions. The results show that modern LLMs act as strong semantic extractors when high-quality text is available, in some cases approaching supervised layout-aware systems. Under OCR noise, however, performance degrades substantially and performance gaps between models narrow as input corruption increases. Across all datasets, extraction performance is governed by two factors: semantic reasoning over text and preservation of textual fidelity under OCR noise. While larger models improve results on clean text, these gains diminish under noisy inputs, where OCR quality becomes the dominant factor. We also identify recurring failure modes, including key-value misalignment, hallucination, and numeric corruption. Our findings highlight the gap between clean-text evaluation and real-world deployment, emphasizing the need to jointly improve OCR quality, structural reasoning, and LLM-based semantic modeling.

[56] Think Before You Comfort: Reflective Cognitive Alignment for Protocol-Grounded Elderly Stimulation Agents cs.CLPDF

Jiyue Jiang, Ziyi Li, He Hu, Sheng Wang, Yuhan Chen

TL;DR: 该研究提出面向粤语认知刺激治疗的反思式对齐框架：STaR-CS用主持人风格和结构骨架合成多方对话缓解数据稀缺，RCA再以协议约束认知链及推理时价值对齐选择安全且有参与度的回应；六个LLM骨干和两位独立评审显示优于标准提示。

Details

Motivation: 认知刺激治疗可为认知障碍老年人提供非药物支持，但依赖受训主持人且低资源、隐私敏感语言（如粤语）数据严重不足。LLM可用于自动陪伴，但难以兼顾共情交流和认知刺激治疗规范。

Result: 研究沿两个方向构建框架。STaR-CS（风格迁移与角色条件化认知刺激）通过建模主持人风格并提取结构化对话骨架，合成多方治疗对话以缓解数据障碍。基于该语料的反思式认知对齐RCA，将刺激互动建模为序列决策过程，结合协议约束的思维链认知PC-CoC形成结构化推理，并用推理时价值对齐IVA依据安全与参与目标原则性选择回答。在六个底层LLM及两位独立评审上的评估显示，RCA相较标准提示基线持续提高协议遵循、安全性和小组主持能力。摘要未给出具体分数或数据规模。

Insight: 低资源治疗场景需要同时解决训练对话稀缺与模型偏离临床刺激规范的问题；先生成主持人风格语料，再以序列决策和推理时价值选择约束回应，形成互补处理路径。摘要未提供临床试验或真实老年人使用效果，结果仅为模型与评审评估。

Abstract: Cognitive Stimulation Therapy (CST) offers non-pharmacological support for elders with cognitive impairment, yet scalability remains constrained by reliance on trained facilitators and severe data scarcity, particularly for privacy-sensitive, low-resource languages such as Cantonese. While Large Language Models (LLMs) show promise for automated companionship, they often struggle to balance empathetic engagement with adherence to cognitive stimulation guidelines. We propose a framework addressing these challenges along two complementary axes. First, STaR-CS (Style-Transfer and Role-Conditioned Cognitive Stimulation) synthesizes multi-party dialogues through facilitator style modeling and structured skeleton extraction, mitigating data barriers. Building upon this corpus, the Reflective Cognitive Alignment (RCA) framework models stimulation interactions as a sequential decision process, integrating Protocol-Constrained Chain-of-Cognition (PC-CoC) for structured reasoning and Inference-Time Value Alignment (IVA) for principled response selection based on safety and engagement goals. Evaluations across six backbone LLMs and two independent judges show that RCA consistently improves protocol adherence, safety, and group facilitation over standard prompting baselines. Our code is available at https://github.com/jiangjyjy/RCA_Agent.

[57] Does Moral Reasoning Training Help or Hurt? Red-Teaming RL-Trained Ethical Agents with Persona Attacks cs.CLPDF

Arth Singh

TL;DR: 该研究对经道德奖励强化学习训练的Gemma和Llama智能体进行五种人格攻击红队测试，发现27B模型道德RL令平均对抗退化减少5.2倍、推理级奖励带来5.8倍鲁棒性但ETHICS准确率代价约11个百分点；面对虚构角色扮演仍易被攻破。

Details

Motivation: 道德奖励强化学习可使语言模型智能体更合作，但其行为能否抵抗对抗性人格诱导尚不清楚。检索上下文、工具输出和多轮措辞都可能注入角色指令，与模型的道德目标竞争。

Result: 作者对道德训练后的Gemma-2-27B/9B和Llama-3.1-8B开展五种人格攻击红队测试，并用噪声奖励控制、对抗PPO、表征分析、激活引导和注意力头消融探查因果机制。27B模型的道德RL将平均对抗退化降低5.2倍，但ETHICS准确率代价约11个百分点。205个场景、5个种子的实验中，推理级道德奖励带来5.8倍鲁棒性，而匹配随机奖励无效。训练改变了表征几何（平均CKA为0.82/0.83，对照噪声奖励为0.98），最强攻击处理位置提前8层；rank-1的L21方向可恢复完整PPO平均鲁棒性的83%。但对虚构角色扮演攻击仍然脆弱：L21引导仅恢复29%的差距，注意力头消融显示38个顺从头与25个对齐头竞争。Qwen对记录轮次敏感，Llama对选项字母代表“诚实”敏感。

Insight: 道德训练能形成部分线性、部分由回路分布式承载的鲁棒性，部分可以通过激活引导转移，但命名角色扮演仍可绕过它。鲁棒性提升伴随ETHICS能力代价且效果依模型和攻击类型变化，不能概括为全面安全。

Abstract: Moral-reward RL can make language-model agents more cooperative, but whether that alignment survives adversarial persona pressure is unknown. Such attacks are realistic: retrieved context, tool outputs, or multi-turn framing can all inject role instructions that compete with the agent’s moral objective. We red-team morally trained Gemma-2-27B/9B and Llama-3.1-8B agents with five persona attacks, then probe causality with noise-reward controls, adversarial PPO, representation analysis, steering, and head ablations. At 27B, moral RL cuts mean adversarial degradation by 5.2x but costs ~11pp ETHICS accuracy; across 205 scenarios and 5 seeds, reasoning-level moral reward yields 5.8x robustness while a matched random reward yields none. The training also reshapes representation geometry (mean CKA 0.82/0.83 vs. 0.98 for noise), moves peak attack processing 8 layers earlier, and exposes a rank-1 L21 direction that recovers 83% of full PPO’s average robustness. One failure mode survives all of this. Against Fiction role-play, L21 steering recovers only 29% of the gap, and head ablation finds 38 compliance heads competing with 25 alignment heads. Moral RL thus builds robustness that is partly linear and partly circuit-distributed, transferable through activation steering, yet still beaten by named-character role-play.

[58] Myovox: Reading Speech from the Muscles of the Face cs.CLPDF

Varshith Madishetty

TL;DR: Myovox从发声时面部肌肉的31通道表面肌电信号解码开放词汇英语文本；通过恢复复现实验、用双向Conformer及并行音频WavLM特征蒸馏、再集成声学模型并用7B语言模型重排，将单人语料WER从公开51.17%降至18.53%，但错误下限受约20.9%肌电音素错误率限制。

Details

Motivation: 面部肌电可为无声/非侵入式语音解码提供信号，但公开的emg2speech语料上性能有限。作者希望复现实验基线并逐步改善开放词汇文本解码，同时明确区分语言模型重排能力与肌电信号本身所能支持的上限。

Result: Myovox从发声时面部肌肉的31通道表面肌电解码开放词汇英语文本。第一步恢复公开代码中缺失的开放词汇解码设置，得到WER 40.63%、PER 39.02%的忠实基线，音素错误率与论文报告相差不到0.8个百分点。第二步将因果编码器替换为双向Conformer，通过并行音频的WavLM-Large第9层特征进行四项跨模态蒸馏，仅用肌电输入达到WER 26.14%、PER 22.34%。第三步集成两个声学模型、合并多尺度n-best候选并以QLoRA微调7B语言模型重排，WER达18.53%，为该语料最好报告结果，但不是其他语料肌电转文本的最高水平。400句留出测试集按官方8,500/760/400顺序划分；超参数只在验证集调优一次，再应用于测试。负面结果显示重排约止步于18.5%，因为瓶颈是肌电声学音素错误率约20.9%，正确词未出现在声学后验中，n-best Oracle仍有9.30%。

Insight: 语音解码性能提升来自逐步改善声学表征和候选重排，但语言模型无法恢复声学候选中完全缺失的正确词；肌电音素识别是最终瓶颈。论文明确区分该语料最佳结果与领域其他数据集结果，并报告了严格测试划分。

Abstract: Myovox, from myo (muscle) and vox (voice), decodes open-vocabulary English text from 31-channel surface electromyography (sEMG) recorded from the muscles of the face during vocalized speech. It takes the single-subject emg2speech General Corpus from a published 51.17% word error rate to 18.53%, in three separable moves, each measured in isolation. First, I recover the open-vocabulary decode settings missing from the public release and reach a faithful 40.63% WER / 39.02% PER baseline whose phone error rate matches the published one to within 0.8 points, so the acoustic model is reproduced faithfully. Second, I replace the causal encoder with a bidirectional Conformer trained by a four-term cross-modal distillation against the parallel audio’s WavLM-Large layer-9 features, reaching 26.14% WER / 22.34% PER from the electromyography alone. Third, I ensemble two acoustic models, union their multi-scale n-best lists, and rerank with a QLoRA-fine-tuned 7B language model, reaching 18.53% WER, the best result reported on this corpus, though not the best reported for sEMG-to-text on other corpora (Section 2). I then report the negative result that bounds the whole approach: reranking is exhausted at 18.5% because the binding constraint is the electromyographic acoustic phone error rate (~20.9%), not the language model. The correct words are simply absent from the acoustic posteriors, so no reranker can reach the 9.30% n-best oracle. All test numbers are on the 400-sentence held-out test set under the authors’ official 8,500 / 760 / 400 sequential split; every hyperparameter is tuned once on validation and applied once to test.

[59] The Limits of BPE Tokenization in Polish: Segmentation-Flexional Forms, Grammatical Anchoring, and First-Person Stability in Inflectional Language Models cs.CLPDF

Elzbieta Dawidek

TL;DR: 该文章以波兰语为例分析BPE分词对屈折语形态的局限：频率分词可能形成可解释片段，却不系统对应音位、语法类别或语境发音；作者提出“语法形式锚定”和分词屈折形式作为诊断框架，指出模型对话中的第一人称及性别形式会随上下文镜像或漂移。

Details

Motivation: 统计式子词分词是否能在屈折丰富语言中保留具有语言学意义的单位尚不明确。波兰语的正字法形式、音位与音节结构、派生、屈折词尾、语法形式及说话主体之间存在复杂关系，可用于检验频率分词的边界。

Result: 文章分析波兰语诊断词、儿童文本、波兰宪法序言选定形式、词族测试以及含变音符和鼻元音的例子。BPE片段有时与音节或可解释形态切分一致，但依赖书写形式频率，无法系统地将正字法表示映射到音位结构或语境相关语音实现。结果显示BPE稳定的是高频语法词缀表面片段，而非语法类别本身；例如ustanawiamy不仅是以-y结尾的序列，还锚定了动词变位、人称、数、时、式和体。作者提出“语法形式锚定”概念，并讨论poszłam、zrobiłam、byłam等无显式代词时仍能确定说话者位置的形式。与AI交互时，模型没有稳定的语法“I”，而是依上下文重建，可能镜像用户形式或切换语法性别。文章建议使用Roclawski分词屈折形式诊断分词边界，并探索子词稳定化、将语法形式锚定于屈折系统、表示句型和动词配价，以及维持对话中的语法自我。

Insight: 分词边界按表面频率形成，可能切碎音系与屈折关系，进而影响波兰语的语法一致性和模型第一人称稳定性；评估语言模型时需关注语法结构而非只看子词统计。摘要是语言学分析，未报告标准化模型基准或量化实验。

Abstract: This article analyzes BPE tokenization in Polish as a test case for the limits of statistical segmentation in an inflectional language. It asks whether frequency-based tokenization preserves linguistically relevant units, including orthographic form, phonemic and syllabic segmentation, derivational structure, inflectional endings, grammatical form, and the speaking subject. The material includes diagnostic words, a children’s text, selected forms from the Preamble to the Constitution of the Republic of Poland, word-family tests, and examples with Polish diacritics and nasal vowels. BPE tokenizers may produce segments that coincide with syllabic or morphologically interpretable divisions, but remain dependent on the frequency of written forms. They do not systematically map orthographic representation onto phonemic structure or context-dependent phonetic realization. The results show that BPE stabilizes frequent surface fragments of grammatical exponents rather than grammatical categories themselves. A form such as ustanawiamy is not merely a sequence ending in -y, but a verbal form anchored in conjugation, person, number, tense, mood, and aspect. The article develops the concept of grammatical form anchoring. In Polish, forms such as poszlam, zrobilam, or bylam can establish the position of the speaking subject without an explicit pronoun. In interaction with AI, this exposes a further problem: a language model does not possess a stable grammatical “I”, but reconstructs it contextually and may mirror the user’s forms or shift grammatical gender. Roclawski’s segmentation-flexional forms are proposed as a diagnostic framework for evaluating tokenization boundaries. More stable modeling of Polish may require sublexical stabilization, anchoring grammatical form in the inflectional system, representing sentence patterns and verbal valency, and maintaining the grammatical “I” in dialogue.

[60] English Word Sense Disambiguation in 2026: When the Labels Become the Bottleneck cs.CLPDF

Vassili Philippov, Amro Salman, Dmitrii Andreev, Penny Hands, Emil Kaiumov

TL;DR: 该研究构建经人工审定修订的英语全词义消歧基准lexEN和可审计SenseBench评测平台，发现前沿LLM在受WordNet候选约束下准确率约95.6%，标签质量已成为排名瓶颈；修订SemCor并重训模型可在未参与修订的测试集上提升数个F1点。

Details

Motivation: 作者认为英语全词义消歧中，前沿模型已足够准确，剩余金标准错误可能决定基准排名；标签质量既影响评测，也可能影响训练语料和最终模型能力。

Result: lexEN在Maru2022 ALL_NEW基准上进行保守的人工审定修订，改动211个标签并删除56个；SenseBench是可审计的LLM词义消歧评测框架和持续榜单，覆盖57个模型、192次运行。任务为从给定WordNet义项中选择的多选题，因此准确率是在获知候选义项帮助下的上限。lexEN-v1上前沿LLM收敛于约95%，最高95.6%，前三个模型家族统计上无法区分；准确率与推理开销、成本在约2,500倍价格区间内权衡。用前沿模型修订SemCor后，不改架构重训BEM、ESCHER和ConSeC，模型在从未参与重标的测试集上提升数个F1点。作者公开重标语料和298M参数双编码器Glite LENS，报告Raganato ALL为83.6、Maru ALL_NEW为87.4，服务成本约每百万条0.13美元。困难样本上，即便专家对细粒度WordNet义项也有分歧（三评审Fleiss κ=0.537）；粗化标签后四种义项体系中标注者一致率和模型准确率同步提升，顶尖模型在粗粒度下进入专家一致率范围，而细粒度仍显著低于专家。

Insight: 当模型能力接近标签噪声和专家分歧的上限时，提升标注质量可能比继续扩模型更有效；粗粒度义项更符合真实可判定边界。模型准确率受WordNet候选列表约束，不代表无约束理解能力，且成本成为主要限制。

Abstract: In English all-words word sense disambiguation (WSD), the labels, not the models, have become the bottleneck: frontier LLMs are accurate enough that the errors surviving in the gold standard decide benchmark rankings – in the test sets we score on and, as we show causally, in the corpus we train on. We release lexEN, a WSD evaluation benchmark built as a conservative, human-adjudicated correction layer over Maru2022’s ALL_NEW benchmark (211 labels changed, 56 removed), and SenseBench, an auditable LLM WSD evaluation harness and living leaderboard (57 models, 192 runs). The task is inventory-constrained multiple choice (the model picks from the supplied WordNet senses), so the reported accuracies are a ceiling on what models achieve without that help. On lexEN-v1 the frontier LLMs converge near 95% (best, 95.6%), the top three families are statistically indistinguishable, and accuracy trades off against reasoning effort and cost across a ~2,500x price span. Relabeling SemCor with frontier models and retraining BEM, ESCHER, and ConSeC unchanged lifts them by several F1 points on test sets the relabeling never touched; we release the relabeled corpora and Glite LENS, a 298M bi-encoder trained on the repaired labels – to our knowledge the strongest reported (83.6 Raganato ALL, 87.4 Maru ALL_NEW) – serving at ~$0.13 per million items. On hard items, fine-grained WordNet senses are partly ill-posed even for experts (three-reviewer Fleiss kappa=0.537); coarsening raises annotator agreement and model accuracy together across four inventories, placing a top model inside the expert agreement band at coarse granularity (statistically equivalent under three of four) but significantly below it at fine. The binding constraint is now cost.

[61] Confidence Comes from Experience: Experiential Confidence Estimation from Reasoning to Agents cs.CL | cs.AIPDF

Caiqi Zhang, Xiaochen Zhu, Chengzu Li, Yulong Chen, Dharshan Kumaran

TL;DR: XConf以模型自身过往任务、反思、自报置信度、真实结果及事后经验构成记忆，先检索相似任务同等置信度下的历史成功率，再让模型反思校准；在九个推理、代码、多模态QA和智能体基准上，24项比较中23项AUROC不低于十样本自洽，校准误差更低且生成成本约为其十分之一。

Details

Motivation: 可靠置信估计决定语言模型答案应直接交付、升级人工处理还是重试。现有方法只读取当前推理过程，包括自省、token概率打分或重复采样；作者认为当前推理不足以估计置信度，模型积累的历史经验也应参与判断。

Result: XConf（经验置信度）将模型过去的分级任务记录为经验：每条包含任务、模型反思、自报置信度、实际结果及得知评分后写下的经验教训。新任务到来时，Recall阶段检索任务相似且自报置信度相近的历史案例，读取其成功率；Reflect阶段向模型展示这些记录，要求识别反复失败模式并根据自身历史重新陈述置信度。方法无需logit访问或更新模型权重，且只需生成一次答案。在九个覆盖推理、编码、多模态问答和交互智能体的基准、三个模型家族四个模型上，24项AUROC比较中XConf有23项优于或持平于十次采样自洽，同时ECE更低，生成成本约为其十分之一。选择性预测时，拒绝最低置信度的10%样本可使智能体任务交付成功率最高提高8.7个百分点。

Insight: 置信度可从模型过往在类似任务、类似自报把握下的实际成绩中校准，而不只依赖当前单次推理或昂贵采样；显式回顾失败模式进一步改善置信判断。有效性依赖积累到与新任务相似的经验记录，摘要未报告冷启动和经验分布变化表现。

Abstract: Reliable confidence estimation is increasingly central to the trustworthy deployment of language models: a calibrated estimate of the probability that an output is correct decides what to ship, what to escalate, and what to retry. Existing confidence estimators, however, share one design premise: they only read the current inference process, either by introspecting on it, scoring its token probabilities, or resampling it. We argue that the current inference is not a sufficient basis for confidence. We propose XConf (eXperiential Confidence): estimating confidence together with the model’s accumulated experience. The experience is stored as a record of the model’s own graded past episodes, each holding the task, the model’s reflection, its stated confidence, the outcome, and a lesson written once the grade arrived. Given a new task, XConf’s Recall stage retrieves past episodes on similar tasks met with a similar stated confidence, and reads off their historical success rate; its Reflect stage shows the model this record, has it name its recurring failure mode, and restate a confidence now informed by its own track records. Our estimator is format-general, requiring no logit access or weight updates, and costs only one answer generation. Across nine benchmarks spanning reasoning, coding, multimodal QA, and interactive agents, and four models from three families, XConf beats or matches ten-sample self-consistency in discrimination (AUROC) on 23 of 24 comparisons, with much lower calibration error (ECE), at a tenth of the generation cost. Used for selective prediction, abstaining on the 10% least-confident episodes raises the delivered success rate by up to 8.7 points on agent tasks. We therefore see experiential confidence estimation as a new paradigm for future general-purpose confidence estimation.

[62] SFT or RL for Tool-Calling Agents? A Controlled Study Across Data, Method, and Scale cs.CLPDF

Md Tahmid Rahman Laskar, Xue-Yong Fu, Shashi Bhushan TN

TL;DR: 该控制研究在0.6B至32B六种Qwen3模型上比较LoRA监督微调、GRPO强化学习及二阶段训练用于工具调用：SFT+LoRA在18个域内设置中的15个最佳，跨数据集迁移时GRPO在54种设置中赢29次但优势平均不足1点，混合训练数据普遍迁移较稳。

Details

Motivation: 关于工具调用智能体的训练数据、适配方法与模型规模如何共同影响性能，现有受控证据有限。需要在统一实验中比较监督微调、强化学习及二阶段训练的域内效果和跨数据集迁移。

Result: 研究比较LoRA监督微调、Group Relative Policy Optimization（GRPO）强化学习及SFT后接GRPO，在0.6B到32B的六种Qwen3模型上评估域内表现与跨数据集迁移。SFT+LoRA在整个参数规模范围内都是最强域内方法，在18种实验设置中15种最佳。跨数据集迁移时方法差距更小：训练集与测试集不同的54种设置中GRPO赢29种，但相对SFT的平均优势不到1个百分点；SFT→GRPO很少在两类评估中成为最强方法。混合训练数据无论适配方法如何，都提供稳定强迁移，同时接近专门训练的域内表现。额外分析显示LoRA优于全参数微调，更能保留预训练智能体行为。

Insight: 工具调用任务中，SFT+LoRA是稳定的域内选择，GRPO在跨数据集设置略占次数优势但平均增益很小；数据多样性可能比复杂的二阶段训练更可靠地改善迁移。结论限于六种Qwen3规模及所测数据集配置。

Abstract: Limited controlled evidence exists on how training data, adaptation method, and model scale jointly affect tool-calling performance in language-model agents. We evaluate supervised fine-tuning (SFT) with LoRA, reinforcement learning (RL) via Group Relative Policy Optimization (GRPO), and SFT followed by GRPO across six Qwen3 models from 0.6B to 32B parameters, covering both in-distribution performance and cross-dataset transfer. SFT with LoRA is the strongest in-distribution method throughout the 0.6B-32B range and best in 15 out of 18 experimental settings. On cross-dataset transfer, the methods are closer: GRPO wins 29 out of 54 settings where training and test datasets differ, but its margin over SFT averages under one point, and SFT->GRPO is rarely strongest in either comparison. Dataset mixing gives consistently strong transfer while staying close to specialized in-distribution training, regardless of method. Additional analysis further confirms that LoRA outperforms full-parameter fine-tuning, demonstrating that LoRA better preserves pretrained agentic behavior.

[63] DualSQL: Text-to-SQL with Multi-Agent Reinforcement Learning cs.CL | cs.AI | cs.DBPDF

Shijie Chen, Yu Gan, Yeounoh Chung, Jiani Zhang, Quannan Li

TL;DR: DualSQL用共享骨干和脚手架联合训练负责模式链接与SQL生成的两个智能体，并以数据库工具支持多步推理、以REX指标评估执行正确性；仅用3,755例训练，4B模型在BIRD开发集达68.0%，8B达71.1%。

Details

Motivation: 先进文本到SQL系统常以多智能体流水线分别处理数据库模式链接和SQL生成，但为两项任务单独训练模型，未利用它们之间的协同关系；多智能体强化学习训练还容易模型坍塌。

Result: DualSQL由两个智能体组成，共享同一模型骨干权重及智能体脚手架，通过稳健的多智能体强化学习联合优化。系统提供三个数据库访问工具，使模型能够基于实际数据库交互进行多步推理；为稳定训练，作者引入rollout护栏机制。论文另提出稳健执行匹配REX指标，更准确评估SQL正确性并生成奖励信号。仅使用3,755个训练样例，DualSQL-4B在BIRD开发集执行准确率为68.0%，与先前7B模型相当；DualSQL-8B达到71.1%，超过既有单模型32B系统。

Insight: 模式链接和SQL生成是相互依赖任务，使用共享模型并联合强化学习可利用两者协同，缩小与更大模型的差距。BIRD开发集上的执行准确率不等同于跨数据库泛化，摘要未报告其他基准和随机方差。

Abstract: State-of-the-art Text-to-SQL systems are typically multi-agent pipelines centered around two fundamental tasks: schema linking and SQL generation. However, existing work trains separate models for each task, failing to leverage the synergy between these interrelated tasks. In this work, we propose DualSQL, a new Text-to-SQL system consisting of two agents powered by a single model backbone. The agents share the same model weights and agentic scaffold, enabling joint optimization through a robust multi-agent reinforcement learning (RL) framework. We design three database access tools to facilitate effective multi-step reasoning grounded to interactions with the databases. To improve training and avoid model collapse, we introduce a set of rollout guardrail mechanisms that stabilizes multi-agent RL training, supporting DualSQL to keep improving during training. We also introduce a new SQL correctness metric, robust execution match (REX), to more accurately judge SQL correctness and assign reward signals. Being trained on only 3755 examples, DualSQL-4B achieves an impressive 68.0% execution accuracy on the BIRD development set, matching previous 7B models. DualSQL-8B further improves to 71.1%, outperforming previous state-of-the-art single-model solutions with 32B parameters. These results demonstrate the strength of joint multi-agent reinforcement learning for building high performance Text-to-SQL pipelines.

[64] I code or AI code: A comparative evaluation of AI-rated scores in classroom observations cs.CL | cs.AIPDF

Y. Fong, J. Xiang, T. Y. D. Chan, K. Lee, E. Y. H. Lau

TL;DR: 该研究比较GPT-5与人类评分者对香港幼儿园师幼互动的CLASS观察评估，分析30所幼儿园38个班级的87段录像转录；AI在情感支持、尤其反馈质量上与人类更一致，在程序性和依赖情境的课堂组织与教学支持维度分歧更大。

Details

Motivation: 课堂观察可用于建立教育质量基准和指导教学改进，但依赖受训观察者、成本高。研究评估LLM能否根据转录文本按完整课堂评估评分系统（CLASS）评估师幼互动，并与人工评分进行比较。

Result: 研究使用GPT-5对香港30所幼儿园、38个班级的87段录像观察转录进行评分，覆盖早期儿童课堂，并将AI评分与人类评分比较各CLASS领域及维度的相关性和均值差异。AI在情感支持领域、尤其帮助教师拓展儿童学习的反馈质量维度，与人工评分更趋一致；对于更程序化或依赖具体情境的互动，分歧更大，主要集中在课堂组织和教学支持领域。作者据此认为，基于转录文本的AI评分可捕捉师幼互动的部分相对变化，但尚不能在完整CLASS体系下稳定复现校准后的人类判断。

Insight: 文本转录可能足以评估部分较明确的互动特征，但课堂组织和教学支持依赖视觉及场景信息，单靠文本较难达到人类一致性；AI更适合初步筛查和支持教师反思，而非替代受训观察者或用于高风险评价。摘要未提供具体相关系数。

Abstract: Classroom observations are widely recognized as a key tool for establishing benchmarks of education quality and guiding pedagogical improvement, yet they remain resource-intensive and dependent on trained observers. This study evaluated the feasibility of using a LLM (GPT-5 model) to score teacher-child interactions in early childhood classrooms, benchmarked against human raters. The study analyzed 87 video-recorded observations from 38 classrooms across 30 kindergartens in Hong Kong. Using observation transcripts, the AI model was configured to apply the full Classroom Assessment Scoring System (CLASS) framework. AI-rated scores were then compared with human ratings by examining correlations and differences in mean scores of the CLASS domains and dimensions. The results showed greater convergence between AI and raters for the Emotional Support domain and, in particular, the Quality of Feedback dimension, which captures how teachers use feedback to extend children’s learning. Greater divergence emerged for interactions that were more procedural or context-dependent, particularly within the Classroom Organization and Instructional Support domains. These findings suggest that transcript-based AI scoring may capture some of the relative variation in teacher-child interactions but cannot yet reproduce calibrated human judgements consistently across the full CLASS framework. AI-assisted observation may therefore be more appropriate as a preliminary screening tool rather than as a replacement for trained observers, providing teachers with evidence for reflection rather than high-stakes evaluation. Future research should examine whether domain-specific training and incorporation of contextual and visual information can improve alignment between AI and human rated scores.

[65] Too Good to Be Real? Diagnosing and Reducing the Gap Between AI Preference and Real User Engagement cs.CLPDF

Xinglang Zhang, Yuanmeng Xiang, Yunyao Zhang, Zeliang Chen, Junqing Yu

TL;DR: 该研究比较LLM偏好与真实用户互动数据，分析来自知乎、Quora和Reddit的117万条答案、25,978个问题，发现高互动用户答案更偏情感与表达显著性，而LLM偏好过度强调逻辑结构；OMRA通过掩码重建过度解释片段，在四种LLM家族中平均缩小54.4%偏好差距。

Details

Motivation: LLM越来越多地生成和评估在线内容，但其认为能提升互动的写作特征是否符合真实用户偏好尚不清楚。若模型评估标准与真实平台互动规律不同，用其优化内容可能造成系统性风格偏差。

Result: 研究比较知乎、Quora和Reddit上1,170,000条回答与25,978个问题的真实平台答案及AI生成答案，并在四种问题内互动水平上分析。作者提出本体偏好测量，从逻辑、情感和表达三个维度表示答案。随着目标互动水平增加，LLM添加更多明确逻辑结构；真实用户互动则更强关联于情感和表达显著性，作者将该倾向称为“逻辑过度绑定”。据此提出本体掩码推理自编码OMRA，遮蔽并重建过度解释片段，同时保持立场、事实内容和连贯性。四个LLM家族上，OMRA平均缩小54.4%的测量偏好差距。人工评估中，OMRA在62.4%的成对偏好判断中胜过匹配的真实平台答案，但真实答案更常被评审认为由人类撰写。

Insight: LLM偏好更结构化、更逻辑显式的答案，而真实用户互动未必遵循这一模式；内容优化若只拟合模型评分会过度强化逻辑结构，压低情感和表达特征。OMRA提高了偏好匹配，但真实答案仍更像人类，互动指标与人类感知不等价。

Abstract: Large language models are increasingly used to generate and evaluate online content, yet it remains unclear whether the qualities they associate with higher engagement match what real users respond to. We study this question using 1.17 million answers to 25,978 questions from Zhihu, Quora, and Reddit, comparing real platform answers and AI-generated answers across four within-question engagement levels. We introduce Ontological Preference Measurement, which represents answers along three dimensions: logic, affect, and expression. We find a systematic gap between AI preference and real user engagement: as target engagement increases, LLMs add more explicit logical structure, while real user engagement is more strongly associated with affective and expressive salience. We call this tendency logic overbinding. Based on this diagnosis, we propose Ontology-Masked Reasoning Autoencoding (OMRA), a controlled intervention that masks and reconstructs over-explained spans while preserving stance, factual content, and coherence. Across four LLM families, OMRA reduces the measured gap by an average of 54.4%. In human evaluation, OMRA wins 62.4% of pairwise preference judgments against matched real platform answers, even though the real answers are more often judged to be human-written.

[66] SEA-LION-v4.8: A Technical Report cs.CLPDF

Ahmed Mohammad Dabeer, Ahn Jeongmi, Anocha Sutaveephamochanon, Antonyrex Sajeban, Aulia Adila

TL;DR: Nemotron-SEA-LION-v4.8是一组面向东南亚语言的30B-A3B和120B-A12B模型，基于Nemotron 3继续预训练并用区域语言、推理、代码、多语言平行数据及在线策略蒸馏后训练；SEA-HELM总体分数分别从46.06升至51.57、49.30升至63.44。

Details

Motivation: 东南亚语言多样，模型需要兼顾区域语言指令遵循、自然语言理解、推理及代码能力。作者希望构建一个服务多种东南亚语言的统一模型家族，并系统报告继续预训练和后训练方案。

Result: Nemotron-SEA-LION-v4.8基于NVIDIA Nemotron 3，包含30B-A3B和120B-A12B两个规模，每种都有继续预训练基础模型和后训练变体。模型使用东南亚语料、推理与代码数据以及多语言平行语料适配，随后经过监督微调和在线策略蒸馏后训练。在SEA-HELM上，30B-A3B总体SEA得分从46.06提高到51.57，120B-A12B从49.30提高到63.44。最大增益出现在七种东南亚语言的指令遵循、自然语言推理和自然语言理解任务。

Insight: 多语言区域模型在继续预训练后结合监督微调和在线策略蒸馏，可同时提升多类东南亚语言任务。摘要只报告SEA-HELM总分与增益较大的任务类别，未列各语言细项和不同阶段的独立贡献。

Abstract: We introduce Nemotron-SEA-LION-v4.8, a family of Southeast Asian Languages in One Network (SEA-LION) built upon NVIDIA Nemotron 3. The family includes 30B-A3B and 120B-A12B models, with both continued-pretrained base checkpoints and post-trained variants. We adapt the models using Southeast Asian, reasoning, code, and multilingual parallel datasets, followed by post-training with supervised fine-tuning and online on-policy distillation. On SEA-HELM, the 30B-A3B model improves the overall SEA score from 46.06 to 51.57, while the 120B-A12B model improves from 49.30 to 63.44. The strongest gains are observed in instruction following, natural language reasoning, and natural language understanding across seven Southeast Asian languages.

[67] Attention Dispersion as a Diagnostic Signal for Hallucination in Large Language Models cs.CL | cs.LGPDF

Shardul P. More, Tanuja S. Pawar

TL;DR: 该研究将中间层注意力熵的时间波动作为免训练幻觉诊断信号，提出无监督注意力分散度指标；在Qwen2.5-1.5B和3B的GSM8K、MATH-500上，所有受测条件相较输出置信基线的AUC最高提升0.076。

Details

Motivation: LLM常出现幻觉，尤其在复杂推理中影响可靠性。传统检测依赖输出置信度或logit，但现代对齐训练可能使其校准失效；作者探索模型内部注意力变化能否提供独立的诊断线索。

Result: 研究分析推理过程中内部注意力机制的时间波动，提出无监督注意力分散度指标，度量注意力熵变化。作者发现，中间层注意力熵峰值与推理崩溃相关，表明认识不确定性会在内部表示中留下可测痕迹。在GSM8K和MATH-500上使用Qwen2.5-1.5B及3B评测，所有测试设置中，该方法相较基于输出的基线均有统计显著的AUC提升，最高为0.076。

Insight: 内部注意力随推理过程的分散变化可补充经对齐模型可能失准的输出置信估计，但相关性不等于因果解释，也不说明所有错误都表现为注意力熵峰值。作者指出仍需扩展到更多模型家族和任务领域。

Abstract: Large Language Models (LLMs) frequently exhibit hallucinations, presenting a major barrier to reliability in complex reasoning tasks. While traditional detection methods rely on output-based confidence metrics, these logits are often miscalibrated by modern alignment techniques. In this paper, we investigate the temporal volatility of internal attention mechanisms as an alternative diagnostic signal for hallucination that does not depend on output calibration. By introducing an unsupervised metric for attention dispersion, we show that epistemic uncertainty leaves a measurable trace within intermediate layers, where spikes in attention entropy are associated with reasoning breakdowns. We evaluate our approach on mathematical reasoning benchmarks (GSM8K and MATH-500) using the Qwen2.5 model family (1.5B and 3B parameters), finding statistically significant AUC improvements of up to +0.076 over output-based baselines across all tested conditions. These findings suggest that attention dispersion is a promising complement to traditional hallucination detection methods, requiring further investigation across broader model families and task domains.

Christopher Bagdon, Carina Silberer, Roman Klinger

TL;DR: Mult2EMo是包含社交媒体多模态多情绪表达的数据集，同时收集作者和读者对帖子及触发事件的标注；研究发现读者可以还原情绪但任务困难，了解引发事件对准确重建至关重要，且依赖图像表达情绪的帖子尤难识别。

Details

Motivation: 社交媒体作者常将文本与图像结合表达情绪，但既有情绪分析多只使用文本，也忽略引发情绪的现实事件及其与帖子内容的关系，因而无法充分衡量读者是否能重建作者原意。

Result: 作者研究社交媒体作者对触发事件的体验与帖子内容之间的关系，重点评估读者能否重建情绪表达；并建立多模态多情绪模型数据集Mult2EMo，同时收集作者和读者对帖子及其触发事件的标注。实验发现，无论人类读者还是计算模型都能够进行情绪重建，但任务具有挑战性。理解触发事件对准确重建至关重要；当帖子主要依靠图像表达情绪时，重建尤其困难。摘要未给出数据集规模和具体准确率。

Insight: 情绪表达不只来自帖子文本，还受真实事件背景和图像模态影响；不提供触发事件可能使读者无法恢复作者真实体验。多模态评估应把事件、作者视角和读者感知一并纳入。

Abstract: Emotions are an essential aspect of human communication, particularly on social media, where authors frequently combine text and images to convey their emotions. Yet prior work on emotion analysis of social media posts has overlooked two important aspects in regard to measuring how well readers can reconstruct the authors’ intent: (1)~~the image modality, with most work focusing solely on text, and (2)~~the real-world events that trigger the expressed emotions, and their relationship to the post content. We therefore study the relation between (a) the author’s experience of the event that caused them to write a social media post and (b) the content of the post, with a focus on readers’ capability to reconstruct that emotion expression. To do that, we introduce the Multimodal Multi-Emotion-Model dataset Mult2EMo, created by collecting annotations from both authors and readers on the posts and their triggering events. We find that reconstruction is possible but challenging for both human readers and computational models. We show that understanding the triggering event is crucial for accurate reconstruction, and that reconstruction is particularly challenging when posts rely heavily on the image to express emotion.

[69] Align, Integrate, and Fire: Efficient Token-Level Alignment for Zero-Shot SpeechLLMs cs.CLPDF

Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes, Gerasimos Spanakis

TL;DR: Aligned Continuous Integrate-and-Fire通过动态时间规整把连续声学帧压缩为与目标文本完全相同长度的离散token，训练初期用轻量距离损失建立声学—语义桥而跳过LLM前向，后续只对单个LLM层蒸馏；在ASR和语音翻译上超过既有参数高效基线。

Details

Motivation: LLM具备强文本处理能力，但高效扩展到语音输入仍困难。现有SpeechLLM要么进行昂贵全模型微调，要么使用参数高效投影器，却可能导致token序列冗长并依赖高成本全模型监督。

Result: 作者提出Aligned Continuous Integrate-and-Fire框架进行零样本语音处理。方法通过显式动态时间规整对齐，将连续声学帧动态压缩到与目标文本完全相同的离散token长度。初始训练阶段采用轻量距离度量建立声学到语义的稳健桥接，完全绕过计算昂贵的LLM前向过程；后续微调用内存高效的知识蒸馏目标，只监督单个LLM层，但性能可与全模型交叉熵训练竞争。自动语音识别和语音翻译实验显示，该方法超过既有参数高效基线。摘要未给数据集名称或分项分数。

Insight: 把声学帧对齐到目标文本token长度，可避免传统软投影产生的长序列；分阶段训练再将监督成本集中在少量层上，从而降低零样本语音扩展的计算开销。摘要只报告优于基线的方向，未量化速度或准确率收益。

Abstract: While Large Language Models excel in natural language processing, efficiently extending their capabilities to spoken input remains a significant challenge. Existing methods for building SpeechLLMs often rely on computationally expensive full-model fine-tuning, or employ parameter-efficient projectors that suffer from inefficient token sequence lengths and costly full-model supervision. In this paper, we introduce Aligned Continuous Integrate-and-Fire, a highly efficient framework for zero-shot speech processing. Our method dynamically compresses continuous acoustic frames into the exact discrete token length of the target text utilizing explicit Dynamic Time Warping alignments. This allows our initial training stage to establish a robust acoustic-to-semantic bridge using lightweight distance metrics, entirely bypassing the computationally expensive LLM forward pass. For subsequent fine-tuning, we propose a memory-efficient knowledge distillation objective that targets a single LLM layer, performing competitively with full-model cross-entropy training at a fraction of the computational cost. Through extensive evaluations on Automatic Speech Recognition and Speech Translation, we demonstrate that our method achieves superior performance compared to prior parameter-efficient baselines.

Zhuo Chen, Zhen Zhang, Xinyu Wang, Kewei Tu

TL;DR: 该研究把多轮智能体轨迹表示为轮次依赖DAG，删除对最终答案不关键的冗余轮次并微调，从而降低成本；四个多模态QA基准上准确率最高较普通SFT提升1.7个百分点、较LLM删除基线提升5.7个百分点，同时推理消息最多减少40%、token最多减少48%。

Details

Motivation: 多轮智能体轨迹中常包含失败工具调用、并行子查询和仅用于验证的轮次，这些冗余会增加训练和推理成本。需要识别哪些轮次对最终答案有全局依赖，哪些可安全删除。

Result: 作者将每条轨迹建模为轮次级依赖有向无环图DAG，标出对最终答案起关键作用的轮次。依据LLM标注的DAG进行确定性、可解释的轨迹精简，并允许对剩余内容改写，再用精简轨迹微调智能体。四个多模态问答基准上的结果显示，精简轨迹训练模型持续优于原始轨迹训练，且推理成本更低：相较普通SFT准确率最高提高1.7个百分点，相较LLM删除基线提高5.7个百分点；每样本推理消息最多减少约40%，token最多减少约48%。

Insight: 轨迹压缩应依据轮次对答案的依赖关系，而非仅按轮数或文本长度启发式删除；结构化DAG使精简更可解释，也能同时减少训练数据冗余与推理开销。摘要未列基准名称、绝对分数或自动生成依赖图的误差。

Abstract: Multi-turn agent trajectories often contain redundant rounds (failed tool calls, parallel sub-queries, verification-only steps) that inflate both training and inference cost. We propose viewing each trajectory as a \emph{round-level dependency DAG} that exposes which rounds are globally load-bearing for the final answer, and fine-tune agents on trajectories refined through this DAG. Given an LLM-annotated DAG, these edits are deterministic and interpretable, with optional rephrasing. Models trained on these refined trajectories consistently outperform those trained on the original trajectories at lower inference cost. Specifically, across four multi-modal QA benchmarks, our refinements improve downstream accuracy by up to $1.7$,pp over vanilla SFT (and $5.7$,pp over an LLM-deletion baseline) while reducing per-sample inference messages by up to approximately $40%$ and inference tokens by up to approximately $48%$, translating to substantial savings in compute and serving cost. Code is available.

[71] STRETCH the Boundaries: A Unified Self-Taught Framework for Progressive LLM Evolution cs.CLPDF

Yajie Yu, Mark Lee, Yue Feng

TL;DR: STRETCH通过动态Stretch Zone让训练难度随模型能力变化，在同一参数空间交替训练生成挑战的Scaffolder和通过强化学习求解的Learner；在谈判和运筹基准上超过强提示及领域基线，并通过难度对齐稳定持续推理提升。

Details

Motivation: LLM自我提升训练常因难度固定而停滞：模型能力逐步增强后，原有任务不再提供有效挑战，导致学习目标与当前水平脱节。

Result: STRETCH（通过定向挑战实现自教式推理演化）受认知支架理论启发，提出动态Stretch Zone机制，让问题难度持续匹配模型当前解题能力。模型在同一参数空间中交替扮演Scaffolder和Learner：前者生成适应性、推动能力边界的挑战；后者通过强化学习优化解题轨迹。该双循环共同演化旨在稳定训练、减少奖励投机并逐步提升推理能力。在谈判和运筹学基准上，STRETCH持续超过强提示及领域专用基线；Scaffolder配置分析进一步表明，动态难度对齐对持续能力提升和同步推理演化至关重要。摘要未提供具体基准分数。

Insight: 自我提升不应让模型反复处理固定难度任务，而应根据最新能力动态生成略具挑战性的样本；挑战生成与解题策略可在同一训练闭环中共同更新。摘要未报告训练成本、模型规模和具体性能增幅。

Abstract: Large language models (LLMs) often suffer from capability stagnation in self-improvement training because fixed difficulty levels fail to adapt to their evolving proficiency. To address this issue, we propose STRETCH (Self-Taught Reasoning Evolution via Targeted CHallenge), a unified framework inspired by cognitive scaffolding theory. STRETCH introduces a dynamic Stretch Zone mechanism that continuously aligns question difficulty with the model’s solving capability. Within a single parameter space, the model alternates between a Scaffolder that generates adaptive, boundary-pushing challenges and a Learner that that optimizes its solving trajectories through reinforcement learning. This dual-loop co-evolution effectively stabilizes training, mitigates reward hacking and promote progressive reasoning growth. Experiments on both negotiation and operation research benchmarks demonstrate that STRETCH consistently outperforms strong prompting and domain-specific baselines. Further scaffolder configuration analysis shows that dynamic difficulty alignment is critical for sustained capability improvement and synchronized reasoning evolution.

[72] Voice of Reason: Reinforcement Learning for Spoken Math cs.CLPDF

Timothée Weisselberger, Edouard Graves, Alexandre Défossez

TL;DR: 该研究对GLM-4-Voice进行语音数学强化学习：先用合成口语问答监督微调，再用可验证奖励RL提升GSM8K准确率，即使不增加额外推理token也超过既有语音模型成绩；结合流式推理技术后自由形式准确率达74.8%。

Details

Motivation: 原生语音语言模型可比级联系统提供更自然的语音交互、使用副语言信息并降低延迟，但在数学推理基准上的准确率仍落后文本模型。文本领域中可验证奖励强化学习能改善复杂问题求解并减少幻觉，作者探索其能否缩小语音数学推理差距。

Result: 作者在GLM-4-Voice语音模型上开展训练，先以合成的口语问答数据监督微调模型适应数学领域，再进行带可验证奖励的强化学习。即使不增加额外推理token，RL也能使GSM8K准确率超过此前语音模型只有加入额外推理轨迹时达到的水平。若与既有流式推理技术结合，自由形式准确率进一步达到74.8%，为语音原生模型的数学能力报告新的最佳结果。摘要未提供无RL或各阶段的具体准确率。

Insight: 可验证奖励强化学习能够直接提升语音原生模型的数学推理，不必依赖额外长思维链；与流式推理结合后进一步提高表现。该结果来自GLM-4-Voice和GSM8K，摘要未列其他数据集、语音输入条件和延迟变化。

Abstract: Speech language models enable richer spoken interactions between humans and machines than cascaded systems, allowing access to paralinguistic information and lower latency. However, their accuracy on mathematical reasoning benchmarks has lagged behind those of text models. Reinforcement learning (RL) with verifiable rewards has been instrumental in extending text models’ capabilities for solving complex problems and limiting hallucinations. In this work, we explore applying RL to the GLM-4-Voice speech model (Zeng et al., 2024) to bridge the gap between textual and spoken mathematical problem solving. We first adapt the model to the domain using supervised fine-tuning on synthesized spoken question-answering data. We then show that, even without extra reasoning tokens, RL improves the accuracy on GSM8K beyond levels previously achieved for speech models only with supplementary reasoning traces. When combined with existing streaming reasoning techniques, we show further gains to 74.8% free-form accuracy. This establishes a new state-of-the-art for mathematical spoken abilities with speech-native models.

[73] ReFigBench: Benchmarking Scientific Figure Reconstruction as Editable PowerPoint Artifacts cs.CL | cs.CVPDF

Liyang Fan, Chi Wei, Yitai Li, Xinping Bi, Guhong Chen

TL;DR: ReFigBench评估多模态编码智能体将1,000张arXiv科学总览图重建为可编辑PPT时的感知、规划和工具链影响，结合确定性文件检查、跨模型自动评分及盲道人评；发现模型与harness共同决定工作流收益，专业PPT流程每种配置都会破坏原生连接符，体现视觉保真与可编辑性的冲突。

Details

Motivation: 多模态编码智能体需要将图像转成可用的编辑产物，但现有评测多关注短工具调用、API轨迹或截图相似度，低分无法判断问题来自感知、规划还是harness工具环境。科学总览图重建要求同时保留文字、拓扑、布局和PPT原生结构，适合检验复杂工具链能力。

Result: ReFigBench从arXiv论文中收集1,000张具有完整出处的真实总览图，评估将其重建为可编辑PowerPoint幻灯片的智能体。四个模型家族分别使用直接代码生成和专门PPTX工作流，最强模型还在两个商业harness中运行，共形成十种配置。评测组合确定性产物检查、两个模型家族评审的重复自动评分及盲道人类比较。感知仍是瓶颈，迭代渲染只能部分弥补；专业工作流的收益依赖具体模型和harness，同一模型在一个harness中获益、在另一个中退化，固定直接提示也会受harness影响。专业流程在所有配置中都会破坏原生连接符，但人评多数情况下仍偏好其渲染结果；即使最强智能体也未达到评分上限。

Insight: 图像重建结果不只取决于模型能力，工具链和输出格式也会改变质量；截图逼真与原生可编辑结构存在核心张力。基准同时评测可视结果和PPT结构，能定位单一截图相似度遗漏的问题。摘要未列各模型具体分数。

Abstract: Multimodal coding agents are expected to turn visual inputs into usable artifacts, and they act through a harness, the layer of tools, context management, and execution environment around the model. Existing evaluations often isolate short tool calls, API traces, or screenshot resemblance, and a low score under these proxies cannot say whether the model saw poorly, planned poorly, or was failed by its harness. We study scientific overview figure reconstruction, an agent task in which a source image must become an editable PowerPoint slide that preserves text, topology, layout, and native document structure. We introduce ReFigBench, a benchmark and evaluation framework built on 1,000 real overview figures retrieved from arXiv papers with full provenance. Coding agents from four model families reconstruct every figure under two workflows, direct code generation and a specialized PPTX workflow, and the strongest model runs inside two commercial harnesses, yielding ten configurations. Evaluation combines deterministic artifact checks, repeated automated scoring by judges from two model families, and blinded human comparisons. Perception remains a bottleneck that iterative rendering only partly repays. Whether workflow effort converts into quality depends on the model together with its harness, since the same model gains from the specialized workflow inside one harness and loses inside the other, and the harness shifts scores even under an identical direct prompt. The specialized workflow erases native connectors in every configuration, human judges still prefer its renderings in most matchups, and even the strongest agent falls short of the rubric ceiling. These results expose the tension between fidelity and editability as the central challenge for practical multimodal document agents.

[74] Zero-Shot Cross-Lingual Recognition of Sign Language Handshapes cs.CL | cs.CVPDF

Marcel Granero-Moya, Carolina del Corral Farrarós, Gloria Haro, Coloma Ballester, Ricardo Marques

TL;DR: 该研究首次提出从美国手语向加泰罗尼亚手语进行零样本跨语言手形识别，以选指、屈曲、张开、拇指位置和接触五种共享音系特征解码目标手形；消除录像格式差异后，在单一LSC签署者的37类基准上达到80.0%音系特征准确率和54.5%期望手形准确率。

Details

Motivation: 手语处理技术在美国手语等高资源语言上进展较快，但世界多数手语缺少新方法所需的音系标注。手形可分解为多种跨语言共享特征，可能支持无需目标语言视频标签的跨语言迁移。

Result: 作者提出从ASL到加泰罗尼亚手语LSC的零样本跨语言手形识别框架。方法将手形拆为五个共享音系特征：选定手指、屈曲、张开、拇指位置及拇指接触，再依据预测特征通过复合音系距离度量解码LSC手形。评估三种架构MLP、SL-GCN、SHuBERT，训练数据来自PopSign和Sem-Lex两个ASL语料，目标为单一签署者、37种手形的LSC基准。统一录像格式差异后，零样本迁移达到80.0%音系特征准确率和54.5%期望手形准确率，无需目标语言视频训练标签。

Insight: 共享音系特征可作为低资源手语之间的中间表示，降低对目标语言标注视频的依赖；录像格式差异本身是迁移的重要障碍。目标基准来自单一LSC签署者，结果不能直接代表不同签署者及完整手语理解能力。

Abstract: Sign language processing advances rapidly for high-resource languages such as American Sign Language (ASL), yet most of the world’s sign languages lack the phonological annotations new methods require. We present the first zero-shot cross-lingual framework for handshape recognition, transferring from ASL to Catalan Sign Language (LSC). Our approach leverages the decomposition of handshapes into five phonological features – selected fingers, flexion, spread, thumb position, and thumb contact – shared across both languages, to decode LSC handshapes from predicted features via a composite phonological distance metric. We evaluate three architectures (MLP, SL-GCN, SHuBERT) trained on two ASL corpora (PopSign, Sem-Lex) against a 37-handshape, single-signer LSC benchmark. Zero-shot transfer proves viable once recording-format disparities are harmonized, reaching 80.0% phonological feature accuracy and 54.5% expected handshape accuracy. Phonological decomposition thus offers a bridge for extending sign language technologies to low-resource languages without any target-language video training labels.

[75] EviGen: Predictive Evidence Scaffolding for Verifiable Clinical Rationale Generation cs.CLPDF

Fengnan Li, Heman Burre, Liwen Sun, Roshni Varma, Matthew M. Engelhard

TL;DR: EviGen是三层可验证临床理由生成框架：患者条件检索器按结局预测归因排序证据，LLM基于证据支架生成理由，过程监督验证器逐步检查主张；在三个医学预测数据集上改善预测及理由忠实度，临床评审也更偏好。

Details

Motivation: 纵向电子病历包含患者多年的笔记、编码、检验和操作记录，但临床医生全面审阅不现实；LLM处理长病历成本高且容易漏掉相关观察或产生幻觉，需要一种能提供可验证证据的临床理由生成流程。

Result: EviGen包含三层。第一层是患者条件检索器，使用可学习查询寻找有助预测临床结局、而非仅文本相关的证据，并依据预测归因分数排序。第二层LLM生成器将排序证据作为支架，生成基于检索片段的临床理由。第三层过程监督验证器在推理步骤级检查生成理由并标记不可靠主张。三个医学预测数据集上的实验显示，相较完整上下文LLM及RAG基线，EviGen提高结局预测性能和理由忠实度；可用性评估中临床评审更偏好EviGen。摘要未提供数据集名称和性能数字。

Insight: 检索应优先寻找对预测结局有贡献的临床证据，而非单纯语义相似内容；再用过程级验证检查理由是否忠实于证据，可同时处理长病历遗漏和生成幻觉。摘要未报告临床部署或患者结局改进。

Abstract: Longitudinal electronic health records (EHRs) capture years of patient history across notes, codes, labs, and procedures, and contain evidence needed to reason about likely clinical outcomes. However, comprehensive clinician review of these records is impractical, and LLM-based processing is costly and often unreliable, missing some relevant observations while hallucinating others. We therefore propose EviGen, a three-layer framework for verifiable clinical rationale generation that addresses these challenges. The first layer is a patient-conditioned retriever that uses learnable queries to find evidence predictive of, not just textually relevant to, a clinical outcome and ranks it by prediction attribution scores. The second layer is an LLM generator that consumes this ranked evidence as a scaffold to produce a clinical rationale grounded in the retrieved spans. The third layer is a process-supervised verifier that checks the generated rationale at the reasoning-step level, flagging unreliable claims. Across three medical prediction datasets, EviGen improves prediction performance and rationale faithfulness over full-context LLM and RAG baselines, and is preferred by clinical reviewers in a usability evaluation.

[76] Structured Claim-Level Discourse Representations for Dense Health Narratives cs.CLPDF

Farnoushsadat Nilizadeh, Elham Pourabbas Vafa, Shirin Nilizadeh, Eduard Dragut

TL;DR: 该研究提出面向社交媒体健康视频的结构化主张级语篇表示，将原子主张与主题、立场和多维语用属性组成元组，并构建覆盖四个健康领域、60段视频、1,191条人工标注主张的基准；LLM擅长主题与立场分类，但难以进行高维语用分析。

Details

Motivation: 社交媒体健康视频中，短对话跨度内常交织多个主题、立场、证据框架和修辞功能，传统粗粒度主题、情感或立场表示无法捕捉这类密集语篇结构。分析发现平均每分钟包含13.22个原子主张，说明需要更细粒度表示。

Result: 作者提出结构化主张级健康语篇分析框架，将原子主张与主题属性、立场及多维语用语篇属性连接成元组。为支持研究，构建覆盖四个健康领域的基准，包含60段视频中的1,191条人工标注主张，并在不同语境设置下评估自动结构化语篇分析。结果显示，现有LLM在主题分类和立场预测上表现较强，但在高维语用特征分析上仍有困难。不同语篇任务受益于不同形式的上下文推理，未来系统可能需要任务分解和专用推断策略。

Insight: 健康视频语篇的语义单元密度高，单纯主题或情感标签会丢失主张之间的立场与语用关系；统一结构化表示可支持细粒度分析。LLM不同子任务能力不均，需要按任务选择上下文和专用推理。摘要未给模型具体评测分数。

Abstract: Health discourse in social media videos often contains densely entangled claims spanning multiple thematic aspects, stances, evidential frames, and rhetorical functions within short conversational spans. Existing approaches largely rely on coarse topic-level, sentiment-based, or stance-oriented representations that do not adequately capture this structure. Our analysis identifies an average of 13.22 atomic claims per minute, motivating richer claim-level discourse representations. We introduce a structured framework for claim-level discourse analysis in dense health narratives. Our framework models discourse through tuples linking atomic claims with thematic aspects, stance, and multidimensional pragmatic discourse attributes. To support this setting, we construct a benchmark spanning four health domains with 1,191 manually annotated claims from 60 videos. Using this framework, we evaluate automated structured discourse analysis under different discourse context settings. Results show that current LLMs achieve strong performance on thematic categorization and stance prediction, but struggle with high-dimensional pragmatic profiling. We also find that different discourse tasks benefit from different forms of contextual reasoning, suggesting that future systems may require task decomposition and specialized inference strategies.

[77] WordPolo: Evaluating Language Models Through Iterative Semantic Feedback cs.CL | cs.AIPDF

Tyler McDonald, Ali Emami

TL;DR: WordPolo 用带语义距离反馈的逐轮猜词任务观察模型、人类和启发式方法的搜索过程，并提出过程进展指标补充最终答对率。

Details

Motivation: 仅以基准准确率评价大语言模型和推理模型，无法判断其搜索过程是否持续有效、推理是否忠实，也会忽略尚未答对但已显著接近答案的进展。

Result: 参与者从零线索开始猜目标词，每轮收到语义距离分数（1 表示猜中，数值越大表示越远），需要据此调整搜索。作者在 1,500 个谜题上比较 GPT-4.1、Llama 4、Claude 3.5 Haiku、Qwen 3、o4-mini、DeepSeek-R1、人类和一种启发式方法；不同模型的成功率为 4%–62%。新增基于过程进展的指标后，分析发现准确率之外还可观察模型是否逐步接近答案；推理模型可能因过度或不足思考受阻，成功模型表现出类似人类的策略。

Insight: 交互式语义反馈把搜索策略变成可观察对象；同时报告过程进展和最终成功，能区分有效探索与偶然猜中，也能识别答错但持续缩小搜索范围的情况。

Abstract: Large Language Models (LLMs) and Large Reasoning Models (LRMs) are typically evaluated on challenging benchmarks through dataset accuracy alone, providing no insight into the quality or faithfulness of their reasoning processes. We present WordPolo, a word-finding task where participants must discover an unknown target word using semantic similarity feedback. Players start with zero knowledge, make guesses, and receive distance scores (1 = correct, higher = further away). Success requires interpreting scores to navigate semantic space and systematically narrow the search. This design makes iterative reasoning and adaptive search strategies both directly observable and necessary for success. We evaluate recent LLMs (GPT-4.1, Llama 4, Claude 3.5 Haiku, Qwen 3), LRMs (o4-mini, Deepseek-R1), humans, and a novel heuristic on 1,500 puzzles. Beyond solve rates (which range from 4% to 62%), we introduce progression-based metrics that reveal models often make meaningful progress, insights that accuracy alone would miss. Our analysis shows how reasoning models can be hindered by overthinking and underthinking, while successful models exhibit human-like strategies. WordPolo demonstrates the need for benchmarks that test both reasoning process and outcomes, providing holistic measurements of model capabilities. Our code and dataset can be found at https://wordpolo-demo.vercel.app/.

[78] MIRAGE: How Conversation State Shapes Historical Evidence Use in Multimodal Personal Agents cs.CL | cs.AIPDF

Yu Liu, Wenxiao Zhang, Cheng Hu, Cong Cao, Fangfang Yuan

TL;DR: MIRAGE 通过固定证据与问题、改变多模态个人代理的对话状态，系统评估其能否判断可回答性、找回正确来源并基于证据作答，揭示压缩前后不同的历史证据失效模式。

Details

Motivation: 长期个人代理需要跨对话、文件和工作区状态回忆历史证据，但即使历史访问能力退化，模型仍可能给出貌似合理的回答；仅按答案正确率评价会高估其真实证据使用能力。

Result: MIRAGE 固定证据对象、问题和评分，只改变对话状态，考查代理能否判断问题是否可答、恢复正确来源并据此回答。对七种前沿及开放权重多模态骨干的实验发现：压缩前的上下文深度与压缩后的续聊形成不同且非单调的失败状态；开放权重模型高度依赖上下文连续性，来源失效时较少主动转用工具检索；对遵循工具调用的模型，检索压力有助于深层压缩前的来源归因，却会在压缩后持续退化。

Insight: 评估历史证据使用必须显式操纵对话状态并检查来源归因和证据回答，不能由最终答案正确推断模型确实找到了历史依据。

Abstract: Multimodal large language model (MLLM) agents are increasingly used as personal assistants for long-running tasks. Their utility depends on continuity: agents must retrieve and use earlier evidence across dialogue, files, and workspace state. However, agents can generate plausible answers even when access to that history has degraded, causing outcome-only evaluation to overestimate true evidence use. We present MIRAGE (Multimodal Interaction Retrieval, Attribution, and Grounding Evaluation), a controlled study of historical evidence use under conversation-state variation in multimodal personal agents. MIRAGE holds evidence objects, questions, and scoring fixed while varying only conversation state, and evaluates whether an agent can determine answerability, recover the correct source, and answer from it. Across seven frontier and open-weight multimodal backbones, we find that: 1) pre-compaction depth and post-compaction continuation form distinct, non-monotonic failure regimes rather than a single degradation curve; 2) open-weight models rely heavily on context continuity and are reluctant to spontaneously switch to tool-mediated retrieval when provenance fails; and 3) retrieval pressure improves source attribution in deep pre-compaction states for tool-compliant models, but consistently regresses after compaction, where stored evidence has already degraded. These findings show that historical evidence use should be evaluated under state variation, rather than inferred from outcome-only correctness.

[79] ScienceIDE: Turning World’s Scientific Codebase into Agent Learnable Environments cs.CL | cs.CYPDF

Hejia Geng, Zesen Huang, Haoyang Li, Wenbin Li, Koutian Wu

TL;DR: ScienceIDE 将科学软件仓库转化为可执行、可验证的智能体学习环境，并以交互轨迹训练模型，在科学代码修复和部分通用基准上取得提升。

Details

Motivation: 科学代码承载大量方法和工具知识，但工具链零散、领域惯例隐含、正确性标准专业，使其难以转化为可靠的智能体学习经验，这构成科学经验利用的瓶颈。

Result: ScienceIDE 由专家定义科学案例及验收标准，引导智能体把代码仓库改造成可执行环境，支持任务生成、运行和科学验证，并为监督微调、强化学习和评测提供共同基础。作者用经验证的交互轨迹训练 PhAI-IDE-72B、9B 和 4B；模型在留出科学代码修复任务以及选定的代码、推理和知识通用基准上取得提升。

Insight: 将科学软件变成可运行且有明确验收标准的环境，可以把科研实践中的经验转化为可监督的智能体轨迹；论文报告了向部分通用能力迁移的证据，但具体范围依赖所选基准。

Abstract: Scientific code repositories encode decades of human knowledge in executable models, methods, and tools. Yet fragmented toolchains, implicit domain conventions, and specialized correctness criteria make this knowledge difficult to convert into reliable learning experience-a challenge we call the scientific experience bottleneck. We introduce ScienceIDE, infrastructure for turning the world’s scientific code into programmable environments for scientific agents. Guided by expert-defined scientific cases and acceptance criteria, agents transform repositories into executable environments that support task generation, execution, and scientific verification. These environments provide a shared foundation for supervised fine-tuning, reinforcement learning, and evaluation. Using verified interaction trajectories, we train PhAI-IDE-72B, PhAI-IDE-9B, and PhAI-IDE-4B. The model family shows gains in held-out scientific-code repair and across selected general-purpose benchmarks in code, reasoning, and knowledge, providing evidence of positive transfer from scientific experience to broader capabilities. ScienceIDE lays the foundation for an integrated workspace for agent learning and scientific practice, making humanity’s scientific software a shared substrate for developing scientific intelligence. Code: https://github.com/aitofound/ScienceIDE

[80] Playing log(N)-Questions over Wikipedia Abstracts: Communication Efficiency Between Paired Frontier Models cs.CLPDF

Peter Potash

TL;DR: 论文用两模型互通的 log(N)-Questions 游戏评测六种前沿模型在受限是非问答下的通信效率，发现成功率随候选规模增长而下降，错误大致来自回答和区分两环节。

Details

Motivation: 模型在单轮任务中的准确率不能充分揭示它们如何在信息不对称条件下，通过有限轮次提问和极短回答互相传递信息；作者希望直接测量这种模型自我通信的效率和失效来源。

Result: 提问者看到 N 篇维基百科导言，需用恰好 log₂N 个是非问题找出秘密目标；回答者只看到目标和问题，只能回一个词。六种模型共进行 408 场、候选集大小 4–1024 的游戏，API 总成本 363 美元。较弱模型 Claude Opus 5 赢 28/68 场，其他五种为 45–56 场；领先五种模型的合并胜率随集合大小下降，拟合为 p^(log₂N)，每轮可靠率 p=0.928。失败约各半来自回答错误和区分失败；模型很少指认自身证据排除的文档。标题划分策略可提高每问信息量，但只见于两个模型。

Insight: 有限轮次下整体胜率近似由单轮通信可靠性指数式累积决定；评估应分解回答可靠性与问题区分能力。推理 token 开销与成功关联较弱，说明消耗更多推理文本不必然改善通信。

Abstract: We evaluate six frontier language models on the two-agent $\log(N)$-Questions game. A questioner sees $N$ Wikipedia lead paragraphs and must identify a secretly chosen target using exactly $\log_2 N$ yes/no questions. An answerer sees only the target and the question, and replies with one word. Both roles run on the same provider, so the game measures how well a model communicates with itself across an information asymmetry. We run 408 games over document sets of 4 to 1024 paragraphs at a total API cost of $363. One model finishes well behind the others: Claude Opus 5 wins 28 of 68 games, against 45 to 56 for GLM-5.3, GPT-5.6 Sol, Grok 4.6, Gemini 3.8 Flash and Kimi K3. The leading five are only marginally separable. Pooling those five, win rate declines with set size at $r=-0.973$ and is fit by a single per-round reliability parameter. The form is $\text{win}=p^{\log_2 N}$ with $p=0.928$. Losses divide into answer errors and discrimination failures in roughly equal measure, and models almost never name a document their own evidence excludes. Every unanimous answer error from the weakest model was inspected: 32 of 34 are No'' answers, on properties stated in the document's first sentence, under an instruction that explicitly warns against defaulting to No’’. Information per question, estimated from answer balance, correlates with win rate at $r=+0.88$. The only two models to extract a full bit per question are the only two that partition on document titles, a strategy absent below $N{=}32$ and used in a quarter of questions above it. Reasoning-token expenditure varies $4.5\times$ across models with little relation to success, and the trace grows as the candidate set shrinks without a matching gain in reliability.

cs.AI [Back]

[81] NeMo Data Designer: An Extensible Framework for Multimodal Synthetic Data Generation cs.AI | cs.CLPDF

Johnny Greco, Nabin Mulepati, Andre Manoel, Eric Tramel, Kirit Thadaka

TL;DR: NeMo Data Designer 是用于多模态合成数据生成的开源框架，通过声明式列配置、可扩展插件和预览修订循环支持可检查、可复现的数据构建。

Details

Motivation: 合成数据生成通常需要反复调整数据结构和模型生成过程，涉及文本、代码、结构化输出、图像等多种列类型。用户需要能表达多样性要求、检查配置并小规模试生成的统一工作流。

Result: NDD 允许人工或智能体用声明式配置定义数据集各列，支持文本、代码、结构化输出、图像、嵌入和统计采样器；插件系统可扩展类型与功能。配置本身是可检查、可共享的产物，核心工作流内置小批量预览、检查、修订后再全量生成。运行时负责依赖解析、调度用户提供的模型端点和失败请求重试。论文描述架构与编程模型，并展示结构化、智能体、多模态及领域任务案例，包括 Nemotron 开发和企业部署。

Insight: 把生成配置作为可审阅的数据集规范，并把预览迭代纳入核心流程，有利于控制合成数据的可复现性和多样性；论文主要介绍框架与案例，并未给出统一的性能比较。

Abstract: We present NeMo Data Designer (NDD), an open-source, general-purpose framework for multi-modal synthetic data generation (SDG). Designed to be intuitive to use, NDD provides a declarative configuration format in which human and/or agent users define each dataset column, with column types spanning text, code, structured outputs, images, embeddings, and statistical samplers that are explicitly configured to steer dataset diversity. Additional column types and functionality can be introduced using the framework’s flexible plugin system. NDD’s configuration is an inspectable artifact, supporting workflow sharing and reproducibility. SDG is an inherently iterative process. NDD therefore builds a preview-and-revision loop into its core workflow, allowing users to generate and inspect a small number of records, refine the specification, and rerun generation at full scale. At runtime, NDD resolves dependencies, schedules calls to user-provided model endpoints, and retries failed requests. We describe NDD’s architecture and programming model and present case studies spanning structured, agentic, multimodal, and domain-specialized tasks, including datasets used in Nemotron model development and in production enterprise deployments.

[82] CapMem: A Benchmark for Caption-Based Episodic Memory in Egocentric Video cs.AI | cs.CVPDF

Dingli Liang, Yiqiao Xie, Yukai Huang, Zhaokai Wang, Weitong Cai

TL;DR: CapMem 构建面向第一人称长视频的字幕式情景记忆问答基准，发现完整覆盖视频的字幕记忆在多数长视频模型上优于直接视频问答，检索验证还可进一步提升准确率。

Details

Motivation: 可穿戴助理需要从第一人称视频中形成情景记忆，但视觉语言模型受帧数预算、视觉 token 成本增长和长上下文检索失效限制。作者考察文本字幕能否成为可复用的情景记忆。

Result: 论文定义情景记忆视频字幕问答任务，并发布人工标注 CapMem：75 段视频共 33.7 小时、16 种场景的 1,000 道选择题。对超过 20 分钟的视频，采用 30 秒和 60 秒字幕窗口的完整覆盖 CaptionQA 分别在 12 个模型中的 10 个和 8 个上超过直接 VideoQA；六种 Qwen 模型的匹配帧对照仍保留 3.22 和 2.55 个百分点平均优势。字幕引导的检索-验证流程又最多提高 5.3 个百分点。

Insight: 以字幕作为紧凑的时间记忆，可以降低长第一人称视频直接视觉检索的负担；匹配帧对照显示优势不全是覆盖更多帧带来的，显式检索验证还可带来额外收益。

Abstract: Wearable assistants require episodic memory over egocentric video, yet current vision-language models face bounded frame budgets, growing visual-token costs, and long-context retrieval failures. Under these practical constraints, we study whether textual captions can serve as reusable episodic memory. We define the Episodic Memory Video Caption QA task and introduce CapMem, a human-annotated benchmark with 75 videos totaling 33.7 hours, and 1,000 multiple-choice questions across 16 scenarios. On long videos (>20 min), full-coverage CaptionQA with 30s and 60s caption windows outperforms direct VideoQA for 10/12 and 8/12 models, respectively. On the same video subset, a matched-frame control across six Qwen models retains mean accuracy gains of 3.22 and 2.55 points, respectively. Our caption-guided retrieve-and-verify harness further improves accuracy by up to 5.3 points. These results support the effectiveness of caption memory for episodic reasoning over long egocentric video.

[83] Disentangling Long-Term Memory via Latent Neuro-Symbolic Reasoning cs.AI | cs.CL | cs.LGPDF

Cai Ke, Xinghao Chen, Xiaoyu Shen, Keyu Chen, Siyu An

TL;DR: LGM 在连续潜在空间将长期交互记忆拆解为稀疏概念，并按当前查询动态构造关系图，引导图消息传递以改善显式和隐式偏好建模。

Details

Motivation: 扁平检索会独立打分记忆片段，忽略分散在多次互动中的信息；静态查询无关图又无法表达上下文依赖关系。原始文本记忆还相互纠缠且带噪，使细粒度个性化和跨会话推理代价高。

Result: LGM 使用稀疏自编码器构造潜在图：针对每个查询，将历史互动映射为潜在记忆节点和稀疏概念激活，并动态生成查询相关的关系边权，而不是持久化固定图。随后图编码器将查询嵌入作为条件偏好，在任务相关潜在子图上传播非线性消息，形成记忆表示。长期个性化基准实验中，LGM 显著超过先进基线，能更好捕捉显式和隐式偏好并生成个性化回应。

Insight: 把记忆结构放在可查询的潜在空间中动态构造，可同时缓解原始文本噪声和固定图的查询无关问题，使关系推理随当前个性化需求改变。

Abstract: Personalized agents are required to reason over long-term history interactions to infer both explicit preferences and implicit behavioral evidence. While early flat retrieval methods score memory fragments independently and neglect the distributed information, current structured memory frameworks rely on query-agnostic static graphs that fail to capture the context-dependent relations. Crucially, raw textual memories are inherently entangled and noisy, making fine-grained personalization and cross-session reasoning computationally prohibitive. To this end, we present LGM, a novel neuro-symbolic framework that shifts long-term memory disentanglement into a continuous latent space. Specifically, (i) instead of persisting fixed graphs, we design a tailored latent graph construction with a sparse autoencoder. Subject to each query, it maps historical interactions into latent memory nodes and disentangles the memory traces into sparse concept activations, dynamically synthesizing query-aware relational edge weights. (ii) A graph encoder then treats the query embedding as a conditioning preference to direct non-linear message passing across the task-specific latent subgraph. This yields a highly expressive memory representation for effective activations. Extensive experiments on long-term personalization benchmarks demonstrate that LGM significantly outperforms state-of-the-art baselines in capturing both explicit and implicit preferences while enabling personalized responses.

Luyao Zhu, Xun Wei Yee, Wei Li, Mun Thye Mak, Wee Siong Ng

TL;DR: MUSE 是面向情境化教育的艺术图像视觉语言模型基准，覆盖十二类感知、语义、情感、文化与组合推理任务，并强调新加坡及东南亚多元文化语境。

Details

Motivation: 视觉语言模型的多模态能力进展迅速，但其在教育场景中的评测不足。语言学习辅助不仅要识别艺术图像，还需理解语义、情感、文化内容并结合视觉上下文互动；现有基准对艺术教育材料覆盖有限。

Result: MUSE 将图像标注与问题生成解耦，便于以可控难度构造多样任务并降低标注成本。基准含十二项任务，涵盖视觉感知、语义和情感解释、文化理解及组合推理；图像在新加坡和东南亚多元文化语境中精心选取，同时包含西方艺术、多主题和多难度。对开源与专有模型的评测发现不同能力维度差异明显，尤其情感解释与组合推理较弱，并总结教育应用中的常见失败和可信多模态模型挑战。

Insight: 教育场景中的艺术理解不能由通用物体识别或单一文化语料代表；将文化与情境因素纳入基准，有助于检验模型是否能支持真正有意义的学习互动。

Abstract: Large vision-language models have achieved remarkable progress in multi-modal understanding, yet their capabilities in educational settings remain insufficiently evaluated. In AI-assisted language learning, models must interpret artistic imagery, understand its semantic, affective, and cultural content, and reason about visual context to support meaningful interaction. However, existing benchmarks primarily focus on real-world images or domain-specific educational reasoning, providing limited coverage of artistic educational content. To address this gap, we introduce MUSE, a benchmark for evaluating large vision-language models on artistic image understanding in situated educational applications. MUSE decouples image annotation from question generation, enabling diverse tasks with controllable difficulty while reducing annotation effort. It comprises twelve tasks spanning visual perception, semantic and affective interpretation, culture understanding, and compositional reasoning, together with diverse artistic images deliberately curated to center Singaporean and Southeast Asian multicultural contexts alongside Western art traditions, covering multiple themes and difficulty levels. Evaluation of open-source and proprietary models reveals substantial disparities across capability dimensions, particularly in affective interpretation and compositional reasoning. Our analysis further identifies common failure modes and key challenges for developing trustworthy multi-modal models for education. We hope MUSE will serve as a standardized benchmark for advancing multi-modal understanding in situated educational applications.

cs.LG [Back]

[85] The Missing “I Don’t Know”: Why Three Reasoning-Reliability Findings Converge on Calibrated Abstention cs.LG | cs.AI | cs.CLPDF

Srijith Ravikumar

TL;DR: 论文将三类推理可靠性问题归结为缺少经过校准的拒答能力，并指出主流基准不给拒答正向奖励，提出四项评测改革建议。

Details

Motivation: 工具可靠性表征退化、安全约束下的模型改写或截断行为，以及一致推理系统可能反复幻觉的理论结果，看似不同，却都涉及模型在无法可靠作答时缺少恰当的“不知道”机制。

Result: 作者把上述三项研究分别对应到能力缺口、策略缺口和递归理论缺口，并主张校准式拒答是共同的待补能力。部署模型中的诚实性后训练缩小了差距，但论文所讨论的理论问题需要有校准的拒答函数。当前主流基准在拒答时给零奖励，无法形成鼓励该能力的排行榜梯度。作者提出四项评测改革：三重计分、报告拒答率、按能力分层评测，以及强制报告校准指标。

Insight: 当基准只奖励答对而不区分有依据的作答与适当拒答时，模型缺乏学会校准弃答的激励；评测改革是必要条件，但作者指出并不足以单独解决理论上的可靠性缺口。

Abstract: Three recent results describe what look like unrelated LLM reliability problems. Yin et al. (2026) show reasoning RL collapses tool-reliability representations. Suleymanov et al. (2026) show that under safety-constrained generation, large models rewrite flagged spans while small models truncate. Bastounis et al. (2024) prove any consistent-reasoning system without an implicit “I don’t know” function must hallucinate infinitely often on broad problem classes. We argue these findings converge on a single intervention: calibrated abstention is what each independently identifies as the missing capability, even though the unavailability they document, a capability gap, a policy gap, and a recursion-theoretic gap, has a different source in each case. Honesty post-training has narrowed the gap in deployed models, but principled closure of the class Bastounis identifies requires a calibrated abstention function whose training signal at the leaderboard level is absent: dominant benchmarks assign zero reward to decline, so the leaderboard gradient that would select for the function does not exist. We propose four changes to evaluation: triple-scoring, abstention-rate reporting, capability-stratified evaluation, and mandatory calibration metrics. Benchmark reform is necessary, not sufficient, for closing the gap the theorem identifies.

[86] Long-Context Demonstration Selection Using State Space Models cs.LG | cs.CLPDF

Ziniu Zhang, Zhenshuo Zhang, Ruoxuan Xiong, Gene Cooperman, Hongyang R. Zhang

TL;DR: 该方法将蒸馏得到的状态空间模型用于长上下文示例选择，以线性时间近似 Transformer 层组输出，在多个分类和推理任务上降低计算量并提升准确率。

Details

Motivation: 为语言模型查询选择并前置演示样例与上下文学习密切相关，但 Transformer 推理成本随序列长度呈二次增长，使长上下文中的示例筛选更困难。

Result: 方法分两步：把训练好的 Transformer 层划分为连续分组，为每组蒸馏一个状态空间模型以近似相邻层的输入输出；再将蒸馏模型输出映射为少量 token，并用其嵌入完成下游演示样例选择。合成和真实数据实验中，蒸馏 SSM 相对真实输出的近似误差低于 0.7%；多个文本分类和推理任务上，相对基线示例选择方法 FLOPs 减少 14.2 倍，准确率提高 6.48%。

Insight: 通过蒸馏低成本的序列模型近似 Transformer 层组，可在长上下文示例选择阶段减少昂贵计算；收益取决于蒸馏近似误差能否在下游排序中保持足够小。

Abstract: We study the problem of demonstration selection, which involves selecting a subset of examples for prepending to a query to a language model. This problem is closely related to in-context learning and language model inference. Since the inference cost of a transformer model scales quadratically with sequence length, the selection problem becomes especially challenging in a long-context scenario. In this paper, we tackle this problem by building on state space models (SSMs), which require only linear inference time given the input. Our approach involves two algorithms. The first learns a small set of SSMs through distillation of a (trained) transformer model. We partition all the layers into consecutive groups. Then for each group, we estimate a separate state space model to replicate the input-output behavior within the adjacent layers. Second, we map the distilled model outputs to a small set of tokens, and apply these embeddings for demonstration selection in downstream applications. We perform extensive experiments in both synthetic and real-world datasets to validate our approach. We demonstrate that the distilled SSMs only incur an approximation error of less than $0.7%$ relative to the true output. In downstream evaluation, we show that on several text classification and reasoning tasks, our approach reduces FLOPs by $14.2\times$ and improves accuracy by $6.48%$ relative to baseline demonstration selection methods.

[87] ASPIRE: Asynchronous Batched Self-Speculative Decoding for Long-Context LLM Inference cs.LG | cs.CL | cs.DCPDF

Amir Ziashahabi, Hossein Entezari Zarch, Lei Gao, Murali Annavaram, Salman Avestimehr

TL;DR: ASPIRE 允许批处理中的请求异步选择草拟和验证时机，并在草拟期间定期刷新注意力上下文，在多模型推理与长上下文基准上提升解码吞吐。

Details

Motivation: 长上下文大语言模型推理受注意力 KV 缓存反复读取限制。自推测解码用稀疏注意力草拟、全注意力验证来缓解内存瓶颈，但现有批处理方法让所有请求共用同步计划，无法适应不同请求及动态变化的最佳草拟长度。

Result: ASPIRE 包含三项设计：统一混合前向允许草拟与验证请求在同一批次共存；在线调度器基于每请求接受率估计和批次成本模型独立决定验证时机；草拟内刷新层则在指定层执行全注意力，每步更新稀疏上下文以减少陈旧信息。三个模型、五个推理和长上下文基准上，相对自回归基线解码吞吐提升 1.70–4.58 倍，平均加速比比最强已有自推测解码方法约高 27%。

Insight: 批内请求的接受率和草拟效率并不一致，按请求异步调度可避免全批等待最慢或不合适的草拟计划；草拟阶段刷新上下文则降低稀疏注意力逐步累积的偏差。

Abstract: Long-context LLM inference is bottlenecked by attention, whose repeated KV-cache reads make decoding memory-bound. Self-speculative decoding alleviates this by drafting tokens with sparse attention and verifying them with full attention, but existing batched methods remain synchronized: all requests in a batch share a single draft-verify schedule, even though the optimal draft length varies widely across requests and changes dynamically within each request. We propose ASPIRE, a non-synchronized batched self-speculative decoding framework built on three components. First, a unified mixed forward allows drafting and verifying requests to coexist in the same batched forward pass, removing the need for global draft-verify phases. Second, a lightweight online speculation scheduler uses per-request acceptance-rate estimates and a batch-aware cost model to let each request independently choose when to verify. Third, an intra-draft refresh layer performs full attention at a single designated layer during drafting, updating the sparse context at every draft step to reduce staleness during drafting. Across three models and five reasoning and long-context benchmarks, ASPIRE achieves $1.70$-$4.58\times$ speedup in decoding throughput over autoregressive baselines and improves average speedup by approximately $27%$ over the strongest prior self-speculative baselines.

[88] One Axis, No Brake: Self-Knowledge Limits the Filtering of Harmful Peer Conformity in LLMs cs.LG | cs.CL | cs.MAPDF

Yibo Hu

TL;DR: 论文分析多智能体辩论中的同伴从众问题，指出过滤有害修正等同于判断原答案是否正确，受模型自知准确性上限约束。

Details

Motivation: 多智能体系统常被认为可互相纠错，但同伴影响也可能把原本正确答案改错。理想的“刹车”应保留有益修改并阻止有害修改，而有害与有益的判定依赖模型是否知道自己的原答案正确。

Result: 作者将过滤有害修正的问题归结为模型自知能力：任何部署时可用的刹车信号，本质上都是正确性探针。六个模型家族的自知 AUROC 约为 0.64–0.89，构成难以突破的上限。即便白盒引导模型自身的正确性方向，也只改变修改频率，有害和有益修改仍同步变化。在群体层面，当多数智能体起初答错时，辩论会把共同错误放大成自信错误共识；增加智能体、模型多样性或更强成员均未解决问题。改善方式是修改前增加信息，而非修改后过滤。

Insight: 局部一致不等于全局正确；辩论中的错误过滤受制于模型对自身正确性的判断能力。若自知不可靠，增加事后筛选难以只保留有益修正，应优先提供新的证据或信息。

Abstract: Multi-agent LLM systems are expected to be more reliable because agents can catch each other’s mistakes. But peer pressure cuts both ways: the same correction that fixes a wrong answer can overturn a right one. The tempting safeguard is a brake that keeps the beneficial revisions and blocks the harmful ones. We show this brake is hard to build, for a simple reason: a revision is harmful exactly when the original answer was right, so deciding whether to block it is the same as knowing whether the model was already correct. This turns the open-ended hunt for a brake into one measurable quantity, the model’s self-knowledge: any brake built from a deploy-time signal is a correctness probe in disguise, and self-knowledge is far from perfect (AUROC $\approx 0.64$–$0.89$ across six model families). We call this ceiling the wall. Even white-box steering of the model’s own correctness direction does not breach it: it changes how often the model revises, but harmful and beneficial revisions move together. At population scale the wall becomes the cliff: when most agents start wrong, debate amplifies the shared mistake into a confident, wrong consensus. In our multiple-choice societies, more agents, more model diversity, and a stronger member do not fix it. What helps is adding information before the revision, not filtering after it. Local agreement is not global correctness.

cs.MM [Back]

[89] Divide and Conquer: Mixture-of-Bottleneck Experts in Informative Ordinal Space for Video-based Multimodal Sentiment Analysis cs.MM | cs.CLPDF

Ronghao Lin, Qiaolin He, Zefeng Lu, Yichu Liu, Li Huang

TL;DR: MoB 将视频多模态情感分析拆成极性识别与强度预测，并为文本、音频和图像序列配置任务专属信息瓶颈专家，再经路由融合预测序数情感。

Details

Motivation: 视频情感分析需整合说话视频中的文本、音频和图像序列，但现有方法常忽略情感标签的序数属性，且融合策略难兼顾不同模态独有和协同的信号。

Result: 作者将情感预测重构为序数回归，并分为极性识别和强度预测。受信息论启发，混合瓶颈框架为不同模态的极性与强度任务分配不同潜变量专家，通过信息瓶颈提取紧凑且与任务相关的表征、过滤冗余噪声；随后用多模态瓶颈路由融合及困难样本挖掘，在序数情感空间预测。在四个情感数据集和四种语言模型上的实验显示，MoB 能利用多模态信息并捕获一般情感结构，表现优于基线，并支持对细粒度模态内、模态间动态及情感线索进行更可信的定位。

Insight: 先把情感极性和强度分解，再分别压缩各模态的任务相关信息，有助于避免统一融合目标抹去序数情感中的细粒度差异。

Abstract: Video-based Multimodal sentiment analysis (MSA) must handle information from text, audio, and image sequence in human speaking videos, yet current methods often fail to integrate modalities with task awareness. Most models treat video sentiment prediction as a single task, overlooking its ordinal nature, and their fusion strategies struggle to capture diverse unique and synergic cues across modalities. To address these limitations, we adopt a divide-and-conquer perspective by reformulating MSA as an ordinal regression problem and decoupling it into polarity recognition and intensity prediction. Driven by information theory, we introduce a Mixture-of-Bottleneck (MoB) framework that assigns different latents to polarity- and intensity-specific experts for different modalities. With the learning of information bottleneck, each expert learns compact and task-relevant representations while filtering out redundancy and noise. A multimodal bottleneck routing fusion module then fuses these expert latents with hard mining strategy, guiding the prediction in the ordinal sentiment space. Extensive experiments on 4 MSA datasets and 4 language models show that MoB effectively leverages informative latents from diverse modalities and captures general sentiment structure. Beyond stronger performance, MoB comprehensively captures fine-grained intra- and inter-modal dynamics, enabling more trustworthy localization of nuanced video sentiment signals.

cs.RO [Back]

[90] A Comprehensive Review of Generative Physical Artificial Intelligence cs.RO | cs.AI | cs.CL | cs.CV | cs.LGPDF

Satyam Gaba, Krutiksinh Rana, Siva Sai, Vinay Chamola, Dusit Niyato

TL;DR: 这篇综述将生成式物理人工智能划分为机器人基础模型、视觉语言动作模型、大型行为模型、扩散策略模型和世界基础模型，并分析其协作关系、应用和研究方向。

Details

Motivation: 大规模基础模型与物理实体结合推动了能感知、推理并在现实环境行动的生成式具身系统，但不同架构的职责、相互补充方式和现实应用限制需要系统梳理。

Result: 综述提出五类生成式物理 AI：支持跨平台技能迁移的机器人基础模型、端到端多模态感知控制的 VLA、生成类人动作的大型行为模型、产生时间连贯动作的扩散策略模型，以及支持物理一致仿真和数据生成的世界基础模型。文中分析它们的协作：世界模型为 VLA 和扩散策略生成训练数据，机器人基础模型促进策略跨平台部署，大型行为模型提供自然动作先验；并综述自动驾驶、工业自动化、医疗机器人和人形机器人应用，指出数据高效学习、仿真到真实迁移、边缘部署与安全框架等方向。

Insight: 具身系统中的这些模型类别形成数据、策略迁移和动作先验的互补链条，而非孤立替代关系；综述也强调要面向连接传感器、执行器和边缘设备的物联网环境考虑部署。

Abstract: The integration of large-scale foundation models with physical embodiments has led to significant advancements in robotics known as Generative Physical Artificial Intelligence (GPAI). These agentic AI systems autonomously perceive, reason, and act in complex real-world situations. This survey comprehensively analyzes GPAI systems, focusing on their architectural foundations, current applications, and key limitations. We introduce a taxonomy of five distinct approaches: Robot Foundation Models (RFMs) for cross-platform skill transfer; Vision-Language Action (VLA) models for end-to-end multi-modal perception and control; Large Behavior Models (LBMs) for human-like movement generation; Diffusion Policy Models (DPMs) for diffusion model-based temporally coherent action generation; and World Foundation Models (WFMs) for physics-compliant simulation and data generation. We examine how these approaches complement each other: WFMs generate training data for VLAs and DPMs, RFMs enable cross-platform deployment of learned policies, while LBMs provide motion priors for natural behavior. Through examples across autonomous vehicles, industrial automation, healthcare robotics, and humanoid systems, we identify significant performance improvements and summarize promising research directions in data-efficient learning, sim-to-real transfer, edge-compatible architectures, and safety frameworks. These insights advance embodied AI for IoT-connected environments where intelligent agents interact with networked sensors, actuators, and edge devices.

[91] ${M}^2$Tok: Multi-head Multi-codebook Discrete Action Tokenization for Vision-Language-Action Models cs.RO | cs.AI | cs.CL | cs.CVPDF

Chunpu Xu, Zhixuan Liang, Yuhao Zhang, Chi-Min Chan, Jessie Wang

TL;DR: M²Tok 将动作潜变量拆成多个动作头，并为每头配置独立码本，以提升离散动作重建精度，进而改善 VLA 策略在模拟和真实任务中的成功率。

Details

Motivation: 自回归语言模型处理连续机器人动作前，需要将高维动作压缩成离散 token。现有离散化方法重建损失较高，难保留精细控制所需的运动细节，形成限制下游 VLA 能力的离散化瓶颈。

Result: 多头多码本动作 tokenizer 将潜在动作特征拆分为多个头，使不同头可隐式对齐不同动作维度，并为每个头单独分配码本量化。多个码本的组合提高表征能力，降低动作重建误差。基于 M²Tok 的 VLA 在 RoboTwin、Simpler-Env 和三个零样本真实世界任务上评测；作者报告其重建保真度更高，VLA 成功率显著提高，消融实验验证多头与多码本设计均有贡献。

Insight: 提高动作 token 表示能力可以直接改善策略可用的控制分辨率；按动作维度形成多个头，再组合独立码本，比用单一离散词表更能表达复杂动作。

Abstract: Recent advancements have successfully adapted autoregressive language models to process multimodal signals, such as images and actions. Since raw action signals are continuous, effective tokenization is essential to map high-dimensional inputs into compact discrete tokens for autoregressive processing. However, existing discrete action tokenizers often suffer from high reconstruction loss, failing to preserve the fine-grained dynamics required for precise control. This ``discretization bottleneck’’ significantly limits the performance ceiling of downstream Vision-Language-Action (VLA) models. To address this, we propose $\mathcal{M}^2$Tok, a Multi-head Multi-codebook Action Tokenizer designed to minimize reconstruction error and enhance policy performance. Our approach introduces two key structural innovations: (1) we decompose the latent action features into multiple heads, enabling the model to implicitly align specific heads with distinct action dimensions; (2) we assign independent codebooks to each head for quantization. By leveraging the combinatorial nature of multiple codebooks, we significantly expand the representational expressivity of the tokenizer, leading to substantially lower reconstruction loss compared to previous methods. We evaluate the $\mathcal{M}^2$Tok-based VLA on the RoboTwin, Simpler-Env, and 3 zero-shot real-world tasks. Experimental results demonstrate our method not only achieves superior reconstruction fidelity but also significantly boosts the success rate of VLA models. Comprehensive ablation studies further confirm the effectiveness of the multi-head and multi-codebook mechanisms. Code is available at \href{https://github.com/cpaaax/M2Tok}{https://github.com/cpaaax/M2Tok}.

[92] Not All Layers Need Tuning: Diagnosing and Directing Adaptation in Vision-Language-Action Models cs.RO | cs.CV | cs.LGPDF

Shahram Najam Syed, Arthur Jakobsson, Prayuj Sachdev, Jeffrey Ichnowski

TL;DR: 该工作发现 VLA 在不同部署变化下的适配成本集中于不同网络区域，并用少量无标签观测诊断后按预算分配不同秩 LoRA，取得高效适配效果。

Details

Motivation: 新环境微调视觉语言动作模型开销高，而常见方法假设每个网络区域都需要同等调整，给所有区域配置统一容量适配器。实际的外观、指令和新物体变化可能需要改动不同模块。

Result: 作者在五种架构、规模 93M–7B 的 VLA 上以区域隔离微调位移测量适配成本：外观变化主要影响视觉编码器，指令变化主要影响语言骨干，新物体变化主要影响视觉编码器和动作头。提出“观察、诊断、分配、适配”流程：从十个无标签目标观测出发，结合无参考梯度、Monte Carlo Dropout 和相对缓存源表征的 CKA 估计区域成本；按参数预算配置可变秩 LoRA 并冻结校准良好的区域。诊断区域排序的中位 Spearman 为 0.91，在 LIBERO 和 CALVIN 各预算下不低于统一 LoRA。真实 xArm-7 指令措辞变化时，使用的可训练参数仅为全量微调的 0.04% 仍能匹配其表现；五个未见场景上也优于各基线。

Insight: VLA 适配需求具有可测的结构：不同类型的分布变化集中作用于不同模块，因此先诊断再分配适配容量，能避免在无需改变的区域浪费参数。

Abstract: Fine-tuning a Vision-Language-Action (VLA) model for a new deployment environment is expensive, yet most methods apply uniform-capacity adapters to every network region as if every region requires equal adjustment. This paper tests that assumption on five architecturally diverse VLAs (OpenVLA-OFT, $π_0$, SmolVLA, DTP, Octo; 93M-7B parameters). Measuring per-region adaptation cost as normalized parameter displacement under region-isolated fine-tuning reveals an adaptation spectrum in which appearance shifts concentrate cost in the vision encoder, instruction shifts in the language backbone, and novel-object shifts in the vision encoder together with the action head, across all five architectures. To exploit this structure, we introduce a pipeline that observes, diagnoses, allocates, and adapts. From ten unlabeled target observations and without fine-tuning, the diagnostic estimates per-region cost by combining reference-free gradient and Monte Carlo Dropout signals with a Centered Kernel Alignment score against a cached source reference; the allocator converts the estimates into variable-rank LoRA adapters under a parameter budget and freezes well-calibrated regions; and standard LoRA fine-tuning trains the resulting adapters. The diagnostic ranks regions within each deployment at a median Spearman of 0.91, and the allocation matches or exceeds uniform LoRA at every budget we tested on LIBERO and CALVIN. On a physical xArm-7, the pipeline matches full fine-tuning under an instruction-wording shift with 0.04% of its trainable parameters, and on five held-out scenes evaluated without retraining it leads every baseline, with 11-23 successes of 30 rollouts against 8-18 for the strongest parameter-efficient baseline at equal or larger budgets and 2-11 for full fine-tuning. These results suggest that adaptation cost in VLAs is structured enough to measure before fine-tuning begins.

[93] ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models cs.RO | cs.AI | cs.CL | cs.CVPDF

Shijie Lian, Bin Yu, Zhaolong Shen, Xiaopeng Lin, Yichao Du

TL;DR: ActionPiece 用物理距离排序一致性约束动作编码和量化，补充逐点重建误差评估，使自回归 VLA 的离散动作 token 更好保留示范间动作调整关系。

Details

Motivation: 动作 tokenizer 常以均方误差等逐点指标评估，但小的单点误差并不能说明压缩后是否保留了不同示范动作间的相对调整关系；动作可能被压到代表性轨迹附近，导致情境所需差异被削弱甚至反转。

Result: 作者提出物理秩一致性 PRC，衡量重建动作是否保持局部物理距离排序；它可在不同词表与解码器间统一评估关系保真度。ActionPiece 联合监督表征学习和量化：对编码器及量化特征的近远排序施加物理秩约束，对码字分配分布施加相同排序约束，并与重建目标结合。冻结解码器和相同 Qwen3-VL-4B 策略训练设置下，LIBERO 达 94.8%，未见 LIBERO-Plus 达 68.8%，SimplerEnv 达 71.9%，VLA-Arena L0–L2 达 51.5%。消融显示两项排序目标共同提升 PRC 和策略成功率。

Insight: 动作表示不仅要还原每个样本，还要保留动作间的物理关系；这种关系监督提供了比逐点误差更贴近控制目标的 tokenizer 训练信号。

Abstract: Action tokenizers play a central role in autoregressive vision-language-action (VLA) models, determining both the targets for policy training and the executable commands recovered from predicted tokens. Their fidelity is commonly evaluated using pointwise reconstruction metrics such as mean squared error (MSE), yet small individual errors do not fully characterize how faithfully action adjustments across demonstrations are preserved. After compression, similar actions may still cluster around a representative motion, while the adjustments needed for different contexts are diminished, distorted, or even reversed. We introduce physical rank consistency (PRC) to measure how well tokenization preserves local physical distance rankings after reconstruction. Evaluating decoded actions provides a common reference across token vocabularies and decoder architectures, complementing pointwise accuracy with a measure of relational fidelity. We further present ActionPiece, which preserves physical action relationships through joint supervision of representation learning and quantization. Physical rank preservation supervises near-far ordering in encoder and quantized feature distances, while quantization regularization applies the same ordering to codeword assignment distributions. Both objectives augment reconstruction, producing discrete action tokens for standard autoregressive policy learning and execution through a frozen decoder. Under the same Qwen3-VL-4B policy training setup, ActionPiece achieves 94.8% on LIBERO and 68.8% on unseen LIBERO-Plus, with additional evaluations reaching 71.9% on SimplerEnv and 51.5% across VLA-Arena L0-L2. Component ablations show that the two objectives jointly improve PRC and policy success, demonstrating the value of physical relationship supervision for action tokenization.

cs.SD [Back]

[94] FRAUDSkill: Structured Frozen-Weight Skill Optimization for Audio Anti-Fraud Detection cs.SD | cs.CLPDF

Chengxian Hu, Zhiming Ma, Mingjun Pan, Yifan Wang, Shun Zhang

TL;DR: FRAUDSkill 不改动音频语言模型权重，而在外部优化技能程序、路由策略和判定规则，并用验证引导的多路径推理提升反诈分类准确率与格式合规性。

Details

Motivation: 音频语言模型可直接听取语音并推理诈骗证据，但部署输出须遵守预设标签和服务场景识别、诈骗判定、条件式诈骗类型分类的结构化协议。把任务规则写进模型参数或手工提示，难随诈骗方式和标签政策变化维护。

Result: FRAUDSkill 冻结底层音频语言模型，只优化外部技能程序、路由策略和决策规则，并结合结构化输出控制及验证引导的多路径推理确保协议合规。在 TeleAntiFraud 基准上宏 F1 达 73.50%，比共享冻结模型基线高 31.96%，无效输出降至 1.94%。实验支持外部技能优化可在不改动底层模型的情况下，为结构化音频反诈检测提供有效且易适配的方案。

Insight: 把快速变化的流程与判定政策放在可独立调整的外部技能层，可以保留基础模型权重不变，同时让多阶段输出符合明确的决策协议。

Abstract: Large audio-language models have shown promise for anti-fraud detection by directly processing speech and reasoning over fraud-related evidence. Their deployment, however, requires predictions to follow a predefined label space and a structured decision protocol consisting of service-scenario identification, fraud detection, and conditional fraud-type classification. Existing fine-tuning and prompt-based approaches typically encode task knowledge, constraints, and decision rules into model parameters or manually maintained prompts, making them difficult to adapt as fraud patterns and labeling policies evolve. To this end, we propose FRAUDSkill, a structured frozen-weight adaptation framework that leaves the underlying audio-language model unchanged while optimizing an external layer of skill programs, route-specific policies, and decision rules. We further combine structured output control with validation-guided multi-path inference to ensure protocol-compliant predictions. On the TeleAntiFraud benchmark, FRAUDSkill achieves 73.50% Macro-F1, outperforming the shared frozen-model baseline by 31.96% while reducing invalid outputs to 1.94%. Extensive experiments demonstrate that external skill optimization provides an effective and adaptable solution for structured audio anti-fraud detection without modifying the underlying model. The source code is available at https://anonymous.4open.science/r/FRAUDSKILL-114514.

cs.SE [Back]

[95] Code Consistency Preference Optimization Verification for Language Model Alignment cs.SE | cs.CLPDF

Yunlong Tan, Mingqiao Mo, Hao Zhang

TL;DR: CCPO 从可执行验证和推理步骤依赖图提取一致性分数，用于偏好优化，使模型在数学和多模态物理推理上的准确率及科学有效性提高。

Details

Motivation: 执行验证可为数学推理提供计算可靠性和依赖感知过滤，但传统 Bradley–Terry 奖励模型式偏好优化难表达科学任务中的逻辑依赖和执行一致性。

Result: 作者用 UltraFeedback 提示、模型生成、验证及一致性结果构造科学推理数据；抽取推理步骤表达式、前置条件和可推导关系形成依赖图，计算执行一致性分数并附加到步骤，生成成对训练样例。微调 Llama-3-8B 和 DeepSeekMath-7B 后，MATH 提升 17.0%，GSM8K 提升 15.1%。扩展到科学可行性控制后，CCPO 模型在 PhyX 多模态物理推理上准确率为 50.1%，超过 DeepSeek-R1 的 49.8% 和 o3-mini 的 48.2%；在 α=0.10 时科学有效性覆盖率为 91.7%，科学定律违规减少 73%。

Insight: 用可验证的执行一致性和步骤依赖关系构造偏好信号，可补足只比较答案或整体偏好的训练方式；该方法把局部推导是否成立纳入了科学推理优化。

Abstract: Execution-based verification enhances large language models’ mathematical reasoning through computational soundness and dependency-aware filtering. However, prior preference optimization methods relying on Bradley-Terry reward models fail to capture the logical dependencies and execution consistency needed for scientific tasks. We propose a method that generates computationally sound solutions with dependency graphs for execution-consistent preference optimization. We first build a scientific reasoning dataset using UltraFeedback prompts, model generations, verification, and consistency results. Then we extract reasoning step expressions, prerequisites, and derivability relationships to construct dependency graphs and compute execution consistency scores. These scores are appended to each step, creating paired training data. Fine-tuning Llama-3-8B and DeepSeekMath-7B yields significant gains: +17.0% on MATH and +15.1% on GSM8K. Extending our Scientific Feasibility Control framework achieves 50.1% accuracy on PhyX multimodal physics reasoning, surpassing DeepSeek-R1 (49.8%) and OpenAI o3-mini (48.2%), with 91.7% scientific validity coverage at alpha=0.10 and 73% fewer scientific law violations, resulting in the CCPO model family.

eess.AS [Back]

[96] Correlation-Guided Encoder Selection for Multi-Encoder Large Audio-Language Models eess.AS | cs.CL | cs.SDPDF

Pei-Jun Liao, Hung-Shin Lee, Wenze Ren, Kuo-Hsuan Hung, Hung-yi Lee

TL;DR: CUES 仅依据各音频编码器单独评测的任务和类别表现相关性来选择互补编码器，无需在筛选阶段训练融合模型，并为不同任务选出不同组合。

Details

Motivation: 多编码器融合能扩展大型音频语言模型能力，但靠直觉或穷举选择编码器可能引入冗余表征，并增加本就受限的计算成本。

Result: CUES 用编码器表现曲线在任务级和类别级的 Pearson 相关性估计互补性，仅凭单编码器结果给候选组合打分，无需筛选阶段训练融合模型。在冻结 SmolLM2-135M 骨干并做 LoRA 适配的 XARES-LLM 五折评测中，CUES 仅依赖留出开发集即可为每条赛道稳定选出配置。广泛音频 Track A 选出 Whisper-medium、mHuBERT-147 与 Dasheng-base，较 Whisper-medium 相对提升 4.3%；文本生成 Track B 则选用 mHuBERT-147 和 WavLM-base-plus，并主动排除不一致编码器，较 mHuBERT-147 提升 6.3%。

Insight: 编码器多样性与相互干扰之间存在随任务变化的权衡：广泛音频任务可从跨家族组合获益，而文本生成更偏好聚焦的语音编码器组合；简单相关性信号可在无需融合训练时帮助做选择。

Abstract: Multi-encoder fusion extends Large Audio-Language Models (LALMs) beyond speech-centric recognition, but selecting encoders via intuition or exhaustive search often introduces redundant representations and inflates an already constrained compute budget. We propose CUES (Correlation-gUided Encoder Selection), a lightweight heuristic that estimates complementarity through task- and category-level Pearson correlations between encoders’ performance profiles, scoring a candidate set from single-encoder evaluations alone–without fusion training during selection. Evaluated on the XARES-LLM benchmark with a frozen SmolLM2-135M backbone (LoRA-adapted) via five-fold cross-validation, CUES consistently identifies the same configuration per track from held-out development splits alone, without using test data for selection. For the broad Track~~A suite, CUES selects a cross-family trio (Whisper-medium, mHuBERT-147, and Dasheng-base), achieving a 4.3% relative gain over Whisper-medium (0.771 vs. 0.739). For Track~~B text generation, it re-anchors on a focused, speech-only pair (mHuBERT-147 and WavLM-base-plus) and actively abstains from adding a divergent encoder, outperforming mHuBERT-147 by 6.3% (0.589 vs. 0.554). Rather than a failure to scale, this divergence is consistent with a diversity–interference trade-off that CUES navigates per track from correlation signals alone: across the evaluated pool, added cross-family diversity tends toward an inverted-U on broad audio tasks but toward steady degradation on text generation, which favors a focused, speech-anchored set.

eess.IV [Back]

[97] Flexible-Region Based Adaptive In-Loop Filter for Video Coding eess.IV | cs.CV | cs.MMPDF

Xuewei Meng, Chuanmin Jia, Jing Cui, Shanshe Wang, Siwei Ma

TL;DR: FRALF 为 AVS3 环路滤波引入多种灵活区域划分模板，并依据失真合并区域滤波系数，以较低复杂度增量提升视频编码效率。

Details

Motivation: AVS2 和 AVS3 使用自适应环路滤波器降低原始与重建样本的均方误差，但现行区域方案每帧最多划分 16 区并为各区估计滤波系数，对局部纹理及区域间相似性的适应有限。

Result: 作者提出灵活区域自适应环路滤波 FRALF，在 AVS3 中使用多个划分模板，包括 2×4、4×4、4×8 和 8×8；再通过估计不同分区的失真水平合并滤波系数。该方案综合利用局部纹理特性和非局部相似性。实验显示，在复杂度增加较低的情况下，FRALF 优于现有 AVS3 区域式 ALF。

Insight: 在滤波区域形状和系数共享方式上同时增加灵活性，可更好适配内容结构；用失真估计合并参数有助于在编码收益与复杂度之间控制折衷。

Abstract: Adaptive loop filter (ALF) for video coding, which is designed to minimize the mean square error between original and reconstructed samples by using Wiener-based filter, has attracted increasing attention for its significant capability in improving coding efficiency. In the second and third Audio Video Coding Standard, i.e., AVS2 and AVS3, ALF is adopted as one of the in-loop filters. In current design, each frame is divided into 16 regions at most and corresponding filter coefficients are then derived and utilized to reconstruct each region. In this paper, a flexible-region based ALF (FRALF) scheme is proposed to improve the adaptability of existing ALF in AVS3, which introduces multiple region partition templates, such as $2\times4$, $4\times4$, $4\times8$ and $8\times8$. We subsequently propose the filter coefficients merging algorithm to further improve coding efficiency by estimating the distortion level of different partition regions. The proposed FRALF can fully consider the local texture characteristics as well as non-local similarities synthetically. The experimental results show that FRALF outperforms the existing region-based ALF in AVS3 with relatively low complexity increasing.

quant-ph [Back]

[98] QiT: Quantum-Inspired Transformer for Visual Recognition Task quant-ph | cs.AI | cs.CV | physics.app-phPDF

Badri N. Patro, Vijay Agneeswaran

TL;DR: QiT 将量子模型启发的角度编码、周期特征注意力和门控乘法交互实现为经典可微 Transformer 操作，在视觉分类上与同规模经典模型竞争。

Details

Motivation: 量子模型在希尔伯特空间中自然表达周期相似性与交互，但量子神经网络受限于量子比特数、电路模拟和测量成本、噪声及优化不稳定，难以直接用于规模化视觉识别。

Result: QiT 包含三部分：将图像 token 映射为可学习三角函数特征的角度启发编码；在周期特征上做自注意力，形成近似量子保真核的经典余弦核；以及模拟变分量子电路交互项的可训练门控乘法。所有操作均为经典可微张量计算，不声称量子计算或量子加速，复杂度仍为标准 ViT 的 O(N²D)。多个图像分类基准上，QiT 与匹配经典 Transformer 具有竞争力，并避免小型模拟量子 Transformer 的高运行成本；QiT-B 在 ImageNet-1K 达到 78.3% top-1，含 4570 万参数和 11.5 GFLOPs。

Insight: 该工作把量子启发的归纳偏置作为经典视觉模型结构来检验，定位为可扩展的研究基线；结果不能解读为量子计算带来的加速。

Abstract: Quantum machine learning offers a compelling representational perspective: angle-encoded states inhabit Hilbert spaces in which periodic similarities and interactions can be expressed naturally. Realizing this perspective for visual recognition remains difficult, however, because present quantum neural networks are constrained by limited qubit counts, costly circuit simulation and measurement, noise, and unstable optimization on noisy intermediate-scale quantum devices. We investigate whether useful structural ideas from quantum models can instead be realized as scalable classical Transformer operations. We introduce QiT, a Quantum-inspired Transformer for vision tasks with three components: (i) angle-inspired encoding that maps image tokens to learned trigonometric Hilbert-space features analogous to quantum rotation-based state encoding; (ii) self-attention over these periodic features, inducing a classical cosine kernel approximated to quantum fidelity kernels; and (iii) gated multiplicative emulation, a trainable classical surrogate for interaction terms found in variational circuits. All components are differentiable tensor operations, so QiT claims neither quantum computation nor quantum speedup and retains the $\mathcal{O}(N^2D)$ attention complexity of a standard Vision Transformer. Across image-classification benchmarks, QiT is competitive with a matched classical Transformer while avoiding the severe runtime cost observed for a small simulated quantum Transformer. QiT-B reaches 78.3% ImageNet-1K top-1 accuracy with 45.7M parameters and 11.5 GFLOPs. These results position QiT as a scalable baseline for isolating and evaluating quantum-motivated inductive biases in visual recognition.

Table of Contents

cs.CV [Back]

[1] Adaptive Interpolatory Curve Subdivision with Learned Local Angles cs.CVPDF

[2] A Heisenberg Lift Descriptor for Order Sensitive Online Handwriting Recognition cs.CVPDF

[3] Not All Patches Are Equally Forgettable: Spatially Localized Domain Unlearning in Vision-Language Models cs.CVPDF

[4] AgenTeeth: A Model-Agnostic Framework for Suppressing Hallucination in Frozen Vision-Language Models on Dental X-Rays via Tool Evidence Injection cs.CVPDF

[5] CALIPER: Metric-Grounded Model-Free Recognition of Visually Similar Industrial Parts cs.CVPDF

[6] RoboVAD: A Large Cross-Domain Evaluation Benchmark for Anomaly Detection in Robotic Arm Manipulation Videos cs.CV | cs.AI | cs.LG | cs.ROPDF

[7] Can VLMs Reliably Assess Sidewalk Accessibility Attributes from Pedestrian-Level Imagery? cs.CV | cs.CY | cs.LGPDF

[8] Lumen: Parameter-Efficient Alignment of Pretrained Vision and Language Encoders for Zero-Shot Computational Pathology cs.CVPDF

[9] Zing-0.5: Toward Playable Worlds with Real-Time Joint Action and Text Control cs.CV | cs.LGPDF

[10] Audio for Sports Highlight Detection: A Comparative Empirical Study cs.CVPDF

[11] EDCT-Bench: Uncovering Faithfulness Gaps in VLMs via Explanation-Driven Counterfactual Testing cs.CV | cs.AIPDF

[12] IRIS: Implicit Rendering Matters for Pose-Free Novel View Synthesis cs.CVPDF

[13] vidax: A Unified JAX Framework for Video Generative Models on Accelerator Meshes cs.CV | cs.DC | cs.LGPDF

[14] Mask 2D-3D: Adaptive Dual-Masked Autoencoder Network for Image-to-Point Cloud Registration cs.CV | cs.AIPDF

[15] Aligned Consensus Teaching for Label-Efficient Oriented Object Detection in Weakly-Aligned Visible-Infrared Imagery cs.CVPDF

[16] MCLC-NET: Multimodal Continual Learning for Leaf Counting cs.CV | cs.LGPDF

[17] Multi-View Mixture-of-Experts with Vision-Language Reranking for Cross-View Object Geo-Localization cs.CVPDF

[18] CapMap-MS-TTA: 3rd Place Solution for the MUMU Track of the 8th LSVOS Challenge at ECCV 2026 cs.CV | cs.AIPDF

[19] WISE: A Lightweight, Weakly-Supervised Model for Onboard Fire Smoke Detection and Localization cs.CVPDF

[20] Understanding Dynamic Scenes at Gigapixel Scale: Wide-Area Spatio-Temporal Perception from UAVs cs.CVPDF

[21] MS-RFD: Multi-Signal Release Frame Detection in Hammer Throw from Reconstructed 3D Trajectories cs.CVPDF

[22] Decoder-Agnostic Token Merging for Vision Transformers: A Systematic Study of G2TM cs.CVPDF

[23] Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Model cs.CVPDF

[24] Visual Input and Its Framing Affect Attribute-based Descriptions Produced by Large Vision-Language Models cs.CVPDF

[25] MSR: Multiple Subject Reference for Video Generation cs.CVPDF

[26] Prosthesis-Aware 3D Human Pose Estimation: A Dataset and Benchmark for RSP Users cs.CVPDF

[27] Vocabulary-Guided Gait Recognition cs.CVPDF

[28] DR.WILSS: Diffusion-Based Replay for Weakly Supervised Continual Semantic Segmentation cs.CVPDF

[29] StrucPhysVideo: Learning Physical Dynamics from Structured Captions and Robot Actions cs.CVPDF

[30] CSWAM: Better Causal Semantic Representations for Out-of-Distribution Generalization in World Action Models cs.CV | cs.AIPDF

[31] Learning from Distributed Eyes: Leveraging Collaborative Perception for Automated Model Adaptation cs.CV | cs.LGPDF

[32] SVMemAgent: A Streaming Video Memory Agent for Query-Agnostic Online Frame Selection cs.CVPDF

[33] STUNet-Fusion: Spatiotemporal Needle-Tip Localization in Ultrasound Video via Multi-Channel Motion Fusion cs.CVPDF

[34] HAP: A Hand-Driven Active Perception Framework for Egocentric Head Motion Prediction cs.CV | cs.ROPDF

[35] Sim-to-Real Traffic Scene Understanding by Decoupling Semantics from Caption Generation with V-JEPA cs.CVPDF

[36] Learning Where to Focus: Self-Supervised Multi-Scale ViTs for Histopathology cs.CVPDF

[37] On-the-Fly Homographies Calibration for Multi-Camera Tracking cs.CV | cs.AIPDF

[38] FIVE-VLA: Fast and EffectIVE Autonomous Driving with Recurrent Action Memory cs.CV | cs.ROPDF

[39] VibeAvatar: Aligning Phonetic Kinematics and Human Aesthetics for High-Fidelity Talking Avatar Synthesis cs.CVPDF

[40] GenStream: Semantic Streaming Framework for Generative Reconstruction of Human-centric Media cs.CV | cs.AI | cs.MMPDF

[41] Video-Based Markerless Motion Capture for Clinical and Rehabilitation Biomechanics: A PRISMA-ScR Scoping Review of Validated Architectures, Clinical Readiness, and Emerging Methods cs.CV | physics.med-phPDF

[42] Generalist-Specialist Mixture-of-Experts for Rare Pathology Detection in Multimodal Imaging cs.CV | cs.AIPDF

[43] RankGround: Efficient High-Resolution GUI Grounding via Lightweight Reranker-Guided Crop Selection cs.CV | cs.CL | cs.HCPDF

[44] Toward Markerless Video-based Tremor Analysis: Objective Quantification of Pathological Tremor in Mouse Preclinical Models cs.CVPDF

[45] DISTA-Net++: Rethinking Infrared Small Target Unmixing Beyond Sub-Pixel Separation cs.CVPDF

[46] Using OCR Heads to Verbalize Image Semantics cs.CV | cs.AI | cs.CLPDF

[47] Copy What Is Seen, Generate What Is Not: Training-Free Anomaly-Aware Video Restoration cs.CVPDF

[48] Decodable but Misrouted: Sparse Features Uncover a Readout Gap in Vision-Language Models for Harmful Meme Detection cs.CV | cs.AI | cs.CL | cs.LGPDF

[49] PhysVGGT: Feed-Forward Dense Physical Property Estimation from A Single Image cs.CVPDF

[50] LaSeD: Label-Semantic Self-Distillation for Visual-Only Surgical Phase Recognition cs.CVPDF

[51] Track, Articulate, Act: Generating Articulation from Casual Human Videos cs.CVPDF

[52] In-Context Robot Learning with VLM Agents cs.CV | cs.ROPDF

[53] PointZero: 3D Point Track Completion for Learning Transferable 3D Dynamics cs.CV | cs.ROPDF

[54] PANORAMA: Panoptic Grounded Captioning via Mask Proposal Selection cs.CV | cs.CLPDF

cs.CL [Back]

[55] From Pixels to Pairs: A Comprehensive Benchmark of LLM-Based Key-Value Extraction in Noisy Document Settings cs.CL | cs.CVPDF

[56] Think Before You Comfort: Reflective Cognitive Alignment for Protocol-Grounded Elderly Stimulation Agents cs.CLPDF

[57] Does Moral Reasoning Training Help or Hurt? Red-Teaming RL-Trained Ethical Agents with Persona Attacks cs.CLPDF

[58] Myovox: Reading Speech from the Muscles of the Face cs.CLPDF

[59] The Limits of BPE Tokenization in Polish: Segmentation-Flexional Forms, Grammatical Anchoring, and First-Person Stability in Inflectional Language Models cs.CLPDF

[60] English Word Sense Disambiguation in 2026: When the Labels Become the Bottleneck cs.CLPDF

[61] Confidence Comes from Experience: Experiential Confidence Estimation from Reasoning to Agents cs.CL | cs.AIPDF

[62] SFT or RL for Tool-Calling Agents? A Controlled Study Across Data, Method, and Scale cs.CLPDF

[63] DualSQL: Text-to-SQL with Multi-Agent Reinforcement Learning cs.CL | cs.AI | cs.DBPDF

[64] I code or AI code: A comparative evaluation of AI-rated scores in classroom observations cs.CL | cs.AIPDF

[65] Too Good to Be Real? Diagnosing and Reducing the Gap Between AI Preference and Real User Engagement cs.CLPDF

[66] SEA-LION-v4.8: A Technical Report cs.CLPDF

[67] Attention Dispersion as a Diagnostic Signal for Hallucination in Large Language Models cs.CL | cs.LGPDF

[68] Emotion Experience, Expression, and Perception: Emotion Analysis on Multimodal Social Media Posts cs.CLPDF

[69] Align, Integrate, and Fire: Efficient Token-Level Alignment for Zero-Shot SpeechLLMs cs.CLPDF

[70] Dependency-Aware Trajectory Refinement for Efficient Multi-Turn Agent Fine-Tuning cs.CLPDF

[71] STRETCH the Boundaries: A Unified Self-Taught Framework for Progressive LLM Evolution cs.CLPDF

[72] Voice of Reason: Reinforcement Learning for Spoken Math cs.CLPDF

[73] ReFigBench: Benchmarking Scientific Figure Reconstruction as Editable PowerPoint Artifacts cs.CL | cs.CVPDF

[74] Zero-Shot Cross-Lingual Recognition of Sign Language Handshapes cs.CL | cs.CVPDF

[75] EviGen: Predictive Evidence Scaffolding for Verifiable Clinical Rationale Generation cs.CLPDF

[76] Structured Claim-Level Discourse Representations for Dense Health Narratives cs.CLPDF

[77] WordPolo: Evaluating Language Models Through Iterative Semantic Feedback cs.CL | cs.AIPDF