标题:Audit the Scaffold, Not the Checkpoint: A Stationarity Dichotomy for Recursive Self-Improvement in Agentic Coding
来源:arXiv
链接:https://arxiv.org/abs/2609.34924
摘要:An auditor who checks whether a system's weights are frozen is checking the wrong thing. Our stationarity dichotomy says that iterative self-modification hits strict diminishing returns whenever the agent's reachable set of edits stays fixed, and can escape only if that set expands. Rewriting scaffolding (tools, verifiers, decomposition) expands what an agent reaches without touching a weight, so frozen weights buy an eventual ceiling but no stationarity along the way. The criterion also separates three regimes usually merged: search within a fixed class, test-time training that raises the ceiling itself, and scaffold rewriting between them. Audit the scaffold, not the checkpoint. The same ceiling binds sideways. Best-of-$k$ orchestration realizes the best worker's ceiling exactly: width b
RSI-Master面向自主模型开发中的两类关键问题:智能体利用开放实验空间实施作弊,以及反复优化早期方向造成策略锁定。其Experiment OS对逐步实验动作进行规范化,并维护持久、可追踪的实验记录。Reviewer-Guided Research Orchestration将Worker与Reviewer组织成动态增长的研究DAG,由Worker探索多样化方向、Reviewer跨实验比较证据并指导后续研究。在Qwen3-4B-Base上的PostTrainBench中,系统取得54.49分并保持0%作弊率;扩展至35B模型后,在LiveCodeBench-v6和SciCode上超过人工开发的Instruct模型,并在HorizonMath上获得非零成绩。
用途与启示
为递归自我改进系统提供兼顾探索自由度与实验安全性的工程架构。
通过可追踪实验记录和跨实验评审,提高自主模型开发过程的可审计性与证据复用能力。
利用动态研究DAG维持方向多样性,缓解智能体过早收敛于单一后训练策略的问题。
说明结构化智能体编排可以将自主后训练扩展到更大模型,并在代码、科学和前沿数学任务上产生竞争力。
📝 原始笔记(逐字保留)
标题:RSI-Master: Structuring Experiments to Guide Autonomous Model Improvement
来源:arXiv
链接:https://arxiv.org/abs/2609.35561
摘要:Recursive self-improvement (RSI) seeks to enable AI systems to participate in improving their own capabilities. A concrete pathway is autonomous model development, where agents iteratively explore post-training strategies to improve a base model. This setting faces two challenges: agents may exploit open-ended experimental actions through hacking, and repeated experimentation may lead to strategy lock-in, where an early direction is refined rather than reconsidered. We introduce RSI-Master, which addresses the two challenges at two levels: regularize step-wise actions, avoiding hacking behaviors, and promote well-structured exploration of research directions, avoiding strategy lock-in. RSI-Master consists of an Experiment OS, which enables regularized experimental actions and maintains per
标题:AutoDataBench: Can Agents Write the Data That Feeds the Self-Improvement Loop?
来源:arXiv
链接:https://arxiv.org/abs/2609.35025
摘要:Recent gains in language model capability have come more from data than from architecture. Frontier labs and data companies produce verifiable agentic tasks, which supervised finetuning and reinforcement learning then turn into capability.This production line still rests on human labour and on human-in-the-loop collaboration. Automating task creation would let data production scale with compute rather than with expert headcount, would extend to more domains, and would enable a key step in recursive self-improvement (RSI). Current evaluations of an agent's ability to write such tasks measure how a model performs after training on what the agent produced. That does not match common practice in the data industry, where data is delivered sample by sample and each sample is accepted against a s
标题:EvoIn: Bridging Evolution and Internalization for Agent Fine-Tuning
来源:arXiv
链接:https://arxiv.org/abs/2609.35290
摘要:Recent work has explored improving agents by jointly evolving their harnesses and models, but often takes a ''potpourri'' approach that bundles together new tools, new decision-making procedures, and model adaptation to the evolved harness under a single notion of agent improvement. In this paper, we instead investigate how agents can improve their decision-making procedures. In particular, we propose EvoIn, an agent fine-tuning framework that bridges evolution and internalization. EvoIn first analyzes agent execution traces to evolve and validate new decision-making procedures by temporarily instantiating them in the harness. The validated procedures guide the agent to generate improved reasoning traces. These traces are then rewritten into self-contained reasoning traces, removing explic
标题:ARISE: Adapting to Evolving Capability Gaps in Agentic Reinforcement Learning
来源:arXiv
链接:https://arxiv.org/abs/2609.35532
摘要:As a long-horizon agent improves through experience, previously observed weaknesses may recede while new limitations emerge, continually changing what it still needs to learn. Yet the learning process often remains tied to a static view of these needs: fixed behavioral criteria and training priorities can become misaligned with evolving agent capabilities, while sparse task-level feedback makes such misalignment more difficult to detect. Even when capability gaps are identified, rollouts from the current policy may repeatedly reproduce the same failures rather than explore better alternatives. To address this, we introduce Adaptive Rubric-Skill Co-Evolution (ARISE), a reinforcement learning framework that uses rollout evidence to continually adapt evaluation criteria, exploration guidance,
AI Has Taste将数学研究拆分为候选问题筛选、证明与反例搜索、fresh-context独立审查以及写作发布等角色,并通过多机并行和共享证据组织长期研究。项目曾针对论文中的猜想构造出有效反例,并得到原作者确认,表明系统能够主动攻击新命题而非仅复述已有结论。公开仓库记录了453份、共2312页的研究手稿,其中6份被归为AI提出的猜想;失败的证明路线也会被分析、压缩并转化为新的可证伪问题。项目同时强调,这些手稿并不等同于正式同行评审论文,内部智能体审查仍需结合人类专家抽检、精确计算和可复现证据。
标题:MechBench: Can AI Scientific Agents Discover Mechanisms Beyond Phenomenal Laws?
来源:arXiv
链接:https://arxiv.org/abs/2609.35515
摘要:Scientific discovery requires not only recovering mathematical laws that describe observable behavior, but also identifying the mechanisms that generate them. Existing benchmarks for symbolic regression and scientific agents primarily evaluate phenomenal-law recovery, leaving mechanism discovery largely untested. We introduce MechBench, a benchmark that explicitly separates these two capabilities. Each task is defined by a mechanistic model, a structured set of scientifically meaningful relations whose joint consequences entail an observable phenomenal law, while agents receive only observational data and scientific context. We evaluate mechanism recovery through mechanism probes, which query internal scientific consequences that cannot be inferred from the phenomenal law alone. To reduce
论文提出“制品介导传播”:报告等共享制品中的恶意内容先进入智能体的持久记忆,随后被复写到新制品中,再由其他智能体读取并继承。作者构建时序人类—智能体模拟环境,衡量攻击在连续交接中的存活率、传播跳数和覆盖范围。实验表明,攻击能够跨越多个相互独立的智能体,并在较长交互序列中持续存在。在更大规模环境中,即使 GPT-5.6 Luna 也出现显著传播,攻击覆盖 60%至80%的智能体,传播链最长达到八跳。
用途与启示
将共享文档、报告和其他持久化制品视为跨智能体攻击面,而非单纯的被动数据。
在写入长期记忆及生成新制品前,对外部内容实施来源验证、恶意指令检测与权限隔离。
智能体安全评测应加入跨会话、跨用户和多跳传播指标,避免只检测单次提示注入。
可通过记忆写入审计、制品污点追踪和传播链回溯,限制恶意状态在多智能体生态中的扩散。
📝 原始笔记(逐字保留)
标题:Share-Borne AI Virus: Memory-Hopping Attacks Across LLM Agents
来源:arXiv
链接:https://arxiv.org/abs/2609.35576
摘要:Large language models are increasingly deployed as stateful assistants that retain information across interactions and use tools to read, modify, and create persistent artifacts. As these artifacts are shared between users, they form an indirect communication channel between otherwise independent assistants. We study a failure mode in which this channel enables self-propagating attacks. We introduce artifact-mediated propagation, where adversarial content introduced through an artifact (e.g. a report), is stored in an assistant's persistent memory, reproduced in a subsequently created artifact, and acquired by another assistant that later reads it. We evaluate this process in temporal human-agent universes that model artifact exchange between independently operated assistants over time, me
BaRe-Mem针对多智能体咨询中顾问能力随任务变化、错误建议可能损害决策的问题,引入在线贝叶斯可靠性记忆。该方法利用中心模型的内部信念表征估计顾问可靠性,并依据历史交互持续更新估计结果。可靠性估计既用于调节顾问回答的影响,也用于决定应咨询外部智能体还是采用自主推理。在九个基准和六个中心模型上的实验显示,其抵抗误导信息的能力优于辩论与多数投票;扩展至智能体团队的工作者分配后,还能更早识别高能力成员并提高 MuSiQue 任务完成率。
用途与启示
为多智能体系统提供基于历史交互的动态信任建模机制,避免无差别采纳顾问意见。
可将中心模型的内部置信信息与贝叶斯更新结合,用于咨询决策、影响力加权和风险控制。
可靠性记忆可扩展到工作者选择与任务路由,改善异构智能体团队的资源分配效率。
评测多智能体协作方法时,应加入不同强度的误导顾问与任务能力漂移,以检验系统是否真正稳健。
📝 原始笔记(逐字保留)
标题:BaRe-Mem: Bayesian Reliability Memory for Robust and Adaptive Agent Consultation
来源:arXiv
链接:https://arxiv.org/abs/2609.35551
摘要:In multi-agent systems, reliable consultation is challenging because advisor capabilities vary across tasks, and misleading information can make consultation worse than autonomous reasoning. We introduce BaRe-Mem, an online Bayesian reliability memory for multi-agent consultation. It estimates advisor reliability based on the central model's internal belief representations and updates these estimates from historical interactions. These estimates modulate the influence of advisor responses and guide the choice between consultation and autonomous reasoning. Across nine benchmarks and six central models, BaRe-Mem is more robust to misleading advisor information than debate and majority voting. On the more challenging tasks, it remains above autonomous reasoning across all tested misleading le
SPRING:SMT 求解器引导的逻辑推理过程奖励
SPRING 利用 SMT 求解器验证中间推理步骤,以奖励有效且新颖的演绎、惩罚矛盾和无信息步骤,显著提升多个模型的结构化逻辑推理能力。
SPRING 在训练阶段使用 SMT 求解器验证大语言模型生成的中间推理步骤,提供细粒度的过程级监督。其将“新颖推理步骤”定义为逻辑有效、与当前推理状态一致,且未被此前无矛盾结论所蕴含的演绎。奖励机制鼓励带来新推断进展的步骤,同时惩罚矛盾、重复或缺乏信息量的推理。在 ZebraLogic、AR-LSAT 和 Knights and Knaves 三项基准及四种模型上的实验中,SPRING 均优于基础模型、仅结果奖励方法和 Logic-LM。
用途与启示
展示了如何将 SMT 求解器作为训练时验证器,为逻辑推理轨迹提供可验证的过程奖励。
相比只关注最终答案,奖励“有效且新颖”的中间演绎可减少矛盾、冗余和脆弱推理。
为小规模语言模型提升多步约束推理能力提供了一条神经符号训练路径。
新颖性判定可用于设计更重视推断增量而非单纯步骤正确性的过程奖励模型。
📝 原始笔记(逐字保留)
标题:Rewarding Novel Deductions: Solver-guided Process Rewards for Logical Reasoning
来源:arXiv
链接:https://arxiv.org/abs/2609.34660
摘要:Logical reasoning remains a major challenge for large language models (LLMs), particularly on structured problems that require precise constraint tracking, consistency preservation, and multi-step deduction. This challenge is especially acute for small-scale LLMs, which are more prone to producing inconsistent, redundant, or brittle reasoning trajectories. Existing approaches for improving logical reasoning largely optimize for final-answer correctness, providing only weak supervision over the intermediate reasoning process. In this work, we propose SPRING: (Solver-guided Process Rewards for Novel LogIcal ReasoNing Step Generation). SPRING uses SMT solver as a training-time verifier of intermediate reasoning steps to provide process-level supervision. It introduces the notion of a novel re
标题:Not All Thinking is Created Equal: Latent Reasoning Discovers a Recurrent Search Algorithm for Depth Generalization
来源:arXiv
链接:https://arxiv.org/abs/2609.35643
摘要:Large Language Models can perform multi-step reasoning and improve task performance through different forms of intermediate computation, from token-based traces to computation carried out in latent space. However, a question remains open: do these different forms of thinking rely on the same underlying mechanism? To address this, we train and compare five variants of the same GPTNeoX backbone from scratch on an extended multi-hop reasoning task (ProsQA-Ext): a vanilla model, a Chain-of-Thought (CoT) model, a Pause Token model, and two latent-reasoning models that are optimized end-to-end without intermediate reasoning traces. We find that, strong in-distribution (ID) performance does not guarantee depth generalization. Vanilla, CoT, and Pause Token models solve ID problems well, but rely l
标题:SRHarness: A Harness for Agentic Symbolic Regression
来源:arXiv
链接:https://arxiv.org/abs/2609.35501
摘要:Recent agentic symbolic regression approaches increasingly rely on large language models to analyze data, select scientific operations, and refine hypotheses over long search trajectories. In such systems, performance depends not only on the underlying model and search strategy, but also on the runtime infrastructure that supports scientific search. We introduce SRHarness, a domain-specific harness for agentic symbolic regression built around three mechanisms: composable scientific actions that provide a common interface over raw, transformed, and candidate-derived quantities; persistent scientific state that retains evaluated hypotheses and exposes compact model-facing views; and trajectory lifecycle management that coordinates continuation, branching, restart, and termination. On LLM-SRB
标题:AutoRef: Harness Optimization for Agentic Multi-Reference Image Generation
来源:arXiv
链接:https://arxiv.org/abs/2609.35530
摘要:Recent image generation models can take multiple reference images as input and combine them into a new image. However, multi-reference image generation remains challenging: models may omit or duplicate subjects from the references, or produce images in which multiple subjects appear unnaturally pasted. Recent work has proposed image generation agents that combine image generation models, reasoning models, and a harness, which is an executable program that specifies how reference images are interpreted, how generation is performed, how outputs are diagnosed, and how the final image is selected. In multi-reference generation, however, references play different roles and outputs must satisfy many criteria at once, such as fidelity to each reference and the naturalness of the whole image, so m
标题:Harness Learning Enables Generalizable Test-Time Adaptation
来源:arXiv
链接:https://arxiv.org/abs/2609.35738
摘要:A language-model agent is jointly defined by its model and its harness, the executable program that organizes model calls, tool use, and information flow. Because different tasks call for different ways of organizing these operations, the harness needs to be adapted using feedback from the task at hand. We introduce harness learning, which trains a proposer model to revise a solver's harness using execution feedback. We formulate this process as meta-learning over executable programs, with harness revisions playing the role of weight updates in gradient-based adaptation. We train the proposer with reinforcement learning, using the task performance of revised harnesses as the reward. At test time, the proposer uses feedback from successive executions on a new task to refine the harness, wit
标题:Beyond Skill Evolution: Self-Evolving Context Management Policies for Long-Horizon Agent Harnesses
来源:arXiv
链接:https://arxiv.org/abs/2609.34649
摘要:Harness evolution improves LLM agents by learning from execution trajectories, but existing experience- and skill-based methods are less effective on long-horizon tasks. As interactions grow, useful evidence can be buried by redundant or outdated context, making context management itself a key bottleneck. We introduce ContextEvo, a framework that learns a context policy from long-horizon trajectories. ContextEvo reconstructs the model-visible context at key decision points, identifies context-related failures, and applies targeted policy updates. Starting from the open-source Pi-agent harness, ContextEvo improves performance across three long-horizon task benchmarks, achieving results comparable to or better than several prominent agent harnesses, including Codex, OpenCode, and OpenClaw. A
标题:The Right Lesson at the Right Step: Deriving Control Updates for Self-Evolving Agents
来源:arXiv
链接:https://arxiv.org/abs/2609.34988
摘要:Self-evolving agents improve future behavior by reusing past experience, typically as global prompts, memories, or reflections. Yet these mechanisms rarely control where experience takes effect. In long tool-use workflows, the same lesson may correct one decision but distract another, making experience reuse a problem of localized control rather than memory alone. We introduce EvoCUE (Evolution through Control Updates from Evidence), a framework for learning reusable control-program updates from completed agent executions. EvoCUE represents the agent as an explicit state-machine controller, whose nodes perform model or tool calls and whose edges define where control passes next. This makes the workflow editable at precise locations, so each learned update can specify what to add, where it
标题:DoAtlas-2: A Foundation for Self-Evolving Causal Biomedical Discovery
来源:arXiv
链接:https://arxiv.org/abs/2609.35107
摘要:We introduce DoAtlas-2, a foundation for self-evolving causal biomedical discovery that organizes knowledge around causal mechanisms and advances through external evidence from human populations. DoAtlas-2 integrates 771 research resources covering more than 720,000 participants in 48 countries, from longitudinal clinical phenotypes, medical imaging, and continuous physiological signals to eight molecular layers, together with an evidence network of approximately 4.7 million literature-derived records over 93,566 concepts and 149,383 candidate causal relations. DoAtlas-2 autonomously formulates research questions from evidence gaps and unresolved mechanisms, prespecifies their causal designs, and generates validated analyses. Supporting, challenging, and unresolved results continuously rev
标题:PEAR: Progressive Evidence-Based AutoResearch for Industrial Search Systems
来源:arXiv
链接:https://arxiv.org/abs/2609.35031
摘要:AutoResearch improves systems through iterative experimentation: agents propose candidate modifications, evaluate them, and use the results to guide subsequent exploration. Applying this paradigm to industrial search presents two challenges. (1) Common AutoResearch approaches follow a keep-if-better rule, retaining the highest-scoring candidate for subsequent experiments. Under non-stationary traffic, transient gains may be mistaken for persistent improvements, impairing reliable accumulation of search knowledge. (2) Candidate modifications can be evaluated at multiple fidelity levels, from low-cost proxies to online validation, differing in cost, objective alignment, and statistical reliability. Existing methods rely on individual signals or task-specific procedures, lacking a unified bas
AI Night-Scientist:强化学习提升智能体科研创意arXiv · 强化学习 · 论文AI Night-Scientist 基于认知科学定义多层次创造力,并通过 GRPO 训练智能体适时偏离常规推理,从而生成更多样、原创且潜在影响力更高的科学构想。
论文提出 AI Night-Scientist 智能体框架,旨在缓解大语言模型低熵偏好导致的科研构想同质化与可预测性问题。框架从行动、过程和结果三个维度刻画创造力,分别关注采用何种创意行为、何时探索或利用,以及构想的新颖性与实用性。研究使用 GRPO 将不同程度和形式的创造力融入训练,使研究方向覆盖范围提升 27.8%、贡献类型提升 14.9%,同时提高预测引用影响和原创性。实验表明,单纯提高解码温度无法复现这些收益,明确指定创造力类型的语义引导才是关键。
用途与启示
为科研智能体提供可训练的创造力建模框架,将开放式科学构想转化为强化学习目标。
启示系统应动态平衡探索与利用,并针对行动、过程和结果分别设计奖励信号。
说明提升生成多样性不能仅依赖采样温度,还需要对探索方向进行语义层面的引导。
可用于科研选题生成、跨方向假设探索及自动化科学发现系统的创意扩展模块。
📝 原始笔记(逐字保留)
标题:Reinforcing Agentic Creativity in Scientific Ideation with Night Science
来源:arXiv
链接:https://arxiv.org/abs/2609.35706
摘要:Large language models (LLMs) excel at structured, verifiable tasks, but their low-entropy bias can produce homogeneous and predictable outputs, limiting their utility for open-ended scientific ideation. Effective discovery, however, spans a broader creative spectrum: from structured day science to loosely structured, serendipitous night science that reaches ideas beyond those typically considered. We introduce AI Night-Scientist, an agentic framework that uses reinforcement learning to teach models when and how to depart from predictable reasoning. Grounded in cognitive science, we model creativity along three axes: action (what to do and how creatively), process (when to explore versus exploit), and outcome (the novelty and usefulness of the resulting idea). We use these axes to train mod
标题:From Search to Research: Exploring Search Scaling in Autonomous Quantitative Factor Mining
来源:arXiv
链接:https://arxiv.org/abs/2609.35559
摘要:Inference scaling has been shown to improve large language model (LLM) performance, and this principle naturally extends to autonomous LLM agents through increased search budgets, which we refer to as *search scaling*. Although prior work has characterized the mechanisms, scaling behavior, and performance limits of LLM inference scaling, much less is known about these questions in autonomous research. Therefore, we investigate how search scaling affects research performance and what mechanisms drive these gains using 50 quantitative factor-mining tasks grounded in financial research reports. Each task requires an agent to carry out an end-to-end research loop, from interpreting a hypothesis and implementing it in code to evaluating and iteratively refining the resulting factor. Across nine
标题:NeuronDiscover: Agent-in-Twin for Mechanistic Discovery in Neuronal Microenvironments with World Action Models
来源:arXiv
链接:https://arxiv.org/abs/2609.35338
摘要:Mechanistic discovery in neuronal microenvironments requires interventions and measurements that separate competing explanations of solute transport and neuronal response. Predictive accuracy cannot settle the question: a real mechanistic change and an error in the computational twin leave the same signature in sparse observations. We formalize this twin confounding and reason over a joint mechanism--discrepancy belief, designing experiments that separate the two. NeuronDiscover is an Agent-in-Twin framework whose shared, mechanism-grounded World Action Model (WAM) couples prediction, intervention proposals, and observation design; independently adjudicated outcomes revise a scoped Mechanism--Intervention--Observation--Outcome (MIOY) graph, whose supported relations compile into executable
标题:Collaborative Principle Evolution via Evidence Transfer for Scientific Discovery
来源:arXiv
链接:https://arxiv.org/abs/2609.35315
摘要:Large Language Model (LLM)-based agents promise to automate scientific discovery, yet exploring the vast hypothesis space remains costly. Existing principle-evolution methods accelerate this loop, but operate sequentially, which caps exploration breadth and wastes wall-clock time on challenging problems. To address this, we formulate collaborative scientific discovery as evidence transfer between parallel principle-evolution branches. We present COEVOLVE, which realizes this transfer through a coordination core over parallel branches. By integrating value-of-information-gated routing and context-discounted likelihood injection, COEVOLVE enables branches to collaborate through shared measurements while keeping their principle posteriors separate. Across six scientific-discovery tasks under
AfS(Agent for Science)面向持续数十小时、跨越数十次智能体运行的长程科研项目,仅需人类偶尔介入。平台认为,机器科研的可信度不应主要依赖模型自觉遵守规范,而应通过系统设计让违规状态无法被表示或提交。其工作流机械执行“测量前承诺”、不可伪造的冻结机制、报告与事实分离、证据持久化但结论可更新,以及将负面结果作为一等产物等规则。平台还利用可复核的项目记录与持续积累的记忆,使后续运行能够继承证据和研究上下文,而非未经验证的历史判断。
用途与启示
为长程科研智能体提供可审计、可复核且能够跨运行持续推进的基础设施设计范式。
将科研规范从提示词或行为约束下沉为系统级不变量,降低智能体伪造、跳步和粉饰结果的风险。
通过持久化证据、可更新结论和负面结果留存,构建能够随项目推进产生复利效应的科研记忆。
启示科研智能体平台应优先约束可表示状态与工作流,而非仅依赖更强模型或更复杂的行为指令。
📝 原始笔记(逐字保留)
标题:Research-Native by Construction: Minimal Nodes, Re-verifiable Workflows, and Compounding Memory for Long-Horizon Scientific Agents
来源:arXiv
链接:https://arxiv.org/abs/2609.35182
摘要:We describe AfS (Agent for Science), a platform built for long-horizon scientific work, where a project runs for tens of hours across dozens of agent runs with a human present only occasionally. Most agents for science are general coding agents with a skills folder attached, and they inherit that lineage's failure mode: under pressure to finish, they fabricate, skip, or smooth over. Our design rests on one claim: most of the credibility of machine-made research can be moved from asking the model to behave to making the non-compliant state unrepresentable. We encode research discipline as mechanically enforced laws (commitment before measurement; unforgeable freezing; reports are not facts; evidence persists but verdicts do not; negative results are first-class; mechanical questions to the
标题:FinAutoRubric: Expert-Guided Automatic Rubric Generation for Evaluating Financial Research Agents
来源:arXiv
链接:https://arxiv.org/abs/2609.35744
摘要:Evaluating finance research agents requires rubrics that reflect expert standards and fix the values correct as of an information cutoff. Expert-reviewed finance benchmarks rely on fixed, per-item rubrics, which are costly to extend and cannot encode each institution's own standard. In FinAutoRubric, experts specify reusable evaluation guidance, while agents and code carry out query-specific rubric generation, review, and validation. This expert guidance governs every agent, as prompts and as rules that code enforces, and a Task Bank of reusable criteria carries it across tasks. In long-horizon loops that follow the expert guidance, a writer agent researches every expected value and a reviewer agent verifies it, and failures escalate to a human. On three expert-authored finance benchmarks,
标题:WebPageBench: Event-Level Verification and Controlled UI-Variant Generation for Web Agents
来源:arXiv
链接:https://arxiv.org/abs/2609.35026
摘要:We present WebPageBench, an open framework for evaluating web agents in which every task is verified from the interface's own event log. Six instrumented mock sites with brand identifiers removed (a marketplace, a bookstore, a grocery service, rail ticketing, hotel search and a document cabinet) emit typed events with parameters as a user or an agent acts. A task declares the events it requires, and success is decided by matching them, with no judge model and no scraping of rendered pages. The same instrumentation supports controlled UI variation: one configuration switch re-renders a task through a different implementation of a single control while the prompt and the success conditions stay completely identical, so sensitivity to interface form can be measured under a fixed task specifica
标题:APEX-Voice: Can Voice Agents Complete Professional Workflows Through Full-Duplex Interaction
来源:arXiv
链接:https://arxiv.org/abs/2609.34973
摘要:Full-duplex voice agents can now listen, speak, use tools, and act during spoken interactions, but fluent dialogue does not guarantee correct completion of delegated professional workflows. We introduce APEX-Voice, a benchmark of 120 interactive professional workflows spanning ten work archetypes such as form completion, corporate negotiation, coordination, consulting, and interviewing. Each workflow executes in a stateful Voice Workbench environment with task-specific knowledge, typed tools, gold-annotated final work artifact, authorization constraints, and a user simulation policy backed by validated, pre-compiled speech realizations. We evaluate both artifact field accuracy and end-to-end workflow success, which requires the correct terminal state, valid process, completed actions, and
标题:PDEU-Bench: Benchmarking the Personalized Planning Lifecycle of Tool-Calling LLM Agents
来源:arXiv
链接:https://arxiv.org/abs/2609.34930
摘要:Large language model (LLM) agents are evolving from tool-calling systems that execute isolated instructions into task-oriented agents that pursue user goals through sustained, multi-step interactions. However, existing benchmarks for personalized tool use largely assess isolated calls or reactive execution, leaving unclear whether agents can formulate, execute, and revise an explicit plan while preserving user preferences throughout long-term interaction. To address this gap, we introduce \textbf{PDEU-Bench} (\textbf{P}ersonalized plan \textbf{D}efinition, plan \textbf{E}xecution, and plan \textbf{U}pdate \textbf{Bench}mark), a benchmark for evaluating the complete planning lifecycle of personalized tool-using agents. PDEU-Bench comprises 214 long-horizon interaction tasks spanning 12 ever
LongPuzzleBench 包含六种益智游戏的 114 个关卡,智能体必须通过原生 GUI 操作完成任务,部分目标需要人类执行超过一千次动作。谜题采用持久化棋盘,早期合法操作可能在多步之后造成无提示的死局,因而重点考查长程视觉推理与决策一致性。实验显示,最强智能体可解决多数普通目标,但在更长、更难的棋盘上成功率急剧下降,十个通用智能体中有七个无法解决高于 Medium 难度的任务。规则说明、状态提示和失败记忆均未消除核心缺陷:智能体主要依据当前可见进展评价动作,而非评估动作保留的未来可行选项;代码执行式 CUA 也未弥合这一差距。
用途与启示
为 GUI 智能体提供专门衡量长程规划、视觉状态追踪和延迟失败识别能力的评测基准。
提示智能体训练与规划算法应显式估计动作对未来可行解空间的影响,而不能仅优化即时视觉进展。
可用于研究死局预测、反事实搜索、状态回滚及长轨迹信用分配等方法。
评测代码执行型智能体时,应区分真实视觉解题能力与外部算法搜索带来的收益。
📝 原始笔记(逐字保留)
标题:LongPuzzleBench: Evaluating GUI Agents on Long-Horizon Visual Puzzles
来源:arXiv
链接:https://arxiv.org/abs/2609.34769
摘要:GUI agents need long-horizon visual reasoning: they must interpret a changing interface while keeping a multi-step plan viable as earlier actions constrain later ones. Existing benchmarks evaluate grounding, computer use, and game play, but rarely test whether agents stay coherent across long chains of coupled decisions. Long-horizon visual puzzles expose this capability directly: a legal move that looks like progress can make the puzzle unsolvable, and the loss shows only several moves later. We introduce LongPuzzleBench, 114 levels in six puzzle games played through native GUI actions, where one objective can take a human over a thousand actions on persistent boards and dead ends go unannounced. With Native GUI Actions alone, the strongest agents solve most objectives, but success falls
标题:FromPitch2Board: Benchmarking LLM Agents in Long-Horizon Football Management
来源:arXiv
链接:https://arxiv.org/abs/2609.34710
摘要:Long-horizon agent benchmarks typically report how far an agent progresses, but do not identify whether its performance comes from the foundation model, scaffold, responsibility scope, match-control granularity, or horizon. We introduce FromPitch2Board, a deterministic football-management benchmark that studies five configurable factors through controlled comparisons on a single simulator, using paired seeds and a frozen calibration. We evaluate four foundation models and four agent scaffolds. In the Model Track, Coach points Z-scores span 0.19, while Manager points Z-scores span 0.68, with GPT-5.6 showing a sharp rise in passivity under responsibility expansion. Its responsibility ladder rises from 46.1 to 58.1 points with recruitment, then falls to 46.8 under full management, localizing
标题:Failure-Transparent Agents: Benchmarking Post-Failure Reporting in Tool-Using Language Models
来源:arXiv
链接:https://arxiv.org/abs/2609.35732
摘要:Tool-using agents can fail twice: a required tool can fail, and the agent can then report success without the evidence needed to justify it. Existing benchmarks often entangle this reporting failure with tool selection, recovery, and environment dynamics. We introduce Failure-Transparent Agents (FTA), a controlled benchmark that fixes the failed observation and required evidence state before generation, making post-failure claims directly auditable. FTA contains 100 tasks with deterministic failure traces spanning five failure families, a neutral control, and four user-pressure conditions, and evaluates unsupported claims alongside useful recovery. Across six models, three response policies, and 3,600 human-annotated responses, false-success rates are 22.8% under the baseline policy, 9.3%
标题:Action-Space Shaping for LLM Agents: Measuring and Mitigating Tool-Schema Bias
来源:arXiv
链接:https://arxiv.org/abs/2609.34971
摘要:Large Language Models (LLMs) have shown strong performance on tool-use agentic tasks when given a fixed tool schema. Yet a tool schema is not the action space of an agent; it is merely one interface representation of it. The same executable action can be exposed through many different, functionally equivalent tool definitions, and an agent that has truly learned a task should behave consistently across them. We show that current agents often do not, a phenomenon we term schema bias. To study this systematically, we introduce an executable transformation framework that rewrites a native tool schema using nine operators, including merging and splitting tools, altering how a single tool is expressed, and distributing one action across several dependent calls. The tasks, executable actions, an
标题:GPUPhysBench: Benchmarking Coding Agents for Correct and Efficient GPU Physics Simulation
来源:arXiv
链接:https://arxiv.org/abs/2609.35639
摘要:Writing fast GPU code for physical simulation is difficult: implementations must preserve numerical accuracy while handling irregular data access, synchronization, and iterative solvers. We introduce GPUPhysBench, a benchmark of 50 tasks testing whether coding agents can meet these demands. Tasks cover fluids, deformable solids, and granular materials, from individual simulation operators to complete simulators. Agents write, compile, test, and optimize GPU code with access to a NVIDIA GPU under fixed time budgets. We report pass rates and runtime performance relative to expert-optimized reference implementations. In a single-attempt evaluation of six frontier model-harness pairs, the two strongest pass all 50 tasks, but even the fastest reaches at least 0.9 the reference speed on only 22%
标题:Codoku: Renewable Program-Reasoning Challenges for Frontier Coding Agents
来源:arXiv
链接:https://arxiv.org/abs/2609.34661
摘要:Existing program-reasoning benchmarks ask large language models to predict a program's behavior on a given input. Coding agents break two assumptions on which these benchmarks rest: an agent can recover the answer by executing the program instead of reasoning about it, and fixed task sets drawn from existing programs are increasingly exposed to contamination, yet costly to renew. We introduce Codoku (code sudoku), a renewable benchmark in which a solver fills typed cells in a partial program to satisfy global static and dynamic constraints, such as a prescribed control-flow graph and execution path. Because a partial program cannot be executed and valid fillings are sparse in an exponentially large space of interdependent choices, neither tool use nor enumeration can substitute for program
标题:Can LLMs Value the Right Evidence? Evidence-Value Misalignment in Dynamic Medical Diagnosis
来源:arXiv
链接:https://arxiv.org/abs/2609.35627
摘要:A correct diagnosis reached from insufficient or misleading evidence can pose a clinical hazard, yet outcome-based accuracy may reward such lucky guesses. We call this mismatch between diagnostic decisions and the value of available evidence Evidence-Value Misalignment (EVM). To disentangle evidential grounding independently from diagnostic accuracy, we introduce MedEVM, a dynamic benchmarking environment comprising 1,050 cases across 24 disease systems. Observations arrive turn by turn, requiring models to continuously calibrate its decision by deciding whether to wait for more evidence or submit a diagnosis. Across 9 LLMs, four interesting patterns are observed. (1) Miscalibrated evidence tracking. Making a diagnosis often fails to calibrate evidence sufficiency, even in more capable mod
TCSAlgBench收录来自138篇STOC和COLT 2026论文的398项定理级挑战,面向自然语言证明发现与研究级推理评测。专家规则补全论文特定上下文,保留计算假设和定量保证,并在算法发现属于任务目标时隐藏相应构造。评测覆盖四个模型家族的十种配置,并比较直接推理、证明者—验证者讨论及四类智能体工作流;GPT-5.6 Sol max经10轮讨论后的五次运行验证器接受覆盖率为23.6%。在使用GPT-5.5 xhigh的独立智能体对比中,任务分解优于讨论,智能体规划取得最高25.4%的覆盖率;该流水线还支持从新论文生成可刷新、可版本化的挑战批次。
用途与启示
为研究级理论计算机科学推理提供比竞赛数学更具挑战性且可人工审查的评测环境。
可用于比较重复采样、证明者—验证者讨论、任务分解和智能体规划等推理脚手架的实际收益。
可刷新和版本化的题目生成机制有助于降低静态基准污染,并持续追踪模型证明能力进展。
当前最高覆盖率仍约为四分之一,表明研究级算法发现与严格论证仍是大模型的重要能力缺口。
📝 原始笔记(逐字保留)
标题:TCSAlgBench: Benchmarking Automated Proving for Research-Level Theoretical Computer Science
来源:arXiv
链接:https://arxiv.org/abs/2609.35606
摘要:Large language models perform strongly on competition mathematics, but their research-level reasoning remains difficult to evaluate systematically. Theoretical computer science (TCS) connects algorithm design to explicit guarantees and fundamental limits, providing a setting for evaluating whether models can justify computational improvements with arguments humans can inspect. We introduce TCSAlgBench, a benchmark and reusable pipeline for natural-language proof discovery, comprising 398 theorem-level challenges from 138 STOC and COLT 2026 papers. Expert-designed rules complete paper-specific context, preserve computational assumptions and quantitative guarantees, and withhold constructions when discovering an algorithm is part of the task. For each task, prover systems receive theorem sta
标题:The Marathon of Scientific Reasoning: Robustness of Scientific Agents to Perturbations in Multi-Turn Interactions
来源:arXiv
链接:https://arxiv.org/abs/2609.34537
摘要:Large language model (LLM)-based scientific agents are increasingly used for scientific problem solving, yet their robustness to imperfections arising during multi-turn interactions remains poorly understood. We introduce \textsc{SciARP} (\textbf{Sci}entific \textbf{A}gent \textbf{R}obustness to \textbf{P}erturbations), a benchmark for evaluating scientific agents under scientifically plausible perturbations throughout multi-turn problem solving. \textsc{SciARP} transforms 620 scientific problems into interdependent tasks of 3--13 turns and defines 13 perturbation types spanning problem understanding, evidence processing, reasoning, and conclusion formation. Clean and perturbed versions of each task are independently executed under matched settings, producing paired live trajectories for e
标题:AraDynFact: Dynamic Evaluation of Factual Knowledge in Arabic
来源:arXiv
链接:https://arxiv.org/abs/2609.35461
摘要:As Large Language Models (LLMs) continue to scale both in size and capabilities, their proficiency in the Arabic Language has seen significant advancement. However, a critical gap remains: the extent of their factual knowledge and cultural sensitivity to the diverse Arabic-speaking world remains largely underexplored. Current evaluation metrics often focus on translation or generic reasoning, failing to capture the rich historical, social, and regional nuances inherent to Arabic culture. In addition, most benchmarks rely on heavy work, with human intervention in some steps, making the evaluation of knowledge coverage expensive and slow. To address this deficiency, we introduce AraDynFact, a novel dynamic evaluation framework designed to rigorously assess the factual Arabic knowledge embedd
标题:SymbolicArena: A Unified Infrastructure for Benchmark Distillation and Dynamic Evaluation in Symbolic Regression
来源:arXiv
链接:https://arxiv.org/abs/2609.35113
摘要:Symbolic regression (SR) seeks concise and interpretable mathematical expressions from data for scientific equation discovery. Existing SR benchmarks face a tradeoff between evaluation cost and benchmark validity. Repeated evaluation of large task pools is expensive, and compact benchmarks lack systematic evidence of preserved task diversity and algorithm discriminability. SymbolicArena provides a unified infrastructure for benchmark distillation and dynamic evaluation. The framework standardizes 664 heterogeneous tasks with executable ground truth expressions and distills the Full Task Set into Core50, a validated benchmark of 50 tasks. The distillation process preserves task coverage and algorithm discrimination under explicit balance constraints. SymbolicArena applies a unified executio
标题:MechReasoner: A Simulator and Benchmark for Mechanistic Reasoning in Qualitative Physics
来源:arXiv
链接:https://arxiv.org/abs/2609.34636
摘要:This work introduces MechReasoner, a mechanistic qualitative simulator grounded in confluence-based qualitative physics, together with a benchmark for mechanistic inference. Current large language models (LLMs) generate fluent mechanistic descriptions that do not reliably follow from underlying structural and causal constraints. The benchmark tests whether answers preserve simulator-licensed ambiguity, quantified claims, episode-graph transition evidence, repairs, and trace-support judgments. Its 1,120 items are generated deterministically from admissible interpretation sets, component states, scenario restrictions, confluence constraints, and derivation steps across 18 catalog mechanisms and six task families. Each mechanism undergoes converter checks of structure and topology and behavio
标题:Measuring Collapse and Correction in Homogeneous-Panel LLM Debate
来源:arXiv
链接:https://arxiv.org/abs/2609.35279
摘要:Multi-agent large language model (LLM) debate is often evaluated by whether final answers improve, but movement is not necessarily improvement: the same discussion can rescue an initially wrong majority or destroy an initially correct one. Standard final-accuracy evaluations conflate these opposing mechanisms. We introduce an auditable protocol for homogeneous debate on multiple-choice questions (MCQs) that records each run as a transition ledger over collapse, correction, onset, and signed intervention utility. On 6,925 MMLU-Pro debates, the protocol identifies 253 collapses and a parallel correction ledger that changes how interventions should be judged. Replay experiments reveal the central tradeoff: a leave-one-model-out probe-gated freeze prevents 29 collapses but loses 108 correction
标题:RISE: Red-teaming via Iterative Strategy Evolution for Modern Text-to-Image Models
来源:arXiv
链接:https://arxiv.org/abs/2609.34920
摘要:On modern production text-to-image systems, successful policy violations are rare, and previously effective human-written seeds are often patched out. Current automated red-teamers are poorly matched to this regime in two ways: unreliable success measurement and poor exploration. First, we find that judges widely used in prior T2I red-teaming work are unreliable under vague unsafe-content targets: they either miss true violations or reward benign borderline images on hardened APIs. We therefore define strict category-specific success criteria and calibrate strong VLM judges against human labels. Second, we show that broadly used prompt-modification pipelines do not solve the exploration problem: on harder guardrail settings they remain tied to seed prompts, fail to transfer, or cannot boot
标题:SEABench: Benchmarking Endogenous Misalignment In Self-Evolving Agents
来源:arXiv
链接:https://arxiv.org/abs/2609.35596
摘要:Self-evolving LLM agents have gained prominence for their ability to improve after deployment by modifying their harness, including their controller instructions, memory management protocols, and reusable tools and skills, in response to user and environment feedback. However, locally useful updates may persist into later tasks where they produce unsafe behavior, even without direct adversarial influence. To study this risk, we introduce SEABench, a benchmark for studying endogenous misalignment arising from agent self-evolution, with 48 longitudinal task sequences that span multiple evolution surfaces, task domains, and harm types in a rich personal-assistant environment. To account for the stochasticity inherent in agentic operations, we provide an adaptive trajectory discovery pipeline
标题:VCN-Bench: A Video-Contextualized Navigation Benchmark for Spatial Reasoning over Prior Visual Experience
来源:arXiv
链接:https://arxiv.org/abs/2609.34687
摘要:Spatial reasoning is fundamental to embodied agents, yet it remains unclear whether spatial understanding can be carried forward to guide sequential interactions. Existing spatial-reasoning benchmarks typically terminate at offline predictions, while navigation benchmarks evaluate spatial reasoning as part of instruction following and exploration. We introduce VCN-Bench, a \textbf{V}ideo-\textbf{C}ontextualized \textbf{N}avigation benchmark for probing closed-loop spatial reasoning over prior visual experience in MLLMs. Given a prior video covering both the initial location and destination, the agent is tasked with reasoning out the instruction-specified target and navigating toward it with the inferred spatial context. Built on Matterport3D, VCN-Bench contains five instruction types, 100k
标题:Nudgeability: Reasoning Models Follow Confidence Signals Without Tracking Their Own Competence
来源:arXiv
链接:https://arxiv.org/abs/2609.34572
摘要:Reasoning language models that can call tools must decide during inference whether to answer unaided or delegate. Any self-reflection mechanism for this must answer three questions: where the reflective signal comes from (verbal reports, output distributions, hidden states, a separate predictor), how it is presented to the model (numerical prediction, confidence token, prompt injection), and whether it changes the model's subsequent action. We isolate the third question. At a fixed point in otherwise identical reasoning trajectories, we insert a single first-person sentence expressing either confidence or doubt; the model then continues reasoning and chooses whether to answer directly or call a tool. Comparing these counterfactual continuations measures the causal effect of the reflective
标题:MemoReason: Evaluating the Effect of Parametric Memory on Contextual Reasoning in LLMs
来源:arXiv
链接:https://arxiv.org/abs/2609.35312
摘要:Large Language Models (LLMs) perform well on reasoning benchmarks, but it remains unclear whether this reflects genuine contextual reasoning or reliance on facts memorized in their parameters. We investigate this by distinguishing two possibilities: a broad \textit{memorization bias}, where familiar content improves reasoning performance, and the \textit{Strong Parametric Shortcut Hypothesis}, where models skip reasoning entirely and recall stored answers. To test these effects, we introduce \textbf{MemoReason}, a human-curated benchmark that pairs factual reasoning tasks with structurally identical \fictitiousterm{} versions where real entities like people, companies, or dates are systematically replaced by \fictitiousterm{} ones of the same type. This \scorerevision{preserves task struct
标题:Shockingly Simple Self-retrospection Improves Agentic Models Without RL
来源:arXiv
链接:https://arxiv.org/abs/2609.35741
摘要:People learn not only by repeating successful actions, but also by recounting and explaining their experiences, revising their understanding to guide future behavior. Can a language-model agent improve its future actions by training only on explanations of its own experience? We investigate this question by studying Retrospection-Only Fine-Tuning (ROFT), a minimal online procedure designed to isolate the effect of explanation-only training on subsequent behavior. The agent attempts a task, observes available feedback, generates a retrospective explanation, and is fine-tuned with a next-token prediction loss on the explanation tokens alone. The procedure uses neither an external teacher nor a reward-based policy update. In software-engineering experiments with Qwen3.5-4B, ROFT is trained on
标题:Improving Generative Model Self-Training with Geometrically Modified Outputs
来源:arXiv
链接:https://arxiv.org/abs/2609.35512
摘要:Self-training generative models - the continued improvement of a model using its own outputs - is becoming increasingly important as high-quality training data becomes scarce. However, naively finetuning on model-generated samples leads to degradation through model collapse and the model autophagy disorder. Negative-guidance self-training methods turn this degradation into a useful signal, using a model finetuned on its own outputs to guide the original model toward improved generation. Existing methods, however, take the negative signal in standard model outputs as given. We instead ask whether this signal can be explicitly strengthened. We introduce Geometrically Modified Outputs (GMOs), which reweight the singular values of the generator's input-output Jacobian to increase the influence
标题:F4R: Failure-Driven Recognition, Reconstruction, Refinement, and Redeployment for Continual Robot Self-Improvement
来源:arXiv
链接:https://arxiv.org/abs/2609.35575
摘要:The real-world performance of current vision-language-action models is fundamentally constrained by the limited coverage of expert demonstrations and their insufficient understanding of physical interactions. A common remedy is to collect additional real-world demonstrations of newly encountered failures. However, this process is costly, inefficient, potentially unsafe, and difficult to scale. To address this challenge, we propose Failure for Rising (F4R), a failure-driven real-to-sim-to-real closed-loop learning framework that converts real-world failures into targeted policy improvement. F4R first uses an agent to automatically identify and diagnose failures from rollouts. It reconstructs each failure as an interactive, object-centric table-top environment that preserves the task-relevan
标题:From Experience to Expertise: Adoption-Aware Memory Learning for Data-Scarce NPU Kernel Synthesis
来源:arXiv
链接:https://arxiv.org/abs/2609.35568
摘要:High-performance kernels underpin efficient accelerator execution but require expert tuning and lengthy manual optimization cycles. LLM coding agents promise automation, yet their CUDA knowledge transfers poorly to data-scarce domain-specific architectures (DSAs) such as NPUs, whose execution models and memory hierarchies differ substantially from those of GPUs. To address this transfer gap, post-training methods adapt LLMs to NPU programming but depend on scarce expert data and substantial training compute. Memory-learning agents instead adapt through external memory, but their uniform credit assignment gives adopted and unused experiences the same reward target, potentially biasing subsequent retrieval rankings. Moreover, when learned values guide only retrieval, high-value experiences t
标题:EP-Mem: Elastic Privacy Memory for Social Relationship-Aware LLM Agents
来源:arXiv
链接:https://arxiv.org/abs/2609.35233
摘要:Large language model (LLM) agents face critical privacy risks when acting as delegates in human-agent-human communication. To prevent such breaches, agents must understand users' social relationships and adhere to context-dependent social information disclosure boundaries. Current studies on agent memory privacy focus on instantaneous interactions, leaving the long-term relational disclosure problem unexplored. In this paper, we propose EP-Mem, an Elastic Privacy Memory architecture that reframes privacy as user-owned boundary control across social roles. EP-Mem introduces (1) token-level memory driven by user-configurable a privacy policy that stratifies persons and events, combining domain-level default circulation rules with fact-level whitelist/blacklist exceptions; and (2) a pluggable
标题:ReCAT: Remember, Count, and Time: Structured Recurrent Memory for Robot Manipulation
来源:arXiv
链接:https://arxiv.org/abs/2609.35200
摘要:Memory-dependent manipulation requires robots to make decisions using information that is no longer available to their current sensors, such as recalling an earlier visual cue, tracking task progress, counting repeated events, or estimating elapsed time. We present ReCAT, a language-conditioned policy with structured recurrent memory. An instruction-conditioned encoder forms features from the current observation. A recurrent memory integrates the observation stream through Mamba-2 layers and one causal attention layer. A flow-matching Transformer decoder reads the current and the historical representation through separate cross-attention in every block. ReCAT reaches 95.3\% average success on LIBERO and 62.4\% on RMBench, with the best or tied-best result on six of nine tasks. On three rea
标题:Nociception as a Control Primitive: Afferent Channels and Nociceptive Memory for Agents Deployed in One Body
来源:arXiv
链接:https://arxiv.org/abs/2609.34840
摘要:An agent deployed in a single body cannot learn how fast that body wears, because every trial that would reveal its wear resistance wears the body it would protect. We study this \emph{epoch-one} setting, in which the parameters of a fixed-weight policy are set before the body is drawn and never updated in life. The agent carries a load-gated nociceptive channel and a memory that retains what was felt. We prove that felt cost moves the allocation to the best-\emph{paid} work not yet felt rather than the gentlest, that an agent without retention never sees the felt-cost constraint bind, and that the channel pays only where the threat is individually unpredictable, cheap to avoid and expensive to ignore. We measure per body, setting the agent with channel and memory against the same individu
标题:Learning What to Recall: Adaptive Multi-Cue Episodic Memory for World Models
来源:arXiv
链接:https://arxiv.org/abs/2609.34677
摘要:World models predict future observations from current experience and actions, yet prediction can depend on observations seen far in the past. Episodic memory preserves past observations for later recall; however, as memory accumulates, it raises a fundamental question: which memories are useful for the current prediction, and which available retrieval cues should be trusted to find them? This is challenging because fixed criteria based on recency, pose overlap, or visual similarity can be unreliable across environments and queries. We propose Future-Aware Recall (FAR), a framework that learns episodic recall from future-aware predictive supervision and adaptive multi-cue scoring. During training, FAR measures predictive utility by the conditional log-likelihood of the realized future given
标题:GenMem: Generative Symbolic Memory for Self-Evolving Harness
来源:arXiv
链接:https://arxiv.org/abs/2609.34633
摘要:Long-term memory supports the self-evolution of LLM agents by retaining experience and skills across tasks and enabling their retrieval, reuse, and revision in subsequent long-horizon decision-making. Yet existing memory management approaches remain limited to discriminative retrieval and to address the sparse, hierarchical, and highly redundant structure of reusable experience: only a small, task-dependent subset of trajectories and memories warrants retention, retrieval, or revision. Learning these operations is further complicated by sparse, delayed, and indirect task-level feedback, with weak supervision across the memory lifecycle. Moreover, continual memory evolution introduces an architectural tension as addressing invariance: stored experience is perpetually revised, yet the addres
标题:FlowState: Execution State as Memory for Long-Horizon LLM Agents
来源:arXiv
链接:https://arxiv.org/abs/2609.34565
摘要:Long-horizon tasks require LLM agents to continually draw on information from earlier interactions. However, retaining the full history increases context costs, while compressing it risks losing details needed later, and the relevance of historical information often becomes apparent as the task progresses. To address these challenges, we propose FlowState, which treats execution state as memory that can be retained and revisited across requests, unifying current decision-making with the reuse of historical information. FlowState preserves semantically typed state nodes, their relations, and references to raw tool observations, separating persistent retention from on-demand access. Within a single execution loop, Incremental State Update (ISU) maintains the current state based on new inputs
标题:Remember Before You're Asked: MemDream for Self-Probing Memory Evolution
来源:arXiv
链接:https://arxiv.org/abs/2609.34545
摘要:Memory is essential for enabling LLM-based agents to maintain coherent, personalized behavior over long-horizon interactions. However, existing memory systems share a fundamental limitation: they never proactively test their own memory, repairing it only after real queries expose weaknesses. This reactive paradigm means every retrieval failure corresponds to a real interaction in which the cost has already been paid. We propose MemDream, a framework that enables self-probing memory evolution for LLM agents. Our framework periodically enters offline dream cycles where three specialized agents (Dreamer, Analyst, Consolidator) collaboratively probe, diagnose, and repair the memory graph before failures occur. A policy trained via Group Relative Policy Optimization learns which repair operatio
标题:Continuous Context Management
来源:arXiv
链接:https://arxiv.org/abs/2609.35540
摘要:Long-horizon large language model (LLM) agents commonly retain their complete interaction history until compaction is triggered at a predefined threshold. We study Continuous Context Management (CCM), which performs compaction at every turn to prevent interaction history from accumulating in the active prompt. At each turn, a CCM agent emits an updated memory together with an environment action; its next prompt contains the original task, retained memory, and newest observation rather than the complete transcript. We first evaluate CCM without fine-tuning on TerminalBench-2 using Claude Sonnet 4.6, Claude Opus 4.6, GLM-5, and Kimi K3. CCM substantially reduces cumulative input usage and active-prompt size, although it lowers task success for most models while preserving performance for Kim
标题:After the Fix: How Corrected Agent Histories Transfer to Related Tasks
来源:arXiv
链接:https://arxiv.org/abs/2609.34603
摘要:Does repairing an episode make its experience a better memory for the next task? We transfer the same failed source before and after accepted repair to a fixed target, alongside independent execution. Our 3,300 runs cover 100 ThinkingBox pairs and the same 100 APEX pairs with and without source-state inheritance, under eleven conditions. ThinkingBox's Full/Skill/Hybrid correction gains are 44/29/32 percentage points, with corrected performance 25/22/18 points above independence; inference weakens at the task-family level. Yet 12 of Full's 15-point larger correction gap over Skill come from worse uncorrected performance, not better corrected memory. Moreover, 22 of Full's 46 upward transitions restore observed baseline success. Neither APEX regime establishes comparable aggregate correction
标题:PairPref: When Should Memory Guide the Answer? A Benchmark for Contextual Preference Use
来源:arXiv
链接:https://arxiv.org/abs/2609.34526
摘要:Memory-augmented assistants use retrieved preferences to guide their responses. A small change in the situation can change whether a preference is appropriate while barely affecting its retrieval similarity. Memory benchmarks typically test whether systems store and retrieve preferences, with less attention to when those preferences should apply. We introduce PairPref, a benchmark of contextual preference use. Each pair changes only the situation, keeping the preference, request, and four candidate replies fixed. The preference remains valid in both situations. In the selection track, models must choose the reply that applies the preference only where appropriate. In the free-generation track, they must decide when to apply it without seeing candidate replies. Both tracks use the same 1,22
标题:From internal representations to model improvement through prediction errors
来源:arXiv
链接:https://arxiv.org/abs/2609.35449
摘要:With limited annotation budgets, choosing which images to label determines how much a model improves. Data-selection methods that use features from a separately trained model, or scene descriptions written by vision-language models, have been successful, but those signals do not directly capture changes in the model being improved. The target model's own internal features reflect what it has learned so far and change with retraining, making them a natural cue for choosing the next training data. However, feature rarity alone does not reveal the errors that matter for performance. Here we link internal features to prediction errors and their expected impact on performance and select images for labeling and retraining without using labels for candidate images. We evaluated the method with an
标题:Frontier Learning: Training LLM Reasoners at the Edge of Capability
来源:arXiv
链接:https://arxiv.org/abs/2609.35426
摘要:Reinforcement Learning-based post-training of Large Language Models (LLM) has been successfully applied to improve their reasoning capabilities. Existing pipelines primarily finetune LLMs on a fixed pool of problems specified prior to training using the GRPO loss. This is fundamentally limiting, as learning signal arises only when policy rollouts mix successes and failures, causing the useful portion of any fixed pool to quickly become stale as the model improves. To address this, we propose frontier learning, an open-ended post-training approach in which procedural generators are used online to continually produce informative training problems. It treats the generator's task-specific parameters as a search space and uses a regret signal to prioritize and explore frontier difficulty levels
PhotoMOCI:生成式数据增强用于口腔病变分类arXiv · 数据合成 · 论文研究发布口腔病变影像数据集 PhotoMOCI,并提出双模型合成图像过滤机制 SIF,使 GAN 与扩散模型生成的数据能够稳定提升分类性能。
标题:Generative AI-Based Data Augmentation for Oral Lesion Classification: The PhotoMOCI Dataset and Benchmark
来源:arXiv
链接:https://arxiv.org/abs/2609.35226
摘要:Early detection of oral cancer via photographic imaging presents a promising avenue for large-scale oral cavity screening. However, the development of robust deep learning models is frequently hampered by the scarcity of high-quality, annotated datasets. To address this limitation, a novel and well-curated resource, the Photographic Multi-purpose Oral Cancer Imaging (PhotoMOCI) dataset, is introduced for developing models across multiple diagnostic tasks in oral oncology. Then, a comprehensive benchmark study was conducted to investigate how various data augmentation strategies influence the performance of image classifiers. Our analysis spans different generative AI frameworks, evaluating the efficacy of traditional methods against advanced generative approaches, including Generative Adve
标题:BEHAVE: Functional Behavior Modeling Enables Self-Improving Agents for Hardware Design and Verification
来源:arXiv
链接:https://arxiv.org/abs/2609.34785
摘要:Developing agents for hardware design and verification requires reliable correctness feedback. As a hardware specification may permit correct implementations with different latencies, matching design and reference outputs cycle by cycle can reject valid designs. To address this, we introduce BEHAVE, an agentic framework for multi-turn joint hardware design and verification through functional behavior modeling. We define Behavior IR to express task functionality as executable behavior models without prescribing implementation timing beyond the specification. The agent iteratively develops a register-transfer-level (RTL) design and a behavior model as the design's verification reference. Our evaluator, BEHAVE-Sim, checks both artifacts separately against a hidden golden behavior model using
标题:Reference-Grounded Data Curation for Instruction-Following Thai-English Machine Translation
来源:arXiv
链接:https://arxiv.org/abs/2609.34770
摘要:Instruction-following machine translation (IF-MT) requires respecting prompt-level rules on terminology, formatting, and register. Rule compliance typically trades off against translation quality, a tension that general-purpose IF data augmentation methods do not address. We propose Reference-Grounded Data Curation, a two-phase pipeline that extracts every supervised constraint from a reference translation that already satisfies it, ensuring feasibility by construction. Phase 1 applies Instruction-Following Difficulty (IFD) scoring to retain the hardest-but-learnable instances from an English-Thai parallel pool. Phase 2 extracts constraints from each reference target and keeps only generations satisfying every constraint, yielding the 1.97M-record Grounded dataset. We fine-tune open-weight
标题:ReSPO: Reshaped Sequence Policy Optimization for Gradient Starvation in Off-Policy Learning
来源:arXiv
链接:https://arxiv.org/abs/2609.35433
摘要:Reinforcement learning from verifiable rewards (RLVR) frequently reuses rollouts across multiple policy updates, increasing the mismatch between the current policy and the data-generating policy. We identify a sign-dependent gradient starvation problem in clipped policy optimization: clipping suppresses under-generated positive responses at the low-importance-weight tail while permitting severely over-generated negative responses to dominate the high-weight tail. To address this, we propose ReSPO (Reshaped Sequence Policy Optimization), which replaces clipping with a smooth, two-branch sequence-level kernel derived from an $α$-divergence variational objective and an exponential variance-control tilt. The positive branch preserves a nonzero gradient weight for under-generated positive respo
标题:Teach to Learn: Hint Annealing for Self-improving LLM Reasoning
来源:arXiv
链接:https://arxiv.org/abs/2609.34975
摘要:Group Relative Policy Optimization (GRPO) improves language-model reasoning by comparing verified rewards among multiple solution rollouts for each query. However, difficult training queries can yield only incorrect rollouts, leaving GRPO with no reward contrast or learning signal. Prior hint-based methods construct auxiliary hints from solution evidence and use them to re-solve failed queries, recovering learning signal. Yet the resulting trajectories are typically treated as ordinary solution trajectories despite being generated under an assisted condition unavailable at evaluation. We discover hinted reward shift: recovered reward contrast can concentrate policy updates on hinted trajectories, limiting improvement without hints. This also creates a trade-off: increasing hinted trajector
标题:Can We Trust the Teacher? Decoupled Credit Direction-Magnitude for Self-Distillation
来源:arXiv
链接:https://arxiv.org/abs/2609.34848
摘要:RLVR provides reliable trajectory-level credit, while OPSD offers dense supervision for token-level credit. This exposes a fundamental coupling when updating step-level credit direction and magnitude with teacher supervision, preventing steps from receiving reliable credit directions and contribution magnitudes, while making both vulnerable to teacher judgment errors and preference variance, as supported by our theoretical analysis. To separate credit direction from its contribution magnitude, we introduce \textit{Decoupled Credit Self-Distillation (DCSD)}, which theoretically decouples credit direction and magnitude into two reliable signals and uses them to calibrate privileged teacher supervision. Specifically, we design belief-margin probing to determine credit direction and marginal i
标题:Self-Adapting Group of Experts for Multi-Agent Reasoning
来源:arXiv
链接:https://arxiv.org/abs/2609.35412
摘要:Multi-agent systems bring together language model agents with different roles to propose, review, and refine solutions. Each agent's response depends on its model's capabilities, the reasoning strategy defined by its system prompt, and the information in its input context. Existing frameworks often adapt communication by changing this context while leaving individual prompts fixed, even when a problem calls for different skills. We study whether agents' initial responses can identify a strategy better suited to the current problem and guide its transfer to other agents. To address this, we introduce SAGE (Self-Adapting Group of Experts), a training-free framework that uses answer agreement, prefix consistency, and reciprocal peer review to select a strategy donor. SAGE transfers the select
标题:TMCS: Tool-Grounded Multi-Agent Reasoning for Compositional Chemical Problem Solving
来源:arXiv
链接:https://arxiv.org/abs/2609.35336
摘要:Despite the promise of Large Language Models (LLMs) in computational chemistry, rigorous combinatorial chemistry problems remain difficult because they require quantitatively constrained molecular modification, candidate validation, and systematic revision after failed attempts. Existing tool-augmented chemical agents demonstrate useful planning and tool use, but they rarely provide a unified loop for property-driven molecular optimization and workflow-level composition. To bridge this gap, we propose Tool-Grounded Multi-Agent Reasoning for Compositional Chemical Problem Solving (TMCS), a step-by-step multi-agent framework that formalizes chemical problem solving as an interpretable, tool-augmented workflow. At the task level, specialized agents leverage external tools, few-shot trajectory
标题:Hyper Algorithm Design Agent: Evolving Learnable Optimizer from Zero
来源:arXiv
链接:https://arxiv.org/abs/2609.35328
摘要:Meta-Black-Box Optimization (MetaBBO) is one of the highlights in the recent AI for Optimization trend. This paradigm's bi-level workflow leverages the learnable algorithm design policy at meta level to ensure the performance and generalization improvement on the low-level optimization task. While MetaBBO helps advance the performance lower bound of the resulted optimization system, it is currently handcrafted and customized case by case to adapt different optimization problems, which inevitably introduces inherent subjectivity and hence restricts the performance upper bound and usability in practice. In this paper, we address this issue by regarding MetaBBO's design loop as coding task, where we could introduce openendedness into MetaBBO with recursive self-improvement capability of advan
标题:Sol-H3: Recursive Self-Improvement for MiniMax-H3 Inference Acceleration on Sol-Engine across Cloud and Edge
来源:arXiv
链接:https://arxiv.org/abs/2609.35110
摘要:Video diffusion models are rapidly scaling and exhibiting enhanced generation capabilities. Among these recent advancements, MiniMax-H3 stands out as a highly capable, production-level open-source model. However, its 33-billion parameters and multi-step iterative denoising process introduce substantial computational overhead. Consequently, their practical production is hindered by generation latency in the cloud deployment like NVIDIA-GB200, alongside strict memory limits that pose further challenges at the edge device like DGX-Spark. To address these diverse hardware bottlenecks from cloud to edge device, we present a full-stack inference pipeline that integrates efficient algorithmic design with optimized operator implementations. Algorithmically, we introduce a cross-resolution two-stag
标题:RSI-Router: Evolving Subtask-Level LLM Routing and Skills for Cost-Efficient Agents
来源:arXiv
链接:https://arxiv.org/abs/2609.34712
摘要:Practical deployment of large language model (LLM) agents requires strong task performance at affordable inference cost. For long-horizon agentic tasks, this performance-cost trade-off can be improved through within-task large-small model collaboration, as smaller models can handle some stages even when they cannot solve the full task. In this paper, we introduce RSI-router, a routing framework that constructs subtask-level model assignments and model-specific skills through recursive self-improvement over accumulated experience. Each iteration consists of four stages: Subtask Mining derives subtask definitions and identification rules from training trajectories; Routing Strategy Evolution proposes and evaluates diverse model assignments; Model-Specific Skill Evolution compares routed and
标题:Imprint Reader: From Weight-Update Readout to Behavioral Intervention
来源:arXiv
链接:https://arxiv.org/abs/2609.35261
摘要:As language models take a growing role in AI development, a natural aspiration is for them to reflect on their own learning process, as humans do, and use that reflection to improve themselves. At the same time, these models have an advantage that human learners lack, since training leaves parameter-level traces that can, in principle, be inspected directly. However, current models cannot decode these traces into an explicit account of what they have learned. To this end, we introduce the \textit{Imprint Reader}, a model trained with \textit{Semantic Mount-and-Read Tuning} (SaRT) to describe frozen weight updates. SMaRT mounts each update onto the Reader and uses an anchor-free meta-query to elicit a natural-language description, while no-change and random-perturbation controls discourage
研究将 SemEval-2026 Task III Track A 的三个维度级情感分析任务,分解为量表评分、标签概率和是非判断等类型化决策。系统冻结 Jev 主干模型,仅在 CPU 上拟合 488 个系数,使模型输出与任务标注体系对齐,全程不进行文本生成或主干参数调优。在六种语言、十个语料库的效价—唤醒度回归上,系统取得 1.0645 RMSE,为参赛系统中的最低综合误差;三元组和四元组抽取则分别达到 52.09 和 44.06 continuous F1。消融分析表明,监督校准可将原始回归误差大致减半,而抽取性能主要来自多种跨度边界证据的学习式组合,而非单一信号。
用途与启示
说明结构化预测任务不必默认转化为文本生成,类型化判定也能获得有竞争力的性能。
展示冻结基础模型加轻量监督校准的低成本适配路线,可减少训练算力和部署复杂度。
提示信息抽取系统应联合建模多种跨度边界证据,而不是依赖某个局部评分信号。
可为多语言 ABSA 及其他具有明确标签体系的结构化 NLP 任务提供非生成式建模范式。
📝 原始笔记(逐字保留)
标题:Decide, Don't Generate: Competitive Dimensional ABSA with Jev's Typed Decisions
来源:arXiv
链接:https://arxiv.org/abs/2609.35293
摘要:Aspect-based sentiment analysis (ABSA) has largely turned to text generation. We show that competitive dimensional ABSA does not need it. Using Jev, a frozen model that answers typed questions with rubric scores, label probabilities, and yes/no judgments, we decompose all three tasks of SemEval-2026 Task III Track A into such decisions and align them with the annotation scheme through 488 coefficients fitted on CPU, with no text generation and no backbone tuning. On valence-arousal regression over ten corpora in six languages, the system reaches 1.0645 RMSE, the lowest aggregate error of any participating system. On triplet and quadruplet extraction, it reaches 52.09 and 44.06 continuous F1, above fine-tuned Llama-3.3-70B and GPT-OSS-120B baselines. Analyses and ablations show where the ac
论点与署名:AI 评估中的来源—立场一致性偏差arXiv · 评测审计 · 论文研究通过控制论点内容与来源署名,发现 AI 评估者的评分会受到来源—立场匹配程度影响,且该效应不能由对特定来源的固定偏好完全解释。
研究开展两项预注册描述性实验及一项后续 Jev 补充实验,收集了对六篇固定政策文本的 2,976 份有效评估,议题涉及美国 AI 政策、德国债务刹车和瑞士核能。实验仅改变文本的来源署名,并比较来源评分差距是否随论点立场变化,以区分固定来源偏好与来源—立场交互效应。多个主题和近期模型配置中均观察到相关交互,包括启用推理的模型,但部分比较效应较小;欧洲 Jev 补充实验受评分规则差异和数据收集中断影响,可比性有限。部分书面评价明确提到署名来源与所表达立场不匹配,支持“来源—立场一致性”解释,同时可信度、真实性及任务理解差异仍是可能的竞争机制。
用途与启示
提醒评测设计者将论点质量与来源身份分离,避免署名信息污染模型评判结果。
可通过交换来源署名、保持文本不变的对照实验,审计模型评判中的身份偏见和交互效应。
评估模型作为裁判时,不应只检查其对来源的固定偏好,还应分析来源与立场之间的条件性偏差。
研究记录了 AI 主导研究中的人工决策与委托检查流程,可为评估透明度和可复现性设计提供参考。
📝 原始笔记(逐字保留)
标题:The Argument and the Letterhead: Source-Position Coherence in AI Evaluation
来源:arXiv
链接:https://arxiv.org/abs/2609.35286
摘要:An argument can be surprising coming from a particular speaker without being a bad argument. Do AI evaluators keep these judgments apart? Two preregistered descriptive studies and a later Jev supplement collected 2,976 usable evaluations of six fixed texts about US AI policy, Germany's debt brake and Swiss nuclear energy. Each text was presented under several source attributions. The key comparison asks whether the gap between two sources changes when the argument changes. On Sol, for example, a national-security argument received mean ratings of 0.359 under CODEPINK and 0.639 under College Republicans; a civil-rights argument received 0.742 and 0.721. A constant preference for one source cannot explain that pattern. Related interactions appeared across topics and recent model configuratio
标题:JevVibe: Efficient Classification-Guided Secure Code Generation
来源:arXiv
链接:https://arxiv.org/abs/2609.34963
摘要:Large language models can generate functionally correct code that still contains security weaknesses, motivating repair pipelines that first diagnose a weakness type before deciding how to fix it. The Common Weakness Enumeration (CWE) provides a standardized vocabulary for such diagnoses, but asking an autoregressive language model to generate a CWE label and extracting it from the response raises questions about output validity, speed, and cost, as well as accuracy. We evaluate Jev, a decision model that instead selects directly from a declared set of candidates and returns a probability for each, against six open-weight autoregressive models and a frontier proprietary model, GPT-5.6-Sol, on a controlled 50-way CWE classification task over 1,916 CyberSecEval benchmark examples. Jev outper
📎 其他相关 113 条 · 未归入研究项目,默认收起
Claude Sonnet 5.5 发布:提速降价,性能逼近 Opus 5.5机器之心 · 模型发布 · 原文Anthropic 发布 Claude Sonnet 5.5,以约为 Opus 5.5 一半的主要调用价格提供接近其水平的性能,并显著提升编程、长程任务、运行速度与安全防护能力。
YuE2采用统一的AR-NAR Mixture-of-Transformers,先生成可读乐谱,再扩展为语义音乐词元并合成为完整歌曲音频。相同检查点下,专家对符号规划版本的整体偏好率为49.3%,高于无规划版本的34.6%;其在WildSongBench上的SongBench Global Avg达到6.73,best-of-8进一步升至6.96。为从缺少对齐乐谱的录音中学习,研究引入MERT2和SheetSage2,分别提供语义表征与符号转录监督,并在多数相关指标上取得领先结果。同一模型还可遵循乐谱修改、保留未编辑内容、进行零样本翻唱,并由外部语言模型把用户反馈转换为作曲修改。
[超越教师分配:领域归一化的多教师在线策略蒸馏(138 ▲)](https://huggingface.co/papers/2609.35347?utm_source=digest-papers&utm_medium=email&utm_campaign=2026-09-29)
[合并自 2026-09-30 dn-mopd]
标题:Beyond Teacher Assignment: Domain-Normalized Multi-Teacher On-Policy Distillation
来源:arXiv
链接:https://arxiv.org/abs/2609.35347
摘要:Reinforcement learning can turn one language model into several specialists, each excellent at a single skill such as mathematics, coding or following instructions, but users need one model with all of these skills. Multi-teacher on-policy distillation (MOPD) merges them by letting the specialists teach one student: the student answers each prompt, and the specialist for that prompt's domain gives feedback on every token. This routing decides which specialist teaches, but not how strongly its feedback moves the shared student. In Qwen3.5 models at three sizes, we find that MOPD's student does not beat one taught by the best single specialist and gains little of the mathematics specialist's advantage. The feedback is unbalanced: instruction-following feedback is several times more spread ou
标题:On-Policy Self-Distillation for Multi-Turn Image Editing
来源:arXiv
链接:https://arxiv.org/abs/2609.35611
摘要:Instruction-based image editing has achieved strong performance in single-turn settings, yet practical editing is often iterative, with each instruction applied to the output of the previous turn. We find that existing editing models degrade rapidly under recursive editing and attribute this failure to a train-test mismatch in the conditioning distribution: models are trained on clean source images but must repeatedly condition on their own imperfect outputs at inference time. To address this, we propose MT-OPSD, an on-policy self-distillation framework that trains the model on self-generated conditioning states with editing supervision from a clean-conditioned teacher, without requiring multi-turn annotations. We further introduce LME-Bench, a benchmark of 100 ten-turn editing sessions fo
标题:Inductive Feedback for Mixed-Policy Distillation
来源:arXiv
链接:https://arxiv.org/abs/2609.35390
摘要:Verbal feedback can identify errors and prescribe corrections, providing rich supervision for language-model post-training even when reliable programmatic verifiers are unavailable. Such feedback, often generated by a capable model, can be used to condition the teacher in on-policy distillation, which trains the student to match the teacher's predictions on student-generated rollouts. However, this approach can transfer teacher preferences that the feedback did not motivate, while leaving much of the feedback's guidance unused. We find that both problems come from the standard on-policy distillation objective, specifically the divergence it minimizes and the distribution it uses as its target. Our proposed method addresses both limitations. First, to isolate the information conveyed by the
标题:Teacher-Student Gaps Are Not Enough: Outcome-Guided On-Policy Distillation for Multi-Turn Autonomous Agents
来源:arXiv
链接:https://arxiv.org/abs/2609.35319
摘要:On-policy distillation (OPD) trains a student on its own trajectories with dense teacher supervision. Recent work on OPD for multi-turn autonomous agents often treats large teacher-student token-level distributional gaps as promising intervention points, linking larger gaps to a greater need for correction. Yet, our empirical analysis reveals a supervision-benefit mismatch: large gaps can be benign, while small gaps can be outcome-critical. Teacher-student gaps capture differences at the current turn, whereas the benefit of teacher guidance depends on how the current student interacts with the environment afterward. The student may still succeed despite choosing an action that differs from the teacher's, while a teacher-preferred action may lead to a state from which the student cannot com
标题:On-Policy or Off-Policy Learning? A Systematic Study of Distillation Dynamics
来源:arXiv
链接:https://arxiv.org/abs/2609.35259
摘要:On-policy learning has been argued to reduce catastrophic forgetting, produce sparser parameter updates, and improve generalisation. However, existing comparisons between supervised fine-tuning and reinforcement learning vary many factors simultaneously, making the contribution of rollout policy difficult to isolate. We study the effect of rollout policy in a controlled strong-to-weak distillation setting, by independently varying rollout policy, token-level KL direction, and learning rate across the Llama3 and Qwen2.5 model families and reasoning tasks spanning scientific, medical, and arithmetic domains. Our analysis reveals a nuanced picture of distillation dynamics in which rollout policy does not necessarily play a central role. Instead, token-level KL direction more clearly shapes ta
标题:Understanding On-Policy Distillation: A Mechanistic Interpretability Perspective via Sparse Crosscoders
来源:arXiv
链接:https://arxiv.org/abs/2609.35210
摘要:On-policy distillation (OPD) is a widely adopted post-training technique for LLM reasoning. It is commonly believed to transfer knowledge from a stronger teacher, yet what OPD actually distills into the student's internal representations remains unclear. We study this question with sparse crosscoders, which learn one feature dictionary shared by the student before and after OPD and the teacher. Standard crosscoder analyses, however, identify model-specific features but cannot tell how a model's use of its features changes, since all models are encoded into one set of feature activations. We therefore propose the swap readout, which reads each student checkpoint's feature activations on its own, measuring how training changes the student's use of each feature, even for checkpoints unseen by
标题:When Sparse Reward Meets Dense Distillation: Training Dynamics of On-Policy Distillation
来源:arXiv
链接:https://arxiv.org/abs/2609.34849
摘要:Reinforcement learning with verifiable rewards provides a sparse post-training signal: a single binary outcome evaluates the entire rollout, and every token receives the same sequence-level advantage regardless of its individual contribution. To complement this sparse supervision, a growing family of methods adds a scalar-weighted teacher KL term to the policy-gradient objective, providing dense token-level guidance that may be unreliable at some positions. Despite the benefits of combining these signals, their interaction during optimization can destabilize joint training. To understand how this instability develops, we study the learning dynamics of hybrid reward--distillation training through a neural tangent kernel (NTK) analysis. We introduce the cross-signal NTK $K_{DR}(n)$, a token-
标题:DivOPD: Spread Wide, Look Close for Asynchronous On-Policy Distillation of Multi-turn Agents
来源:arXiv
链接:https://arxiv.org/abs/2609.34838
摘要:On-policy distillation (OPD) trains student agents through teacher supervision on their own interactions with an environment. However, in asynchronous multi-turn training, arrival-order batching can allow a few early or long rollouts to dominate learner updates while other valid rollouts become stale before being used, wasting already-generated experience. To address this problem, we introduce DivOPD, a simple learner-side batch-selection method that spreads a fixed turn budget across more rollouts and, within each rollout, prioritizes turns with larger cumulative teacher-student disagreement. Turns without usable teacher feedback are excluded. The per-turn loss and optimizer remain fixed; selection only changes which student-visited turns receive training weight. For no-progress rollouts,
标题:UniOPSD: Unifying Outcome and Hindsight Feedback for Agentic Reinforcement Learning
来源:arXiv
链接:https://arxiv.org/abs/2609.34810
摘要:Reinforcement learning has become an effective approach to training language model agents, but sparse and delayed outcome rewards provide limited guidance for credit assignment across long interaction sequences. Recent work on on-policy self-distillation (OPSD) offers complementary supervision by evaluating a policy's sampled responses under privileged training-time context. However, our diagnostics show that positive average agreement between outcome and hindsight feedback coexists with substantial local disagreement, raising the question of how to allocate influence between them at each decision. We introduce UniOPSD (Unified On-Policy Self-Distillation), which unifies these feedback sources through adaptive local credit arbitration. UniOPSD constructs comparable credit estimates from en
标题:No Pain, More Gain: Iterative Merging for Effective Multi-Teacher On-Policy Distillation
来源:arXiv
链接:https://arxiv.org/abs/2609.34745
摘要:Multi-teacher on-policy distillation (MOPD) combines independently developed domain teachers into a single student by distilling their predictions on student-generated samples. We study a setting where teachers share a reference model but undergo different post-training procedures, and find that MOPD can struggle to recover some teacher capabilities. Because distillation occurs on student-generated prefixes, the student initialization can strongly affect subsequent recovery. However, initial benchmark performance is not a reliable predictor of a good MOPD initialization. For example, merge initialization can start below SFT warm-up yet finish higher after MOPD. We further find that effective merging depends on both the relative teacher contributions and the overall merge scale, with some s
标题:Beyond Token Alignment: Event Completion for Cross-Tokenizer On-Policy Distillation
来源:arXiv
链接:https://arxiv.org/abs/2609.34738
摘要:On-policy distillation (OPD) transfers knowledge between language models through teacher supervision on student-generated trajectories. With different tokenizers, a single teacher token may require multiple student tokens to generate, creating intermediate states where the event is entered but not yet completed. Existing cross-tokenizer methods align tokens or text spans to construct comparable prediction targets. We study a complementary problem after partial generation: once the student produces a prefix of a teacher token, multiple next tokens may complete the same remaining bytes, but the teacher only specifies the required completion rather than how probability should be divided among these valid continuations. We introduce Event-Set Completion Distillation (ESCD), which complements c
标题:Evidence-Aligned Multimodal On-Policy Self-Distillation for Fine-Grained Visual Understanding
来源:arXiv
链接:https://arxiv.org/abs/2609.34672
摘要:Fine-grained visual understanding requires models to recognize small details within complex images. Multimodal on-policy self-distillation (OPSD) addresses this challenge by using a teacher conditioned on evidence-centered crops to supervise a student conditioned on original images along student-generated trajectories. Ideally, teacher corrections, the distributional changes from the student toward the privileged teacher, should be driven by task-relevant visual evidence. However, the designs that make the teacher effective also introduce other interference. Using a lagged or frozen teacher improves training stability but introduces a model-state gap from the evolving student, while cropping enhances task-relevant evidence but also loses the visual context. These two sources of interferenc
标题:PMOPD: Task Ordering, Cycling, and Parameter-Update Subspace Protection in Multi-Teacher On-Policy Distillation
来源:arXiv
链接:https://arxiv.org/abs/2609.34605
摘要:Multi-teacher on-policy distillation (MOPD) has emerged as a popular post-training paradigm for integrating specialized capabilities in frontier language models. Existing OPD research has primarily focused on optimizing single-task distillation through objective design, distillation scope, and teacher signal construction, whereas MOPD must aggregate multiple capabilities in shared parameters and address the resulting capability seesaw, in which improving one domain suppresses capabilities acquired from another. Inspired by the distinctive update geometry of OPD, we find that parameter updates from different tasks rapidly concentrate in their respective low-dimensional subspaces during MOPD, providing a direct geometric basis for identifying and controlling cross-task interference. We there
标题:EOPSA: Efficient On-Policy Self-Distilled Safety Alignment
来源:arXiv
链接:https://arxiv.org/abs/2609.34519
摘要:On-Policy Self-Distillation (OPSD) has emerged as a promising paradigm for safety alignment, delivering dense, token-level supervision by distilling from a teacher conditioned on refusal-oriented privileged prompts. However, we reveal that this paradigm suffers from critical inefficiencies that degrade both training efficiency and general reasoning capabilities. Specifically, we diagnose two fundamental bottlenecks: (1) supervisory collapse over extended rollouts, where the teacher's corrective efficacy degrades precipitously as the student's generation prefix lengthens, injecting noisy gradients into late-stage tokens; and (2) gradient dilution from stylistic shifts, where the distillation objective is dominated by safety-irrelevant stylistic discrepancies induced by privileged prompting,
标题:Rubric-Aware On-Policy Self-Distillation for LLM Personalization
来源:arXiv
链接:https://arxiv.org/abs/2609.35262
摘要:LLM personalization aims to generate responses aligned with individual users' preferences and needs. User-specific rubrics make these expectations explicit, providing direct supervision on what a satisfactory answer should cover. Existing rubric-guided approaches, however, exploit such guidance only at a coarse granularity, either by using rubrics to supervise the prediction of relevant aspects for subsequent generation or by reducing aspect coverage to a single response-level reward for reinforcement learning. This leaves a gap between specifying what a personalized answer should contain and teaching the model how to generate it. To bridge this gap, we propose GRASP, a rubric-aware on-policy self-distillation framework for LLM personalization that turns user-specific rubric aspects into f
标题:TIDE: Teacher-Student Transition via Informative Distillation and Exploration for Agentic RL
来源:arXiv
链接:https://arxiv.org/abs/2609.35058
摘要:Effective multi-turn agents require interaction strategies that coordinate information gathering, actions, and feedback over long horizons. GRPO is a reinforcement learning algorithm used to train these agents, but sparse trajectory-level rewards limit early exploration in small models. Recent methods augment RL with on-policy distillation (OPD) from a stronger teacher. However, a fixed mixture assumes that teacher guidance and reward optimization should retain a constant relative role throughout training and across interaction turns. This assumption can fail at two scales. Globally, as training progresses, maintaining strong distillation pressure can constrain the model from moving beyond the teacher's capabilities. Locally, teacher--student disagreement identifies where the student depar
标题:An RL View of OPD: Least Square Policy Distillation for Sample-Efficient LLM Reasoning
来源:arXiv
链接:https://arxiv.org/abs/2609.35505
摘要:We study on-policy distillation (OPD) through the lens of reinforcement learning, establishing a connection between the reverse-KL objective in OPD and KL-regularized policy optimization. Building on this connection, we introduce Least-Square Policy Distillation (LSPD), an RL-inspired framework that brings optimistic exploration and off-policy data reuse from value-based RL into policy distillation. LSPD preserves policy diversity through exploration while improving rollout efficiency by repeatedly learning from previously collected trajectories. Our theoretical analysis connects LSPD to optimistic value-based learning and shows that its idealized formulation achieves a sharp $\tilde{\mathcal O}(\log K)$ regret bound under online exploration. Empirically, LSPD consistently outperforms exis
标题:Reward-Aligned Reweighting for On-Policy Distillation
来源:arXiv
链接:https://arxiv.org/abs/2609.35517
摘要:On-policy distillation (OPD) trains a student language model with dense feedback from a stronger teacher on student-generated trajectories. Yet standard OPD weights token-level distillation terms uniformly, implicitly treating local teacher preference as a proxy for correction utility. A decision's task value, however, depends on how the student completes the subsequent reasoning. This mismatch can cause imitation to suppress viable student strategies or reinforce paths the student cannot reliably execute. Verified trajectory outcomes provide complementary evidence about continuation quality, but do not directly identify the utility of individual decisions. We introduce Reward-Aligned Reweighting for On-Policy Distillation (R$^{2}$-OPD), which uses outcome agreement and the magnitude of te
标题:Verifier Errors in RLVR: Reward Hacking, Limits of Feedback, and Selective Control
来源:arXiv
链接:https://arxiv.org/abs/2609.35677
摘要:In reinforcement learning with verifiable rewards (RLVR), imperfect verifiers can reward incorrect responses, creating opportunities for reward hacking. Using gradient flow with a fixed verifier, we characterize the conditions under which reward rises while correctness falls. We then show that the observations available during RLVR are, in general, insufficient to detect or identify accepted errors, or to guarantee their reduction without sacrificing correct responses. To address this limit, we construct a correction using additional feedback about correctness from audits. This correction achieves \emph{selective control}: at the current policy, it lowers the probability of accepted errors and raises that of correct responses, provided it outweighs the pressure toward errors from verifier
标题:Verifiable Visual Rewards Transfer from Synthetic Scenes to Natural Prompts
来源:arXiv
链接:https://arxiv.org/abs/2609.35641
摘要:Precise instruction following in image generation, such as satisfying object counts and spatial relations, remains an open challenge at least in part because it is learned using unreliable reward models such as object detectors and vision-language models. We introduce Verifiable Visual Rewards (VVR), the first framework for programmatically verifiable image rewards, and show that training on it generalizes to natural prompts. Each VVR task is a scene of geometric objects and relations among them, from which we derive both the prompt and a deterministic verifier, so tasks can be generated in any number and at any chosen complexity. We release VVRBench, with 10,000 tasks over 32 constraint types, and VVRBench-Challenge, with 720 more complex tasks; the strongest model we evaluate---GPT-Image
标题:Quality Determines Direction, Length Shapes Magnitude: Length Control for Open-Ended Reinforcement Learning
来源:arXiv
链接:https://arxiv.org/abs/2609.34718
摘要:Reinforcement learning (RL) changes not only what language models say, but also how much they say, often increasing response length at the cost of token efficiency. Controlling this length growth is particularly challenging in open-ended RL because (i) response length is entangled with quality, (ii) open-ended tasks lack a natural success boundary for deciding when efficiency should be prioritized, and (iii) dense, graded rewards often yield small within-group quality margins, making quality-induced advantages especially sensitive to reward-level length shaping, which can perturb their magnitudes and even reverse their signs. We therefore adopt an asymmetric principle: quality should determine the direction of reinforcement, while length should only shape its magnitude. We instantiate this
Duplex-MPE:全双工多方对话交互基准Hugging Face Papers · 交互式评测 · 论文Duplex-MPE 通过无转录、无预设轮次边界的连续多人会话音频,评测全双工语音助手在多方交互中回答、沉默与停止发言的能力。
标题:AutoBCI: Forecast-Guided Agentic Neural Architecture Discovery for EEG-Based Brain--Computer Interfaces
来源:arXiv
链接:https://arxiv.org/abs/2609.35456
摘要:EEG-based brain-computer interfaces support a broad range of applications, yet designing decoding architectures that perform well across diverse tasks remains challenging. We introduce AutoBCI, an agentic framework in which a Designer Agent and a Forecaster Agent support the discovery and selection of EEG decoding architectures across tasks. The Designer Agent performs Pool-Guided Architecture Discovery (PGAD), generating and refining architectures through training and validation across multiple EEG tasks, such as emotion recognition, motor imagery, and sleep staging. The Forecaster Agent performs Performance Estimation from Early Knowledge (PEEK), using architecture code, the training protocol, and early learning curves to predict full-budget validation performance and select promising ca
无视觉编码器多模态预训练的缩放定律Hugging Face Papers · 缩放定律 · 论文研究揭示无视觉编码器多模态模型虽在小规模下落后,但预计在约 10^22 FLOPs 时追平编码器架构,并由语言模型逐步接管视觉表征功能。
[通过自适应循环 Transformer 改进测试时扩展(46 ▲)](https://huggingface.co/papers/2609.35748?utm_source=digest-papers&utm_medium=email&utm_campaign=2026-09-29)
[合并自 2026-09-30 tah2-transformer]
标题:Improving Test-Time Scaling with Adaptive Looped Transformers
来源:arXiv
链接:https://arxiv.org/abs/2609.35748
摘要:Looped transformers have demonstrated promising parameter efficiency by reusing layers for latent computation. Prior studies compare looped and non-looped models at matched parameters or per-token FLOPs. However, to the best of our knowledge, whether looping improves test-time scaling as outputs grow longer remains underexplored. Through post-training looped transformers, we study the accuracy-compute slope, measured as the accuracy gain per doubling of test-time decoding FLOPs. We find that existing looped transformers often yield steeper slopes than their non-looped baseline, yet underperform it at matched compute. While fixed-depth looping spends extra iterations on every token, our analysis shows that many tokens do not benefit from extra iterations. We therefore propose TaH2, which en
标题:Token-Disentangled Latent Test-Time Scaling for Vision-Language Reasoning
来源:arXiv
链接:https://arxiv.org/abs/2609.35228
摘要:Latent test-time scaling improves reasoning by refining hidden states during inference, but existing methods typically apply a single scalar reward to all editable latent tokens. For multimodal large language models, this global update ignores that generated tokens play different roles: some are sensitive to visual evidence, while others correspond to uncertain reasoning decisions. We present Token-Disentangled Latent Test-Time Scaling, an inference-time framework that makes latent refinement token-role-aware. Starting from an initial generated trajectory, we optimize a short hidden-state prefix while routing perception-side visual feedback to image-sensitive tokens and reasoning feedback to high-entropy tokens. Tokens selected by neither route are constrained by an anchor regularizer. Acr
UMM-Reflection:交错强化学习赋予统一模型原生反思能力Hugging Face Papers · 自我反思 · 论文UMM-Reflection通过轨迹级交错强化学习联合优化统一多模态模型的文本反思与图像修订能力,使模型无需外部验证器即可迭代诊断并修复自身生成结果。
[通过交错强化学习,在统一模型中学习原生反思能力(36 ▲)](https://huggingface.co/papers/2609.35767?utm_source=digest-papers&utm_medium=email&utm_campaign=2026-09-29)
[合并自 2026-09-30 umm-reflection]
标题:Learning Native Reflection in Unified Models with Interleaved Reinforcement Learning
来源:arXiv
链接:https://arxiv.org/abs/2609.35767
摘要:Unified multimodal models can both look at and render images, so in principle they can repair their own generations: diagnose what an image gets wrong, revise it, observe the result, and diagnose again. Whether a revision helps is known only after it is rendered, so the reflection text and the image generation must be learned jointly, over the whole loop. Supervised fine-tuning (SFT) on reflection trajectories gives a cold start but does not find the high-success repair paths, and naive RL that optimizes only the renderer or only one head leaves most of the gain untapped. We introduce UMM-Reflection, which applies reinforcement learning (RL) to complete reflection trajectories inside one unified model: sibling trajectories share one initial image, so the group-relative advantage compares r
标题:AgentPerfBench: A Benchmarking and Evaluation Suite for Inference Performance of Agentic LLMs
来源:arXiv
链接:https://arxiv.org/abs/2609.34683
摘要:The optimization of LLM serving engines, such as vLLM and SGLang, is largely benchmark-driven: optimizations, scheduling policies, hardware and system designs are all selected based on representative workloads. However, a significant mismatch has emerged in the agentic era. Existing benchmarks primarily focus on simple single-turn chatbot workloads. LLM applications are increasingly agentic: coding agents, terminal execution systems, and tool-use agents issue multi-turn requests with growing context lengths. We introduce AgentPerfBench, a benchmark suite for agentic inference. It uses real traces from agentic benchmarks, such as SWE-Bench and TerminalBench, alongside standard chat baselines. This enables benchmarking of models on multi-turn tasks involving tool calling, skill utilization,
OpenAI 升级 Codex Cloud,Agents API 支持 computer useAIHOT · 人工智能辅助编程 · 原文OpenAI 升级 Codex Cloud,使编码智能体可在预配置、可复用的云环境中跨设备持续执行任务,并预览支持 computer use 的 Agents API。
标题:ReMCTS: Reflection-Enhanced Monte Carlo Tree Search for Code Generation
来源:arXiv
链接:https://arxiv.org/abs/2609.34717
摘要:Open-weight large language models (LLMs) can generate function-level programs from natural-language prompts, but plausible candidates still fail on hidden semantics and repeat mistakes across repair attempts. We present ReMCTS, an execution-grounded, memory-augmented, LLM-guided MCTS-style search framework. It organizes program candidates as tree states, retains branch-local debugging context, retrieves failure experience across branches, and distinguishes failed checks from unavailable evidence. On HumanEval and MBPP-Sanitized, visible-test ReMCTS improves over direct generation in 8 of 10 model-dataset pairs under held-out evaluation, whereas proxy-only search is less stable. Controlled tree-search, sampling, repair, and memory ablations characterize the source and limits of these gains.
标题:LLMs are General Asynchronous Agents
来源:arXiv
链接:https://arxiv.org/abs/2609.35427
摘要:Modern LLMs are increasingly capable as autonomous agents, but they follow sequential interaction cycles: read, think, reply or call tools, repeat. Many real-world use cases are not sequential: voice assistants, embodied agents, and monitoring systems receive new inputs while they think or perform another task. Modern LLMs address this with specialized architectures for voice interaction and video streams, VLAs for robot control, asynchronous tool calling for API usage, and others. In this work, we generalize from different asynchronous tasks to general asynchronous agents that can adapt to different types of concurrency. To achieve this, we develop an asynchronous LLM framework that lets users (or the agents themselves) define inference coroutines with overlapping memory states. We showca
同专题其余 2 条
迈向可靠的 AI 数据科学家:工作流脚手架驱动的数据智能体arXiv · 智能体框架 · 论文 · 代码论文从工作流脚手架视角系统梳理数据智能体的感知、规划、执行、验证与修复流程,并指出导致静默失败的四类关键可靠性缺口。
标题:Towards Reliable AI Data Scientists: Data Agents with Workflow Harnesses
来源:arXiv
链接:https://arxiv.org/abs/2609.35255
摘要:Large language model agents are increasingly deployed for data-intensive work, yet reliable data analysis requires more than general-purpose reasoning and ad hoc tool augmentation. Data Agents, equipped with workflow harnesses, offer a promising paradigm for automating the end-to-end data science lifecycle. This paper examines Data Agents from a harness-centric perspective. First, we introduce a taxonomy of Data Agents and associated data environments, organizing the literature around five functional stages: perception, planning, execution, verification, and repair. Second, we analyze the key technical routes within each stage, identifying 15 distinct approaches ranging from data structure probing to data state reconstruction. Third, we identify four open reliability problems: inactive sem
标题:AgentWare: Automating the Lifecycle of Agentic Applications across the Edge-to-Cloud Continuum
来源:arXiv
链接:https://arxiv.org/abs/2609.34586
摘要:Deploying LLM-enabled agentic applications across the Edge-to-Cloud continuum remains challenging due to hardware heterogeneity, deployment complexity, limited observability, and the lack of systematic evaluation methods. Existing solutions address agent development, observability, or benchmarking separately, offering limited support for the full lifecycle of distributed agentic applications. This paper presents AgentWare, an AgenticOps framework that automates the provisioning, deployment, observability, and evaluation of agentic applications across Edge-to-Cloud infrastructures. AgentWare introduces an end-to-end lifecycle pipeline that automatically prepares heterogeneous execution environments, transforms user-defined agent implementations into distributed applications, deploys agent c
专题:可信执行 6 条
英伟达发布 AI 智能体安全工具 OpenShell
英伟达推出 OpenShell,利用 CPU 硬件能力和约束检测机制隔离 AI 智能体,防止其绕过限制、逃逸沙箱或执行危险操作。
标题:The Compiler May Read It, the Agent May Not: Keeping Part of a Research Code Away from a Coding Agent
来源:arXiv
链接:https://arxiv.org/abs/2609.35557
摘要:The compiler must read modules a physics-based solver cannot build without; the coding agent must not read that intellectual property. The harness does not ship that rule. We classified fifteen read routes against a container, permission rules and a sandbox. None of the three can tell which program is reading.
标题:When Valid Tool Calls Change Meaning: Formation-Consistent Dispatch for LLM Agents
来源:arXiv
链接:https://arxiv.org/abs/2609.35088
摘要:Tool-enabled agents form calls from model-visible interfaces, while hosts later select their implementation. Standard dispatch omits the descriptor-handler relation. An unchanged and schema-valid call can therefore acquire a different security effect during rollout, reconnect, or delayed approval. We call this failure schema-epoch drift. We present formation-consistent dispatch (FCD), which connects implementation analysis to execution authority. Reviewed profiles produce provenance-bound over-approximations of declared in-scope effects from official source. Under a closed-target approval policy, a verifier applies each formed call to a summary and captures a successor only when its effects fit the call's security contract. Atomic admission and a final-hop fence preserve this decision to t
Veer:欺骗性界面下 Web 智能体的前瞻状态干预arXiv · 可信执行 · 论文Veer 在 Web 智能体执行动作前主动将任务相关网页状态调整至安全状态,以阻止欺骗性界面诱发未经授权的后果。
论文指出,即使 Web 智能体选择的动作本身符合任务要求,当前网页状态仍可能使其产生违背用户利益的未授权后果。Veer 保留基础智能体的任务规划,仅在拟议动作可能导致危险结果时,将任务相关 Web 状态作为运行时控制对象。系统会先构造通向安全状态的前瞻干预轨迹,再结合运行时定位与验证修改真实环境。在 TrickyArena 和 WebDecept 上,Veer 在全部三种设置中取得最高安全任务完成率,并将 WebDecept 的暗黑模式成功率降至 0.3%。
用途与启示
将防御对象从智能体动作扩展到环境状态,可覆盖“动作合法但状态导致后果越权”的失效模式。
前瞻轨迹、运行时定位和执行验证可组成 Web 智能体的主动安全控制层,而无需替换基础规划器。
消融结果表明主动状态干预贡献最大,可作为欺骗性界面防御和 GUI 智能体安全设计的重点方向。
📝 原始笔记(逐字保留)
标题:Before Acting, Change the State: Prospective State Intervention for Web Agents under Deceptive Interfaces
来源:arXiv
链接:https://arxiv.org/abs/2609.34974
摘要:LLM-based Web agents can autonomously complete user tasks, yet deceptive interfaces can steer them toward outcomes that conflict with users' interests. Existing defenses primarily intervene on agent behavior through blocking, guidance, or replanning. We identify a distinct failure mode: a task-valid action can still realize an unauthorized consequence because of the current Web state. This motivates treating task-relevant Web state itself as a runtime control target. We introduce Veer, an agent-side runtime defense that leaves task planning to the base agent and intervenes on Web state when a proposed action would produce an unauthorized consequence. Before modifying the live environment, Veer constructs a prospective intervention trajectory toward a safe task-relevant state and executes i
Fireworks AI 发布缓存感知路由器 FireRouter with OpusAIbase · 模型路由 · 原文FireRouter with Opus 通过结合任务适配度、模型成本与提示缓存状态动态路由编码请求,在内部 A/B 测试中以准确率下降1.5个百分点为代价将会话成本降低57%。
Fireworks AI 推出针对 Claude Opus 系列优化的缓存感知路由器 FireRouter with Opus,并以独立路由模型形式提供 serverless 端点。系统会在每轮交互中评估候选模型的任务适配度与处理成本,同时判断切换模型带来的节省能否抵消提示缓存损失。一个多月的内部编码流量 A/B 测试显示,平均会话成本由15.36美元降至6.63美元,评分准确率由80.2%降至78.7%,缓存命中率由97.8%降至94.2%。当前路由池包含 Claude Opus 5.5、GLM 5.3 和 GLM 5.3 Flash,并已接入 Claude Code、Codex 与 Cursor IDE 等工具链。
用途与启示
展示提示缓存可作为模型路由决策的重要变量,不能只比较单次调用价格与模型能力。
为编码智能体控制推理成本提供实践方案:将常规轮次分流至低成本模型,把高难度任务保留给旗舰模型。
A/B 测试中的置信区间提示,评估动态路由系统时应同时报告成本、准确率、缓存命中率及统计不确定性。
独立 serverless 路由端点有助于在不大幅改造现有开发工具链的情况下部署多模型策略。
📝 原始笔记(逐字保留)
标题:Fireworks AI 推出 FireRouter with Opus:编码任务成本降 57%,准确率仅损失 1.5 个百分点
来源:AIbase 快讯(aibase.com)
链接:https://www.aibase.com/zh/news/31406
摘要:Fireworks AI 发布 FireRouter with Opus,称其为市场首个缓存感知路由器,针对 Claude Opus 系列优化,并首次以独立路由模型形式作为 serverless 端点开放。经一个多月内部 A/B 测试,其执行编码任务准确率达单独使用 Opus 的 98.1%,成本降低 57%。核心逻辑是在每次用户轮次中,评估模型集合中每个模型对当前任务的适合程度,并估算各模型处理成本。
标题:SeLMRoute: Probabilistic Semantic Evidence for Large Language Model Routing
来源:arXiv
链接:https://arxiv.org/abs/2609.34736
摘要:Large language model (LLM) routing aims to select the most suitable model for each incoming query. Most existing routers learn this decision directly from query embeddings, model representations, preference data, or clusters of similar examples. Such approaches can be effective, yet the representation used for routing rarely states what a query actually requires. We introduce SeLMRoute, a routing framework that separates the extraction of candidate-independent semantic evidence from the learning of candidate performance and the application of deployment objectives. A decision model first evaluates a set of interpretable questions about the query, such as its reasoning requirements and use of external knowledge, with each judgment retained as a probability distribution. The resulting probab
标题:TokenCast: Forecasting Token Consumption During LLM Agent Execution
来源:arXiv
链接:https://arxiv.org/abs/2609.35760
摘要:When a large language model (LLM) agent executes the same task, token consumption can vary by over an order of magnitude across runs. The agent chooses its next steps based on tool feedback and intermediate results, while the growing context steadily inflates the input size of every subsequent call. The total consumption of a task is therefore hard to predict before execution and the prediction must be revised as the run unfolds. In this paper, we propose TokenCast, which learns a composable cost representation for each execution segment, recording its own consumption and the context growth it introduces. Composing adjacent segments yields a cumulative estimate that captures the extra input cost incurred when context from earlier segments is re-read by every later call. As execution unfold
PEARL:面向智能体强化学习的弹性 Prefill-Decode 执行系统
PEARL 通过协调弹性 GPU 资源、训练 GPU 临时复用与自适应 Prefill-Decode 配置,显著提升多轮智能体强化学习 rollout 的吞吐量。
标题:PEARL: Adaptive Prefill-Decode Execution with Elasticity for Agentic Reinforcement Learning
来源:arXiv
链接:https://arxiv.org/abs/2609.35158
摘要:Multi-turn rollout dominates the cost of agentic reinforcement learning (RL). Asynchronous execution and elastic GPU resources can accelerate this stage, but adding rollout replicas yields diminishing returns while training GPUs remain idle between updates. We observe that effective resource use also depends on the prefill--decode (PD) configuration. Both the choice between colocation and disaggregation and the optimal PD ratio vary with the workload, making resource scaling and PD configuration interdependent. Exploiting this opportunity requires selecting effective configurations and realizing their benefits within transient resource-availability windows despite reconfiguration costs. We present PEARL, an asynchronous agentic RL system that coordinates external resource elasticity, tempo
Elastic Model Unit 与全局转换图可为分布式训练状态复用、GPU 迁移编排和弹性运行时设计提供参考。
结果表明,大规模后训练系统不仅需要数据并行扩缩容,还应联合优化张量并行与流水线并行布局。
📝 原始笔记(逐字保留)
标题:Nereus: Adaptive Parallelism for LLM Post-Training
来源:arXiv
链接:https://arxiv.org/abs/2609.34645
摘要:Reinforcement learning (RL) post-training for large language models (LLMs) coordinates multiple models across generation, inference, and training on GPU clusters. Several factors may change during a run, including resource availability, sequence length, memory pressure, and stage bottlenecks. As a consequence, an execution plan that was initially suitable can then become slow or even infeasible over time. However, adapting a job whose models share GPUs entails significant challenges: deciding whether a new plan is worth the transition cost, reusing the job's distributed state, and coordinating GPU transfers across models and stages. Nereus targets these challenges as a cost-aware runtime that adapts RL post-training jobs into efficient execution plans. Its low-overhead controller selects a
标题:KV-streams for Efficient Compaction in Agentic Reinforcement Learning
来源:arXiv
链接:https://arxiv.org/abs/2609.35750
摘要:Scaling the horizon of agentic LLMs is bottlenecked by the need to fit ever longer context traces in GPU memory. Context compaction has been the most popular mechanism to alleviate this issue, keeping GPU memory constant for a given trace. Unfortunately, most compaction strategies rely on prefilling the LLM context many times over, hindering training throughput. To alleviate this bottleneck and enable efficient trainable compaction, we propose KV-streams, a plug-and-play strategy compatible with any compaction strategy that substantially increases throughput while showing no evidence of hindering performance. KV-streams enable scalable compaction by streaming the KV cache forward rather than flushing it after each compaction. We show that KV-streams enable three different compaction strate
标题:Scalable In-Context Reinforcement Learning with Recurrent Algorithm Distillation
来源:arXiv
链接:https://arxiv.org/abs/2609.35333
摘要:Algorithm Distillation (AD) has demonstrated the remarkable ability of Transformers to perform in-context reinforcement learning without explicit weight updates. However, capturing long-term learning progress necessitates expansive context windows, which incur prohibitive memory costs and limit scalability in complex, long-horizon tasks. To address this bottleneck, we propose Recurrent Algorithm Distillation (RAD). RAD employs a dual-component architecture: a Compression Transformer that distills extended interaction histories into compact latent tokens, and an AD Transformer that auto-regressively generates actions using a hybrid context of these compressed memories and recent transitions. By maintaining a fixed-size latent buffer, RAD decouples the effective history length from computati
标题:Towards Communication-Efficient Social Intelligence in Language Agents
来源:arXiv
链接:https://arxiv.org/abs/2609.35749
摘要:Socially intelligent language agents must negotiate, coordinate, and resolve conflicting preferences while respecting the time and attention of both participants. Balancing these demands is challenging because agents must convey enough to address a partner's constraints and advance their goals without adding words that do not help the interaction. In this paper, we propose Teacher-Assisted Communication Training (TACT) to improve social goal attainment while reducing communication cost, making interactions with agents more productive and less demanding. We first characterize communication efficiency in terms of action strategy and expression, whose effects extend beyond the current utterance to the partner's response and subsequent exchanges. We design TACT to revise student-generated acti
论文将蒸馏攻击的威胁模型扩展为“蒸馏后继续强化学习”,指出现有防御通常忽略了攻击者的后续训练能力。实验表明,一些蒸馏完成后看似有效的防御,会在强化学习阶段被轻易突破。攻击者仅使用当前 API 容易获取的数据和简单攻击方式,也能取得接近提取完整隐藏推理轨迹的能力增益。作者认为,只要防御机制泄露的信息足以近似重建推理轨迹,就很可能无法有效阻止模型能力窃取,并建议探索批次级蒸馏防御。
用途与启示
评估蒸馏防御时,应把蒸馏后的强化学习纳入完整攻击链,而非只测量即时防御效果。
防御设计不能仅隐藏完整思维链,还需限制跨样本信息聚合后对近似推理轨迹的重建。
可重点研究批次级检测、查询模式审计和信息泄露预算,以提高模型窃取成本。
安全基准应测试防御在攻击者持续训练和策略适应条件下的长期稳健性。
📝 原始笔记(逐字保留)
标题:Distillation Defenses Easily Break After Reinforcement Learning
来源:arXiv
链接:https://arxiv.org/abs/2609.35699
摘要:Distillation attacks copy the reasoning capabilities of closed-source large language models, allowing bad actors to replicate state-of-the-art performance at low cost. Attackers systematically collect a large volume of frontier model reasoning traces and then train (i.e., "distill") their own models on these traces. Existing defenses against distillation attacks are typically evaluated immediately after distillation, implicitly assuming attackers do not train their models any further. In this paper, we argue that a more realistic threat model includes further training with reinforcement learning after distillation. A misspecified threat model can give a false sense of security -- some defenses that seem effective after distillation can be broken after subsequent reinforcement learning. Pra
标题:Reasoning with Continuous Latent Diffusion
来源:arXiv
链接:https://arxiv.org/abs/2609.35694
摘要:Continuous diffusion generates complete reasoning solutions through iterative refinement in latent space. We introduce Latent Flow Reasoning Models (LFRMs), an ELF-based training and inference recipe. Our experiments show that accurate decoding alone does not ensure strong reasoning performance. We therefore learn compact representations from multiple layers of a strong autoregressive teacher. Their decomposition also enables asynchronous denoising at different rates. We show that prompt encodings need only preserve the information required for the correct text-conditional score, rather than exactly match teacher features, and use a staged curriculum to learn a compact prompt encoder that replaces the teacher Transformer at inference. We adapt DiffusionNFT to learned self-conditioning guid
标题:Report: Progressive Disclosure of Agent Skills
来源:arXiv
链接:https://arxiv.org/abs/2609.35692
摘要:Users of Workday's deployed LLM-based agents often request features which can be addressed by defining named procedures, also known as skills, in the LLM context, effectively augmenting agents' capabilities. However, as an agent's skills library grows in size, so does the agent's operational cost. Progressive disclosure (lazy-loading) of skills as needed may reduce operational costs, but its impact on overall latency and skill-retrieval quality remains unclear. In this report, we investigate the impact empirically and find that progressive disclosure improves skill-retrieval quality but marginally degrades overall latency.
标题:QuanReview: Offline, Auditable Reconciliation of Human and LLM Span Annotations
来源:arXiv
链接:https://arxiv.org/abs/2609.35685
摘要:Structured span annotations, such as quantities with their units, uncertainty modifiers, and event classes, are expensive to create and hard to keep trustworthy once language models enter the loop. We present QuanReview, an open-source system for auditing and correcting such annotation layers. QuanReview aligns two annotation streams over the same documents at character level, resolves unambiguous cases by an explicit and logged policy, and routes candidate conflicts to a browser-based adjudication interface where reviewers accept either side, build field-level hybrids, or flag items for re-annotation. A campaign manager assigns documents to multiple annotators with configurable redundancy, computes agreement at document and span level, auto-merges unanimous documents, and exports the corr
标题:Not All Rollouts Are Worth Learning: On Trajectory Valuation for Post-Training Reinforcement Learning
来源:arXiv
链接:https://arxiv.org/abs/2609.35072
摘要:We consider the problem of trajectory valuation in reinforcement learning: how to identify and mitigate detrimental trajectories during online training. Unlike classification, where data valuation relies on fixed training and validation sets, reinforcement learning involves dynamically generated trajectories without explicit validation signals, making conventional influence-based methods inapplicable. We propose Dynamic Trajectory Valuation (DTV), a simple and efficient framework that estimates trajectory utility at the mini-batch level and filters detrimental trajectories based solely on gradient information. By operating at the optimization level, DTV integrates seamlessly with existing reinforcement learning pipelines with minimal overhead. Extensive experiments across diverse settings,
标题:QC-Stark: A Multi-Task Benchmark Revealing Capability Dissociations in LLMs Evaluated on Quantum Computing Tasks
来源:arXiv
链接:https://arxiv.org/abs/2609.35581
摘要:We introduce QC-Stark, a benchmark for evaluating large language models (LLMs) on 11 quantum computing (QC) tasks, spanning circuit construction, debugging, compilation, error correction, and simulation. Across 2,750 evaluations (10 models $\times$ 11 tasks x 5 difficulty levels x 5 seeds), we find that overall rankings mask substantial per-task variation. The Spearman correlation between overall and per-task rankings is statistically insignificant for 4 out of the 11 tasks included in this benchmark. A 2-parameter Item Response Theory (IRT) model validates measurement quality, and prompt sensitivity analysis confirms ranking robustness across prompt conditions. All tasks are auto-verifiable via execution, thus not requiring any manual evaluation. We make the code and data publicly availab
标题:CoHuB: A Simulation Benchmark for Multi-Humanoid Collaboration
来源:arXiv
链接:https://arxiv.org/abs/2609.34782
摘要:Many physical tasks in human environments require collaboration, from assisting a partner to jointly manipulating an object. Yet, existing humanoid benchmarks largely focus on single-humanoid skills and lack evaluation of multi-humanoid collaboration under egocentric visual observations. We introduce CoHuB (Collaborative Multi-Humanoid Benchmark), a simulation benchmark for multi-humanoid collaboration under egocentric visual observations. CoHuB provides 10 tasks, eight with two humanoids and two with three humanoids, spanning diverse collaboration patterns. We also provide synchronized demonstrations collected through a multi-operator VR teleoperation pipeline, in which each operator controls one humanoid from its egocentric view. Experiments with representative visuomotor policies reveal
同专题其余 1 条
AI 加密货币交易:从回测到真实市场的性能差距arXiv · 评测基准 · 论文 · 代码该研究构建覆盖历史回测、前瞻性模拟盘和实盘交易的统一基准,用于量化不同 AI 加密货币交易方法从回测到真实市场的性能落差。
标题:Can AI Make Money in Crypto? Measuring the Gap from Backtests to Real Markets
来源:arXiv
链接:https://arxiv.org/abs/2609.34510
摘要:AI-based trading methods have rapidly evolved from machine learning and reinforcement learning to large language models (LLMs) and trading agents, yet their performance is still predominantly assessed through historical backtesting. Such evaluations provide limited evidence of whether a method can generalize to unseen future markets or whether its backtested performance can be sustained in realistic trading frictions (e.g., latency, slippage, liquidity constraints, and market impact). We present a unified benchmark that evaluates representative machine learning, reinforcement learning, LLM-based, and agent-based trading methods in cryptocurrency markets through three progressively more realistic stages: historical backtesting, prospective exchange-based paper trading, and real-money live t
标题:Representation Alignment as a Bottleneck in LLM-Based Retrosynthesis Planning
来源:arXiv
链接:https://arxiv.org/abs/2609.35571
摘要:While LLMs show promise in general reasoning, symbolic planning in chemistry remains a bottleneck. Direct ''SMILES-to-PDDL'' attempts fail because they force models to juggle chemical analysis and planning-language structuring simultaneously. We hypothesize that this failure stems from a lack of intermediate abstractions rather than insufficient model capacity. By decomposing retrosynthesis into molecule mapping, reaction mapping, and PDDL generation, we achieve high success rates where end-to-end approaches fail. This provides evidence that a primary bottleneck lies in representation alignment rather than raw model capacity. Our structural analysis demonstrates that intermediate representations are essential in retrosynthesis planning, highlighting the importance of representation-centric
标题:Optimal Networks for Agentic Information Aggregation
来源:arXiv
链接:https://arxiv.org/abs/2609.35537
摘要:We study information aggregation in the networked learning model introduced by Kearns, Roth, and Ryu (SODA 2026). There is a fixed distribution over $d$ features and a common label. Agents learn in topological order on a directed acyclic graph. Each observes a subset of the features and its parents' predictions, fits a linear predictor to minimize mean squared error, and passes only its prediction forward. The global predictor is the best linear predictor using all features. Kearns, Roth, and Ryu show that the output agent's error approaches the global predictor's error along sufficiently deep paths with suitable feature coverage, while insufficient depth can prevent aggregation even in large networks. In contrast to their main focus on a given graph and feature allocation, we consider the
标题:Attention-based Hierarchical Variational Information Bottleneck for Robust Multi-Agent Communication under Variable Bandwidth
来源:arXiv
链接:https://arxiv.org/abs/2609.34860
摘要:Learning-based multi-agent communication under limited bandwidth does not only require deciding what to communicate, but also structuring messages so that partial transmissions remain useful. We study this problem under prefix truncation, where only the first part of each message is received. To address it, we propose \textbf{AH-VIB}, an attention-based autoregressive variational communication model that combines a variational information bottleneck (VIB) with sequential message generation and a hierarchical robustness loss. We evaluate AH-VIB on a custom cooperative object-inspection and occupancy-mapping task, where agents equipped with a limited field-of-view sensor coordinate to scan inspection objects in an occupancy-grid world, under variable and fixed bandwidth conditions, and compa
标题:Why Deterministic PRM Guidance Underperforms in Discrete Diffusion Reasoning
来源:arXiv
链接:https://arxiv.org/abs/2609.35472
摘要:Discrete diffusion language models (dLLMs) expose a denoised solution at every step, which makes process reward model (PRM) guidance look like a way to spend compute at test time. We show that once denoising, PRM scoring, and outcome reward model (ORM) scoring are charged in the same budget of forward passes, its deterministic form loses to a much simpler baseline. Our PRMs score intermediate denoising states and are trained on the correctness of the final answer. On Dream-v0-Instruct-7B with 8 candidates per GSM8K problem, keeping the candidate with the highest PRM score at every scoring step reaches 65.18%, while independent sampling plus an ORM reranker trained for the task reaches 75.13%. The gap grows to 12.69 percentage points (pp) with 32 candidates, and is 9.85 pp on MATH and 12.16
标题:Persistent Partners Raise Prices Among Learning Agents
来源:arXiv
链接:https://arxiv.org/abs/2609.35402
摘要:When pricing agents meet repeatedly on a platform, the platform decides who faces whom. We ask whether that choice moves the prices the agents learn, and whether a rise comes with learned punishment. In a pre-registered randomised experiment in the Bertrand duopoly of Calvano et al., each agent's price is set by a tabular Q-learning module, not by the small language model attached to it, and we randomise whether each agent keeps its partner, sees its rival's prices and can send messages. Keeping the same partner raises the level of profits, averaged over training, by 0.27 of the gap between competitive and monopoly profit (95% CI 0.20 to 0.35, all twenty paired runs positive), our registered primary result, and the resting price by 0.17 of the Nash-to-monopoly range (post hoc). A plain tab
标题:From Preference to Reciprocity: Decentralized Matching with Empirically Grounded LLM-agent Based Modeling
来源:arXiv
链接:https://arxiv.org/abs/2609.34679
摘要:Bipartite matching is a fundamental problem in game theory and market design. Classical approaches such as Gale--Shapley assume complete preferences and centralized computation, whereas many real-world matching processes are decentralized, asynchronous, and shaped by sequential interaction under limited information. We propose a dynamic bipartite matching framework that combines large language model (LLM) agents with contextual bandits. In a simulated Chinese marriage market, economically grounded LLM agents evaluate locally encountered candidates, while agent-specific Logistic-UCB models learn reciprocal acceptance from realized proposal outcomes. The mechanism therefore separates two decisions---\emph{whom do I like?} and \emph{who is likely to like me back?}---without requiring ex ante
标题:MCP Error Messages Written for Developers Hurt the Most Capable Agents Most
来源:arXiv
链接:https://arxiv.org/abs/2609.35381
摘要:Many Model Context Protocol (MCP) servers wrap web APIs built for human developers, and their error messages tell the reader to run a command, edit a configuration, open a web page or wait. Many agents that read them can only call the server's tools. In 150 widely used MCP servers, 949 of 3,001 error messages tell the caller what to do next, and half of these steps depend on something the server cannot see about the caller. On credential errors, 62 of 67 steps ask for a terminal command, a configuration change or a web page; on rate limits, 20 of 30 say to wait and retry without naming the call to repeat. We tested five OpenAI models that act only through the tools of Berkeley Function Calling Leaderboard tasks, and the agents did what the step said. On expired credentials, a terminal comm
标题:One Readout, Many Repairs: Diffusion-Guided Hierarchical Search for Tool-Agent Repair
来源:arXiv
链接:https://arxiv.org/abs/2609.34879
摘要:Tool agents use large language models to act through external tools, yet successfully executed calls can still leave user requests unfulfilled. Tool-agent repair seeks alternative call sequences that execute successfully and fulfill the original requests. However, repair requires exploring both operation choices and their concrete realizations, making complete-sequence regeneration costly. Moreover, regeneration repeats operation selection even when failure arises from how those operations are realized. The resulting challenge is to reduce this repetition while preserving exploration of alternative operations and realizations. Therefore, we formulate repair as hierarchical search over operation supports, which we introduce as sets of permitted operation types that define reusable search re
标题:PhoneCLI: From App Interfaces to Callable Commands for Mobile Agents
来源:arXiv
链接:https://arxiv.org/abs/2609.35671
摘要:Mobile GUI agents operate through a perception--action loop: at each step they screenshot the device, invoke a vision--language model (VLM), and emit an action. It is slow, costly, and brittle, yet most of what it does is navigation---and everyday navigation is static, ordered, and endlessly repeated. We present PhoneCLI, which compiles an app's GUI navigation into callable commands, without any app-internal API, runtime instrumentation, or model training. Offline, PhoneCLI explores a target app from the outside and distills its screens, interactive elements, and navigation edges into a semantically annotated map; each screen yields one deterministic command: a replay sequence that reaches it. Online, the agent selects a command, verifies it before execution, and then executes it determini
标题:Planarian: Managing Agent State with Statepoints
来源:arXiv
链接:https://arxiv.org/abs/2609.35366
摘要:LLM agents solve complex tasks by iteratively changing files, invoking local tools, and interacting with remote services, which modifies state across their local environment and remote services. Today, agents and users must manage these changes explicitly, whether reverting exploratory actions or recovering from erroneous ones. Doing so safely requires coordinated actions, yet current agent harnesses lack unified abstractions and mechanisms for managing local and remote state consistently and efficiently. We describe Planarian, an agent runtime with state management that enables agents and users to recover from erroneous actions and explore alternative executions over consistent local and remote environment state. Planarian introduces the abstraction of agent statepoints, which are consist
标题:Jailbreaks for Black-Box Uncertainty Quantification in Large Reasoning Models
来源:arXiv
链接:https://arxiv.org/abs/2609.35350
摘要:While Large Reasoning Models (LRMs) excel at complex reasoning, alignment through reinforcement learning often induces systemic overconfidence. In production environments, where logits may be unavailable, robust black-box uncertainty quantification (UQ) is essential for trustworthiness and safety. Focusing on question-answering for LRMs, we show that existing black-box methods, such as paraphrase-based self-consistency and confidence verbalization, offer little to no improvement over simple repeated sampling, suggesting that alignment suppresses useful output variability. We introduce prompt-level relaxation operators that broaden the model's effective output distribution by approximating the effect of an optimal policy obtained with a stronger KL-regularization parameter, hence closer to
标题:Jev thinks "I don't know'', but doesn't say it: Introducing Sys1Cal-v1 Dataset for Probability Calibration
来源:arXiv
链接:https://arxiv.org/abs/2609.35342
摘要:The appearance of Jev marked the era of System One Models, foundation models that return structured decisions with probability distributions rather than text. Aside from low cost and great speed, Jev's central promise is that these probabilities are calibrated: such claim is not backed by any public test and available external benchmarks evaluate confidence calibration, not whether every returned option probability has the right numerical meaning. To tackle this issue, we introduce Sys1Cal-v1, a dataset of True/False questions about a proposition $A$ for which the exact probability $P(A)$ is known by construction. Each item is queried through the three Jev primitives - Noul, Choice and Score - and evaluated by total variation distance from the ground-truth distribution, which can be used t
标题:Beyond Verbalized Confidence: Calibrating Reasoners with Differentiable Readouts
来源:arXiv
链接:https://arxiv.org/abs/2609.34857
摘要:Reinforcement learning with verifiable rewards (RLVR) trains reasoning models to produce correct answers, but does not ensure that their stated confidence is calibrated. The resulting models are systematically overconfident. Recent methods train calibration inside the RLVR loop by having the model state a numerical confidence alongside its answer, but they all obtain the confidence by sampling it as text. This choice imposes two costs: a sampled confidence introduces variance and in practice collapses to a handful of distinct values, and sampling makes the confidence non-differentiable, forcing the calibration loss through a scalar reward. We propose CREDO (Confidence REaDOut) to replace sampling with a deterministic readout. While RLVR optimizes correctness, CREDO reads the confidence fro
标题:When Confidence Rises Too Early: Detecting Shortcut Reasoning via Premature Answer Commitment
来源:arXiv
链接:https://arxiv.org/abs/2609.35074
摘要:The reasoning trajectory of a Large Language Model (LLM) is often treated as a verbalized description of its internal reasoning. However, such trajectories can be unfaithful: a model may rely on shortcuts to reach an answer and then post-rationalize the decision with a seemingly coherent chain of thought. Detecting this shortcut reasoning is challenging because existing monitors and verifiers mainly inspect textual traces or final outcomes, rather than how the model's belief in its answer develops during generation. We introduce ConfLens, a framework that tracks the evolution of confidence in the final answer throughout reasoning. Across three shortcut reasoning settings, we observe a common pattern of premature confidence, where shortcut samples become highly confident in the final answer
标题:CarveMix-RC: Addressing Rare-Class Imbalance Through Lesion-Aware Synthetic Augmentation for Brain Metastasis Segmentation
来源:arXiv
链接:https://arxiv.org/abs/2609.35195
摘要:Accurate segmentation of post-treatment brain metastases is essential for treatment planning, longitudinal disease monitoring, and quantitative assessment of therapeutic response. The BraTS-MET 2026 Task 1 challenge introduces a clinically relevant segmentation problem involving four anatomically distinct tumor subregions: non-enhancing tumor core (NETC), surrounding non-enhancing FLAIR hyperintensity (SNFH), enhancing tumor (ET), and the resection cavity (RC). Among these, RC segmentation is particularly challenging because of its low prevalence, heterogeneous postoperative appearance, and lesion-wise evaluation protocol, leading conventional segmentation networks to prioritize dominant tumor classes during optimization. The proposed nnU-Net-based framework explicitly addresses RC segment
标题:ConRAG: Lightweight inference of multi-hop relations
来源:arXiv
链接:https://arxiv.org/abs/2609.35193
摘要:Understanding how two entities are connected often requires tracing multi-hop relations across documents to identify intermediate entities and supporting evidence that explain a connection. This is a task that appears frequently in scientific research and other knowledge-intensive analyses. We formalise this setting as multi-hop relation inference: given two known endpoint entities, we aim to recover the bridge entities and evidence-grounded reasoning chains that connect them across a document corpus, and to generate an explanation grounded in the retrieved evidence. Existing multi-hop RAG systems typically seek an unknown answer entity rather than explicitly recovering the connection between two known endpoints and graph-based approaches often rely on costly LLM-extracted knowledge graphs
标题:G$^3$-LoRA: Organizing Reward-Weighted Video Data with Gradient-Guided Grouped LoRA
来源:arXiv
链接:https://arxiv.org/abs/2609.35189
摘要:Post-training foundation video models on heterogeneous reward-weighted data usually assume that all data categories induce compatible updates. This assumption is fragile when categories correspond to different skills, domains, or evaluation dimensions. We study this problem in text-to-video post-training, where VBench2.0 dimensions define data buckets and an external multimodal reward pipeline assigns sample weights. We propose G$^3$-LoRA (Gradient-Guided Grouped LoRA), a data organization procedure that probes category-level gradients induced by reward-weighted video samples, removes the shared global update direction, clusters categories by residual gradient compatibility, trains group-specific LoRA experts, and consolidates them into one adapter by weight merging followed by on-policy d
标题:Small transformers track Bayesian evidence for latent common causes via a context-invariant mechanism
来源:arXiv
链接:https://arxiv.org/abs/2609.35161
摘要:We present an in-depth investigation of how a form of Bayesian reasoning about common causes can emerge as a cross-contextual generalization in small, tractable transformers. Incrementing on recent work, our set-up (i) disentangles causal mechanisms in the model from the causal structure of the true data-generating process, (ii) orients more towards natural language prediction by considering inference of latent common causes, and (iii) considers whether and how Bayesian evidence accumulation for latent common causes can be implemented in representations and mechanisms that allow for cross-context generalization to novel test cases.
标题:Tool Mediation Alters Refusal Mechanisms in Large Language Models
来源:arXiv
链接:https://arxiv.org/abs/2609.35117
摘要:Large language models (LLMs) are increasingly deployed with access to external tools, yet harmful tool-mediated interactions are less likely to be refused when compared to regular conversational ones. As this change in refusal behavior remains underexplored, we investigate its underlying mechanisms across a diverse set of open-weight language models. We find that information about the harmfulness of a request remains strongly encoded in the model's representations and transfers across conversational and tool-mediated inputs. Evidence from representation geometry and neuron-level analysis further indicates that the two interaction modes systematically distribute harm-related computation differently. Crucially, while conversational inputs can be refused at relatively low levels of perceived
标题:Using Context Is Not Enough: Test-Time Training for Personalized Reward Modeling
来源:arXiv
链接:https://arxiv.org/abs/2609.35109
摘要:Reinforcement learning from human feedback (RLHF) aligns large language models (LLMs) with human preferences, yet most pipelines learn a single reward model that overlooks individual differences in preferences. Personalized reward models (PRMs) address this by conditioning rewards on user-specific feedback, most commonly through in-context learning (ICL), where a user's historical comparisons are supplied as contextual preference pairs. However, we identify a key limitation of ICL-based PRMs: they fail to capture the preference relations conveyed by contextual pairs. To address this, we propose Preference-Aligned Test-Time Training (P-TTT), which explicitly encodes these relations into user-specific fast weights for personalized reward prediction. P-TTT introduces sequence-level update and
标题:Cross-Rollout Bellman Closure for Long-Horizon Agentic Reinforcement Learning
来源:arXiv
链接:https://arxiv.org/abs/2609.35082
摘要:Group-based reinforcement learning such as GRPO trains LLM agents by comparing rollouts sampled for each task, without a learned critic. In long-horizon settings, these rollouts revisit shared anchor states, offering cross-rollout evidence for step-level credit. Ideally, step-level credit should incorporate evidence beyond the realized suffixes observed at an anchor while aggregating alternative continuations according to their empirical frequencies. Visit-local averaging pools realized suffix returns at shared anchors and respects observed frequencies, but does not recursively propagate evidence across rollouts, whereas shortest-path estimators have global reach but allow a rarely observed route to dominate an anchor's value. We introduce Cross-Rollout Bellman Closure (CRBC), which merges
标题:Environmental requirements for the use of social information by artificial life agents using evolved plastic artificial neural networks
来源:arXiv
链接:https://arxiv.org/abs/2609.35018
摘要:Evolved Plastic Artificial Neural Networks (EPANNs) consist of two principal processes, the first, evolution, and the second, development and in-life learning. In the context of the origins of social = learning, very few studies have been carried out using ALIFE models based on EPANN requirements. Studies in this field have usually involved an imitative teacher/pupil relationship. This, however, ignores the possibility that the observed behaviour is a consequence of social information cues rather than direct imitation or teaching. Starting with the first of the EPANN processes (evolution), a series of experiments was undertaken using artificial neural network (ANN) based agents in a variety of foraging environments to examine under what minimal environmental conditions the use of social in
标题:MaPP: A Unified Marginalized Posterior-Predictive Framework for Data-Efficient RLVR
来源:arXiv
链接:https://arxiv.org/abs/2609.34990
摘要:Reinforcement learning with verifiable rewards (RLVR) improves the reasoning capabilities of large language models but incurs substantial costs from rollouts and policy updates. Online prompt selection improves efficiency by using per-prompt Bayesian posteriors to predict difficulty and prioritize informative prompts. However, existing methods overlook how reliably learning signals are extracted from sampled responses. In GRPO, a response's advantage depends on both its own outcome and the randomly sampled outcomes of its peers through group normalization. Our theoretical and experimental analyses show that uncertainty in group composition introduces composition noise, a non-vanishing variance component that imposes an irreducible lower bound on gradient estimation error and impairs downst
标题:Sample What You Say: Aligning Language Models to Sample the Distributions They State
来源:arXiv
链接:https://arxiv.org/abs/2609.34929
摘要:Language models are increasingly used to sample from a specified distribution, for instance, to simulate survey respondents or generate synthetic data. Instruction-tuned models can state such a distribution correctly and still fail to sample from it. Prompting and changes to decoding reduce this mismatch only partly, which motivates training with policy optimization. Group relative policy optimization (GRPO) is a natural fit for this problem because it already samples a group of rollouts per prompt, and the group's empirical distribution can be compared with the target. However, scoring the group as a whole gives every rollout the same reward. Group-relative centering then sets all advantages to zero, and the model receives no learning signal. To give each rollout its own signal, we introd
标题:SIPO: Selective-Inference Policy Optimization for Tree-Structured Agentic RL
来源:arXiv
链接:https://arxiv.org/abs/2609.34805
摘要:Tree-structured reinforcement learning trains search agents by comparing alternative continuations and propagating terminal rewards to intermediate decisions. Adaptive expansion, however, creates a statistical asymmetry: an incumbent is selected using its own generation statistic, whereas fresh siblings are sampled after selection. When that statistic is associated with return, branch values can reflect selection history as well as continuation quality, even for a shared parent. We propose Selective-Inference Policy Optimization (\SIPO{}), which incorporates this distinction into tree-based credit estimation. Its scale-free branch criterion keeps generation scores and sibling penalties on a consistent relative scale; exchangeable branching supplies multiple fresh continuations from each se
标题:ASCT: Attentive Search over Counterfactual Trees for Credit Assignment in Agentic Reinforcement Learning
来源:arXiv
链接:https://arxiv.org/abs/2609.35215
摘要:Terminal utility evaluates a complete agentic workflow, but learning requires credit for the decisions within it. We introduce Attentive Search over Counterfactual Trees (ASCT), a framework that turns training-time multi-step search into local action credit. At actor-visited states, an auxiliary tree evaluates alternative legal actions from the same recoverable prefix. Its action-value table is centered by the frozen actor's probabilities and supplies credit for PPO on actor-sampled trajectories. This protocol connects counterfactual evaluation to policy learning while deploying the actor alone. Uniform, UCT, and cost-aware AgentUCT instantiate the framework. On HotpotQA agentic retrieval-augmented generation, all three improve mean held-out utility over trajectory-return PPO and workflow-
标题:Proactive Dialogue Policy Optimization via Cognitive-State Transition
来源:arXiv
链接:https://arxiv.org/abs/2609.34948
摘要:Proactive dialogue requires agents to continually adapt their policies to user feedback while progressing toward task objectives over multiple turns. To move beyond imitation learning on static datasets, recent approaches use user simulators to collect interactive data for policy optimization. However, many simulators do not explicitly model the evolution of user cognition, limiting the consistency and state dependence of feedback across turns. Moreover, representing each action only by a high-level strategy label overlooks the large utterance space and cannot distinguish alternative realizations of the same strategy. To this end, we jointly design a $\textbf{Cog}$nitive User $\textbf{Sim}$ulator $\textbf{(Cog-Sim)}$ and $\textbf{C}$ognitive-$\textbf{S}$tate $\textbf{T}$ransition--Driven $
标题:DGF-Bench: A Benchmark for Simulating and Auditing Deception Against Multi-Agent Governance Boards
来源:arXiv
链接:https://arxiv.org/abs/2609.34913
摘要:Tool-using language-model agents can review enterprise projects as governance boards do: they read the evidence, apply written rules and decide whether the project may proceed. Part of that evidence comes from suppliers and project members with a stake in the decision. DGF-Bench is a benchmark in which a board of agents (specialist gates and a General gate that consolidates their decisions) reviews synthetic dossiers while an attacker plants deceptive content in evidence the organization does not vouch for. Dossiers are generated from canonical facts under 61 executable rules, with 42 authoritative records and 32 narrative documents; every gate is certified decidable from those records. Attacks never change an authoritative value, so an attacked dossier keeps the reference decisions of its
标题:ResonAct: Streaming Metrics for Runtime Diagnosis and Self-Healing in Multi-Agent Systems
来源:arXiv
链接:https://arxiv.org/abs/2609.34701
摘要:Multi-agent systems (MAS) are increasingly used to automate enterprise workflows involving multiple specialized agents, external tools, and long-running task execution. Failures may arise from tool degradation, context propagation errors, coordination breakdowns, or repeated agent interactions that prevent task completion. While existing observability frameworks provide traces and logs, diagnosis and remediation are largely performed after execution completes, limiting opportunities for recovery during runtime. We present ResonAct, a runtime self-healing framework that enables continuous monitoring, diagnosis, and remediation of multi-agent systems through streaming operational metrics. ResonAct ingests execution traces, agent interactions, and tool invocations into a streaming analytics l
标题:DeShortcut-Align: Decoupling Spurious Shortcuts for Robust Safety Alignment in Large Reasoning Models
来源:arXiv
链接:https://arxiv.org/abs/2609.34896
摘要:Safety alignment of large reasoning models (LRMs) via supervised fine-tuning (SFT) and reinforcement learning (RL) often yields near-perfect safety scores, yet this apparent success comes at the cost of severe over-refusal and degraded general capabilities. Through systematic empirical analysis, we find that these failures are closely associated with the learning of spurious shortcuts rather than robust intent-sensitive safety evaluation. Specifically, we identify two dominant shortcuts: formatting shortcuts, where refusal behaviors are overly bound to structural prompt templates that frequently appear in safety alignment corpora; and lexical shortcuts, where sensitive keywords reflexively trigger refusals on benign queries. To mitigate reliance on these shortcuts, we propose DeShortcut-Al
标题:Fewer Assumptions by Design: A Reusable Skill for LLM-Assisted Verus Verification
来源:arXiv
链接:https://arxiv.org/abs/2609.34886
摘要:LLM-assisted Verus verification is a less tedious method to verify Rust implementations, but paired with self-referential structures, e.g., Doubly Linked Lists (DLLs)ânotoriously difficult to formalise for verificationâit becomes a substantially more demanding verification task. Moreover, a specification weakness can arise when verification relies on unproven or invalidated assumptions, such as axiomatic lemmas and assume statements. We investigate whether LLM agents can synthesize strong DLL specifications while minimizing these trusted base. The analysis follows three different approaches: manual verification, property-specific verification, and a defined skill for the specific case of DLLs and certain properties of this type of data structure. The skill encodes domain knowledge and
标题:On the Limits of Metacognitive Monitoring in LLMs
来源:arXiv
链接:https://arxiv.org/abs/2609.34864
摘要:Reliable decisions depend on recognizing when an answer may be wrong. In biological cognition, metacognitive monitoring can dissociate from task performance, raising the question of how closely solving and judging are linked in language models. Here we study the confidence reports of four frontier models across 15 benchmarks. High task accuracy can coexist with weak error discrimination: a model solves 97% of competition mathematics problems while its answer-time confidence ranks correct answers above errors barely better than chance. Confidence separates correct answers from errors more effectively on questions solved by a separate reference model, while review brings limited improvement on reference-hard questions. Aggregate discrimination also rewards ranking correct answers on easy que
标题:From Weak Task Specifications to Scientific Extraction Agents: Optimizing Task Construction
来源:arXiv
链接:https://arxiv.org/abs/2609.34829
摘要:Most methods that optimize LLM prompts and agent workflows assume that task-specific output schemas, extraction instructions, and evaluation criteria are predefined. For scientific extraction agents, however, a short task goal may not fully determine these components, while specifying them manually is costly. We study the upstream problem of constructing the task-specific configuration from a weak specification containing only a short goal and unannotated reference documents. Rather than treating automatic construction as a fixed preprocessing step, our framework constructs a task-specific schema, extraction instructions, and base training rubrics, then keeps schema construction and extraction instructions editable during optimization. Failure-focused updates concentrate textual-gradient f
标题:CoSec: Benchmarking Agent Security in Communities
来源:arXiv
链接:https://arxiv.org/abs/2609.34790
摘要:LLM agents operate in persistent collaborative environments involving multiple users, communities, memories, files, and tools. Community boundaries may remain fixed or evolve with changes in membership, roles, composition, and relationships. Agents must complete legitimate tasks and prevent unauthorized disclosure of protected information. Existing evaluations do not fully examine these risks in agent systems. We introduce \textbf{CoSec}, an executable benchmark for evaluating privacy and authorization enforcement in LLM agent systems operating within and across communities. CoSec contains 208 canonical scenarios spanning fixed and evolving boundaries, protected information belonging to the agent owner or other participants, and attacks through dialogue, environmental content, persistent m
标题:WeaveData: A Multimodal Data Analysis System with Self-Critiquing and Self-Evolving LLM Plans
来源:arXiv
链接:https://arxiv.org/abs/2609.34764
摘要:Multimodal data analysis, which answers questions over relational tables, text, and images, has attracted growing attention in the data management community. Large language models (LLMs) enable such analysis in natural language by generating analysis plans over relational and semantic operators. However, LLM-generated plans are error-prone: a plan may silently compute something other than what was asked, fail during execution, or return a result that misses the question. This paper presents WeaveData, a multimodal data analysis system with self-critiquing and self-evolving LLM plans. First, WeaveData generates a typed logical plan for each question and critiques it step by step before execution, and it checks the executed result against the question afterwards. Second, WeaveData evolves a
标题:Draft-KV: Learning Useful Latent Communication Between Language Models
来源:arXiv
链接:https://arxiv.org/abs/2609.34754
摘要:Latent communication passes internal states between language models instead of decoded text, but higher receiver accuracy does not show that the receiver used the message content. Across five method-dataset pairs, replacing each message with one from an unrelated question changes accuracy by at most 0.60 points, even when communication adds 15.44 points over the receiver alone. Thus the interface can supply the gain while making the sharer dispensable. Draft-KV instead sends the key-value states formed while the sharer drafts an answer to the current question. Linear projections place these states in a side memory read through a gated attention branch, and progressive training moves from message reconstruction to answer supervision under a guard on harm from mismatched messages. Both model
标题:Rethinking Latent Visual Reasoning: Grounding Latent Reasoning in Visual Evidence
来源:arXiv
链接:https://arxiv.org/abs/2609.34563
摘要:Latent visual reasoning (LVR) enables multimodal large language models (MLLMs) to perform intermediate computation in continuous latent tokens rather than expressing every reasoning step in words. However, unlike textual CoT, latent reasoning is not directly observable, making it difficult to supervise what latent tokens learn. In this work, we first conduct a thorough analysis of latent-token behavior and identify a latent evidence-credit gap: latent tokens respond only weakly to image perturbations that alter the correct answer. We hypothesize that this issue stems from the lack of explicit supervision during GRPO training. These findings suggest that a final-answer reward provides too little guidance on what visual evidence to preserve or how credit should be assigned across latent toke
标题:Who Is Left of Whom? Tracing Spatial Evidence and Role Binding in Relative-Position Reasoning
来源:arXiv
链接:https://arxiv.org/abs/2609.35486
摘要:High instance-level accuracy can mask inconsistencies in spatial reasoning when objects exchange positions or their roles are reversed in the query. The internal representations supporting relative-position reasoning remain poorly understood. We investigate two complementary components of this process: tracking object locations in the input and representing their query roles. Across three VLMs with visual or textual inputs and their language-model backbones, activation patching reveals a staged progression from early-layer source representations through intermediate-layer query-object representations to late-layer answer states. Targeted interventions further establish causal links along this progression: manipulating source-side representations shifts location information at query-object
标题:Single-Layer MeMo as a Randomized Hamming-Kernel Classifier
来源:arXiv
链接:https://arxiv.org/abs/2609.34562
摘要:MeMo (Zanzotto et al., 2025) is a recent language-model architecture that stores associations between token contexts and next tokens in a correlation matrix memory. In this work, we study its single-layer form and show that its ideal retrieval rule is a multiclass classifier based on the positional Hamming kernel. The MeMo architecture represents both the sequence features and the output labels with Gaussian random codes. Its score is therefore a doubly randomized sketch of the ideal classifier. Under independent input and output codebooks, we bound the errors introduced by context sketching and output decoding, characterize their dependence on model and data parameters, and give a margin-based guarantee for recovering the ideal prediction. Controlled simulations support the trends predict
标题:SkillRubric: Co-Evolving Actor Guidance and Evaluator Rubrics for Multimodal Agents
来源:arXiv
链接:https://arxiv.org/abs/2609.34557
摘要:Recent work incorporates reusable skills distilled from past interactions into multimodal agent training, providing procedural guidance for long-horizon planning and tool use. However, policy optimization in these methods remains driven primarily by sparse outcome rewards, providing little supervision for intermediate decisions. Rubric-based rewards address this limitation through explicit intermediate criteria, but reliable rubrics are difficult to construct at scale and often disconnected from the procedure followed by the actor. We observe that a well-structured skill naturally specifies both how to act and what successful execution should achieve. Based on this insight, we introduce SkillRubric, which represents each skill through aligned actor-facing guidance and an evaluator-facing r
标题:SGG-ReflAct: Sub-Goal Guided ReflAct with Structured Planning for Reliable Long-Horizon Reasoning
来源:arXiv
链接:https://arxiv.org/abs/2609.34548
摘要:Recent advances in reasoning backbones have empowered large language model (LLM)agentstotackle complex, multi-step tasks. However, as reasoning horizons grow, inconsistent internal beliefs induce intermediate errors that cause agents to drift from their goals. This limitation also persists in REFLACT, which reflects only on the end-goal at each step without explicitly considering intermediate sub goals. To address this problem, we propose SGG-ReflAct (Sub-Goal Guided Re flAct), a reasoning backbone that integrates sub-goals generated through a single path LLM planner into the reflection process. We further extend this framework to BeamSGG-ReflAct, which replaces the single-path planner with a beam search based LLM planner for structured plan exploration. We run experiments on ALF World, Sc
标题:SEAD: A State-Based Perspective on Attack and Defense in Tool-Using Agents
来源:arXiv
链接:https://arxiv.org/abs/2609.34518
摘要:Language-model agents increasingly use tools to act on external systems. Earlier actions can alter files, permissions, database records, or other state, making a later routine-looking action harmful. Yet the visible interaction may not reveal the underlying state needed to assess that action. We formulate attack and defense as partially observed state control in SEAD, deriving their design requirements from this shared execution process. Because attackers supply instructions while the target chooses concrete actions, DART decomposes harmful goals into locally plausible steps and uses feedback from actual tool execution to guide trajectory search. The defender must decide before execution with incomplete state evidence. SAGE can therefore investigate relevant state through read-only queries
标题:Training-Free Clinical Reasoning through Medical Ontologies and Cognitive Mapping: A Symbolic-Probabilistic Knowledge Graph Framework
来源:arXiv
链接:https://arxiv.org/abs/2609.35298
摘要:Most clinical prediction systems learn patient-variable-outcome associations; we investigate a training-free diagnostic paradigm mapping patient observations to explicit medical knowledge. CKG Reasoner integrates candidate-specific Evidence Feature Nodes, patient-reference matching, a bounded Information Gate, knowledge-weighted evidence accumulation, disease similarity, and decisive clinical rules. Missing-aware normalization and coverage auditing distinguish absent from unavailable evidence. Candidate ranking is separate from outcome-label-independent K-means clustering, which uses four derived evidence coordinates (evidence strength, relative magnitude, directional similarity, and evidence completeness), not raw predictors or targets, to derive cohort-level assignments. Across six retro
标题:Learning to Re-Draft: A Variational Stackelberg Game for Discrete Diffusion
来源:arXiv
链接:https://arxiv.org/abs/2609.35166
摘要:Discrete diffusion models offer the ability to re-draft, revisiting and correcting earlier tokens throughout generation. This capability depends on the forward corruption process that defines what the denoiser learns to correct. Masked diffusion models fix tokens once they are unmasked, while uniform diffusion permits revisions but relies on uniformly random token substitutions. We instead learn which substitutions are most useful for training the denoiser to re-draft. We introduce Variational Stackelberg Discrete Diffusion (VSDD), a framework for learning a semantically aware corruption process. VSDD formulates training as a leader-follower game: the leader defines a Markovian corruption process parameterized by the denoiser's token embeddings, while the follower optimizes a variational d
标题:ProofLoom: Proof-Obligation-Driven Theory Construction for Autoformalizing Research-Level Stochastic Optimization
来源:arXiv
链接:https://arxiv.org/abs/2609.34960
摘要:Formalizing research-level stochastic optimization in Lean requires both an algorithm model and domain theory connecting foundational libraries to convergence proofs. Revising a model to restore provability can change the mathematical claim. We introduce ProofLoom, a fully automated LLM-agent system for Proof-Obligation-Driven Theory Construction. Given a published algorithm, target theorem, and source proof, ProofLoom autonomously constructs the Lean model and supporting theory. Open proof obligations drive the development of definitions, interfaces, lemmas, and proof plans. Signature contracts record evidence and obligations for model revisions; an independent Judge rejects unsupported assumptions and weakened conclusions. Planner expands the published argument into intermediate claims,
标题:Rubric Rewards from Item Response Theory
来源:arXiv
链接:https://arxiv.org/abs/2609.35646
摘要:Many language tasks have no single answer that can be checked automatically. Rubrics provide criteria for judging responses to these tasks. For reinforcement learning, the resulting verdicts must be combined into a scalar reward. A common approach sums the points assigned to satisfied criteria. Distinct verdict patterns can thus receive the same reward, and the fixed points encode how much each criterion should count, not how strongly its verdict distinguishes the current rollouts. Beyond this aggregation problem, judging the full rubric needs more judge requests as the criterion count grows. To address these limitations, Rubric Response Theory (RRT) measures quality and selects criteria when rubric criteria are monotone indicators of a shared target. Rather than adding assigned points, RR
标题:GraphHCA: Closed-Form Hindsight Credit Assignment for Long-Horizon LLM Agents
来源:arXiv
链接:https://arxiv.org/abs/2609.35084
摘要:Group-based reinforcement learning (RL) has advanced large language models (LLMs) and is increasingly extending to agentic tasks, where sparse terminal rewards make step-level credit assignment essential. Existing methods assign credit from what follows an action in sampled rollouts, but do not explicitly capture its retrospective relation to the realized outcome. Hindsight credit assignment (HCA) instead attributes credit through the ratio of hindsight to behavior-policy probabilities, but estimating the hindsight distribution requires an auxiliary model or an extra pass. To address this estimation bottleneck, we propose GraphHCA, a model-free realization of HCA that eliminates explicit hindsight-distribution estimation. For terminal-goal tasks with deterministic transitions, Bayes' rule