Multi-Turn Credit Assignment, Agent Tool Creation, and Structural Causal Discovery

Today’s selection of 8 noteworthy AI/ML papers from arXiv covers significant advances in multi-turn agent credit assignment, joint tool creation and use, large-scale web data generation, and task-specific Mixture-of-Experts architectures, alongside novel contributions in software engineering environments, causal inference, system simulation, and quantum kernels.

1. IAPO: Influence-Aware Policy Optimization for Credit Assignment in Multi-Turn Service Agents

Authors: Bo Ren, Yirong Mao, Yi Yang, Wenhui Que | Categories: cs.LG Link: arxiv.org/abs/2608.24588v1

This paper tackles credit assignment in multi-turn LLM service agents—a challenge where a final reward alone cannot indicate which actions contributed to task resolution. The authors propose IAPO, which represents each rollout as a typed influence-dependency graph over agent actions, with user/tool observations as evidence, and uses this structure to redistribute trajectory-level advantages. Experiments on three service-agent benchmarks with Qwen3-4B/8B show superior performance over multi-turn RL baselines while preserving multi-turn function-calling ability.

Takeaway: IAPO addresses a critical gap in sparse-reward RL for long-horizon tasks by deriving richer learning signals from already-completed rollouts without needing comparative data or hand-crafted step-level rewards.

2. Joint Optimization of Tool Creation and Use for Large Language Model Agents

Authors: Zhi Rui Tam, Chieh-Yen Lin, Yun-Nung Chen, Shao-Hua Sun, Hung-yi Lee | Categories: cs.AI, cs.SE Link: arxiv.org/abs/2608.24571v1

This work introduces SMITH, a reinforcement learning framework that jointly trains tool creation and tool use within a single policy, addressing the decoupling problem in prior tool-creation systems. Each rollout alternates between build tasks (write a tool from examples) and use tasks (invoke pooled tools), with three reward axes capturing schema, code, and outcome failures independently. A 4B Qwen3 trained with SMITH achieves 79.8% macro-average accuracy on held-out procedural reasoning tasks, outperforming an untrained 30B model and showing strong out-of-domain generalization.

Takeaway: Jointly optimizing the gap between tool writers and tool users is a breakthrough—it allows models to produce schemas they can actually invoke, significantly boosting practical tool-augmented performance with much smaller backbones.

3. BrowserForge: Scaling Web Episode via Parallel Browser Sandboxes

Authors: Fei Tang, Huawen Shen, Zhiqiong Lu, Zhengxi Lu, Pengyuan Lyu et al. | Categories: cs.CL Link: arxiv.org/abs/2608.24848v1

BrowserForge is a framework that generates large-scale web interaction data by driving parallel browser sandboxes over the open web, exposing agents to hundreds of thousands of real websites. It couples open-web sourcing, a sandbox cluster manager, and a Proposer-Solver dual-agent loop, producing a corpus of 203,238 verified trajectories from distinct websites. Fine-tuning a compact multimodal model on this corpus raises its live Online-Mind2Web success rate from 25.66% to 33.33%, and produces a release that acts purely from screenshots.

Takeaway: This represents a major scale-up in web-agent training data diversity—by sourcing from the open web rather than predefined site lists, it overcomes a key bottleneck limiting web agent generalization.

4. MoTE: Mixture of Task Experts for Multi-Task Video Understanding

Authors: Muhammad Asad Ali, Umar Khan, Nadia Robertini, Didier Stricker | Categories: cs.CV, cs.LG Link: arxiv.org/abs/2608.24763v1

MoTE converts large language model feed-forward networks into task-specific experts within a decoder architecture while keeping the multimodal backbone shared, addressing the entanglement of tasks in dense transformer decoders. Each example follows a sample-level task route, keeping active expert computation independent of the total number of stored experts. The five-expert VideoLLM-MoTE activates ~2B parameters per sample and achieves higher average top-1 accuracy than recent VideoLLM baselines on five COIN benchmarks.

Takeaway: Task-structured routing offers an interpretable, compute-efficient alternative to token-level sparse MoE, proving that explicit task routes outperform dense activation and learned sparse routing in multi-task video-language learning.

5. A Literate Programming Environment for Human and Machine Agents

Authors: Adam T. Burke | Categories: cs.SE, cs.AI, cs.PL Link: arxiv.org/abs/2608.24644v1

This paper introduces an environment for constructing literate programs in concert with language-aware machine agents, featuring a grammar for executable program essays and a parser treating names as first-class objects. The environment includes an internal name-graph relating prose, names, and executable artifacts, plus bindings for established programming languages and testing toolsets. This design co-locates code with its most relevant natural language context, making better use of LLM context windows and providing coding agents with symbol-aware search akin to human-facing IDEs.

Takeaway: By reimagining literate programming as a structured collaboration medium between humans and LLM agents, this work offers a practical path toward richer, more reliable human-AI pair programming.

6. Scalable and Versatile Identification for Hierarchical Structural Causal Models: A New Look at Project STAR

Authors: Janis Aiad, Aghiles Drali, Aymen El Ouadrhiri, Anass Ettahiri, Yasser Oufqir et al. | Categories: stat.ML, cs.AI Link: arxiv.org/abs/2608.24500v1

This paper develops a complete, scalable pipeline for Hierarchical Structural Causal Models (HSCM) that bridges symbolic identification and practical estimation, applied to the landmark Project STAR dataset. A key innovation is an adapted Abstract Syntax Tree that decomposes pyAgrum-identified formulas into independent density, expectation, and marginalization tasks for parallel computation. Results show that flat baselines fail to encode class-level interventions, and symbolic identification alone is insufficient—scalable estimation and stability checks are essential components.

Takeaway: This work provides a robust open-source bridge between formal causal identification and practical numerical estimation in hierarchical settings, offering an important validation tool for educational and social science research.

7. Simthesizer: An Agent-Driven Simulation Framework for LLM Serving Systems

Authors: Wonung Kim, Hyunmin Choi, Minsu Kim, Jaehong Cho, Yeongwook Kim et al. | Categories: cs.AR, cs.AI Link: arxiv.org/abs/2608.24650v1

Borg is a framework that uses agent-driven development to evolve LLM serving simulators, closing the widening gap between deployed serving systems and their models. It introduces a composable simulator infrastructure expressed as a unified dynamic graph, where a harnessed coding agent lowers natural-language feature requests under fidelity validation. Extensions built on Borg follow a vLLM-based real system with 2.51% average throughput error (vs. 6.03% for existing simulators) and simulate up to 284.96x faster than state-of-the-art baselines.

Takeaway: By treating the simulator itself as a software system that AI agents can continuously evolve, Borg solves the urgent problem of keeping simulation infrastructure in lockstep with the rapidly changing LLM serving landscape.

8. When Similarity Is Interaction-Driven: Quantum Kernels for Regime-Sensitive Learning

Authors: Hanqiu Peng, Jianlong Lu, Ying Chen | Categories: quant-ph, cs.LG Link: arxiv.org/abs/2608.24631v1

This paper introduces an interaction-driven quantum kernel built from entangled Pauli-string feature maps that explicitly encode sparse high-order block interactions, addressing settings where similarity is governed by interactions rather than distance. The fidelity kernel is positive semidefinite, allows exact block-factorized formulation, and induces geometry sensitive to interaction-regime changes. It consistently outperforms classical kernels across synthetic experiments and achieves the highest mean accuracy on Credit Card Fraud Detection, while remaining exactly evaluable on classical computers.

Takeaway: This work shows that quantum-kernel value lies in the alignment between feature-map geometry and underlying predictive structure rather than Hilbert-space dimension—a key insight for designing practical quantum-inspired models without requiring quantum hardware.

This content was generated with AI assistance. Paper information sourced from arXiv.