SPROUT checkpoints the environment and the agent context after every turn. When a rollout fails, the same policy reads the failure, picks the turn before the mistake, and continues from there.
An illustrative episode. Press play or step through; hover over anything for what it is.
A long-horizon attempt reads an unfamiliar codebase, installs dependencies, builds and runs tests, sometimes for hours. The verifier's reward is binary, so one bad decision anywhere makes it zero, and in group-relative RL a task whose attempts all fail contributes no gradient. The hardest tasks burn the most compute and teach the least.
Every new attempt redoes the exploration and the setup, and may repeat the same mistake. Branching only inside the conversation does not help either: the files the agent edited, the packages it installed and the artifacts it built are gone.
Pass@4 = 4× the tokens for a few more solved tasksKeep the turns before the mistake. Restore the environment snapshot and the agent context to exactly that boundary, let the verifier's feedback decide where and how to continue, and spend new turns only on what was wrong.
SPROUT beats Pass@4 with 32–52% of its tokensA checkpoint is a turn boundary with two halves: the environment snapshot after the turn's tool calls, and the agent context up to the same observation. Restoring both gives a continuation the exact state of its prefix, with nothing re-executed.
Each attempt runs in its own microVM. After every turn the environment's changed disk blocks are saved as an incremental snapshot layer, so a checkpoint grows with what the turn wrote, not with the size of the environment. Tool calls are synchronous shell commands with no background processes, so nothing persists only in memory and guest memory is never stored. Branches write to their own copy-on-write layer.
The actor also plays analyst and reviewer under role prompts; the roles share weights. The analyst reads a failed attempt with the verifier's diagnostics. The reviewer picks a parent attempt and a boundary and writes the starting turn that replaces the parent's turn there. Two rounds, 4 then 3 branches, stopping at the first pass.
Each task gets two roots and three children per root. When both roots fail, the children start with reviewer-written turns; otherwise they continue unguided from a chosen boundary. Successes are credited, shared prefixes are counted once, and the analysis never enters the learner's input, so the policy learns the corrective turn from its ordinary context.
| Role | Reads | Writes |
|---|---|---|
| Actor | the task and its own history in the sandbox | a turn: reasoning, then shell calls |
| Analyst | one failed attempt, its patch and the verifier's diagnostics | why it failed, what is worth keeping, candidate boundaries |
| Reviewer | all analyses, the attempts' histories, earlier starting turns and their outcomes | for each branch: a parent, a boundary, and the starting turn in the actor's voice |
| Learner | the retained prefix, the starting turn and the continuation, never the analysis or the corrective direction | one GRPO step over roots and branches |
The three roles share weights; only the analyst and reviewer prompts carry verifier diagnostics. Boundaries lie between turns, never inside one: within a turn the actor has already committed to all of its calls.
DeepSWE, Terminal-Bench 2.1 and SWE-bench Pro v2 Hard, all with mini-swe-agent and a single shell tool. Pass@1 is the mean of four independent passes; Pass@4 counts a task solved by any of them.
SPROUT beats Pass@4 on every model and Shepherd by 3.6–23.0 points, recovering 64.8–93.6% of the tasks a single attempt fails. The entropy-based selector needs token log-probabilities, so it runs only with Qwen3.8-27B (71.7%). The largest gain over Pass@4 is Qwen3.8-27B on Terminal-Bench 2.1: 66.3% → 89.9% with 61.3% fewer tokens.
| Qwen3.8-27B | Pass@1 | Pass@4 | Entropy-based | Shepherd | SPROUT |
|---|---|---|---|---|---|
| DeepSWE · resolve rate, % | 37.2 | 71.7 | 71.7 | 67.3 | 78.6 |
| tokens, M | 1,921 | 8,673 | 8,589 | 9,026 | 4,123 |
| Terminal-Bench 2.1 · resolve rate, % | 49.4 | 66.3 | 76.4 | 79.8 | 89.9 |
| tokens, M | 27.7 | 319.9 | 112.8 | 103.9 | 123.7 |
| SWE-bench Pro v2 Hard · resolve rate, % | 62.7 | 88.2 | 90.2 | 78.4 | 94.1 |
| tokens, M | 550.8 | 1,918.5 | 1,238.9 | 1,004.0 | 771.3 |
Qwen3.8-27B, full-parameter, on SWE-bench Rebench v2 Filtered (the top 2,115 tasks by golden-patch length). GRPO and SPROUT share the data and the configuration: 32 prompts × 8 rollouts per update, 128K context, no KL penalty, 8 NVIDIA B300.
| Training | SWE-bench Pro v2 Hard | DeepSWE | Terminal-Bench 2.1 | Hours per step |
|---|---|---|---|---|
| Base (no RL) | 62.7 | 37.2 | 49.4 | – |
| GRPO | 74.5 | 33.6 | 39.3 | 3.77 |
| SPROUT | 84.3 | 40.7 | 64.0 | 2.28 |
GRPO helps on SWE-bench Pro v2 Hard, the benchmark closest to the training tasks, and hurts on the other two. SPROUT rollouts improve the base model on all three, by 9.8, 7.1 and 24.7 points over GRPO, and a training step takes 2.28 h instead of 3.77 h because branches reuse their prefixes.
With the same budget of branches, more review rounds with fewer branches each resolve more tasks for fewer tokens, because each round starts from the previous round's failures and their diagnostics. Breadth alone saturates: 32 branches in one round barely improve on 4, though they still beat eight independent attempts with a third fewer tokens.
Eight branches in one round: 76.1% for 0.75B tokens. The same eight over eight rounds: 92.0% for 0.53B, recovering 52 instead of 34 of the 61 tasks the baseline fails. Widening one round from 4 to 32 branches adds 3.5 points for 3.2× the tokens. Depth does add sequential dependencies between rounds; these runs measure the success–token trade-off, not latency.
A trajectory passing 45 of 46 tests scores the same as one passing none, yet it has localized the problem to one test. Of 26 branch points in the SWE-bench Pro case study, 21 fall in the middle or final third of the parent's progress.
Four Ansible branches resume from the same checkpoint and only one passes 46 of 46: the one whose fix matches the caller's real constraints, which the verifier's error message exposed and the agent's own assumptions did not.
On the Flipt ECR task, 4 baseline attempts, 7 BPO branches and 7 Shepherd branches all fail. SPROUT's first round fails 2 of 6 tests; the verifier names two undefined constants; the second round resumes from that branch and passes in ten more turns.
@misc{liu2026sproutscalingagenticreinforcement,
title={SPROUT: Scaling Agentic Reinforcement Learning for Long-Horizon Software Engineering},
author={Mengyang Liu and Weiming Wang and Haixin Nan and Yu Wu and Peize Li and Shihong Huang and Xuning He and Xue Jiang and Yihong Dong},
year={2026},
eprint={2610.14405},
archivePrefix={arXiv},
primaryClass={cs.SE},
url={https://arxiv.org/abs/2610.14405},
}