Three Agent Self-Improvement Papers Claim Gains—Only One Shows Cross-Benchmark Numbers

Two recent papers and a report describe ways to make AI agents improve themselves, but only one reports gains on two benchmarks after training on a different benchmark description. ¹²³ For teams deciding whether to build on any of this, that distinction—not the “self-improving” label—is what matters.

The three methods intervene at different points in the training pipeline. EnvHarness adapts training environments. EvolvingBrain builds and selects a library of agent skills. Recursive Reward Improvement (RRI) generates training tasks and grading rubrics. Each needs a different kind of evidence, and the supplied evidence is uneven. ¹²³

What each agent self-improvement method actually changes

Method	What changes	Evidence reported
EnvHarness	A framework dynamically adapts agent-training environments to an agent’s weaknesses, reportedly without modifying the agent, environment, or evaluation system	Researchers say the approach can improve learning effectiveness and efficiency; the supplied report gives no quantitative results or benchmark details 3
EvolvingBrain	Extracts patterns from model-specific trajectories, turns them into skills, organizes them, and selects skills based on agent state	Experiments on VAB-Minecraft and DiscoveryWorld reportedly show improved success rates and generalization; the supplied abstract gives no numbers 1
RRI	A proposer generates training tasks and rubrics; a solver learns from them via reinforcement learning	Reports score gains on HealthBench Professional and transfer to PRBench Hard and ProfBench 2

The real axis of comparison is where adaptation happens: the training environment, the agent’s skill memory, or the tasks and rewards it trains against. That suggests what to isolate when evaluating. If a full system improves, you still need to know which piece caused it—and whether the gain survives when the method no longer controls the test conditions. ¹²³

EnvHarness: adaptive agent training with no numbers attached

Google researchers have open-sourced EnvHarness, a framework that dynamically adapts agent-training environments to an agent’s weaknesses. The report says researchers believe it can “improve both learning effectiveness and efficiency,” while not modifying the agent, environment, or evaluation system. ³

That last point raises an obvious question: if the environment itself isn’t modified, what is the harness actually changing? The supplied report doesn’t resolve it. It gives no benchmark name, baseline, sample size, effect size, or training cost. The stated benefits are claims in the report, not quantified findings there. ³

A useful test would train one agent with EnvHarness and an otherwise identical agent on a fixed environment, then score both on held-out tasks the adaptation loop never touched. Training curves, compute cost, and performance on weaknesses not used to steer adaptation could help distinguish sample-efficiency gains from final-performance gains. The supplied report does not describe such a test. ³

EvolvingBrain: better agent, or just a better memory?

EvolvingBrain is an adaptive skill system for multimodal agents. It extracts procedural and problematic patterns from model-specific trajectories, generates reusable skills organized into domain, plan, and action/trap levels, summarizes long-horizon interactions into structured abstractive memory, and uses the agent’s current state to guide skill selection. ¹

The authors report improved success rates and generalization on VAB-Minecraft and DiscoveryWorld, but the supplied abstract gives no numbers, baselines, or ablations. That is enough to describe the architecture, but not to size the improvement or determine which component accounts for it. ¹

A fair test would compare it with simpler alternatives: plain conversation memory, a static skill library, and versions with components removed. New tasks, environments, and base models would test transfer; degraded performance when the skill store is incomplete could reveal how much the system depends on already having the right experience banked. Without those comparisons, gains might reflect “more memory helps” rather than the benefit of adaptive skill selection. ¹

The abstract says code is provided in supplementary materials and will be released publicly in the camera-ready version. That is a release plan, not replication. ¹

RRI: reinforcement learning for agents with the strongest transfer claim

RRI runs a proposer-solver loop: the proposer writes training tasks and, for open-ended ones, a grading rubric describing what a good answer should include and what errors it should avoid. The solver learns to solve the tasks through reinforcement learning. ²

The method targets a failure mode in rubric-based training: solvers may exploit gaps in a rubric rather than solve the task. Before training, an adversary writes a rubric-gaming answer, while the same model writes an honest answer without seeing the rubric. The rubric can then undergo up to two repair steps to make the honest answer score higher. The exploits found also inform the generation of future tasks and rubrics. ²

On HealthBench Professional, the authors report adjusted-score gains over the Generation Only self-improvement baseline of 10.9 points for a 9B model and 12.4 points for a 27B model. They also report a 10.3-point gain without an external model. ²

For transfer, a 27B model trained on the HealthBench Professional description improved by 6.6 points on PRBench Hard and 8.0 points on ProfBench, without seeing tasks from either benchmark. ²

That is the strongest evidence of the three because it is the only reported cross-benchmark result with numerical gains and an explicit statement that the model saw no tasks from the target benchmarks. It remains a result reported in one paper: the supplied abstract does not provide uncertainty estimates or deployment evidence, and the supplied sources do not report an independent replication. ²

Teams evaluating RRI internally could audit generated tasks and rubrics for validity, leakage, reward hacking, and grader instability, then rerun the Generation Only comparison across model sizes, with and without an external model, on held-out benchmarks and with uncertainty reported. ²

What to demand before adopting any agent evaluation claim

The evaluation should match the mechanism. EnvHarness needs a test of environment adaptation. EvolvingBrain needs tests of its skill and memory components. RRI needs tests of task and rubric quality. A single benchmark score does not isolate these effects. For a broader approach to scoping agent trials, see this pre-pilot checklist for evaluating AI agents.

• Test transfer, not recall. Keep test tasks untouched by whatever the method steers—environment selection, skill creation, or rubric generation. Score on new tasks or environments, not only on the weaknesses the method was tuned against. ¹²³
• Isolate the intervention. Use a relevant baseline: fixed-environment training for EnvHarness, static memory for EvolvingBrain, and Generation Only for RRI. ¹²³
• Price the cost and the risk. Track training and evaluation compute alongside scores, and examine failure modes as well as averages. The supplied sources do not offer a shared cost or risk framework. ¹²³
• Insist on reproducibility. Fix the evaluation protocol, document the test set, and seek an independent rerun. The supplied sources do not report one. ¹²³

Which method fits depends on where an agent actually fails. Failures tied to task exposure or environment dynamics may point toward EnvHarness; failures that recur across long sessions may point toward EvolvingBrain’s skills and memory (a demo of long-running agents is not a reliability case on its own); weak task coverage or reward design may point toward RRI. That is a diagnosis, not a guarantee that any of these methods will fix the problem.

Before committing engineering time, run a small internal replication on held-out tasks with a go/no-go bar that weighs reliability, training cost, and failure severity—not just the headline score. Once an adapted agent moves toward production, the question shifts to making its actions, dependencies and cost accountable. The supplied sources do not report independent validation or production testing for these methods, so that remains an evidence gap to address.

FAQ

What is EnvHarness? An open-sourced framework reported to dynamically adapt agent-training environments to an agent’s weaknesses without modifying the agent, environment, or evaluation system. The supplied report gives no quantitative results. ³

How does EvolvingBrain train multimodal agents? It extracts patterns from model-specific trajectories, turns them into organized skills, summarizes long-horizon interactions into structured memory, and uses the agent’s current state to guide skill selection. ¹

What is Recursive Reward Improvement? A proposer-solver loop in which the proposer writes training tasks and rubrics, and the solver learns via reinforcement learning. An adversary tests for rubric gaming, followed by rubric repair. ²

Do any of these transfer beyond their training tasks? RRI reports gains on PRBench Hard and ProfBench after training on the HealthBench Professional description, without seeing tasks from either target benchmark. EvolvingBrain reports generalization on VAB-Minecraft and DiscoveryWorld without numerical results in the supplied abstract. The EnvHarness report does not describe a transfer result. The supplied sources do not report independent or production validation. ¹²³

Sources

¹ EvolvingBrain: An Adaptive Skill System for Multi-modal Agents Self-Evolution in Open Worlds, arXiv, Oct. 16, 2026

² Recursive Reward Improvement via Adversarial Meta-Prompt Evolution, arXiv, Oct. 16, 2026

³ Google EnvHarness Enables Self-Evolving Agents to Learn in Dynamics Environments, InfoQ AI, ML & Data Engineering, Oct. 16, 2026