The two new classes: Agent and Env

The previous verifiers release already had the parts for running a single agent: a Taskset (the work to do), a HarnessThe program the model runs inside. It defines which tools the model can call, how to read results, and how to proceed step by step. The model is the engine; the harness is the whole car. The same engine in a race car and in a pickup truck does completely different work. (the program that drives the model), and a Runtime (the machine that executes that program). This release does something simple: it wraps those three pieces into one unit called an Agent.

Give it a task, and it hands back a TraceThe complete record of one agent run: what it said, what tools it called, what score it got, how many tokens it burned. Think of it as the dashcam for each car; the compiled record of all agents in one run is called an Episode. of what happened: the messages it sent, the tools it called, the score it received, the tokens it burned. The messy work—spinning up a sandbox, installing tool services, handling timeouts and retries—is all hidden behind that single action. Any harness, model, or runtime can be mixed and matched, then driven by a few lines of plain code.

Env is where multi-agent runs live, and its signature is just as short: Env.run(task, agents). You give it an initial task and a set of ready-made agents; the control flow in between is yours to write. Each time an agent finishes, its trace is automatically folded into an Episode—the complete record of that run.

Env.run is just ordinary async Python: await twice for sequential, launch a TaskGroup for concurrent, write a while loop for back-and-forth. No DSL, no graph orchestration, no state machine config files. The old single-agent setup collapses into a single line, and the abstraction layer adds nothing on top.

class SingleAgentEnv(Env[SingleAgentEnvConfig]):
    async def run(self, task: Task, agents: Agents) -> None:
        await agents.agent.run(task)

A neat detail: agent names are derived automatically from config field names. If your config class has solver and judge fields, then agents.solver and agents.judge work in your code, no extra registration needed.

Let an agent be the judge and overturn the tests

Start with how common this problem is: In software engineering training tasks, correctness is decided by test cases—and test cases often assert specific implementation details: the name of a variable, whether a function returns a list or a generator. A solution that does the job beautifully can still get a zero just for not matching the test's expectations. Feed that signal into training and the model learns the wrong lesson: it practices guessing what the test wants to see rather than doing the job right.

What if a big model did the grading? A single call can't reach an accurate verdict: it can't see the codebase, can't run tests, can't verify anything—only guess from a block of text.

A judge built as an Agent is a different story. It has its own sandbox. It can read the code, see which test fails on which line, run things itself—then overturn the rigid test's conclusion.

class AgenticJudgeEnv(vf.Env[AgenticJudgeEnvConfig]):
    async def run(self, task, agents) -> None:
        solution = await agents.solver.run(task)
        if not solution.ok:
            raise
        await agents.judge.run(JudgeTask.from_trace(solution))

The judge's prompt is blunt:

Reconstruct what this agent did from the trace, verify it with real execution in your own sandbox, and never trust what the trace claims about anything you can verify yourself.

Judge scoring prompt in the verifiers repo

How a verdict becomes a score: the judge scores against several predefined criteria, writes the result as JSON to a fixed file in the sandbox; the framework validates it strictly, records each verdict as a metric on the solver's trace, and averages them into a reward named judge. The default trusts the judge's score alone—the task set's own reward weight is 0. Set the weight to 1 if you want both.

Three design choices the blog doesn't mention

Which sandbox the judge works in is a choice. The shared version lets the judge into the same sandbox the solver used, with the workspace left intact—any changes are visible. The isolated version gives the judge a fresh box, restoring only the artifacts the solver officially submitted. Choosing the environment id chooses the security boundary, rather than a toggle that could contradict the actual semantics.

The judge's harness must be able to execute code. This is checked at environment construction time; attaching a chat-only, no-tool harness throws an error. A verdict that doesn't require execution is a plugin-style scorer, not an agentic judge.

Clean the slate before starting. Since the solver used the box first, the framework deletes any existing verdict and trace files before the judge begins. It guards against two things: a solver pre-planting a verdict to impersonate the judge's result, or placing a symlink in the upload path that redirects the judge's writes to a file of its choosing.

One rule runs throughout: the judge's tokens never enter the training data. It's the measuring stick; the stick shouldn't be optimized along the way.

A proposing agent and solving agents

For reinforcement learning to have a useful signal, task difficulty has to track the model's current level: too easy is uninformative, too hard is equally uninformative. A static dataset can't do this—it won't get harder just because the model got stronger.

The approach is to have the model write its own problems. One agent plays the proposer, picking a seed topic and constructing a new problem. A group of agents plays the solvers, each attempting it. Solvers score points for correct answers—that part's obvious. The proposer's score, though, isn't about whether the problem looks good. It's about what the problem did to the solver group.

If everyone solves it, too easy. If nobody does, too hard—or the problem itself is broken (maybe the proposer's own reference answer is wrong). When about half solve it, the problem provides the most training signal. That quantity is called learnability, and it's written as 4p(1−p), where p is the solve rate. At p=0.5, it peaks at exactly 1.

That curve is the entire secret of automatic curriculum: the model gets stronger, and its problems get harder accordingly, no human tuning required. The idea comes from Absolute Zero (Zhao et al., 2025).

async def run(self, task: vf.Task, agents: vf.Agents) -> None:
    proposed = await agents.proposer.run(task)
    solve_task = SolveTask.from_trace(proposed)
    async with asyncio.TaskGroup() as tg:
        for _ in range(self.config.n):
            tg.create_task(agents.solver.run(solve_task))

The in-repo implementation is considerably more concrete than this sketch. There are 6 seed topics: ratio mix, number theory, combinatorics, geometry, probability, modular arithmetic. The proposer is instructed to first write code that constructs the numbers, verify the answer end-to-end, and finally emit one line of JSON. The answer must be an integer—no room for hand-waving. By default, 4 solvers attempt the same problem.

The two seats can be configured completely differently, which is genuinely useful:

Which side participates in learning is a switch too: train both, train only the proposer, or train only the solvers.

Two models play poker, and the opponent is itself

Kuhn Poker is the textbook toy model of game theory, small enough to describe in one sentence: three cards in total—J, Q, K (K highest). Each player antes 1 chip and draws one private card. First player chooses check or bet 1 chip. After at most one more action, the hand settles at ±1 or ±2 chips.

It's chosen precisely because it's small. Small enough to run tens of thousands of hands and watch where a strategy converges, rather than burning budget on the complexity of the game itself.

The environment manages three things: who holds which private card, the current public state, and which legal actions are available at this step—all judged host-side. The model just needs to include one bracketed action in its reply, e.g. [check]. Two actions or a malformed one gets a re-ask; fail again and it's a forfeit. The task set is infinite: one random seed per hand, run as many hands as you like.

The two seats bind to the same model by default—that's self-play: the strategy gets stronger, so the opponent gets stronger too, and the curriculum climbs on its own. No separate opponent service required.

How credit is divided among different roles

Both self-play examples above force the same question: what should this run's score be compared against?

To score a performance, you compare against "the usual level" to know whether it's good or bad; that baseline is what's called the baseline. An 80 on a final exam is good or bad depending on the class average—the average is the baseline. GRPO does exactly this: run a batch of answers on the same problem, and reinforce whichever ones beat the batch average.

But if half the class took a different exam, the average is meaningless. Multi-role training runs into exactly this, in two distinct ways.

One person's failure shouldn't count against another

In the proposer-solver run, a solver's score can only be compared against "other attempts on the same problem." Comparing across problems means punishing a normally performing solver for a hard problem's failure. A proposer's score can only be compared against "other problems generated from the same seed topic." Classic GRPO can't express this hierarchy; it flattens a set of rollouts into one average, mixing problems of different difficulty and different roles together.

Hierarchical GRPO does one thing: grouping. Split into groups by (role, episode), and each group computes its own baseline. The code is almost comically short: group, subtract the group mean, done.

In a zero-sum game, the average is always 0

Poker is a different kind of failure. In a zero-sum game, the mean reward across a batch of games is always approximately 0: one side's wins are the other side's losses, no matter whether the strategies are good or bad. Using it as a baseline is the same as having no baseline at all.

Worse, the two seats' reward scales are inverted. Mixed together, the first-player's structural advantage gets read as permanent credit—the role is merely seated well, but it keeps collecting rewards.

RAE's answer: maintain a separate moving average for each role, remembering "how much this role usually scores." Credit equals this run's score minus its own usual level. In a single-agent environment, RAE automatically degenerates into REINFORCE with a moving-average baseline.

Both algorithms live in prime-rl, a few dozen lines each. RAE comes from the SPIRAL paper (arXiv:2506.24119).

Making the user an agent too

Assistant-style tasks share a trait: they can't be represented as "one prompt, one reply." A real person has context you can't see, reveals information bit by bit, reacts to your answers, and decides when the job counts as done.

So the user becomes an agent too. One episode is a back-and-forth conversation between user and assistant, and both traces stay in the episode record. The simulated user is frozen by default—same principle as the judge; it's part of the environment. The assistant's trace is scored against the original task and used for training.

Here's how information is split. The data row in the task set is read as the user-side world: its prompt becomes "your situation" in the user's system message, while the assistant gets the same task with the prompt blanked out. The assistant starts with no idea what to do and has to figure it out through the conversation.

The user's persona includes a few rules: speak in your own words, keep it short, stay in character, the assistant does the work (any tool call, answer, or formatting must be done by the assistant, not taken over), and details come only when asked. When the goal is met—or the assistant is clearly unable—it replies with an end marker. The default cap is 8 turns, to keep conversations from drifting off.

async with (
    agents.user.interaction(user_task) as user,
    agents.assistant.interaction(assistant_task) as assistant,
):
    ask = await user.turn("Hello! How can I help you today?")
    while True:
        answer = await assistant.turn(ask.last_reply)
        if answer.terminated:
            break
        ask = await user.turn(answer.last_reply)
        if ask.terminated:
            break

The result: different user groups, personas, hidden goals, and interaction strategies all plug into the same interface, and the assistant side can be configured independently. It's the same set of parts assembled differently, not a separate evaluation pipeline.

What's available now

The repo ships 13 ready-made harnesses, including real coding agents like Claude Code, Codex, and Kimi Code—just attach one to a model and go.

• claude_code
• codex
• kimi_code
• bash
• browser_use
• mini_swe_agent
• hermes_agent
• openclaw
• terminus_2
• rlm
• pi
• pool
• null

There are 4 runtime types: local process, Docker, Modal, and Prime Intellect's own cloud sandbox. Beyond the four environments above, the repo also includes a BestOfNEnv: run the same task n independent times, flag the best one, and check whether any attempt crossed a threshold—useful for rejection sampling and pass@k evaluation.

The same Agent abstraction is already powering their own synthetic data generation and filtering pipelines: every trace has a uniform format and is auditable, which makes it a data product in itself. The blog's final example is barely a dozen lines: Poolside's Laguna-S2.1 model runs inside the pool harness and queries PyPI for the latest verifiers release version. Same Agent as training.

The first example treats grading. Hardcoded tests obsess over writing style and implementation details; a clean solution that misses the test's expected pattern still gets a zero, and the model learns to guess what the test wants to see.

Before the judge starts, any existing verdict files are cleared so nothing can be planted in advance; the judge's own conversation records never enter training. It's the measuring stick, and the stick shouldn't be tuned along the way.

The second example treats the shortage of problems. A useful learning signal has to track the model's current level—too easy yields nothing, too hard yields nothing, and a static bank can't do that.

The approach: one agent proposes problems, several agents solve them. The proposer's score doesn't care whether the problem looks elegant; it cares about how many solvers got it right. Everyone solves it means too easy. Nobody does means too hard, or the problem itself is broken. When about half solve it, the problem carries the most training signal.

Two more modes run out of the box: two agents play a minimal poker game to practice mind games, with the strategy and opponent getting stronger together—no separate opponent service needed. And the user itself becomes an agent, with task information deliberately hidden on the user's side, forcing the assistant to ask its way toward understanding.

Both self-play scenarios hit the same question: what should this run's score be compared against? The two seats' rewards sometimes can't be compared on the same scale, so Prime Intellect wrote two new algorithms: one groups by role and episode for separate baselines, the other gives each role its own personal history line.