Arbiter: Sample-Efficient Test-Time Scaling with an Agentic Verifier

Prior attempts at IOI-level competitive programming scale test-time compute in parallel: overgenerate, then search. AlphaCode drew up to a million candidate programs per problem and distilled them to ten submissions with filtering and clustering. OpenAI's o1-ioi sampled 10,000 solutions per subtask at IOI 2024, then picked 50 with clustering and a learned reranker. NVIDIA's GenCluster reached the IOI 2025 gold line by drawing 5,000 solutions per subtask from gpt-oss-120b and selecting 50 through behavioral clustering and a tournament. The paradigm works the way gold panning works: scoop blind, sift everything, keep the rare fleck that glitters. How can we do better than search?

Arbiter (Agentic Reinforcement-learned Bug-finding Interactive Test ExamineR) hands the solver a metal detector instead. It reads the problem and one candidate, decides Accept or Reject, and on rejection returns a diagnosis plus a concrete input that breaks the code. That input is a bearing on where to dig next. A solver refining against that signal corrects its prior mistake instead of redrawing from scratch, so the compute goes into revising one chain deeper rather than widening the pool.

Beyond IOI, Arbiter is an example of environment engineering. A typical coding environment is built to ensure correctness and nothing more: it scores what the model did, and the score is where the interaction ends. We design the environment to also carry signal. Every rejection arrives with a diagnosis and a replayable counterexample, so the model has something concrete to improve from. Because that feedback comes from the verifier rather than the judge, querying it spends no submissions and exposes no hidden tests. The same signal can drive RL training and test-time refinement alike.

Base model & RL loop. Arbiter is trained from Qwen3.6-35B-A3B with GRPO-style group-normalized advantages in an asynchronous agentic-RL loop. The verifier is trained in a sandbox environment with two tools (execute_code, execute_command) and no judge access. It can compile the candidate code solution and probe it on inputs of its own devising, but must commit to a verdict on its own evidence. Adjudication happens only after the episode ends, when the candidate is re-hosted on a live judge and the submitted hack is replayed against it. This keeps the training signal unmetered and leak-free.

Training data. We mine training data from a strong solver's RL rollouts and sample it down to 20K submissions across 6,145 ICPC and Codeforces problems predating 2026, under a score-stratified quota. The near-miss band (score 76–99) gets the largest share at 40%, since a solution that fails a single subtask cannot be broken by a careless input and forces the verifier to actually read the code. Every label is a live judge verdict. The pool is deduplicated by code content, capped per problem, and kept disjoint from every evaluation problem, with a separate hard validation set (100 problems, 400 candidates) to pick checkpoints.

Reward design. Every Arbiter output includes a verdict (AC/no-AC), a diagnosis and a hack. Rewarding only the verdict makes the reward easily gamable due to label imbalance, and the model quickly degenerates. Rewarding the diagnosis is hard due to its free-form text nature. Thus our reward heavily relies on the verifiability of the hack.

The reward is a layered funnel built to make evidence cheaper to produce than to fake, with three-quarters of it sitting at the one step that cannot be faked: a validator-legal input that actually breaks the candidate. Guessing "rejected" and submitting garbage therefore caps an order of magnitude below a verified break.

Length penalty. Late in training, responses balloon until episodes die to timeouts rather than to wrong verdicts. The fix, adopted from Kimi K2.5, is a length penalty applied in alternating phases against a relative budget, the 75th percentile of length within each group's best-reward tier. Under it, response length comes down while reward keeps rising.

Refinement is only as good as the verifier behind it. We measure Arbiter's judging quality directly on two complementary benchmarks.

- UOJ-Bench is an existing public hacking benchmark: every candidate is known to be wrong (479 Easy / 1,046 Hard), so it probes depth against covert bugs that survive an official test suite.

- USACO-Judge is a benchmark we built from the held-out 2026 USACO contests: 900 candidates (270 Easy / 630 Hard) over 39 problems spanning Bronze through Platinum, balanced at one correct solution for every two wrong ones, so a verifier must accept correct code as well as break wrong code. Candidates come from five model families plus five human competitive programmers, and every label is a live judge verdict on expert-authored validators and checkers.

The table ranks models by hack success: the share of wrong candidates broken with a legal generated input, split Easy/Hard on each benchmark. Training does the heavy lifting. On UOJ-Bench Easy, the same 35B backbone breaks 16.7% of candidates untrained, 41.3% with an agentic scaffold, and 61.5% after RL. That places Arbiter above DeepSeek-V4-Pro, a model 45× its size, on every split.

On IOI 2026 itself, gpt-oss-120b attempts every subtask, drawing 50 seed solutions each. Arbiter reviews every candidate. An Accepted verdict sends the program to the judge and retires its trajectory. A rejection returns feedback that the solver revises against in the next round, under a 5-round budget. A trajectory that never earns an accept submits its final revision instead, so the verifier decides both what to fix and when to stop. The climb is also cheap: where GenCluster drew 5,000 solutions per subtask, refinement reaches gold on fewer than 100 generations per subtask on average, under two rounds of revision. Capping refinement at round R shows how quickly the feedback pays off.

Every score above is backed by its raw data. Click a subtask to open its fan of 50 trajectories, then any node for the verifier's full turn-by-turn review, or the icon for the program that round submitted and the reasoning that wrote it. The badge beside a problem's score opens its merged final submission.