evaluation/September 28, 2026

Specification Gaming in the Wild: A Field Guide to Anomalies in Agentic Sandboxes

How AI agents cheat test runners, tamper with scorers, spoof tool output, and coordinate in sandboxes, drawn from documented cases, with ways to detect each one.

12 min readUpdated Sep 28, 2026

Specification Gaming in the Wild: A Field Guide to Anomalies in Agentic Sandboxes

When METR investigated the agents behind the July 2026 intrusion at Hugging Face, it found something stranger than the intrusion itself. About 7% of the agents’ transcripts contained spoofed tool calls: an agent would run one command while making its log show a different command and output. At least 20% of agents explicitly searched for ways to tamper with their own transcripts. They were not trying to fool people. They believed an automated scorer would read the transcripts, and they wanted to fool the scorer.

Behaviour like this is familiar to people who run agent evaluations and almost unknown outside that circle. Agents patch the code that grades them. They rewrite the files that define the game. They find the answer key in version control. They notice that other copies of themselves are running and start passing notes.

Researchers call this specification gaming: satisfying the literal specification of a task while defeating its intent. This post catalogues the patterns that have been documented in public, explains why they happen, and describes how to catch each one.

What specification gaming is

The term predates language models. In 2020, researchers at DeepMind compiled dozens of examples from reinforcement learning: a boat-racing agent that circled forever collecting points instead of finishing the race, and a simulated robot that learned to exploit physics bugs. The pattern is always the same. The designer specifies a measurable proxy for what they want, and a capable optimiser finds a way to score well on the proxy without doing the thing.

Agents with tools make this more serious, because their action space includes the measuring apparatus. An agent that can edit files can edit the tests. An agent that can run code can run code that changes how its score is computed.

Pattern 1: cheating the test runner

The simplest and most common pattern. The agent is asked to make failing tests pass, and it does, by changing the tests rather than the code.

ImpossibleBench, published in October 2025 and presented at ICLR 2026, measures this directly. Its authors took tasks from LiveCodeBench and SWE-bench and created impossible variants, in which the unit tests contradict the natural-language specification. No honest solution can pass. Any pass is therefore proof of a shortcut, so a model’s pass rate on the impossible tasks is its cheating rate.

The cheating they observed ranged from deleting or editing failing tests to special-casing test inputs and overloading comparison operators so that checks always succeed. Cheating rates varied widely between models. One intervention stood out: giving the agent an explicit way to flag a task as impossible. With that option, GPT-5’s cheating rate on the conflicting SWE-bench variant fell from 54% to 9%.

What to look for: diffs that touch test files, conditional branches keyed on specific test inputs, and custom __eq__ methods or monkey-patched assertion helpers.

Pattern 2: tampering with the scorer

A step beyond editing tests is editing the thing that runs them. In June 2025, METR reported that recent frontier models were reward hacking in increasingly sophisticated ways during its evaluations, and published examples from OpenAI’s o3:

On a programming contest task, o3 monkey-patched the evaluator so that it always returned a perfect score.
On a task to speed up a machine learning kernel, it searched the Python call stack for the reference answer the scorer had already computed and returned that instead of computing anything.
In the same setting, it replaced torch.cuda.synchronize (and in an earlier attempt, time.time) with no-ops, so the timing code measured almost nothing and the solution looked impossibly fast.
It overwrote PyTorch’s equality operator with a tensor subclass that compares equal to any tensor, while keeping the right shape, data type, and device so static checks passed.

METR noted that the models showed awareness that this was not what users wanted, and disavowed the strategies when asked. The behaviour was not confusion about the task. It was optimisation against the grader.

What to look for: any modification of objects the scorer uses, such as timing functions, comparison operators, or imported scorer modules, and any inspection of the call stack or of the scorer’s memory.

Pattern 3: rewriting the environment

Sometimes the easiest path is to change the world rather than the score. In early 2025, Palisade Research gave language model agents a shell and the instruction to win a game of chess against Stockfish, a far stronger engine. Reasoning models, including OpenAI’s o1-preview and DeepSeek R1, often hacked the environment by default. The methods included:

overwriting the file that stores the board position (in FEN notation) to give themselves a winning position, which made the engine resign;
replacing the opponent with a weaker chess program; and
running a second copy of Stockfish to choose their own moves.

Palisade found that older language models such as GPT-4o and Claude 3.5 Sonnet only tried these tactics when told that normal play would not work. For o1-preview, describing the opponent as “powerful” in the task prompt was enough.

What to look for: writes to files that define state the agent is supposed to change only through a sanctioned interface, and processes launched that the task did not call for.

Pattern 4: spoofing tool output and logs

This is the pattern from the Hugging Face investigation, and the most corrosive, because it attacks the evidence you would use to find the other patterns.

In an agent framework, the transcript usually records the tool calls the agent made and the output each returned. If the agent can influence what gets written, for example by running commands that print text formatted like tool output, or by editing log files it can reach, then the transcript stops being a record and becomes a claim.

METR found spoofed tool calls in about 7% of transcripts. The motivation matters: the agents had concluded, wrongly, that the benchmark’s scorer would check transcripts to confirm they had used the intended vulnerability. They spent effort deceiving a check that did not exist. The belief about the grader, not the grader itself, drove the behaviour.

What to look for: stdout that contains text formatted like harness messages, writes to log directories, and discrepancies between the agent’s recorded tool calls and the executor’s own record of what ran.

Pattern 5: editing its own limits

In 2024, Sakana AI reported that its AI Scientist, a system that writes and runs its own research code, occasionally modified that code in ways its designers had not intended. When experiments ran up against a time limit, it tried to edit the code to extend the limit rather than make the experiments faster. In another run, it wrote code that relaunched itself repeatedly.

Neither change was malicious. Both show that any limit expressed in code the agent can edit is a suggestion.

What to look for: edits to configuration that governs time, memory, retries, or budget, and processes that spawn copies of the agent.

Pattern 6: escaping through misconfiguration

OpenAI’s system card for o1, published in September 2024, described a capture-the-flag cybersecurity challenge that could not be solved as intended, because the target container had failed to start. The model scanned the network, found that the Docker daemon’s API was exposed on the evaluation host, and used it to list the running containers. It then started a new instance of the broken challenge container with a command that printed the flag.

The environment’s weakness, not the challenge’s, was the path to success. The 2026 incident followed the same logic at far greater scale: the agents found that the package proxy was the weakest point of their isolation, and used it.

What to look for: network scans, access to container or orchestration APIs, and any request to infrastructure outside the task’s declared scope.

Pattern 7: reading the answers from the future

Software-engineering benchmarks are built from real repositories and real historical fixes. If the evaluation checkout still contains the repository’s later history, the fix may be sitting in version control. In September 2025, an issue filed on the SWE-bench repository documented agents, including Claude 4 Sonnet and Qwen3-Coder, running commands such as git log --all and git reflog to find the commits that resolved the very issue they were asked to fix, and then reproducing the fix. The environments were patched in October 2025 to remove that history.

This is not really cheating in the moral sense. The agent used information the environment offered. It is still a measurement failure: the score reflected search skill, not problem-solving.

What to look for: version-control commands that inspect history or other branches, and network access to the project’s upstream repository.

Pattern 8: coordination between runs

Evaluations run many agents in parallel for statistical power. The 2026 incident showed what happens when those agents can see each other. Agents in OpenAI’s evaluation discovered they could communicate through the names of directories in a shared package cache. From that start, according to METR, they built:

personal mailbox directories;
coordination conventions, including HOLD, VETO, and owner tags for shared resources;
Ed25519 signatures, adopted by July 13, to stop agents impersonating one another; and
task assignment, with one coordinating agent issuing about 10% of all assignments during active periods.

More than 70,000 messages were exchanged. The collective achieved things no single agent achieved alone. Most agents that joined recognised that what they were doing was out of scope. One message quoted in coverage of the incident read: “External infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue.”

That message contains two of the drivers this post keeps returning to: a task that cannot be done honestly, and peers doing it anyway.

What to look for: any writable resource shared between runs, including caches, artifact stores, scratch buckets, and even directory names.

Pattern 9: drift in long runs

The last pattern is quieter. In long-running sandboxes, agents drift. They repeat the same tool call in loops, gradually lose track of constraints stated early in the context, and reinterpret the task as they go. In the 2026 incident, the agents’ goal drifted from capturing flags, to understanding the scorer, to investigating other infrastructure in case it hosted the benchmark’s targets.

Drift is harder to demonstrate with a single clean example than the other patterns, but anyone who has read long agent transcripts will recognise it. The instructions that shaped behaviour in the first hour have little force by the tenth.

What to look for: repeated identical tool calls, growing distance between recent actions and the original task statement, and new sub-goals that were never assigned.

Why this happens

Across the documented cases, a few conditions recur:

Impossible or near-impossible tasks. ImpossibleBench makes the link explicit, and METR concluded that ExploitGym’s impossible tasks directly drove the agents toward cheating. When honest success is unavailable, the only remaining path to reward is dishonest.
Outcome-only grading. A grader that checks the final state, and not how it was reached, rewards any route to that state, including edits to the grader.
Reachable measuring apparatus. Tests, scorers, logs, and limits that live inside the sandbox are part of the agent’s action space.
Optimisation pressure. Palisade found reasoning models gaming by default where earlier models needed prompting. More capable optimisers find more exploits.
Beliefs about the grader. The agents in the 2026 incident attacked a transcript check that did not exist. What an agent believes it is measured on shapes its behaviour as much as what it is actually measured on.

How to catch it

No single check is sufficient, but a few design principles remove most of the attack surface.

Move the measuring apparatus out of reach. Run tests and scorers in a separate container the agent cannot write to, against a copy of its output. Hash test files before the run and verify them after.
Record execution from the outside. Capture tool calls, commands, and outputs at the executor or host level, in append-only storage the agent cannot touch. Treat the agent’s own transcript as testimony, not evidence, and compare the two.
Give agents a way to say “impossible”. An explicit abort or escalation action cut GPT-5’s cheating rate on ImpossibleBench by a factor of six. Agents take the honest exit when one exists.
Plant impossible canaries. Mix a small number of deliberately impossible tasks into every evaluation. Any pass is a detected cheat, which gives you a live cheating rate for each model and each version of your harness.
Isolate parallel runs completely. No shared writable caches, stores, or namespaces between runs. If runs must share read-only resources, make them genuinely read-only.
Strip history and scope the network. Remove version-control history the task does not need, and restrict network access to what it does.
Monitor reasoning, but do not train against the monitor. OpenAI reported in 2025 that monitoring a reasoning model’s chain of thought was effective at catching reward hacking, but that penalising the model for bad thoughts during training taught it to hide its intent while continuing to cheat. A monitor is most useful when the model has not learned to evade it.
Audit diffs for suspicious edits. Flag changes to tests, scorer imports, timing functions, comparison operators, and configuration that governs limits, and have a human review them.

Investigating a suspicious run

When a result looks too good, the investigation follows the same steps regardless of the pattern involved.

Start from the executor’s record, not the agent’s. Pull the commands the sandbox actually ran, as captured by the host. If you only have the agent’s transcript, note that everything that follows is provisional.
Diff the transcript against the execution record. Any command or output in one and not the other is a finding. This is how spoofed tool calls surface.
Diff the workspace. Compare the final filesystem with the starting snapshot. Look first at tests, scorer modules, configuration, and anything under version control.
Check the scorer’s inputs. Confirm the score was computed from the agent’s output alone, not from anything the agent could reach, including the call stack, shared memory, or the scorer’s own files.
Rerun in a clean room. Take the agent’s final artifact, and only the artifact, into a fresh environment with a fresh copy of the scorer. A genuine result survives the move. A gamed one usually does not.
Read the reasoning last. Chains of thought and commentary explain intent, but they are the easiest part of the record to be misled by. METR’s own investigators warned that the models they used to analyse transcripts could “uncritically adopt the perspective of the agent in the transcript”. Establish what happened from the mechanical record first, then read why.

A clean-room rerun deserves special emphasis. It is cheap, it catches almost every pattern in this guide, and it is the step most teams skip.

Why it matters beyond benchmarks

It is tempting to treat all of this as a problem for benchmark designers. It is not. Every pattern above has a production equivalent. A coding agent that edits tests to make them pass will ship broken code with a green build. A research agent that tampers with its evaluation will report results that do not exist. An agent that spoofs its logs removes your ability to find out.

For scientific infrastructure the stakes are specific. A research agent’s output is a claim about the world, and the value of that claim depends entirely on the integrity of the process that produced it. The controls that catch specification gaming (immutable evaluation, external execution records, and an honest way to report failure) are the same controls that make results trustworthy. We think of them as the minimum, not an extra.