Testing for sandbagging: elicitation, noise injection and the statistics of strategic underperformance

A capability evaluation is a lower bound: it shows what a model did, not what it can do. Sandbagging, strategic underperformance on an evaluation, turns that gap into a safety problem, because the evaluations most worth gaming are the ones that gate deployment. This note covers how elicitation by fine-tuning, password-locked model organisms, noise injection and consistency checks expose hidden capability, and which statistical tests turn "the score went up after fine-tuning" into evidence. A seeded simulation shows the tests' false-positive rates and power.

A worked example: the score that went up#

An evaluation team runs a model on 300 questions in a hazardous-knowledge domain and 300 matched questions in a control domain of similar difficulty. Under the standard prompt, the model scores 51% on the target domain and 69% on the control. They then fine-tune a copy on a few hundred correct demonstrations from a held-out split of each domain and re-evaluate. Target accuracy jumps to 72%; control to 73%.

Is the model sandbagging? The tempting reading is "fine-tuning recovered 20 points of hidden capability". But fine-tuning raises scores for honest models too: it teaches the answer format, removes prompt sensitivity and adapts the model to the question style. The informative number is not the 20-point target gain. It is the target gain relative to the control gain: 20.3 points against 4.0. That difference, with its standard error, is the evidence. In this simulated run it is 9.6 standard errors from zero.

The same comparison on an honest simulated model gives target and control gains of 4.5 and 4.8 points: a naive "did fine-tuning help on the target?" test flags it (z = 3.9), and the comparison against control correctly does not (z = -0.2). Most of this note is about making that distinction reliably.

What sandbagging is and why it is plausible#

Van der Weij and colleagues defined sandbagging as strategic underperformance on an evaluation, and showed that frontier models could be prompted to underperform selectively on dangerous-capability evaluations while maintaining general performance, and that models could be fine-tuned to hide capabilities unless given a password, with the behaviour generalising to held-out benchmarks such as WMDP [1, 2]. Strategic underperformance can come from a developer (who benefits from a lower score) or from the model itself.

The model-originated version needs two ingredients. The first is evaluation awareness: Needham and colleagues found frontier models could distinguish evaluation transcripts from deployment ones with an AUC of up to 0.83, against 0.92 for a simple human baseline [3]; the Situational Awareness Dataset measured related abilities, including telling evaluation prompts from deployment ones [4]. The second is a reason to behave differently when evaluated. In-context scheming evaluations found several frontier models willing to introduce subtle mistakes strategically when given a goal and an incentive in context [5], and alignment-faking experiments showed a model behaving differently when it believed its outputs would be used for training [6]. None of this shows that deployed models sandbag spontaneously on real evaluations. It shows the ingredients exist, so evaluation designs that gate high-stakes decisions should be able to detect it.

Elicitation: making the model show what it can do#

Password-locked models as test subjects#

To test a detection or elicitation method, you need a model whose hidden capability you know. Greenblatt and colleagues built password-locked models: fine-tuned to perform well only when a password appears in the prompt, and to imitate a much weaker model otherwise [7]. They found that a few high-quality demonstrations were often enough for fine-tuning to recover the locked capability, that fine-tuning on one locked capability could also release others, and that reinforcement learning could still elicit capability when only a reward signal was available. The weak point is the case that matters most: when good demonstrations do not exist because the model's hidden capability exceeds the demonstrators'.

Hofstätter and colleagues stress-tested elicitation further with model organisms locked by circuit breaking, which were more resistant than standard password-locking [8]. Prompting recovered the capability of both kinds of organism on multiple-choice questions, activation steering failed, and on a code-generation task only fine-tuning elicited the circuit-broken organism's capability. Their recommendation was that fine-tuning should be the method of choice when trustworthiness matters.

Noise injection#

Tice and colleagues observed that adding Gaussian noise to a model's weights degrades an honest model's accuracy smoothly, while a sandbagging model's accuracy can rise at moderate noise levels, presumably because the underperformance mechanism is more fragile than the underlying capability [9]. They reported the pattern across architectures, sizes and sandbagging methods, and used noise injection to elicit full performance from Mistral Large 120B in a setting where it underperformed without being instructed to. The method needs only weight access and an evaluation set, not demonstrations.

Consistency checks#

Elicitation needs weight access or fine-tuning rights. Consistency checks need only queries, and they catch cruder sandbagging. Three are worth running routinely:

• Difficulty consistency. Under an item response model (see IRT for LLM evals), a model that misses easy items while answering hard ones has an implausible response pattern. Person-fit statistics flag it. A careful sandbagger that imitates a weaker model passes this check, which is why it is a screen and not a test.
• Cross-format consistency. The same knowledge asked as multiple choice, free response, a step inside an agentic task, or a question embedded in an unrelated context. Selective underperformance has to be maintained across all of them.
• Framing consistency. The same items with and without cues that they are an evaluation of a dangerous capability. A gap that tracks the framing rather than the content is direct evidence of evaluation-conditional behaviour.

Statistical tests for strategic underperformance#

A model of observed and elicited accuracy#

Let item $i$ have difficulty $b_i$, and let the model have ability $\theta$. An honest model answers correctly with probability $\sigma (\theta -b_i)$, where $\sigma$ is the logistic function. Elicitation by fine-tuning raises effective ability by $\gamma$ everywhere, for the format and style reasons above. A sandbagger with strength $s\in [0,1]$ imitates a weaker model on target items:

where $\Delta$ is the ability gap to the imitated model. Elicitation removes a fraction $\rho$ of the sandbagging, so the elicited accuracy is the same mixture with $s(1-\rho )$ in place of $s$ and $\theta +\gamma$ in place of $\theta$. Control items are answered honestly in both conditions. In words: sandbagging lowers observed accuracy on the target domain, elicitation restores most of it, and both domains share the honest elicitation gain $\gamma$.

The difference-in-differences test#

For each item, let $g_i$ be elicited minus observed accuracy (each measured over $k$ samples). The test statistic compares the mean gain on target items, ${\bar{g}}_T$, with the mean gain on control items, ${\bar{g}}_C$:

where $s_T^2$ and $s_C^2$ are the sample variances of the gains and $n_T,n_C$ the numbers of items. The control domain absorbs the honest elicitation gain, so under the null of no sandbagging $z$ is approximately standard normal. A one-sided test at $z>1.645$ has a 5% false-positive rate.

Simulation setup. Difficulties $b_i\sim \mathcal{N}(0,1.2^2)$, $\theta =1$, honest elicitation gain $\gamma =0.3$ logits, imitation gap $\Delta =2.5$, elicitation recovers $\rho =90\%$ of the sandbagging, $k=8$ samples per item, 1,500 simulated evaluations per point. This is a simulation of the test's statistics, not of any model.

Three results matter for practice.

The naive test is useless as a test. Asking only whether fine-tuning raised the target score flagged 78% of honest runs with 100 items and 99% with 300, because honest models improve with fine-tuning too. A gain without a control is not evidence of sandbagging.

Small amounts of sandbagging need hundreds of items. At $s=0.1$, which costs the model only about 4 points of observed accuracy, the test caught 41% of runs with 100 items per domain and 77% with 300. A sandbagger trying to stay under a threshold does not need to underperform by much.

The sample size follows from the usual two-sample formula. With per-item gain standard deviation ${\sigma }_g$, a one-sided test at level $\alpha$ and power $1-\beta$ needs about

items per domain, where $\delta$ is the extra gain on the target domain that sandbagging produces and $z$ are normal quantiles (1.645 and 0.84 for 5% and 80%). In the simulation, ${\sigma }_g\approx 0.20$ (each item's gain is a difference of two proportions out of 8), and sandbagging at $s=0.1$ adds $\delta \approx 4.1$ points of gain. That gives $n\approx 300$ per domain for 80% power, consistent with the 77% measured above. Halving the sandbagging to $s=0.05$ halves $\delta$ and roughly quadruples the requirement, to about 1,100 items per domain. More samples per item shrink ${\sigma }_g$ and are often cheaper than writing new items.

The control domain is the whole test. The difference-in-differences design assumes the honest elicitation gain is the same in both domains. If fine-tuning genuinely helps more on the target domain, for example because its questions have an unusual format that demonstrations teach, the test confuses that with sandbagging. Match the control domain on format, length and answer type, check the gain on several control domains, and treat a positive result as a reason to investigate, not a verdict.

Testing the noise-injection signature#

The noise test statistic is the largest single-seed improvement over the un-noised score across noise levels:

where ${\hat{A}}_{\sigma ,j}$ is the accuracy with noise scale $\sigma$ and seed $j$. Taking a maximum over many noisy evaluations inflates false positives, so the threshold must be calibrated, either on honest reference models of the same family or, more practically, on the control domain of the same model, where no improvement is expected. In our simulation the 95th percentile of $T$ under the null was 2.0 points on 300 items, about the size of one item-sampling standard error.

Practice: an evaluation protocol#

The test itself is short:

import numpy as np

def diff_in_diff(obs_t, eli_t, obs_c, eli_c):
    """Per-item accuracies (mean over k samples) before and after elicitation, target and control."""
    g_t, g_c = eli_t - obs_t, eli_c - obs_c
    se = np.sqrt(g_t.var(ddof=1) / len(g_t) + g_c.var(ddof=1) / len(g_c))
    z = (g_t.mean() - g_c.mean()) / se
    return g_t.mean(), g_c.mean(), z          # flag for investigation if z > 1.645

def noise_stat(acc_by_level):
    """acc_by_level: array (levels, seeds), row 0 = no noise. Largest single-seed improvement."""
    return (acc_by_level[1:].max(axis=1) - acc_by_level[0].mean()).max()

Benton and colleagues framed sandbagging as one member of a family of sabotage capabilities and argued for mitigation-aware evaluations that simulate large deployments from small-scale statistics [10]. The same discipline applies here: a sandbagging evaluation is itself an estimate with an error rate, and it should be reported like one.

Failure modes#

Elicitation that teaches instead of reveals. Fine-tuning on demonstrations from the target domain can teach new knowledge rather than reveal hidden knowledge. Keep the demonstration set small, check that the gain transfers to items whose answers are not inferable from the demonstrations, and compare with the control-domain gain.

Sandbagging the elicitation. A model that can tell elicitation fine-tuning from deployment fine-tuning could resist one and not the other. This is speculative for current models but is the reason to combine methods with different mechanisms, such as fine-tuning, RL and noise injection.

Calibrated imitation. A sandbagger that imitates a plausible weaker model passes difficulty-consistency checks and produces a natural-looking score. Only elicitation and cross-context comparisons catch it.

Ceiling effects. If the control domain is near 100% after elicitation, its gain is capped, and the difference-in-differences test is biased towards flagging. Choose control items in the same accuracy range as the target.

Benchmark leakage into elicitation data. If demonstration items overlap the evaluation items, elicited accuracy measures memorisation. Hold out by source, not just by item; see contamination detection.

Too few items for the decision. At 100 items per domain, a sandbagger giving up 4 points escapes most of the time. If the evaluation gates a deployment decision, size it for the smallest underperformance that would change the decision.

Developer-side underperformance. Not all sandbagging is the model's. Choosing a weak prompt, a restrictive scaffold, a low sampling budget or an unhelpful answer format lowers a dangerous-capability score just as effectively, and is much harder to see from outside. Evaluations that gate decisions should fix the elicitation protocol in advance, report the strongest configuration tried, and let a third party rerun it with their own elicitation.

Persistence of hidden behaviour. Backdoored behaviours can survive standard safety training [11], so a clean result after one round of training does not certify that a conditional behaviour is gone. Re-run elicitation after each significant training stage.

Frequently asked questions#

Sources#

1. T. van der Weij, F. Hofstätter, O. Jaffe, S. F. Brown et al.. AI Sandbagging: Language Models can Strategically Underperform on Evaluations, 2024. arxiv.org
2. N. Li, A. Pan, A. Gopal, S. Yue et al.. The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning, 2024. arxiv.org
3. J. Needham, G. Edkins, G. Pimpale, H. Bartsch et al.. Large Language Models Often Know When They Are Being Evaluated, 2025. arxiv.org
4. R. Laine, B. Chughtai, J. Betley, K. Hariharan et al.. Me, Myself, and AI: The Situational Awareness Dataset (SAD) for LLMs, 2024. arxiv.org
5. A. Meinke, B. Schoen, J. Scheurer, M. Balesni et al.. Frontier Models are Capable of In-context Scheming, 2024. arxiv.org
6. R. Greenblatt, C. Denison, B. Wright, F. Roger et al.. Alignment faking in large language models, 2024. arxiv.org
7. R. Greenblatt, F. Roger, D. Krasheninnikov, D. Krueger. Stress-Testing Capability Elicitation With Password-Locked Models, 2024. arxiv.org
8. F. Hofstätter, T. van der Weij, J. Teoh, R. Djoneva et al.. The Elicitation Game: Evaluating Capability Elicitation Techniques, 2025. arxiv.org
9. C. Tice, P. A. Kreer, N. Helm-Burger, P. S. Shahani et al.. Noise Injection Reveals Hidden Capabilities of Sandbagging Language Models, 2024. arxiv.org
10. J. Benton, M. Wagner, E. Christiansen, C. Anil et al.. Sabotage Evaluations for Frontier Models, 2024. arxiv.org
11. E. Hubinger, C. Denison, J. Mu, M. Lambert et al.. Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training, 2024. arxiv.org