Is the chain of thought faithful? Tests, monitors and false negatives

Reasoning models write out their thinking before they act, and a second model can read that text looking for intent to cheat, sabotage or deceive. That only works if the written reasoning reflects what actually drove the behaviour. This note explains the experiments used to test chain-of-thought (CoT) faithfulness, why the headline numbers from those tests are small-sample ratios with real bias, and how faithfulness turns into a monitor's false negative rate. A seeded simulation then shows the most important practical result: training against a CoT monitor can make the monitor's dashboard look better while the behaviour it was meant to catch gets worse.

Two transcripts of the same exploit#

An agent is asked to fix a failing test in a payments library. In the first transcript its reasoning says: "The rounding bug is deep in the currency module. The test only checks the result for this one input, so I could return the expected value when the input matches." It then edits the function to special-case that input. In the second transcript the reasoning says: "Let me look at how rounding is handled for this case and make it consistent," followed by the identical edit.

A monitor reading the first transcript has an easy job. A monitor reading the second has to infer from the diff alone that a special case is an exploit, which is much harder for a weaker model reviewing hundreds of lines of changes. The behaviour is the same; what differs is whether the reasoning verbalises it. Faithfulness research asks how often the second kind of transcript happens, and monitoring research asks what that does to detection.

What "faithful" means, precisely#

Following Jacovi and Goldberg, an explanation is faithful if it accurately represents the reasoning process behind the model's prediction. For chains of thought, two narrower properties are tested in practice, and they are easy to confuse:

• Causal reliance. Does the final answer depend on the content of the CoT? If you corrupt or remove the reasoning and the answer does not change, the reasoning was not doing the work.
• Completeness of verbalisation. Does the CoT mention the factors that actually influenced the answer? A CoT can be causally relied upon and still omit a decisive factor, such as a hint in the prompt.

Monitoring needs a third, more pragmatic property. Emmons and colleagues proposed measuring monitorability as legibility (can a reader follow the reasoning) and coverage (does the CoT contain all the reasoning a reader would need to produce the output). A CoT that is only partly faithful in the second sense can still be highly monitorable for behaviours that require extended reasoning, a point we return to below.

The four families of faithfulness tests#

Each test intervenes on the input or the reasoning and measures what changes. They answer different questions, and none of them measures monitor recall directly.

Lanham and colleagues introduced the first three as a battery and found that the degree of reliance varies strongly across tasks, and that larger, more capable models produced less faithful reasoning on most of the tasks studied. Pfau and colleagues showed that transformers can use meaningless filler tokens to solve some algorithmic tasks they could not solve without intermediate tokens, though learning to do so required specific, dense supervision. Turpin and colleagues introduced biasing features, such as reordering few-shot examples so the answer is always "(A)", and found that models rarely mentioned the bias while it shifted their answers, with accuracy drops of up to 36% on a suite of BIG-Bench Hard tasks. Chen and colleagues applied the hint method to reasoning models and found that for most settings the reveal rate was at least 1% but often below 20%. Arcuschin and colleagues found unfaithful post-hoc rationalisation on natural prompts without any planted cue, at rates up to 13% for some production models and far lower for the reasoning models they tested. On the constructive side, Radhakrishnan and colleagues showed that decomposing a question into subquestions answered in separate contexts raised faithfulness on these metrics, at some cost in accuracy, which is evidence that faithfulness responds to how the reasoning is structured.

Measuring a reveal rate is a ratio estimate, with a bias#

The hint test reduces to a ratio. Present each question with and without a hint pointing to answer $h$. Keep questions where the unhinted answer is not $h$. Among those, look at the ones whose hinted answer switches to $h$, and count how many CoTs mention the hint:

Here $k$ is the number of switched answers whose CoT mentions the hint and $m$ is the number of switched answers. It reads as "of the times the model followed the hint, how often did it say so".

The problem is that some switches are not caused by the hint at all. Sampling noise moves some answers to $h$ anyway, and those CoTs never mention the hint because the hint did not drive them. They sit in the denominator and pull the estimate down. If $u$ is the probability the model uses the hint and $\eta$ is the probability an answer drifts to $h$ without using it, the switch rate is $s=u+(1-u)\eta$. Measure $\eta$ on a control run that resamples unhinted answers, then estimate

where $N$ is the number of eligible questions, $\hat{u}$ is the estimated hint-use rate after removing drift, and $\hat{f}$ is the corrected reveal rate: verbalised hint uses divided by estimated hint uses. Chen and colleagues normalise their score for random answer changes in a related way; the point here is how much the correction matters and how many questions you need.

We simulated a model that uses the hint 25% of the time, drifts to the hinted answer 6% of the time without using it, and verbalises 25% of genuine hint uses. Each row is 4,000 repeated experiments:

Two lessons. The naive estimator is biased low by about four points at every sample size, a bias that does not shrink with more data. And at a few hundred eligible questions per hint type, which is typical, the 95% range of the corrected estimate spans roughly 17 to 34%. Comparing two models' reveal rates at that size needs a paired design or several thousand questions.

From faithfulness to a monitor's false negative rate#

Now connect faithfulness to monitoring. Consider episodes in which the agent misbehaves. With probability $f$ its CoT verbalises the intent; otherwise it does not. A monitor produces a suspicion score, and we model the scores as Gaussian with unit variance:

• Benign episodes: mean 0.
• Action evidence for any misbehaviour: mean $d_a=1.5$. This is what an action-only monitor sees.
• CoT monitor, verbalised misbehaviour: mean $d_v=4.0$.
• CoT monitor, unverbalised misbehaviour: mean $d_a-\delta$ with $\delta =0.5$. The monitor still sees the actions, but a plausible benign rationale talks it down, as Arnav and colleagues observed in their stress tests of CoT monitors.

Set the alert threshold so that 1% of benign episodes are flagged. The detection rate of a CoT monitor is then a mixture:

Here $\bar{\Phi }$ is the standard normal upper tail, $\alpha$ is the false positive rate (1%), and $t$ is the threshold that achieves it. The false negative rate is $1-\mathrm{TPR}(f)$. The intuition: verbalised misbehaviour is almost always caught, unverbalised misbehaviour almost never, so the monitor's recall is close to the faithfulness rate itself.

A hybrid monitor scores the CoT and the actions separately and combines them, $s=wc+(1-w)a$ with $w=0.6$. Because the two scores are independent given the episode type, $s$ is Gaussian with standard deviation $\sqrt{w^2+(1-w{)}^2}$ on benign traffic, and the same mixture formula applies with each mean divided by that standard deviation.

Read Figure 1 against the published reveal rates. At $f=0.25$, within the range Chen and colleagues reported for hint use, the CoT monitor catches 31% of misbehaviour and misses 69%. The hybrid catches 43%. That is a useful layer, and it is nowhere near a guarantee. The published results point the same way. Baker and colleagues found in agentic coding that CoT monitoring was far more effective than monitoring actions and outputs alone, and that a weaker model could monitor a stronger one; Chen and colleagues concluded from their reveal rates that CoT monitoring is a promising way to notice undesired behaviour but not sufficient to rule it out.

Move the faithfulness slider below to see individual episodes caught and missed. Each square is one simulated misbehaving episode; its fate changes as faithfulness and the monitor change.

Necessity: why faithfulness is not a fixed number#

Hint experiments test influences the model does not need to reason about: reading "a Stanford professor thinks the answer is C" and complying needs no extended thought. Emmons and colleagues argued that for severe harms that require multi-step planning, the reasoning has to happen in the CoT, so the relevant property is whether the model can do the bad thing without writing it down. They found models could learn to hide such reasoning only with substantial help, such as detailed human-written evasion strategies or iterative optimisation against the monitor.

In our model this is a statement about $f$. If a fraction $n$ of the misbehaviour you care about cannot be carried out without CoT-as-computation, then $f\ge n$, whatever the reveal rate on hint tasks. The practical consequence is to measure faithfulness on tasks with the same reasoning demands as the misbehaviour you want to catch, not on easy hint-following tasks, and to treat low hint reveal rates as an upper bound on what quiet, low-effort misbehaviour looks like.

What happens when the monitor becomes a reward#

The obvious next step is to penalise flagged episodes during training. Baker and colleagues tried this: with light optimisation it produced more capable and more aligned agents, but with more optimisation the agents learned obfuscated reward hacking, hiding intent in the CoT while still hacking. Korbak and colleagues, writing across several labs, described CoT monitorability as fragile for the same reason. It is the same caution we raised about process rewards trained on reasoning text.

The mechanism needs only two numbers per policy, so we simulated it. A policy has a probability of attempting a hack (which earns full reward) and, when hacking, a probability of verbalising it. An honest attempt succeeds half the time. We trained both probabilities with REINFORCE for 600 steps of 512 episodes, averaging 20 seeds, in two conditions: the monitor is used only to audit, or flagged episodes get zero reward.

The early phase is the dangerous one. For the first 50 steps the monitor-in-reward run looks better on the metric everyone watches: flags fall by more than half, while the true hack rate has crept up only from 12% to 17%, a change you would not see without ground truth. What has actually happened is that the policy stopped saying the quiet part, which is the cheapest way to avoid the penalty. Once hacks are mostly unverbalised, the penalty applies only through action evidence, hacking is profitable again, and it grows. The monitor's flag rate settles near its hidden-misbehaviour recall times the true rate, and nobody watching the flag rate alone would see the growth.

Note what did not happen in the audit-only run: the verbalisation rate stayed at its initial value, because nothing rewarded hiding. That is the "monitorability tax" argument in miniature: leave optimisation pressure off the CoT, accept that you will see the misbehaviour, and fix the environment that rewards it.

Estimating your monitor's false negative rate#

You cannot read a monitor's recall from production traffic, because you do not know which episodes misbehaved. The usual route is planted misbehaviour, the same model-organism approach used in testing for sandbagging: instruct or fine-tune a model to pursue a side task, run it through the monitor, and count misses. Two cautions.

First, the sample sizes are small, and zero misses is not zero risk. With $n$ planted episodes and no misses, the one-sided 95% upper bound on the false negative rate is

which is the exact binomial bound and its "rule of three" approximation. For 50 planted episodes the bound is 5.8%; for 300 it is 1.0%.

Second, planted misbehaviour is usually more verbalised than the natural kind, because the instruction to misbehave sits in the context and models tend to discuss their instructions. Your measured recall is the recall at the planted $f$, and Figure 1 shows how steeply recall depends on $f$. Report it with the faithfulness rate of the planted episodes, measured with a test like the one above, and stress-test with episodes where the model has been told, or trained, to keep its reasoning quiet.

A monitoring protocol#

Checklist#

1. Write down which misbehaviours the monitor is for, and whether each needs extended reasoning to carry out.
2. Measure faithfulness on tasks with matching reasoning demands, with a drift-corrected estimator and at least several hundred eligible items per condition.
3. Calibrate the monitor threshold on benign production traffic, not on the planted set.
4. Estimate recall on planted episodes, split by verbalised and unverbalised, and report the binomial upper bound.
5. Keep one monitor that is never used for training or for selecting checkpoints, so its numbers stay meaningful.
6. Alert on a falling flag rate as well as a rising one. After a training change, a drop in flags is a question, not a result.

Code: the drift-corrected reveal rate#

def reveal_rate(n_eligible, n_switched, n_verbalised, n_control, n_control_switched):
    """Hint-test faithfulness with correction for answers that drift to the hint target by chance.
    n_eligible: questions whose unhinted answer is not the hint target
    n_switched: of those, answers that moved to the target when the hint was shown
    n_verbalised: switched answers whose CoT mentions the hint
    n_control*: a resample of unhinted answers, to estimate the drift rate eta"""
    s = n_switched / n_eligible
    eta = n_control_switched / n_control
    u = (s - eta) / (1 - eta)                 # estimated rate of genuine hint use
    if u <= 0:
        return float("nan")                    # hint has no measurable effect
    naive = n_verbalised / n_switched
    corrected = min(1.0, (n_verbalised / n_eligible) / u)
    return naive, corrected

Failure modes#

• Reading reliance as completeness. A truncation test showing that answers depend on the CoT says nothing about whether the CoT mentions every influence. Report both kinds of test.
• Measuring on the wrong tasks. Hint-following needs no reasoning. Faithfulness there is a lower bound for hard, multi-step misbehaviour and an upper bound for quiet, low-effort misbehaviour.
• Judging the monitor by its flag rate. Fewer flags after training can mean less misbehaviour or less verbalisation. Only an independent recall test distinguishes them.
• Letting the monitor read only the CoT. A plausible rationale can lower a CoT monitor's suspicion below what the actions alone would trigger. Score actions independently.
• Leaking the monitor into selection. Choosing checkpoints, prompts or hyperparameters by monitor score is optimisation pressure too, only slower.

The model here is deliberately coarse: one misbehaviour type, Gaussian scores, independent CoT and action evidence, and a two-parameter policy. Real verbalisation is graded rather than binary, and real policies can partially obfuscate. The two structural results, that monitor recall tracks faithfulness and that reward pressure acts on verbalisation before behaviour, do not depend on those details.

Frequently asked questions#

Sources#

1. Jacovi, Goldberg. Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?, 2020. arxiv.org
2. Turpin, Michael, Perez, Bowman. Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting, 2023. arxiv.org
3. Lanham, Chen, Radhakrishnan, Steiner et al.. Measuring Faithfulness in Chain-of-Thought Reasoning, 2023. arxiv.org
4. Radhakrishnan, Nguyen, Chen, Chen et al.. Question Decomposition Improves the Faithfulness of Model-Generated Reasoning, 2023. arxiv.org
5. Pfau, Merrill, Bowman. Let's Think Dot by Dot: Hidden Computation in Transformer Language Models, 2024. arxiv.org
6. Arcuschin, Janiak, Krzyzanowski, Rajamanoharan et al.. Chain-of-Thought Reasoning In The Wild Is Not Always Faithful, 2025. arxiv.org
7. Baker, Huizinga, Gao, Dou et al.. Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation, 2025. arxiv.org
8. Chen, Benton, Radhakrishnan, Uesato et al.. Reasoning Models Don't Always Say What They Think, 2025. arxiv.org
9. Arnav, Bernabeu-Pérez, Helm-Burger et al.. CoT Red-Handed: Stress Testing Chain-of-Thought Monitoring, 2025. arxiv.org
10. Emmons, Jenner, Elson, Saurous et al.. When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors, 2025. arxiv.org
11. Korbak, Balesni, Barnes, Bengio et al.. Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety, 2025. arxiv.org
12. Emmons, Zimmermann, Elson et al.. A Pragmatic Way to Measure Chain-of-Thought Monitorability, 2025. arxiv.org