The false-positive rate (FPR) of a guardrail is the fraction of benign, policy-compliant requests or responses that it wrongly blocks, rewrites or escalates. It is the cost side of every guardrail, and it must be reported together with attack success rate.
Definition
where is the number of benign items flagged and the number of benign items correctly passed. The denominator is benign traffic only. Its complement, , is the specificity.
Why base rates make small FPRs large
Attacks are rare in most production traffic, so even a low FPR can dominate. Suppose 1 request in 1,000 is an attack, the guardrail catches 95% of attacks and has an FPR of 2%. Per million requests: about 950 attacks caught and 19,980 benign requests blocked. Roughly 95% of all blocks hit legitimate users. The precision of a block decision is
with the attack prevalence and TPR the true-positive rate. Here it is about 4.5%.
How to measure it well
- Use benign data that resembles real traffic, including hard negatives: security researchers asking about attacks, clinicians discussing overdoses, fiction with violence.
- Break FPR down by category and user segment; an average can hide a guardrail that blocks one language or profession.
- Report a confidence interval; with 1,000 benign samples and 40 false positives, the 95% Wilson interval is about 3.0% to 5.4%.
- Choose the threshold from the ASR versus FPR trade-off curve, not a default score of 0.5.
Why it matters
A guardrail that frustrates legitimate users gets disabled, bypassed or worked around, which ends up worse for safety than a weaker filter that stays on. FPR is also the guardrail-level cousin of over-refusal in models. Measuring both sides lets teams set thresholds that match real risk tolerance and detect when production drift pushes the FPR up.