Where Measure sits#
AI RMF 1.0 (NIST AI 100-1, January 2023) [1] defines four functions. Govern sets culture, policy and accountability. Map establishes context and identifies risks. Measure analyses, assesses, benchmarks and monitors those risks with quantitative and qualitative methods. Manage prioritises and acts on them. NIST AI 600-1, the Generative AI Profile published in July 2024 [2], adds suggested actions specific to generative AI, tagged by function (MS for Measure), including red-teaming actions under Measure such as assessing resilience to prompt injection and other attacks.
The checklist#
Each item is phrased as something that can be demonstrated with an artefact.
Metrics and methods
- Every risk identified in Map has at least one metric, with a written definition, dataset, grader and statistical method. Artefact: eval plan.
- Metrics are reported with uncertainty (intervals) and sample sizes justified by the smallest change that matters. Artefact: release report.
- Helpfulness and over-refusal are measured alongside harm, so trade-offs are visible.
Measurement validity
- Automated graders are validated against qualified human labels per category, with sensitivity, specificity and drift checks. Artefact: grader validation report; see LLM-as-a-judge.
- Evaluation data is versioned, representative of deployment, and separated from training and prompt-tuning data.
Adversarial testing
- Red teaming covers prompt injection (direct and indirect), jailbreaks, data leakage and misuse relevant to the use case, mapped to a taxonomy. Artefact: red-team report.
- Testing is repeated after fine-tuning and model updates, to check that safety controls weren't weakened.
- Adversarial testing recurs on a schedule, not only before launch.
Monitoring in deployment
- Production traffic is sampled and scored with the same graders; drift thresholds trigger review.
- Guardrail effectiveness is tracked: block rates, false-positive rates, bypasses found.
- Incidents and user feedback feed back into Map and into the eval suite.
Common gaps we see#
- Measure filled with policies. A policy that says "we evaluate for bias" isn't a measurement. The artefact is the result table.
- One-off red teaming before launch, never repeated after the model was swapped.
- Unvalidated automated graders producing numbers nobody can defend. See the judge error analysis.
- No link back to Map. Metrics exist, but nobody can say which risk each one addresses.
Frequently asked questions#
What is the NIST AI RMF Measure function?
Measure is one of the four core functions of the NIST AI Risk Management Framework. It covers using quantitative and qualitative methods to analyse, assess, benchmark and monitor AI risk and related impacts identified in the Map function.
What is NIST AI 600-1?
NIST AI 600-1 is the Generative AI Profile of the AI RMF, published in July 2024. It describes risks specific to or exacerbated by generative AI and suggests actions, organised by AI RMF function, including red teaming and evaluation actions under Measure.