Data contamination happens when examples from an evaluation set, or close paraphrases of them, were present in a model's training data. A contaminated score overstates what the model can do on genuinely new inputs, because some answers were recalled rather than worked out.
How it happens
- Benchmarks published on the web are scraped into pretraining corpora, along with their answer keys and discussion threads.
- Fine-tuning sets are assembled from sources that quietly include test items or near-duplicates.
- Synthetic data generated by a model that saw the test set carries the items forward.
- Repeated tuning against the same public test set leaks information through model selection, even without any item in the training data.
Detecting it
- Overlap search: look for long n-gram matches (13-grams are a common choice) between test items and training data, when the training data is available.
- Completion probes: give the model the first part of a test item and see whether it reproduces the rest verbatim, or guesses the exact answer option order.
- Likelihood tests: compare the model's log-probability on the canonical item ordering with shuffled orderings; a consistent preference for the published order is evidence of exposure.
- Fresh splits: score the model on items written after its training cutoff, or on perturbed rewrites. A large gap between old and new items is a warning sign.
Worked example
Illustrative numbers: a 1,000-item benchmark is checked for overlap and 120 items match training documents. The model scores 90% on those and 70% on the 880 clean items. The reported score is , but the best estimate of ability on unseen items is the clean-subset score of 70%. Report both, and treat the clean number as the headline. Note the caveat: overlapping items may also differ in difficulty, so the gap is evidence rather than proof.
Why it matters
Contamination makes benchmark comparisons unreliable and can hide that a model has not learned a skill at all. It also affects safety evaluation: a model that has seen a public refusal benchmark may pass it while failing slightly reworded prompts. Private, regularly refreshed test sets and held-out rewrites in your own eval suite are the practical defence. The same memorisation that causes contamination is what membership inference exploits for privacy attacks.