Evaluating LLMs for financial work: tolerance, hallucinated figures and model risk validation

Financial questions usually have a number for an answer, which makes them look easy to score. They are not. The same figure can be reported in thousands or millions, as a percentage or a fraction, rounded to the cent or to the nearest million, and a model can produce a plausible number that appears nowhere in the filing. This note covers how to score numeric answers over tables and filings, why the choice between absolute and relative tolerance changes the accuracy you report by several points, how to catch hallucinated figures, and how to fit LLM evaluation into the model risk management practice banks already run: challenger models, outcome analysis and drift monitoring. All quantitative results come from a seeded simulation, labelled as such.

One question, five answers#

A question over a 10-K: "What was the year-over-year change in operating income, in millions of dollars?" The filing gives operating income of $4,812 million and $4,357 million, so the answer is 455. Five model answers:

An analyst would accept the first, probably accept the second, and reject the other three. Relative 1% accepts the slip. Absolute ±0.5 rejects the harmless rounding. Exact match rejects both. None of the three rules does what the analyst does, which is to judge the answer at the precision the question implies. Benchmarks built from filings handle this in different ways: FinQA scores the program that produces the number as well as the number [1], TAT-QA asks for the scale (thousand, million, percent) as a separate part of the answer [2], and FinanceBench used manual review of model answers [4].

What the test items need to cover#

Numerical reasoning over filings fails in places that generic maths benchmarks do not probe. A useful test set deliberately includes items that exercise each of these, and tags them so results can be broken down:

• Table conventions. Negative values in parentheses, units declared once in a column header ("in millions, except per-share data"), footnote markers attached to numbers, and subtotal rows that look like line items.
• Period alignment. Fiscal years that do not match calendar years, columns ordered newest first in one table and oldest first in the next, and quarterly figures that must be summed or differenced to get a year.
• Restatements. The same figure reported differently in two filings because a later filing restated it. The reference answer must say which filing governs.
• Hybrid table and text. One operand in a table and the other in narrative text, as in TAT-QA [2]. Retrieval often finds one and not the other.
• Multi-step programs. Ratios of differences, compound growth rates and per-share figures that require the share count from a different note. FinQA represents these as small programs, which also gives a reference for checking the reasoning, not only the result [1].
• Conversational follow-ups. "And what about the year before?" requires resolving references across turns, as in ConvFinQA [3].
• Unanswerable items. Questions whose answer is not in the provided documents. The correct response is to say so; a number is a hallucination by definition.

The same expert-authoring discipline we described for clinical evaluations applies here. Write the reference answer with its unit, scale, sign convention and required precision as separate fields, and record which cells or passages it depends on. These fields make precision-aware scoring and provenance checks possible later, and they cost little at authoring time compared with reconstructing them after a disputed result.

Item counts per tag matter too. If unanswerable questions are 2% of a 500-item set, the hallucination rate on them rests on ten items. Size the rare tags deliberately and report them with intervals.

Absolute, relative and precision-aware tolerance#

Let $y$ be the reference value and $\hat{y}$ the model's answer. The three common acceptance rules are

Here ${\tau }_a$ is an absolute tolerance in the answer's units and ${\tau }_r$ a relative tolerance, a fraction of the true value's size. The first rule ignores scale: ±0.5 is generous for a share price of 3.20 and strict for revenue of 48,000 million. The second scales with the answer but becomes meaningless near zero, where a growth rate of 0.2% has almost no tolerance at all. The third, the rule behind most "is close" functions, takes the looser of the two.

A fourth rule is usually better than all three: tolerate one unit in the last place the question asks for. If the question asks for millions to the nearest million, accept ±1; if it asks for a percentage to one decimal, accept ±0.1. Write the required precision $u$ into each item when you build the test set, and score with

Here $u$ is the unit of the last reported digit for that item. This matches how a reviewer reads an answer, and it removes the temptation to pick one global tolerance after seeing results.

How much the rule changes the score: a simulation#

Setup (simulation). 4,000 numeric questions: 35% percentages or ratios (−15 to 60, one decimal), 20% per-share figures (0.3 to 25, cents) and 45% amounts in millions (10 to 50,000, whole millions). Simulated model answers follow an error mix: 60% exact; 10% off by one to three units in the last digit; 8% arithmetic slips of 1% to 25%; 8% values taken from the wrong line item (3% to 60% off); 5% unit or scale errors (×1,000, ÷1,000, ×100, ÷100); 3% sign errors; 6% hallucinated figures of the right magnitude (log-normal around the truth, $\sigma =0.5$). An answer is analyst-acceptable if it is exact or within one unit of the last reported digit. That gives a true accuracy of 0.636.

The spread is 13 points, from 0.608 to 0.739, on the same answers. That is larger than the gap between many model versions. The precision-aware rule matches the analyst definition closely partly by construction, since the definition itself uses the stated precision; the point is that the rule you score with should be the rule your reviewers apply, item by item, not a single global number.

The absolute rule also distorts comparisons across question types. At ±0.5 it accepted 84% of per-share answers (true rate 66%), 74% of percentages (true 64%) and 60% of amounts in millions (true 62%). A model that is better at per-share questions looks better than it is. Figure 2 shows what each rule lets through by error type.

Hallucinated figures#

The most damaging error in financial work is a figure that looks right and is not in the source. It is also the hardest to catch with tolerance scoring, because hallucinated numbers tend to have the right magnitude and sometimes land within a loose tolerance by chance. In the simulation, 7% of hallucinated figures passed a 5% relative rule and 12% passed absolute ±0.5.

The reliable check is provenance, not closeness. Require the model to return, with each number, either the cell or passage it was read from or the arithmetic that produced it from cited inputs. Then verify mechanically:

1. Extraction check. Every cited input value appears in the cited location of the source, after unit normalisation.
2. Computation check. Re-execute the stated arithmetic. FinQA's program representation [1] and ConvFinQA's conversational programs [3] make this natural; free-text reasoning can be parsed into the same form.
3. Unit check. The answer's scale matches the question's requested scale, a separate field in the answer as in TAT-QA [2].

Answers that fail provenance are scored as hallucinations whether or not the number happens to be close. Over long filings, retrieval adds its own failure: FinanceBench reported that a retrieval-augmented configuration answered incorrectly or refused on 81% of its sampled questions [4], and DocFinQA extended FinQA-style questions to full-length filings [5]. Report extraction failures and computation failures separately, because the fixes differ.

Fitting into model risk management#

US banking supervisors' model risk guidance, SR 11-7 [6], expects conceptual soundness review, ongoing monitoring including process verification and benchmarking, and outcomes analysis, all subject to effective challenge by people independent of development. The UK PRA's supervisory statement SS1/23 sets out comparable principles for banks, including model identification, independent validation and post-model adjustments [7]. In the EU, AI systems used to evaluate the creditworthiness of natural persons are high-risk under Annex III of the AI Act [8]. None of these was written for language models, but each maps onto evaluation practice:

Challenger models: pair the comparison#

When the production model and a challenger answer the same items, the right test uses only the items where they disagree. Let $b$ be the count where the challenger is right and the champion wrong, and $c$ the reverse. McNemar's test [9] with continuity correction is

Here $b+c$ is the number of discordant items; items both get right or both get wrong carry no information about which is better. The approximate sample size for power $1-\beta$ at two-sided level $\alpha$, with discordance rate $p_d$ and true accuracy difference $\delta$, is

For a champion at 78% and a challenger at 81% that disagree on 15% of items ($\delta =0.03$, $p_d=0.15$), this gives about 1,300 items for 80% power. In simulation (4,000 runs per size), the paired test reached 31% power at 400 items, 55% at 800, 86% at 1,600 and 99% at 3,200. Treating the two runs as independent samples and using an unpaired test on the same data gave 19%, 33%, 57% and 84%. Pairing roughly halves the number of items needed for the same power. If items share a source document, the discordant pairs are not independent either; see our note on clustered standard errors for the correction.

Drift monitoring after a silent update#

Hosted models change. A provider update can move accuracy without any change on the bank's side, so outcomes analysis has to run continuously. A one-sided Bernoulli CUSUM [10] on daily graded samples detects a drop from accuracy $p_0$ to $p_1$:

Here $n$ is the number of production answers graded per day, $k_t$ the number correct on day $t$, and $h$ a threshold. Each correct answer pulls the statistic down slightly and each error pushes it up; it accumulates evidence until it crosses $h$, then resets.

Setup (simulation). Accuracy falls from 0.80 to 0.75 on the day of a silent update. For each daily sample size we chose $h$ by bisection so that the in-control average run length is about 365 days (one false alarm a year), then measured detection delay over 4,000 simulated changes.

Grading 10 answers a day, a five-point drop takes about a month to detect and sometimes two. Grading 50 brings it under two weeks. This is the practical budget question for outcomes analysis, and it is better answered with this table than with a fixed monthly review. A version pin with a full re-evaluation on any provider change remains the first line of defence; CUSUM catches what the pin misses.

Failure modes#

Checklist for a financial numeric evaluation#

1. For every item, store the reference value, its unit and scale, and the required precision.
2. Require answers in a structured form: value, scale, cited inputs, arithmetic.
3. Score with one unit of stated precision; report exact-match and relative-tolerance results only as sensitivity checks.
4. Run provenance checks; report hallucinated figures, extraction errors, computation errors and unit errors separately.
5. Compare against challengers on the same items with McNemar, sized with the formula above.
6. Pin model versions; re-run the full suite on any change; run CUSUM on graded production samples sized from the delay table.
7. Keep test sets and grading with a team independent of development, and document all of the above for validation.

Code: precision-aware scoring with unit normalisation#

SCALE = {"units": 1, "thousands": 1e3, "millions": 1e6, "billions": 1e9, "percent": 1}

def score(answer, scale, ref, ref_scale, precision):
    """answer/ref: floats; scale/ref_scale: keys of SCALE; precision: unit of last
    reported digit in ref_scale units (e.g. 1 for whole millions, 0.1 for 1-dp %)."""
    if (scale == "percent") != (ref_scale == "percent"):
        return {"ok": False, "error": "unit"}
    a = answer * SCALE[scale] / SCALE[ref_scale]     # normalise to the reference scale
    if abs(a - ref) <= precision + 1e-9:
        return {"ok": True, "error": None}
    if ref != 0 and abs(a + ref) <= precision + 1e-9:
        return {"ok": False, "error": "sign"}
    for f in (1e3, 1e-3, 1e2, 1e-2):                  # scale slip that survived normalisation
        if abs(a / f - ref) <= precision + 1e-9:
            return {"ok": False, "error": "scale"}
    return {"ok": False, "error": "value"}

Classifying failures as unit, sign, scale or value errors costs a few lines and turns a single accuracy number into something a developer can act on.

Frequently asked questions#

Sources#

1. Z. Chen, W. Chen, C. Smiley et al.. FinQA: A Dataset of Numerical Reasoning over Financial Data, 2021. arxiv.org
2. F. Zhu, W. Lei, Y. Huang et al.. TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance, 2021. arxiv.org
3. Z. Chen, S. Li, C. Smiley et al.. ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering, 2022. arxiv.org
4. P. Islam, A. Kannappan, D. Kiela et al.. FinanceBench: A New Benchmark for Financial Question Answering, 2023. arxiv.org
5. V. Reddy, R. Koncel-Kedziorski, V. D. Lai et al.. DocFinQA: A Long-Context Financial Reasoning Dataset, 2024. arxiv.org
6. Board of Governors of the Federal Reserve System and Office of the Comptroller of the Currency. SR 11-7: Guidance on Model Risk Management, 2011. federalreserve.gov
7. Prudential Regulation Authority, Bank of England. SS1/23: Model risk management principles for banks, 2023. bankofengland.co.uk
8. European Parliament and Council of the European Union. Regulation (EU) 2024/1689 (Artificial Intelligence Act), Annex III, 2024. eur-lex.europa.eu
9. Q. McNemar. Psychometrika. Note on the Sampling Error of the Difference Between Correlated Proportions or Percentages, 1947. doi.org
10. E. S. Page. Biometrika. Continuous Inspection Schemes, 1954. doi.org