Model failure · United Provinces of Kestrel 2026 (B) · GBP 2033 Notes · CDSBench #219 · benchmark v0.1
Claude reached the verified outcome but cited evidence that does not exist
Question
In which currency is the instrument denominated? Answer with the ISO currency code.
- Verified answer
- GBP Stated in the currency condition.
- Claude answered
- GBP The currency is GBP.
- Decisive issue
- Extraction
- Evidence for the label
- Claude cited
- 1 non-existentevidence score 67%
Failure classification
Hallucinated evidence
- Hallucinated evidence. 1 cited section id(s) do not exist in the case record.
Classifications are produced by the failure analyzer against the verified label and the evidence map; the model never grades itself. Failure taxonomy →
Everyone else
The same question, every other participant
- MastermindCorrectanswered GBPits failure page →
- GPTCorrectanswered GBPits failure page →
- GeminiCorrectanswered GBPits failure page →
Related failures
No other extraction misses on this run
Benchmark version
v0.1
This page records Claude’s answer from its latest official run on the frozen benchmark v0.1. If the model is re-run and the answer changes, the page shows the new result and the model page keeps the full run history. Corrections policy →
Same kind of question on your own documents?
Mastermind runs the deterministic tests, the skeptic and the evidence verification on your record. Private documents never enter the benchmark.
Analyze your own documents →