Model failure · Orinoco Retail Holdings plc 2027 · JPY 2035 Notes · CDSBench #31 · benchmark v0.1
Gemini answered 2030-01-01; the verified outcome was 2035-04-10
Question
What is the maturity date of the instrument? Answer as YYYY-MM-DD.
- Verified answer
- 2035-04-10 Stated in the maturity condition.
- Gemini answered
- 2030-01-01 Estimated from context.
- Decisive issue
- Extraction
- Evidence for the label
- Gemini cited
- evidence score 0%
Failure classification
Retrieval failure
- Retrieval failure. None of the cited sections is among those that establish the answer; the relevant evidence was not retrieved.
- Wrong fact. Extracted 2030-01-01; record states 2035-04-10.
- Overconfident. Confidence 0.85 on an incorrect answer.
Classifications are produced by the failure analyzer against the verified label and the evidence map; the model never grades itself. Failure taxonomy →
Everyone else
The same question, every other participant
- MastermindCorrectanswered 2035-04-10
- ClaudeCorrectanswered 2035-04-10
- GPTCorrectanswered 2035-04-10its failure page →
Related failures
Gemini also missed at least 6 other extraction questions
- #1 Republic of Norland 2024Retrieval failure
- #11 Halvard Steel AG 2025Retrieval failure
- #21 Kingdom of Vestmark 2026Hallucinated evidence
- #51 Brightwater Energy Corp 2024Hallucinated evidence
- #71 Meridian Telecom S.A. 2026Retrieval failure
- #90 Northgate Shipping Ltd 2028Retrieval failure
Benchmark version
v0.1
This page records Gemini’s answer from its latest official run on the frozen benchmark v0.1. If the model is re-run and the answer changes, the page shows the new result and the model page keeps the full run history. Corrections policy →
Same kind of question on your own documents?
Mastermind runs the deterministic tests, the skeptic and the evidence verification on your record. Private documents never enter the benchmark.
Analyze your own documents →