Model failure · Republic of Marovia 2024 · USD 2039 Notes · CDSBench #106 · benchmark v0.1
Gemini reached the verified outcome but cited evidence that does not exist
Question
Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, Not Contingent, Transferable, Maximum Maturity 30 years, Not Bearer), is the instrument a Deliverable Obligation for this credit event?
- Verified answer
- Review Bearer form requires the clearing-system exception to be reviewed.
- Gemini answered
- Review Interpretation needed: Not Bearer.
- Decisive issue
- Deliverability
- Evidence for the label
- Gemini cited
- 1 non-existentevidence score 0%
Failure classification
Hallucinated evidence
- Hallucinated evidence. 1 cited section id(s) do not exist in the case record.
Classifications are produced by the failure analyzer against the verified label and the evidence map; the model never grades itself. Failure taxonomy →
Everyone else
The same question, every other participant
- MastermindCorrectanswered Reviewits failure page →
- ClaudeCorrectanswered Reviewits failure page →
- GPTCorrectanswered Review
Related failures
Gemini also missed at least 6 other deliverability questions
- #67 Republic of Sanderia 2025Hallucinated evidence
- #76 Meridian Telecom S.A. 2026Hallucinated evidence
- #86 United Provinces of Kestrel 2027Retrieval failure
- #135 Solvane Airlines S.A. 2027Hallucinated evidence
- #205 Republic of Sanderia 2024 (B)Hallucinated evidence
- #214 Meridian Telecom S.A. 2025 (B)Hallucinated evidence
Benchmark version
v0.1
This page records Gemini’s answer from its latest official run on the frozen benchmark v0.1. If the model is re-run and the answer changes, the page shows the new result and the model page keeps the full run history. Corrections policy →
Same kind of question on your own documents?
Mastermind runs the deterministic tests, the skeptic and the evidence verification on your record. Private documents never enter the benchmark.
Analyze your own documents →