Model failure · Republic of Marovia 2028 (B) · CHF 2037 Notes · CDSBench #247 · benchmark v0.1
GPT answered Yes; the verified outcome was No
Question
Does the record contain enough evidence to reach a reliable deliverability conclusion without further documents?
- Verified answer
- No Bearer form requires the clearing-system exception to be reviewed.
- GPT answered
- Yes Assumed satisfied.
- Decisive issue
- Uncertainty
- Evidence for the label
- follows from the whole record
- GPT cited
- noneevidence score 100%
Failure classification
Wrong final answer
- Wrong final answer. Answered YES; verified outcome NO.
- Overconfident. Confidence 0.92 on an incorrect answer.
- Unsupported claim. No evidence cited for the conclusion.
Classifications are produced by the failure analyzer against the verified label and the evidence map; the model never grades itself. Failure taxonomy →
Everyone else
The same question, every other participant
- MastermindIncorrectanswered Yesits failure page →
- ClaudeIncorrectanswered Yesits failure page →
- GeminiIncorrectanswered Yesits failure page →
Related failures
GPT also missed at least 6 other uncertainty questions
- #70 Republic of Sanderia 2025Wrong final answer
- #72 Meridian Telecom S.A. 2026Wrong final answer
- #79 Meridian Telecom S.A. 2026Wrong final answer
- #109 Republic of Marovia 2024Wrong final answer
- #138 Solvane Airlines S.A. 2027Wrong final answer
- #208 Republic of Sanderia 2024 (B)Wrong final answer
Benchmark version
v0.1
This page records GPT’s answer from its latest official run on the frozen benchmark v0.1. If the model is re-run and the answer changes, the page shows the new result and the model page keeps the full run history. Corrections policy →
Same kind of question on your own documents?
Mastermind runs the deterministic tests, the skeptic and the evidence verification on your record. Private documents never enter the benchmark.
Analyze your own documents →