Model failure · Solvane Airlines S.A. 2027 · JPY 2045 Notes · CDSBench #130 · benchmark v0.1
GPT answered 2030-01-01; the verified outcome was Insufficient information
Question
What is the maturity date of the instrument? If it is not stated in the record, answer INSUFFICIENT_INFORMATION.
- Verified answer
- Insufficient information No document in the record states the maturity date.
- GPT answered
- 2030-01-01 Estimated from context.
- Decisive issue
- Extraction
- Evidence for the label
- GPT cited
- noneevidence score 0%
Failure classification
Missed uncertainty
- Missed uncertainty. Expected INSUFFICIENT_INFORMATION; the answer committed to 2030-01-01 without sufficient evidence.
- Overconfident. Confidence 0.92 on an answer the record cannot support.
Classifications are produced by the failure analyzer against the verified label and the evidence map; the model never grades itself. Failure taxonomy →
Everyone else
The same question, every other participant
- MastermindCorrectanswered Insufficient information
- ClaudeCorrectanswered Insufficient information
- GeminiIncorrectanswered 2030-01-01its failure page →
Related failures
GPT also missed 1 other extraction question
- #268 Solvane Airlines S.A. 2026 (B)Missed uncertainty
Benchmark version
v0.1
This page records GPT’s answer from its latest official run on the frozen benchmark v0.1. If the model is re-run and the answer changes, the page shows the new result and the model page keeps the full run history. Corrections policy →
Same kind of question on your own documents?
Mastermind runs the deterministic tests, the skeptic and the evidence verification on your record. Private documents never enter the benchmark.
Analyze your own documents →