Model failure · Republic of Sanderia 2024 (B) · USD 2029 Notes · CDSBench #204 · benchmark v0.1
GPT reached the verified outcome but cited the wrong section
Question
Is the instrument transferable without the consent of the Issuer?
- Verified answer
- Yes Stated in the transfer condition.
- GPT answered
- Yes Based on the transferable provision, the answer is YES.
- Decisive issue
- Transferability
- Evidence for the label
- GPT cited
- evidence score 0%
Failure classification
Evidence selection failure
- Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0.00).
Classifications are produced by the failure analyzer against the verified label and the evidence map; the model never grades itself. Failure taxonomy →
Everyone else
The same question, every other participant
- MastermindAbstainedanswered Reviewits failure page →
- ClaudeAbstainedanswered Reviewits failure page →
- GeminiCorrectanswered Yesits failure page →
Related failures
GPT also missed 2 other transferability questions
- #56 Brightwater Energy Corp 2024Failed multi document reasoning
- #194 Brightwater Energy Corp 2028 (B)Failed multi document reasoning
Benchmark version
v0.1
This page records GPT’s answer from its latest official run on the frozen benchmark v0.1. If the model is re-run and the answer changes, the page shows the new result and the model page keeps the full run history. Corrections policy →
Same kind of question on your own documents?
Mastermind runs the deterministic tests, the skeptic and the evidence verification on your record. Private documents never enter the benchmark.
Analyze your own documents →