GPT · GPT-5 · benchmark v0.1
GPT missed 56 verified fact extraction questions
Wrong answers, abstentions the record did not justify, and correct answers with unsupported evidence. Every row has a permanent page that can be cited or forwarded.
| # | Question | Category | Verified | GPT | Failure |
|---|---|---|---|---|---|
| 268 | Solvane Airlines S.A. 2026 (B) · GBP 2038 Notes What is the maturity date of the instrument? If it is not stated in the record, answer INSUFFICIENT_… | Fact extraction | Insufficient information | Incorrect · 2030-01-01 | Missed uncertainty. Expected INSUFFICIENT_INFORMATION; the answer committed to 2030-01-01 without sufficient e… |
| 269 | Solvane Airlines S.A. 2026 (B) · GBP 2038 Notes In which currency is the instrument denominated? Answer with the ISO currency code. | Fact extraction | GBP | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 277 | Republic of Norland 2027 (C) · JPY 2040 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. | Fact extraction | 2040-05-04 | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 278 | Republic of Norland 2027 (C) · JPY 2040 Notes In which currency is the instrument denominated? Answer with the ISO currency code. | Fact extraction | JPY | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 287 | Halvard Steel AG 2028 (C) · CHF 2064 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. | Fact extraction | 2064-06-07 | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 288 | Halvard Steel AG 2028 (C) · CHF 2064 Notes In which currency is the instrument denominated? Answer with the ISO currency code. | Fact extraction | CHF | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
Categories on this page: Fact extraction. Category pages explain what each one tests: fact extraction.