Gemini · Gemini 2.5 Pro · benchmark v0.1
Gemini missed 238 verified questions
Wrong answers, abstentions the record did not justify, and correct answers with unsupported evidence. Every row has a permanent page that can be cited or forwarded.
| # | Question | Category | Verified | Gemini | Failure |
|---|---|---|---|---|---|
| 125 | Commonwealth of Tessaly 2026 · GBP 2043 Notes Is the instrument transferable without the consent of the Issuer? | Transferability | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 126 | Commonwealth of Tessaly 2026 · GBP 2043 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Ineligible | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 128 | Commonwealth of Tessaly 2026 · GBP 2043 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer with the s… | Evidence | DOC-1#S4 | Incorrect · DOC-1#S1 | Retrieval failure. None of the cited sections is among those that establish the answer; the relevant evidence… |
| 130 | Solvane Airlines S.A. 2027 · JPY 2045 Notes What is the maturity date of the instrument? If it is not stated in the record, answer INSUFFICIENT_… | Fact extraction | Insufficient information | Incorrect · 2030-01-01 | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 131 | Solvane Airlines S.A. 2027 · JPY 2045 Notes In which currency is the instrument denominated? Answer with the ISO currency code. | Fact extraction | JPY | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 132 | Solvane Airlines S.A. 2027 · JPY 2045 Notes Is the instrument denominated in a Specified Currency (USD, EUR, JPY, GBP, CHF, CAD)? | Currency | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 133 | Solvane Airlines S.A. 2027 · JPY 2045 Notes Is the instrument subordinated? | Seniority | No | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 134 | Solvane Airlines S.A. 2027 · JPY 2045 Notes Is the instrument transferable without the consent of the Issuer? | Transferability | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 135 | Solvane Airlines S.A. 2027 · JPY 2045 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Insufficient information | Incorrect · Eligible | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 137 | Solvane Airlines S.A. 2027 · JPY 2045 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer with the s… | Evidence | DOC-1#S4 | Incorrect · DOC-1#S1 | Retrieval failure. None of the cited sections is among those that establish the answer; the relevant evidence… |
| 138 | Solvane Airlines S.A. 2027 · JPY 2045 Notes Does the record contain enough evidence to reach a reliable deliverability conclusion without furthe… | Uncertainty | No | Incorrect · Yes | Wrong final answer. Answered YES; verified outcome NO. |
| 139 | Republic of Norland 2028 (B) · CHF 2047 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. | Fact extraction | 2047-03-16 | Incorrect · 2030-01-01 | Retrieval failure. None of the cited sections is among those that establish the answer; the relevant evidence… |
| 140 | Republic of Norland 2028 (B) · CHF 2047 Notes In which currency is the instrument denominated? Answer with the ISO currency code. | Fact extraction | CHF | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 141 | Republic of Norland 2028 (B) · CHF 2047 Notes Does the instrument's maturity satisfy a maximum maturity of 30 years measured from the credit event… | Maturity | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 142 | Republic of Norland 2028 (B) · CHF 2047 Notes Is the instrument denominated in a Specified Currency (USD, EUR, JPY, GBP, CHF, CAD)? | Currency | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 143 | Republic of Norland 2028 (B) · CHF 2047 Notes Is the instrument subordinated? | Seniority | No | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 144 | Republic of Norland 2028 (B) · CHF 2047 Notes Is the instrument transferable without the consent of the Issuer? | Transferability | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 145 | Republic of Norland 2028 (B) · CHF 2047 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Eligible | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 147 | Republic of Norland 2028 (B) · CHF 2047 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer with the s… | Evidence | DOC-1#S4 | Incorrect · DOC-1#S1 | Retrieval failure. None of the cited sections is among those that establish the answer; the relevant evidence… |
| 149 | Halvard Steel AG 2024 (B) · USD 2056 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. | Fact extraction | 2056-04-19 | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 150 | Halvard Steel AG 2024 (B) · USD 2056 Notes In which currency is the instrument denominated? Answer with the ISO currency code. | Fact extraction | USD | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 151 | Halvard Steel AG 2024 (B) · USD 2056 Notes Does the instrument's maturity satisfy a maximum maturity of 30 years measured from the credit event… | Maturity | No | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 152 | Halvard Steel AG 2024 (B) · USD 2056 Notes Is the instrument denominated in a Specified Currency (USD, EUR, JPY, GBP, CHF, CAD)? | Currency | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 153 | Halvard Steel AG 2024 (B) · USD 2056 Notes Is the instrument subordinated? | Seniority | No | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 154 | Halvard Steel AG 2024 (B) · USD 2056 Notes Is the instrument transferable without the consent of the Issuer? | Transferability | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 155 | Halvard Steel AG 2024 (B) · USD 2056 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Ineligible | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 157 | Halvard Steel AG 2024 (B) · USD 2056 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer with the s… | Evidence | DOC-1#S4 | Incorrect · DOC-1#S1 | Retrieval failure. None of the cited sections is among those that establish the answer; the relevant evidence… |
| 159 | Kingdom of Vestmark 2025 (B) · NOK 2046 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. | Fact extraction | 2046-05-22 | Incorrect · 2030-01-01 | Retrieval failure. None of the cited sections is among those that establish the answer; the relevant evidence… |
| 160 | Kingdom of Vestmark 2025 (B) · NOK 2046 Notes In which currency is the instrument denominated? Answer with the ISO currency code. | Fact extraction | NOK | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 161 | Kingdom of Vestmark 2025 (B) · NOK 2046 Notes Does the instrument's maturity satisfy a maximum maturity of 30 years measured from the credit event… | Maturity | Yes | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 162 | Kingdom of Vestmark 2025 (B) · NOK 2046 Notes Is the instrument denominated in a Specified Currency (USD, EUR, JPY, GBP, CHF, CAD)? | Currency | No | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 163 | Kingdom of Vestmark 2025 (B) · NOK 2046 Notes Is the instrument subordinated? | Seniority | No | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 164 | Kingdom of Vestmark 2025 (B) · NOK 2046 Notes Is the instrument transferable without the consent of the Issuer? | Transferability | Yes | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 165 | Kingdom of Vestmark 2025 (B) · NOK 2046 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Ineligible | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 167 | Kingdom of Vestmark 2025 (B) · NOK 2046 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer with the s… | Evidence | DOC-1#S4 | Incorrect · DOC-1#S1 | Retrieval failure. None of the cited sections is among those that establish the answer; the relevant evidence… |
| 169 | Orinoco Retail Holdings plc 2026 (B) · GBP 2048 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. | Fact extraction | 2048-06-25 | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 170 | Orinoco Retail Holdings plc 2026 (B) · GBP 2048 Notes In which currency is the instrument denominated? Answer with the ISO currency code. | Fact extraction | GBP | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 171 | Orinoco Retail Holdings plc 2026 (B) · GBP 2048 Notes Does the instrument's maturity satisfy a maximum maturity of 30 years measured from the credit event… | Maturity | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 172 | Orinoco Retail Holdings plc 2026 (B) · GBP 2048 Notes Is the instrument denominated in a Specified Currency (USD, EUR, JPY, GBP, CHF, CAD)? | Currency | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 173 | Orinoco Retail Holdings plc 2026 (B) · GBP 2048 Notes Is the instrument subordinated? | Seniority | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 174 | Orinoco Retail Holdings plc 2026 (B) · GBP 2048 Notes Is the instrument transferable without the consent of the Issuer? | Transferability | Yes | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 175 | Orinoco Retail Holdings plc 2026 (B) · GBP 2048 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Ineligible | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 177 | Orinoco Retail Holdings plc 2026 (B) · GBP 2048 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer with the s… | Evidence | DOC-1#S4 | Incorrect · DOC-1#S1 | Retrieval failure. None of the cited sections is among those that establish the answer; the relevant evidence… |
| 179 | Federal Republic of Aurelia 2027 (B) · JPY 2050 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. | Fact extraction | 2050-07-01 | Incorrect · 2030-01-01 | Retrieval failure. None of the cited sections is among those that establish the answer; the relevant evidence… |
| 180 | Federal Republic of Aurelia 2027 (B) · JPY 2050 Notes In which currency is the instrument denominated? Answer with the ISO currency code. | Fact extraction | JPY | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 181 | Federal Republic of Aurelia 2027 (B) · JPY 2050 Notes Does the instrument's maturity satisfy a maximum maturity of 30 years measured from the credit event… | Maturity | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 182 | Federal Republic of Aurelia 2027 (B) · JPY 2050 Notes Is the instrument denominated in a Specified Currency (USD, EUR, JPY, GBP, CHF, CAD)? | Currency | Yes | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 183 | Federal Republic of Aurelia 2027 (B) · JPY 2050 Notes Is the instrument subordinated? | Seniority | No | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 184 | Federal Republic of Aurelia 2027 (B) · JPY 2050 Notes Is the instrument transferable without the consent of the Issuer? | Transferability | No | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 185 | Federal Republic of Aurelia 2027 (B) · JPY 2050 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Ineligible | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
Categories on this page: Transferability, Deliverability, Evidence, Fact extraction, Currency, Seniority, Uncertainty, Maturity. Category pages explain what each one tests: transferability, deliverability, evidence.