Gemini · Gemini 2.5 Pro · benchmark v0.1
Gemini missed 238 verified questions
Wrong answers, abstentions the record did not justify, and correct answers with unsupported evidence. Every row has a permanent page that can be cited or forwarded.
| # | Question | Category | Verified | Gemini | Failure |
|---|---|---|---|---|---|
| 187 | Federal Republic of Aurelia 2027 (B) · JPY 2050 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer with the s… | Evidence | DOC-1#S4 | Incorrect · DOC-1#S1 | Retrieval failure. None of the cited sections is among those that establish the answer; the relevant evidence… |
| 189 | Brightwater Energy Corp 2028 (B) · CHF 2052 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. | Fact extraction | 2052-08-04 | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 190 | Brightwater Energy Corp 2028 (B) · CHF 2052 Notes In which currency is the instrument denominated? Answer with the ISO currency code. | Fact extraction | CHF | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 191 | Brightwater Energy Corp 2028 (B) · CHF 2052 Notes Does the instrument's maturity satisfy a maximum maturity of 30 years measured from the credit event… | Maturity | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 192 | Brightwater Energy Corp 2028 (B) · CHF 2052 Notes Is the instrument denominated in a Specified Currency (USD, EUR, JPY, GBP, CHF, CAD)? | Currency | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 193 | Brightwater Energy Corp 2028 (B) · CHF 2052 Notes Is the instrument subordinated? | Seniority | No | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 195 | Brightwater Energy Corp 2028 (B) · CHF 2052 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Eligible | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 197 | Brightwater Energy Corp 2028 (B) · CHF 2052 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer with the s… | Evidence | DOC-1#S4 | Incorrect · DOC-1#S1 | Retrieval failure. None of the cited sections is among those that establish the answer; the relevant evidence… |
| 201 | Republic of Sanderia 2024 (B) · USD 2029 Notes In which currency is the instrument denominated? Answer with the ISO currency code. | Fact extraction | USD | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 202 | Republic of Sanderia 2024 (B) · USD 2029 Notes Is the instrument denominated in a Specified Currency (USD, EUR, JPY, GBP, CHF, CAD)? | Currency | Yes | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 203 | Republic of Sanderia 2024 (B) · USD 2029 Notes Is the instrument subordinated? | Seniority | No | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 204 | Republic of Sanderia 2024 (B) · USD 2029 Notes Is the instrument transferable without the consent of the Issuer? | Transferability | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 205 | Republic of Sanderia 2024 (B) · USD 2029 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Review | Incorrect · Eligible | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 207 | Republic of Sanderia 2024 (B) · USD 2029 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer with the s… | Evidence | DOC-1#S4 | Incorrect · DOC-1#S1 | Retrieval failure. None of the cited sections is among those that establish the answer; the relevant evidence… |
| 208 | Republic of Sanderia 2024 (B) · USD 2029 Notes Does the record contain enough evidence to reach a reliable deliverability conclusion without furthe… | Uncertainty | No | Incorrect · Yes | Wrong final answer. Answered YES; verified outcome NO. |
| 209 | Meridian Telecom S.A. 2025 (B) · EUR 2031 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. | Fact extraction | 2031-10-10 | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 210 | Meridian Telecom S.A. 2025 (B) · EUR 2031 Notes Is there enough evidence in the record to establish the currency of denomination of the instrument? | Uncertainty | No | Incorrect · Yes | Wrong final answer. Answered YES; verified outcome NO. |
| 211 | Meridian Telecom S.A. 2025 (B) · EUR 2031 Notes Does the instrument's maturity satisfy a maximum maturity of 30 years measured from the credit event… | Maturity | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 212 | Meridian Telecom S.A. 2025 (B) · EUR 2031 Notes Is the instrument subordinated? | Seniority | No | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 213 | Meridian Telecom S.A. 2025 (B) · EUR 2031 Notes Is the instrument transferable without the consent of the Issuer? | Transferability | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 214 | Meridian Telecom S.A. 2025 (B) · EUR 2031 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Insufficient information | Incorrect · Eligible | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 216 | Meridian Telecom S.A. 2025 (B) · EUR 2031 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer with the s… | Evidence | DOC-1#S4 | Incorrect · DOC-1#S1 | Retrieval failure. None of the cited sections is among those that establish the answer; the relevant evidence… |
| 217 | Meridian Telecom S.A. 2025 (B) · EUR 2031 Notes Does the record contain enough evidence to reach a reliable deliverability conclusion without furthe… | Uncertainty | No | Incorrect · Yes | Wrong final answer. Answered YES; verified outcome NO. |
| 218 | United Provinces of Kestrel 2026 (B) · GBP 2033 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. | Fact extraction | 2033-11-13 | Incorrect · 2030-01-01 | Retrieval failure. None of the cited sections is among those that establish the answer; the relevant evidence… |
| 219 | United Provinces of Kestrel 2026 (B) · GBP 2033 Notes In which currency is the instrument denominated? Answer with the ISO currency code. | Fact extraction | GBP | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 220 | United Provinces of Kestrel 2026 (B) · GBP 2033 Notes Does the instrument's maturity satisfy a maximum maturity of 30 years measured from the credit event… | Maturity | Yes | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 221 | United Provinces of Kestrel 2026 (B) · GBP 2033 Notes Is the instrument denominated in a Specified Currency (USD, EUR, JPY, GBP, CHF, CAD)? | Currency | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 222 | United Provinces of Kestrel 2026 (B) · GBP 2033 Notes Is the instrument subordinated? | Seniority | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 223 | United Provinces of Kestrel 2026 (B) · GBP 2033 Notes Is the instrument transferable without the consent of the Issuer? | Transferability | Yes | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 224 | United Provinces of Kestrel 2026 (B) · GBP 2033 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Ineligible | Incorrect · Eligible | Retrieval failure. None of the cited sections is among those that establish the answer; the relevant evidence… |
| 226 | United Provinces of Kestrel 2026 (B) · GBP 2033 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer with the s… | Evidence | DOC-1#S4 | Incorrect · DOC-1#S1 | Retrieval failure. None of the cited sections is among those that establish the answer; the relevant evidence… |
| 228 | Northgate Shipping Ltd 2027 (B) · JPY 2035 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. | Fact extraction | 2035-12-16 | Incorrect · 2030-01-01 | Retrieval failure. None of the cited sections is among those that establish the answer; the relevant evidence… |
| 229 | Northgate Shipping Ltd 2027 (B) · JPY 2035 Notes In which currency is the instrument denominated? Answer with the ISO currency code. | Fact extraction | JPY | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 230 | Northgate Shipping Ltd 2027 (B) · JPY 2035 Notes Does the instrument's maturity satisfy a maximum maturity of 30 years measured from the credit event… | Maturity | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 231 | Northgate Shipping Ltd 2027 (B) · JPY 2035 Notes Is the instrument denominated in a Specified Currency (USD, EUR, JPY, GBP, CHF, CAD)? | Currency | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 232 | Northgate Shipping Ltd 2027 (B) · JPY 2035 Notes Is the instrument subordinated? | Seniority | No | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 233 | Northgate Shipping Ltd 2027 (B) · JPY 2035 Notes Is the instrument transferable without the consent of the Issuer? | Transferability | Yes | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 234 | Northgate Shipping Ltd 2027 (B) · JPY 2035 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Eligible | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 236 | Northgate Shipping Ltd 2027 (B) · JPY 2035 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer with the s… | Evidence | DOC-1#S4 | Incorrect · DOC-1#S1 | Retrieval failure. None of the cited sections is among those that establish the answer; the relevant evidence… |
| 238 | Republic of Marovia 2028 (B) · CHF 2037 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. | Fact extraction | 2037-01-19 | Incorrect · 2030-01-01 | Retrieval failure. None of the cited sections is among those that establish the answer; the relevant evidence… |
| 239 | Republic of Marovia 2028 (B) · CHF 2037 Notes In which currency is the instrument denominated? Answer with the ISO currency code. | Fact extraction | CHF | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 240 | Republic of Marovia 2028 (B) · CHF 2037 Notes Does the instrument's maturity satisfy a maximum maturity of 30 years measured from the credit event… | Maturity | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 241 | Republic of Marovia 2028 (B) · CHF 2037 Notes Is the instrument denominated in a Specified Currency (USD, EUR, JPY, GBP, CHF, CAD)? | Currency | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 242 | Republic of Marovia 2028 (B) · CHF 2037 Notes Is the instrument subordinated? | Seniority | No | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 243 | Republic of Marovia 2028 (B) · CHF 2037 Notes Is the instrument transferable without the consent of the Issuer? | Transferability | Yes | Correct | Evidence selection failure. Correct answer but cited evidence does not match the supporting sections (evidence score 0… |
| 244 | Republic of Marovia 2028 (B) · CHF 2037 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Review | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 246 | Republic of Marovia 2028 (B) · CHF 2037 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer with the s… | Evidence | DOC-1#S4 | Incorrect · DOC-1#S1 | Retrieval failure. None of the cited sections is among those that establish the answer; the relevant evidence… |
| 247 | Republic of Marovia 2028 (B) · CHF 2037 Notes Does the record contain enough evidence to reach a reliable deliverability conclusion without furthe… | Uncertainty | No | Incorrect · Yes | Wrong final answer. Answered YES; verified outcome NO. |
| 248 | Calder Chemicals N.V. 2024 (B) · USD 2034 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. | Fact extraction | 2034-02-22 | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 249 | Calder Chemicals N.V. 2024 (B) · USD 2034 Notes In which currency is the instrument denominated? Answer with the ISO currency code. | Fact extraction | USD | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
Categories on this page: Evidence, Fact extraction, Maturity, Currency, Seniority, Deliverability, Transferability, Uncertainty. Category pages explain what each one tests: evidence, fact extraction, maturity.