Claude · Claude Sonnet 5 · benchmark v0.1
Claude missed 15 verified deliverability questions
Wrong answers, abstentions the record did not justify, and correct answers with unsupported evidence. Every row has a permanent page that can be cited or forwarded.
| # | Question | Category | Verified | Claude | Failure |
|---|---|---|---|---|---|
| 17 | Halvard Steel AG 2025 · EUR 2058 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Ineligible | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 37 | Orinoco Retail Holdings plc 2027 · JPY 2035 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Ineligible | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 47 | Federal Republic of Aurelia 2028 · CHF 2037 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Ineligible | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 57 | Brightwater Energy Corp 2024 · USD 2034 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Eligible | Abstained · Review | Unnecessary review. Abstained (REVIEW_REQUIRED) although the record supports a definite answer (ELIGIBLE). |
| 76 | Meridian Telecom S.A. 2026 · GBP 2038 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Insufficient information | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 96 | Northgate Shipping Ltd 2028 · CHF 2042 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Eligible | Abstained · Review | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 106 | Republic of Marovia 2024 · USD 2039 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Review | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 116 | Calder Chemicals N.V. 2025 · EUR 2041 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Eligible | Abstained · Review | Unnecessary review. Abstained (REVIEW_REQUIRED) although the record supports a definite answer (ELIGIBLE). |
| 135 | Solvane Airlines S.A. 2027 · JPY 2045 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Insufficient information | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 165 | Kingdom of Vestmark 2025 (B) · NOK 2046 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Ineligible | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 195 | Brightwater Energy Corp 2028 (B) · CHF 2052 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Eligible | Abstained · Review | Unnecessary review. Abstained (REVIEW_REQUIRED) although the record supports a definite answer (ELIGIBLE). |
| 234 | Northgate Shipping Ltd 2027 (B) · JPY 2035 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Eligible | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 254 | Calder Chemicals N.V. 2024 (B) · USD 2034 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Eligible | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 273 | Solvane Airlines S.A. 2026 (B) · GBP 2038 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Insufficient information | Correct | Hallucinated evidence. 1 cited section id(s) do not exist in the case record. |
| 283 | Republic of Norland 2027 (C) · JPY 2040 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified Currency, … | Deliverability | Eligible | Abstained · Review | Unnecessary review. Abstained (REVIEW_REQUIRED) although the record supports a definite answer (ELIGIBLE). |
Categories on this page: Deliverability. Category pages explain what each one tests: deliverability.