Benchmark category · v0.1 · 30 verified questions
Deliverability
The full deliverable-obligation analysis: every characteristic tested together on one instrument and one credit event.
Ranking
Accuracy on deliverability questions
| # | Model | Accuracy | Correct | |
|---|---|---|---|---|
| 1 | Claude Sonnet 5 | 83% | 25/30 | |
| 2 | GPT -5 | 73% | 22/30 | |
| 3 | Gemini 2.5 Pro | 73% | 22/30 | |
| Reference · CDSBench’s own system · not ranked | ||||
| — | Mastermind | 83% | 25/30 | |
Common failures
How models go wrong here
| Hallucinated evidence | 68 |
| Overconfident | 16 |
| Retrieval failure | 14 |
| Unnecessary review | 10 |
| Evidence selection failure | 9 |
| Missed uncertainty | 8 |
Across every model’s latest official run. Failure taxonomy →
Hardest questions
Most models wrong
- #57 Brightwater Energy Corp 2024 · USD 2034 NotesApplying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified …3/4
- #195 Brightwater Energy Corp 2028 (B) · CHF 2052 NotesApplying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified …3/4
- #76 Meridian Telecom S.A. 2026 · GBP 2038 NotesApplying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified …2/4
- #96 Northgate Shipping Ltd 2028 · CHF 2042 NotesApplying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified …2/4
- #116 Calder Chemicals N.V. 2025 · EUR 2041 NotesApplying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified …2/4
- #135 Solvane Airlines S.A. 2027 · JPY 2045 NotesApplying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified …2/4
- #205 Republic of Sanderia 2024 (B) · USD 2029 NotesApplying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified …2/4
- #214 Meridian Telecom S.A. 2025 (B) · EUR 2031 NotesApplying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified …2/4
Latest
Most recently published questions
- #293 Halvard Steel AG 2028 (C)2026-09-15
- #283 Republic of Norland 2027 (C)2026-09-15
- #273 Solvane Airlines S.A. 2026 (B)2026-09-15
- #264 Commonwealth of Tessaly 2025 (B)2026-09-15
- #254 Calder Chemicals N.V. 2024 (B)2026-09-15
Cases
30 cases with deliverability questions
- Federal Republic of Aurelia 2028 Federal Republic of Aurelia · 2028
- Northgate Shipping Ltd 2028 Northgate Shipping Ltd · 2028
- Republic of Norland 2028 (B) Republic of Norland · 2028
- Brightwater Energy Corp 2028 (B) Brightwater Energy Corp · 2028
- Republic of Marovia 2028 (B) Republic of Marovia · 2028
- Halvard Steel AG 2028 (C) Halvard Steel AG · 2028
- Republic of Norland 2027 (C) Republic of Norland · 2027
- Orinoco Retail Holdings plc 2027 Orinoco Retail Holdings plc · 2027
- United Provinces of Kestrel 2027 United Provinces of Kestrel · 2027
- Solvane Airlines S.A. 2027 Solvane Airlines S.A. · 2027
- Federal Republic of Aurelia 2027 (B) Federal Republic of Aurelia · 2027
- Northgate Shipping Ltd 2027 (B) Northgate Shipping Ltd · 2027
and 18 more in the case explorer.
Following this category sends an email only when meaningful new deliverability questions are published, a model’s deliverability accuracy moves materially, or a research report on it is published. No newsletter.