Model comparison · benchmark v0.1
Claude vs GPT on CDSBench
Same frozen questions, same scoring, latest official run per model. Choose a category to compare on one kind of question.
Dimensions
Every scored dimension
By category
Accuracy on each kind of question
| Category | Claude | GPT | n |
|---|---|---|---|
| Fact extraction | 100% | 96% | 56 |
| Maturity | 100% | 100% | 26 |
| Currency | 100% | 100% | 28 |
| Seniority | 100% | 100% | 30 |
| Transferability | 57% | 93% | 30 |
| Multi-document | 100% | 100% | 2 |
| Deliverability | 83% | 73% | 30 |
| Uncertainty | 69% | 69% | 32 |
| Restructuring | 100% | 0% | 2 |
| Entity match | 100% | 93% | 30 |
| Evidence | 100% | 0% | 30 |
Where they disagree
12+ verified questions where at least one is right and one is wrong
| # | Question | Verified | Claude | GPT |
|---|---|---|---|---|
| 9 | Republic of Norland 2024 · USD 2029 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer … · Evidence | DOC-1#S4 | DOC-1#S4 | DOC-1#S1 |
| 16 | Halvard Steel AG 2025 · EUR 2058 Notes Is the instrument transferable without the consent of the Issuer? · Transferability | Yes | Review | Yes |
| 19 | Halvard Steel AG 2025 · EUR 2058 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer … · Evidence | DOC-1#S4 | DOC-1#S4 | DOC-1#S1 |
| 29 | Kingdom of Vestmark 2026 · ZAR 2033 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer … · Evidence | DOC-1#S4 | DOC-1#S4 | DOC-1#S1 |
| 36 | Orinoco Retail Holdings plc 2027 · JPY 2035 Notes Is the instrument transferable without the consent of the Issuer? · Transferability | Yes | Review | Yes |
| 39 | Orinoco Retail Holdings plc 2027 · JPY 2035 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer … · Evidence | DOC-1#S4 | DOC-1#S4 | DOC-1#S1 |
| 49 | Federal Republic of Aurelia 2028 · CHF 2037 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer … · Evidence | DOC-1#S4 | DOC-1#S4 | DOC-1#S1 |
| 59 | Brightwater Energy Corp 2024 · USD 2034 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer … · Evidence | DOC-1#S4 | DOC-1#S4 | DOC-1#S1 |
| 60 | Brightwater Energy Corp 2024 · USD 2034 Notes Did the documented amendment change whether the instrument satisfies the Transferable char… · Restructuring | Yes | Yes | No |
| 66 | Republic of Sanderia 2025 · EUR 2036 Notes Is the instrument transferable without the consent of the Issuer? · Transferability | Yes | Review | Yes |
| 67 | Republic of Sanderia 2025 · EUR 2036 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified … · Deliverability | Review | Review | Eligible |
| 69 | Republic of Sanderia 2025 · EUR 2036 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer … · Evidence | DOC-1#S4 | DOC-1#S4 | DOC-1#S1 |
Overall = 100 × (0.7 accuracy + 0.2 evidence + 0.1 uncertainty handling) − 15 × false-confidence rate. Mastermind is CDSBench’s own system built on the ranked models; it is compared here on request, never ranked. Methodology → · Runs: Claude 2026-09-15 · GPT 2026-09-15
Related: Claude model page · GPT model page · Full leaderboard · Collections