Model comparison · benchmark v0.1
Claude vs Gemini on CDSBench
Same frozen questions, same scoring, latest official run per model. Choose a category to compare on one kind of question.
Dimensions
Every scored dimension
By category
Accuracy on each kind of question
| Category | Claude | Gemini | n |
|---|---|---|---|
| Fact extraction | 100% | 68% | 56 |
| Maturity | 100% | 100% | 26 |
| Currency | 100% | 100% | 28 |
| Seniority | 100% | 100% | 30 |
| Transferability | 57% | 100% | 30 |
| Multi-document | 100% | 100% | 2 |
| Deliverability | 83% | 73% | 30 |
| Uncertainty | 69% | 69% | 32 |
| Restructuring | 100% | 100% | 2 |
| Entity match | 100% | 100% | 30 |
| Evidence | 100% | 0% | 30 |
Where they disagree
12+ verified questions where at least one is right and one is wrong
| # | Question | Verified | Claude | Gemini |
|---|---|---|---|---|
| 1 | Republic of Norland 2024 · USD 2029 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. · Fact extraction | 2029-01-01 | 2029-01-01 | 2030-01-01 |
| 9 | Republic of Norland 2024 · USD 2029 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer … · Evidence | DOC-1#S4 | DOC-1#S4 | DOC-1#S1 |
| 11 | Halvard Steel AG 2025 · EUR 2058 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. · Fact extraction | 2058-02-04 | 2058-02-04 | 2030-01-01 |
| 16 | Halvard Steel AG 2025 · EUR 2058 Notes Is the instrument transferable without the consent of the Issuer? · Transferability | Yes | Review | Yes |
| 19 | Halvard Steel AG 2025 · EUR 2058 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer … · Evidence | DOC-1#S4 | DOC-1#S4 | DOC-1#S1 |
| 21 | Kingdom of Vestmark 2026 · ZAR 2033 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. · Fact extraction | 2033-03-07 | 2033-03-07 | 2030-01-01 |
| 29 | Kingdom of Vestmark 2026 · ZAR 2033 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer … · Evidence | DOC-1#S4 | DOC-1#S4 | DOC-1#S1 |
| 31 | Orinoco Retail Holdings plc 2027 · JPY 2035 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. · Fact extraction | 2035-04-10 | 2035-04-10 | 2030-01-01 |
| 36 | Orinoco Retail Holdings plc 2027 · JPY 2035 Notes Is the instrument transferable without the consent of the Issuer? · Transferability | Yes | Review | Yes |
| 39 | Orinoco Retail Holdings plc 2027 · JPY 2035 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer … · Evidence | DOC-1#S4 | DOC-1#S4 | DOC-1#S1 |
| 49 | Federal Republic of Aurelia 2028 · CHF 2037 Notes Which section of the record establishes the seniority (ranking) of the instrument? Answer … · Evidence | DOC-1#S4 | DOC-1#S4 | DOC-1#S1 |
| 51 | Brightwater Energy Corp 2024 · USD 2034 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. · Fact extraction | 2034-06-16 | 2034-06-16 | 2030-01-01 |
Overall = 100 × (0.7 accuracy + 0.2 evidence + 0.1 uncertainty handling) − 15 × false-confidence rate. Mastermind is CDSBench’s own system built on the ranked models; it is compared here on request, never ranked. Methodology → · Runs: Claude 2026-09-15 · Gemini 2026-09-15
Related: Claude model page · Gemini model page · Full leaderboard · Collections