Model comparison · benchmark v0.1
Gemini vs GPT on CDSBench
Same frozen questions, same scoring, latest official run per model. Choose a category to compare on one kind of question.
Dimensions
Every scored dimension
By category
Accuracy on each kind of question
| Category | Gemini | GPT | n |
|---|---|---|---|
| Fact extraction | 68% | 96% | 56 |
| Maturity | 100% | 100% | 26 |
| Currency | 100% | 100% | 28 |
| Seniority | 100% | 100% | 30 |
| Transferability | 100% | 93% | 30 |
| Multi-document | 100% | 100% | 2 |
| Deliverability | 73% | 73% | 30 |
| Uncertainty | 69% | 69% | 32 |
| Restructuring | 100% | 0% | 2 |
| Entity match | 100% | 93% | 30 |
| Evidence | 0% | 0% | 30 |
Where they disagree
12+ verified questions where at least one is right and one is wrong
| # | Question | Verified | Gemini | GPT |
|---|---|---|---|---|
| 1 | Republic of Norland 2024 · USD 2029 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. · Fact extraction | 2029-01-01 | 2030-01-01 | 2029-01-01 |
| 11 | Halvard Steel AG 2025 · EUR 2058 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. · Fact extraction | 2058-02-04 | 2030-01-01 | 2058-02-04 |
| 21 | Kingdom of Vestmark 2026 · ZAR 2033 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. · Fact extraction | 2033-03-07 | 2030-01-01 | 2033-03-07 |
| 31 | Orinoco Retail Holdings plc 2027 · JPY 2035 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. · Fact extraction | 2035-04-10 | 2030-01-01 | 2035-04-10 |
| 51 | Brightwater Energy Corp 2024 · USD 2034 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. · Fact extraction | 2034-06-16 | 2030-01-01 | 2034-06-16 |
| 56 | Brightwater Energy Corp 2024 · USD 2034 Notes Reconciling all documents in the record, is the instrument transferable without the consen… · Transferability | Yes | Yes | No |
| 57 | Brightwater Energy Corp 2024 · USD 2034 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified … · Deliverability | Eligible | Eligible | Ineligible |
| 60 | Brightwater Energy Corp 2024 · USD 2034 Notes Did the documented amendment change whether the instrument satisfies the Transferable char… · Restructuring | Yes | Yes | No |
| 71 | Meridian Telecom S.A. 2026 · GBP 2038 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. · Fact extraction | 2038-08-22 | 2030-01-01 | 2038-08-22 |
| 86 | United Provinces of Kestrel 2027 · JPY 2040 Notes Applying the standard Deliverable Obligation Characteristics (Not Subordinated, Specified … · Deliverability | Ineligible | Eligible | Ineligible |
| 90 | Northgate Shipping Ltd 2028 · CHF 2042 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. · Fact extraction | 2042-10-01 | 2030-01-01 | 2042-10-01 |
| 100 | Republic of Marovia 2024 · USD 2039 Notes What is the maturity date of the instrument? Answer as YYYY-MM-DD. · Fact extraction | 2039-11-04 | 2030-01-01 | 2039-11-04 |
Overall = 100 × (0.7 accuracy + 0.2 evidence + 0.1 uncertainty handling) − 15 × false-confidence rate. Mastermind is CDSBench’s own system built on the ranked models; it is compared here on request, never ranked. Methodology → · Runs: Gemini 2026-09-15 · GPT 2026-09-15
Related: Gemini model page · GPT model page · Full leaderboard · Collections