Skip to content
CDSBench
Google · Gemini 2.5 Pro · benchmark v0.1 · 2026-09-15

Gemini

CDSBench score58.6on 296 verified questions

Not currently inside Mastermind. The promoted Mastermind version (v0) uses a different analyst model; this page is what tells us whether that should change.

Scores (CSV)

Every failure with a permanent page →Gemini vs Claude →Gemini vs GPT →Compare with Mastermind →

Accuracy
78%
Evidence
21%
cited the right sections
Complex reasoning
88%
multi-document, deliverability
Uncertainty
33%
abstains only when it should
Hallucination
21.3%
cites non-existent evidence
False confidence
22.3%
wrong at ≥ 80% confidence
Strongest

Where it is reliable

  • Transferability100%n=30
  • Restructuring100%n=2
  • Multi-document100%n=2
Weakest

Where it fails

  • Evidence0%n=30
  • Fact extraction68%n=56
  • Uncertainty69%n=32
Failure profile

Accuracy by category and question level

CategoryAcc.n
Evidence0%30
Fact extraction68%56
Uncertainty69%32
Deliverability73%30
Currency100%28
Maturity100%26
Seniority100%30
Entity match100%30
Multi-document100%2
Restructuring100%2
Transferability100%30
LevelAcc.n
Uncertainty69%32
Deliverability73%30
Multi document100%8
Fact extraction44%86
Rule application100%112
Document interpretation100%28
Failure types

How it goes wrong

Evidence selection failure117
Overconfident66
Hallucinated evidence63
Retrieval failure55
Wrong fact46
Wrong final answer12
Unsupported claim10
Missed uncertainty6
Failed to notice contradiction2
Prompt injection followed2
Recent failures

66 questions wrong or weakly supported

All 66 →

Clean successes

58 questions right with correct evidence

Performance over time

All official runs on this benchmark version

5075100Gemini 2.5 Pro2026-09-152026-09-15
DateVersionOverallAccuracy
2026-09-15 17:16Gemini 2.5 Pro58.678%
2026-09-15 19:59Gemini 2.5 Pro58.678%
2026-09-15 20:14Gemini 2.5 Pro58.678%
2026-09-15 20:36Gemini 2.5 Pro58.678%