Release analysis · draft, not yet published as a report
Claude Sonnet 5 on CDSBench v0.1
Claude is reliable on entity match (100%) but transferability remains its weakest category at 57%; its citations need checking before use (6.1% non-existent evidence).
Score
91.2
Previous generation
—
Change
—
Headline
What the measurements say
- Claude Sonnet 5 scores 91.2 on 296 verified questions, 1st of 3 frontier models on CDSBench v0.1.
- This is the first Claude generation on this benchmark version, so there is no predecessor to compare with.
- Weakest category: transferability at 57% (17/30).
- It cited evidence that does not exist in the record in 6.1% of answers, and answered wrongly at ≥ 80% confidence in 0.0%.
- On the 28 verified questions it missed, Mastermind (CDSBench's own system, reference only, 91.2) answered 0 correctly and abstained on 18.
Categories
Accuracy by category
| Category | Acc. | n | Δ |
|---|---|---|---|
| Transferability | 57% | 30 | — |
| Uncertainty | 69% | 32 | — |
| Deliverability | 83% | 30 | — |
| Currency | 100% | 28 | — |
| Evidence | 100% | 30 | — |
| Maturity | 100% | 26 | — |
| Seniority | 100% | 30 | — |
| Fact extraction | 100% | 56 | — |
| Entity match | 100% | 30 | — |
| Multi-document | 100% | 2 | — |
| Restructuring | 100% | 2 | — |
Hardest failures
Highest confidence first
- #56 Brightwater Energy Corp 2024 — answered Review, verified Yes · Unnecessary review
- #57 Brightwater Energy Corp 2024 — answered Review, verified Eligible · Unnecessary review
- #194 Brightwater Energy Corp 2028 (B) — answered Review, verified Yes · Unnecessary review
- #195 Brightwater Energy Corp 2028 (B) — answered Review, verified Eligible · Unnecessary review
- #96 Northgate Shipping Ltd 2028 — answered Review, verified Eligible · Hallucinated evidence
- #116 Calder Chemicals N.V. 2025 — answered Review, verified Eligible · Unnecessary review
The founder publishes this as a versioned report from the workbench release workflow; it then lives under /research with PDF, citation and share actions. Model page →