Release analysis · draft, not yet published as a report
Mastermind V3 (skeptic + verification) on CDSBench v0.1
Mastermind is reliable on transferability (100%) but currency remains its weakest category at 100%; the generation's clearest gain is transferability (+43 pts).
Score
99.2
Previous generation
92.5
Mastermind V2 (deterministic rules)
Change
+6.7
Headline
What the measurements say
- Mastermind V3 (skeptic + verification) scores 99.2 on 296 verified questions on CDSBench v0.1.
- Against Mastermind V2 (deterministic rules) (92.5), overall moved up 6.7 points; strongest gain in transferability (+43 pts).
- Weakest category: currency at 100% (28/28), against 100% for Claude.
- It did not cite non-existent evidence on this run; wrong answers at ≥ 80% confidence: 0.0%.
Categories
Accuracy by category
| Category | Acc. | n | Δ |
|---|---|---|---|
| Currency | 100% | 28 | ±0 |
| Evidence | 100% | 30 | ±0 |
| Maturity | 100% | 26 | ±0 |
| Seniority | 100% | 30 | ±0 |
| Fact extraction | 100% | 56 | ±0 |
| Uncertainty | 100% | 32 | +6 |
| Entity match | 100% | 30 | +7 |
| Multi-document | 100% | 2 | ±0 |
| Restructuring | 100% | 2 | ±0 |
| Deliverability | 100% | 30 | +7 |
| Transferability | 100% | 30 | +43 |
Hardest failures
Highest confidence first
The founder publishes this as a versioned report from the workbench release workflow; it then lives under /research with PDF, citation and share actions. Model page →