What’s new
Meaningful additions to the benchmark
New benchmark versions, published cases, model versions, Mastermind versions, reports and corrections — read from the data, so nothing is listed that did not happen. Interface changes are not listed. RSS feed.
2026-09
40 changes
- 2026-09-15caseCase published: Halvard Steel AG 2028 (C)Halvard Steel AG · 10 questions
- 2026-09-15caseCase published: Republic of Norland 2027 (C)Republic of Norland · 10 questions
- 2026-09-15caseCase published: Solvane Airlines S.A. 2026 (B)Solvane Airlines S.A. · 9 questions
- 2026-09-15caseCase published: Commonwealth of Tessaly 2025 (B)Commonwealth of Tessaly · 10 questions
- 2026-09-15caseCase published: Calder Chemicals N.V. 2024 (B)Calder Chemicals N.V. · 10 questions
- 2026-09-15caseCase published: Republic of Marovia 2028 (B)Republic of Marovia · 10 questions
- 2026-09-15caseCase published: Northgate Shipping Ltd 2027 (B)Northgate Shipping Ltd · 10 questions
- 2026-09-15caseCase published: United Provinces of Kestrel 2026 (B)United Provinces of Kestrel · 10 questions
- 2026-09-15caseCase published: Meridian Telecom S.A. 2025 (B)Meridian Telecom S.A. · 9 questions
- 2026-09-15caseCase published: Republic of Sanderia 2024 (B)Republic of Sanderia · 9 questions
- 2026-09-15caseCase published: Brightwater Energy Corp 2028 (B)Brightwater Energy Corp · 11 questions
- 2026-09-15caseCase published: Federal Republic of Aurelia 2027 (B)Federal Republic of Aurelia · 10 questions
- 2026-09-15caseCase published: Orinoco Retail Holdings plc 2026 (B)Orinoco Retail Holdings plc · 10 questions
- 2026-09-15caseCase published: Kingdom of Vestmark 2025 (B)Kingdom of Vestmark · 10 questions
- 2026-09-15caseCase published: Halvard Steel AG 2024 (B)Halvard Steel AG · 10 questions
- 2026-09-15caseCase published: Republic of Norland 2028 (B)Republic of Norland · 10 questions
- 2026-09-15caseCase published: Solvane Airlines S.A. 2027Solvane Airlines S.A. · 9 questions
- 2026-09-15caseCase published: Commonwealth of Tessaly 2026Commonwealth of Tessaly · 10 questions
- 2026-09-15caseCase published: Calder Chemicals N.V. 2025Calder Chemicals N.V. · 10 questions
- 2026-09-15caseCase published: Republic of Marovia 2024Republic of Marovia · 10 questions
- 2026-09-15caseCase published: Northgate Shipping Ltd 2028Northgate Shipping Ltd · 10 questions
- 2026-09-15caseCase published: United Provinces of Kestrel 2027United Provinces of Kestrel · 10 questions
- 2026-09-15caseCase published: Meridian Telecom S.A. 2026Meridian Telecom S.A. · 9 questions
- 2026-09-15caseCase published: Republic of Sanderia 2025Republic of Sanderia · 9 questions
- 2026-09-15caseCase published: Brightwater Energy Corp 2024Brightwater Energy Corp · 11 questions
- 2026-09-15caseCase published: Federal Republic of Aurelia 2028Federal Republic of Aurelia · 10 questions
- 2026-09-15caseCase published: Orinoco Retail Holdings plc 2027Orinoco Retail Holdings plc · 10 questions
- 2026-09-15caseCase published: Kingdom of Vestmark 2026Kingdom of Vestmark · 10 questions
- 2026-09-15caseCase published: Halvard Steel AG 2025Halvard Steel AG · 10 questions
- 2026-09-15caseCase published: Republic of Norland 2024Republic of Norland · 10 questions
- 2026-09-15modelModel version added: Gemini 2.5 ProGoogle
- 2026-09-15modelModel version added: Claude Sonnet 5Anthropic
- 2026-09-15modelModel version added: GPT-5OpenAI
- 2026-09-15mastermindMastermind V5 full orchestrationAdds multi-model consensus on contested conclusions, alternative hypotheses, and system confidence computed from pipeline signals with observed calibration.
- 2026-09-15mastermindMastermind V4 precedent + evidence retrievalAnalogous historical cases retrieved from the CDSBench corpus with similarities/differences; section-level evidence retrieval for large records.
- 2026-09-15mastermindMastermind V3 adversarial verificationPrimary analyst → skeptic → claim-level evidence verification → contradiction check → revised result.
- 2026-09-15mastermindMastermind V2 deterministic rulesObjective logic moved into code: maturity, currency, seniority, identifiers, required fields.
- 2026-09-15mastermindMastermind V1 specialist structureCDS case schema, specialist prompt, strict JSON, evidence IDs, insufficient-information state.
- 2026-09-15mastermindMastermind V0 raw baseline (promoted)Standardized prompt → frontier model → structured answer.
- 2026-09-15benchmarkBenchmark v0.1 opened (current)