← Evaluations/EVAL-20260402-182524
reasoning
Apr 02, 2026REASON-029

During WWII, analysts studied bullet holes on returning bombers to decide where to add armor. They found most damage on the wings and fuselage, almost none on the engines. Their recommendation: armor the wings. Abraham Wald disagreed. (1) What was Wald's reasoning? (2) Give 5 modern examples of survivorship bias in business/tech. (3) 'We studied 100 successful startups and found they all did X.' Why is this analysis worthless without a control group?

Winner
Claude Opus 4.6
openrouter
9.65
WINNER SCORE
matrix avg: 9.39
↓ results.json↓ report.mdFull dataset (CSV) →
10×10 Judgment Matrix · 89 judgments
OPEN DATA
Judge ↓ / Respondent →GPT-OSS-120BGemini 3.1 ProDeepSeek V4Claude Opus 4.6GPT-5.4Grok 4.20Claude Sonnet 4.6MiMo-V2-FlashGemini 2.5 FlashMiniMax M2.5
GPT-OSS-120B—8.78.38.78.38.78.78.78.78.3
Gemini 3.1 Pro9.8—9.810.09.89.810.010.010.09.8
DeepSeek V49.49.4—10.09.49.410.09.4·9.4
Claude Opus 4.69.89.49.0—9.49.49.69.89.49.8
GPT-5.49.08.69.310.0—9.49.88.89.89.8
Grok 4.209.48.89.09.29.8—9.08.89.08.8
Claude Sonnet 4.69.88.89.310.09.49.8—9.28.89.4
MiMo-V2-Flash9.89.09.010.09.810.09.8—9.89.8
Gemini 2.5 Flash9.410.09.410.010.09.810.010.0—9.8
MiniMax M2.59.78.48.69.09.08.89.08.69.8—