reasoning
Mar 18, 2026EVAL-20260318-163754During WWII, analysts studied bullet holes on returning bombers to decide where to add armor. They found most damage on the wings and fuselage, almost none on the engines. Their recommendation: armor the wings. Abraham Wald disagreed. (1) What was Wald's reasoning? (2) Give 5 modern examples of survivorship bias in business/tech. (3) 'We studied 100 successful startups and found they all did X.' Why is this analysis worthless without a control group?
Winner
GPT-5.4
openrouter
9.73
WINNER SCORE
matrix avg: 9.48
10×10 Judgment Matrix · 56 judgments
OPEN DATA
| Judge ↓ / Respondent → | MiniMax M2.7 | MiniMax M2.5 | MiniMax M2.1 | MiniMax M2 | MiniMax M1 | MiniMax-01 | Claude Sonnet 4.6 | GPT-5.4 |
|---|---|---|---|---|---|---|---|---|
| MiniMax M2.7 | — | 9.4 | 9.3 | 10.0 | 9.4 | 9.0 | 9.4 | 9.0 |
| MiniMax M2.5 | 10.0 | — | 9.8 | 10.0 | 10.0 | 9.6 | 10.0 | 10.0 |
| MiniMax M2.1 | 10.0 | 9.6 | — | 9.8 | 10.0 | 9.1 | 9.8 | 10.0 |
| MiniMax M2 | 10.0 | 10.0 | 9.8 | — | 9.8 | 8.8 | 9.8 | 9.4 |
| MiniMax M1 | 10.0 | 8.3 | 9.0 | 9.6 | — | 8.8 | 9.6 | 10.0 |
| MiniMax-01 | 9.8 | 10.0 | 9.8 | 10.0 | 9.7 | — | 10.0 | 9.8 |
| Claude Sonnet 4.6 | 9.4 | 9.3 | 9.7 | 9.4 | 9.8 | 8.3 | — | 9.8 |
| GPT-5.4 | 8.4 | 9.3 | 7.7 | 9.0 | 9.0 | 8.3 | 8.7 | — |