reasoning
Mar 17, 2026EVAL-20260317-030901During WWII, analysts studied bullet holes on returning bombers to decide where to add armor. They found most damage on the wings and fuselage, almost none on the engines. Their recommendation: armor the wings. Abraham Wald disagreed. (1) What was Wald's reasoning? (2) Give 5 modern examples of survivorship bias in business/tech. (3) 'We studied 100 successful startups and found they all did X.' Why is this analysis worthless without a control group?
Winner
Qwen 3.5 397B-A17B
openrouter
9.95
WINNER SCORE
matrix avg: 9.02
10×10 Judgment Matrix · 39 judgments
OPEN DATA
| Judge ↓ / Respondent → | Qwen 3 8B | Qwen 3 32B | Qwen 3 Coder Next | Qwen 3.5 35B-A3B | Qwen 3.5 27B | Qwen 3.5 122B-A10B | Qwen 3.5 397B-A17B | Qwen 3.5 9B |
|---|---|---|---|---|---|---|---|---|
| Qwen 3 8B | — | 9.8 | 9.6 | 8.7 | 8.2 | 9.3 | 9.8 | 9.0 |
| Qwen 3 32B | 9.6 | — | 10.0 | 8.6 | 8.4 | 10.0 | 10.0 | 8.4 |
| Qwen 3 Coder Next | 10.0 | 10.0 | — | 10.0 | 9.6 | 10.0 | 10.0 | 9.6 |
| Qwen 3.5 35B-A3B | 9.8 | · | · | — | · | · | · | · |
| Qwen 3.5 27B | 9.8 | · | · | 5.5 | — | 9.8 | · | 6.3 |
| Qwen 3.5 122B-A10B | · | 9.8 | 9.8 | 6.4 | 5.7 | — | 10.0 | 5.3 |
| Qwen 3.5 397B-A17B | 9.8 | 10.0 | 10.0 | 7.5 | 6.3 | 10.0 | — | 6.4 |
| Qwen 3.5 9B | · | · | · | · | · | · | · | — |