← Evaluations/EVAL-20260207-133335
reasoning
Feb 04, 2026REASON-004

On an island, there are 100 people with blue eyes and 100 people with brown eyes. Everyone can see everyone else's eye color but not their own. There are no mirrors or ways to discover your own eye color. There's a rule: if you ever figure out your own eye color, you must leave the island at midnight that day. One day, a visitor announces to everyone: "At least one person on this island has blue eyes." What happens? When? Why? Be precise about the logical chain of events.

Winner
Grok 3 (Direct)
xAI
9.98
WINNER SCORE
matrix avg: 9.62
↓ results.json↓ report.mdFull dataset (CSV) →
10×10 Judgment Matrix · 100 judgments
OPEN DATA
Judge ↓ / Respondent →Gemini 2.5 FlashMiMo-V2-FlashGemini 3Claude Sonnet 4.5DeepSeek V3.2Claude Opus 4.5Gemini 3GPT-OSS-120BOLMo ThinkGrok 3 (Direct)
Gemini 2.5 Flash—10.09.810.010.09.89.110.010.010.0
MiMo-V2-Flash10.0—10.010.010.010.010.010.010.010.0
Gemini 310.010.0—10.010.010.00.010.010.010.0
Claude Sonnet 4.510.010.010.0—9.810.00.010.010.010.0
DeepSeek V3.29.79.59.39.3—9.39.89.89.89.8
Claude Opus 4.59.79.89.79.79.8—0.910.09.810.0
Gemini 310.00.010.00.010.010.0—0.00.010.0
GPT-OSS-120B8.80.00.09.39.30.00.0—0.00.0
OLMo Think0.010.00.00.00.010.00.00.0—0.0
Grok 3 (Direct)9.59.89.89.89.59.58.39.89.5—