{ "evaluation_id": "EVAL-20260318-163203", "question_id": "EVAL-20260318-163203", "question_text": "You must choose between three investments. Investment A returns 10% with 90% probability, -50% with 10% probability. Investment B returns 5% with certainty. Investment C returns 100% with 20% probability, 0% with 80% probability. (1) Rank them by expected value. (2) Rank them by the Kelly criterion. (3) You have $10,000 your entire savings. Does this change your answer? Why? (4) Now you have $10,000,000. Does it change again? Derive the general principle.", "category": "reasoning", "timestamp": "2026-03-18T16:32:03.732Z", "display_date": "Mar 18, 2026", "winner": { "name": "GPT-5.4", "provider": "openrouter", "score": 9.71 }, "avg_score": 8.79, "matrix_size": 21, "models_used": [ { "id": "minimax_01", "name": "MiniMax-01", "provider": "openrouter" }, { "id": "judge_claude_sonnet", "name": "Claude Sonnet 4.6", "provider": "openrouter" }, { "id": "judge_gpt54", "name": "GPT-5.4", "provider": "openrouter" } ], "rankings": { "judge_gpt54": { "display_name": "GPT-5.4", "provider": "openrouter", "average_score": 9.71, "score_count": 7, "min_score": 8.8, "max_score": 10, "rank": 1 }, "judge_claude_sonnet": { "display_name": "Claude Sonnet 4.6", "provider": "openrouter", "average_score": 9.42, "score_count": 7, "min_score": 7.6, "max_score": 10, "rank": 2 }, "minimax_01": { "display_name": "MiniMax-01", "provider": "openrouter", "average_score": 7.24, "score_count": 7, "min_score": 4.35, "max_score": 9.1, "rank": 3 } }, "judgments": [ { "judge_key": "minimax_m27", "judge_name": "MiniMax M2.7", "respondent_key": "minimax_01", "respondent_name": "MiniMax-01", "weighted_score": 9.1, "brief_justification": "The response provides accurate calculations for expected values and Kelly criterion for all three investments, with correct rankings for both methods. The analysis of how wealth affects investment decisions is thoughtful, though the discussion of the general principle could be more rigorous in connecting Kelly fraction sizing to absolute dollar amounts and risk of ruin." }, { "judge_key": "minimax_m27", "judge_name": "MiniMax M2.7", "respondent_key": "judge_claude_sonnet", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 10, "brief_justification": "The response provides mathematically correct calculations for expected value and Kelly criterion, correctly ranks investments, and offers a sophisticated analysis of how wealth constraints fundamentally alter optimal strategy. The formal treatment of utility near subsistence thresholds, the unified three-regime framework, and the ergodicity insight at the end demonstrate exceptional depth while remaining accessible." }, { "judge_key": "minimax_m27", "judge_name": "MiniMax M2.7", "respondent_key": "judge_gpt54", "respondent_name": "GPT-5.4", "weighted_score": 9.8, "brief_justification": "Calculations are accurate for both expected value (C=20%, B=5%, A=4%) and Kelly criterion (C≈0.139, B≈0.049, A≈0.016). Part 3 correctly identifies that small savings change the optimal choice to B due to survival/risk concerns, and Part 4 derives the key principle linking wealth level to optimal strategy, with appropriate distinction between EV/Kelly as special cases vs. expected utility as the general framework." }, { "judge_key": "minimax_m25", "judge_name": "MiniMax M2.5", "respondent_key": "minimax_01", "respondent_name": "MiniMax-01", "weighted_score": 7.15, "brief_justification": "Calculations for expected value and Kelly criterion are correct, but the analysis of how wealth changes the answer lacks depth and actionable guidance. The response doesn't clearly articulate why the ranking should or shouldn't change with different wealth levels." }, { "judge_key": "minimax_m25", "judge_name": "MiniMax M2.5", "respondent_key": "judge_claude_sonnet", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 10, "brief_justification": "Exceptionally thorough response with correct EV and Kelly calculations, well-structured explanations, and profound insight into how wealth level fundamentally changes optimal decision-making through the subsistence threshold concept and the transition between EV/Kelly regimes." }, { "judge_key": "minimax_m25", "judge_name": "MiniMax M2.5", "respondent_key": "judge_gpt54", "respondent_name": "GPT-5.4", "weighted_score": 10, "brief_justification": "Response correctly calculates expected values (A=4%, B=5%, C=20%) and Kelly criterion values (A≈0.0165, B≈0.0488, C≈0.1386), providing proper rankings. Thoroughly addresses all four parts of the question with excellent depth, explaining how wealth level affects risk tolerance through utility theory, diminishing marginal utility, and the distinction between one-shot bets vs. repeated opportunities. Clear structure with mathematical rigor and practical general principle." }, { "judge_key": "minimax_m21", "judge_name": "MiniMax M2.1", "respondent_key": "minimax_01", "respondent_name": "MiniMax-01", "weighted_score": 9, "brief_justification": "Mathematically correct calculations for expected value and Kelly criterion with clear structure. Slightly less depth on the general principle derivation, but covers the key considerations well." }, { "judge_key": "minimax_m21", "judge_name": "MiniMax M2.1", "respondent_key": "judge_claude_sonnet", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 10, "brief_justification": "The response provides mathematically correct calculations for both expected value and Kelly criterion, with thorough derivations. It comprehensively addresses all four parts of the question and introduces the cushion ratio framework and ergodicity concepts, demonstrating exceptional depth while maintaining clarity through excellent structure, tables, and step-by-step derivations." }, { "judge_key": "minimax_m21", "judge_name": "MiniMax M2.1", "respondent_key": "judge_gpt54", "respondent_name": "GPT-5.4", "weighted_score": 9.8, "brief_justification": "The response provides mathematically correct calculations for both expected value and Kelly criterion, thoroughly addresses all four parts of the question, presents information with excellent structure and clarity, and demonstrates strong analytical depth by connecting wealth levels to utility theory and Kelly assumptions. The general principle derived is insightful and practically valuable." }, { "judge_key": "minimax_m2", "judge_name": "MiniMax M2", "respondent_key": "minimax_01", "respondent_name": "MiniMax-01", "weighted_score": 8.25, "brief_justification": "The response demonstrates strong mathematical accuracy in expected value and Kelly criterion calculations, with clear organization and step-by-step explanations. While it addresses all parts of the question and provides practical insights about capital size considerations, it could have explored deeper the theoretical foundations (like utility functions and risk aversion) and more explicitly derived the general principle about capital size affecting optimal investment strategies." }, { "judge_key": "minimax_m2", "judge_name": "MiniMax M2", "respondent_key": "judge_claude_sonnet", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 10, "brief_justification": "Comprehensive and accurate analysis covering all four parts of the question with correct EV and Kelly calculations. The response excels in depth by deriving a formal utility framework with subsistence thresholds and explaining the ergodicity problem, providing genuinely actionable principles that apply across wealth levels." }, { "judge_key": "minimax_m2", "judge_name": "MiniMax M2", "respondent_key": "judge_gpt54", "respondent_name": "GPT-5.4", "weighted_score": 10, "brief_justification": "Mathematically correct with accurate EV and Kelly calculations. Thoroughly addresses all four parts of the question. Clear structure with well-explained reasoning. Provides insightful discussion of utility theory, diminishing marginal returns, and practical constraints. Derives a useful general principle about risk tolerance based on wealth relative to needs." }, { "judge_key": "minimax_m1", "judge_name": "MiniMax M1", "respondent_key": "minimax_01", "respondent_name": "MiniMax-01", "weighted_score": 7, "brief_justification": "The response correctly calculates expected values and provides a reasonable Kelly criterion analysis, but uses a non-standard Kelly formula interpretation. It addresses wealth-dependent preferences but lacks depth on utility theory, risk of ruin, and fractional Kelly strategies." }, { "judge_key": "minimax_m1", "judge_name": "MiniMax M1", "respondent_key": "judge_claude_sonnet", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 9.55, "brief_justification": "The response correctly calculates EV and Kelly criterion rankings, provides comprehensive analysis covering all four parts of the question, and demonstrates exceptional depth by formalizing the utility transitions and subsistence threshold principle. The clarity is excellent with well-structured sections, though minor notational issues slightly reduce correctness." }, { "judge_key": "minimax_m1", "judge_name": "MiniMax M1", "respondent_key": "judge_gpt54", "respondent_name": "GPT-5.4", "weighted_score": 9.8, "brief_justification": "Response correctly computes all expected values and Kelly criterion values, provides accurate rankings, and thoroughly explains how wealth level changes the optimal choice due to utility considerations and risk tolerance. Well-structured with clear mathematical notation and a useful general principle." }, { "judge_key": "minimax_01", "judge_name": "MiniMax-01", "respondent_key": "judge_claude_sonnet", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 8.8, "brief_justification": "The response demonstrates a high level of correctness and depth, with a comprehensive analysis of the problem using both expected value and the Kelly criterion. It provides clear reasoning for each scenario and the transitions between them. However, the structure could be slightly more concise, and some explanations could be streamlined for better clarity." }, { "judge_key": "minimax_01", "judge_name": "MiniMax-01", "respondent_key": "judge_gpt54", "respondent_name": "GPT-5.4", "weighted_score": 9.8, "brief_justification": "The response provides a comprehensive and accurate analysis of the problem, correctly deriving the expected values and Kelly criterion rankings, and thoroughly explains the impact of wealth levels on decision-making. The reasoning is clear and insightful, with only minor improvements possible in structure for enhanced readability." }, { "judge_key": "judge_claude_sonnet", "judge_name": "Claude Sonnet 4.6", "respondent_key": "minimax_01", "respondent_name": "MiniMax-01", "weighted_score": 5.85, "brief_justification": "The expected value calculations are correct, and Investment B's Kelly fraction of 100% is correctly identified. However, the Kelly criterion formula is misapplied for Investment A—the standard Kelly formula for investments with two outcomes uses log-growth maximization, and the b parameter should represent the win/loss ratio differently (b=0.1 for win, loss=0.5, giving f*=(0.9/0.5)-(0.1/0.1)... actually the correct formula yields a different result). The response also fails to derive the general" }, { "judge_key": "judge_claude_sonnet", "judge_name": "Claude Sonnet 4.6", "respondent_key": "judge_gpt54", "respondent_name": "GPT-5.4", "weighted_score": 8.8, "brief_justification": "The response correctly computes expected values and Kelly criterion rankings, properly identifies that both give C>B>A, and provides a nuanced discussion of how wealth level relative to needs affects optimal choice. The general principle about expected utility maximization is well-articulated, though the Kelly section could have more explicitly noted that Kelly is about optimal fraction sizing rather than all-in choices, and the transition from Kelly to utility theory could be developed more rig" }, { "judge_key": "judge_gpt54", "judge_name": "GPT-5.4", "respondent_key": "minimax_01", "respondent_name": "MiniMax-01", "weighted_score": 4.35, "brief_justification": "The response gets expected values right, but its Kelly analysis is substantially incorrect: it applies the binary-bet formula improperly, concluding C has negative Kelly when its expected log growth at full investment is positive and actually exceeds A and B. It also fails to derive the key principle correctly: with entire savings, utility/ruin risk and one-period consumption needs matter; with large wealth, optimal choice depends on fraction of total wealth at risk, not absolute dollars." }, { "judge_key": "judge_gpt54", "judge_name": "GPT-5.4", "respondent_key": "judge_claude_sonnet", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 7.6, "brief_justification": "The EV and Kelly rankings are computed correctly under the implicit assumption you may hold cash, and the discussion of wealth-dependent risk tolerance is thoughtful. However, the answer blurs Kelly with ad hoc utility/liquidity concerns, does not clearly state that with a single all-in choice and no cash option B may be preferred under expected log wealth, and includes some overreaching claims (e.g. lottery tickets/ergodicity) that are not cleanly derived from the problem." } ], "meta": { "source": "The Multivac (app.themultivac.com)", "methodology": "10x10 blind peer matrix evaluation", "criteria": "correctness, completeness, clarity, depth, usefulness", "self_judgments": "excluded from rankings", "license": "Open data — cite as: The Multivac (2026)" } }