{ "evaluation_id": "EVAL-20260402-122330", "question_id": "CODE-004", "question_text": "This function works but is O(n³). Optimize it to O(n log n) or better while maintaining correctness.\n\n```python\ndef find_triplets_with_sum(arr, target_sum):\n \"\"\"Find all unique triplets in arr that sum to target_sum\"\"\"\n n = len(arr)\n result = []\n \n for i in range(n):\n for j in range(i + 1, n):\n for k in range(j + 1, n):\n if arr[i] + arr[j] + arr[k] == target_sum:\n triplet = sorted([arr[i], arr[j], arr[k]])\n if triplet not in result:\n result.append(triplet)\n \n return result\n```\n\nExplain your optimization approach and prove the new time complexity.", "category": "code", "timestamp": "2026-02-03T00:00:00.000Z", "display_date": "Feb 03, 2026", "winner": { "name": "Grok 4.20", "provider": "openrouter", "score": 9.49 }, "avg_score": 8.87, "matrix_size": 89, "models_used": [ { "id": "claude_opus_46", "name": "Claude Opus 4.6", "provider": "openrouter" }, { "id": "gemini_31_pro", "name": "Gemini 3.1 Pro", "provider": "openrouter" }, { "id": "claude_sonnet_46", "name": "Claude Sonnet 4.6", "provider": "openrouter" }, { "id": "grok_420", "name": "Grok 4.20", "provider": "openrouter" }, { "id": "deepseek_v4", "name": "DeepSeek V4", "provider": "openrouter" }, { "id": "gpt_oss_120b", "name": "GPT-OSS-120B", "provider": "OpenAI" }, { "id": "gemini_3_flash", "name": "Gemini 3 Flash Preview", "provider": "Google" }, { "id": "minimax_m25", "name": "MiniMax M2.5", "provider": "openrouter" }, { "id": "mimo_v2_flash", "name": "MiMo-V2-Flash", "provider": "Xiaomi" }, { "id": "gpt_5_4", "name": "GPT-5.4", "provider": "openrouter" } ], "rankings": { "grok_420": { "display_name": "Grok 4.20", "provider": "openrouter", "average_score": 9.49, "score_count": 9, "min_score": 8.8, "max_score": 10, "rank": 1 }, "gpt_5_4": { "display_name": "GPT-5.4", "provider": "openrouter", "average_score": 9.46, "score_count": 9, "min_score": 8.8, "max_score": 10, "rank": 2 }, "claude_sonnet_46": { "display_name": "Claude Sonnet 4.6", "provider": "openrouter", "average_score": 9.42, "score_count": 8, "min_score": 8.6, "max_score": 10, "rank": 3 }, "claude_opus_46": { "display_name": "Claude Opus 4.6", "provider": "openrouter", "average_score": 9.09, "score_count": 9, "min_score": 8.45, "max_score": 9.8, "rank": 4 }, "mimo_v2_flash": { "display_name": "MiMo-V2-Flash", "provider": "Xiaomi", "average_score": 9.04, "score_count": 9, "min_score": 7.75, "max_score": 10, "rank": 5 }, "gemini_3_flash": { "display_name": "Gemini 3 Flash Preview", "provider": "Google", "average_score": 8.94, "score_count": 9, "min_score": 7.75, "max_score": 10, "rank": 6 }, "gpt_oss_120b": { "display_name": "GPT-OSS-120B", "provider": "OpenAI", "average_score": 8.78, "score_count": 9, "min_score": 6.55, "max_score": 9.6, "rank": 7 }, "minimax_m25": { "display_name": "MiniMax M2.5", "provider": "openrouter", "average_score": 8.39, "score_count": 9, "min_score": 5.65, "max_score": 9.65, "rank": 8 }, "deepseek_v4": { "display_name": "DeepSeek V4", "provider": "openrouter", "average_score": 8.37, "score_count": 9, "min_score": 6.75, "max_score": 9.8, "rank": 9 }, "gemini_31_pro": { "display_name": "Gemini 3.1 Pro", "provider": "openrouter", "average_score": 7.72, "score_count": 9, "min_score": 3.6, "max_score": 9.8, "rank": 10 } }, "judgments": [ { "judge_key": "gpt_5_4", "judge_name": "GPT-5.4", "respondent_key": "claude_opus_46", "respondent_name": "Claude Opus 4.6", "weighted_score": 9.15, "brief_justification": "Accurate and well-structured optimization to O(n^2) using sorting plus two pointers, with solid duplicate handling, correctness reasoning, and complexity proof. Minor nit: Python list.sort() space details are a bit oversimplified, but overall highly useful and correct." }, { "judge_key": "gpt_5_4", "judge_name": "GPT-5.4", "respondent_key": "gemini_31_pro", "respondent_name": "Gemini 3.1 Pro", "weighted_score": 3.6, "brief_justification": "The response correctly notes that O(n log n) is not known for general 3SUM and suggests the standard sort-plus-two-pointers idea, but the provided code is incomplete and lacks duplicate-handling details, full explanation, and the requested time-complexity proof. As written, it is not a complete correct answer." }, { "judge_key": "gpt_5_4", "judge_name": "GPT-5.4", "respondent_key": "claude_sonnet_46", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 8.6, "brief_justification": "Accurately presents the standard sorted two-pointer optimization and correctly notes the true worst-case is O(n²), not O(n log n). Explanation and complexity discussion are strong. Minor issues: the docstring initially incorrectly claims O(n log n), and one verification assert expects a specific order that the function may not return." }, { "judge_key": "gpt_5_4", "judge_name": "GPT-5.4", "respondent_key": "grok_420", "respondent_name": "Grok 4.20", "weighted_score": 9.6, "brief_justification": "Accurate and standard O(n^2) 3Sum optimization with correct deduplication, clear explanation, and solid complexity argument including why O(n log n) is generally impossible due to output size. Slightly not fully formal as a proof, but very strong overall." }, { "judge_key": "gpt_5_4", "judge_name": "GPT-5.4", "respondent_key": "deepseek_v4", "respondent_name": "DeepSeek V4", "weighted_score": 8.4, "brief_justification": "Accurately replaces the cubic approach with the standard sort + two-pointer O(n^2) method, explains duplicate handling, and gives a reasonable complexity argument. Clear and practical, though the correctness proof is somewhat informal and does not deeply justify why the two-pointer scan finds all valid pairs." }, { "judge_key": "gpt_5_4", "judge_name": "GPT-5.4", "respondent_key": "gpt_oss_120b", "respondent_name": "GPT-OSS-120B", "weighted_score": 6.55, "brief_justification": "Good explanation of the standard sorted two-pointer optimization and solid O(n^2) complexity proof, but it does not meet the requested O(n log n) target claim, contains a few inaccuracies (e.g. original sorted/in check not truly constant time, Python sort space claim), and the example/code output is truncated." }, { "judge_key": "gpt_5_4", "judge_name": "GPT-5.4", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 8.8, "brief_justification": "Accurately replaces the cubic approach with the standard sorted two-pointer O(n^2) solution, explains duplicate handling and complexity clearly, and correctly notes that O(n log n) is not generally achievable for 3Sum. Minor caveat: the claim that uniqueness is ensured in O(1) via duplicate skipping is a bit simplified, and the impossibility statement is asserted rather than rigorously justified." }, { "judge_key": "gpt_5_4", "judge_name": "GPT-5.4", "respondent_key": "minimax_m25", "respondent_name": "MiniMax M2.5", "weighted_score": 5.65, "brief_justification": "Clear explanation of the standard sorted two-pointer O(n^2) approach and generally sound duplicate-handling ideas, but the response is incomplete/truncated, omits the actual code, and includes an overstated claim that two-pointers 'visit all pairs,' which is not literally true though the algorithm is still correct." }, { "judge_key": "gpt_5_4", "judge_name": "GPT-5.4", "respondent_key": "mimo_v2_flash", "respondent_name": "MiMo-V2-Flash", "weighted_score": 7.75, "brief_justification": "Strong overall answer with the standard correct O(n^2) two-pointer optimization, clear duplicate handling, and good complexity breakdown. However, the lower-bound argument is flawed: not every triplet can sum to one target, and the claim about worst-case output size is imprecise/inconsistent. Still practically useful and mostly correct." }, { "judge_key": "claude_opus_46", "judge_name": "Claude Opus 4.6", "respondent_key": "gpt_5_4", "respondent_name": "GPT-5.4", "weighted_score": 10, "brief_justification": "The response is exemplary across all dimensions. The optimized code is correct, using the standard sorting + two-pointer approach for 3Sum. The correctness argument is thorough, covering why no triplets are missed and why uniqueness is preserved. The time complexity proof is rigorous and well-structured, correctly arriving at O(n²) and noting this is optimal for the problem. The response also correctly addresses that the question's requested O(n log n) bound is not achievable for general 3-sum e" }, { "judge_key": "claude_opus_46", "judge_name": "Claude Opus 4.6", "respondent_key": "gemini_31_pro", "respondent_name": "Gemini 3.1 Pro", "weighted_score": 7.1, "brief_justification": "The response correctly identifies that O(n log n) is not achievable and proposes the optimal O(n^2) two-pointer approach, which is factually accurate. The explanation of the 3SUM lower bound is appropriate. However, the code is clearly truncated - it's missing the critical logic for advancing the two pointers (handling duplicates for left/right, and the cases for current_sum < target_sum and current_sum > target_sum), as well as the return statement. The complexity proof is also incomplete/missi" }, { "judge_key": "claude_opus_46", "judge_name": "Claude Opus 4.6", "respondent_key": "claude_sonnet_46", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 9.75, "brief_justification": "The response provides a correct O(n²) two-pointer solution with proper duplicate handling, early termination optimizations, and immutability of the original array. Crucially, the response honestly corrects the premise of the question — acknowledging that O(n log n) is not achievable for this problem due to potential O(n²) output size, which demonstrates deep understanding. The complexity proof is rigorous and well-structured. The correctness table, verification tests, and practical performance c" }, { "judge_key": "claude_opus_46", "judge_name": "Claude Opus 4.6", "respondent_key": "grok_420", "respondent_name": "Grok 4.20", "weighted_score": 9.8, "brief_justification": "The solution is correct, well-implemented, and handles all edge cases including duplicates and small arrays. The complexity analysis is rigorous and accurate, correctly noting that O(n²) is optimal for 3Sum due to potential output size. The explanation of the two-pointer technique, deduplication strategy, and correctness proof sketch are thorough and clear. The note about O(n log n) being unachievable due to output size is an excellent addition showing deep understanding." }, { "judge_key": "claude_opus_46", "judge_name": "Claude Opus 4.6", "respondent_key": "deepseek_v4", "respondent_name": "DeepSeek V4", "weighted_score": 8.8, "brief_justification": "The solution correctly optimizes from O(n³) to O(n²) using the well-known sorting + two-pointer technique. The code is correct, the complexity analysis is accurate, and the explanation is clear and well-structured. Minor note: the question asked for O(n log n) or better, and the response achieves O(n²) which doesn't meet that specific request—but O(n²) is the best known general solution for 3SUM, so this is practically the right answer. The proof of correctness could be slightly more rigorous, b" }, { "judge_key": "claude_opus_46", "judge_name": "Claude Opus 4.6", "respondent_key": "gpt_oss_120b", "respondent_name": "GPT-OSS-120B", "weighted_score": 9.2, "brief_justification": "The response provides a correct O(n²) optimization using the standard two-pointer approach after sorting. The time complexity proof is rigorous and well-structured. The explanation covers why the original is O(n³), how the optimization works, formal complexity analysis, space analysis, and includes a helpful comparison table. Minor issues: the response claims O(n log n) or better was the target but delivers O(n²) which is appropriate since O(n²) is the known optimal for 3-sum in the comparison m" }, { "judge_key": "claude_opus_46", "judge_name": "Claude Opus 4.6", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 9.2, "brief_justification": "The solution is correct, well-structured, and thoroughly explained. The sorting + two-pointer approach is the standard optimal solution for 3Sum. The response correctly notes that O(n log n) is not achievable and that O(n^2) is the best known complexity. The complexity proof is rigorous, duplicate handling is properly implemented, and the comparison table adds practical value. Minor note: the question asked for O(n log n) which is impossible, and the response correctly addresses this. One very m" }, { "judge_key": "claude_opus_46", "judge_name": "Claude Opus 4.6", "respondent_key": "minimax_m25", "respondent_name": "MiniMax M2.5", "weighted_score": 8.65, "brief_justification": "The response correctly identifies the O(n²) two-pointer approach, provides a thorough correctness proof with lemmas, and explains the complexity analysis well. The algorithm description is accurate and the duplicate-skipping logic is correctly explained. Minor issues: the complexity analysis table appears truncated at the end, and while the code is described in detail, the actual Python implementation is not explicitly shown (though the pseudocode is clear enough to implement). The claim about O" }, { "judge_key": "claude_opus_46", "judge_name": "Claude Opus 4.6", "respondent_key": "mimo_v2_flash", "respondent_name": "MiMo-V2-Flash", "weighted_score": 9.2, "brief_justification": "The response is excellent. The code is correct, implementing the classic sort + two-pointer approach for 3Sum. The explanation of why O(n log n) is impossible is a valuable insight, though the argument about output size could be slightly more rigorous (the worst case for unique triplets summing to a target is indeed O(n^2), not O(n^3) as briefly suggested before self-correcting). The complexity proof is clear and correct at O(n^2). The duplicate-skipping logic is properly implemented. The code c" }, { "judge_key": "gemini_31_pro", "judge_name": "Gemini 3.1 Pro", "respondent_key": "gpt_5_4", "respondent_name": "GPT-5.4", "weighted_score": 10, "brief_justification": "The response provides an optimal O(n^2) solution, correctly identifying that the requested O(n log n) bound is impossible for 3Sum due to the maximum possible number of valid triplets. The code, explanation, and complexity analysis are flawless." }, { "judge_key": "gemini_31_pro", "judge_name": "Gemini 3.1 Pro", "respondent_key": "claude_opus_46", "respondent_name": "Claude Opus 4.6", "weighted_score": 9.15, "brief_justification": "" }, { "judge_key": "gemini_31_pro", "judge_name": "Gemini 3.1 Pro", "respondent_key": "claude_sonnet_46", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 9, "brief_justification": "" }, { "judge_key": "gemini_31_pro", "judge_name": "Gemini 3.1 Pro", "respondent_key": "grok_420", "respondent_name": "Grok 4.20", "weighted_score": 10, "brief_justification": "The response correctly identifies that O(n log n) is impossible for 3Sum, provides the optimal O(n²) solution, and thoroughly explains the approach, complexity, and correctness." }, { "judge_key": "gemini_31_pro", "judge_name": "Gemini 3.1 Pro", "respondent_key": "deepseek_v4", "respondent_name": "DeepSeek V4", "weighted_score": 8.15, "brief_justification": "The AI correctly provided the optimal O(n²) solution for the 3SUM problem with clear explanations. However, it failed to address why the requested O(n log n) complexity is theoretically impossible (3SUM conjecture), missing an opportunity for deeper insight." }, { "judge_key": "gemini_31_pro", "judge_name": "Gemini 3.1 Pro", "respondent_key": "gpt_oss_120b", "respondent_name": "GPT-OSS-120B", "weighted_score": 8.05, "brief_justification": "" }, { "judge_key": "gemini_31_pro", "judge_name": "Gemini 3.1 Pro", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 10, "brief_justification": "The model correctly identifies that O(n log n) is impossible for the 3Sum problem, provides the optimal O(n^2) solution using the two-pointer approach, and thoroughly explains the logic and complexity." }, { "judge_key": "gemini_31_pro", "judge_name": "Gemini 3.1 Pro", "respondent_key": "minimax_m25", "respondent_name": "MiniMax M2.5", "weighted_score": 7.2, "brief_justification": "The AI correctly identifies that O(n^2) is the optimal complexity for 3-Sum and provides excellent theoretical proofs. However, the response is cut off at the end and completely missing the actual code implementation." }, { "judge_key": "gemini_31_pro", "judge_name": "Gemini 3.1 Pro", "respondent_key": "mimo_v2_flash", "respondent_name": "MiMo-V2-Flash", "weighted_score": 10, "brief_justification": "The AI correctly identifies that an O(n log n) solution is impossible due to the O(n^2) maximum output size, provides the optimal O(n^2) two-pointer solution, and clearly explains the logic, complexity, and duplicate handling." }, { "judge_key": "claude_sonnet_46", "judge_name": "Claude Sonnet 4.6", "respondent_key": "gpt_5_4", "respondent_name": "GPT-5.4", "weighted_score": 9.4, "brief_justification": "The response is excellent overall. The optimized O(n²) solution is correct and well-implemented with proper duplicate handling. The complexity analysis is rigorous and honest—correctly noting that O(n²) is the best achievable for full enumeration, not O(n log n) as the question requested, which is an important clarification. The proof of correctness covers all key aspects (no missed triplets, uniqueness). Minor deduction on correctness only because the question asked for O(n log n) and the respo" }, { "judge_key": "claude_sonnet_46", "judge_name": "Claude Sonnet 4.6", "respondent_key": "claude_opus_46", "respondent_name": "Claude Opus 4.6", "weighted_score": 9.2, "brief_justification": "The response correctly identifies that O(n²) is achievable (not O(n log n) as the question asked, but this is the actual optimal bound), implements the two-pointer approach properly with duplicate handling, provides a clear step-by-step trace, and proves complexity rigorously. Minor issue: the question asked for O(n log n) or better, and the response achieves O(n²) which is better than O(n³) but not O(n log n) — however, O(n²) is the theoretical optimum for this problem, so the response is actua" }, { "judge_key": "claude_sonnet_46", "judge_name": "Claude Sonnet 4.6", "respondent_key": "gemini_31_pro", "respondent_name": "Gemini 3.1 Pro", "weighted_score": 7.25, "brief_justification": "The response correctly identifies that O(n log n) is impossible for 3SUM and provides the optimal O(n²) solution with good explanation. However, the code appears to be truncated - the duplicate-skipping logic for left/right pointers and the pointer advancement after finding a match are missing, making the solution incomplete. The theoretical explanation about 3SUM lower bounds is accurate and adds good depth." }, { "judge_key": "claude_sonnet_46", "judge_name": "Claude Sonnet 4.6", "respondent_key": "grok_420", "respondent_name": "Grok 4.20", "weighted_score": 9.2, "brief_justification": "The response correctly implements the O(n²) two-pointer approach with proper deduplication. The complexity analysis is accurate and honest about why O(n log n) isn't achievable in general. The correctness proof sketch is solid. Minor note: the question asked for O(n log n) or better, and the response correctly explains why O(n²) is the practical optimum, which shows good depth. The code is clean and well-commented." }, { "judge_key": "claude_sonnet_46", "judge_name": "Claude Sonnet 4.6", "respondent_key": "deepseek_v4", "respondent_name": "DeepSeek V4", "weighted_score": 8.35, "brief_justification": "The response correctly optimizes to O(n²) using sort + two-pointer, with clear explanation and working code. Minor issue: the question asked for O(n log n) or better, but O(n²) is the best achievable for this problem (finding all triplets), so the response should have addressed why O(n log n) isn't achievable rather than implicitly accepting O(n²). The duplicate-skipping logic is correct and well-explained. The proof of correctness is adequate but could be more rigorous." }, { "judge_key": "claude_sonnet_46", "judge_name": "Claude Sonnet 4.6", "respondent_key": "gpt_oss_120b", "respondent_name": "GPT-OSS-120B", "weighted_score": 9, "brief_justification": "Excellent response with rigorous complexity proof, clear algorithm explanation, and well-structured code. The two-pointer approach is correctly implemented with duplicate handling. Minor issue: the response claims O(n log n) in the title but correctly identifies O(n²) throughout - this is actually honest and accurate. The sanity check example appears cut off at the end, which is a minor flaw. The formal proof is mathematically sound and the space analysis adds valuable depth." }, { "judge_key": "claude_sonnet_46", "judge_name": "Claude Sonnet 4.6", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 8.55, "brief_justification": "The response correctly identifies that O(n log n) is not achievable for 3Sum and delivers the optimal O(n²) solution with proper explanation. The implementation is correct with proper duplicate handling. The complexity analysis is accurate and well-explained. Minor issue: the claim that O(n log n) is 'theoretically impossible' for 3Sum is stated without nuance (it's a conjecture, not proven), but this is a minor point. The comparison table and step-by-step explanation add good practical value." }, { "judge_key": "claude_sonnet_46", "judge_name": "Claude Sonnet 4.6", "respondent_key": "minimax_m25", "respondent_name": "MiniMax M2.5", "weighted_score": 9, "brief_justification": "The response correctly identifies and implements the O(n²) two-pointer optimization, provides formal correctness proofs with three lemmas, and includes a clear complexity analysis table. The algorithm handles duplicates properly. Minor issue: the response appears truncated at the end of the complexity table, cutting off the final summary row. The proof of exhaustiveness for the two-pointer approach could be slightly more rigorous, but overall this is an excellent, well-structured response." }, { "judge_key": "claude_sonnet_46", "judge_name": "Claude Sonnet 4.6", "respondent_key": "mimo_v2_flash", "respondent_name": "MiMo-V2-Flash", "weighted_score": 9, "brief_justification": "The response correctly identifies that O(n log n) is impossible for this problem and provides an optimal O(n²) solution with the two-pointer technique. The explanation is thorough, covering sorting, two-pointer logic, duplicate handling, and complexity analysis. The theoretical argument about why O(n log n) is impossible is sound and well-explained. Minor issue: the duplicate-skipping logic has a subtle edge case (checking arr[left+1] when left+1 might equal right), but the overall approach is c" }, { "judge_key": "grok_420", "judge_name": "Grok 4.20", "respondent_key": "gpt_5_4", "respondent_name": "GPT-5.4", "weighted_score": 8.85, "brief_justification": "The response provides a correct O(n^2) two-pointer solution after sorting, with solid proofs of correctness, uniqueness handling, and time complexity analysis. It clearly explains the approach, includes an example, and notes why O(n^2) is optimal. Minor deduction for not achieving the requested O(n log n) bound (though it correctly explains why it's impossible in the worst case)." }, { "judge_key": "grok_420", "judge_name": "Grok 4.20", "respondent_key": "claude_opus_46", "respondent_name": "Claude Opus 4.6", "weighted_score": 8.45, "brief_justification": "The response correctly optimizes to O(n^2) using standard two-pointer after sorting, with good explanations, proofs, and examples. Minor issues: early termination conditions assume positive numbers (incorrect for negatives), and duplicate skipping for right pointer has a small off-by-one risk, but overall solid and maintains correctness." }, { "judge_key": "grok_420", "judge_name": "Grok 4.20", "respondent_key": "gemini_31_pro", "respondent_name": "Gemini 3.1 Pro", "weighted_score": 7.9, "brief_justification": "Response correctly identifies 3SUM lower bound, provides standard O(n^2) two-pointer solution after sorting, and explains approach well. However, code is incomplete (missing duplicate skipping for left/right pointers and the full while-loop logic for advancing pointers), so it doesn't fully maintain correctness for unique triplets or handle all cases." }, { "judge_key": "grok_420", "judge_name": "Grok 4.20", "respondent_key": "claude_sonnet_46", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 9, "brief_justification": "The response provides a correct O(n^2) two-pointer solution after sorting, with clear explanations, duplicate handling, early termination, correctness guarantees, complexity analysis acknowledging output-size lower bound, and verification tests. Minor inaccuracy in stated time complexity title but honestly corrected in text." }, { "judge_key": "grok_420", "judge_name": "Grok 4.20", "respondent_key": "deepseek_v4", "respondent_name": "DeepSeek V4", "weighted_score": 6.75, "brief_justification": "The response correctly optimizes to O(n^2) using sorting + two pointers and explains the approach well, but contains a bug in the duplicate-skipping logic for the left/right pointers (incorrect while conditions that can cause index errors or miss skips). It claims O(n^2) while the query asked for O(n log n) or better, and the proof is somewhat superficial without rigorous pair coverage argument." }, { "judge_key": "grok_420", "judge_name": "Grok 4.20", "respondent_key": "gpt_oss_120b", "respondent_name": "GPT-OSS-120B", "weighted_score": 8.85, "brief_justification": "The response provides a correct O(n²) two-pointer optimization with clear explanations, formal proofs, and duplicate handling. It maintains correctness while improving complexity significantly. Minor deduction for not achieving the requested O(n log n) and a truncated example at the end." }, { "judge_key": "grok_420", "judge_name": "Grok 4.20", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 7.75, "brief_justification": "The response correctly optimizes to O(n^2) using standard two-pointer technique after sorting, with good explanations and duplicate handling. However, it has minor bugs: duplicate-skipping logic after finding a triplet is incorrect (should advance past all duplicates for left/right separately), and it doesn't sort the triplets in the result as the original function did. It also claims O(n log n) is impossible without justifying why, though O(n^2) is indeed optimal." }, { "judge_key": "grok_420", "judge_name": "Grok 4.20", "respondent_key": "minimax_m25", "respondent_name": "MiniMax M2.5", "weighted_score": 8.65, "brief_justification": "Response provides correct O(n^2) two-pointer algorithm after sorting with strong proofs for correctness, uniqueness, and complexity. Minor incompleteness as code implementation is missing despite describing the algorithm, and complexity table is cut off." }, { "judge_key": "grok_420", "judge_name": "Grok 4.20", "respondent_key": "mimo_v2_flash", "respondent_name": "MiMo-V2-Flash", "weighted_score": 8.6, "brief_justification": "The response provides a correct and standard O(n^2) two-pointer solution after sorting, with good explanations, duplicate handling, and a solid proof of why O(n log n) is impossible in the general case due to output size. Minor issues include an incomplete comment about positive numbers and a minor inaccuracy in the worst-case triplet count (it's O(n^2) not O(n^3)), but overall highly accurate and useful." }, { "judge_key": "deepseek_v4", "judge_name": "DeepSeek V4", "respondent_key": "gpt_5_4", "respondent_name": "GPT-5.4", "weighted_score": 9.45, "brief_justification": "The response provides a correct and optimized solution with a thorough explanation of the approach, correctness proof, and time complexity analysis. It covers all aspects requested and offers additional optional improvements." }, { "judge_key": "deepseek_v4", "judge_name": "DeepSeek V4", "respondent_key": "claude_opus_46", "respondent_name": "Claude Opus 4.6", "weighted_score": 9.45, "brief_justification": "Highly accurate and thorough explanation with clear optimization steps and complexity proof." }, { "judge_key": "deepseek_v4", "judge_name": "DeepSeek V4", "respondent_key": "gemini_31_pro", "respondent_name": "Gemini 3.1 Pro", "weighted_score": 9.05, "brief_justification": "Accurate and clear explanation of the 3SUM problem's complexity limits, with a good start to the optimized solution." }, { "judge_key": "deepseek_v4", "judge_name": "DeepSeek V4", "respondent_key": "claude_sonnet_46", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 9.6, "brief_justification": "Detailed explanation with correctness guarantees and complexity proof, very useful and actionable." }, { "judge_key": "deepseek_v4", "judge_name": "DeepSeek V4", "respondent_key": "grok_420", "respondent_name": "Grok 4.20", "weighted_score": 9.6, "brief_justification": "The response provides a correct and optimal solution with clear explanation and proof of time complexity. It covers all aspects thoroughly and is highly actionable." }, { "judge_key": "deepseek_v4", "judge_name": "DeepSeek V4", "respondent_key": "gpt_oss_120b", "respondent_name": "GPT-OSS-120B", "weighted_score": 9.6, "brief_justification": "The response is highly accurate, thorough, and clearly explains the optimization approach with a detailed proof of time complexity. It provides practical value with actionable insights and maintains clarity throughout." }, { "judge_key": "deepseek_v4", "judge_name": "DeepSeek V4", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 9.6, "brief_justification": "Accurate, thorough, and clear explanation with strong depth and practical value." }, { "judge_key": "deepseek_v4", "judge_name": "DeepSeek V4", "respondent_key": "minimax_m25", "respondent_name": "MiniMax M2.5", "weighted_score": 9.65, "brief_justification": "The response provides a thorough and accurate optimization approach, including a detailed correctness proof and complexity analysis. It is clear and well-structured, with insightful depth and practical value." }, { "judge_key": "deepseek_v4", "judge_name": "DeepSeek V4", "respondent_key": "mimo_v2_flash", "respondent_name": "MiMo-V2-Flash", "weighted_score": 9.45, "brief_justification": "The response is factually accurate, thoroughly covers the topic, and provides a clear, well-structured explanation. The depth of analysis is excellent, addressing both the optimization approach and theoretical limits. The solution is highly practical and actionable." }, { "judge_key": "gpt_oss_120b", "judge_name": "GPT-OSS-120B", "respondent_key": "gpt_5_4", "respondent_name": "GPT-5.4", "weighted_score": 8.8, "brief_justification": "Accurate O(n^2) solution with clear code, thorough correctness and complexity proof, good explanations, and practical value." }, { "judge_key": "gpt_oss_120b", "judge_name": "GPT-OSS-120B", "respondent_key": "claude_opus_46", "respondent_name": "Claude Opus 4.6", "weighted_score": 8.8, "brief_justification": "Accurate O(n^2) solution with clear code, thorough explanation, complexity proof, and practical guidance." }, { "judge_key": "gpt_oss_120b", "judge_name": "GPT-OSS-120B", "respondent_key": "gemini_31_pro", "respondent_name": "Gemini 3.1 Pro", "weighted_score": 7.85, "brief_justification": "Accurately explains lower bound and offers an O(n^2) solution with clear reasoning, though the code snippet is incomplete and lacks full duplicate handling." }, { "judge_key": "gpt_oss_120b", "judge_name": "GPT-OSS-120B", "respondent_key": "grok_420", "respondent_name": "Grok 4.20", "weighted_score": 8.8, "brief_justification": "Provides a correct O(n²) algorithm with clear code, thorough explanation, proof of complexity, and practical utility." }, { "judge_key": "gpt_oss_120b", "judge_name": "GPT-OSS-120B", "respondent_key": "deepseek_v4", "respondent_name": "DeepSeek V4", "weighted_score": 7.5, "brief_justification": "Provides a correct O(n²) solution with clear explanation and code, but does not achieve the requested O(n log n) complexity." }, { "judge_key": "gpt_oss_120b", "judge_name": "GPT-OSS-120B", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 8.8, "brief_justification": "Provides a correct O(n^2) two-pointer solution, explains approach, handles duplicates, and gives a solid complexity proof; clear and practical." }, { "judge_key": "gpt_oss_120b", "judge_name": "GPT-OSS-120B", "respondent_key": "minimax_m25", "respondent_name": "MiniMax M2.5", "weighted_score": 8.25, "brief_justification": "Accurate O(n^2) solution with solid proof and clear explanation, but answer is slightly truncated and lacks concrete code, reducing completeness." }, { "judge_key": "gpt_oss_120b", "judge_name": "GPT-OSS-120B", "respondent_key": "mimo_v2_flash", "respondent_name": "MiMo-V2-Flash", "weighted_score": 8.6, "brief_justification": "Accurate O(n^2) solution with clear explanation, thorough handling of duplicates, solid complexity proof, and practical code." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "gpt_5_4", "respondent_name": "GPT-5.4", "weighted_score": 10, "brief_justification": "The response is perfect. It provides the standard optimal O(n^2) solution, correctly identifies that O(n log n) is not possible for this problem (as the output size can be O(n^2)), and provides a rigorous complexity proof and explanation of duplicate handling." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "claude_opus_46", "respondent_name": "Claude Opus 4.6", "weighted_score": 9.8, "brief_justification": "The response provides a perfect O(n^2) solution, which is the optimal complexity for the 3Sum problem. It includes early termination optimizations, handles duplicates correctly without using a set, and provides a clear complexity proof and step-by-step walkthrough." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "gemini_31_pro", "respondent_name": "Gemini 3.1 Pro", "weighted_score": 9.8, "brief_justification": "The response correctly identifies that the requested O(n log n) complexity is impossible for the 3SUM problem and provides the optimal O(n^2) solution instead. The explanation is technically sound and the code is efficient." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "claude_sonnet_46", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 10, "brief_justification": "The model correctly identifies that the prompt's requested complexity of O(n log n) is mathematically impossible for this problem and provides the optimal O(n^2) solution instead. It provides a thorough explanation, handles duplicates correctly, and includes early termination optimizations." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "grok_420", "respondent_name": "Grok 4.20", "weighted_score": 10, "brief_justification": "The response is excellent. It correctly identifies that while the prompt asked for O(n log n), O(n^2) is the actual optimal complexity for the 3Sum problem. It provides a perfect implementation, clear explanation, and accurate complexity analysis." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "deepseek_v4", "respondent_name": "DeepSeek V4", "weighted_score": 9.8, "brief_justification": "The response correctly identifies that 3Sum cannot be solved in O(n log n) and provides the optimal O(n^2) solution. The code is correct, handles duplicates properly, and the complexity analysis is accurate." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "gpt_oss_120b", "respondent_name": "GPT-OSS-120B", "weighted_score": 9.6, "brief_justification": "The response provides a correct O(n^2) solution, which is the standard optimal complexity for the 3-Sum problem. While the prompt asked for O(n log n), that complexity is generally impossible for this problem, and the model correctly identifies and proves the O(n^2) bound. The explanation and code are excellent." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "minimax_m25", "respondent_name": "MiniMax M2.5", "weighted_score": 9.45, "brief_justification": "The response provides a highly accurate and well-reasoned optimization from O(n^3) to O(n^2). While the prompt asked for O(n log n) or better, the response correctly identifies that O(n^2) is the optimal complexity for the 3-sum problem and provides a rigorous proof of correctness and complexity. The response was cut off at the very end, but the content provided is excellent." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "mimo_v2_flash", "respondent_name": "MiMo-V2-Flash", "weighted_score": 10, "brief_justification": "The model correctly identifies that the prompt's requested complexity of O(n log n) is mathematically impossible for the 3Sum problem, which is a known hard problem with an O(n^2) optimal solution. It provides the standard two-pointer optimization, correctly handles duplicates (which the original code did poorly), and provides a rigorous complexity proof." }, { "judge_key": "minimax_m25", "judge_name": "MiniMax M2.5", "respondent_key": "gpt_5_4", "respondent_name": "GPT-5.4", "weighted_score": 9.8, "brief_justification": "Excellent solution with correct O(n²) two-pointer approach, thorough proof of correctness, detailed complexity analysis, and clear explanation of why O(n²) is optimal for 3-sum enumeration. Well-structured with code, proofs, and examples." }, { "judge_key": "minimax_m25", "judge_name": "MiniMax M2.5", "respondent_key": "claude_opus_46", "respondent_name": "Claude Opus 4.6", "weighted_score": 8.8, "brief_justification": "Correct algorithm using two-pointer technique. Thorough explanation with step-by-step example, complexity proof, and correctness argument. Well-structured and practical. Note: achieves O(n²), which is optimal for this problem (3SUM cannot be solved faster than O(n²) in worst case), though the question asked for O(n log n) or better." }, { "judge_key": "minimax_m25", "judge_name": "MiniMax M2.5", "respondent_key": "gemini_31_pro", "respondent_name": "Gemini 3.1 Pro", "weighted_score": 8.35, "brief_justification": "Correctly explains the O(n²) lower bound for 3SUM and provides the optimal two-pointer solution. However, the code snippet is incomplete (missing the else branches for pointer movement and return statement), which significantly impacts completeness and practical usefulness." }, { "judge_key": "minimax_m25", "judge_name": "MiniMax M2.5", "respondent_key": "claude_sonnet_46", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 10, "brief_justification": "Excellent response that correctly identifies O(n²) as optimal (not O(n log n) as requested), provides working optimized code with proper duplicate handling, includes thorough complexity proof, and demonstrates strong insight into why the lower bound exists. Well-structured with verification tests." }, { "judge_key": "minimax_m25", "judge_name": "MiniMax M2.5", "respondent_key": "grok_420", "respondent_name": "Grok 4.20", "weighted_score": 9.25, "brief_justification": "Correct O(n²) solution using two-pointer technique with thorough explanation. Properly proves time complexity and explains why O(n log n) isn't achievable due to potential O(n²) output size. Well-structured with clear code and reasoning." }, { "judge_key": "minimax_m25", "judge_name": "MiniMax M2.5", "respondent_key": "deepseek_v4", "respondent_name": "DeepSeek V4", "weighted_score": 8.8, "brief_justification": "Well-structured response with correct O(n²) optimization using two-pointer technique. Code is correct, explanations are clear, and complexity proof is provided. Minor扣分 for not explicitly mentioning space complexity implications of sorting in-place vs. out-of-place." }, { "judge_key": "minimax_m25", "judge_name": "MiniMax M2.5", "respondent_key": "gpt_oss_120b", "respondent_name": "GPT-OSS-120B", "weighted_score": 8.6, "brief_justification": "Correct O(n²) two-pointer solution with thorough complexity proof and clear explanations. Well-structured response with good depth, though the example output is truncated at the end." }, { "judge_key": "minimax_m25", "judge_name": "MiniMax M2.5", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 8.4, "brief_justification": "Correctly optimizes from O(n³) to O(n²) using the standard sorting + two-pointer technique. Well-structured code with clear explanations of the optimization approach and complexity analysis. Accurately notes that O(n log n) is impossible for 3Sum. Minor deduction for depth - could explain why lower complexity is theoretically impossible." }, { "judge_key": "minimax_m25", "judge_name": "MiniMax M2.5", "respondent_key": "mimo_v2_flash", "respondent_name": "MiMo-V2-Flash", "weighted_score": 8.8, "brief_justification": "Correct O(n²) two-pointer solution with thorough explanation. Well-structured code and clear reasoning for why O(n log n) is theoretically impossible. Minor depth deduction for not fully proving the O(n²) output bound mathematically." }, { "judge_key": "mimo_v2_flash", "judge_name": "MiMo-V2-Flash", "respondent_key": "gpt_5_4", "respondent_name": "GPT-5.4", "weighted_score": 8.8, "brief_justification": "Accurate solution with correct complexity analysis, thorough explanation, and practical code. Minor deduction for not addressing the requested O(n log n) bound explicitly." }, { "judge_key": "mimo_v2_flash", "judge_name": "MiMo-V2-Flash", "respondent_key": "claude_opus_46", "respondent_name": "Claude Opus 4.6", "weighted_score": 9, "brief_justification": "Accurate optimization to O(n²), thorough explanation with proof, clear structure, and practical code." }, { "judge_key": "mimo_v2_flash", "judge_name": "MiMo-V2-Flash", "respondent_key": "gemini_31_pro", "respondent_name": "Gemini 3.1 Pro", "weighted_score": 8.6, "brief_justification": "Accurate explanation of 3SUM lower bound, correct O(n^2) optimization, clear code and complexity proof, though response is cut off mid-code." }, { "judge_key": "mimo_v2_flash", "judge_name": "MiMo-V2-Flash", "respondent_key": "claude_sonnet_46", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 9.4, "brief_justification": "Accurate and thorough explanation with correct complexity analysis, though O(n log n) is not achievable for this problem." }, { "judge_key": "mimo_v2_flash", "judge_name": "MiMo-V2-Flash", "respondent_key": "grok_420", "respondent_name": "Grok 4.20", "weighted_score": 9.2, "brief_justification": "Accurate O(n²) solution with clear explanation, proof, and complexity analysis." }, { "judge_key": "mimo_v2_flash", "judge_name": "MiMo-V2-Flash", "respondent_key": "deepseek_v4", "respondent_name": "DeepSeek V4", "weighted_score": 8.8, "brief_justification": "The response provides a correct and standard optimization (two-pointer technique) achieving O(n^2) complexity. It includes clear code, explanation, and proof. Minor deduction in depth for not explicitly stating the target was O(n log n) but achieving O(n^2), though O(n^2) is the best known for this specific problem (finding all unique triplets)." }, { "judge_key": "mimo_v2_flash", "judge_name": "MiMo-V2-Flash", "respondent_key": "gpt_oss_120b", "respondent_name": "GPT-OSS-120B", "weighted_score": 9.55, "brief_justification": "Accurate O(n²) solution with thorough explanation, proof, and code; clear structure and high practical value." }, { "judge_key": "mimo_v2_flash", "judge_name": "MiMo-V2-Flash", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 9.35, "brief_justification": "The response correctly identifies the standard O(n^2) solution for the 3Sum problem, provides a working implementation, and accurately explains the time complexity. It correctly notes that O(n log n) is impossible for this problem class." }, { "judge_key": "mimo_v2_flash", "judge_name": "MiMo-V2-Flash", "respondent_key": "minimax_m25", "respondent_name": "MiniMax M2.5", "weighted_score": 9, "brief_justification": "The response provides a correct, optimal O(n^2) algorithm, a clear explanation of the approach, a formal proof of correctness, and a complexity analysis. It correctly identifies the lower bound and handles duplicates." } ], "meta": { "source": "The Multivac (app.themultivac.com)", "methodology": "10x10 blind peer matrix evaluation", "criteria": "correctness, completeness, clarity, depth, usefulness", "self_judgments": "excluded from rankings", "license": "Open data — cite as: The Multivac (2026)" } }