{ "evaluation_id": "EVAL-20260402-125437", "question_id": "CODE-010", "question_text": "Create TypeScript types that enforce these compile-time constraints:\n\n1. A `Route` type where the path parameters in the URL template must match the params object:\n - `createRoute('/users/:userId/posts/:postId', { userId: '123', postId: '456' })` ✓\n - `createRoute('/users/:userId', { wrongParam: '123' })` ✗ (compile error)\n\n2. A `Builder` pattern type where methods can only be called in a specific order:\n - `builder.setName('x').setAge(25).build()` ✓\n - `builder.setAge(25).setName('x').build()` ✗ (compile error)\n - `builder.setName('x').build()` ✗ (compile error - age not set)\n\nShow the type definitions and example usage.", "category": "code", "timestamp": "2026-03-17T00:00:00.000Z", "display_date": "Mar 17, 2026", "winner": { "name": "Claude Sonnet 4.6", "provider": "openrouter", "score": 9.14 }, "avg_score": 7.6355557, "matrix_size": 75, "models_used": [ { "id": "gemini_31_pro", "name": "Gemini 3.1 Pro", "provider": "openrouter" }, { "id": "claude_opus_46", "name": "Claude Opus 4.6", "provider": "openrouter" }, { "id": "claude_sonnet_46", "name": "Claude Sonnet 4.6", "provider": "openrouter" }, { "id": "grok_420", "name": "Grok 4.20", "provider": "openrouter" }, { "id": "deepseek_v4", "name": "DeepSeek V4", "provider": "openrouter" }, { "id": "gpt_oss_120b", "name": "GPT-OSS-120B", "provider": "OpenAI" }, { "id": "gemini_3_flash", "name": "Gemini 3 Flash Preview", "provider": "Google" }, { "id": "mimo_v2_flash", "name": "MiMo-V2-Flash", "provider": "Xiaomi" }, { "id": "gpt_5_4", "name": "GPT-5.4", "provider": "openrouter" } ], "rankings": { "claude_sonnet_46": { "display_name": "Claude Sonnet 4.6", "provider": "openrouter", "average_score": 9.14, "score_count": 9, "min_score": 8.4, "max_score": 10, "rank": 1 }, "claude_opus_46": { "display_name": "Claude Opus 4.6", "provider": "openrouter", "average_score": 9.04, "score_count": 7, "min_score": 8.2, "max_score": 10, "rank": 2 }, "gpt_5_4": { "display_name": "GPT-5.4", "provider": "openrouter", "average_score": 8.94, "score_count": 9, "min_score": 8.4, "max_score": 9.8, "rank": 3 }, "gemini_3_flash": { "display_name": "Gemini 3 Flash Preview", "provider": "Google", "average_score": 8.72, "score_count": 9, "min_score": 8.3, "max_score": 9.45, "rank": 4 }, "mimo_v2_flash": { "display_name": "MiMo-V2-Flash", "provider": "Xiaomi", "average_score": 8.49, "score_count": 8, "min_score": 7.35, "max_score": 9.8, "rank": 5 }, "deepseek_v4": { "display_name": "DeepSeek V4", "provider": "openrouter", "average_score": 7.71, "score_count": 9, "min_score": 6.55, "max_score": 9.8, "rank": 6 }, "grok_420": { "display_name": "Grok 4.20", "provider": "openrouter", "average_score": 7.64, "score_count": 9, "min_score": 5.75, "max_score": 9.55, "rank": 7 }, "gpt_oss_120b": { "display_name": "GPT-OSS-120B", "provider": "OpenAI", "average_score": 6.91, "score_count": 9, "min_score": 3.25, "max_score": 9.05, "rank": 8 }, "gemini_31_pro": { "display_name": "Gemini 3.1 Pro", "provider": "openrouter", "average_score": 2.13, "score_count": 6, "min_score": 0.45, "max_score": 5.05, "rank": 9 } }, "judgments": [ { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "gemini_31_pro", "respondent_name": "Gemini 3.1 Pro", "weighted_score": 1.45, "brief_justification": "The response is incomplete and appears to have been cut off or corrupted. It contains fragments of a justification but fails to provide the actual TypeScript type definitions or the requested code implementation." }, { "judge_key": "gpt_5_4", "judge_name": "GPT-5.4", "respondent_key": "claude_opus_46", "respondent_name": "Claude Opus 4.6", "weighted_score": 8.2, "brief_justification": "Well-structured and mostly correct. The builder pattern is strong and enforces ordering cleanly. Route param extraction works for the given examples, but the first createRoute does not truly enforce exact keys despite claiming so; exactness is only properly handled in the later strict variant. Also the builder examples omit the explicit compile-error example for calling build before age is set, though the types imply it." }, { "judge_key": "gpt_5_4", "judge_name": "GPT-5.4", "respondent_key": "gemini_31_pro", "respondent_name": "Gemini 3.1 Pro", "weighted_score": 0.45, "brief_justification": "The response is truncated and incomplete, providing only fragmentary builder examples with no actual type definitions, no route solution, and no usable implementation." }, { "judge_key": "gpt_5_4", "judge_name": "GPT-5.4", "respondent_key": "claude_sonnet_46", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 8.4, "brief_justification": "Accurately demonstrates both requested compile-time patterns with solid examples and clear structure. Minor issues: the route param extractor may capture suffixes in more complex path syntaxes, and using never-typed properties for builder methods can yield less ergonomic errors than omitting methods entirely. Overall strong and practical." }, { "judge_key": "gpt_5_4", "judge_name": "GPT-5.4", "respondent_key": "grok_420", "respondent_name": "Grok 4.20", "weighted_score": 5.75, "brief_justification": "The route typing is mostly correct and demonstrates the intended constraint well. However, the builder solution does not actually enforce call order at compile time: methods are available on all states, and the checks shown are largely runtime or rely on return types rather than restricting method availability. The alternative builder is also flawed and not reliably type-safe. Clear presentation and examples, but the core builder requirement is not correctly satisfied." }, { "judge_key": "gpt_5_4", "judge_name": "GPT-5.4", "respondent_key": "deepseek_v4", "respondent_name": "DeepSeek V4", "weighted_score": 6.95, "brief_justification": "Covers both requested patterns with readable examples, but the Route validation is not fully correct: the function signature fixes params to Record, so it does not properly relate the passed object type to the path, and the stated exact-match guarantee is weaker than claimed. Builder ordering is mostly well modeled and practical." }, { "judge_key": "gpt_5_4", "judge_name": "GPT-5.4", "respondent_key": "gpt_oss_120b", "respondent_name": "GPT-OSS-120B", "weighted_score": 3.25, "brief_justification": "The route-param extraction is mostly on the right track, but the builder solution is cut off and incomplete, so it fails to satisfy a major part of the request. Clear formatting, but incomplete and not practically usable as given." }, { "judge_key": "gpt_5_4", "judge_name": "GPT-5.4", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 8.35, "brief_justification": "Accurate and well-structured solution covering both requested constraints with good examples. Route param extraction and staged builder typing are practical and mostly correct, though the route typing does not strictly forbid extra keys in all TS situations and the builder implementation exposes extra methods on the concrete class if not referenced through the staged interface." }, { "judge_key": "gpt_5_4", "judge_name": "GPT-5.4", "respondent_key": "mimo_v2_flash", "respondent_name": "MiMo-V2-Flash", "weighted_score": 7.75, "brief_justification": "Clear and mostly correct with good examples. The builder ordering via `this`-typed methods works well. The route typing enforces required keys, but `P extends Record<...>` may still allow extra keys in some cases, so it does not fully guarantee exact param matching." }, { "judge_key": "claude_opus_46", "judge_name": "Claude Opus 4.6", "respondent_key": "gpt_5_4", "respondent_name": "GPT-5.4", "weighted_score": 8.6, "brief_justification": "The solution is correct and well-structured. The Route type correctly extracts path parameters using template literal types and enforces exact matching with the Exact helper type. The Builder pattern correctly uses stage types to enforce method ordering. Minor depth deduction: the Exact type approach for preventing excess properties works but has known edge cases in TypeScript (excess property checking can sometimes be bypassed). The builder solution is simple and effective but uses concrete sta" }, { "judge_key": "claude_opus_46", "judge_name": "Claude Opus 4.6", "respondent_key": "claude_sonnet_46", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 9.2, "brief_justification": "The response provides correct, well-structured TypeScript type definitions for both challenges. The Route type correctly uses template literal types to extract path parameters and enforce exact matching. The Builder pattern correctly uses phantom types/conditional types to enforce method ordering. The code is thoroughly commented, includes both valid and invalid usage examples with clear error explanations, and the implementations are practical. Minor deductions: the ValidateParams approach usin" }, { "judge_key": "claude_opus_46", "judge_name": "Claude Opus 4.6", "respondent_key": "grok_420", "respondent_name": "Grok 4.20", "weighted_score": 6, "brief_justification": "The Route type solution is mostly correct and well-explained. However, the Builder pattern has significant issues: the first Builder class doesn't actually enforce compile-time constraints because setAge and build are available on all states - there's no conditional type or method hiding based on the generic parameter T. The methods are always callable regardless of state. The 'alternative' StrictBuilder using conditional return types (returning `never`) is closer to correct but still allows cal" }, { "judge_key": "claude_opus_46", "judge_name": "Claude Opus 4.6", "respondent_key": "deepseek_v4", "respondent_name": "DeepSeek V4", "weighted_score": 6.55, "brief_justification": "The Route type has a significant correctness issue: the `ValidateParams` type doesn't properly constrain the `params` parameter in `createRoute`. The function signature accepts `Record` which means TypeScript won't actually enforce the constraint at the call site - the params type is widened. A correct approach would use the extracted params directly as the type of the second parameter. The conditional error message trick is clever but may not produce the intended compile error in p" }, { "judge_key": "claude_opus_46", "judge_name": "Claude Opus 4.6", "respondent_key": "gpt_oss_120b", "respondent_name": "GPT-OSS-120B", "weighted_score": 6.1, "brief_justification": "The Route type solution is correct and well-explained with proper ExtractParams template literal type and Exact helper. However, the Builder pattern solution is clearly cut off mid-response - the State type is defined but the actual Builder class/type implementation, the method chaining logic, and the usage examples are all missing. This means only half the question is answered. The Route portion demonstrates good depth and clarity, but the incomplete Builder pattern significantly hurts complete" }, { "judge_key": "claude_opus_46", "judge_name": "Claude Opus 4.6", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 8.8, "brief_justification": "Both implementations are correct and well-structured. The Route type correctly uses template literal types with recursive inference to extract path parameters, and the Builder pattern correctly enforces method ordering through interface chaining. Minor note: the Route implementation doesn't enforce exact keys (excess property checking in TS helps but isn't airtight in all contexts), but for the demonstrated usage it works correctly. The explanations are clear and insightful, connecting the patte" }, { "judge_key": "claude_opus_46", "judge_name": "Claude Opus 4.6", "respondent_key": "mimo_v2_flash", "respondent_name": "MiMo-V2-Flash", "weighted_score": 8, "brief_justification": "The Route type implementation is correct and well-done, properly extracting params from URL templates using template literal types. However, it doesn't enforce that the params object has *exactly* the right keys (extra keys beyond the required ones would still compile due to how Record works with the generic P constraint - though in practice TypeScript's excess property checking on object literals helps). The Builder pattern uses `this` parameter typing which is a valid approach, but the state t" }, { "judge_key": "gemini_31_pro", "judge_name": "Gemini 3.1 Pro", "respondent_key": "gpt_5_4", "respondent_name": "GPT-5.4", "weighted_score": 9.8, "brief_justification": "The response provides perfectly accurate and elegant TypeScript solutions for both constraints. The use of recursive template literal types and exact property checking demonstrates strong TS knowledge, and the stage-based builder pattern effectively enforces method order." }, { "judge_key": "gemini_31_pro", "judge_name": "Gemini 3.1 Pro", "respondent_key": "claude_opus_46", "respondent_name": "Claude Opus 4.6", "weighted_score": 9.6, "brief_justification": "" }, { "judge_key": "gemini_31_pro", "judge_name": "Gemini 3.1 Pro", "respondent_key": "claude_sonnet_46", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 9.6, "brief_justification": "" }, { "judge_key": "gemini_31_pro", "judge_name": "Gemini 3.1 Pro", "respondent_key": "grok_420", "respondent_name": "Grok 4.20", "weighted_score": 7.75, "brief_justification": "" }, { "judge_key": "gemini_31_pro", "judge_name": "Gemini 3.1 Pro", "respondent_key": "deepseek_v4", "respondent_name": "DeepSeek V4", "weighted_score": 6.6, "brief_justification": "" }, { "judge_key": "gemini_31_pro", "judge_name": "Gemini 3.1 Pro", "respondent_key": "gpt_oss_120b", "respondent_name": "GPT-OSS-120B", "weighted_score": 4.65, "brief_justification": "The response provides an excellent and correct solution for the first requirement (Route parameters), but it gets abruptly cut off before providing the solution for the second requirement (Builder pattern)." }, { "judge_key": "gemini_31_pro", "judge_name": "Gemini 3.1 Pro", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 8.3, "brief_justification": "" }, { "judge_key": "claude_sonnet_46", "judge_name": "Claude Sonnet 4.6", "respondent_key": "gpt_5_4", "respondent_name": "GPT-5.4", "weighted_score": 8.8, "brief_justification": "The response correctly implements both challenges with accurate TypeScript types. The route parameter extraction using template literal types is well-implemented and handles nested params. The Exact type helper properly rejects extra keys. The builder pattern using staged interfaces is clean and correct. Minor issue: the Exact type approach may not perfectly reject extra keys in all TypeScript versions without additional constraints, but the approach is sound. The builder implementation is strai" }, { "judge_key": "claude_sonnet_46", "judge_name": "Claude Sonnet 4.6", "respondent_key": "claude_opus_46", "respondent_name": "Claude Opus 4.6", "weighted_score": 9, "brief_justification": "Excellent response with accurate TypeScript type definitions for both challenges. The route parameter extraction using template literal types is correct and well-explained. The phantom type builder pattern correctly enforces ordering constraints. Code includes runtime implementations, compile-time tests with @ts-expect-error annotations, and clear explanations of the type mechanics. Minor issue: the strict exact-key enforcement for extra params in createRoute (without createStrictRoute) isn't fu" }, { "judge_key": "claude_sonnet_46", "judge_name": "Claude Sonnet 4.6", "respondent_key": "gemini_31_pro", "respondent_name": "Gemini 3.1 Pro", "weighted_score": 1, "brief_justification": "The response is severely incomplete and appears to be a fragment of internal reasoning/scratchpad rather than an actual answer. It contains no TypeScript type definitions, no working code, and no example usage. It fails to address either of the two required constraints." }, { "judge_key": "claude_sonnet_46", "judge_name": "Claude Sonnet 4.6", "respondent_key": "grok_420", "respondent_name": "Grok 4.20", "weighted_score": 6.95, "brief_justification": "The route type extraction is mostly correct but has a subtle bug - the recursive call adds a leading slash which may cause issues with param extraction. The first Builder approach using state strings is good and functional, correctly preventing out-of-order calls via return types. However, the claim about extra properties causing compile errors is incorrect - TypeScript's excess property checking won't catch this in all cases with the generic approach. The alternative StrictBuilder using conditi" }, { "judge_key": "claude_sonnet_46", "judge_name": "Claude Sonnet 4.6", "respondent_key": "deepseek_v4", "respondent_name": "DeepSeek V4", "weighted_score": 7.75, "brief_justification": "The builder pattern solution is well-implemented and correct. The route type solution has a subtle issue: `ValidateParams` checks if `Record, string> extends Params` but doesn't prevent extra keys in Params, so `{ userId: '123', wrongParam: '456' }` might pass when it shouldn't. The approach of using `never` for disallowed methods in the builder is clever and practical. The `never` properties in the builder runtime implementation are assigned functions cast to `never`, wh" }, { "judge_key": "claude_sonnet_46", "judge_name": "Claude Sonnet 4.6", "respondent_key": "gpt_oss_120b", "respondent_name": "GPT-OSS-120B", "weighted_score": 7.25, "brief_justification": "The createRoute solution is well-implemented with correct type extraction and exact matching. However, the Builder pattern solution is incomplete - it gets cut off mid-implementation, leaving the second requirement entirely unaddressed. The first solution is solid with good explanation of how it works, but the truncated response significantly impacts completeness and usefulness scores." }, { "judge_key": "claude_sonnet_46", "judge_name": "Claude Sonnet 4.6", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 8.8, "brief_justification": "The response correctly implements both constraints with accurate TypeScript types. The ExtractRouteParams recursive template literal type is well-constructed and handles nested path segments. The builder pattern using interface chaining is a clean, idiomatic solution. The explanation of techniques is clear and insightful. Minor gap: doesn't address edge cases like params with special characters or query strings, and the builder explanation could mention why `implements` all three interfaces work" }, { "judge_key": "claude_sonnet_46", "judge_name": "Claude Sonnet 4.6", "respondent_key": "mimo_v2_flash", "respondent_name": "MiMo-V2-Flash", "weighted_score": 8.8, "brief_justification": "The solution correctly implements both type-level constraints using template literal types for route params and phantom types/polymorphic `this` for the builder pattern. The ExtractRouteParams type handles nested params correctly. The builder uses a clever approach with typed `this` parameters to enforce ordering. Minor issues: the error message in the builder comment is slightly misleading (methods aren't missing, just `this` type mismatch), and the `S` generic parameter in Builder isn't actual" }, { "judge_key": "grok_420", "judge_name": "Grok 4.20", "respondent_key": "gpt_5_4", "respondent_name": "GPT-5.4", "weighted_score": 8.6, "brief_justification": "The solution accurately implements both requested compile-time constraints with clean, working TypeScript. The route param extraction and exactness check are solid (minor edge cases like repeated params or trailing slashes not covered). The builder uses a simple but effective state-machine approach via distinct interfaces. Code is clear, well-commented, and includes usage examples demonstrating both valid and invalid cases." }, { "judge_key": "grok_420", "judge_name": "Grok 4.20", "respondent_key": "claude_sonnet_46", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 8.8, "brief_justification": "The solution is factually accurate, logically sound, and demonstrates advanced TypeScript techniques. It fully addresses both requirements with working compile-time constraints, thorough utility types, clear explanations, and comprehensive examples. Minor deduction for slight over-complexity in ValidateParams and a small runtime cast in the builder, but overall excellent." }, { "judge_key": "grok_420", "judge_name": "Grok 4.20", "respondent_key": "deepseek_v4", "respondent_name": "DeepSeek V4", "weighted_score": 7.4, "brief_justification": "The solution mostly works but has flaws: the Route validation is inverted (checks if required extends provided instead of vice-versa, leading to poor errors and false positives), and the Builder uses runtime errors with 'never' which doesn't fully prevent invalid calls at compile time as cleanly as a proper branded state-machine approach would. Still demonstrates good understanding and provides working examples." }, { "judge_key": "grok_420", "judge_name": "Grok 4.20", "respondent_key": "gpt_oss_120b", "respondent_name": "GPT-OSS-120B", "weighted_score": 8.6, "brief_justification": "The response provides accurate, well-explained TypeScript solutions for both constraints with good examples. The route extraction and Exact helper are solid. The builder section is cut off, reducing completeness slightly, but the approach shown is correct and insightful." }, { "judge_key": "grok_420", "judge_name": "Grok 4.20", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 8.6, "brief_justification": "The solution is factually accurate, uses correct advanced TS features, provides working examples with @ts-expect-error, and clearly explains the techniques. Minor incompleteness: ExtractRouteParams doesn't perfectly handle all edge cases like trailing slashes or multiple colons, but it works for the given examples." }, { "judge_key": "grok_420", "judge_name": "Grok 4.20", "respondent_key": "mimo_v2_flash", "respondent_name": "MiMo-V2-Flash", "weighted_score": 8.55, "brief_justification": "The solution accurately implements both requirements using advanced TypeScript techniques. The route param extraction works correctly for the given cases. The builder uses phantom types/state tokens effectively to enforce call order and required steps. Code is well-commented, includes comprehensive examples, and handles edge cases like templates without params." }, { "judge_key": "deepseek_v4", "judge_name": "DeepSeek V4", "respondent_key": "gpt_5_4", "respondent_name": "GPT-5.4", "weighted_score": 9.05, "brief_justification": "Accurate and complete implementation with clear examples. Depth could be slightly improved with the promised generic builder." }, { "judge_key": "deepseek_v4", "judge_name": "DeepSeek V4", "respondent_key": "gemini_31_pro", "respondent_name": "Gemini 3.1 Pro", "weighted_score": 5.05, "brief_justification": "Incomplete response with unclear structure and missing key details." }, { "judge_key": "deepseek_v4", "judge_name": "DeepSeek V4", "respondent_key": "claude_sonnet_46", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 9.45, "brief_justification": "Highly accurate and complete with clear examples and deep analysis." }, { "judge_key": "deepseek_v4", "judge_name": "DeepSeek V4", "respondent_key": "grok_420", "respondent_name": "Grok 4.20", "weighted_score": 9.45, "brief_justification": "Highly accurate and complete with excellent depth and practical value." }, { "judge_key": "deepseek_v4", "judge_name": "DeepSeek V4", "respondent_key": "gpt_oss_120b", "respondent_name": "GPT-OSS-120B", "weighted_score": 9.05, "brief_justification": "Accurate, thorough, and clear with good practical value." }, { "judge_key": "deepseek_v4", "judge_name": "DeepSeek V4", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 9.45, "brief_justification": "Accurate, complete, and clear with advanced TypeScript techniques demonstrated." }, { "judge_key": "deepseek_v4", "judge_name": "DeepSeek V4", "respondent_key": "mimo_v2_flash", "respondent_name": "MiMo-V2-Flash", "weighted_score": 9.6, "brief_justification": "Accurate, thorough, and practical with clear examples and explanations." }, { "judge_key": "gpt_oss_120b", "judge_name": "GPT-OSS-120B", "respondent_key": "gpt_5_4", "respondent_name": "GPT-5.4", "weighted_score": 8.8, "brief_justification": "" }, { "judge_key": "gpt_oss_120b", "judge_name": "GPT-OSS-120B", "respondent_key": "claude_opus_46", "respondent_name": "Claude Opus 4.6", "weighted_score": 8.75, "brief_justification": "" }, { "judge_key": "gpt_oss_120b", "judge_name": "GPT-OSS-120B", "respondent_key": "gemini_31_pro", "respondent_name": "Gemini 3.1 Pro", "weighted_score": 2, "brief_justification": "The answer is incomplete, missing full type definitions and examples, providing only a fragment, so it fails to demonstrate correct or useful solutions." }, { "judge_key": "gpt_oss_120b", "judge_name": "GPT-OSS-120B", "respondent_key": "claude_sonnet_46", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 8.8, "brief_justification": "The answer presents working TypeScript types that enforce the required compile‑time constraints, includes thorough examples and error cases, is well‑organized and clearly explained, offers insight into the type‑level mechanisms, and provides practical, reusable patterns." }, { "judge_key": "gpt_oss_120b", "judge_name": "GPT-OSS-120B", "respondent_key": "grok_420", "respondent_name": "Grok 4.20", "weighted_score": 7.35, "brief_justification": "Route types work well; builder pattern lacks proper compile-time ordering enforcement in the primary example, though an alternative is hinted. Coverage and explanation are thorough and clear, with decent depth and practical value." }, { "judge_key": "gpt_oss_120b", "judge_name": "GPT-OSS-120B", "respondent_key": "deepseek_v4", "respondent_name": "DeepSeek V4", "weighted_score": 7.35, "brief_justification": "Provides clear type definitions and examples for both problems, but the Route validation uses a fixed Record instead of inferring the actual params type, so it doesn't fully enforce the constraints. Builder pattern is correctly modeled. Overall answer is well‑structured and useful despite the flaw." }, { "judge_key": "gpt_oss_120b", "judge_name": "GPT-OSS-120B", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 8.8, "brief_justification": "Accurate TypeScript implementations with clear examples; covers both constraints fully and explains underlying techniques, offering practical, reusable code." }, { "judge_key": "gpt_oss_120b", "judge_name": "GPT-OSS-120B", "respondent_key": "mimo_v2_flash", "respondent_name": "MiMo-V2-Flash", "weighted_score": 7.35, "brief_justification": "The answer includes functional type definitions and examples, clearly presented, but the Route type allows extra params, so it doesn't fully enforce the constraint." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "gpt_5_4", "respondent_name": "GPT-5.4", "weighted_score": 9.8, "brief_justification": "The response provides perfect solutions for both constraints. The Route type uses advanced template literal types and an 'Exact' helper to prevent extra keys, and the Builder pattern uses a state-machine approach with interfaces to enforce method ordering." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "claude_opus_46", "respondent_name": "Claude Opus 4.6", "weighted_score": 10, "brief_justification": "The response provides perfect solutions for both complex TypeScript challenges. The Route parameter extraction uses recursive template literal types correctly, and the Builder pattern implementation using state-machine phantom types is the industry-standard way to enforce method ordering at compile time. The code is well-commented and includes test cases." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "claude_sonnet_46", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 10, "brief_justification": "The response provides perfect solutions for both complex TypeScript challenges. The Route type uses advanced template literal types and recursive inference to extract parameters, and the Builder pattern uses phantom types and conditional mapping to enforce a strict state machine. The code is well-commented and includes clear usage examples." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "grok_420", "respondent_name": "Grok 4.20", "weighted_score": 9.55, "brief_justification": "The response provides excellent solutions for both constraints. The Route type correctly uses template literal types and recursive inference. The Builder pattern uses phantom types/state tracking to enforce method ordering and completion. While the Builder implementation uses 'as any' for the internal transition (common in TS for this pattern), the external API enforcement is exactly what was requested." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "deepseek_v4", "respondent_name": "DeepSeek V4", "weighted_score": 9.8, "brief_justification": "The response provides elegant and technically accurate solutions for both constraints. The use of template literal types for path extraction and a state-machine approach for the builder pattern perfectly addresses the prompt's requirements with clear examples." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "gpt_oss_120b", "respondent_name": "GPT-OSS-120B", "weighted_score": 8.8, "brief_justification": "The solution for the Route type is excellent and uses advanced template literal types correctly. The Builder pattern solution is cut off at the end, which reduces completeness, but the provided logic for the first half is sound and demonstrates high technical depth." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "mimo_v2_flash", "respondent_name": "MiMo-V2-Flash", "weighted_score": 9.8, "brief_justification": "The response provides perfect solutions for both constraints. The Route type uses template literal types effectively, and the Builder pattern uses 'this' parameter constraints to enforce method ordering at compile time, which is the idiomatic way to solve this in TypeScript." }, { "judge_key": "minimax_m25", "judge_name": "MiniMax M2.5", "respondent_key": "gpt_5_4", "respondent_name": "GPT-5.4", "weighted_score": 8.4, "brief_justification": "Well-structured TypeScript types that correctly enforce compile-time constraints for both requirements. The Route type properly extracts params and uses intersection types to reject extras. The Builder pattern correctly enforces method ordering. Minor issues: the ExtractRouteParams logic has a subtle edge case with trailing params, and the Builder implementation returns new objects rather than chaining on same builder instance. Overall thorough and practical." }, { "judge_key": "minimax_m25", "judge_name": "MiniMax M2.5", "respondent_key": "claude_opus_46", "respondent_name": "Claude Opus 4.6", "weighted_score": 8.75, "brief_justification": "Comprehensive solution with working TypeScript code for both requirements. Route params extraction uses recursive conditional types correctly; builder uses phantom types for state machine. Minor issue: basic createRoute may not strictly prevent extra keys (the advanced version handles this). Well-documented with valid/invalid usage examples." }, { "judge_key": "minimax_m25", "judge_name": "MiniMax M2.5", "respondent_key": "claude_sonnet_46", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 9, "brief_justification": "Excellent implementation using advanced TypeScript features: template literal types with recursion for path extraction, conditional types for validation, and phantom types for builder state tracking. Well-organized with clear examples showing both valid and invalid usage." }, { "judge_key": "minimax_m25", "judge_name": "MiniMax M2.5", "respondent_key": "grok_420", "respondent_name": "Grok 4.20", "weighted_score": 7.4, "brief_justification": "The Route type solution is correct and works well. However, the Builder pattern implementations have significant flaws: the class-based approach uses runtime errors and 'as any' casts which defeats compile-time safety, and the branded types alternative has a fundamental issue where the conditional type in setAge won't properly enforce ordering (it's evaluated based on the initial type parameter, not the actual chain of calls). The concepts are demonstrated clearly but the implementations would n" }, { "judge_key": "minimax_m25", "judge_name": "MiniMax M2.5", "respondent_key": "deepseek_v4", "respondent_name": "DeepSeek V4", "weighted_score": 8.4, "brief_justification": "Well-structured TypeScript solutions using advanced type features. Route validation correctly extracts path params and validates against provided params. Builder pattern uses state machine approach effectively. Minor deduction for slight complexity in validation logic that could be more explicit about exact vs partial matching." }, { "judge_key": "minimax_m25", "judge_name": "MiniMax M2.5", "respondent_key": "gpt_oss_120b", "respondent_name": "GPT-OSS-120B", "weighted_score": 5.9, "brief_justification": "" }, { "judge_key": "minimax_m25", "judge_name": "MiniMax M2.5", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 8.4, "brief_justification": "Well-structured response with working implementations for both constraints. The template literal types and interface chaining techniques are correctly applied. Minor gap: RouteParams type could potentially accept extra keys due to TypeScript's structural typing, but overall the solution is solid and clearly explained." }, { "judge_key": "minimax_m25", "judge_name": "MiniMax M2.5", "respondent_key": "mimo_v2_flash", "respondent_name": "MiMo-V2-Flash", "weighted_score": 8.1, "brief_justification": "Well-structured response with correct TypeScript type implementations. The Route type correctly extracts and validates params. The Builder pattern uses clever 'this' parameter typing but has a minor flaw - setName can technically be called at any point (not just first). The code is clearly organized with good comments and shows both valid and invalid usage examples." }, { "judge_key": "mimo_v2_flash", "judge_name": "MiMo-V2-Flash", "respondent_key": "gpt_5_4", "respondent_name": "GPT-5.4", "weighted_score": 8.6, "brief_justification": "Accurate TypeScript types for both constraints, clear examples, and good depth with advanced techniques like Exact helper and type-state builder." }, { "judge_key": "mimo_v2_flash", "judge_name": "MiMo-V2-Flash", "respondent_key": "claude_opus_46", "respondent_name": "Claude Opus 4.6", "weighted_score": 9, "brief_justification": "The response provides highly accurate and advanced TypeScript solutions for both constraints. The Route type uses conditional types and template literal types correctly. The Builder pattern uses a sophisticated phantom state/conditional interface approach to enforce strict ordering, which is a robust solution. The code is well-commented, includes test cases, and explains the concepts clearly." }, { "judge_key": "mimo_v2_flash", "judge_name": "MiMo-V2-Flash", "respondent_key": "gemini_31_pro", "respondent_name": "Gemini 3.1 Pro", "weighted_score": 2.85, "brief_justification": "Response is incomplete, lacks actual type definitions and examples, and contains confusing, non-functional code snippets." }, { "judge_key": "mimo_v2_flash", "judge_name": "MiMo-V2-Flash", "respondent_key": "claude_sonnet_46", "respondent_name": "Claude Sonnet 4.6", "weighted_score": 9, "brief_justification": "The solution is highly accurate and robust, using advanced TypeScript features (conditional types, template literal types, recursive types, and conditional method availability) to solve both problems correctly at compile time. The code is well-structured, commented, and includes comprehensive usage examples. Minor deduction in correctness for the `ValidateParams` type which uses a string literal error message as a type, which technically allows the string to be passed as a valid argument (though" }, { "judge_key": "mimo_v2_flash", "judge_name": "MiMo-V2-Flash", "respondent_key": "grok_420", "respondent_name": "Grok 4.20", "weighted_score": 8.55, "brief_justification": "Accurate and thorough solutions with clear examples; minor issues in builder runtime checks and alternative implementation." }, { "judge_key": "mimo_v2_flash", "judge_name": "MiMo-V2-Flash", "respondent_key": "deepseek_v4", "respondent_name": "DeepSeek V4", "weighted_score": 8.6, "brief_justification": "Accurate and complete solutions with clear explanations; minor issues in Route validation logic and builder implementation details." }, { "judge_key": "mimo_v2_flash", "judge_name": "MiMo-V2-Flash", "respondent_key": "gpt_oss_120b", "respondent_name": "GPT-OSS-120B", "weighted_score": 8.6, "brief_justification": "Accurate and thorough type definitions with clear examples; depth shown in explaining type mechanics; highly useful for compile-time safety." }, { "judge_key": "mimo_v2_flash", "judge_name": "MiMo-V2-Flash", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 9, "brief_justification": "Accurate and complete solutions with clear explanations and practical examples." } ], "meta": { "source": "The Multivac (app.themultivac.com)", "methodology": "10x10 blind peer matrix evaluation", "criteria": "correctness, completeness, clarity, depth, usefulness", "self_judgments": "excluded from rankings", "license": "Open data — cite as: The Multivac (2026)" } }