{ "evaluation_id": "EVAL-20260207-142929", "question_id": "CODE-010", "question_text": "Create TypeScript types that enforce these compile-time constraints:\n\n1. A `Route` type where the path parameters in the URL template must match the params object:\n - `createRoute('/users/:userId/posts/:postId', { userId: '123', postId: '456' })` ✓\n - `createRoute('/users/:userId', { wrongParam: '123' })` ✗ (compile error)\n\n2. A `Builder` pattern type where methods can only be called in a specific order:\n - `builder.setName('x').setAge(25).build()` ✓\n - `builder.setAge(25).setName('x').build()` ✗ (compile error)\n - `builder.setName('x').build()` ✗ (compile error - age not set)\n\nShow the type definitions and example usage.", "category": "code", "timestamp": "2026-03-17T00:00:00.000Z", "display_date": "Mar 17, 2026", "winner": { "name": "Claude Sonnet 4.5", "provider": "Anthropic", "score": 9.49 }, "avg_score": 7.873, "matrix_size": 90, "models_used": [ { "id": "claude_opus", "name": "Claude Opus 4.5", "provider": "Anthropic" }, { "id": "gemini_3_flash", "name": "Gemini 3 Flash Preview", "provider": "Google" }, { "id": "claude_sonnet", "name": "Claude Sonnet 4.5", "provider": "Anthropic" }, { "id": "gemini_3_pro", "name": "Gemini 3 Pro Preview", "provider": "Google" }, { "id": "minimax_m2", "name": "MiniMax M2", "provider": "MiniMax" }, { "id": "glm_4_7", "name": "GLM-4-7", "provider": "Zhipu" }, { "id": "deepseek_v3", "name": "DeepSeek V3.2", "provider": "DeepSeek" }, { "id": "gpt_codex", "name": "GPT-5.2-Codex", "provider": "OpenAI" }, { "id": "grok_direct", "name": "Grok 3 (Direct)", "provider": "xAI" }, { "id": "grok_code_fast", "name": "Grok Code Fast", "provider": "xAI" } ], "rankings": { "claude_sonnet": { "display_name": "Claude Sonnet 4.5", "provider": "Anthropic", "average_score": 9.49, "score_count": 8, "min_score": 8.8, "max_score": 10, "rank": 1 }, "grok_direct": { "display_name": "Grok 3 (Direct)", "provider": "xAI", "average_score": 9.42, "score_count": 8, "min_score": 8.8, "max_score": 9.8, "rank": 2 }, "gpt_codex": { "display_name": "GPT-5.2-Codex", "provider": "OpenAI", "average_score": 9.36, "score_count": 8, "min_score": 8.6, "max_score": 9.8, "rank": 3 }, "gemini_3_flash": { "display_name": "Gemini 3 Flash Preview", "provider": "Google", "average_score": 9.28, "score_count": 8, "min_score": 8.6, "max_score": 9.8, "rank": 4 }, "grok_code_fast": { "display_name": "Grok Code Fast", "provider": "xAI", "average_score": 9.16, "score_count": 8, "min_score": 8.25, "max_score": 10, "rank": 5 }, "claude_opus": { "display_name": "Claude Opus 4.5", "provider": "Anthropic", "average_score": 9.07, "score_count": 8, "min_score": 7.8, "max_score": 9.8, "rank": 6 }, "deepseek_v3": { "display_name": "DeepSeek V3.2", "provider": "DeepSeek", "average_score": 8.84, "score_count": 7, "min_score": 6.8, "max_score": 10, "rank": 7 }, "gemini_3_pro": { "display_name": "Gemini 3 Pro Preview", "provider": "Google", "average_score": 5.62, "score_count": 9, "min_score": 4.1, "max_score": 7.2, "rank": 8 }, "glm_4_7": { "display_name": "GLM-4-7", "provider": "Zhipu", "average_score": 4.25, "score_count": 4, "min_score": 0.5, "max_score": 8.55, "rank": 9 }, "minimax_m2": { "display_name": "MiniMax M2", "provider": "MiniMax", "average_score": 4.24, "score_count": 4, "min_score": 0.5, "max_score": 8.15, "rank": 10 } }, "judgments": [ { "judge_key": "grok_code_fast", "judge_name": "Grok Code Fast", "respondent_key": "claude_opus", "respondent_name": "Claude Opus 4.5", "weighted_score": 9.8, "brief_justification": "The response accurately implements TypeScript types for both Route and Builder constraints with correct compile-time enforcement, covers all requirements thoroughly with examples, is well-structured and explained, provides insightful advanced features, and offers highly practical, actionable code." }, { "judge_key": "grok_code_fast", "judge_name": "Grok Code Fast", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 9.8, "brief_justification": "The response accurately implements TypeScript types for both constraints with correct compile-time enforcement, fully covers the requirements including examples and explanations, is clearly structured, provides insightful details on underlying concepts like finite state machines, and offers highly actionable code for developers." }, { "judge_key": "grok_code_fast", "judge_name": "Grok Code Fast", "respondent_key": "claude_sonnet", "respondent_name": "Claude Sonnet 4.5", "weighted_score": 9.6, "brief_justification": "The response accurately implements TypeScript types for route parameter validation and ordered builder patterns with correct logic, full coverage of requirements including alternatives, clear structure and examples, insightful use of advanced TypeScript features, and highly practical, actionable code that can be directly applied." }, { "judge_key": "grok_code_fast", "judge_name": "Grok Code Fast", "respondent_key": "gemini_3_pro", "respondent_name": "Gemini 3 Pro Preview", "weighted_score": 5.5, "brief_justification": "The response correctly implements the Route type using template literals, though the ExtractParams recursion has potential flaws for complex paths; however, it is incomplete as it cuts off mid-example and entirely omits the Builder pattern requirement." }, { "judge_key": "grok_code_fast", "judge_name": "Grok Code Fast", "respondent_key": "minimax_m2", "respondent_name": "MiniMax M2", "weighted_score": 0, "brief_justification": "The response is empty and does not provide any TypeScript type definitions or address the compile-time constraints specified in the question." }, { "judge_key": "grok_code_fast", "judge_name": "Grok Code Fast", "respondent_key": "glm_4_7", "respondent_name": "GLM-4-7", "weighted_score": 0, "brief_justification": "The response is empty, providing no type definitions or examples, thus failing all criteria." }, { "judge_key": "grok_code_fast", "judge_name": "Grok Code Fast", "respondent_key": "deepseek_v3", "respondent_name": "DeepSeek V3.2", "weighted_score": 9.55, "brief_justification": "The response accurately implements TypeScript types for both constraints using advanced features like template literals and state machines, with correct enforcement of compile-time errors; it thoroughly covers the requirements, provides clear examples, and offers practical, adaptable code." }, { "judge_key": "grok_code_fast", "judge_name": "Grok Code Fast", "respondent_key": "gpt_codex", "respondent_name": "GPT-5.2-Codex", "weighted_score": 9.8, "brief_justification": "The response accurately implements TypeScript types that enforce the specified compile-time constraints using advanced features like template literals and conditional types, with clear examples demonstrating validity and errors. It provides thorough, practical code that directly addresses the query without unnecessary complexity." }, { "judge_key": "glm_4_7", "judge_name": "GLM-4-7", "respondent_key": "minimax_m2", "respondent_name": "MiniMax M2", "weighted_score": 0, "brief_justification": "No response was provided to evaluate. The message ended after 'Response to evaluate:' without any content." }, { "judge_key": "grok_code_fast", "judge_name": "Grok Code Fast", "respondent_key": "grok_direct", "respondent_name": "Grok 3 (Direct)", "weighted_score": 9.6, "brief_justification": "The response accurately implements TypeScript types for both constraints with correct logic, covers all requirements including examples and explanations, is well-structured and easy to understand, provides insightful details on the type system mechanics, and offers highly practical, actionable code for developers." }, { "judge_key": "claude_opus", "judge_name": "Claude Opus 4.5", "respondent_key": "grok_code_fast", "respondent_name": "Grok Code Fast", "weighted_score": 8.25, "brief_justification": "The solution correctly implements both requirements with working TypeScript code. The Route type uses template literal types effectively to extract path parameters, and the Builder pattern uses state classes to enforce method ordering. Minor deductions: the Route solution may not handle exact object types perfectly (excess property checks can be bypassed), and the Builder could have shown a more type-level approach using generics instead of multiple classes. Overall, well-explained and practical." }, { "judge_key": "claude_opus", "judge_name": "Claude Opus 4.5", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 9, "brief_justification": "The response provides correct, working TypeScript implementations for both constraints. The route parameter extraction using template literal types is accurate, and the builder pattern using interface chaining correctly enforces method ordering. The code is exceptionally well-organized with clear comments, usage examples showing both success and failure cases, and a helpful summary of key concepts. Minor deduction for not handling edge cases like optional parameters or showing the exact compiler error messages." }, { "judge_key": "claude_opus", "judge_name": "Claude Opus 4.5", "respondent_key": "claude_sonnet", "respondent_name": "Claude Sonnet 4.5", "weighted_score": 9, "brief_justification": "Excellent response providing working TypeScript implementations for both requirements. The route parameter extraction uses template literal types correctly, and the builder pattern uses generic state tracking with contextual 'this' types effectively. Includes comprehensive examples of both valid and invalid cases, plus a bonus flexible builder alternative. Minor deduction because the RouteParams type with Record for empty params might not work perfectly in all edge cases, but overall this is a thorough, well-explained solution." }, { "judge_key": "claude_opus", "judge_name": "Claude Opus 4.5", "respondent_key": "gemini_3_pro", "respondent_name": "Gemini 3 Pro Preview", "weighted_score": 5.3, "brief_justification": "The response correctly implements the Route type with template literal types for parameter extraction, but the code is cut off mid-example and completely missing the Builder pattern implementation (Part 2). The Route solution shown is technically sound with good explanation, but only half the question was answered, and even that half is incomplete with truncated examples." }, { "judge_key": "claude_opus", "judge_name": "Claude Opus 4.5", "respondent_key": "minimax_m2", "respondent_name": "MiniMax M2", "weighted_score": 1.25, "brief_justification": "The response is completely empty - no TypeScript types, no code, no explanation whatsoever. The question asked for specific advanced TypeScript type definitions for route parameter extraction and ordered builder patterns, but nothing was provided." }, { "judge_key": "claude_opus", "judge_name": "Claude Opus 4.5", "respondent_key": "glm_4_7", "respondent_name": "GLM-4-7", "weighted_score": 0.9, "brief_justification": "The response is completely empty, providing no TypeScript type definitions, no implementation, and no examples. This fails to address any part of the complex type-level programming question about template literal types for route parameters and state-machine builder patterns." }, { "judge_key": "claude_opus", "judge_name": "Claude Opus 4.5", "respondent_key": "deepseek_v3", "respondent_name": "DeepSeek V3.2", "weighted_score": 6.8, "brief_justification": "The Route type implementation has issues - ExtractParams doesn't properly enforce exact matching (allows extra properties) and the constraint logic is flawed. The Builder pattern works conceptually but uses 'as any' casts which undermines type safety. The explanations are clear and well-structured, but the actual implementations have subtle bugs that would cause issues in practice. The approach is sound but execution needs refinement." }, { "judge_key": "claude_opus", "judge_name": "Claude Opus 4.5", "respondent_key": "gpt_codex", "respondent_name": "GPT-5.2-Codex", "weighted_score": 8.6, "brief_justification": "The solution is correct and demonstrates proper use of template literal types for route parameter extraction and state-machine-style interfaces for the builder pattern. The code is exceptionally clear with good formatting and examples. Minor deductions: lacks actual runtime implementations (only declarations), doesn't show the compile error messages, and could explain the type mechanics more deeply for educational value." }, { "judge_key": "claude_opus", "judge_name": "Claude Opus 4.5", "respondent_key": "grok_direct", "respondent_name": "Grok 3 (Direct)", "weighted_score": 9, "brief_justification": "The response provides correct, working TypeScript implementations for both requirements. The Route type correctly uses template literal types for parameter extraction, and the Builder pattern properly enforces method ordering through state interfaces. The explanations are exceptionally clear with well-structured code examples. Minor deduction for not handling edge cases like consecutive parameters or optional parameters in routes, and not showing a more generic/extensible builder pattern." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "grok_code_fast", "respondent_name": "Grok Code Fast", "weighted_score": 9.8, "brief_justification": "The response provides accurate and working TypeScript implementations for both complex constraints. The Route type uses advanced template literal types correctly, and the Builder pattern uses state-transitioning classes to enforce method ordering at compile time." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "claude_opus", "respondent_name": "Claude Opus 4.5", "weighted_score": 9.6, "brief_justification": "The response provides high-quality, idiomatic TypeScript solutions for both problems. The route parameter extraction uses template literal types correctly, and the builder pattern uses 'this' parameter types to enforce state transitions, which is the industry standard for this requirement." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "claude_sonnet", "respondent_name": "Claude Sonnet 4.5", "weighted_score": 10, "brief_justification": "The response provides perfect solutions for both constraints. It uses advanced TypeScript features like template literal types for route parsing and polymorphic 'this' types for the builder pattern, including clear examples of both valid and invalid usage." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "gemini_3_pro", "respondent_name": "Gemini 3 Pro Preview", "weighted_score": 6.45, "brief_justification": "The response provides a correct implementation for the first constraint (Route Parameters) but completely fails to provide the second requested constraint (the Builder pattern with specific method ordering)." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "minimax_m2", "respondent_name": "MiniMax M2", "weighted_score": 0, "brief_justification": "The response is empty and provides no content, failing to address any part of the prompt." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "glm_4_7", "respondent_name": "GLM-4-7", "weighted_score": 0, "brief_justification": "The response is empty and provides no content, types, or code to evaluate." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "deepseek_v3", "respondent_name": "DeepSeek V3.2", "weighted_score": 9.8, "brief_justification": "The response provides highly accurate and idiomatic TypeScript solutions for both problems. The Route type uses template literal types effectively, and the Builder pattern correctly uses state-tracking via generics to enforce method ordering at compile time." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "gpt_codex", "respondent_name": "GPT-5.2-Codex", "weighted_score": 9.8, "brief_justification": "The solution correctly uses template literal types for the route parser and a finite state machine approach for the builder pattern, perfectly meeting all prompt requirements." }, { "judge_key": "gemini_3_flash", "judge_name": "Gemini 3 Flash Preview", "respondent_key": "grok_direct", "respondent_name": "Grok 3 (Direct)", "weighted_score": 9.8, "brief_justification": "The response provides perfect solutions for both requirements. The Route type uses advanced template literal inference correctly, and the Builder pattern uses a state-machine interface approach that strictly enforces the requested method order and completion." }, { "judge_key": "claude_sonnet", "judge_name": "Claude Sonnet 4.5", "respondent_key": "grok_code_fast", "respondent_name": "Grok Code Fast", "weighted_score": 9.2, "brief_justification": "Excellent implementation with correct TypeScript advanced types. Both solutions work as specified with clear explanations. Minor limitation: Route type could handle edge cases like consecutive slashes, but the core implementation is sound and well-documented." }, { "judge_key": "claude_sonnet", "judge_name": "Claude Sonnet 4.5", "respondent_key": "claude_opus", "respondent_name": "Claude Opus 4.5", "weighted_score": 9.4, "brief_justification": "Excellent implementation of advanced TypeScript type constraints with sophisticated use of template literal types, phantom types, and conditional types. The response is cut off at the end but demonstrates deep understanding of TypeScript's type system with clear explanations and practical examples." }, { "judge_key": "claude_sonnet", "judge_name": "Claude Sonnet 4.5", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 9.35, "brief_justification": "Excellent implementation with correct type-level programming techniques. The route parameter extraction using template literals is accurate, and the builder pattern using interface chaining effectively enforces ordering. Minor deduction for not handling edge cases like optional parameters or empty routes, but the core solution is sound and well-explained." }, { "judge_key": "claude_sonnet", "judge_name": "Claude Sonnet 4.5", "respondent_key": "gemini_3_pro", "respondent_name": "Gemini 3 Pro Preview", "weighted_score": 4.7, "brief_justification": "The Route type implementation is mostly correct and demonstrates good understanding of template literal types, but the response is incomplete - it cuts off mid-example and completely omits the Builder pattern solution (50% of the requirements). The provided Route solution works but lacks edge case handling." }, { "judge_key": "claude_sonnet", "judge_name": "Claude Sonnet 4.5", "respondent_key": "minimax_m2", "respondent_name": "MiniMax M2", "weighted_score": 0.5, "brief_justification": "No response was provided to evaluate. The 'Response to evaluate' field is empty, so there is no content to assess for completeness, clarity, depth, or usefulness. Only minimal correctness points given for the absence of incorrect information." }, { "judge_key": "claude_sonnet", "judge_name": "Claude Sonnet 4.5", "respondent_key": "glm_4_7", "respondent_name": "GLM-4-7", "weighted_score": 0.5, "brief_justification": "No response was provided to evaluate. The response field is empty, making it impossible to assess any of the requested TypeScript type definitions or examples." }, { "judge_key": "claude_sonnet", "judge_name": "Claude Sonnet 4.5", "respondent_key": "deepseek_v3", "respondent_name": "DeepSeek V3.2", "weighted_score": 8.4, "brief_justification": "The response provides working TypeScript implementations for both requirements with good type safety. The Route type correctly extracts parameters, though the recursive type could be simplified. The Builder pattern effectively enforces method order using state machines, with both a basic and fluent implementation provided." }, { "judge_key": "claude_sonnet", "judge_name": "Claude Sonnet 4.5", "respondent_key": "gpt_codex", "respondent_name": "GPT-5.2-Codex", "weighted_score": 8.6, "brief_justification": "The solution correctly implements both requirements using appropriate TypeScript features (template literal types and state-based interfaces). Minor deduction for the builder being declared rather than implemented, though this is acceptable for type demonstration purposes." }, { "judge_key": "claude_sonnet", "judge_name": "Claude Sonnet 4.5", "respondent_key": "grok_direct", "respondent_name": "Grok 3 (Direct)", "weighted_score": 9.2, "brief_justification": "Excellent implementation of both TypeScript patterns with correct type definitions and clear explanations. The route parameter extraction uses template literal types properly, and the builder pattern correctly enforces method ordering through state interfaces. Minor limitation: the route type doesn't strictly prevent extra parameters in all TypeScript versions, but the core solution is sound and practical." }, { "judge_key": "gemini_3_pro", "judge_name": "Gemini 3 Pro Preview", "respondent_key": "grok_code_fast", "respondent_name": "Grok Code Fast", "weighted_score": 0, "brief_justification": "" }, { "judge_key": "gemini_3_pro", "judge_name": "Gemini 3 Pro Preview", "respondent_key": "claude_opus", "respondent_name": "Claude Opus 4.5", "weighted_score": 0, "brief_justification": "" }, { "judge_key": "gemini_3_pro", "judge_name": "Gemini 3 Pro Preview", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 0, "brief_justification": "" }, { "judge_key": "gemini_3_pro", "judge_name": "Gemini 3 Pro Preview", "respondent_key": "claude_sonnet", "respondent_name": "Claude Sonnet 4.5", "weighted_score": 0, "brief_justification": "" }, { "judge_key": "gemini_3_pro", "judge_name": "Gemini 3 Pro Preview", "respondent_key": "minimax_m2", "respondent_name": "MiniMax M2", "weighted_score": 0, "brief_justification": "The response to evaluate was empty, making it impossible to assess any criteria." }, { "judge_key": "gemini_3_pro", "judge_name": "Gemini 3 Pro Preview", "respondent_key": "glm_4_7", "respondent_name": "GLM-4-7", "weighted_score": 0, "brief_justification": "The response to evaluate was empty, so no solution was provided." }, { "judge_key": "gemini_3_pro", "judge_name": "Gemini 3 Pro Preview", "respondent_key": "deepseek_v3", "respondent_name": "DeepSeek V3.2", "weighted_score": 0, "brief_justification": "" }, { "judge_key": "gemini_3_pro", "judge_name": "Gemini 3 Pro Preview", "respondent_key": "gpt_codex", "respondent_name": "GPT-5.2-Codex", "weighted_score": 0, "brief_justification": "" }, { "judge_key": "gemini_3_pro", "judge_name": "Gemini 3 Pro Preview", "respondent_key": "grok_direct", "respondent_name": "Grok 3 (Direct)", "weighted_score": 0, "brief_justification": "" }, { "judge_key": "minimax_m2", "judge_name": "MiniMax M2", "respondent_key": "grok_code_fast", "respondent_name": "Grok Code Fast", "weighted_score": 10, "brief_justification": "The response provides accurate, working TypeScript implementations for both requirements with clear explanations, well-structured code, and practical examples showing both valid and invalid usage patterns." }, { "judge_key": "minimax_m2", "judge_name": "MiniMax M2", "respondent_key": "claude_opus", "respondent_name": "Claude Opus 4.5", "weighted_score": 8.45, "brief_justification": "Excellent TypeScript implementations using template literal types and phantom types. Both constraints are correctly enforced. Minor deduction for incomplete final example and missing error messages for builder pattern." }, { "judge_key": "minimax_m2", "judge_name": "MiniMax M2", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 9.8, "brief_justification": "Excellent implementation of both compile-time constraints with working code, clear examples, and educational explanations of the key TypeScript concepts used." }, { "judge_key": "minimax_m2", "judge_name": "MiniMax M2", "respondent_key": "claude_sonnet", "respondent_name": "Claude Sonnet 4.5", "weighted_score": 9.8, "brief_justification": "Excellent implementation of both compile-time constraints with working code, clear examples, and additional alternative patterns. Thoroughly covers the requirements with well-documented TypeScript code." }, { "judge_key": "minimax_m2", "judge_name": "MiniMax M2", "respondent_key": "gemini_3_pro", "respondent_name": "Gemini 3 Pro Preview", "weighted_score": 4.65, "brief_justification": "The response correctly implements the Route type with template literal types but is severely incomplete - it only addresses the first requirement, completely omits the Builder pattern, and the code example is truncated mid-word." }, { "judge_key": "minimax_m2", "judge_name": "MiniMax M2", "respondent_key": "glm_4_7", "respondent_name": "GLM-4-7", "weighted_score": 0, "brief_justification": "" }, { "judge_key": "minimax_m2", "judge_name": "MiniMax M2", "respondent_key": "deepseek_v3", "respondent_name": "DeepSeek V3.2", "weighted_score": 10, "brief_justification": "Excellent response that fully implements both requirements with working TypeScript code. Uses advanced type features (template literal types, conditional types, discriminated unions) correctly. Provides clear examples, explains the mechanisms, and even offers alternative implementations." }, { "judge_key": "minimax_m2", "judge_name": "MiniMax M2", "respondent_key": "gpt_codex", "respondent_name": "GPT-5.2-Codex", "weighted_score": 9.45, "brief_justification": "Excellent solution demonstrating advanced TypeScript type techniques. Correctly implements both constraints using template-literal types and stateful builder interfaces. Well-organized with clear examples. Minor deduction for completeness as it doesn't show a full working builder implementation with actual methods." }, { "judge_key": "minimax_m2", "judge_name": "MiniMax M2", "respondent_key": "grok_direct", "respondent_name": "Grok 3 (Direct)", "weighted_score": 9.8, "brief_justification": "Excellent solution that correctly implements both compile-time constraints using TypeScript's advanced type system. The Route type uses template literal types to extract parameters, and the Builder pattern uses a state machine with interfaces. Well-organized with clear explanations, valid/invalid examples, and practical ready-to-use code." }, { "judge_key": "glm_4_7", "judge_name": "GLM-4-7", "respondent_key": "grok_code_fast", "respondent_name": "Grok Code Fast", "weighted_score": 9.8, "brief_justification": "The response provides correct, complete implementations for both constraints using advanced TypeScript features like template literal types and state-based classes. Code is well-documented with clear examples showing valid and invalid usage, making it highly practical for real-world use." }, { "judge_key": "glm_4_7", "judge_name": "GLM-4-7", "respondent_key": "claude_opus", "respondent_name": "Claude Opus 4.5", "weighted_score": 9.25, "brief_justification": "The type definitions are technically correct and elegantly solve both constraints using advanced TypeScript features (template literal types for routes, phantom types for builder ordering). However, the response is cut off mid-example, leaving the second usage example incomplete." }, { "judge_key": "glm_4_7", "judge_name": "GLM-4-7", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 9.8, "brief_justification": "The response provides correct, production-ready TypeScript implementations for both constraints using advanced features like template literal types and interface chaining. The code is well-structured with clear examples demonstrating both success and failure cases, and includes helpful explanations of the key concepts used." }, { "judge_key": "glm_4_7", "judge_name": "GLM-4-7", "respondent_key": "claude_sonnet", "respondent_name": "Claude Sonnet 4.5", "weighted_score": 9.8, "brief_justification": "Excellent implementation using advanced TypeScript features (template literal types, conditional types, contextual this types) that correctly enforces both compile-time constraints. Includes comprehensive examples, error cases, and even provides an alternative flexible builder pattern." }, { "judge_key": "glm_4_7", "judge_name": "GLM-4-7", "respondent_key": "gemini_3_pro", "respondent_name": "Gemini 3 Pro Preview", "weighted_score": 5.75, "brief_justification": "The Route type implementation using template literal types is correct and well-explained, but the response is incomplete - it cuts off mid-example and completely fails to address the Builder pattern requirement (constraint #2)." }, { "judge_key": "glm_4_7", "judge_name": "GLM-4-7", "respondent_key": "deepseek_v3", "respondent_name": "DeepSeek V3.2", "weighted_score": 0, "brief_justification": "" }, { "judge_key": "glm_4_7", "judge_name": "GLM-4-7", "respondent_key": "gpt_codex", "respondent_name": "GPT-5.2-Codex", "weighted_score": 9.8, "brief_justification": "The response provides correct, complete solutions using advanced TypeScript features (template literal types, recursive types, state machine pattern) with clear examples demonstrating both valid and invalid usage cases." }, { "judge_key": "glm_4_7", "judge_name": "GLM-4-7", "respondent_key": "grok_direct", "respondent_name": "Grok 3 (Direct)", "weighted_score": 9.8, "brief_justification": "The response provides excellent, working TypeScript implementations for both requirements using advanced features like template literal types and state machine patterns. All examples are correct, well-explained, and include proper error cases with @ts-expect-error comments." }, { "judge_key": "deepseek_v3", "judge_name": "DeepSeek V3.2", "respondent_key": "grok_code_fast", "respondent_name": "Grok Code Fast", "weighted_score": 8.8, "brief_justification": "The response accurately implements both TypeScript constraints with clear code and explanations, though depth could be improved by exploring more edge cases or alternatives." }, { "judge_key": "deepseek_v3", "judge_name": "DeepSeek V3.2", "respondent_key": "claude_opus", "respondent_name": "Claude Opus 4.5", "weighted_score": 8.8, "brief_justification": "The response provides accurate and logically valid TypeScript implementations for both constraints, with clear examples and thorough explanations, though the builder example is cut off at the end." }, { "judge_key": "deepseek_v3", "judge_name": "DeepSeek V3.2", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 8.6, "brief_justification": "The response provides accurate and logically valid TypeScript solutions that correctly enforce the specified compile-time constraints, with clear examples and explanations of the key type-level concepts used." }, { "judge_key": "deepseek_v3", "judge_name": "DeepSeek V3.2", "respondent_key": "claude_sonnet", "respondent_name": "Claude Sonnet 4.5", "weighted_score": 9.35, "brief_justification": "The response provides accurate, complete TypeScript solutions for both constraints, includes clear examples and error cases, and offers an insightful alternative builder pattern, making it highly practical and actionable." }, { "judge_key": "deepseek_v3", "judge_name": "DeepSeek V3.2", "respondent_key": "gemini_3_pro", "respondent_name": "Gemini 3 Pro Preview", "weighted_score": 7.2, "brief_justification": "The response provides a solid solution for the Route type using template literal types and demonstrates understanding of TypeScript's advanced type system, but it's incomplete as it doesn't address the Builder pattern requirement at all and the Route implementation cuts off mid-sentence." }, { "judge_key": "deepseek_v3", "judge_name": "DeepSeek V3.2", "respondent_key": "minimax_m2", "respondent_name": "MiniMax M2", "weighted_score": 8.15, "brief_justification": "The response provides technically correct TypeScript solutions for both constraints with good completeness and practical usefulness, though the clarity could be improved with more explanatory comments and the Route solution has a minor correctness issue with optional parameters." }, { "judge_key": "deepseek_v3", "judge_name": "DeepSeek V3.2", "respondent_key": "glm_4_7", "respondent_name": "GLM-4-7", "weighted_score": 8.55, "brief_justification": "The response provides technically correct TypeScript implementations for both constraints with clear examples, though the Route implementation could be slightly more robust for edge cases like optional parameters." }, { "judge_key": "deepseek_v3", "judge_name": "DeepSeek V3.2", "respondent_key": "gpt_codex", "respondent_name": "GPT-5.2-Codex", "weighted_score": 9.8, "brief_justification": "The solution is fully correct, complete, and clearly presented with excellent practical examples, though the builder pattern could be extended for more complex sequences." }, { "judge_key": "deepseek_v3", "judge_name": "DeepSeek V3.2", "respondent_key": "grok_direct", "respondent_name": "Grok 3 (Direct)", "weighted_score": 9.35, "brief_justification": "The solution correctly implements both compile-time constraints with robust TypeScript types, provides comprehensive examples with error cases, and offers practical, reusable patterns for real-world applications." }, { "judge_key": "gpt_codex", "judge_name": "GPT-5.2-Codex", "respondent_key": "grok_code_fast", "respondent_name": "Grok Code Fast", "weighted_score": 8.8, "brief_justification": "The response correctly defines TypeScript types for route parameter extraction and ordered builder methods with clear examples. It covers the requirements well, though the route params type relies on standard excess property checks rather than strict exactness in all cases." }, { "judge_key": "gpt_codex", "judge_name": "GPT-5.2-Codex", "respondent_key": "claude_opus", "respondent_name": "Claude Opus 4.5", "weighted_score": 7.8, "brief_justification": "The types correctly enforce route params and ordered builder calls, with solid examples and extra advanced usage, but the response is truncated at the end and lacks a finished second example, reducing completeness." }, { "judge_key": "gpt_codex", "judge_name": "GPT-5.2-Codex", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 8.8, "brief_justification": "The response correctly implements both type-level constraints with clear examples and explains the approach. Minor edge cases like optional params/no-params handling could be refined, but overall it is accurate and actionable." }, { "judge_key": "gpt_codex", "judge_name": "GPT-5.2-Codex", "respondent_key": "claude_sonnet", "respondent_name": "Claude Sonnet 4.5", "weighted_score": 8.8, "brief_justification": "The response provides accurate TypeScript types enforcing both route parameter matching and ordered builder calls, with clear examples and compile-time errors. It is thorough and well-structured, though a bit extra with the alternative builder pattern." }, { "judge_key": "gpt_codex", "judge_name": "GPT-5.2-Codex", "respondent_key": "gemini_3_pro", "respondent_name": "Gemini 3 Pro Preview", "weighted_score": 4.1, "brief_justification": "The route parameter typing is generally correct, but the response is incomplete and cut off, and it entirely omits the required builder pattern type. As a result it lacks thorough coverage and practical usability." }, { "judge_key": "gpt_codex", "judge_name": "GPT-5.2-Codex", "respondent_key": "minimax_m2", "respondent_name": "MiniMax M2", "weighted_score": 0, "brief_justification": "No response was provided to evaluate, so none of the requested criteria are satisfied." }, { "judge_key": "gpt_codex", "judge_name": "GPT-5.2-Codex", "respondent_key": "glm_4_7", "respondent_name": "GLM-4-7", "weighted_score": 0, "brief_justification": "The response contains no content, so it does not address the question or provide any type definitions or examples." }, { "judge_key": "gpt_codex", "judge_name": "GPT-5.2-Codex", "respondent_key": "deepseek_v3", "respondent_name": "DeepSeek V3.2", "weighted_score": 8.55, "brief_justification": "The response provides valid TypeScript type patterns that enforce parameter extraction and builder ordering with clear examples. It is largely correct and thorough, though the route type doesn’t strictly forbid extra params and some generic constraints are a bit indirect." }, { "judge_key": "gpt_codex", "judge_name": "GPT-5.2-Codex", "respondent_key": "grok_direct", "respondent_name": "Grok 3 (Direct)", "weighted_score": 8.8, "brief_justification": "Response provides correct TypeScript types and examples for both constraints, with clear explanations. Minor edge cases (like non-literal params) aren’t discussed, but overall it is accurate and practical." }, { "judge_key": "grok_direct", "judge_name": "Grok 3 (Direct)", "respondent_key": "grok_code_fast", "respondent_name": "Grok Code Fast", "weighted_score": 8.6, "brief_justification": "The response provides accurate and mostly complete TypeScript solutions for both constraints with clear code examples. Minor limitations are acknowledged, though some edge cases could be explored further." }, { "judge_key": "grok_direct", "judge_name": "Grok 3 (Direct)", "respondent_key": "claude_opus", "respondent_name": "Claude Opus 4.5", "weighted_score": 9.45, "brief_justification": "The response provides accurate and detailed TypeScript solutions for both route parameter enforcement and builder pattern constraints, with clear examples and error cases. It misses only minor additional edge cases or alternative approaches for completeness." }, { "judge_key": "grok_direct", "judge_name": "Grok 3 (Direct)", "respondent_key": "gemini_3_flash", "respondent_name": "Gemini 3 Flash Preview", "weighted_score": 9.05, "brief_justification": "The response is factually accurate with correct TypeScript implementations for both constraints. It provides clear examples and explanations, though the depth could be enhanced with edge case handling." }, { "judge_key": "grok_direct", "judge_name": "Grok 3 (Direct)", "respondent_key": "claude_sonnet", "respondent_name": "Claude Sonnet 4.5", "weighted_score": 9.6, "brief_justification": "The response is highly accurate, covers both constraints comprehensively with working code examples, and provides clear explanations. The depth and practical utility are excellent, though minor clarity could be improved in type annotations." }, { "judge_key": "grok_direct", "judge_name": "Grok 3 (Direct)", "respondent_key": "gemini_3_pro", "respondent_name": "Gemini 3 Pro Preview", "weighted_score": 6.9, "brief_justification": "The response correctly addresses the Route type constraint with a solid implementation, but it is incomplete as it does not cover the Builder pattern. Clarity and depth are good for the provided solution, though usefulness is limited by the missing second part." }, { "judge_key": "grok_direct", "judge_name": "Grok 3 (Direct)", "respondent_key": "minimax_m2", "respondent_name": "MiniMax M2", "weighted_score": 7.05, "brief_justification": "The response provides accurate TypeScript type definitions for the Route and Builder pattern with constraints, but lacks complete example usage and detailed explanations. The clarity could be improved with better structure and comments in the code." }, { "judge_key": "grok_direct", "judge_name": "Grok 3 (Direct)", "respondent_key": "glm_4_7", "respondent_name": "GLM-4-7", "weighted_score": 7.05, "brief_justification": "The response provides accurate TypeScript type definitions for the Route and Builder pattern with constraints, but lacks detailed explanation of the implementation. Example usage is included, though clarity could be improved with better comments or structure." }, { "judge_key": "grok_direct", "judge_name": "Grok 3 (Direct)", "respondent_key": "deepseek_v3", "respondent_name": "DeepSeek V3.2", "weighted_score": 8.8, "brief_justification": "The response provides accurate and mostly complete TypeScript solutions for both constraints with detailed examples. Minor clarity issues arise from complex type definitions without extensive explanation for beginners." }, { "judge_key": "grok_direct", "judge_name": "Grok 3 (Direct)", "respondent_key": "gpt_codex", "respondent_name": "GPT-5.2-Codex", "weighted_score": 9.05, "brief_justification": "The response provides accurate and functional TypeScript type definitions for both constraints with clear examples. It lacks minor additional context or edge-case handling for completeness and depth." } ], "meta": { "source": "The Multivac (app.themultivac.com)", "methodology": "10x10 blind peer matrix evaluation", "criteria": "correctness, completeness, clarity, depth, usefulness", "self_judgments": "excluded from rankings", "license": "Open data — cite as: The Multivac (2026)" } }