# The Multivac — Evaluation Report

**Evaluation ID:** EVAL-20260402-124253
**Date:** Mar 03, 2026
**Category:** code
**Question ID:** CODE-008

---

## Question

Implement a production-ready API rate limiter with the following requirements:
1. Token bucket algorithm
2. Support for different rate limits per API key
3. Redis backend for distributed systems
4. Graceful degradation when Redis is unavailable
5. Proper async support
6. Comprehensive logging

Include the main class, Redis integration, and a FastAPI middleware example.

---

## Winner

**Gemini 3 Flash Preview** (Google)
- Winner Score: 8.28
- Matrix Average: 6.70
- Total Judgments: 87

---

## Rankings

| Rank | Model | Provider | Avg Score | Judgments |
|------|-------|----------|-----------|----------|
| 1 | Gemini 3 Flash Preview | Google | 8.28 | 8 |
| 2 | Grok 4.20 | openrouter | 7.94 | 8 |
| 3 | GPT-5.4 | openrouter | 7.71 | 9 |
| 4 | MiMo-V2-Flash | Xiaomi | 7.44 | 9 |
| 5 | Claude Opus 4.6 | openrouter | 7.00 | 9 |
| 6 | GPT-OSS-120B | OpenAI | 6.87 | 9 |
| 7 | Claude Sonnet 4.6 | openrouter | 6.82 | 8 |
| 8 | DeepSeek V4 | openrouter | 6.81 | 9 |
| 9 | MiniMax M2.5 | openrouter | 4.97 | 9 |
| 10 | Gemini 3.1 Pro | openrouter | 3.12 | 9 |

---

## 10×10 Judgment Matrix

Rows = Judge, Columns = Respondent. Self-judgments excluded (—).

| Judge ↓ / Resp → | GPT-5.4 | Gemini 3 | Claude Opus | Gemini 3.1 Pro | Claude Sonnet | Grok 4.20 | DeepSeek V4 | GPT-OSS-120B | MiniMax M2.5 | MiMo-V2-Flash |
|---|---|---|---|---|---|---|---|---|---|---|
| GPT-5.4 | — | 7.8 | 3.7 | 1.6 | 4.0 | 5.4 | 4.8 | 3.0 | 2.9 | 5.2 |
| Gemini 3 | 9.6 | — | 9.6 | 3.0 | 8.8 | 9.3 | 9.2 | 8.8 | 7.0 | 9.0 |
| Claude Opus | 7.0 | 7.8 | — | 0.8 | 6.0 | 8.0 | 7.0 | 6.3 | 3.7 | 6.6 |
| Gemini 3.1 Pro | 7.0 | 8.8 | 5.4 | — | 5.8 | 8.3 | 4.8 | 5.8 | 3.2 | 7.8 |
| Claude Sonnet | 8.2 | 8.6 | 8.4 | 1.6 | — | 8.3 | 7.5 | 7.2 | 5.7 | 7.4 |
| Grok 4.20 | 8.4 | 7.8 | 7.4 | 3.6 | · | — | 6.6 | 6.4 | 5.8 | 6.6 |
| DeepSeek V4 | 8.6 | · | 8.6 | 8.4 | 8.8 | 8.8 | — | 9.2 | 7.6 | 9.0 |
| GPT-OSS-120B | 5.7 | 8.6 | 5.1 | 2.4 | 5.8 | 7.0 | 6.2 | — | 5.7 | 7.5 |
| MiniMax M2.5 | 6.3 | 7.5 | 6.8 | 1.0 | 6.5 | · | 7.4 | 6.5 | — | 7.8 |
| MiMo-V2-Flash | 8.6 | 9.3 | 8.0 | 5.5 | 8.6 | 8.3 | 7.8 | 8.6 | 3.2 | — |

---

## Methodology

- **10×10 Blind Peer Matrix:** All models answer the same question, then all models judge all responses.
- **5 Criteria:** Correctness, completeness, clarity, depth, usefulness (each scored 1–10).
- **Self-judgments excluded:** Models do not judge their own responses.
- **Weighted Score:** Composite of all 5 criteria.

---

## Citation

The Multivac (2026). Blind Peer Evaluation: CODE-008. app.themultivac.com

## License

Open data. Free to use, share, and build upon. Please cite The Multivac when using this data.

Download raw JSON: https://app.themultivac.com/api/evaluations/EVAL-20260402-124253/results
Full dataset: https://app.themultivac.com/dashboard/export