Cheapest LLM that's good enough.
Snapshot 2026-10-31 · Quality step Balanced · Cost mode: batch if supported · 54 tasks · 27 models
In our busiest week we made over 180,000 calls to large-language models, and we let the outputs be evaluated by a panel of leading models. Cost per token doesn't track quality — and when you say something like "give me the most cost-effective model that's reliably nearly as good as the best one" (the Near-best step), the best-value model typically saves north of 90% of the cost.
This benchmark publishes our results so other builders can answer the same question for their own pipelines: which model(s) are good enough and the most cost-efficient for each category of tasks. The data also shows that the most expensive model — even when your budget can pay for it — is not always the best performer.
Have a look. Tell us what you think. We can also run tests for you — reach us at benchmark@fronset.ai.
Right-sized per step
Best-value model per capability, at your quality step. Greyed cells mean no model qualifies on that category yet.
1.0x = this model is the best-value good-enough option, no overpayment; 3.2x = you pay ~3× more on average for the same quality).
★ marks any model that is the best-value pick on at least one of the category's tasks (multiple stars per column are possible — they're the per-task cost winners).
Greyed cells: model doesn't qualify on any task in that category at the current step.
Rows grouped by provider; click a model name for the full per-task breakdown.Fronset Benchmark · snapshot 2026-10-31 · Balanced step
By capability
Financial Analysis & Trading Decisions
Read business/financial docs deeply enough to make forward-looking judgments about opportunity and risk.
| Model | Tasks | Avg $/1k runs | Overpay |
|---|---|---|---|
| MiniMax M3 ★ | 1/4 | $0.75 | best value |
| Claude Haiku 5.5 ★ | 1/3 | $1.03 | best value |
| GPT-6 Luna ★ | 2/3 | $1.27 | best value |
| DeepSeek V4.1 Flash ★ | 3/4 | $3.53 | 2.2x |
| Xiaomi MiMo V2.6 Pro | 1/4 | $7.76 | 1.6x |
Fronset Benchmark · snapshot 2026-10-31 · Balanced step
Structured Data & Fact Extraction
Precise pattern recognition and field-level information retrieval, strict schema adherence.
| Model | Tasks | Avg $/1k runs | Overpay |
|---|---|---|---|
| Claude Haiku 5.5 ★ | 1/3 | $0.34 | best value |
| Xiaomi MiMo V2.6 Flash ★ | 1/3 | $0.74 | best value |
| NVIDIA Nemotron-3 Super 120B | 1/1 | $0.93 | 2.7x |
| GPT-6 Luna | 2/3 | $1.41 | 1.9x |
| Xiaomi MiMo V2.6 Pro ★ | 2/3 | $1.48 | 2x |
Fronset Benchmark · snapshot 2026-10-31 · Balanced step
Content Summarization & Synthesis
Compress long input into essential nuggets without losing material detail.
| Model | Tasks | Avg $/1k runs | Overpay |
|---|---|---|---|
| Xiaomi MiMo V2.6 Flash ★ | 2/3 | $0.57 | 1.2x |
| GPT-6 Luna ★ | 4/4 | $2.22 | 1.5x |
| Claude Haiku 5.5 ★ | 2/4 | $2.33 | best value |
| MiniMax M3 | 2/3 | $3.05 | 7.8x |
| DeepSeek V4.1 Flash | 3/4 | $3.21 | 4.1x |
Fronset Benchmark · snapshot 2026-10-31 · Balanced step
Long-form Content Generation
Sustained compositional skill, voice consistency, coherent extended prose.
| Model | Tasks | Avg $/1k runs | Overpay |
|---|---|---|---|
| GPT-6 Luna ★ | 2/3 | $0.71 | best value |
| Claude Haiku 5.5 | 2/3 | $0.91 | 1.3x |
| MiniMax M3 ★ | 3/3 | $1.05 | 1.5x |
| Tencent Hy3 | 1/3 | $1.31 | 1.8x |
| DeepSeek V4.1 Flash | 1/2 | $1.65 | 2.4x |
Fronset Benchmark · snapshot 2026-10-31 · Balanced step
Social & Promotional Content
Conciseness, platform-native conventions, engagement under tight character limits.
| Model | Tasks | Avg $/1k runs | Overpay |
|---|---|---|---|
| GPT-6 Luna ★ | 2/3 | $0.33 | best value |
| Xiaomi MiMo V2.6 Flash ★ | 2/4 | $0.34 | 1.3x |
| Gemini 3.5 Flash Lite | 1/4 | $0.49 | 1.3x |
| Xiaomi MiMo V2.6 Pro | 2/2 | $0.71 | 2.8x |
| Claude Haiku 5.5 ★ | 4/4 | $0.75 | 1.3x |
Fronset Benchmark · snapshot 2026-10-31 · Balanced step
Relevance, Classification & Matching
Semantic similarity judgment: does this thing belong in that bucket / match that target?
| Model | Tasks | Avg $/1k runs | Overpay |
|---|---|---|---|
| Gemini 3.5 Flash Lite | 3/9 | $0.23 | 3x |
| NVIDIA Nemotron-3 Nano 30B-A3B | 2/6 | $0.23 | 2.9x |
| NVIDIA Nemotron 3.5 Lightning | 2/7 | $0.66 | 6.4x |
| NVIDIA Nemotron-3 Super 120B | 3/9 | $0.71 | 6.8x |
| Claude Haiku 5.5 ★ | 8/14 | $0.75 | 2.1x |
Fronset Benchmark · snapshot 2026-10-31 · Balanced step
Topic Organization & Clustering
Discover natural groupings without external schema, name and order them coherently.
| Model | Tasks | Avg $/1k runs | Overpay |
|---|---|---|---|
| Xiaomi MiMo V2.6 Flash ★ | 2/2 | $0.21 | best value |
| Claude Haiku 5.5 | 1/3 | $0.73 | 3.9x |
| GPT-6 Luna | 1/2 | $0.80 | 4.3x |
| Xiaomi MiMo V2.6 Pro | 1/2 | $1.10 | 4.5x |
| DeepSeek V4.1 Flash | 1/3 | $1.69 | 9.1x |
Fronset Benchmark · snapshot 2026-10-31 · Balanced step
Infrastructure & Utility
Mechanical competence at format conversion, metadata manipulation, prompt rewriting, translation; minimal domain expertise required.
| Model | Tasks | Avg $/1k runs | Overpay |
|---|---|---|---|
| Xiaomi MiMo V2.6 Flash ★ | 3/6 | $0.12 | best value |
| Claude Haiku 5.5 | 3/4 | $0.23 | 2x |
| Gemini 3.5 Flash Lite | 1/6 | $0.59 | 1.8x |
| GPT-6 Luna ★ | 7/7 | $0.60 | 1.3x |
| Xiaomi MiMo V2.6 Pro | 4/5 | $0.60 | 2.1x |
Fronset Benchmark · snapshot 2026-10-31 · Balanced step
Evidence-Grounded Hypothesis Generation
Derive distinct, arguable hypotheses or claims from a source document while grounding and citing each one strictly in a supplied set of approved evidence.
| Model | Tasks | Avg $/1k runs | Overpay |
|---|---|---|---|
| Thinking Machines Inkling Small ★ | 1/1 | $4.91 | best value |
| MiniMax M3 | 1/1 | $5.39 | 1.1x |
| Gemini 3.8 Flash | 1/1 | $7.54 | 1.5x |
| Qwen 3.7 Plus | 1/1 | $8.20 | 1.7x |
| Meta Muse Spark 1.3 | 1/1 | $15.73 | 3.2x |
Fronset Benchmark · snapshot 2026-10-31 · Balanced step
Methodology, briefly
Quality scores come from LLM-judge verdicts on production workloads, on a 0–10 scale. Only MEDIUM-or-better cells qualify. Each quality step sets the bar at a share of the best-performing model's score on each task — move the slider above to see your own pipeline. The methodology lists the six steps.
What changed this week
What changed — 2026-10-16 → 2026-10-31
- Tasks: 54 · Models: 27 · Model–task pairs: 929 (+23) · Graded samples: 49,974 (+3,263)
Per-model movement
(every model, Balanced step) — best value = tasks where the model is the cheapest good-enough option, counted separately under each cost mode; qualifying = tasks it’s good-enough on (quality only, so it is the same under both modes). A model with no batch API prices batch at its sync rate, so the two best-value columns diverge wherever a provider batch discount decides the winner:
| Model | Best Value (Sync) | Best Value (Async/Batch) | Qualifying |
|---|---|---|---|
| GPT-6 Luna | 17 (+5) | 13 (-2) | 26 (+2) |
| Xiaomi MiMo V2.6 Flash | 11 (-2) | 13 (-1) | 17 (+1) |
| MiniMax M3 | 6 (-1) | 5 | 21 (-2) |
| Claude Haiku 5.5 | 2 (-2) | 6 (+2) | 22 (+5) |
| DeepSeek V4.1 Flash | 5 | 2 (-3) | 23 (+5) |
| Thinking Machines Inkling Small | 4 (+1) | 4 (+1) | 26 (-1) |
| GLM-5.3 Flash | 3 | 3 | 26 (-1) |
| Xiaomi MiMo V2.6 Pro | 1 (+1) | 3 (+3) | 21 (+1) |
| Tencent Hy3 | 2 | 2 (+1) | 15 (-1) |
| Thinking Machines Inkling | 1 | 1 | 27 (-2) |
| Qwen 3.7 Plus | 1 (-2) | 1 (-2) | 21 (-2) |
| NVIDIA Nemotron-3 Ultra 550B | 1 (+1) | 1 (+1) | 14 (+2) |
| Meta Muse Spark 1.3 | 0 | 0 | 37 (+3) |
| Moonshot Kimi K3 | 0 | 0 | 37 |
| Claude Opus 5.5 | 0 | 0 | 34 (+1) |
| Grok 4.7 | 0 | 0 | 31 (+2) |
| GPT-6 Sol | 0 | 0 | 28 |
| GLM-5.3 | 0 | 0 | 27 |
| Claude Sonnet 5.5 | 0 | 0 | 26 (+2) |
| Tencent Hy4 Preview | 0 | 0 | 26 (-1) |
| Gemini 3.8 Flash | 0 | 0 | 21 (-3) |
| Qwen 3.8 Max | 0 | 0 | 20 |
| Gemini 3.5 Flash Lite | 0 | 0 | 6 (-1) |
| NVIDIA Nemotron-3 Super 120B | 0 (-1) | 0 | 6 |
| Qwen 3.8 Flash | 0 | 0 | 6 (+1) |
| NVIDIA Nemotron 3.5 Lightning | 0 | 0 | 2 |
| NVIDIA Nemotron-3 Nano 30B-A3B | 0 | 0 | 2 |