A Pricier Model Only Pays Off When You Raise the Bar

On engagement triage in the Fronset benchmark, Claude Opus 5.5 at $0.01 per task in sync mode is statistically tied with Claude Haiku 5.5 at $0.0003 [^1]. At a bar that accepts any of the leading scores, that is the usual result: the leaders tie, and the defensible pick is the cheapest model in the tied group. A pricier model earns its price only when you raise the bar to require a clear separation, and only on the tasks where the scores actually separate [^2].

How to read a tie

Each quality score here is a mean on a 0–10 scale, reported with a confidence interval, written as ±. Where two models’ intervals overlap, the result is a tie, and the tie is the finding rather than a gap to be rounded away. A loose bar accepts any model in the leading tied group; a strict bar accepts only a model whose interval sits clear of the others.

At a loose bar, the cheapest tied model qualifies

On engagement triage, Claude Opus 5.5 scores 8.56 (±0.3), Claude Haiku 5.5 scores 8.52 (±0.33) and Grok 4.7 scores 8.51 (±0.32), with Grok priced at $0.0091 per task [^1]. The three ranges overlap, so on this task the cost curve is flat and the expensive route is simply overpaying.

Translation shows the same shape. MiniMax M3 scores 8.79 (±0.22) at $0.0011 per task and GLM-5.3 scores 8.92 (±0.31) at $0.07, and both overlap the leader, Gemini 3.8 Flash at 9.01 (±0.24) [^3]. On reviewing profile pool matches, Claude Haiku 5.5 scores 8.28 (±0.33) at $0.0012 in batch mode, against 8.25 (±0.36) at $0.05 in batch mode for Claude Opus 5.5 [^4].

At a strict bar, some premiums pay

Require non-overlapping intervals and the tied groups break apart, but only on specific tasks. On generating section prompts, Claude Opus 5.5 scores 9.12 (±0.22) at $0.10 per task in sync mode, against 8.18 (±0.46) at $0.05 for Claude Sonnet 5.5 [^2]. On selecting authoritative sources, GLM-5.3 scores 8.65 (±0.15) at $0.18, against 8.11 (±0.29) at $0.02 for GLM-5.3 Flash [^5]. On voice profile generation, Tencent Hy4 Preview at 9.47 (±0.15) clears Tencent Hy3 at 7.95 (±0.29); Hy3 costs $0.0005 in sync mode, and no price is given for Hy4 Preview [^6].

The same pair of models can fall on either side of that line. Gemini 3.8 Flash scores 8.12 (±0.27) on analysing profiled documents, against 6.88 (±0.4) for Gemini 3.5 Flash Lite, at $0.0071 per task in sync mode against $0.0018 [^7]. On language identification the same pair tie at 9.92 (±0.15) and 9.91 (±0.12), with the Lite model cheaper at $0.0001 against $0.0005 [^8]. A strict bar justifies the Flash premium on the first task and not on the second.

Where the cost curve bends

The point where cost starts to climb can only be bracketed, not pinned. Many tasks have only one or two models measured, and most carry no cost figure for most models, so the cheapest model that clears a given bar often cannot be named.

Eligibility checks show what the bracket looks like. Set the bar at about 9.1 and only the tied top band qualifies: Claude Opus 5.5 at 9.17 (±0.14), GLM-5.3 at 9.18 (±0.3) and GLM-5.3 Flash at 9.13 (±0.39), none of them with a reported price [^9]. Claude Haiku 5.5 at 8.67 and $0.0006 and DeepSeek V4.1 Flash at 8.95 and $0.0007 sit just below that band on the point scores, and if a score around 8 is acceptable, GPT-6 Luna in batch mode is the cheapest reported route at $0.0001 per task, scoring 8.09 (±0.36) [^9]. So on this task the knee sits somewhere between the sub-cent routes and the top band.

What to do about it

Write down the bar before you look at models: the score the task needs, and whether a difference has to clear the interval to count. Then read the intervals, not the point scores. If the candidates overlap at your bar, take the cheapest one with a reported price.

Pay for the larger or newer model only where its range clears the cheaper model’s at your bar, and only on that task. Where the qualifying band carries no price, as on eligibility checks, the cost of the strict bar is unknown, so confirm it before you commit a budget to it.

The live benchmark at https://fronset.ai/benchmark/ lets you set the bar your task actually needs and see which models clear it.