Check the Error Bars Before You Call It a Lead

On the Fronset benchmark, a higher score is a real lead only when the two models’ confidence intervals do not overlap. In the busy middle of the scale, between roughly 8 and 9, they usually do, and the higher score is then a tie rather than a lead. The numbers worth planning against are the ones that pair a high mean with a narrow interval.

How to read the ± figure

Each score is a mean quality on a 10-point scale, and the ± figure beside it is the 95% confidence interval on that mean. The benchmark treats a model as ranked on a task only when that interval is within ±0.2 points [^1]. Inside that line, the mean is known tightly enough to place; outside it, the score is a rough centre rather than a position.

Most leads in the 8-to-9 band are ties

Selecting social posts for a portfolio shows the pattern. Tencent Hy4 Preview scores 8.65, ±0.16, Claude Opus 5.5 8.6, ±0.21, GLM-5.3 8.6, ±0.17 and Grok 4.7 8.49, ±0.22 [^2]. Tencent Hy4 Preview has the highest figure, and the four are tied.

Models from the same family can tie as well. Claude Opus 5.5 at 8.37, ±0.2 and Claude Sonnet 5.5 at 8.21, ±0.26 are tied on evidence-grounded claim generation [^3]. Reading a higher point estimate that the interval will not support as a better model is the most common way to over-read a published number.

When a gap is real, and when it cannot be judged

A lead holds when the intervals do not overlap. Qwen 3.7 Plus scores 8.46, ±0.17, on writing publication title packages, above Qwen 3.8 Flash at 7.52, ±0.29 [^4], and here the intervals do not overlap.

Where one side has no interval at all, no gap can be judged. GLM-5.3 scores 8.26, ±0.25, on catalyst and scenario analysis, while GLM-5.3 Flash scores 8.03 with no interval [^5].

The firmest results pair a high mean with a narrow interval. MiniMax M3 scored 9.08, ±0.03, after 106 independent judgments on writing newsletter copy [^6], well inside the ±0.2 line. That is a number a team can plan against, with the caveat that it describes newsletter copy only.

Why a score gap is not a ranking

Every score is absolute: one judge, the grader the benchmark uses, scores one output at a time against a rubric [^1]. The scores therefore support statements about each model’s quality, not an ordering against another model. An ordering has to come from intervals that separate, or from direct comparison of answers against each other.

Language identification shows what direct comparison can reveal. Every model measured sits near the top of the scale there, including NVIDIA Nemotron-3 Nano 30B-A3B at 9.89, ±0.08 after 456 independent judgments and GPT-6 Luna at 10.0, ±0.02 [^7]. These come from separate sets of results, not one league table. In direct comparison, though, the top two answers scored exactly the same in 100.0% of 62 judged occasions [^7], so on this work a gap between two models’ published scores tells you very little about what you will get.

What to do with a close score

Before treating a higher score as a lead, check whether the two intervals overlap. If they do, the models are tied for that task, and a team with a quality bar near 8 can treat them as interchangeable and let price or latency decide.

Treat a score with no interval as unplaced, however high it is. Reserve budgets and routes that depend on a specific model for cells whose interval sits inside the ±0.2 line and is separated from the alternatives.

The live benchmark at https://fronset.ai/benchmark/ shows the current interval for the exact task you plan to run. Set the quality bar you actually need there, and compare intervals, not just scores, before you commit a route.