8.98 Out of 10, on the 10% of Calls That Answered
On the Fronset benchmark, a model’s quality score is measured only on the calls that came back, so a high score can describe a small slice of the work you send it. GPT-6 Sol scores 8.98 out of 10 on publication title package generation but answers on only 10% of attempts [^1]. Read every score next to its answer rate, the share of attempts that return an answer, or you will not know how much of that quality you actually receive.
The best scores come with the fewest answers
Across ten kinds of work, GPT-6 Sol’s quality scores run from 6.91 to 8.98 out of 10, and its answer rate runs from 9% to 84%. Its two best scores come with its two lowest answer rates: 8.98 at 10% on publication title packages [^1], and 8.78 at 9% on catalyst and scenario analysis [^2]. Its lowest score, 6.91 on short post batch relevance scoring, comes with a 67% answer rate [^3].
GPT-6 Luna shows the same shape on the same two kinds of work: 8.61 with a 10% answer rate on publication title packages [^1], and 8.6 with 9% on catalyst and scenario analysis [^2]. Qwen 3.7 Plus has the highest score in one set of results, 9.72 on structured output extraction, and answers on 40% of attempts [^4]. In each case the headline number describes a minority of attempts and says nothing about the rest.
On the same task, the lead splits
The trade-off is clearest when several models do the same work. Four models measured on reference-preserving analytical writing show it [^5]:
| Model | Quality score | Answer rate |
|---|---|---|
| GLM-5.3 Flash | 9.22 | 52% |
| GLM-5.3 | 8.86 | 50% |
| Thinking Machines Inkling | 8.44 | 90% |
| Qwen 3.8 Max | 8.16 | 31% |
Among these four, the two highest scorers answer about half the time, while Inkling scores lower and answers far more often. Neither side is better overall. The right pick depends on whether the work needs quality on the calls that return or needs most calls to return.
Claim continuity classification is the case where the answer rate settles the choice. Tencent Hy4 Preview scores 7.57 and answers on 88% of attempts, while Thinking Machines Inkling scores 7.37 but answers on 20% [^6]. The results come without uncertainty ranges, so those two scores are best read as close. A score earned on one call in five says little about what a production path will receive.
What the two numbers measure
A quality score tells you how good the output is when an answer comes back. The answer rate tells you how often it comes back. As the GPT-6 Sol results show, the two can point in opposite directions, and a pipeline lives with both.
The answer rate also belongs to a model on a particular task, not to the model alone. Thinking Machines Inkling answers on 90% of analytical writing attempts [^5] but on 20% of claim continuity classification attempts [^6]. Qwen 3.8 Max answers on 31% of analytical writing attempts [^5], yet on hypothesis post generation it scores 8.72 with 90% answered [^7]. Where a strong score comes with an answer rate near 90%, the score applies to most of what you send.
Choosing a model
Where a missing answer stalls a pipeline, send the work to models with high answer rates: Tencent Hy4 Preview for claim continuity classification [^6], Thinking Machines Inkling for analytical writing [^5], and Qwen 3.8 Max for hypothesis posts [^7]. Use the low-answer, high-score options, such as GLM-5.3 on analytical writing [^5], Qwen 3.7 Plus on structured extraction [^4], and GPT-6 Sol or GPT-6 Luna on publication title packages [^1], only where retries or a fallback model are affordable.
Compare scores only within a task, since scores on differently named tasks cannot be ranked against each other. These results also carry no cost or latency figures, so they cannot decide a choice on price or speed.
The live benchmark at https://fronset.ai/benchmark/ lets you set the quality bar you need and compare batch against sync pricing yourself. The method is documented at https://fronset.ai/benchmark/methodology/.