Median 5.5 s, p90 51 s: Set Timeouts on the Tail
Most latency figures in the Fronset benchmark are medians, and the median understates how long callers actually wait. Across 841,549 synchronous calls, the median is 5.5 seconds, but the 90th percentile is 51 seconds [^1]. A model with the lower median can also have the longer tail, so picking a model or setting a timeout on the median alone can go wrong.
Two numbers, two different promises
The median is the response time that half of all calls beat; the other half take longer. The 90th percentile, or p90, is the time that nine calls in ten beat, so one call in ten takes longer still. A latency budget set on the median is exceeded by roughly half the calls, and one set on the p90 by roughly one in ten.
Every figure here is from synchronous calls measured under a 600-second per-call deadline, a setting chosen for these measurements rather than a property of the models. These are latency results only. They say nothing about answer quality or cost, and a quality bar or a cost limit would each change which model you pick.
How far the tail reaches
Where a p90 is published, it sits well above the median. On Claim Continuity Classification, NVIDIA Nemotron 3.5 Lightning has a median of 18 seconds and a p90 of 173 seconds over 212 calls [^2].
A low median does not remove the problem. Gemini 3.5 Flash Lite answers the same kind of work with a median of 1.6 seconds and a p90 of 27 seconds over 218 calls [^2]. If you only read the median, you would expect near-instant answers, yet one call in ten takes longer than 27 seconds.
The gap is not limited to small samples. deepseek-v4-pro on language-model quality evaluation has a median of 28 seconds and a p90 of 103 seconds over 9,413 calls [^3]. Together with the aggregate figure above, the same shape shows up from a few hundred calls to hundreds of thousands.
A lower median does not mean a shorter tail
The tail does not always follow the median. On relevance scoring of documents for a topic report, three models have both figures [^4]:
| Model | Median | p90 |
|---|---|---|
| deepseek-v4-flash | 4.4 s | 19 s |
| Qwen 3.7 Plus | 20 s | 96 s |
| Tencent Hy3 | 22 s | 68 s |
deepseek-v4-flash is fastest at both points. Between the two slower models, the order flips: Qwen 3.7 Plus has the lower median, but Tencent Hy3 has the lower p90. Choosing between those two on the median would pick the one with the longer tail.
No confidence intervals are given, so treat this ordering as indicative rather than settled. The point is not that one of these models is better. It is that the median and the p90 can disagree, and a comparison on the median alone cannot see it.
What to do about it
If a user or a downstream step is waiting on the call, size the budget on the p90 where one exists. A timeout set on the median is designed around the typical call and will be blown by the slow ones. When you compare candidate models for interactive work, put the median and the p90 side by side, because the ranking can change between them.
Most tasks have no tail figure at all. A team with a tight interactive budget should not assume those tasks behave better than the ones measured. Where no p90 exists, measure the tail yourself on your own traffic before committing to a budget.
The live benchmark at https://fronset.ai/benchmark/ has the per-task figures behind these numbers, and the method is documented at https://fronset.ai/benchmark/methodology/.