Nano, Lite, Haiku: When the Smaller Sibling Costs You Quality

For plumbing work, the usual reflex is to reach for the smallest member of a model family: the Lite, the Nano, the Haiku. On several utility tasks in the Fronset benchmark, that step down brings a clear drop in score, while on others it costs nothing measured. Which case applies depends on the task, so the family gap has to be checked task by task.

Where stepping down costs quality

Each score here is out of 10 and carries the ± range the benchmark reports; where two ranges overlap, the models are treated as tied. Costs are per task run; sync mode, where the source specifies it, means an ordinary synchronous call.

The Gemini family shows the pattern on document work. On profiled document analysis, Gemini 3.8 Flash scores 8.12 (±0.27) at $0.0071 in sync mode [^1], against Gemini 3.5 Flash Lite’s 6.88 (±0.4) at $0.0018 [^1]. On structured content summarization, Gemini 3.8 Flash scores 7.96 (±0.35) and Gemini 3.5 Flash Lite 5.92 (±0.31) [^2]. The two models differ in version number as well as tier, so these gaps cannot be put down to size alone.

NVIDIA’s Nemotron-3 family splits the same way on thematic topic discovery, a task that is weak across the field. NVIDIA Nemotron-3 Ultra 550B scores 8.19 (±0.37) at $0.05 [^3], against 6.76 (±0.44) at $0.01 for NVIDIA Nemotron-3 Super 120B [^3]. The larger model clearly leads inside the family.

Where stepping down is free

The same Nemotron-3 family gives up nothing measured on language identification. NVIDIA Nemotron-3 Ultra 550B scores 9.95 (±0.13) at $0.0007, while NVIDIA Nemotron-3 Super 120B scores 9.92 (±0.11) and NVIDIA Nemotron-3 Nano 30B-A3B 9.89 (±0.08), each at $0.0001, all in sync mode [^4]. All three sit inside the tied top band the benchmark reports for that task, which starts at GLM-5.3 Flash’s 9.77 (±0.24) [^4].

Claude shows the same contrast. On metadata rewriting, Claude Opus 5.5 scores 8.0 (±0.25) and Claude Haiku 5.5 7.95 (±0.23) [^5], part of what the source calls a flat tie. Haiku 5.5 costs $0.0002 in sync mode against Opus 5.5’s $0.0072 [^5], so the step from Opus to Haiku costs nothing measured there.

Research query generation shows that the step that matters can be the second one. Claude Opus 5.5 at 7.24 (±0.42) and Claude Sonnet 5.5 at 7.17 (±0.37) are tied, so Sonnet 5.5 at $0.0064 is the cheaper route than Opus 5.5 at $0.01 [^6]. One step further down, Claude Haiku 5.5 scores 6.19 (±0.31) [^6].

What the pattern means

The family gap tracks the benchmark’s own split between tasks where the field ties and tasks where it separates. Language identification and metadata rewriting are ties across models from many developers, and there the step down within a family is free too. Document analysis, summarization, topic discovery and research query generation are among the tasks where models genuinely separate, and that is where the smaller sibling falls behind.

Calling a task utility work does not tell you which kind it is. Summarizing structured content or analysing a profiled document can look like plumbing, but on these figures it behaves like work where model choice changes the output.

What to do about it

Treat a move to the Lite, Nano or Haiku tier as a per-task decision, not a default for plumbing work. Before switching a task, compare the smaller sibling’s score with the larger one’s on that exact task, and step down freely only where both sit inside a tied band. Check each tier in turn rather than jumping straight to the smallest.

Where a gap shows up, test both siblings on your own inputs and weigh the saving against the lost score. Confirm prices against your live usage as well, since the benchmark’s cost figures mix repriced and measured bases and are indicative rather than exact.

The measurement behind these figures is ongoing. Check the live benchmark at https://fronset.ai/benchmark/ for the family gap on the task you plan to run before committing to a route.