Fronset Benchmark

Best LLMs for Topic Cluster Labeling

Assigns a concise descriptive label and short explanation to a cluster of semantically related claims or content items. Illustrative uses include labeling clusters of customer requests, support issues, software defects, sales notes, research claims, interview excerpts, patient fe

What the steps mean
Cost mode:

Category: Topic Organization & Clustering · Typical I/O: 9201→555 tokens

Runs on model group: All models. Inputs may reach any model in the group, not only the ranked models below.

Run this task on Fronset — request an invitation

Models

Frontier on this task: Moonshot Kimi K3 at 8.87 / 10. Score needed at the Balanced step: 7.98.

point-estimate floor (CI low) · upper CI (less certain) · Bars sorted by blended cost; best-value model first. Greyed rows are MEDIUM+ models whose point estimate does not clear the bar.

Cost breakdown

ModelQualityConfidenceCost / 1k runsOverpayMode
Xiaomi MiMo V2.6 Flash ★ Xiaomi MiMo8.34 / 10 CI [8.04, 8.65]MEDIUM$0.19best valuebatch
Claude Haiku 5.5 Anthropic8.01 / 10 CI [7.66, 8.37]MEDIUM$0.733.9xbatch
GPT-6 Luna OpenAI8.40 / 10 CI [8.16, 8.64]HIGH$0.804.3xbatch
DeepSeek V4.1 Flash DeepSeek8.36 / 10 CI [8.14, 8.57]HIGH$1.699.1xbatch
Gemini 3.8 Flash Gemini8.16 / 10 CI [7.94, 8.38]HIGH$2.0111xbatch
MiniMax M3 OpenRouter8.17 / 10 CI [8.01, 8.32]RANKED$2.4313xbatch
Tencent Hy3 OpenRouter8.14 / 10 CI [7.96, 8.31]RANKED$6.1733xbatch
Thinking Machines Inkling Small OpenRouter8.28 / 10 CI [7.91, 8.65]MEDIUM$7.5841xbatch
GPT-6 Sol OpenAI8.56 / 10 CI [8.35, 8.78]HIGH$9.4551xbatch
Qwen 3.7 Plus Alibaba Cloud (DashScope)8.13 / 10 CI [7.79, 8.48]MEDIUM$10.7658xbatch
Claude Sonnet 5.5 Anthropic8.55 / 10 CI [8.30, 8.79]HIGH$15.2882xbatch
Meta Muse Spark 1.3 OpenRouter8.17 / 10 CI [7.90, 8.44]HIGH$22.43120xbatch
Thinking Machines Inkling OpenRouter8.29 / 10 CI [7.90, 8.69]MEDIUM$26.35141xbatch
Claude Opus 5.5 Anthropic8.69 / 10 CI [8.50, 8.88]RANKED$34.07182xbatch
Grok 4.7 xAI8.39 / 10 CI [8.11, 8.68]HIGH$38.79208xbatch
Moonshot Kimi K3 best Moonshot AI8.87 / 10 CI [8.70, 9.03]RANKED$50.13268xbatch
GLM-5.3 Z.AI8.69 / 10 CI [8.45, 8.93]HIGH$85.64458xbatch

Overpay shows how much more you pay than the best-value model that meets the quality step (marked ★) — the best-value good-enough option. "16x" means you overpay 16× — 16× that reference for no quality benefit the step asks for. Typical call shape for this task: 9201 input tokens → 555 output tokens, EMA-tracked from production traffic. Cost is the observed, all-in $ per 1,000 task runs: each model's own measured usage on this task — output verbosity, thinking/reasoning tokens, cache reads and writes, and the spend on its billed failures — priced at current list rates and adjusted by the billing overhead we actually reconcile against provider invoices. Models that answer tersely cost what they actually cost; models that think at length pay for it. Not comparable to providers' advertised $/1M list rates — this is what running the task costs, not a per-token price.

Evaluation rubric

Judge representativeness, specificity, distinction from neighboring concepts, support across cluster members, label clarity, concise boundary description, and absence of unsupported interpretation.

Output schema

Every answer on this task is checked against this JSON Schema, whichever model wrote it. An answer that doesn't fit counts as a model failure, and the call is retried on another model.

{
  "description": "Output from the topic cluster naming LLM call.\n\nGiven a cluster of semantically similar claims and their synthesis summary,\nnames and describes the topic that the cluster represents.",
  "properties": {
    "topic_description": {
      "default": "",
      "description": "Brief description of what this topic covers and why these claims belong together (max 500 characters)",
      "maxLength": 500,
      "title": "Topic Description",
      "type": "string"
    },
    "topic_name": {
      "description": "Concise, descriptive name for the topic (e.g., 'Revenue Growth Outlook', 'Regulatory Headwinds')",
      "title": "Topic Name",
      "type": "string"
    }
  },
  "required": [
    "topic_name"
  ],
  "title": "TopicClusterNamingOutput",
  "type": "object"
}

Prompt templates

The published template for this task.

default · LLMB_TOPIC_CLUSTER_LABELING_SYSTEM + LLMB_TOPIC_CLUSTER_LABELING_USER (5506 calls in window)

System prompt

Name the shared substantive idea that distinguishes this cluster from adjacent clusters. The name must be reusable across subjects: state the theme in wording that a cluster about a different company, market or programme could match into, and keep company, organisation, product, ticker and person names out of it — subject_name is context for reading the claims, never a component of the name. The specifics belong in the description: name there the entities, mechanisms, outcomes or tensions supported by multiple members, and state the common scope and the boundary that separates this cluster from its neighbours. Avoid generic labels such as “Overview,” unsupported conclusions, category repetition, and wording derived from a single outlier. Treat empty optional values as absent and return only the requested result. Your response must conform exactly to this output schema: {schema_json_string}.

User prompt

Inputs — subject_name: {subject_name}; category: {category}; claim_count: {claim_count}; claims_text: {claims_text}. Use only these inputs to complete the task defined by the system prompt.