The llm-bench API — what it costs and how to start
An OpenAI-compatible API that routes each call to the cheapest model good enough for it. Free tier, bring your own keys, or let us manage the models.
What it is
The same routing the benchmark publishes, as an API. You send an OpenAI-compatible request; we pick the cheapest model that clears the quality bar for that task, and tell you which one answered.
The base URL is OpenAI-compatible, so most clients need only a base URL and a key.
Two ways to pay for the models
Bring your own keys. You vault your provider credentials; your provider bills you directly for the tokens, and we charge a per-call fee for the routing and the evaluation behind it.
Let us manage them. We serve the call on our provider accounts and pass the token cost through at list price, drawn from prepaid credit you load in advance. The per-call fee is the same either way.
The per-call fee
Counted per request, not per model attempt — a call that fans out to several models to pick a winner is one call.
| Calls per month | Per 1,000 calls |
|---|---|
| First 10,000 | included |
| 10,000 – 50,000 | $2.00 |
| 50,000 – 250,000 | $1.50 |
| 250,000+ | $1.00 |
Graduated: each band applies only to the calls inside it.
Plans
Free — $0. Every routing feature, the full model catalogue, and the allowance above. Your evaluation data is pooled, which is what keeps the benchmark honest and the tier free.
Paid tiers that keep your data unpooled and add an SLA are not yet available for self-service. If that is what you need, get in touch rather than signing up — we would rather tell you what we can actually do today.
Getting started
Self-service signup is not open yet.
You will get two keys: one for inference, one for managing the account. They are shown once.
What you need before your first completion depends on how the task is served.
A task we create for you from an unrecognised model string runs on your own
provider keys, so vault one and no credit is required — your provider bills
you directly. A task served on our keys draws prepaid credit for the model
usage, and a call made before you load any is refused. Either way there is a
per-call fee, and every account has a monthly allowance of calls that carry
none.
Listing the models available to you works immediately in both cases, so that is the quickest way to check your key is live.