Error Reference
Every error code the OpenAI-compatible plane can return, what it means, and what to do about it.
Every error on the OpenAI-compatible plane (/v1/) uses the OpenAI error envelope, so an SDK you already have handles it. This page describes the codes that plane returns, what each one means, and what you do next.
The envelope
{
"error": {
"message": "Rate limit exceeded for this tenant. Slow down and retry. (diagnostic: 3f9c1a20d4b7481e9c2f5a6b8d0e7142)",
"type": "rate_limit_error",
"param": null,
"code": "tenant_rate_limited"
}
}
type stays inside the set OpenAI SDKs recognise — invalid_request_error, authentication_error, permission_error, rate_limit_error, insufficient_quota, server_error — so SDK-level branching keeps working. The precise meaning is in code.
code is append-only. New codes get added; existing ones do not change meaning. Write your handling against x-llmbench-error-class (below) so an unrecognised code still lands in the right branch.
Any message we echo from an upstream provider is stripped of key-shaped tokens and truncated to 500 characters before it reaches you.
Diagnostic id vs call id
These are two different headers on two different outcomes. Do not look for one on the other.
| Outcome | Header | What it identifies |
|---|---|---|
| Success (200) | x-llmbench-call-id | the call record for the call that ran |
| Error | x-llmbench-diagnostic | the incident, for support |
The diagnostic id is also appended to error.message as (diagnostic: ...), so it survives a client that logs the body and throws the headers away. Quote it in a support request.
One nuance worth knowing: when a failure got far enough to open a call record, the diagnostic id is that call record’s id — the same value a success would have reported in x-llmbench-call-id. When the request was refused before any model ran (bad key, unknown model, malformed body, streaming, quota), there is no call, and the diagnostic is a freshly minted 32-character hex id instead.
Error headers
| Header | On | Meaning |
|---|---|---|
x-llmbench-diagnostic | errors | incident id — quote it in support |
x-llmbench-error-class | errors | whose problem this is (see below) |
x-llmbench-error-code | errors | same value as error.code |
Retry-After | 429s | seconds to wait before retrying |
x-llmbench-request-id | errors that reached invocation | the route id — quote it in support alongside the diagnostic |
x-llmbench-request-id is deliberately absent on anything refused before invocation. No route was opened, so there is no id, and inventing one would be worse than omitting it.
Not every error carries the header triple. The rule: the triple is attached when the error is raised as a gateway refusal or mapped from an invocation. An error body a plain lookup returns directly carries no x-llmbench-* headers and no (diagnostic: ...) suffix, because the body alone is unambiguous. Lookups in that second group include GET /v1/models/{model_id}, GET /v1/files/{id}, GET /v1/files/{id}/content, DELETE /v1/files/{id}, GET /v1/batches/{id}, and POST /v1/batches/{id}/cancel. Everything from POST /v1/chat/completions, POST /v1/files, and POST /v1/batches carries the full triple.
Triage by class
x-llmbench-error-class tells you who has to act, faster than the code does.
| Class | Whose problem | Default response |
|---|---|---|
provider | a transient upstream fault | retry with backoff |
your_quota | you are out of room | wait for the window, or top up |
your_key | your vaulted provider credential | fix the credential |
your_request | the request content | change the request |
router | configuration of the task you called | fix the task, or pick another |
gateway | our admission control | honour Retry-After |
OpenAI SDKs already retry 429s honouring Retry-After, so short gateway and provider waits self-heal without you writing anything.
Codes refused before a model runs
None of these cost provider spend.
| Code | HTTP | Class | Means | Do |
|---|---|---|---|---|
invalid_api_key | 401 | gateway | No key, or a key we don’t recognise. | Send Authorization: Bearer llmb_i_.... Keys are shown once at issue; if you lost it, rotate rather than hunt for it. |
key_plane_mismatch | 403 | gateway | A real key, on the wrong plane — usually a management key (llmb_m_) sent to /v1/. | Use an inference-scope key (llmb_i_) here. Management keys are for /api/inference/v1/ control routes. |
model_not_found | 404 | router | The model string does not resolve to a task available to your key. | Check GET /v1/models. If you expected a new task to be minted on first use, you may have hit the per-day cap on new drafts — the message says so. On a batch input line, unknown models never mint; fix the line. |
invalid_request_body | 400 | gateway | The body did not parse or a field is out of range. param names the field. One case arrives with class router and param: null instead: a model string longer than 128 characters, which can neither match nor mint. | Read param. Common: an unknown key inside llmbench (the block is strict — objective does not exist, use selection), a bad completion_window. |
unsupported_parameter | 400 | gateway | You asked for a feature this endpoint does not have: tools, tool_choice, functions, function_call, logprobs, audio, modalities, web_search_options, or n other than 1. | Drop the parameter. Benign off-values (tools: [], tool_choice: "none", logprobs: false, modalities: ["text"]) are accepted, so an SDK that always sends them is fine. |
unsupported_value | 400 | gateway | A supported field carried a value v1 cannot serve. | Most often messages: at most one user message, no assistant/tool/function roles. Also llmbench.max_wait_seconds, which is batch-lines-only. |
streaming_not_supported | 400 | router | You sent stream: true. Streaming is not offered — in synchronous or batch mode. | Re-send with stream: false (or omit it). We refuse rather than quietly ignoring the field, so a client written against a streaming API fails visibly instead of appearing to stream. |
credential_mode_conflict | 400 | gateway | You sent an x-llmbench-provider-key-* header, but you already have vaulted credentials. | Pick one mode. Either vault your keys, or send them per request — not both. |
zero_custody_not_enabled | 400 | gateway | You sent a per-request provider key without zero-custody enabled on your account. | Vault the key instead, or ask us to enable zero-custody first. We do not use a credential you were not agreed to send us. |
tenant_rate_limited | 429 | gateway | Your per-second limit on our gateway. | Honour Retry-After and slow down. Not a billing problem. |
tenant_quota_exceeded | 429 | gateway | param names the breached scope. Usually one of your own caps for the window (e.g. calls_day). param: "server_concurrency" is different — that means our gateway is momentarily at its in-flight ceiling, not that you are over a cap. | For your own caps: wait for the window to reset, or raise the limit. For server_concurrency: just honour Retry-After; there is nothing on your side to change. |
internal_error belongs to no band
internal_error (500, or 502 for an unexpected deferred result) carries class router and means something failed on our side. It is listed separately because it is not confined to the pre-invocation stage: it comes from the catch-all around the whole handler, so it can fire after a provider call has already completed. Unlike everything in the table above, it does not guarantee that no provider spend was incurred.
Retry. If it persists, quote the diagnostic id.
The stream: true response, exactly
HTTP/1.1 400 Bad Request
x-llmbench-error-code: streaming_not_supported
x-llmbench-error-class: router
x-llmbench-diagnostic: 7b2e04c9a51f4d3ab8c6e9f012345678
{"error": {
"message": "Streaming is not supported by this Service. Re-send the request with stream=false. (diagnostic: 7b2e04c9a51f4d3ab8c6e9f012345678)",
"type": "invalid_request_error",
"param": "stream",
"code": "streaming_not_supported"
}}
The check runs before a model is selected, so a refused streaming request produces no provider spend and no call record. The same check serves both lanes: on POST /v1/chat/completions it is the response; on a batch input line it becomes that line’s entry in the error file.
Codes from the invocation stage
These come from the invocation stage — either from a model that ran, or from a refusal made once the task had been resolved. Three of them run no model at all, and are marked below.
If a model’s output fails our format checks, or a model returns nothing, we try a different model for that call rather than failing it — unless you pinned one with llmbench.forced_model, in which case there is no alternative and you get the failure directly.
| Code | HTTP | Class | Means | Do |
|---|---|---|---|---|
provider_key_missing | 403 | your_key | No usable credential for the provider that was selected. | Vault one: PUT /api/inference/v1/vault/providers/{slug} with a management key. Partial coverage is fine — one vaulted provider in the pool is enough. |
provider_key_revoked | 403 | your_key | The key was valid and is no longer. | Mint a new one at the provider and re-vault. |
provider_key_invalid | 403 | your_key | The provider rejected the key you vaulted. | Mint a fresh key at the provider and re-PUT it. |
provider_account_suspended | 403 | your_key | The provider rejected a call made on our credentials, or suspended the account behind them. | Nothing for you to do — this is our account, not yours. If it persists, contact support. |
provider_quota_exhausted | 429 | your_quota | Your account at the provider is out of quota. | Raise limits or top up at the provider. This is their quota, not ours. |
insufficient_balance | 429 | your_quota | Runs no model. Not enough prepaid credit for a task we serve on our credentials. The message names the shortfall. | Top up by at least the amount quoted, then resubmit. Nothing was charged and, on a batch, nothing was enrolled. |
billing_not_ready | 503 | router | Runs no model. Your prepaid billing account is still being set up — usually under a minute after signup. Nothing was charged. | Retry after Retry-After (30 seconds). If it persists for more than 15 minutes, contact support. |
provider_rate_limited | 429 | your_quota | Rate-limited upstream, or every eligible model is on a rate-limit cooldown. | Back off and retry — Retry-After is 5 seconds. This clears on its own; it is not a misconfiguration. |
provider_unavailable | 502 | provider | The provider was unreachable or erroring. | Retry with backoff. |
provider_timeout | 504 | provider | The provider did not answer in time. | Retry. If a task times out consistently, consider the batch lane. |
provider_error | 502 | provider | An upstream error we could not classify more precisely. The (redacted) provider message is passed through. | Read the message, then retry. |
content_filtered | 400 | your_request | The provider refused on content grounds, or the input exceeded a length limit. | Change the input. Retrying unchanged will not help. |
output_validation_failed | 500 | router | Either every model we tried produced output that failed the task’s schema or validator, or the output parsed and validated but did not clear the task’s quality bar. The redacted error.message distinguishes the two. | If you pinned forced_model, unpin it and let failover work. For a schema failure, the task’s output schema is likely too strict for the models in its pool. For a quality-bar failure, the knob is the bar or the pool, not the schema. |
model_not_found | 404 | router | The task disappeared between routing and invocation. | Re-check GET /v1/models. |
no_eligible_model | 503 | router | Runs no model. The candidate pool was empty, so nothing was tried. | The message takes one of three shapes — see below. |
capability_config_invalid | 400 | router | Runs no model. The task’s own configuration is wrong — output schema, prompt, or validator. | Fix the task’s configuration on the management plane, or call a different task. |
insufficient_balance also occurs at batch create (POST /v1/batches), refusing the whole batch before any line is enrolled. There it carries class gateway rather than your_quota; the code, the status and the remedy are the same. billing_not_ready occurs at batch create too, with the same code, status, class and remedy.
Re-vaulting a key does not fail on a bad key
PUT /api/inference/v1/vault/providers/{slug} re-runs an auth check against the provider, but a rejected key is not an error: the credential is still stored and you still get a 200. Read the response body instead of the status:
verifiedistrueonly when the check passed.last_errorcarries the provider’s own reason when it did not.
GET /api/inference/v1/vault/providers shows the same two fields for every vaulted credential, so you can confirm a fix without re-sending the key.
What no_eligible_model tells you
The message is one of three:
- A credential message, when the task runs on your own provider credentials and one or more of its providers has no active vaulted key. It names those providers. A revoked key produces this same message as an empty vault, because a revoked credential stops serving.
- A quality-bar message, when a
min_confidencefloor is set on the task and exploration is off, so no model has enough evidence to clear it. It names both settings. - A generic sentence when neither applies, naming nothing further.
Batch and file codes
| Code | HTTP | Means | Do |
|---|---|---|---|
file_not_found | 404 | No such file id, or its content was deleted. | Files are tombstoned on delete, so a deleted id keeps 404ing. Re-upload. |
batch_not_found | 404 | No such batch id for this account. | Cross-account ids 404 by construction. Check the id. |
batch_not_cancellable | 409 | The batch is already in a terminal state: completed, failed, or expired. | Nothing to do. cancelled and cancelling are absent from that list because cancelling an already-cancelling or cancelled batch is idempotent and returns 200. |
too_many_open_batches | 429 | You are at your open-batch cap. Retry-After is 60. | Wait for one to finish, or ask for a higher cap. A cap of 0 means unlimited; an account with no limits row falls back to 5. |
batch_cannot_complete | 400 | The batch cannot finish before its absolute 24-hour expiry. The message names the line count, the lane, the minimum minutes needed, and any of your work already queued ahead of it. | Split the file, or wait for your queued work to drain. |
batch_expired is a per-line code, not an HTTP status
Every batch carries an absolute expires_at of created + 24 hours, independent of completion_window. When it passes, any still-open line becomes an error-file line:
{"id": "batch_req_412", "custom_id": "row-88", "response": null,
"error": {"message": "request did not complete before the batch expired",
"type": "invalid_request_error", "param": null, "code": "batch_expired"}}
You will never see batch_expired as an HTTP response. Read it from the error file named by error_file_id on GET /v1/batches/{id}.
Reading per-line failures
Error-file lines come in two shapes, and the discriminator is whether a status code is known — not when the failure happened.
- A status code is known. The error sits under
response:{"status_code": ..., "request_id": ..., "body": {"error": {...}}}, and top-levelerrorisnull. This covers per-line refusals such as an unknown model orstream: trueon the line (request_idisnullthere, since no call was opened) as well as failures from an invocation. - No status code.
responseisnulland the error sits at top-levelerror. This isbatch_expiredand whole-batch validation failures.
Every line carries custom_id, which is how you map a failure back to your input line.
Line codes drawn from the invocation and gateway stages use the same vocabulary as the tables above. Validation failures do not — see next.
Batch validation uses its own, smaller vocabulary
A structurally malformed line does not fail only that line. Validation runs over the whole input file, and any failure fails the entire batch: status becomes failed, no line is ever launched, and the error file lists the offending lines with id and custom_id both null.
These entries have a different shape from everything else on this page: they carry a line number and no type field.
| Code | Means |
|---|---|
invalid_line | A line is not valid JSON, or its custom_id / method / url / body is wrong or duplicated. line names it. |
heterogeneous_routing | A line routes differently from the first line. Every line in a batch must route identically. param names the disagreeing key. |
empty_file | The input file contains no request lines. |
input_file_unreadable | The input file could not be read — deleted, or a storage error. |
Notes
Not every non-2xx comes from here. The control plane (/api/inference/v1/) uses a different envelope: ninja’s {"detail": ...} for auth failures, and {"detail", "code", "retry_after"} for 429s. The code vocabulary on this page is the /v1/ plane’s.
A 429 is not always billing. tenant_rate_limited means slow down; tenant_quota_exceeded and insufficient_balance mean stop until something changes. The type field separates them for you — rate_limit_error versus insufficient_quota — which is exactly how OpenAI’s own SDKs branch.
Quality headers are absent, never wrong. On a success, x-llmbench-quality-floor / -met / -requested are omitted together if they cannot be resolved. Absent means “not resolvable”, not “not met”. Do not read a missing header as a failed promise.