Quickstart

The API is OpenAI-compatible. You repoint base_url, swap the key, and the request and response shapes stay the same. Three things differ, and all three are visible in the first call you make:

1. model names a task, not an LLM. We pick the model that serves it.
2. You get two kinds of key, and they are not interchangeable.
3. Streaming is not offered. stream: true is refused with an error, not served buffered.

Before you start

You need an inference-scope API key.

Whether you also need a provider key vaulted with us depends on the task. Every task declares one of two access modes. In operator-supplied mode we call the provider on our own account. In bring-your-own-key mode the call runs on your credentials, and model selection only considers providers you have vaulted an active key for.

The tasks we create for you from an unrecognised model string are bring-your-own-key, so if you plan to use that path — most people do on their first call — vault a key first. See Vault a provider key .

1. Get a key

Keys look like llmb_ + a one-letter scope tag + _ + a 43-character secret.

Both are sent the same way: Authorization: Bearer <key>.

Your account’s first keys are issued when it is provisioned. A key is shown once, at creation. After that, issue more with a management key:

curl -X POST https://dtp.kapualabs.com/api/inference/v1/management/keys \
  -H "Authorization: Bearer llmb_m_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{"scope": "inference", "name": "prod-worker"}'

The plaintext comes back in token and is never readable again — we store only a SHA-256 digest of it. Everywhere else you will see just a hint like llmb_i_…wxyz, which is the first 7 characters and the last 4. That hint cannot authenticate anything.

Three behaviours worth knowing now:

• A key on the wrong plane gets 403 key_plane_mismatch, not a 401. Note that 403 is also what a provider-credential problem returns — read x-llmbench-error-code to tell them apart: key_plane_mismatch is the wrong key, provider_key_missing / provider_key_invalid / provider_key_revoked / provider_account_suspended are your vaulted provider key.
• A token that does not begin llmb_ is rejected without a database lookup.
• Revoking your last active management key is refused, because a hash is one-way and that revoke would lock you out of your own account. Inference keys have no such guard — you can revoke every one of them.

Note that the two planes are separate mounts on the same host:

Your dashboard shows the base URL for your own account. If it differs from the above, use the one your dashboard shows — it is the authority.

2. Vault a provider key

When you send a model string we do not recognise, we create the task for you (see below ). A task created that way runs on your provider credentials — its pricing mode is byok. Model selection only considers providers you have vaulted an active key for, so with an empty vault the candidate pool is empty and the call fails with HTTP 503:

{"error": {
  "message": "no eligible model: 'my-classifier' runs on your own provider credentials (pricing_mode=byok), and this tenant has no active vaulted key for anthropic, openai. Vault one via PUT /api/inference/v1/vault/providers/{slug}. (diagnostic: …)",
  "type": "server_error", "param": null, "code": "no_eligible_model"}}

That is an easy failure to misread as an outage, so vault first. It takes one call, with your management key:

curl -X PUT https://dtp.kapualabs.com/api/inference/v1/vault/providers/openai \
  -H "Authorization: Bearer llmb_m_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{"api_key": "sk-EXAMPLE-not-a-real-key", "config": {}}'

This stores the key, enables that provider for your own-key traffic, and runs an auth-check ping against it. GET /api/inference/v1/vault/providers lists what you have — metadata only, never key material. DELETE on the same path revokes.

Partial coverage is fine. One vaulted provider in the pool is enough to serve; you do not need to vault everything.

3. Make the call

Python

from openai import OpenAI

client = OpenAI(
    base_url="https://dtp.kapualabs.com/v1",
    api_key="llmb_i_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
)

resp = client.chat.completions.create(
    model="support-ticket-classifier",
    messages=[{"role": "user", "content": "My invoice is wrong and nobody replied."}],
)
print(resp.choices[0].message.content)

TypeScript

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://dtp.kapualabs.com/v1",
  apiKey: "llmb_i_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
});

const resp = await client.chat.completions.create({
  model: "support-ticket-classifier",
  messages: [{ role: "user", content: "My invoice is wrong and nobody replied." }],
});
console.log(resp.choices[0].message.content);

curl

curl -i https://dtp.kapualabs.com/v1/chat/completions \
  -H "Authorization: Bearer llmb_i_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{"model": "support-ticket-classifier",
       "messages": [{"role": "user", "content": "My invoice is wrong and nobody replied."}]}'

Use -i. Most of what makes this different from a plain OpenAI call is in the response headers.

What model means

model is a capability slug — a named unit of work you call. It is not a specific model. Which LLM runs is our decision, and it can differ between two calls with the same slug: if a model’s output fails our format checks, or a model returns nothing, we try an alternative for that call rather than failing it.

GET /v1/models lists the slugs you can address, not the underlying LLMs:

curl https://dtp.kapualabs.com/v1/models \
  -H "Authorization: Bearer llmb_i_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

{"object": "list", "data": [
  {"id": "support-ticket-classifier", "object": "model", "created": 0, "owned_by": "organization"},
  {"id": "atomic-fact-claim-extraction", "object": "model", "created": 0, "owned_by": "llmbench"}
]}

owned_by is llmbench for tasks we ship and organization for your own. created is always 0 — the projection carries no creation timestamp, and we would rather return an obvious zero than a fabricated date. The response is not paginated: the SDK’s models.list() gets the catalog in one call.

You do not have to create a task first

If your model string is not recognised, POST /v1/chat/completions resolves it in order: exact slug, then alias, then lazy onboarding, which mints a new draft task named after your slug. The draft is servable immediately. The x-llmbench-routing header tells you which happened:

derived (and an alias minted for you on the spot) requires similarity routing, which is active whenever the routing index has been populated on your deployment. There is no separate switch to throw — populating the index is what turns it on, and a daily job keeps it current. Against an empty index an unrecognised slug goes straight to minted.

Two limits on this: a model string longer than 128 characters is a 400, and there is a daily cap on how many new tasks one account can mint. Exceeding it fails as 404 model_not_found with the message The model '<slug>' does not exist (daily new-model limit reached for this tenant).

Lazy onboarding is deliberately off on the batch lane. An unknown model on a batch input line fails that line with 404 model_not_found, so one typo’d bulk file cannot mass-create thousands of tasks.

To pin one exact LLM instead, send forced_model in the extension block (below). Pinning removes the failover: if that model fails, the call fails.

What comes back

The body is a standard chat.completion with one choice and finish_reason: "stop". Two notes: usage includes prompt_tokens_details.cached_tokens and completion_tokens_details.reasoning_tokens, and the body’s model field echoes the slug you requested — the LLM that actually ran is in the headers.

Every 200 carries these six:

And these when applicable: x-llmbench-request-id (the route id — quote it in support requests), x-llmbench-routing, x-llmbench-band (LOW/MEDIUM/HIGH/RANKED — how much evidence we have for this model on this task), x-llmbench-phase (explore/exploit/continuous), and x-llmbench-ignored-params.

The same values are mirrored into the body under a top-level llmbench key, with the x-llmbench- prefix stripped:

"llmbench": {
  "call-id": "…", "capability": "support-ticket-classifier",
  "model": "…", "provider": "…",
  "service-tier": "standard", "cost-usd": "0.00042100",
  "band": "MEDIUM", "phase": "exploit"
}

OpenAI SDKs ignore unknown body keys, so this is safe to leave in place.

Every failure from /v1/chat/completions and the batch routes carries three headers: x-llmbench-diagnostic (a 32-hex id, also appended to the error message — quote it in support requests), x-llmbench-error-class (whose problem it is: gateway, router, provider, your_key, your_quota, your_request) and x-llmbench-error-code. The model list and retrieve routes return the error body only. A 429 also carries Retry-After, which a retrying client should respect.

Two things that will surprise a repointed SDK

Streaming is refused, not ignored. stream: true returns:

{"error": {
  "message": "Streaming is not supported by this Service. Re-send the request with stream=false. (diagnostic: …)",
  "type": "invalid_request_error", "param": "stream", "code": "streaming_not_supported"}}

with HTTP 400. This is contractual — Terms of Service section 2 says the same thing. We refuse rather than quietly serving a buffered response, so a client written against a streaming API fails visibly instead of appearing to stream. The refusal happens before any model is selected, so it costs no provider spend.

Sampling parameters are accepted and ignored. The task owns its generation parameters, so temperature, top_p, max_tokens, max_completion_tokens, presence_penalty, frequency_penalty, seed, stop, user and logit_bias are accepted and do nothing. We do not hide this: their names come back in x-llmbench-ignored-params. Features we cannot honour are refused with 400 unsupported_parameter when you actually enable them — tools, tool_choice, functions, function_call, logprobs, audio, modalities, web_search_options, and n when it is not 1. (Benign off-values such as tools: [] or logprobs: false are accepted.)

Also v1-only limits on messages: exactly one user message, string content only (no content-parts / vision), and no assistant, tool or function roles. Multiple system messages are fine — they are joined.

The extension block

Everything llm-bench-specific goes in a llmbench object at the root of the body. The OpenAI SDK’s extra_body puts it there for you:

resp = client.chat.completions.create(
    model="support-ticket-classifier",
    messages=[{"role": "user", "content": ticket_text}],
    extra_body={"llmbench": {
        "trace_id": "ticket-88213",
        "selection": {"strategy": "relative", "quality_threshold": 0.9},
    }},
)

The block is strict: an unknown key inside it is a 400, never a silent no-op. The eight fields are version, prompt_vars, variant_values, selection, forced_model, idempotency_key, trace_id and max_wait_seconds (batch input lines only — on the interactive endpoint it is a 400).

If you have seen objective referenced anywhere, it no longer exists; selection replaced it. Note that selection.strategy: "cost" is not the old cheapest, which kept a quality bar — the equivalent of that is strategy: "relative" with a quality_threshold.

Where to go next

• API reference — data plane. The full llmbench and selection field lists, every response header, and the Batch API (/v1/files, /v1/batches) including the two things that differ from OpenAI’s: completion_window is a per-attempt budget that renews when a line is retried on another model, and batch webhook events only fire to endpoints you have registered — with none registered, poll instead.
• API reference — control plane. Vault, key management, task and schema registration, usage and quota, webhook endpoints. Management key throughout.
• Error reference. The full code vocabulary and what to do per x-llmbench-error-class.
• Lifecycle and exploration cost. What x-llmbench-phase and x-llmbench-band mean, and how a task is charged while we are still establishing which model serves it best. Read this before your first invoice: see also the Pricing Terms section 1.2.
• Migrating from direct OpenAI. What changes, what does not, and the behavioural deltas.

The Terms of Service , Privacy Policy and Pricing Terms govern use of the API. Where anything on this page differs from those, those govern.