Production inference for open models. OpenAI-compatible API with streaming, tool calling, and logprobs. $0.34 per million input tokens, $2.55 per million output — and cached input at one tenth of that.
| Measurement | Result | Conditions |
|---|---|---|
| Time to first token | 131 ms | single request, short prompt |
| Per-request decode | 90.8 tok/s | one request at a time, speculative decoding |
| Per-request decode | ~27 tok/s | 32 concurrent requests |
| Server throughput | 873 tok/s | sum across 32 concurrent requests |
| GSM8K accuracy | 95.0% | 200 questions, greedy |
These are our own measurements, reproducible on the same hardware. Per-request speed falls as concurrency rises — that is true of every provider, and it is why we publish both numbers rather than the flattering one. Independent per-provider throughput and latency for this model are measured continuously by OpenRouter from live traffic; compare us there once we are listed, not here.
Point your existing OpenAI client at our endpoint. Streaming, function calling, structured output, and logprobs are all supported — the last of which roughly three quarters of endpoints do not return.
# Works with the OpenAI SDK you already have installed. client = OpenAI( base_url="https://api.jetinfer.com/v1", api_key="jf-...", ) response = client.chat.completions.create( model="qwen3.8-27b", messages=[{"role": "user", "content": "Summarize this changelog."}], stream=True, )
| Per million tokens | JetInfer | Market range |
|---|---|---|
| Input | $0.34 | $0.40 – 0.58 |
| Output | $2.55 | $3.00 – 3.45 |
| Cached input | $0.034 | $0.04 – 0.12 |
Cached input is billed at one tenth of the input rate. For agent and coding workloads, where the great majority of each prompt is a repeated prefix, that rate determines the bill more than the headline price does.
We serve int4 weights. That is more aggressive than the fp8 and bf16 endpoints you will see listed beside us, and it is the reason the pricing works. It is also the right thing to be sceptical about, so we publish evidence rather than assurances.
GSM8K: 95.0% across 200 questions, greedy decoding, measured on
this deployment. Speculative decoding is lossless by construction — it
samples the same distribution as decoding without it. The quantization level is
declared in our /v1/models metadata rather than left for you to
discover, and we publish every recovery figure we measure.
Prompts and completions are never written to disk, logs, or analytics. We retain only operational metadata — request ID, token counts, latency, and status — for billing and debugging. No customer data is used for training.
The endpoint is in final validation. Tell us what you intend to run on it and we will issue a key with trial credit when capacity comes online.
Request a key