JETINFER QWEN3.8-27B · PRAGUE, EU

Measurements & specs

Every number here was measured on the production machine, with the conditions that produced it stated next to it.

236 ms

to first token

150,000

context tokens

430 tok/s

under full load

47.6 tok/s

per request

concurrent requests

Throughput and latency

Two profiles, never mixed: what wins on latency loses on throughput. Everything below is the default cheapest profile.

Measured at the real request shape	Result	Conditions
Server throughput	430.5 tok/s	16 concurrent, 6,500-token prompts
Per request, idle	47.6 tok/s	one at a time
Per request, under full load	26.9 tok/s	16 concurrent
Time to first token	236 ms	through the public endpoint
Time to first token, p95	1.18 s	8 concurrent, public endpoint
Context / max output	150,000 / 65,536	tokens

Single RTX 3090, measured 2026-08-22. The first-token rows ran through api.jetinfer.com, so they include the network exactly as you would see it. The throughput rows ran against the gateway on the machine itself, which is why they carry no network time. Going through the edge costs about 34 ms of first-token latency and 3% of throughput, measured.

How to read this against anyone else

We test at 6,500 input tokens. Artificial Analysis uses 1,000 and 10,000. NVIDIA sweeps a range. Vendor marketing often quotes a 128-token prompt, where there is almost no prefill to do. Measured that way, this machine reports 873 tok/s and 90.8 tok/s per request: about 2× what a production prompt returns.

Both are here so you can tell them apart. Prefill runs at roughly 1,810 tokens/second, so first-token time scales with your prompt, unless the prefix is cached. The pricing below is built on that.

Your prompt	Cold first token	With 90% prefix cached
6,500 tokens	3.6 s	0.4 s
16,000 tokens	8.8 s	0.9 s
32,000 tokens	17.7 s	1.8 s

Pricing

Rates live on the OpenRouter provider listing and in /v1/models, generated from one price table. They are not repeated here because they move with the market. What does not move:

What you send	How it bills
Repeated prefix, already cached	one tenth of the input rate
Writing a prefix into the cache	the input rate, no surcharge
A request we failed to serve	nothing

Output is never cached; it is new every time. The opt-in fastest profile is priced separately and higher, because it burns more GPU per token.

The endpoint

You will not call this directly. Buying through OpenRouter means your integration is with them — their base URL, their key, JetInfer selected as the provider. What they route to is a plain OpenAI-compatible surface: streaming and logprobs both work. Only about a quarter of endpoints return logprobs at all.

# What OpenRouter routes to. Ordinary OpenAI SDK, no special client.
client = OpenAI(
    base_url="https://api.jetinfer.com/v1",
    api_key="sk-...",
)

response = client.chat.completions.create(
    model="qwen3.8-27b",
    messages=[{"role": "user", "content": "Summarize this changelog."}],
    stream=True,
)

Supported

Streaming with usage totals in both modes, logprobs and top_logprobs, seed, stop sequences, temperature / top_p / top_k, frequency and presence penalties.

Tool use and structured output

Verified live on 21 August 2026. tool_choice:"auto" returns finish_reason:"tool_calls" with a valid name and JSON arguments, streaming included. response_format returns an object matching your schema. These are exercised against the public endpoint on every deploy, not declared and assumed.

Reasoning traces

Qwen3.8 thinking output is returned in the reasoning field. It is never merged into content, and never billed invisibly.

What you are depending on

One deployment, one RTX 3090, in Prague. EU data residency, verified from the machine. The limits below are the ones that will bind you first.

Capacity

12 concurrent requests for general traffic, 4 held for platform probes. 600 requests per minute per routing key, raised on request. At saturation, an immediate 429. Never a queue, never billed for the wait.

int4 weights, declared

int4 weights (W4A16), declared as "quantization": "int4" in /v1/models so it is filterable. Most endpoints on this model run fp8 or bf16. int4 is what makes the price possible.

Context limit

150,000 tokens, which is what this machine serves. Peers advertise more. Requests above the limit are rejected, never truncated to fit.

When it breaks

Health checks every 15 seconds with a real completion. A failed worker is dropped immediately. The watchdog restarts the engine, then stops after four restarts in three hours: that is broken, not unlucky.

No uptime SLA. No multi-region redundancy. No third-party monitor. Replacing a dead machine is manual today. If you need a committed SLA, write to us before you build on this.

Data handling

Prompts and completions are never written to disk, logs, or analytics. We keep operational metadata for 13 months: request ID, key identifier, token counts, latency, status. Nothing is used for training, fine-tuning, or evaluation. GPUs are currently in the Czech Republic. Full terms in the privacy policy; DPA on request.

0prompts stored

int4declared quantization

13 mometadata retention

CZdatacenter region

Questions

About anything on this page: [email protected].