Onboarding · Open models, your server

You choose a size and an area, and name your model and GPU. We do the rest.

STEP 1

Place an order.

Pick a size. It sets the number of questions and how hard they are. Every size has at least 200 questions, so that the tests can separate the settings.

  • Core200 questions, standard complexity.
  • Advanced400 questions, harder.
  • Expert1,000 questions, hardest.

These counts are representative. We set the exact number for your tasks and sub-tasks. One price, with GPU spend included.

You also give us

Your modelYour GPU familyYour server, or one like it

We control the server, so we use yours or one with the same GPU family. If you cannot change the server, see eval + load test.

STEP 2

Build the questions and the traffic.

There are four ways to create the question set. Choosing your areas is all you have to do. The other three take optional input from you, for when you want a say. You do not have to provide the questions: if the price works for you, we write them all.

Start here

Choose your areas

Pick as many as you like. Each tag is an area or a sub-area, and two related ones work well together.

Filled by RadianVector
The questions, the answers, and how to check them.
Optionally checked by you
All of it, at sign-off.
Optional input from you

Let us look at your product

Filled by RadianVector
The questions, from your site and docs. The answers, and how to check them.
Optionally checked by you
All of it, at sign-off.
Optional input from you

Describe your tasks

Filled by RadianVector
The questions for those tasks, and their answers. We decide how to check each answer.
Optionally checked by you
You can tell us how to check, if you prefer.
Optional input from you

Send your questions or traces

Filled by RadianVector
Any answers you leave empty, and how to check them.
Optionally checked by you
You can flag the answers that matter, or tell us how to check them.

The format: a CSV file

One row per question, for anything you send us. Send as many as you have, and we write the rest. Only question is required. If you leave the answer and how_to_check fields empty, we will fill them in on our end. We send the question set back in the same format.

Download the template
questions.csv
question,answer,how_to_check,area,sub_area
What is the total federal tax?,122175.83,"A number, within $2",Taxes,Investor return
Which clause caps liability here?,,Must name the clause,Law,Contracts
What dose for a 72 kg patient at 5 mg/kg?,360 mg,,Healthcare,Dosing

The traffic

Questions say what to ask. Traffic says how the requests arrive. We build it from your areas, or copy the shape of your traces.

Short requests

Independent questions that share little text.

Long documents

Long prompts that fill the context window.

Agent steps

Each step sends the whole conversation so far.

Batches

Many requests sent at once.

STEP 3 · OPTIONAL INPUT FROM YOU

Sign off, or let our systems decide.

You can read every question with its answer, change a group, and sign off. Or let our systems decide: the run starts once the set has at least 200 questions. If you sign, we record exactly which version you approved.

STEP 4 · NO INPUT FROM YOU

We run them.

We start your model on SGLang, vLLM and TensorRT-LLM, and change the settings that matter. Every viable setup gets the same questions and the same traffic. If we know a setup will do badly, we leave it out.

The controls

Some of what we change on each engine, on top of the weights and the engine version.

SGLang
  • Share of GPU memory
  • Requests at once
  • Prefix cache
  • How waiting requests start
vLLM
  • KV cache size and format
  • Requests at once
  • Speculative decoding
  • Prefix caching
TensorRT-LLM
  • KV cache size
  • Scheduling policy
  • Batch size
  • KV cache reuse

Every start gets read

A server can do something other than what it reports. So we check what each start really did, before we trust its numbers.

Data from our research. TensorRT-LLM ran with placeholder weights, which leave the KV cache and the scheduler as they are. The findings.

Then the load

Each setup gets requests together, batches and KV cache reuse, as on eval + load test. For agents, KV cache reuse often decides the result.

Data from our research: 192 agent tasks, four agents at once, Qwen3.8-27B on vLLM 0.32.0, RTX 4090 GPU family. The bars show the idea, and the totals are measured. The second server had MTP speculative decoding on. The measurements.

The answers have to stay right

A faster setup is no use if the answers change. So we compare every setup's answers with a baseline, and grade them.

Data from our research, on vLLM 0.28. The investigation.

STEP 5 · NO INPUT FROM YOU

Get the report.

We grade every answer, then run the answers and timings through our repos, shown here as rv-1 to rv-n, and our statistical tests, including McNemar, Holm and repetition tests. You get a report of which engine, version and flags to use for which task, and the numbers behind them.

Which setup for which task

SetupKV cache, tokensSolvedSuccessful tasks an hourPrompt tokens from the cacheGPU energy per successful task
No speculation36,253117 of 19275278%1,813 J
MTP, depth 119,727117 of 19233016%4,777 J
n-gram, depth 327,405114 of 19264861%2,306 J

Data from our research: 192 agent tasks, four agents at once, Qwen3.8-27B on vLLM 0.32.0, RTX 4090 GPU family. For these agents, on this model and card, use no speculation. For short, independent requests, MTP at depth 1 was faster: 532 tokens a second against 392, eight at once. The measurements.

Two real examples

What the answer looks like.

The engine, the version and the way requests arrive all move the result.

Example 1

SGLang or vLLM?

It depends on the version, the prompt length and how requests arrive. With 3,000-token prompts SGLang was ahead. With 1,000-token prompts vLLM was.

Output tokens a secondSGLang 0.5.20SGLang 0.5.22vLLM 0.32.0
32 requests, 1,000-token replies, all arriving together1,340 to 1,344805 to 8061,341 to 1,346
32 requests, 1,000-token replies, one arriving first799 to 803801 to 8031,341 to 1,342
32 in flight, 1,000-token prompts664685736 to 737
32 in flight, 3,000-token prompts321325316

Data from our research: Qwen3-8B BF16, the same 32,096-token KV cache, RTX 4090 GPU family. The deep dive.

Example 2

One setting can close a gap.

With 64 requests at once, SGLang 0.5.20 trailed vLLM 0.32.0 by 12%. One setting, which lets SGLang start all 64 together, brought it within 3.5%. At 32 requests, a KV cache with room for every request at full length, 0.90 of GPU memory instead of 0.88, removed the slow runs.

How far SGLang trailed vLLM, 64 requests at once

Default
12 % behind
With the setting
3.5 % behind

Data from our research: Qwen3-8B, RTX 4090 GPU family. The deep dive.