Intelligence, Measured.

Build a better AI system for your workload. Better intelligence. Faster responses. Lower costs. Higher energy efficiency.

RadianVector measures and optimizes entire AI systems. We find the smartest AI setup for each task.

Powered by proprietary optimization technology and original research, RadianVector systematically explores various possibilities to identify and validate the right setup for your workloads - with research-grade rigor.

We test and optimize offline, without slowing down your production pipeline. Share only what you choose - no sensitive production data or live traces required.

Frontier Models Open Models
your workload your workload · model APIs your model · hosted GPUs behavior latency cost quality throughput GPU memory Intelligence, Measured AI MODELS AND AGENTS Frontier Models Open Models FRONTIER MODELS Claude · GPT · Gemini · Grok support → use model A claims → use model B regulatory → AI drafts, human checks OPEN MODELS vLLM · SGLang · TensorRT-LLM vllm · fp8 weights fp8 KV cache max_num_seqs 32
Model Explorer

Options: 249 frontier · 1,968 open-weight

Keep your production traffic private

Teams changing their workflows

Deploying AI is not
deploying software.

Box CEO Aaron Levie put it plainly: changing workflows for AI takes far more work than anyone expects. Workflows need re-engineering for agents, evaluations need to be built and maintained, and the whole system needs updating every time a new model arrives.

If your organization is in the middle of that change, talk to us. Measuring what works, and checking it still works after every model update, is what RadianVector tools are built for.

LinkedIn post by Aaron Levie, CEO of Box: There's a huge opportunity right now in being the deployment layer for AI into the economy. The amount of work it takes to change out workflows in enterprises tends to be far greater than anyone realizes or would prefer.
Aaron Levie, CEO of Box, on LinkedIn ↗
Frontier models

Make an informed choice.

Choose a better model for your task.

Prices are for standard-length prompts. OpenAI, xAI and some Google and Anthropic models charge more for long prompts, above 100K to 272K input tokens depending on the model. See the long-prompt prices →

See every model, setting and price on the Model options page →

YOUR SYSTEMS agent agent agent repeated calls open model on your GPUs your data
Open-weight models

Open models,
inside your own systems.

For some use cases, open-weight models work best. If your agents call a model over and over, or all your data has to stay inside your own systems, running open models is now practical, and they are capable enough for many of these tasks.

We measure and tune how open models are served, on your hardware and your workload.

  • vLLM · SGLang · TensorRT-LLM
  • Quantization
  • KV-cache precision
  • Speculative decoding
  • Batching & concurrency
  • KV-cache offload & preemption
  • Temperature · top-k · top-p
  • Reasoning budget

See every setting we tune →   Our Qwen benchmark →

Original RadianVector results

Frontier Model analysis.

We ran our own questions on models from Anthropic, OpenAI and xAI, through each vendor’s own command-line tool, and evaluated the answers. We have ten findings from October 2026, and here are a few of them.

Finding 02

Max effort: 3.3× the tokens, same answer.

On a 50-step calculation, medium, high and max effort all got it right. Max used 24,875 tokens, against 7,584 for medium.

Tested: Claude Opus 5.5, max, high and medium effort · Oct 2026 · Read finding 2 →

Finding 01

Retrying an overflow overflows again.

Opus hit its 128,000-token output limit before answering. Our test runner retried it three times, and each retry hit the limit again. After 514,985 tokens, there was still no answer.

Tested: Claude Opus 5.5, max effort · Oct 2026 · Read finding 1 →

Finding 04

A tool beat the top reasoning setting.

On a 700-step calculation at its highest setting, GPT-6 Astra got it right with its built-in tool, and wrong without it.

Tested: GPT-6 Astra, xhigh effort · Codex (0.154.0) · Oct 2026 · Read finding 4 →

Read all ten findings → See the results tables →

Original RadianVector results

Open Model analysis.

Measured answer quality, request timings and serving behavior, including a tested vLLM fix.

Same model, three inference engines

SGLang vs vLLM vs TensorRT-LLM.

The same setup, run more than once, gave SGLang two speeds. When all the requests reached it together, it kept pace with vLLM and TensorRT-LLM. When one request arrived a few milliseconds early, SGLang made 3 requests wait, and total throughput fell by 40%.

Swipe to see all columns.

Qwen3-8B BF16 · RTX 4090 GPU family · 16,384-token context · at most 32 running · same load from vllm bench serve
MeasurementvLLM 0.30SGLang 0.5.20TensorRT-LLM 1.3.0rc29
KV pool32,336 tokens
47,888 on a second start		32,096 tokens37,760 tokens
Most running at once323232
Output tokens per second1,2741,3421,389

Count what ran, not only the speed. The SGLang column shows a fast run. With the same load, the same SGLang server sometimes ran only 29 requests, kept the rest waiting, and produced 799 output tokens/s. Across repeat runs, fast runs gave 1,337–1,356 and slow runs 795–811. We first saw the difference between two benchmark clients and traced it to how SGLang’s scheduler starts requests. SGLang vs vLLM, the deep dive →

See the full engine comparison and run details →
Answer quality

Open models, with thinking on.

Results at levels 5 and 6 of our difficulty ladder, on the RTX 4090 GPU family.

Swipe to see all columns.

Thinking on · levels 5 and 6 · 28 questions per level
ModelLevel 5Level 6
gpt-oss-20b, high effort25/2822/28
Qwen3.8-27B, 4-bit21/2821/28
Qwen3-8B, 4-bit18/2810/28
GLM-4.7-Flash, 4-bit14/286/28

How the difficulty ladder works →

Response time

What happens as requests increase.

The tool report separates time spent waiting from time spent generating.

Swipe to see all columns.

27B model · vLLM 0.28 · no speculative decoding · all times in seconds
Requests
open		RunningWait for first token
median		Time to completion
mean		Time generating
median
110.054.84.8
220.135.35.2
440.115.55.4
880.115.65.5
16103.18.85.8
321011.816.65.8

At 8 open requests, mean completion time was 5.6 s. At 32, it was 16.6 s, while median generation time changed only from 5.5 to 5.8 s. Only 10 of the 32 requests were running; the others were waiting in line.

See all timing measurements →
Answer correctness

A vLLM bug, before and after the fix.

MTP speculative decoding repeated tokens under load. We traced the cause to the model runner, patched it and reran the tests.

Swipe to see all columns.

Qwen3.8-27B W4A16 · RTX 4090 GPU family · WSL2 · V1 model runner · 385 replies per run · 8 concurrent requests · async scheduling and prefix caching on
ConfigurationDifferent replies
before → after		Repeated-token replies
before → after
vLLM 0.28 · MTP-160 → 1129 → 0
vLLM 0.28 · MTP-377 → 626 → 0
vLLM 0.30 · MTP-139 → 813 → 0

Repeated-token replies fell to zero in these patched runs. “Different replies” counts token-for-token differences from the no-speculation run; it is not a correctness score.

Read the fix and full results →
Two paths

Offline AI model evaluation. Hosted and open models.

Offline evaluation compares AI configurations outside your live production pipeline. RadianVector tools support hosted model APIs and self-hosted open models, so you can measure answer quality, response time and cost before deploying a change.

Frontier Models

Compare hosted models and API settings.

Which model and prompt combination meets your requirements? Where does reasoning effort help? What changes when the output format or supplied context changes?

For engineering teams improving hosted-model applications, retrieval workflows and agents.

  • questions
  • reasoning effort
  • prompt sensitivity
  • tool use
  • severe errors
  • latency
  • cost
  • human review
Explore hosted-model evaluation
Open Models

Measure answers and the serving stack.

Where do requests queue? Does quantization change the answers? How do cache settings and concurrency affect latency, throughput and GPU use?

For teams serving open-weight models with vLLM, SGLang, TensorRT-LLM or llama.cpp on their own infrastructure.

  • model
  • runtime
  • quantization
  • KV cache
  • concurrency
  • decoding
  • throughput
  • latency
  • GPU utilization
  • cost
Explore open-model evaluation
What RadianVector measures

See beneath the headline score.

RadianVector compares performance and behavior on the same workload. See which tasks each configuration completes, where it fails and how much time and compute it uses.

Explore the engineering behind the measurements →
Workload-specific evaluation

Which tasks does the system get right?

Compare task results against your requirements. See successful completions, important errors and the tasks each configuration struggles with.

Controlled runs & effort analysis

Where does extra computation help?

Compare models, prompts and supported reasoning settings on the same questions. Measure where a change improves answers and what it costs in time and money.

inferlint · Open source

What happened inside the serving stack?

Use inferlint to inspect request queues, memory use and concurrency, and understand what your inference server actually did.

Inspect inferlint and an example report →
Regression & change analysis

Did the change improve the system?

Test before and after a change. See what improved, what regressed and how response time and cost shifted.

Study design

Example problem / task: extracting facts from research documents.

A study starts from 1,000 representative questions about the documents. It screens 10 model and reasoning-level settings with 5 prompt variations, then tests output formats and sampling settings on the promising candidates.

Baseline

Measure the current setup on representative questions.

Screen

Narrow the search using the quality and cost requirements.

Validate

Test the finalists on held-out cases and repeat runs to check stability.

Deliver

Provide configurations, measured trade-offs and failure examples.

Explore offline evaluation for hosted models →
Start a conversation

Want to understand your AI better?
Let’s measure it.

[email protected]