vLLM Semantic Router’s Decision 3.0: Benchmarked, Not Yet Proven

vLLM Semantic Router has released Decision 3.0, a family of five open decision models that take text, JSON, and image inputs and return typed answers (choices, yes/no responses, or scores) with probabilities. That output shape makes the models plausible candidates for AI model routing, moderation, and agent-control systems. The reported benchmarks and single-request latency figures do not establish how well they will perform on a particular production workload. ¹

What vLLM Semantic Router Decision 3.0 changes

Decision 3.0 accepts text, JSON, and images, and answers named questions with typed outputs and probabilities. Multiple questions can be sent in one call. The models return decision outputs rather than the free-form prose expected from a general-purpose generator. ¹

The five Apache-2.0 models range from d3-lite (0.85B parameters) to d3 (26.09B): d3-nano (2.21B), d3-mini (4.54B), and d3-flash (8.39B) sit between them. ¹

That shape suits control points where a system needs a bounded answer, such as selecting a model, assigning a moderation label, scoring an image, or deciding whether an agent should proceed. A typed result may be easier to validate downstream than text that must first be parsed. That is an architectural reason to test Decision 3.0, not evidence that it will make better decisions than a team's current system.

A component designed to answer decision-shaped questions is not necessarily proven on a buyer's decision task. The source reports benchmark scores and single-request latency, but not production accuracy, throughput under concurrency, operating cost, or integration behavior on a buyer's workload. ¹

Where a multimodal decision model could fit

AI model routing. A system could ask which model or workflow from its model catalog should handle an input, then apply its own thresholds or fallback logic to the returned probabilities. The reported results do not test Decision 3.0 against a buyer-specific routing policy. ¹

Moderation. The model family may be worth evaluating for systems that examine text and images, but reported vision results vary by task: d3 scores 97.3 on InfographicVQA and 89.9 on Mind2Web, compared with 46.2 on MMMU-Pro vision and 44.6 on Hateful Memes moderation. ¹ These results come from different tasks, not a single moderation score. Citing the strongest result without the weaker one would give an incomplete picture.

Agent control. Teams might use a model to check whether an input meets a condition, whether an action is permitted, or which tool to select. Typed outputs can simplify the handoff to policy-enforcing code. They do not resolve how a system should handle ambiguous inputs, wrong answers, or missing fallback paths. For a broader view of what evidence is enough before an agent acts, see AI agent governance and evidence before action.

Image scoring. Every model in the family has a vision encoder. Requests can include multiple PNG, JPEG, or WebP images, each read at up to 1.6 megapixels; every question sees all images in the request. ¹ Check that behavior against your own data model before assuming it fits your request patterns.

How Decision 3.0 compares with the reported alternatives

There is no single winner across the reported comparisons. The figures below come from different evaluation setups (text, vision, a public suite, and a comparison with an earlier generation), so they should not be combined into one ranking. The model-card figures use the team's internal Jev Decision Index 0.3.1; the source also reports public-suite and approximate-rebuild results. ¹

Comparison	Result
Text	d3: 64.1; Perplexity Decider v1.1: 62.8; Jev: 60.1 1
Public suite	Torchcast Decision 27B: 65.1; d3: 64.9 1
Vision	d3: 71.6; Perplexity Decider v1.1: 70.6 1
Generation-over-generation	d3-flash: 59.0; Decision 2.0 (27B): 55.9 1

The largest model is competitive in some reported comparisons, but results on a benchmark are not results on your policy and data. The smaller models do not lead every reported comparison: d3-lite leads its text table but ranks third on vision, while d3-nano leads its vision table but trails LiquidAI d1-3B on text. ¹ Parameter count alone does not establish the best accuracy-cost tradeoff for a particular use.

Treat scores from different tasks and evaluation methods as a way to shortlist candidates, not as a unified scoreboard. They can help identify models to test; they cannot establish whether a model improves the decision your business needs to make. The reported tables do not measure your routing policy, image mix, queueing, serving cost, or fallback behavior. ¹

Decision 3.0 latency: what the numbers say, and what they do not

On one AMD Instinct MI325X GPU, the reported median single-request latencies were: ¹

Model	Text latency	Image latency
d3-nano	21.7 ms 1	92.0 ms 1
d3-lite	23.5 ms 1	60.7 ms 1
d3-mini	28.5 ms 1	115.0 ms 1
d3-flash	29.7 ms 1	171.8 ms 1
d3	84 ms 1	342 ms 1

Among these results, d3-lite has the lowest image latency and d3-nano the lowest text latency. ¹ Treat them as a starting point for a shortlist, not a production guarantee.

The source says each question triggers its own forward pass, even when multiple questions are bundled into one request. ¹ That means multi-question requests may have a different latency and cost profile from single-question requests. Benchmark the question counts you actually expect to use rather than extrapolating from a generic request.

These are single-request medians measured on one GPU. The source does not report tail latency, throughput under concurrency, batching, queueing, image preprocessing time, performance on other accelerators, or operating cost. ¹ Teams evaluating cost and performance will need to measure those for their own deployment.

What to test before adopting an open decision model

Choose a bounded workflow where the desired output is naturally a choice, a yes/no response, or a score. Build a labeled evaluation set from your own data, including representative text and JSON, the image types you encounter, ambiguous cases, and known failure modes.

Run Decision 3.0 against your current component and a practical alternative using the same inputs and decision policy. Measure what matters to the business: false accepts, false rejects, the cost of each error type, and whether returned probabilities are useful at your intended thresholds. The source reports probabilities but does not establish their calibration on your data. ¹

Test latency and throughput at expected concurrency with both single-question and multi-question requests. Record tail latency and retry rates alongside medians. Because each question triggers its own forward pass, multi-question load deserves direct measurement rather than extrapolation from a single-question result. ¹ If the decision component sits inside an unattended workflow, testing for retries that cost money and outputs that vary applies here too.

Estimate cost per decision and per completed workflow, accounting for questions per request, image handling, GPU utilization, evaluation traffic, and fallback calls. The reported latency table is a starting point for test design, not a cost result. ¹

Finally, assess the fit with surrounding systems, including thresholds, routing rules, agent controls, audit logs, access controls, and fallback paths. The source lists transformers==5.17.0 as an installation target and says model loading needs trust_remote_code=True; platform teams should assess those requirements against their deployment and security review processes. ¹

What’s still unknown

The source does not disclose a context-length figure or list official quantized variants. ¹ Those are open questions for memory planning and input-size budgeting.

The models are Qwen fine-tunes: d3 is based on Qwen3.8-27B, and the four smaller models use Qwen3.5 checkpoints. The source says all were trained on AMD Instinct MI325X GPUs. ¹ That describes model lineage and training hardware, not performance on your hardware or in your domain.

The reported benchmark set mixes internal evaluations with public-suite and approximate-rebuild numbers. ¹ Before using the figures in a procurement decision, check whether the task and scoring method match the decision you need to make.

What to do next

Treat Decision 3.0 as a candidate for narrow, typed decisions, not as a general replacement for generative models. Compare candidate sizes with your current decision component under identical inputs, thresholds, concurrency, and fallback conditions. ¹

Set acceptance criteria before testing: task-level accuracy, useful probability calibration, latency at realistic load, cost per decision, and integration reliability. Keep the existing fallback in place while testing edge cases and production-like traffic. One reading is that hard constraints such as region, data policy, and allowed providers belong in an enterprise AI gateway layer rather than in a model's probabilities. The reported results provide a reason to evaluate the models, not a reason to skip that evaluation.