DeepSeek's DeepInfer Numbers Don't Add Up Yet

DeepSeek says its new DeepInfer stack serves 155 trillion tokens a day per cluster, with an estimated inference margin above 80% and an 81-day payback. If those figures held up, they could change the economics of self-managed versus hosted inference. But the figures are first-party, the production service isn't fully reproducible from the release, and the revenue arithmetic doesn't reconcile. ¹

What DeepSeek DeepInfer changed, and why it matters

A Latent Space item published October 21, 2026, describes DeepInfer as DeepSeek's open-source inference engine behind V4.1 Flash. It reports 155 trillion tokens per day per cluster, $22 million in daily revenue, an estimated inference margin above 80%, and an 81-day payback. ¹

For infrastructure leaders, the implication is that self-managed capacity on open models could beat hosted inference outright. That's a hypothesis to test, not a conclusion the report establishes: Latent Space says the production figures are first-party claims, not independently audited measurements. ¹ The useful distinction is between what DeepSeek reports, what the release makes available, and what another team could reproduce. The figures belong on an investigation checklist, not in a capacity-planning spreadsheet.

The inference throughput claims, unpacked

The item attributes several operating figures to DeepSeek: a 96% input-cache hit rate, 400 ms average time to first token, 230 generated tokens per second (up to 300 for coding), 1.5 million cache-inclusive input tokens per second per prefill GPU, and 14,000 output tokens per second per decode GPU across roughly 2,400 GPUs. ¹

These are different measures: cache-hit rate, latency, generation speed, and per-GPU input and output throughput. The input-token figure is explicitly cache-inclusive, while the output figure is reported per decode GPU. ¹ A fair comparison with another provider would need to align workload mix and cache accounting, among other factors the source identifies as relevant to the economics. ¹ A result tuned to one traffic pattern or cache policy may tell you little about your own workload.

The one real comparison point: vLLM

Latent Space describes DeepInfer as a DeepSeek customization of vLLM, with credit to SGLang and TRRTLLM. It reports support for distributed inference, speculative decoding, SWA Bounded Replay, and structured output on NVIDIA and Huawei Ascend platforms. ¹

It also reports a concrete comparison: under a matched configuration, vLLM achieved decode throughput within 6% of DeepInfer's published GB200 results, and reproduction resources were published. ¹ That is evidence that a specific decode configuration can be approached elsewhere. It is not an independent reproduction of DeepSeek's production-scale throughput, revenue, margin, or payback; those remain unverified. ¹

Evidence	What's reported	What it doesn't establish
Production figures	155T tokens/day/cluster, plus latency, cache, and throughput figures 1	An independent audit or a fully reproducible production configuration 1
Deployment guidance	Eight-accelerator machines for prefill; 64–128-accelerator supernodes for decode 1	The exact hardware composition of the production cluster 1
vLLM comparison	Decode throughput within 6% of DeepInfer's published GB200 results under a matched configuration 1	Production-scale economics or a reproduction of the full service 1

A decode benchmark does not establish full-service economics or reproduce the production load mix and cache accounting. The source also says the deployment recommendations don't establish the production cluster's exact hardware composition. ¹ An independent evaluation should account for those distinctions, along with infrastructure and orchestration costs.

What's released in the open source inference engine, and what's held back

The release covers an inference engine and an API-processing layer. The accompanying deepseek-recipe component implements production token encoding and decoding, along with official-API-compatible processing. ¹

That's not the full service. Latent Space says KV-cache management and load balancing weren't fully released because they remain coupled to DeepSeek's internal infrastructure. ¹ Access to the engine alone therefore doesn't provide the same cache behavior or scheduling as the production service. The deployment guidance (eight-accelerator prefill machines and 64–128-accelerator decode supernodes) is a starting point for testing, not a bill of materials for reproducing the throughput claims. The item says it doesn't establish the production cluster's actual hardware makeup. ¹

The revenue math and LLM inference costs don't reconcile

The pricing assumptions cited are $0.15 per million input tokens and $0.60 per million output tokens. Latent Space also cites $38 in revenue per accelerator-hour at off-peak prices against estimated costs of $5–$8 per card-hour, implying an inference margin above 80%. ¹

That cost figure needs a clear boundary before it can be applied elsewhere. The source identifies hardware cost and which expenses are included as sensitivities, but doesn't provide a full cost breakdown in the excerpt. ¹ Without that boundary, the margin figure isn't portable to another organization's cost structure.

There's also an apparent contradiction: the headline figure is $22 million per day, while another passage estimates roughly $1 million per hour. ¹ Multiplying $1 million by 24 hours gives about $24 million, not $22 million. The item doesn't reconcile the difference. ¹

A second mismatch follows from the per-accelerator figures. One reading: apply the reported $38-per-accelerator-hour rate to roughly 2,400 accelerators for an hour, and the result is about $91,200 an hour, not $1 million. This is a calculation from the reported figures; the source doesn't explain the denominator or assumptions behind the larger hourly estimate. ¹ Until that's clarified, neither daily revenue figure should be treated as a forecast input.

The 96% cache-hit rate also matters: the reported input throughput includes cached tokens, and the source identifies cache accounting as a factor in evaluating the economics. ¹ How cached and uncached tokens were treated in the revenue estimate would affect how readers interpret it.

Finally, the item estimates roughly $166 billion per gigawatt and an 81-day payback based on inference margin, against DeepSeek's stated 10-month payback plan. ¹ Those figures depend on assumptions about utilization, workload mix, and costs; they aren't universal benchmarks for self-hosted AI model economics.

What a real reproduction would need

Analysis: A credible independent test needs enough workload and accounting detail for another team to repeat the measurement. Teams evaluating the claims should request:

• Model version, prompt and response distributions, sequence lengths, batch sizes, concurrency, cache state, latency targets, and utilization methodology.
• Exact accelerator types and counts, prefill/decode topology, networking, memory configuration, software versions, and scheduling behavior.
• Raw input and output token counts, a definition of “cache hit,” accelerator-hours, prices used, and a breakdown of costs included in the $5–$8 per card-hour estimate.
• A reconciliation of the $22 million/day and roughly $1 million/hour figures, plus the basis for the $38-per-accelerator-hour number. ¹

These are different tiers of evidence. Reproducing a decode benchmark is not the same as reproducing end-to-end throughput, and neither is the same as auditing revenue and payback. The vLLM result supports only the narrower decode comparison. ¹

What deployment and finance teams should do now

Treat DeepInfer as something to test, not as a forecast to build on. The engine and API layer are released; the KV-cache management and load-balancing components aren't fully released. ¹

Build an internal sensitivity model around workload mix, cache-hit rate, input/output pricing, accelerator cost, and utilization. Compare self-managed inference against hosted alternatives (a model catalog helps you hold the model constant) using the same request distribution, latency target, and cache policy; otherwise, a measured gap may reflect different accounting rather than an infrastructure difference. Other inference-cost claims, such as those around vocabulary specialization for small language models, deserve the same scrutiny.

Set a hard gate: don't approve capacity or capital spend on the strength of the 155-trillion-token, $22-million-per-day, above-80%-margin, or 81-day-payback figures alone. ¹ Reconcile the assumptions, reproduce throughput under your own workload, and validate the economics with an explicit cost model.

The defensible takeaway

Latent Space offers an investigation agenda and one concrete data point: a vLLM decode benchmark within 6% of DeepInfer's published results. It does not provide an independently validated benchmark for DeepSeek's production economics. ¹ DeepInfer may warrant evaluation by teams considering open-source inference stacks. But the public release, deployment guidance, and business claims don't yet establish a repeatable path from hardware to revenue to payback. Until the $22 million/day and roughly $1 million/hour figures are reconciled, treat them as unverified estimates. ¹

Frequently asked questions

Is the 155 trillion tokens-per-day figure independently verified?

No. Latent Space describes it as a first-party production claim that hasn't been independently audited. ¹

What hardware produced the reported throughput?

The item cites roughly 2,400 GPUs in translated production data, but says the disclosed deployment recommendations don't establish the production cluster's actual hardware composition. ¹

Does the DeepInfer release reproduce the full production service?

No. KV-cache management and load-balancing components weren't fully released because they remain coupled to DeepSeek's internal infrastructure. ¹

What does the reported inference capacity cost to run?

The item estimates $5–$8 per card-hour against $38 in revenue per accelerator-hour at off-peak prices, but doesn't provide a full cost breakdown in the excerpt. ¹

Why is the 81-day payback hard to use for planning?

It rests on assumptions about margin, hardware cost, workload mix, and cache accounting. The same item gives $22 million/day and roughly $1 million/hour without reconciling the two. ¹

Sources

¹ “[AINews] DeepSeek DeepInfer: Open Source SOTA Inference running 155T toks/day/cluster making $22m/day at ~80% margin for a 81 day payback,” Latent Space, October 21, 2026. https://www.latent.space/p/ainews-deepseek-deepinfer-open-source