Can Smarter SSD Controllers Fix AI's Storage Bottleneck?

ScaleFlux builds SSD and CXL memory controllers. Its AI-optimized SSD platform targets offloading the key-value cache that inference workloads use to store conversational context. ¹ If that approach keeps an accelerator supplied with data, it could change how infrastructure teams weigh storage-side processing against adding memory or compute. Analysis: The SiliconANGLE report provides no comparative benchmark results, so that possibility remains to be tested. ¹

Where the AI storage bottleneck actually sits

ScaleFlux vice president of products and marketing Pichai Balaji put numbers on the gap: GPUs operate at about 20–22 terabytes per second, DRAM is almost 37–40 times slower than what GPUs need, and storage is almost 800 times slower, according to his estimates as reported by SiliconANGLE. ¹ These are attributed estimates, not independently verified benchmarks or a sizing model.

Analysis: The bandwidth comparison should not be read as a requirement for every AI workload to match GPU bandwidth. An accelerator can be underused when data delivery or preparation is a limiting step. Storage bandwidth numbers alone won't establish whether that is happening. Buyers should examine the full GPU data pipeline: cache hits and misses, request latency, queue depth, host-CPU load and accelerator utilization.

Analysis: A hardware change matters only if it addresses the application's actual limiting step. If the accelerator is stalled on cache misses, storage-side offload might help. If computation is the constraint, or the working set already fits in fast memory, a smarter SSD may not help.

What SSD controllers for AI workloads actually do

ScaleFlux builds SSD controllers and CXL memory controllers, pairing them with firmware and a full reference architecture. Its AI-optimized SSD platform targets key-value cache offload, which the report describes as storing conversational context used by inference workloads. ¹

Balaji described NAND as something an intelligent SSD or CXL memory controller can use as a memory device rather than passive storage. ¹ The report does not detail the controller's full processing pipeline, which operations move off the host CPU or GPU, or what work the host still has to do. ¹

That gap matters when comparing this approach with ordinary storage, more DRAM or changes to the inference stack. Before evaluating any system, ask the vendor to specify:

• The exact operation the controller performs, such as placement, lookup, filtering, transformation or compression.
• Which data formats and access patterns it supports.
• How the application feeds and retrieves cache data, and whether that requires changes to cache, storage or inference interfaces.
• What CPU and memory resources the system still consumes after offload.

These are screening questions, not claims about what the controller does. The report identifies key-value cache offload as a target use case, but does not establish that controller-side processing helps AI storage workloads broadly. ¹

Who this helps — and who it doesn't

The clearest reported target is inference that uses a key-value cache to store conversational context. ¹ Analysis: A workload may be worth evaluating if cache access is demonstrably on the critical path, the working set exceeds practical memory capacity, and storage latency or host-side processing is limiting accelerator utilization. These are screening criteria, not reported ScaleFlux results.

Analysis: Workloads dominated by computation, sequential reads or data that already fits in fast memory may have less to gain. SiliconANGLE's report does not provide measured results across inference patterns, cache sizes, read/write mixes, latency targets or accelerator types. ¹ That leaves broad claims about the controllers' value for AI storage unproven.

The report also relays ScaleFlux's claims about NAND placement and endurance: Balaji said the company had demonstrated more than 200 flexible data-placement streams and 7–10 effective drive writes per day on standard triple-level-cell NAND. ¹ The account does not give the test method, workload, duration, drive configuration or comparison baseline. ¹ Request those details before using the figures to size a system or compare it with alternatives. The pattern of a vendor claim without its math is a familiar one; we saw it in a 300× climate-simulation cost claim with missing math.

What to measure before you buy

Start with an apples-to-apples comparison against your current storage path, then, where relevant, against adding DRAM or CXL memory or adding host or accelerator capacity. SiliconANGLE's report provides no comparative measurements of GPU utilization, latency or throughput. ¹

A credible test should report:

• Application results: end-to-end latency, tail latency and throughput.
• Cache behavior: hit and miss rates, including performance during warm-up and eviction.
• Resource use: accelerator utilization, host-CPU load and storage-device utilization.
• Operating conditions: concurrency, request distribution, cache population and read/write mix.
• System configuration: model and context behavior, block or object sizes, firmware version, NAND type, controller count, and PCIe or CXL topology.
• Operational behavior: results under recovery and degraded conditions, and the software changes required to deploy the system.

Analysis: The goal is to connect a change in the data path to the business metric that matters. A throughput number without test conditions does not establish that a controller fixed an application bottleneck.

For the placement-stream and drive-write figures, ask how “streams” and “effective writes” were defined, how the measurements were calculated, and under what workload and duration they were observed. ¹ The report does not provide that methodology. ¹

The real cost of AI infrastructure storage performance

Analysis: The relevant comparison is not SSD price versus GPU price. Model the full set of alternatives: storage and controllers, DRAM or CXL memory, host CPUs, accelerators, networking, power and rack footprint, software integration, operations and replacement cycles.

Ask for a workload-specific cost-per-inference or cost-per-throughput analysis with its assumptions stated. The report includes no product pricing, operating-cost data, deployment requirements or total-cost comparison against added memory or compute. ¹ The economics therefore remain unanswered by the report.

ScaleFlux describes itself as NAND-agnostic and says customers can build around whatever flash they can source. ¹ Analysis: That may widen sourcing options, but the report does not establish which configurations are supported or qualified, what firmware dependencies apply, or how endurance behaves in a specific deployment. Validate those points for the system under consideration. ¹

What to do next

Analysis: Pick a production-shaped inference or cache workload with a bottleneck you can already measure. Record a baseline for accelerator busy time, host-CPU time, cache latency and miss rate, tail latency, throughput, and cost per completed request. Then test the proposed design against the same model, data, concurrency, service target and failure assumptions.

Treat a gain in accelerator utilization as useful only if it also improves the target business metric enough to justify added hardware, software, power and operational load. If controller-side processing shifts work around without reducing end-to-end latency or cost, more memory or compute may be the better option. Teams that consume inference through a hosted API rather than running their own cache hardware can't change this layer; for them, the practical comparison is cost per request across the model catalog.

SiliconANGLE says ScaleFlux sponsored the interview segment and that neither ScaleFlux nor other sponsors had editorial control over the content. ¹ Analysis: That disclosure does not resolve the technical question; treat the interview as a hypothesis to test, not procurement evidence. A similar caution applies to any pitch that arrives without numbers.

Analysis: Proceed only if a reproducible workload test shows a meaningful improvement at acceptable total cost and the operations team can support the deployment. The report makes the storage-side approach worth investigating for cache-heavy inference, but does not establish that it can substitute for more memory or compute. ¹

Quick answers

Can smarter SSD controllers improve GPU utilization?

Possibly, especially for workloads using key-value cache offload for conversational context. ¹ The report contains no comparative GPU-utilization measurements, so validate the effect with an application-level benchmark before buying. ¹

What does ScaleFlux say its AI-optimized SSD platform does?

It targets key-value cache offload. ScaleFlux frames its controllers as enabling NAND to be used as a memory device, but the report does not detail the full set of controller-side operations. ¹

What performance evidence should buyers request?

Request end-to-end and tail latency, throughput, cache behavior, host-CPU use, accelerator utilization, test configuration, baseline comparisons and the methodology behind the endurance and placement figures. The report does not provide comparative test details or that methodology. ¹

Does the report prove SSD controllers are cheaper than more memory or compute?

No. It contains no product pricing, operating-cost analysis or total-cost comparison. ¹ Analysis: Build a workload-specific model that includes hardware, software integration, power, operations and replacement costs before deciding.

Sources

¹ “ScaleFlux targets AI memory bottlenecks with smarter SSD controllers,” SiliconANGLE AI, published 2026-10-14. https://siliconangle.com/2026/10/14/ai-factories-stall-gpus-wait-memory-storage-data-aidatapipelineforum/