Can Agents Make Custom Kernels for Ascend NPUs?

CANNBot-CAKE reports that its generated operators passed all 1,060 standard-set cases and 4,167 of 4,240 hidden-set cases, with an average 1.334× speedup over a baseline ¹. That is a promising benchmark result for teams building on Ascend NPUs. The abstract reports pass rates and speedups, but no engineer-hours, so it does not establish that the approach cuts engineering time or cost ¹.

Those are different outcomes. A system can generate correct, faster operators without reducing the human work needed to ship them. Teams evaluating CAKE should treat it as a kernel-evolution proposal to test against their own baseline, not as a demonstrated productivity gain.

What CANNBot-CAKE actually does

CAKE is described as a CANN-native system for developing deployable custom operators on Ascend NPUs ¹. The abstract says that developing high-performance Ascend kernels remains slow, expert-intensive, and difficult to automate, and that current models remain less capable at generating AscendC kernels for NPUs than agents are at approaching expert performance on GPU kernel benchmarks ¹.

CAKE combines adaptive multi-DSL routing, separate Generation and Evolution modes, reviewed knowledge reuse through CAKE Book, and a Self-Evolving Agent ¹. This architecture suggests a workflow that combines kernel generation, evolution, and reuse of reviewed knowledge; the abstract does not give further operational detail ¹.

That architecture is a plausible response to the stated problem. Whether it saves expert time is a separate question: the abstract provides no engineer-time results ¹.

What the evaluation measured

The abstract reports an evaluation of 53 operators from CANN-Bench v1.1 on an Ascend 950PR ¹:

Evaluation slice	Reported result
Standard set	1,060 / 1,060 cases passed (100%) 1
Hidden set	4,167 / 4,240 cases passed (98.28%) 1
Full evaluation	Mean score 78.63/100; average speedup 1.334× over baseline 1

These results show that CAKE passed nearly all reported cases and achieved an average speedup over the baseline used in the evaluation ¹. They do not measure development time or debugging effort. A pass rate reports benchmark outcomes, not how much human work was required to obtain them.

How solid is the kernel speedup?

The reported 1.334× figure compares CAKE with a baseline ¹. The abstract does not identify the baseline implementation or describe the measurement procedure, so it is not enough to predict how the result would transfer to another team's kernels or hardware setup ¹. A single average against an unnamed baseline deserves the same care as other headline performance claims, such as those we examined in DeepSeek's DeepInfer numbers.

The abstract also reports that, on EngramGateFusion, 20 rounds of evolution reduced aggregate device time by 46.6% and roughly halved the mean roofline gap ¹. That is a notable result for the reported operator. It is one operator-level result, however, and does not establish that other kernels will see similar gains ¹.

What's missing from the abstract: a comparison against hand-optimized kernels or a competing kernel-generation system. The abstract reports a comparison with a baseline but does not identify it ¹. For teams that already use specialist optimization, comparing against their current approach would help answer whether CAKE improves that workflow.

Three questions, one unanswered

• Does it produce operators that pass tests? The reported pass rates say yes for the evaluated cases ¹.
• Does it improve device performance? The abstract reports an average speedup over baseline and a device-time reduction for EngramGateFusion ¹.
• Does it reduce the time or expertise needed to get there? The abstract reports no such measurement ¹.

Agents may reduce manual coding while adding work elsewhere, such as specifying tasks, reviewing generated code, diagnosing failures, and checking performance outside the benchmark environment. Those are considerations for an evaluation, not outcomes measured in the abstract. Teams should track human effort directly rather than infer it from benchmark results.

What the benchmark doesn't tell you

Before comparing the reported speedup with another result, a team would need details such as the baseline implementation, timing methodology, and compiler, runtime, and hardware configuration. The abstract does not provide those details ¹. It reports the mean score and roofline-gap result but does not define how either is calculated ¹.

The abstract also does not describe how the 53 CANN-Bench v1.1 operators are distributed across workload types or how representative they are of production workloads ¹. It does not report run-to-run variance or results on other Ascend NPU models in the supplied excerpt ¹.

The abstract does not report a hand-optimized comparison ¹. Without that comparison, the results here do not show whether CAKE replaces specialist kernel optimization, speeds it up, or adds a parallel workflow.

A validation checklist for Ascend kernel development teams

Reproduce the comparison conditions as far as the available details allow. Record the Ascend hardware, CANN version, compiler and runtime settings, operator inputs, timing method, and baseline implementation. The supplied abstract does not provide these details, which limits direct reproduction from the excerpt alone ¹.

Then pilot on a mix of benchmark operators and actual production bottlenecks. Track pass rate and device time, as well as compilation failures, validation defects, and how many agent iterations it takes to reach a deployable result. Compare against existing hand-optimized operators and repeat runs to assess variability.

Also log human time: setup, generation, debugging, review, benchmarking, and integration. That is the evidence missing from the abstract, and it is needed to determine whether CAKE reduces expert effort rather than only producing benchmark results ¹.

Set deployment gates before the pilot starts, including correctness, performance against your own baseline, repeatability, and reviewability. Treat a high benchmark pass rate as evidence about those benchmark cases, not by itself as a deployment decision. The same discipline applies to any accelerator tooling; our look at TornadoVM 7 as a Java route to GPU acceleration takes a similar evaluate-first stance.

Bottom line: strong benchmark results, no cost evidence yet

CAKE is a research proposal for agent-based kernel evolution, with results reported on 53 CANN-Bench operators on an Ascend 950PR ¹. The reported 100% standard-set pass rate, 98.28% hidden-set pass rate, 78.63 mean score, and 1.334× average speedup demonstrate benchmark performance on that evaluation ¹. The abstract does not demonstrate lower total cost or faster time-to-production ¹.

A useful next comparison would test CAKE against a team's own hand-optimized kernels while tracking engineer-hours on both sides. The abstract does not report that comparison ¹. Until such evidence is available, the benchmark results support a claim about performance on the reported evaluation, not a claim that agents make Ascend kernel development cheaper. Cost questions also extend beyond kernels to the surrounding stack, as we argue in why elastic GPU capacity won't lower agent costs by itself.