ascend-benchmark-evaluator

To benchmark Ascend C code generation accuracy, you can evaluate generated operators against reference implementations using the NPUKernelBench suite. This Skill automates correctness validation and performance timing to produce per-operator and consolidated benchmark reports.

NPUKernelBench is a standardized dataset format for evaluating Ascend C operators at scale. It enables automated batch evaluation by validating the correctness of generated outputs against references and measuring runtime performance for each operator.

Yes, you need a Python/NPU environment with PyTorch, torch_npu, and the custom_OPP PyBind integration to run Ascend C benchmark evaluations. These dependencies are required to execute operators and validate correctness on the NPU hardware.

You generate per-operator performance reports by running the evaluator against your NPUKernelBench dataset. The Skill automates dataset handling, compares timing results, and outputs detailed individual operator reports alongside a global benchmark overview.

Yes, you can batch evaluate multiple Ascend C operators for correctness and performance. The Skill processes the entire standardized dataset, validating functional parity against reference implementations while measuring runtime to deliver a consolidated benchmark report.

A limitation of using NPUKernelBench for Ascend C evaluation is its strict environment requirement: it specifically depends on torch_npu and custom_OPP PyBind integration, meaning it cannot run on standard CPU or non-Ascend GPU hardware setups.