skills.rest
DeepLink-org
DeepLink-orgOFFICIAL
Explore all skills by this author →
WebsiteGitHub

cambricon-nlp-operator

Benchmarks operator accuracy and latency on Cambricon MLU hardware.

7|1|Updated Apr 17, 2026
Search Tags:#latency#gemm#accuracy validation#operator benchmark#conv2d#cambricon#mlu
One-click install
npx skills add https://github.com/DeepLink-org/DeepEval-Skills --skill cambricon-nlp-operator-deeplink-org
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: cambricon-nlp-operator
Source: https://github.com/DeepLink-org/DeepEval-Skills/tree/main/skills/Cambricon/nlp/cambricon-nlp-operator
Command: npx skills add https://github.com/DeepLink-org/DeepEval-Skills --skill cambricon-nlp-operator-deeplink-org

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires torch, torch_mlu, and includes scripts (resource) components.

What problem does it solve? It automates accuracy validation and performance benchmarking of NLP operators on Cambricon MLU chips, producing standardized Result Contract 2.0 output without manual CSV parsing or result assembly. ## Core Features & Use Cases - Operator Accuracy Validation: Generates CPU FP32 ground truth and validates MLU operator results against it via torch_mlu. - Performance Benchmarking: Measures GEMM, Conv2d (forward and backward components), 40 long-tail operators, and Transformer encoder/decoder block latency in FP16/FP32. - Deterministic Result Collection: Validates run tokens, case coverage, and latency values, then atomically writes a result.json conforming to AIBenchAgent Result Contract 2.0. - Use Case: When an evaluation task specifies chip_type Cambricon and task_type operator, run the GEMM benchmark to get per-case latency and summary metrics (avg, p50, p95, min, max) for MLU hardware. ## Quick Start Ask the agent to run the Cambricon MLU GEMM operator benchmark and collect the Result Contract 2.0 output.

Frequently Asked Questions about cambricon-nlp-operator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I benchmark GEMM operators on Cambricon MLU?▼

Run the GEMM task inside the Cambricon operator container: recompile the gemm_sample binary with make, copy the empty-baseline CSV template, then execute mlu_test_gemm.py for FP16 and FP32. The collect_cases.py script converts results into Result Contract 2.0 format.

What operator benchmarks does the Cambricon MLU skill support?▼

It supports accuracy validation, GEMM, Conv2d forward and backward components (convbackdata, convbackfilter), 40 long-tail operators, and Transformer encoder/decoder block inference latency. Each test_case maps to its own BenchmarkSpec YAML.

Does the Cambricon operator benchmark require a special Docker setup?▼

Yes, it requires a privileged container from the cambricon-nlp-operator image with --shm-size=16g and --ipc=host. Only the results and logs directories should be bind-mounted; mounting /workspace/operators would shadow the image's built-in source code and baselines.

Why does MLU accuracy validation only report FP32 results?▼

The reference ground-truth generator only produces FP32 data, so the validator treats missing FP16 directories as passed. The BenchmarkSpec and collector therefore only accept passed_fp32 as a valid measurement until independent FP16 ground truth is added.

Why does the Cambricon benchmark fail with MLU unavailable errors?▼

Check device visibility with cnmon on the host and torch.mlu.is_available() inside the container. Also verify NEUWARE_HOME, libcnnl.so, and that no ldd output shows 'not found' for the compiled sample binaries.

Related Skills

Explore More Software Engineering Skills→
NVIDIANVIDIA
14.5k

exec-local-docker

Executes TensorRT-LLM workloads in local Docker containers with background monitoring and log-based result reporting.

Official
Advanced
dawsonblockdawsonblock

evaluating-llms-harness

Run lm-evaluation-harness benchmarks across 60+ tasks and export metrics.

Community
Advanced
mindspore-aimindspore-ai
258

triton-ascend-attention

Optimize transformer attention kernels on Ascend using Triton-Ascend DSL.

Official
Advanced
SteveGJonesSteveGJones
41

validate

Invoke ruff, pytest, mypy, and bandit for SDLC validation checks.

Community
Intermediate