metax-nlp-operator

Benchmarks GEMM, Conv2d, long-tail operators, and Transformer blocks on MetaX GPUs.

7|1|Updated Apr 17, 2026
One-click install
npx skills add https://github.com/DeepLink-org/DeepEval-Skills --skill metax-nlp-operator-deeplink-org
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: metax-nlp-operator
Source: https://github.com/DeepLink-org/DeepEval-Skills/tree/main/skills/MetaX/nlp/metax-nlp-operator
Command: npx skills add https://github.com/DeepLink-org/DeepEval-Skills --skill metax-nlp-operator-deeplink-org

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires torch, pandas, and includes scripts (resource) components.

What problem does it solve? Evaluating operator-level accuracy and performance on MetaX (MACA) GPUs requires coordinating container setup, environment verification, compilation, baseline generation, and result collection. This Skill automates that entire workflow so engineers can produce consistent, contract-compliant benchmark results without manual orchestration. ## Core Features & Use Cases - Operator Accuracy Validation: Verifies MetaX forward outputs, input gradients, and parameter gradients against image-bundled CPU ground truth, producing CSV/JSON pass results. - Performance Benchmarking: Runs GEMM and Conv2d (FP16/FP32), LongTail-Bench operators, and Transformer encoder/decoder block latency tests, writing measured baselines in milliseconds. - Result Contract 2.0 Collection: Uses runner and collector scripts to validate workload fingerprints, case identities, and latency values, then atomically generates a compliant result.json with summary statistics (avg, P50, P95, min, max). - Use Case: An engineer needs to validate a new MetaX driver release. They trigger the Skill to launch the benchmark container, run all GEMM and Conv2d cases, and receive a verified result.json with per-case latencies and accuracy pass rates. ## Quick Start Ask the agent to run the MetaX operator benchmark for GEMM and Conv2d on the available MetaX GPU and collect the results into result.json.

Frequently Asked Questions about metax-nlp-operator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I benchmark GEMM and Conv2d operators on MetaX GPUs?

Launch the provided Docker container with /dev/mxcd and /dev/dri devices, compile the cuda_ops binaries with CMake, then run test_gemm.py and test_conv.py against the FP16 and FP32 parameter CSVs. Measured latencies are written into the baseline column of result copies in milliseconds.

How to validate operator accuracy on MetaX MACA platform?

Run the run_accuracy.py script, which compares MetaX forward outputs, input gradients, and parameter gradients against the CPU ground truth bundled in the image. It produces mx_val_result.csv and mx_val_result.json with aggregated pass status per operator.

What hardware and environment does MetaX operator benchmarking require?

At least one MetaX GPU with matching host drivers exposing /dev/mxcd and /dev/dri is required. The container image must include /opt/maca, the CUDA compatibility layer, MetaX PyTorch, mcBLAS, mcDNN, CMake, and pandas, and versions must be compatible with the host driver.

Does the Transformer Block test measure full training performance?

No. The Transformer Block benchmark runs FP32 inference forward passes only for encoder and decoder layers, without backward propagation. It reports per-iteration latency_ms and should not be described as complete training performance.

Why does the result collection fail with fingerprint or token errors?

The collector rejects results when the AIBENCH_WORKLOAD_FINGERPRINT environment variable is missing, stale, or mismatched with the run token embedded in output CSVs. Ensure the agent injects the identity variables and that runners and collector execute within the same workload run.

What are the limitations of MetaX FP16 accuracy validation?

CPU FP16 support is incomplete, so missing FP16 ground truth directories may cause FP16 cases to be reported as passed without real verification. Always inspect the reference directory and error logs before claiming FP16 accuracy is validated.