shmem-ops-testcase-gen

Generates correctness test plans, case matrices, golden data, and checker scripts for SHMEM operators.

Updated Sep 15, 2026
One-click install
npx skills add https://github.com/WangWindow/CANN-BatchMatMulMaxsum --skill shmem-ops-testcase-gen-wangwindow
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: shmem-ops-testcase-gen
Source: https://github.com/WangWindow/CANN-BatchMatMulMaxsum/tree/main/.agents/skills/shmem-ops-testcase-gen
Command: npx skills add https://github.com/WangWindow/CANN-BatchMatMulMaxsum --skill shmem-ops-testcase-gen-wangwindow

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Building correct SHMEM (symmetric memory) operators requires rigorous testing across multiple PE counts, data scales, dtypes, and boundary conditions, but manually designing case matrices, golden references, and precision checkers is error-prone and often skipped until after implementation. This Skill converts the correctness contract in a design.md file into an executable test plan and validation scripts before any kernel code is written. ## Core Features & Use Cases - Case Matrix Generation: Produces a test matrix covering XS/S/M/L scale tiers, boundary conditions (chunk tails, PE indivisibility, non-power-of-two shapes, UB capacity limits), stress repeats, and performance cases, with a minimum of 20 cases. - Golden & Checker Scripts: Generates gen_data.py with fixed-seed deterministic inputs and high-precision golden outputs, plus check_result.py implementing dual-statistic judgment (precision_percent and eb bias) with per-OpTypes rtol/atol selection. - Multi-PE Run Scripts: Creates scripts/run.sh that launches one process per PE, sets up the SHMEM runtime environment with dynamic endpoint allocation, enforces a 120-second timeout, and passes explicit tolerance arguments. - Use Case: Given a design.md for a matmul_allreduce fused operator, produce the full test suite including rank-pattern golden construction, fp16 tolerance tables, and an 8-PE performance case pair (perf_s and perf_l) ready for Phase 6 evaluation. ## Quick Start Generate the complete test plan, case matrix, gen_data.py, check_result.py, and run.sh for my SHMEM operator based on the correctness contract in design.md.

Frequently Asked Questions about shmem-ops-testcase-gen

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate test cases for a SHMEM operator?

Extract the correctness contract (oracle, tolerance, invariants, case_matrix) from design.md, then generate a case matrix covering XS/S/M/L scales plus boundary and stress cases. Follow with gen_data.py for golden outputs, check_result.py for precision validation, and run.sh for multi-PE execution.

What tolerance should I use for SHMEM operator precision checking?

Tolerance depends on the operator's OpTypes classification, dtype, and compute_times. MOVE and integer operators require bitwise exact match (rtol=0), while COMPUTE_FLOAT operators use values like 2^-8 for fp16 or 2^-11 for fp32, with atol equal to rtol.

How many test cases does a SHMEM case matrix need?

The minimum is 20 cases covering four scale tiers (XS/S/M/L), at least 4 boundary cases (chunk tail, PE indivisibility, non-aligned shapes, UB boundary), at least 1 stress repeat case, and performance cases at both S and L tiers.

How do I construct golden reference data for collective communication operators?

Use rank pattern inputs where each PE's data encodes its rank (e.g., pe_id * 1000000 + index), making communication misplacement directly visible. For floating-point compute, use numpy or PyTorch references with float32 intermediate accumulation and a fixed random seed of 42.

Why does my SHMEM run script fail when tests run in parallel?

Hardcoded IPPORT or SHMEM_UID_SESSION_ID values cause shmem init failures when multiple runs execute concurrently. Use setup_shmem_dynamic_endpoints to allocate ports dynamically instead of fixed defaults like tcp://127.0.0.1:27010.

Can golden generation or precision checking live in main.cpp?

No. main.cpp must only handle host orchestration, kernel launch, and output file writing. Golden generation and precision verification belong in separate Python scripts (gen_data.py and check_result.py) to keep responsibilities separated.