Runs MNN regression tests and on-device LLM benchmarks on Android and iOS.
npx skills add https://github.com/alibaba/MNN --skill test-ci
Please help me install this Agent Skill. Skill: test-ci Source: https://github.com/alibaba/MNN/tree/main/skills/test-ci Command: npx skills add https://github.com/alibaba/MNN --skill test-ci
Verifying MNN changes across host builds, Android devices, and iPhones requires juggling multiple scripts, stage matrices, and device quirks; this Skill consolidates the regression suite and iOS LLM benchmarking into one guided workflow.
./test.sh with declarative stage configuration in test_stages.json, including filters for CPU, OpenCL, Vulkan, and low-memory stages.ios_llm_bench.sh, compare branches, and validate Metal kernel changes with documented pitfalls and recovery procedures.Ask the agent to run the MNN test suite locally or on a connected device, for example: run ./test.sh local for host regression or benchmark the LLM on my iPhone with the iOS bench script.
High-intent search queries and answers about installing and using this skill.
Run ./test.sh android <serial> where the serial comes from adb devices. You can pass a filter like cpu, opencl, vulkan, gpu, unit, lowmem, or android-ci to run only a subset of the stage matrix.
Run transformers/llm/engine/ios/ios_llm_bench.sh with --model pointing to an exported MNN model, --team set to your Apple Team ID, and --backend metal with prompt/decode lengths. The script builds, signs, installs, and reports prefill/decode tokens per second.
Edit test_stages.json, which declaratively defines all unit, lowmem, smoke, and bench stages. Add an object to android.stages or local.stages, or add a test name to a stage's skip array with rationale documented in the _documentation block.
Yes. LLM model provisioning is lazy and only the llm stage needs a model download; unit, smoke, and bench stages run offline. Set LLM_MODEL_DIR to a local model or LLM_MODEL_SOURCE=modelscope when huggingface.co is unreachable.
Common causes include a wrong Team ID, an untrusted developer certificate on the device, or exceeding the three-app limit of free developer profiles. Check bench_logs for FBSOpenApplicationErrorDomain errors and trust the certificate in device Settings.
SKIP is not a failure; it means a prerequisite was absent, such as a GPU library, model, or build artifact. The exit code is non-zero only when a stage actually fails, so automation should gate on the exit code.
alibabaAdd new LLM models to MNN by downloading, analyzing, mapping, and exporting.
ROCmAutomate Git bisect to locate the commit introducing GPU kernel performance regressions.
VMAFxScaffold a GPU backend package for libvmaf with build wiring and smoke tests.
artofrawrMaintain a CODE_INDEX.md capability index to prevent code duplication.