opencl-optimize

Start by profiling with a build using MNN_GPU_TIME_PROFILE=ON to find the bottleneck kernel, then check the dispatcher's onResize code to confirm which kernel your target shape actually uses. Apply one optimization technique at a time, verifying correctness against a CPU baseline after each change.

MNN does not compile .cl files directly; the runtime reads embedded strings in generated *_mnn_cl.cpp files. After editing any .cl file you must run python3 opencl_codegen.py in the cl directory, then rebuild and confirm the new macro appears in the binary.

Use interleaved A/B testing: keep baseline and optimized binaries, alternate base-opt-base-opt runs back to back, and discard cold-start first runs. Single-session numbers look stable but cross-session thermal drift of about 8 percent makes sequential before/after comparisons unreliable.

Yes, real Android devices with Adreno or Mali GPUs are required for meaningful results. Mac and desktop OpenCL behavior differs significantly from mobile GPUs and cannot represent phone performance or stability.

Obtain runnable sample code from the user first, add runtime feature detection in OpenCLRuntime, and implement the new kernel behind an #ifdef with the original path kept as fallback. Test on both Adreno and Mali since extension behavior differs across vendors.

Operators that only support Buffer execution can silently fall back to Image mode or CPU when only backend=3 is specified, producing false passes. Force buffer mode explicitly (e.g., numThread 68 for LLMs) and confirm GPU execution via profiler output or backend logs.