tda-acceleration-benchmarking

Validates proposed GPU, Dask, Polars, or vectorization accelerations through profiling, benchmarking, and numerical equivalence checks.

1|Updated Dec 13, 2025
One-click install
npx skills add https://github.com/ZK-Theory/TDL --skill tda-acceleration-benchmarking-zk-theory
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: tda-acceleration-benchmarking
Source: https://github.com/ZK-Theory/TDL/tree/main/.agents/skills/tda-acceleration-benchmarking
Command: npx skills add https://github.com/ZK-Theory/TDL --skill tda-acceleration-benchmarking-zk-theory

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Teams often adopt GPU, cloud, Dask, Polars, or vectorization changes for slow topological data analysis jobs without evidence, wasting effort or silently corrupting results. This Skill enforces a profile-first discipline that proves an acceleration is justified before adoption. ## Core Features & Use Cases - Profile-First Gating: Refuses acceleration proposals until the baseline is profiled and the bottleneck is classified (CPU, memory bandwidth, I/O, GIL, backend, or algorithmic complexity). - Rigorous Acceptance Gate: Requires stage parity on at least 3 representative inputs, material wall-time gains at realistic scale, numerical equivalence within a declared tolerance, passing contracts, and surviving checkpoint/resume behavior. - Worker-Sweep Benchmarking: Mandates worker-count sweeps, execution-locus tables for thread-based candidates, and recorded environment drift and provenance. - Use Case: A TDL job running exact Wasserstein-2 distances is slow at scale. Use this Skill to profile the bottleneck, discover memory-bandwidth contention under loky, and validate a serial process-pool alternative with bit-for-bit output equivalence before accepting it. ## Quick Start Ask the AI to evaluate whether moving a slow TDA job to GPU or a process pool is justified, requiring profiling, a worker-sweep benchmark, and numerical equivalence checks before acceptance.

Frequently Asked Questions about tda-acceleration-benchmarking

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I know if moving a TDA job to GPU is justified?

Profile the baseline first to classify the bottleneck as CPU, memory bandwidth, I/O, GIL, backend, or algorithmic complexity. GPU is the last rung after vectorization, process pools, out-of-core, and backend swaps, and acceptance requires material wall-time gains plus numerical equivalence at realistic scale.

How to benchmark a process pool acceleration for persistent homology jobs?

Benchmark at realistic workload scale with a worker-count sweep, capping processes at the preflight-selected safe worker count. Partition inputs into balanced queueable blocks, verify stage parity against the production unit, and compare outputs against the baseline within a declared tolerance.

Why does adding more workers make my exact Wasserstein computation slower?

Memory-bandwidth-bound kernels can scale negatively under joblib or loky concurrency, as seen with exact W2 EMD degrading roughly 6.4x per task at n_jobs=12. The remedy is serial processes over disjoint input blocks through a bounded worker pool, not more parallel backends.

Does threading speed up GIL-holding computations like gudhi exact W2?

No. Threads only parallelize stages that release the GIL, and exact W2 via gudhi holds it, giving zero parallelism. Build an execution-locus table mapping each stage to GIL behavior before crediting any thread-based concurrency benefit.

When should an acceleration candidate be rejected?

Reject when numerical outputs drift beyond the declared tolerance, contracts fail, checkpoint/resume breaks, or wall-time gains are not material at realistic scale. Rejected candidates are recorded as findings, and numerical drift escalates to defect investigation.