tda-resource-preflight

Benchmarks and plans worker counts, checkpointing, and wall-time budgets before launching long-running compute jobs.

1|Updated Dec 13, 2025
One-click install
npx skills add https://github.com/ZK-Theory/TDL --skill tda-resource-preflight-zk-theory
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: tda-resource-preflight
Source: https://github.com/ZK-Theory/TDL/tree/main/.agents/skills/tda-resource-preflight
Command: npx skills add https://github.com/ZK-Theory/TDL --skill tda-resource-preflight-zk-theory

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Long-running stochastic compute jobs (bootstraps, permutation nulls, Markov batteries, MICE refits, large ETL) often launch with guessed parallelism, no checkpointing, and no wall-time budget, leading to stalled runs, wasted hours, and unrecoverable failures. This Skill produces a defensible resource plan before launch. ## Core Features & Use Cases - Production-Entry-Point Benchmarking: Sweeps worker counts 1→N against the real script at realistic scale, flagging sub-scale estimates as PROVISIONAL. - Structured Preflight Record: Emits a dated JSON record capturing data scale, candidate worker counts, selected strategy, heartbeat cadence, and wall-time estimates. - Platform-Specific Guidance: Covers WSL background process survival, Windows PowerShell runner pitfalls, and GIL/backend constraints for kernels like exact Wasserstein-2. - Use Case: Before launching a permutation-null battery with B=1000 at landmark count 5000, run the preflight to measure per-diagram timing, select the optimal loky worker count, and record a checkpoint/resume plan with a wall-time budget. ## Quick Start Ask the AI to run a resource preflight for your planned bootstrap or permutation job, including a worker-count sweep and a written wall-time estimate before launch.

Frequently Asked Questions about tda-resource-preflight

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I choose the right worker count for a parallel compute job?

Benchmark the production entry point with a worker-count sweep from 1 to N at realistic scale, then select the feasible count with the lowest p75-projected wall time. Record memory headroom per candidate; a lower count can win when extra processes contend for RAM, CPU, or I/O.

How to benchmark a long-running bootstrap or permutation test before launching?

Time the actual statistic at realistic n × B through the real script, never a component kernel alone. Record the sweep's call count as a percentage of the target run, and label any sub-scale estimate PROVISIONAL in the preflight record.

Why does joblib threading give no speedup for Wasserstein distance computation?

Exact Wasserstein-2 in gudhi holds the GIL, so joblib's threading backend yields zero parallelism. Measure serial first, then sweep a bounded loky process count, since the kernel can be memory-bandwidth-bound and show negative process scaling.

Why does my background WSL process die silently or show empty output?

WSL 2 processes are tied to their parent session and die when it exits; detached launches only appear to work. Use the Bash tool's run_in_background with a direct exec like MSYS_NO_PATHCONV=1 wsl.exe python -u script, and have the script self-log with flush.

When should I skip a resource preflight?

Skip it for small deterministic unit tests and trivial calculations. It is required for compute that may exceed roughly 30 minutes, including bootstraps, permutation nulls, Markov batteries, MICE refits, and large ETL jobs.