codebase-inode-audit

What problem does it solve?

Parallel filesystems used by HPC and ML training clusters have strict inode quotas, and unoptimized experiment code that writes millions of small files (such as per-sample images, per-epoch artifacts, or unbounded Slurm logs) can exhaust these quotas mid-run, causing costly job failures and wasted compute resources.

Core Features & Use Cases

• High-Risk Pattern Detection: Identifies common inode-heavy code patterns including per-sample image/latent dumps, per-epoch artifact directories, uncurated Hydra multirun output trees, and unbounded Slurm log storage.
• Quantitative Inode Pressure Estimation: Calculates projected total file counts for each risky output surface to measure potential inode usage against backend quotas.
• Readiness Gate Enforcement: Blocks progression to dryrun and fullrun stages if projected file counts exceed 80% of the target backend's inode quota, and mandates implementation of inode-safe fixes such as tar sharding, delete-after-score workflows, or output directory pinning.
• Real-World Use Case: Before launching a large-scale image generation training run on the LUMI supercomputer, run this audit to catch a pattern where the code writes one PNG per generated sample per evaluation step, which would produce 2 million files and exhaust the project's inode budget.

Quick Start

Use the codebase-inode-audit skill to scan your ML experiment codebase for inode risks before running a scale-up dryrun or fullrun.