hpc-dev-testing-workflow

A multi-root VS Code workspace lets you manage repository and project folders simultaneously, maintaining Claude sessions across both. Configure it by creating a `.code-workspace` file that references your repository and external data directories, then open it in VS Code to enable seamless context switching between code and pipeline environments.

Yes. VS Code Remote SSH tunneling connects your local editor to HPC systems, enabling you to edit code directly on the cluster while synchronizing changes with external data directories. This eliminates manual file transfers and keeps your development environment unified across local and remote contexts.

Create a minimal internal test dataset and tile-based processing configuration that mirrors your production pipeline structure. Use tile renumbering scripts and test-parameter updates to validate code changes on a subset of data, enabling rapid iteration without waiting for full-scale runs.

Set up a multi-root VS Code workspace that links your code repository and external data directories, then use VS Code Remote SSH to sync changes automatically. This workflow eliminates manual synchronization steps and keeps your development environment consistent as you iterate on code and test parameters.

Your HPC environment must support VS Code Remote SSH access. Beyond that, the workflow adapts to existing Python scientific pipelines and external data structures. You'll configure a minimal test dataset and tile-based processing parameters to match your pipeline's design.

Tile-based processing breaks data into smaller, manageable chunks that run quickly on test datasets. Combined with tile renumbering scripts, this approach lets you validate code changes reproducibly without processing the entire dataset, significantly reducing iteration time during development.