cupynumeric-parallel-data-load
CommunityParallel load sharded data into cupynumeric arrays
Authorsayalinvidia
Version1.0.0
Installs0
System Documentation
What problem does it solve?
Loading large multi-file datasets into a single cupynumeric array when shards vary in row counts and there is no built-in loader for the whole dataset.
Core Features & Use Cases
- Manual partitioning across processors (CPU/OMP/GPU) to read shards in parallel.
- Works with sharded layouts across several formats (npy, Parquet/Arrow, raw binary, HDF5) by streaming per-file reads in a leaf task.
- Use case: ingest sensor or simulation data produced as shard_NNNN.npy files into a unified array for analytics or training.
Quick Start
Run the included write/read workflow to generate shards and load them into a single cupynumeric array.
Dependency Matrix
Required Modules
numpycupynumericlegate
Components
assets
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: cupynumeric-parallel-data-load Download link: https://github.com/sayalinvidia/sayali-skills-test/archive/main.zip#cupynumeric-parallel-data-load Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 510,000+ vetted skills library on demand.