s3-parquet-sampling

What problem does it solve?

Large Parquet datasets stored on S3 are expensive to load fully for exploration or visualization. This Skill provides sampling and local caching to substantially reduce memory use and accelerate analysis.

Core Features & Use Cases

• Sample and reduce Parquet data on S3 to sizes suitable for interactive exploration.
• Cache the reduced dataset locally as Parquet to avoid repeated downloads.
• Use Dask for scalable processing and hvPlot/Datashader for scalable visualizations in data-heavy workflows.

Quick Start

Run the s3-parquet-sampling workflow to sample a large dataset on S3 and cache the result locally.