s3-parquet-sampling-plot-cached

What problem does it solve?

Efficiently sample a subset of a massive Parquet dataset stored on an S3‑compatible bucket, cache the sampled rows locally as a Parquet file for fast reuse, and produce high‑resolution PNG plots suitable for analysis and publication.

Core Features & Use Cases

• Lazy, scalable sampling of large Parquet datasets from S3 using Dask.
• Local caching of the sampled data as Parquet for fast reuse in subsequent runs.
• High‑resolution PNG plots and publication‑quality vector outputs suitable for reports and papers.
• Example use case: reproducible exploration and visualization of the shboost2024 dataset (≈218M rows).

Quick Start

Run the sampling-and-plot workflow to sample a subset from the S3 Parquet data, cache it locally as Parquet, and generate publication-ready plots.