Generate and optimize TPC-H Parquet datasets with cudf or pyarrow.
npx skills add https://github.com/sirius-db/sirius --skill dataset-manager
Please help me install this Agent Skill. Skill: dataset-manager Source: https://github.com/sirius-db/sirius/tree/main/.claude/skills/dataset-manager Command: npx skills add https://github.com/sirius-db/sirius --skill dataset-manager
This Skill streamlines the management of TPC-H datasets in Parquet format, enabling efficient generation, optimization, and inspection for database benchmarking.
Generate TPC-H data for scale factor 100 in the default output directory.
High-intent search queries and answers about installing and using this skill.
Generate TPC-H Parquet datasets at any scale factor using an optimized Rust-based generator to produce benchmark data in your specified output directory for database performance testing.
Parquet file optimization consolidates and rewrites files by adjusting row group sizes and compression, automatically leveraging cudf on GPU or falling back to pyarrow on CPU to improve query performance.
Yes, automatic backend selection uses cudf for GPU acceleration when rewriting Parquet files, falling back to pyarrow for CPU processing to ensure efficient file optimization.
Inspect TPC-H Parquet datasets by analyzing their structure, row group sizes, and file sizes to understand the layout and ensure efficient benchmarking before running queries.
No, a GPU is not required; the optimizer automatically selects cudf for GPU acceleration if available, otherwise it uses pyarrow on CPU to rewrite and consolidate Parquet files.
kindoflukeGenerate, inspect, consolidate, and optimize TPC-H parquet datasets for Sirius benchmarking.
sirius-dbExecute TPC-DS benchmarks across Legacy Sirius, Super Sirius, and DuckDB CPU engines.
ztdevelopsGenerate a TPC-H query processor project skeleton with benchmarking and validation tooling.
gracefullightDetect CPU, GPU, memory, and disk resources for computational planning.
