Build and review Python data workflows with pandas, Polars, Arrow, and DuckDB.
npx skills add https://github.com/stevenke1981/python_skills --skill py-data
Please help me install this Agent Skill. Skill: py-data Source: https://github.com/stevenke1981/python_skills/tree/main/py-data Command: npx skills add https://github.com/stevenke1981/python_skills --skill py-data
This skill addresses the fragility of data processing by providing a structured, contract-based approach to building ETL pipelines, ensuring that data transformations are reproducible, validated, and performant.
Use the py-data skill to analyze the provided sales dataset by defining a schema-first Polars pipeline that filters for completed orders and aggregates revenue by user.
High-intent search queries and answers about installing and using this skill.
To build a reliable Python ETL pipeline, use a schema-first approach with pandas and Polars to define DataFrame transformations, enforce null handling, and validate output contracts before loading data.
Schema validation and null checks in large-scale data processing are handled by implementing explicit data quality gates that verify schema integrity, join cardinality, and uniqueness to prevent silent data corruption.
Yes, you can use DuckDB and Arrow for streaming execution to ingest large Parquet files, perform complex aggregations, and process data efficiently without loading everything into memory.
The best way to switch between Polars and pandas based on data scale is to use multi-engine support, allowing you to seamlessly transition between libraries depending on query complexity and dataset size.
Silent data corruption during transformations occurs when pipelines lack explicit validation; adding data quality gates for schema integrity and uniqueness ensures reproducible and validated outputs.
xiechyParallelize pandas and NumPy workflows with Dask DataFrames and Arrays.
TiepbmDesign governance-backed analytics architectures with layered data models and metric ownership.
AngelP17Automate end-to-end ML pipelines from data ingestion to deployment.
ancolemanTransform raw data into analytical assets using ETL/ELT patterns and orchestration tools.
