Manipulate large-scale datasets with Polars and Apache Arrow.
npx skills add https://github.com/legout/pi-config --skill polars-legout
Please help me install this Agent Skill. Skill: polars Source: https://github.com/legout/pi-config/tree/main/installed-skills/polars Command: npx skills add https://github.com/legout/pi-config --skill polars-legout
This Skill addresses the performance bottlenecks and memory limitations often encountered when processing large datasets with traditional tools like pandas.
Use the polars skill to load the file data.csv and calculate the mean value of the column sales grouped by region.
High-intent search queries and answers about installing and using this skill.
Use Polars to process multi-gigabyte CSV files, leveraging Apache Arrow for memory-efficient data manipulation and lazy evaluation to optimize query plans before execution, preventing standard Python crashes.
Lazy evaluation in dataframes optimizes query plans before execution to minimize memory usage and maximize speed, enabling efficient processing of large-scale datasets through parallelized execution.
Calculate group-by aggregations and complex joins on large datasets using Polars, which supports parallelized execution and complex transformations efficiently built on Apache Arrow.
Yes, you can use Polars for data science pipelines requiring window functions, as it supports complex transformations including joins, aggregations, and window functions for data engineering.
Use Polars instead of pandas when you encounter performance bottlenecks and memory limitations processing large datasets, utilizing predicate pushdown and parallelized execution for faster results.
ovachieverPerform lazy, expression-based DataFrame operations with Polars for CSV and Parquet pipelines.
crossxwillQuery Parquet data with DuckDB and convert results to pandas DataFrames.
qinyan-aiAccelerate in-memory data analytics with a columnar DataFrame API.
Lord1EgyptManipulate in-memory data with Polars DataFrame library.
