Perform in-memory data manipulation with lazy and eager execution on Apache Arrow.
npx skills add https://github.com/mkurman/tamux --skill polars-mkurman
Please help me install this Agent Skill. Skill: polars Source: https://github.com/mkurman/tamux/tree/main/skills/scientific-skills/polars Command: npx skills add https://github.com/mkurman/tamux --skill polars-mkurman
Polars provides a high-performance in-memory DataFrame library for data analytics, solving the bottlenecks of traditional Python data frames by offering lazy evaluation, parallel execution, and Arrow-backed types.
Load a dataset using Polars and compute a quick summary.
High-intent search queries and answers about installing and using this skill.
Use lazy evaluation for large-scale DataFrame transformations to optimize the entire query plan before execution. This approach enables parallel processing and efficient memory usage when filtering, aggregating, and transforming data.
Process large CSV files for ETL using an Apache Arrow-backed DataFrame library with lazy evaluation. This enables parallel execution and columnar memory layout for fast reads, transformations, and aggregations across large-scale data.
Yes, explicit typing is required for in-memory DataFrame operations. Strict data types ensure reliable parallel execution and efficient columnar memory layout when performing transformations, aggregations, and windowing operations on large datasets.
Yes, Apache Arrow can be used for high-performance data manipulation by backing an in-memory DataFrame library. This columnar format enables parallel execution, lazy evaluation, and efficient I/O for large-scale analytics and ETL tasks.
Use lazy execution for large-scale data pipelines to optimize the query plan before running, whereas eager execution processes immediately. Lazy evaluation suits complex aggregations and transformations, while eager execution fits quick data summaries.
biomaps-infraProcess and analyze large tabular datasets exceeding RAM using Vaex.
ejolietPerform fast, memory-efficient DataFrame operations on large datasets with Polars.
spqwLoad Apache Arrow IPC and Parquet files in Observable Framework projects.
schneidermuProcess dataframes with lazy evaluation, parallelism, and Arrow-backed I/O.
