Select Dask collections and execution models for large Python data workflows.
npx skills add https://github.com/dralkh/seerai --skill dask-dralkh
Please help me install this Agent Skill. Skill: dask Source: https://github.com/dralkh/seerai/tree/main/skills/dask Command: npx skills add https://github.com/dralkh/seerai --skill dask-dralkh
Dask helps you run pandas, NumPy, text, and custom Python workloads when they are too large, too slow, or too dynamic for a single in-memory process.
Use the dask skill to select the right collection, chunking strategy, and scheduler for a large data workflow.
High-intent search queries and answers about installing and using this skill.
Scale pandas DataFrame operations by partitioning data and applying lazy computation across clusters. This enables parallel filtering, grouping, joins, and ETL on tabular analytics without loading everything into memory.
Process unstructured text and JSON logs using Bags to handle raw pipelines before converting to DataFrames. This enables distributed text processing and structured analysis on large unstructured datasets.
Run NumPy array computations out-of-core by using chunked Arrays. Chunking breaks large numerical data into blocks, enabling distributed reductions, linear algebra, and scientific processing across clusters.
Choose the right scheduler by evaluating workload characteristics: use threads for numerical work, processes for Python-bound tasks, synchronous execution for debugging, or distributed clusters for task orchestration with Futures.
Use Futures instead of DataFrames or Arrays for dynamic distributed workflows requiring custom task graphs, dependencies, and stateful computations. Futures orchestrate real-time execution rather than static partitioned data processing.
A distributed cluster is not required to run parallel Python data workflows; local machines support threads, processes, and synchronous execution. Clusters are needed only for scaling across multiple nodes or large out-of-core computations.
viniruggeriDistribute large pandas and NumPy workloads across clusters for parallel processing.
Clay-HHKScale pandas and NumPy workloads across cores or clusters with lazy evaluation.
jasrajtulsiScale pandas, NumPy, and Python workloads across clusters with lazy task graphs.
qinyan-aiScale pandas and NumPy workflows to datasets that exceed memory.
