Run in-process analytical SQL queries on Python DataFrames and files.
npx skills add https://github.com/tondevrel/scientific-agent-skills --skill duckdb-tondevrel
Please help me install this Agent Skill. Skill: duckdb Source: https://github.com/tondevrel/scientific-agent-skills/tree/main/skills/duckdb Command: npx skills add https://github.com/tondevrel/scientific-agent-skills --skill duckdb-tondevrel
This Skill enables users to perform complex SQL queries directly on their Python data structures (like Pandas DataFrames) and large files without needing a separate database server, significantly speeding up analytical tasks.
Use the duckdb skill to query the 'sales.parquet' file and select all columns where the 'region' is 'North'.
High-intent search queries and answers about installing and using this skill.
You can run SQL queries on Pandas DataFrames in Python by using an in-process analytical SQL database engine. This allows you to treat DataFrames as SQL tables, executing complex joins and aggregations directly within your Python environment without needing a separate database server.
Querying large Parquet files directly without loading them entirely into memory is possible. The engine leverages columnar storage and vectorized execution to query Parquet, CSV, and JSON files directly, enabling efficient analysis of terabytes of data stored locally or in cloud storage.
The best way to analyze large CSV files with SQL in Python is using an in-process OLAP database engine. It directly queries large CSV files, utilizing columnar storage and vectorized execution to optimize complex analytical SQL operations like aggregations without requiring explicit data loading.
DuckDB works seamlessly with Polars DataFrames for SQL operations. It provides an in-process analytical SQL database that treats both Pandas and Polars DataFrames as SQL tables, allowing you to perform complex SQL operations like joins and aggregations across both frameworks.
You do not need a separate database server to run analytical SQL queries on local files. An in-process SQL database engine runs directly within your Python environment, allowing you to query files like Parquet and CSV and perform OLAP queries on in-memory data efficiently.
You can join local Pandas DataFrames with cloud storage data using SQL by leveraging an in-process analytical SQL engine. It facilitates complex SQL operations like joins and aggregations, allowing you to analyze terabytes of Parquet data in cloud storage and join it with local DataFrames in a single script.
silvainfmRun SQL analytics directly on CSV, Parquet, and JSON files and Pandas or Polars DataFrames.
crossxwillQuery Parquet data with DuckDB and convert results to pandas DataFrames.
apacheWrite DataFusion DataFrame and SQL queries with Apache Arrow integration in Python.
ryankoleanRun in-process analytical SQL queries on CSV, Parquet, and JSON files.
