motherduck-load-data

Load data into MotherDuck from files, object storage, dataframes, and external databases.

Updated Oct 15, 2019
One-click install
npx skills add https://github.com/kkkaoru/dotfiles --skill motherduck-load-data-kkkaoru
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: motherduck-load-data
Source: https://github.com/kkkaoru/dotfiles/tree/main/.agents/skills-stroage/motherduck-load-data
Command: npx skills add https://github.com/kkkaoru/dotfiles --skill motherduck-load-data-kkkaoru

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Getting data into MotherDuck correctly requires choosing the right ingestion path for each source type, and picking the wrong one leads to slow loads, failed imports, or unsupported operations on the Postgres endpoint. ## Core Features & Use Cases - Source-Based Path Selection: Classifies sources as object storage/HTTPS, local files, in-memory dataframes, or external databases, then picks CTAS, INSERT...SELECT, COPY, or Arrow bulk loads accordingly. - Client Path Guidance: Distinguishes native DuckDB clients (Python duckdb, Node.js @duckdb/node-api) from Postgres-endpoint thin clients, clarifying which operations each supports. - Format and Cloud Coverage: Provides advanced options for CSV, Parquet, JSON, Delta Lake, and Iceberg, plus CREATE SECRET authentication patterns for S3, GCS, and Azure. - Use Case: You have partitioned Parquet files in S3 and need them in MotherDuck. The skill guides you to a remote-read CTAS with hive_partitioning, then validates row counts before promoting to modeled tables. ## Quick Start Load the Parquet files from my S3 bucket into a new MotherDuck staging table and validate the row counts.

Frequently Asked Questions about motherduck-load-data

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I load data into MotherDuck from S3?

Use CREATE TABLE AS SELECT with read_parquet pointing at your s3:// path, authenticating via environment variables or CREATE SECRET. Prefer Parquet over CSV and filter or select only needed columns during the load for better performance.

Should I use the MotherDuck Postgres endpoint or a DuckDB client for loading?

Use a native DuckDB client (Python duckdb or Node.js @duckdb/node-api) for local files, dataframes, CREATE SECRET, and extension-based reads. Use the Postgres endpoint only for remote-read CTAS or batched multi-row inserts against cloud storage.

Can I load Delta Lake or Iceberg tables into MotherDuck?

Yes, MotherDuck pre-installs the delta and iceberg extensions. Use delta_scan() or iceberg_scan() against the table's object storage path inside a CTAS statement to land the data.

Why does my CSV load into MotherDuck have wrong column types?

Auto-detection can misinfer types on messy files. Load with all_varchar = true into a raw table, then cast columns explicitly in a second CTAS, or override types using the columns parameter of read_csv.

How do I replicate a PostgreSQL database into MotherDuck?

For continuous replication use managed ETL tools like Fivetran, Airbyte, or Estuary. For one-time loads, export with pg_dump or COPY to CSV, stage in object storage, then CTAS into MotherDuck; direct ATTACH works only from a local DuckDB instance.