cnxload

Load Parquet files into JDBC tables with configurable write strategies.

209|30|Updated Sep 18, 2021
One-click install
npx skills add https://github.com/starlake-ai/starlake --skill cnxload
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: cnxload
Source: https://github.com/starlake-ai/starlake/tree/main/.agent/skills/cnxload
Command: npx skills add https://github.com/starlake-ai/starlake --skill cnxload

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill helps data teams move Parquet data into relational databases without writing custom loaders, reducing manual scripting and potential errors.

Core Features & Use Cases

  • Parquet to JDBC loading: a simple CLI-based operation to load Parquet data into a target JDBC table.
  • Configurable write strategies: APPEND or OVERWRITE to manage how data is written during loads.
  • Flexible connection options: supply JDBC URL, driver, user, and password via the options parameter to tailor connections.

Quick Start

Load the Parquet file into the target JDBC table using the cnxload skill.

Frequently Asked Questions about cnxload

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I load Parquet files into a JDBC table?

To load Parquet files into a JDBC table, you use a CLI operation specifying the source file and target table. This process moves columnar Parquet data directly into relational databases for ETL workflows without manual scripting.

Can I overwrite existing data when loading Parquet into a relational database?

Yes, you can overwrite existing data when loading Parquet into a relational database by setting the write_strategy option to OVERWRITE. Alternatively, you can use APPEND to add data without removing existing records.

What JDBC connection parameters are needed for a Parquet data pipeline?

Required JDBC connection parameters for a Parquet data pipeline include the database URL, driver, user, and password. You supply these via the options parameter to establish and tailor the connection to your target relational database.

Does this Parquet to JDBC loader support batch processing?

Yes, this Parquet to JDBC loader supports batch processing scenarios. It is designed to handle both single-file and batch operations, allowing data teams to move multiple Parquet files into relational databases efficiently.

What is the best way to move Parquet data into a relational database without custom scripts?

The best way to move Parquet data into a relational database without custom scripts is using a CLI-based ETL loader. It enforces standard options for source files and target tables, reducing manual scripting errors and automating the data pipeline.