skills.rest
Canner
CannerOFFICIAL
Explore all skills by this author →
Website𝕏@getwrenaiGitHub

wren-dlt-connector

Connect SaaS APIs to Wren Engine via dlt pipelines and generate queryable semantic projects.

663|199|Updated May 9, 2022
Search Tags:#sql#etl#duckdb#Data Pipeline#DLT#saaS#Semantic Layer
One-click install
npx skills add https://github.com/Canner/wren-engine --skill wren-dlt-connector
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: wren-dlt-connector
Source: https://github.com/Canner/wren-engine/tree/main/skills/wren-dlt-connector
Command: npx skills add https://github.com/Canner/wren-engine --skill wren-dlt-connector

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires duckdb, pyyaml, dlt, and includes scripts (resource) and references (resource) components.

What problem does it solve?

Getting SaaS data from tools like HubSpot, Stripe, or Salesforce into a SQL-queryable semantic layer normally requires manual ETL scripting, schema mapping, and model authoring. This Skill automates the entire path from API credentials to a verified Wren project that answers real SQL queries.

Core Features & Use Cases

  • Guided dlt Pipeline Setup: Walks through installing dlt, picking a verified SaaS source, configuring credentials, and loading data into a local DuckDB file.
  • Automatic Project Generation: The introspect_dlt.py script introspects the DuckDB schema, filters dlt internal tables and columns, detects parent-child relationships, normalizes types via sqlglot, and emits a complete Wren v2 YAML project.
  • End-to-End Verification: Builds the MDL and runs real SQL queries against every model before declaring success, catching catalog mismatches and type errors.
  • Use Case: A data analyst with a Stripe secret key runs the pipeline, generates a Wren project from stripe_data.duckdb, and immediately queries customers, charges, and subscriptions with SQL.

Quick Start

Ask the assistant to connect your HubSpot account to Wren Engine using your private app token so you can analyze contacts and deals with SQL.

Frequently Asked Questions about wren-dlt-connector

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I load HubSpot or Stripe data into DuckDB with dlt?▼

Install dlt with the DuckDB extra, write a pipeline script that calls the verified source function with your API credential, and run it with destination set to duckdb. The pipeline creates a local .duckdb file containing all extracted tables.

How do I create a Wren project from an existing dlt DuckDB file?▼

Run the introspect_dlt.py script with --duckdb-path pointing to your file and --output-dir for the project. It discovers tables, filters dlt internal columns, detects relationships, and writes wren_project.yml, model files, and relationships.yml.

Why do Wren queries fail with table not found on DuckDB?▼

The most common cause is a catalog mismatch: Wren ATTACHes the DuckDB file using its filename stem as the catalog, so table_reference.catalog must equal that stem. Also confirm the profile url points to the directory containing the .duckdb file.

Which SaaS sources does dlt support for this workflow?▼

dlt provides verified sources for HubSpot, Stripe, Salesforce, GitHub, Slack, Google Analytics, Google Sheets, Notion, Jira, Zendesk, Shopify, and Airtable, among others. Each source has its own credential pattern documented in the references file.

Can I run a dlt sync while querying the DuckDB file?▼

No, DuckDB has a single-writer limitation, so running a dlt pipeline while querying can lock the file. For concurrent access, write to a separate file and swap it atomically after the load completes.

Related Skills

Explore More Data & Analytics Skills→
1337hero1337hero
1

api-gateway

Routes authenticated requests to multiple external services via a centralized API gateway.

Community
Advanced
aaronachermannaaronachermann

databricks-metric-views

Define and manage YAML-based business metrics in Databricks Unity Catalog.

Community
Advanced
dtsongdtsong

dlt-extract

Create portable DLT pipelines for file-based data extraction with destination swapping.

Community
Advanced
dtsongdtsong

duckdb

Analyze and manipulate local CSV, Parquet, and JSON files using DuckDB SQL.

Community
Intermediate