skills.rest
nrakow
nrakowCOMMUNITY
Explore all skills by this author →
GitHub

new-source-onboarding

Onboard new data sources through ingestion, staging, testing, documentation, and lineage phases.

1|1|Updated Feb 27, 2026
Search Tags:#dbt#etl#data-quality#lineage#onboard#data-source#data-catalog
One-click install
npx skills add https://github.com/nrakow/ae-skills-dev --skill new-source-onboarding
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: new-source-onboarding
Source: https://github.com/nrakow/ae-skills-dev/tree/main/skills/new-source-onboarding
Command: npx skills add https://github.com/nrakow/ae-skills-dev --skill new-source-onboarding

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Onboard a new data source from ingestion configuration through staging, testing, documentation, and lineage, coordinating multiple phases to reduce setup time and errors.

Core Features & Use Cases

  • End-to-end onboarding workflow covering ingestion strategy, staging, data quality testing, data cataloging, and lineage registration.
  • Seamless tool integration (Fivetran, Airbyte) with dbt validation to accelerate data source adoption.
  • Real-world scenario: onboard a new source from vendor configuration to ready-to-query lineage in your analytics stack.

Quick Start

Start by invoking the ingestion-strategy phase, then progress through staging-layer, data-quality-testing, data-catalog, and data-lineage to onboard a new source.

Frequently Asked Questions about new-source-onboarding

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I onboard a new data source end-to-end in dbt?▼

Onboarding a new data source in dbt involves configuring ingestion connectors, building staging models, defining data quality tests, and registering lineage. This workflow coordinates those phases to produce sources.yml, staging SQL, schema.yml tests, and catalog documentation.

What is the best way to configure Fivetran or Airbyte ingestion before building dbt staging models?▼

Configuring Fivetran or Airbyte ingestion before dbt staging requires defining connector parameters and mapping raw schemas. This workflow starts with an ingestion-strategy phase to generate connector configuration, then validates the raw source against your dbt staging layer.

How do I add data quality tests and catalog documentation for a new dbt source?▼

Adding data quality tests and catalog documentation for a new dbt source requires defining constraints in a schema.yml file. This workflow generates schema.yml with tests and catalog documentation automatically as part of the data-quality-testing and data-catalog phases.

Does this data source onboarding workflow read existing project context files?▼

Yes, the data source onboarding workflow reads the existing .claude/data-stack-context.md file to understand your current analytics stack. This ensures generated staging models, ingestion configurations, and lineage artifacts align with your established project conventions.

What artifacts are produced when onboarding a new data source through this workflow?▼

Onboarding a new data source through this workflow produces ingestion connector configuration, sources.yml, stg_<source>__<object>.sql files, schema.yml with tests, and catalog documentation. These artifacts wire staging, testing, and lineage together for immediate querying.

Can I use this workflow to onboard data sources with connectors other than Fivetran and Airbyte?▼

The workflow explicitly supports Fivetran and Airbyte for ingestion strategy configuration. While it orchestrates the broader dbt staging and lineage phases, the ingestion-strategy phase is tailored to these connector platforms to ensure proper raw data extraction.

Related Skills

Explore More Data & Analytics Skills→
nrakownrakow
1

data-stack-context

Create and maintain a centralized data stack context document from dbt project files.

Community
Advanced
pkoka888pkoka888
1

using-dbt-for-analytics-engineering

Build and validate dbt models and SQL transformations using dbt CLI.

Community
Advanced
cooper538cooper538

sort-tasks

Sort tasks by dependencies and detect circular references.

Community
Intermediate
dylpickledevdylpickledev
4

dbt-model-scaffolder

Generate dbt model SQL scaffolds and schema.yml files with tests.

Community
Advanced