skills.rest
nrakow
nrakowCOMMUNITY
Explore all skills by this author →
GitHub

incremental-models

Automate incremental dbt modeling for new or changed data across Snowflake, BigQuery, Redshift, Databricks, and DuckDB.

1|1|Updated Feb 27, 2026
Search Tags:#dbt#data-modeling#backfill#warehouse#incremental#upsert#lookback
One-click install
npx skills add https://github.com/nrakow/ae-skills-dev --skill incremental-models
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: incremental-models
Source: https://github.com/nrakow/ae-skills-dev/tree/main/skills/incremental-models
Command: npx skills add https://github.com/nrakow/ae-skills-dev --skill incremental-models

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Incremental-models helps analytics teams implement dbt incremental models that only process new or changed data, dramatically reducing compute time and avoiding costly full-table rebuilds.

Core Features & Use Cases

  • Supports multiple warehouses (Snowflake, BigQuery, Databricks/Delta, Redshift, DuckDB) with warehouse-specific incremental strategies.
  • Provides lookback windows to catch late-arriving data, handles schema changes, and supports composite unique keys for safe upserts.
  • Practical scenarios include processing event streams, large fact tables, and slowly changing dimensions with incremental refresh.

Quick Start

Configure your first incremental model and run a targeted incremental build to see real-time improvements.

Frequently Asked Questions about incremental-models

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I build incremental dbt models to avoid full-table refresh costs?▼

Incremental dbt models process only new or changed data to reduce compute time. You can configure materialization, incremental strategies like append or merge, and lookback windows to avoid rebuilding entire tables.

Does incremental dbt modeling work with Snowflake, BigQuery, and Redshift?▼

Yes, incremental dbt modeling supports multiple warehouses including Snowflake, BigQuery, Redshift, Databricks, and DuckDB. Each platform utilizes specific incremental strategies like insert_overwrite and merge for efficient data processing.

How do I handle late-arriving data and schema changes in dbt incremental models?▼

Handle late-arriving data using lookback windows and manage schema changes via the on_schema_change configuration. This ensures your incremental models safely capture delayed records without missing updates.

What is the best way to run upserts on large fact tables using dbt?▼

The best way to run upserts on large fact tables is using dbt merge strategies with composite unique keys. This safely updates existing records and inserts new ones, processing only changed data.

When should I not use incremental materialization for dbt models?▼

Avoid incremental materialization when processing small tables or when late-arriving data cannot be captured by a lookback window. Full rebuilds are safer if data volume is low or complete schema replacements are needed.

Can I validate dbt incremental model configurations before running a full build?▼

Yes, you can validate configurations using dbt compile, build, and test commands. These commands check your frontmatter fields like materialization and incremental_strategy before executing the actual data processing.

Related Skills

Explore More Data & Analytics Skills→
Snowflake-LabsSnowflake-Labs
1

dbt-fixer

Diagnose and correct compilation errors in dbt models within Snowflake environments.

Official
Advanced
JhabbigJhabbig

dbt-transformation-patterns

Organize dbt transformations into staging, intermediate, and mart layers.

Community
Advanced
accelerate-dataaccelerate-data

elementary-data-quality

Automate anomaly detection and schema validation for Microsoft Fabric data quality monitoring.

Official
Advanced
thbehthbeh

databricks-dbsql

Execute advanced SQL and AI functions in Databricks SQL.

Community
Advanced