skills.rest
AubreyHan
AubreyHanCOMMUNITY
Explore all skills by this author →
GitHub

dataform-bigquery

Generates Dataform pipeline code for BigQuery ELT workflows.

Updated Jun 10, 2026
Search Tags:#pipeline#bigquery#sqlx#elt#dataform#workflow settings#gcs ingestion
One-click install
npx skills add https://github.com/AubreyHan/SKILL_Repo --skill dataform-bigquery-aubreyhan
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: dataform-bigquery
Source: https://github.com/AubreyHan/SKILL_Repo/tree/main/dataform-bigquery
Command: npx skills add https://github.com/AubreyHan/SKILL_Repo --skill dataform-bigquery-aubreyhan

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

It helps create and modify Dataform pipelines for Google BigQuery with accurate SQLX, clean project structure, and reliable validation.

Core Features & Use Cases

  • Pipeline Authoring: Generates or updates Dataform actions, declarations, incremental models, and workflow configuration for BigQuery ELT.
  • GCS and BigQuery Integration: Supports ingestion patterns that move data from GCS into BigQuery and connect existing warehouse tables into a Dataform DAG.
  • Validation and Optimization: Emphasizes compile-time checks, dry-run validation, schema review, automatic cleaning, and BigQuery-focused SQL optimization.
  • Use Case: Use it when setting up a new Dataform project, fixing SQLX logic, or building a production ELT pipeline with source and target tables.

Quick Start

Use the dataform-bigquery skill to generate or modify a Dataform BigQuery pipeline and validate the resulting project structure and SQLX logic.

Frequently Asked Questions about dataform-bigquery

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I build a BigQuery ELT pipeline using Dataform?▼

To build a BigQuery ELT pipeline with Dataform, generate SQLX actions, declare sources, and configure workflow_settings.yaml. This structures your project and connects GCS ingestion and existing warehouse tables into BigQuery.

How do I configure GCS ingestion in a Dataform SQLX pipeline?▼

Configuring GCS ingestion in Dataform requires authoring specific SQLX actions that move data from Google Cloud Storage into BigQuery. This integrates raw external data directly into your ELT DAG as declared sources.

Does Dataform support incremental table loading for BigQuery?▼

Yes, Dataform supports incremental models for BigQuery. You author incremental table logic in SQLX to process only new or changed data, optimizing query costs and pipeline execution time.

What is the best way to validate Dataform SQLX before running it in BigQuery?▼

The best way to validate Dataform SQLX is through compile-time checks and dry-run validation. This inspects schemas and verifies SQL logic against BigQuery to catch errors before execution.

Why does my Dataform BigQuery workflow_settings configuration fail to compile?▼

A Dataform workflow_settings configuration fails to compile due to incorrect YAML syntax or invalid SQLX action references. You must verify project structure and perform schema inspection to fix dependencies.

When should I use Dataform instead of writing raw SQL for BigQuery ELT?▼

Use Dataform instead of raw SQL for BigQuery ELT when you need managed dependencies, automated GCS ingestion, and dry-run validation. It provides a structured project environment using SQLX and workflow_settings configuration.

Related Skills

Explore More Data & Analytics Skills→
interflownexinterflownex
2

dataform-bigquery

Generate and optimize Dataform pipeline code for Google BigQuery ELT workflows.

Community
Advanced
ironkid90ironkid90

dataform-bigquery

Automate Dataform pipeline creation and management for Google BigQuery.

Community
Advanced
i9wa4i9wa4
11

bigquery-expert

Optimize Google BigQuery queries, data management, and cost governance.

Community
Advanced
RxFitRxFit

gcp-pipeline-orchestration

Generate and deploy Cloud Composer orchestration pipeline definitions for GCP workflows.

Community
Advanced