dataform-bigquery

Generates and validates Dataform SQLX pipeline code for BigQuery ELT workflows.

9|Updated Jul 8, 2026
One-click install
npx skills add https://github.com/jerrylin96/dotgemini --skill dataform-bigquery-jerrylin96
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: dataform-bigquery
Source: https://github.com/jerrylin96/dotgemini/tree/main/skills/dataform-bigquery
Command: npx skills add https://github.com/jerrylin96/dotgemini --skill dataform-bigquery-jerrylin96

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Building Dataform pipelines for BigQuery requires precise SQLX syntax, correct incremental configurations, and valid DAG dependencies, and mistakes lead to failed compiles or broken production pipelines. This Skill guides the creation, modification, and validation of Dataform projects so pipelines compile cleanly and follow BigQuery best practices. ## Core Features & Use Cases - Pipeline Generation: Creates SQLX actions, source declarations, incremental tables, and GCS ingestion operations with proper configuration blocks. - Validation Workflow: Runs dataform compile, dry-run executions, and bq query checks to catch syntax and dependency errors before deployment. - Automatic Optimization: Applies data cleaning protocols and SQL optimizations like column pruning and predicate filtering on every pipeline. - Use Case: When asked to ingest CSV files from GCS into an existing BigQuery table, it creates an external table declaration, an incremental SQLX action matching the target schema, and validates the result with a dry run. ## Quick Start Ask the assistant to create a Dataform pipeline that loads data from a GCS bucket into a BigQuery table using incremental updates.

Frequently Asked Questions about dataform-bigquery

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a Dataform pipeline for BigQuery?

Initialize a project with dataform init, declare BigQuery sources in declaration SQLX files, and define table or incremental actions that reference sources via ref(). Validate everything with dataform compile and a dry run before executing.

How to append data to an existing BigQuery table with Dataform?

Use type incremental with the name set to the existing target table, and write only a SELECT statement in the body since Dataform auto-generates the INSERT. Do not create a declaration file for the target table.

Does Dataform support BigLake Iceberg tables with 4-part names?

Dataform does not natively support 4-part Project.Catalog.Dataset.Table naming, but you can concatenate the catalog and namespace into the schema field of a declaration. This works for querying but not for creating views directly from BigLake sources.

Why does dataform run --dry-run fail with table not found errors?

This happens when upstream dependencies defined in the project have not been materialized in BigQuery yet. These specific not-found errors can be ignored, but syntax or permission errors must be fixed before proceeding.

How do I ingest CSV files from GCS into BigQuery using Dataform?

Create an external table in a SQLX operations file pointing at the GCS URI, using STRING for all columns with allow_jagged_rows, allow_quoted_newlines, and ignore_unknown_values enabled. Then build incremental or table actions on top of it.