dataform-bigquery

Generate and optimize Dataform pipeline code for Google BigQuery ELT workflows.

2|1|Updated May 25, 2026

Search Tags:#pipeline #bigquery #sqlx #data-engineering #elt #dataform

Ready to run

npx skills add https://github.com/interflownex/All-in-One --skill dataform-bigquery-interflownex

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: dataform-bigquery
Source: https://github.com/interflownex/All-in-One/tree/main/.gemini/skills/dataform-bigquery
Command: npx skills add https://github.com/interflownex/All-in-One --skill dataform-bigquery-interflownex

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill addresses the complexity of building, maintaining, and optimizing ELT pipelines in Google BigQuery by automating the generation of correct, clean, and efficient Dataform code.

Core Features & Use Cases

Automated Pipeline Generation: Creates and modifies Dataform actions, source declarations, and workflow settings with built-in SQL optimizations.
Data Cleaning & Validation: Enforces strict data cleaning protocols and provides automated SQL optimization summaries for every transformation.
Use Case: When migrating raw data from GCS to BigQuery, this skill automatically generates the necessary SQLX operations, applies required cleaning logic, and validates the pipeline structure to ensure production-ready data models.

Quick Start

Use the dataform-bigquery skill to initialize a new project in the current directory for the project ID my-gcp-project and location us-central1.

Frequently Asked Questions about dataform-bigquery

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I generate Dataform pipelines for BigQuery ELT workflows?▼

You can generate Dataform pipelines for BigQuery by creating SQLX actions, source declarations, and workflow settings with built-in SQL optimizations. This approach automates code creation while enforcing data cleaning protocols to ensure production-ready transformations.

How do I ingest raw data from GCS to BigQuery using Dataform?▼

To ingest raw data from GCS to BigQuery, generate SQLX operations that automatically apply required cleaning logic and validate pipeline structure. This ensures schema alignment and produces production-ready data models for downstream analytics.

Does this approach support incremental materialization in BigQuery?▼

Yes, incremental materialization in BigQuery is fully supported. The skill generates and optimizes Dataform pipeline code to handle incremental data loads efficiently while enforcing strict SQL optimization and data cleaning protocols.

Do I need the Dataform CLI to validate BigQuery pipeline integrity?▼

Yes, you need the Dataform CLI and BigQuery integration to validate pipeline integrity. These tools ensure schema alignment and verify that the generated SQLX transformations maintain strict structural correctness before deployment.

What is the best way to optimize SQLX transformations in Dataform?▼

The best way to optimize SQLX transformations in Dataform is by enforcing strict SQL optimization protocols and automated data cleaning. The skill provides optimization summaries for every transformation to maintain efficient ELT workflows.

Why does my Dataform BigQuery pipeline fail schema validation?▼

Your Dataform BigQuery pipeline fails schema validation when SQLX operations lack proper source declarations or schema alignment. Enforcing strict data cleaning protocols and using Dataform CLI validation ensures pipeline integrity and correct structural configuration.