skills.rest
2nth-ai
2nth-aiOFFICIAL
Explore all skills by this author →
GitHub

tech/google/cloud/data

Query BigQuery, publish to Pub/Sub, and run Dataflow pipelines.

1|Updated Apr 1, 2026
Search Tags:#BigQuery#Google Cloud#Pub/Sub#Dataflow#Datastream
One-click install
npx skills add https://github.com/2nth-ai/skills --skill tech-google-cloud-data
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: tech/google/cloud/data
Source: https://github.com/2nth-ai/skills/tree/main/tech/google/cloud/data
Command: npx skills add https://github.com/2nth-ai/skills --skill tech-google-cloud-data

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires bigquery, pubsub, dataflow, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill addresses the complexity of data operations on Google Cloud Platform, providing streamlined access to BigQuery, Pub/Sub, and Dataflow for analytics, messaging, and pipelines.

Core Features & Use Cases

  • BigQuery Access: Query and load data, build materialized views, and run scheduled queries.
  • Pub/Sub Integration: Publish and subscribe to topics, handle exactly-once delivery, and apply filters.
  • Dataflow Pipelines: Execute batch and streaming pipelines, leverage Google-provided templates, and use Flex Templates for custom workflows.
  • Datastream & CDC: Stream changes from Cloud SQL/Postgres into BigQuery with near-real-time replication.
  • Use Case: Automate the ingestion of data from Cloud SQL into BigQuery for real-time analytics and reporting.

Quick Start

Use the tech/google/cloud/data skill to create a dataset in BigQuery and import data from a CSV file located at 'gs://my-bucket/data.csv'.

Frequently Asked Questions about tech/google/cloud/data

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I stream data from Cloud SQL Postgres to BigQuery?▼

You can stream data from Cloud SQL Postgres to BigQuery using Datastream for near-real-time change data capture. This continuously replicates database changes directly into BigQuery for up-to-date analytics.

What is the best way to build batch and streaming pipelines on Google Cloud?▼

Build batch and streaming pipelines on Google Cloud using Dataflow with Apache Beam. You can execute pipelines leveraging Google-provided templates or deploy custom workflows via Flex Templates for specialized processing.

Do I need Apache Beam to run Dataflow pipelines for BigQuery operations?▼

Yes, you need Apache Beam 2.50 or higher to run Dataflow pipelines. Dataflow uses Apache Beam to define and execute both batch and streaming data processing workflows on Google Cloud.

Can I load CSV files into BigQuery using Google Cloud Platform?▼

Yes, you can load CSV files into BigQuery by creating a dataset and importing data directly from a Cloud Storage bucket. BigQuery supports standard SQL for querying the loaded data.

Does Pub/Sub support exactly-once delivery when publishing messages?▼

Yes, Pub/Sub supports exactly-once delivery when publishing and subscribing to topics. You can also apply message filters to route specific events within your Google Cloud data workflows.

How do I automate scheduled queries and materialized views in BigQuery?▼

Automate scheduled queries and materialized views in BigQuery using standard SQL. This enables recurring data transformations and pre-computed table generation for efficient reporting.

Related Skills

Explore More Data & Analytics Skills→
fritzprixfritzprix
8

system-setup

Install and verify MCP runtime dependencies across Windows, Linux, and macOS.

Community
Intermediate
vikingokftvikingokft

recipe-watch-drive-changes

Monitor Google Drive file and folder changes via gws-events subscriptions.

Official
Intermediate
kubonykubony
10

gcp-alerts

Create and manage Google Cloud Monitoring alert policies via gcloud commands.

Community
Intermediate
ironkid90ironkid90

gcp-dataflow

Manage Apache Beam pipeline lifecycles on Google Cloud Dataflow.

Community
Advanced