skills.rest
jerrylin96
jerrylin96COMMUNITY
Explore all skills by this author →
WebsiteGitHub

federate-lakehouse-catalog

Configures Google Cloud Lakehouse federated catalogs connecting BigQuery to Databricks Unity or AWS Glue.

9|Updated Jul 8, 2026
Search Tags:#bigquery#iceberg#lakehouse#cross-cloud#aws glue#federated catalog#databricks unity
One-click install
npx skills add https://github.com/jerrylin96/dotgemini --skill federate-lakehouse-catalog-jerrylin96
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: federate-lakehouse-catalog
Source: https://github.com/jerrylin96/dotgemini/tree/main/skills/federate-lakehouse-catalog
Command: npx skills add https://github.com/jerrylin96/dotgemini --skill federate-lakehouse-catalog-jerrylin96

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Setting up cross-cloud data federation between Google Cloud BigQuery and remote Iceberg REST catalogs like Databricks Unity or AWS Glue involves many error-prone steps: regional secret management, IAM trust policies, service account bindings, and region pairing. This Skill guides the agent through the entire setup so remote data becomes queryable from BigQuery without manual trial and error. ## Core Features & Use Cases - Databricks Unity Federation: Creates regional Secret Manager secrets for OAuth credentials, provisions a federated BigLake Iceberg catalog, and grants the catalog service account secret access. - AWS Glue Federation: Builds AWS IAM roles with placeholder trust policies, attaches scoped Glue and S3 read permissions, creates the federated catalog, and finalizes trust with the Google service agent ID. - Validation and Troubleshooting: Verifies federation by listing namespaces via the bq CLI and provides region-pairing and troubleshooting guidance. - Use Case: A data engineer wants to query tables stored in AWS Glue from BigQuery. The Skill walks through IAM role creation, catalog provisioning, trust policy updates, and a final validation query. ## Quick Start Ask the agent to federate your lakehouse catalog to Databricks or AWS Glue so you can query that data from BigQuery.

Frequently Asked Questions about federate-lakehouse-catalog

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I query Databricks Unity Catalog data from BigQuery?▼

Create a federated BigLake Iceberg catalog pointing to your Databricks Unity instance, storing the OAuth client ID and secret in a regional Secret Manager secret. Once the catalog service account is granted secret access, you can query tables directly with standard BigQuery SQL.

How to connect AWS Glue Data Catalog to Google BigQuery?▼

Create an AWS IAM role with a federated trust policy for accounts.google.com, attach Glue and S3 read permissions, then create a BigLake federated catalog with your AWS account ID as the glue-warehouse. Finally update the trust policy with the catalog's Google service agent ID.

Which GCP region should I pair with my AWS region for federation?▼

Choose GCP regions with low-latency dedicated or partner interconnect to your AWS region. For example, AWS us-east-1 pairs with GCP us-east4, and AWS eu-west-2 pairs with GCP europe-west2. The secret and catalog must reside in the same GCP region.

Why does my federated catalog return permission errors or empty results?▼

Common causes include the AWS trust policy not referencing the correct biglake-service-account-id, the Databricks secret missing from the catalog's region, or the service account lacking roles/secretmanager.secretAccessor. Verify each binding and consult the cross-cloud lakehouse troubleshooting documentation.

Can this skill run Databricks SQL queries or manage AWS Glue jobs?▼

No. The skill only sets up federated catalogs so remote metadata and data become queryable from GCP engines like BigQuery. It does not execute SQL directly against Databricks or manage remote clusters, Glue jobs, or other remote infrastructure.

Related Skills

Explore More Data & Analytics Skills→
interflownexinterflownex
2

federate-lakehouse-catalog

Federate remote Iceberg REST catalogs into Google Cloud BigLake for cross-cloud querying.

Community
Advanced
supercent-iosupercent-io

looker-studio-bigquery

Connect BigQuery data sources to Looker Studio dashboards.

Official
Intermediate
1919114514yasenpai-maker1919114514yasenpai-maker

federate-lakehouse-catalog

Configures Google Cloud Lakehouse federated catalogs connecting BigQuery to Databricks Unity and AWS Glue.

Community
Advanced
Lathika-laaLathika-laa

federate-lakehouse-catalog

Configures Google Cloud Lakehouse federated catalogs connecting BigQuery to Databricks Unity and AWS Glue.

Community
Advanced