skills.rest
nguyenhungtran18
nguyenhungtran18COMMUNITY
Explore all skills by this author →
GitHub

federate-lakehouse-catalog

Configures Google Cloud Lakehouse federated catalogs to query remote Databricks Unity and AWS Glue data.

1|Updated Aug 31, 2026
Search Tags:#BigQuery#iceberg#lakehouse#cross-cloud#aws glue#federated catalog#databricks unity
One-click install
npx skills add https://github.com/nguyenhungtran18/skill-and-tool-tracker --skill federate-lakehouse-catalog-nguyenhungtran18
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: federate-lakehouse-catalog
Source: https://github.com/nguyenhungtran18/skill-and-tool-tracker/tree/main/skills/federate_lakehouse_catalog
Command: npx skills add https://github.com/nguyenhungtran18/skill-and-tool-tracker --skill federate-lakehouse-catalog-nguyenhungtran18

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Organizations with data stored in Databricks Unity Catalog or AWS Glue cannot query it directly from Google Cloud engines like BigQuery or Spark without complex data migration. This Skill guides the setup of cross-cloud federated catalogs so remote Iceberg data becomes queryable in place. ## Core Features & Use Cases - Databricks Unity Federation: Creates a BigLake Iceberg federated catalog using regional Secret Manager credentials and OAuth service principals. - AWS Glue Federation: Sets up IAM roles with placeholder trust policies, scoped Glue/S3 permissions, and background metadata refresh. - Region Pairing Guidance: Recommends optimal GCP regions paired with AWS regions to minimize cross-cloud query latency. - Use Case: A data team keeps tables in AWS Glue on S3 but wants analysts to run BigQuery SQL against them. This Skill walks through IAM role creation, catalog creation, trust policy updates, and validation queries. ## Quick Start Ask the agent to federate your lakehouse catalog to Databricks Unity or AWS Glue so you can query the remote data from BigQuery.

Frequently Asked Questions about federate-lakehouse-catalog

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I query Databricks data from BigQuery?▼

Create a BigLake Iceberg federated catalog of type unity pointing to your Databricks workspace, storing the OAuth client ID and secret in a regional Secret Manager secret. Once created, query tables using standard BigQuery SQL with the catalog namespace path.

How to connect AWS Glue Data Catalog to Google Cloud?▼

Create an AWS IAM role with a placeholder trust policy, attach Glue and S3 read permissions, then create a federated BigLake catalog with your AWS account ID as the glue-warehouse. Finally update the trust policy with the Google service account ID and enable background refresh.

Which GCP region should I pair with my AWS region for federation?▼

Choose GCP regions with low-latency dedicated or partner CCI connectivity to your AWS region. For example, AWS us-east-1 pairs best with GCP us-east4, and AWS eu-central-1 pairs with GCP europe-west3.

Why does my federated catalog validation fail after creation?▼

Common causes include mismatched regions between the secret and catalog, missing secretAccessor IAM binding for the BigLake service account, or an AWS trust policy not yet updated with the Google service account ID. Verify each binding and consult the troubleshooting documentation.

Can I use this for running Databricks SQL or managing Glue jobs?▼

No. This Skill only federates catalog metadata so remote data can be queried from GCP engines. It does not execute SQL directly on Databricks or manage remote clusters, Glue jobs, or other remote infrastructure.

Related Skills

Explore More Data & Analytics Skills→
ricardoluiricardolui

federate-lakehouse-catalog

Configures Google Cloud Lakehouse federated catalogs connecting to Databricks Unity and AWS Glue.

Community
Advanced
supercent-iosupercent-io

looker-studio-bigquery

Connect BigQuery data sources to Looker Studio dashboards.

Official
Intermediate
dennisvinkdennisvink

creating-secrets-using-best-practices

Creates AWS Secrets Manager secrets with KMS encryption, rotation, and least-privilege IAM policies.

Community
Advanced
BAIZ1DBAIZ1D

federate-lakehouse-catalog

Configure cross-cloud lakehouse catalog federation for GCP analytics engines.

Community
Advanced