trino-federated-query-architecture

What problem does it solve?

It solves the performance and reliability problems of writing cross-catalog Trino queries over heterogeneous sources (Iceberg, PostgreSQL, MySQL, Kafka, ClickHouse) by explaining how connector fetch behavior, predicate pushdown, and data movement affect real execution cost.

Core Features & Use Cases

• Federation architecture guidance: Plan cross-catalog JOINs with an explicit mental model of where data is fetched, filtered, joined, and routed.
• Connector-aware pushdown strategy: Write queries so predicates and projections are pushed down correctly per connector type, avoiding accidental full scans and massive JDBC transfers.
• Materialization patterns: Decide when to live-federate vs when to snapshot/materialize JDBC data into Iceberg (e.g., CTAS/CTAS-like snapshots) for scalable joins.
• Kafka query limitations: Use Kafka reads for preview/exploration with awareness of non-repeatable offset windows.
• Operational safety via catalog isolation: Design separate catalogs and apply resource-group style access controls to reduce blast radius.

Quick Start

Use this skill to ask the AI to propose an efficient query plan for joining an Iceberg fact table with a small PostgreSQL dimension and a large PostgreSQL table by pre-materializing the large table into Iceberg first.