trino-modern-data-stack-reference-architecture

What problem does it solve?

This Skill provides an end-to-end, production-oriented reference architecture for a Trino-based Modern Data Stack so you can quickly bootstrap a lakehouse platform without manually stitching together Kafka, Iceberg, Trino, dbt, Airflow, BI, and observability.

Core Features & Use Cases

• End-to-end component wiring (single-machine Docker Compose): Kafka + MinIO/S3 + Hive Metastore + Trino + Airflow + dbt + Superset + Prometheus/Grafana, with ready-to-run configurations for how the systems connect.
• Medallion lakehouse bootstrapping (Bronze/Silver/Gold DDL): Creates Iceberg schemas and example tables for raw ingest, cleansed typed models, and aggregated business-ready outputs.
• Operational pipeline blueprints: Includes Airflow DAG patterns for Kafka→Bronze ingest, Bronze→Silver transformations with MERGE/dedup logic, and Gold refresh plus routine Iceberg maintenance (compaction, snapshot expiration, orphan file cleanup).
• Transformation and serving integration: Provides a dbt project layout with Trino profiles and example models, plus Superset dataset configuration guidance and monitoring alert rules.

Quick Start

Ask the agent to deploy the full docker-compose stack, bootstrap the Iceberg Bronze/Silver/Gold schemas in Trino, run the dbt models via Airflow, and trigger the example Airflow DAGs while validating Trino/Superset/Grafana connectivity.