skills.rest
ivanshamaev
ivanshamaevCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

trino-airflow-orchestration

Orchestrate idempotent Trino SQL pipelines in Apache Airflow with TrinoOperator.

14|1|Updated May 5, 2026
Search Tags:#orchestration#trino#dbt integration#iceberg#apache airflow#incremental loads#idempotent pipelines
One-click install
npx skills add https://github.com/ivanshamaev/de-agent-skills --skill trino-airflow-orchestration
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: trino-airflow-orchestration
Source: https://github.com/ivanshamaev/de-agent-skills/tree/main/group_skills/trino_group_skills/trino_airflow_orchestration
Command: npx skills add https://github.com/ivanshamaev/de-agent-skills --skill trino-airflow-orchestration

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill solves the problem of reliably scheduling and operating Trino SQL data pipelines with correct connection management, retries, idempotency, and operational observability in Apache Airflow.

Core Features & Use Cases

  • Airflow-to-Trino integration: Use TrinoOperator and TrinoHook with a configurable trino_conn_id (including example connection setup).
  • Idempotent DAG patterns: Make reruns safe using partition-aware DELETE+INSERT and MERGE upserts with deterministic run semantics.
  • Incremental, partition-aware orchestration: Drive loads using Airflow logical dates ({{ ds }}), guard with freshness checks, and include maintenance steps like ANALYZE.
  • Production operations: Provide SLA monitoring hooks, retry configuration guidance, and anti-pattern avoidance for common failure modes.

Quick Start

Use the trino-airflow-orchestration skill to generate an Airflow DAG that runs an idempotent Trino MERGE incremental load for a given {{ ds }} partition and includes SLA monitoring and retry settings for the Trino tasks.

Frequently Asked Questions about trino-airflow-orchestration

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I build idempotent Trino pipelines in Apache Airflow?▼

To build idempotent Trino pipelines in Apache Airflow, use TrinoOperator with partition-aware DELETE+INSERT or MERGE upserts driven by logical dates. This ensures deterministic reruns for incremental ETL loads without duplicating data.

What is the best way to configure Airflow TrinoOperator connections for production?▼

The best way to configure Airflow TrinoOperator connections is by defining a proper trino_conn_id using TrinoHook. Combine this with SLA monitoring hooks and retry configurations to maintain production operational observability and avoid common failure modes.

How do I orchestrate incremental loads in Trino using Airflow logical dates?▼

Orchestrate incremental loads in Trino using Airflow logical dates by scoping partitions with the {{ ds }} template. Guard your pipeline with freshness checks and include maintenance steps like ANALYZE to update table statistics.

Can I use dbt-to-Trino orchestration within Apache Airflow?▼

Yes, you can execute dbt-to-Trino orchestration within Apache Airflow using TrinoOperator and TrinoHook. This integration supports partition-aware ETL runs combined with operational monitoring and deterministic retry logic.

Why do my Airflow Trino incremental loads duplicate data on reruns?▼

Airflow Trino incremental loads duplicate data on reruns when lacking idempotent transaction patterns. Implementing partition-aware MERGE or DELETE+INSERT operations scoped by Airflow logical dates ensures deterministic execution and prevents duplicates.

Related Skills

Explore More Software Engineering Skills→
ivanshamaevivanshamaev
14

trino-modern-data-stack-reference-architecture

Bootstrap a Trino lakehouse with Kafka, Iceberg, dbt, Airflow, and Superset.

Community
Advanced
ivanshamaevivanshamaev
14

airflow-starrocks-backfill

Automate StarRocks historical backfills with Airflow using Broker Load or INSERT OVERWRITE.

Community
Advanced
ivanshamaevivanshamaev
14

trino-airflow-lakehouse-pipelines

Implement Airflow-orchestrated Iceberg Lakehouse ETL workflows with Trino SQL across Bronze, Silver, and Gold layers.

Community
Advanced
miptah21miptah21
2

pipeline-orchestration

Design and refactor data pipeline orchestration for Dagster, Prefect, and Luigi.

Community
Intermediate