annotating-task-lineage

To add data lineage to Airflow tasks lacking native OpenLineage extractors, manually annotate operators with explicit input (inlets) and output (outlets) datasets. This provides table-level lineage visibility for ETL/ELT DAGs by explicitly declaring dataset reads and writes.

You can use OpenLineage Dataset objects or native Airflow Asset and Dataset types for Airflow inlets and outlets. OpenLineage Dataset objects are recommended for annotating Airflow tasks, while Airflow Assets work for versions 2.4+ and 3+.

Manually declare inlets and outlets only as a fallback when higher-priority OpenLineage extractors or get_openlineage_facets_on_* methods are unavailable. OpenLineage precedence rules dictate using native extractors first for automatic lineage tracking.

Airflow pipeline datasets may not show up in the lineage UI if operators lack built-in OpenLineage extractors. Annotate tasks with explicit inlet and outlet datasets to ensure table-level lineage visibility and audit upstream-to-downstream data flow.

Yes, you can track table-level lineage for ETL orchestration in Airflow without custom extractors by manually defining inlets and outlets. This lowest-priority fallback method ensures dataset reads and writes appear in the lineage UI.

Yes, annotating task lineage works with both Airflow Assets and OpenLineage Datasets. You can use either type to declare explicit inputs and outputs for Airflow operators, ensuring dataset lineage visibility across your ETL pipelines.