annotating-task-lineage

Annotate Airflow task inlets and outlets with OpenLineage Datasets or Airflow Assets.

3|Updated Jul 22, 2025

Search Tags:#data lineage #etl #airflow #astro #openlineage #inlets #outlets

Ready to run

npx skills add https://github.com/minyeamer/linkmerce --skill annotating-task-lineage-minyeamer

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: annotating-task-lineage
Source: https://github.com/minyeamer/linkmerce/tree/main/.agents/skills/annotating-task-lineage
Command: npx skills add https://github.com/minyeamer/linkmerce --skill annotating-task-lineage-minyeamer

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps you document and visualize data lineage for Airflow tasks so downstream and upstream data dependencies are transparent, even when operators do not have automatic OpenLineage extraction.

Core Features & Use Cases

Manual lineage via inlets/outlets: Define what each task reads (inlets) and writes (outlets) to produce lineage events.
Works with multiple dataset types: Use OpenLineage Dataset objects, Airflow 3+ Assets, or Airflow 2.4+ Datasets.
Clear operator coverage strategy: Apply inlets/outlets as the lowest-priority fallback when no OpenLineage extractor or built-in method exists.

Quick Start

Configure your Airflow DAG tasks by setting inlets and outlets (as OpenLineage Datasets, Airflow Assets, or Airflow Datasets) so the lineage appears in the Astro Lineage view.

Frequently Asked Questions about annotating-task-lineage

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I add data lineage to Airflow tasks without automatic OpenLineage extraction?▼

To add data lineage without automatic extraction, manually annotate Airflow tasks by defining OpenLineage Dataset objects, Airflow 3+ Assets, or Airflow 2.4+ Datasets and assigning them to the operator inlets and outlets properties.

When do I need to manually set inlets and outlets for Airflow lineage?▼

You need to manually set inlets and outlets when an Airflow DAG operator reads from or writes to external systems and lacks a built-in OpenLineage extractor, serving as the lowest-priority fallback to capture lineage visibility.

What dataset types can I use for Airflow inlets and outlets to show data lineage?▼

You can use OpenLineage Dataset objects, Airflow 3+ Assets, or Airflow 2.4+ Datasets as inlet and outlet types to document task data lineage and make dependencies visible in Astro Lineage.

Can I track multiple inputs and outputs in Airflow data lineage?▼

Yes, you can track multiple inputs and outputs by assigning multiple dataset objects to the inlets and outlets of a single Airflow operator, covering multi-input and multi-output transformations or data exports.

Does annotating Airflow task lineage work with databases and object storage?▼

Yes, annotating task lineage applies to DAG operators that read from and write to external systems including databases and object storage, resolving missing lineage visibility by mapping those sources as inlets and outlets.

Why is my data lineage missing in Astro Lineage for custom Airflow operators?▼

Data lineage is missing in Astro Lineage when operators lack automatic OpenLineage extraction, requiring you to manually define and assign OpenLineage Datasets or Airflow Assets to the task inlets and outlets.