creating-openlineage-extractors

Name: creating-openlineage-extractors
Availability: InStock
Author: Astronomer

Implement OpenLineage extractors for Airflow operators lacking built-in lineage.

419|61|Updated Jan 13, 2026

Search Tags:#python #data #airflow #lineage #extractors #openlineage

Ready to run

npx skills add https://github.com/astronomer/agents --skill creating-openlineage-extractors

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: creating-openlineage-extractors
Source: https://github.com/astronomer/agents/tree/main/skills/creating-openlineage-extractors
Command: npx skills add https://github.com/astronomer/agents --skill creating-openlineage-extractors

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill helps you build OpenLineage extractors to capture data lineage from Airflow operators that do not provide built-in lineage support, enabling end-to-end observability.

Core Features & Use Cases

OpenLineage extraction: Implement extractors for custom or third-party operators to emit inputs and outputs.
Runtime enhancements: Add runtime lineage based on operator state after execution.
Use Case: When you deploy a custom operator and need lineage pushed to your OpenLineage-compatible catalog, this skill provides a pattern to implement the extractor.

Quick Start

Start by creating a MyCustomOperator extractor that handles source_table and target_table, then register it with your Airflow OpenLineage setup.

Frequently Asked Questions about creating-openlineage-extractors

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I capture Airflow lineage from custom operators without built-in support?▼

To capture Airflow lineage from unsupported operators, you build OpenLineage extractors that implement specific facets to define inputs and outputs. This approach enables end-to-end observability for third-party or custom operators.

What is an OpenLineage extractor in Airflow?▼

An OpenLineage extractor in Airflow is a provider class that hooks into operator execution to capture data lineage. It emits inputs, outputs, and metadata facets to an OpenLineage-compatible catalog for pipeline observability.

Can I add runtime lineage to an Airflow operator after it executes?▼

Yes, you can add runtime lineage by evaluating the operator state after execution. Your custom OpenLineage extractor can dynamically determine and emit lineage based on runtime results rather than relying solely on static configurations.

Do I need Python development experience to create OpenLineage extractors?▼

Yes, you need Python development experience to create OpenLineage extractors. Implementing these extractors requires familiarity with Python, OpenLineage facets, and Airflow provider extractors to properly define the lineage classes.

What's the best way to register a custom OpenLineage extractor with Airflow?▼

The best way to register a custom OpenLineage extractor is to first implement the extractor class handling your operator's source and target tables, then configure it within your Airflow OpenLineage setup to automatically emit lineage data.