creating-openlineage-extractors

To capture Airflow lineage from unsupported operators, you build OpenLineage extractors that implement specific facets to define inputs and outputs. This approach enables end-to-end observability for third-party or custom operators.

An OpenLineage extractor in Airflow is a provider class that hooks into operator execution to capture data lineage. It emits inputs, outputs, and metadata facets to an OpenLineage-compatible catalog for pipeline observability.

Yes, you can add runtime lineage by evaluating the operator state after execution. Your custom OpenLineage extractor can dynamically determine and emit lineage based on runtime results rather than relying solely on static configurations.

Yes, you need Python development experience to create OpenLineage extractors. Implementing these extractors requires familiarity with Python, OpenLineage facets, and Airflow provider extractors to properly define the lineage classes.

The best way to register a custom OpenLineage extractor is to first implement the extractor class handling your operator's source and target tables, then configure it within your Airflow OpenLineage setup to automatically emit lineage data.