cosmos-dbt-core

You can convert a dbt Core project into an Airflow DAG by using Cosmos to assemble a DbtDag or DbtTaskGroup with specified project_path, profile_config, and execution_config, bridging dbt workflows into Airflow orchestration.

Orchestrating dbt workflows within Airflow is best achieved using Astronomer Cosmos to parse your dbt project and assemble it into an Airflow DAG or TaskGroup with modular configurations for rendering, execution, and project setup.

Cosmos works with Airflow 3.x and dbt Core projects. The integration applies to Cosmos 1.11 and above, supporting manifest parsing, execution configurations, and project assembly for robust data pipelines.

To configure RenderConfig and ExecutionConfig for a Cosmos DbtDag, define modular configuration objects that manage rendering profiles and execution environments, then pass them with proper operator_args during DAG assembly.

When bridging dbt Core into Airflow, safety checks include validating manifest availability, verifying project configurations, and checking the execution environment before deployment to ensure robust pipeline assembly.

Use a DbtTaskGroup instead of a DbtDag in Airflow when embedding your dbt project workflow within a larger existing DAG structure, whereas a DbtDag creates a standalone DAG specifically for your dbt pipeline.