Track and visualize data flow across pipelines and ETL processes.
npx skills add https://github.com/cdalsoniii/brightpath-coder --skill data-lineage
Please help me install this Agent Skill. Skill: Data Lineage Source: https://github.com/cdalsoniii/brightpath-coder/tree/main/.cursor/skills/data-lineage Command: npx skills add https://github.com/cdalsoniii/brightpath-coder --skill data-lineage
This Skill addresses the complexity of understanding how data moves through various systems, enabling users to track data origins, transformations, and destinations.
Use the data lineage skill to trace the flow of data for the 'customer_orders' table.
High-intent search queries and answers about installing and using this skill.
To trace data flow across ETL pipelines, this Skill maps the path data takes from source to destination by reading pipeline configurations and CDC configurations, generating a comprehensive visual graph of data origins and transformations.
Schema change impact analysis works by scanning query logs and schema registries to identify all dependent pipelines and reports. This allows you to assess downstream effects proactively and prevent breakages before modifications are applied.
Yes, you need read access to pipeline configurations, schema registries, query logs, and CDC configurations. These sources are required to build a comprehensive lineage graph and accurately track data freshness.
The best way to monitor data freshness is by continuously tracking pipeline configurations and CDC logs. This approach visualizes data transformations from source to destination and highlights the downstream effects of any pipeline modifications.
Yes, you can use this approach for impact analysis on critical data tables. When a schema change is proposed, it quickly identifies all dependent pipelines and reports, allowing for proactive adjustments and preventing breakages.
neo4j-contribConfigure Neo4j Kafka Connect sink and source pipelines with CDC query API options.
astronomerTrace downstream data lineage and assess impact across tables, DAGs, and dashboards.
tonone-aiDetect data quality issues and pipeline health across databases and warehouses.
jackspaceParse FCS files to extract events as NumPy arrays and metadata.
