skills.rest
camilooscargbaptista
camilooscargbaptistaCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

data-engineering

Assess data pipelines, ETL/ELT processes, and governance across batch and streaming systems.

7|Updated Mar 19, 2026
Search Tags:#data#dbt#etl#airflow#pipelines#data-quality
One-click install
npx skills add https://github.com/camilooscargbaptista/cto-toolkit --skill data-engineering-camilooscargbaptista
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-engineering
Source: https://github.com/camilooscargbaptista/cto-toolkit/tree/main/data-engineering
Command: npx skills add https://github.com/camilooscargbaptista/cto-toolkit --skill data-engineering-camilooscargbaptista

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Data teams often struggle to gain a holistic, validated view of their pipelines, quality controls, contracts, and governance across batch and streaming workloads. This skill provides a structured review that surfaces architectural gaps, data quality deficits, and compliance issues early in the lifecycle.

Core Features & Use Cases

  • Pipeline architecture & data quality checks: evaluates ingestion, transformation, and serving layers with idempotence, backfill strategies, and lineage awareness.
  • Data contracts & governance: verifies contract definitions, freshness, ownership, and change management to reduce breaking changes.
  • Dbt/Airflow/Spark stack guidance: assesses tool-specific patterns, tests, and documentation to improve reliability and observability.
  • Use Case: ideal for teams implementing compliant data platforms, or auditing ongoing data programs for scalability and resilience.

Quick Start

Provide your data pipeline details (tools, sources, and governance rules) to generate a complete data engineering review.

Frequently Asked Questions about data-engineering

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I audit data pipeline architecture for data quality and governance?▼

Data contracts are verified by checking contract definitions, freshness, ownership, and change management rules to reduce breaking changes and ensure governance compliance across batch and streaming pipelines.

Does this data pipeline review work with dbt, Airflow, and Snowflake?▼

Streaming data pipelines are assessed by verifying schema validation, lineage tracking, data quality tests, backfill strategies, and monitoring to ensure scalability and resilience across data lakes and warehouses.

How to review ETL processes for schema validation and lineage tracking?▼

Pipeline backfill strategies are evaluated by checking idempotence, data quality tests, lineage tracking, and monitoring to ensure reliable reprocessing and resilience across data lakes and warehouses.

Why does my data pipeline lack observability and compliance?▼

Data pipelines lack observability and compliance when missing schema validation, lineage tracking, data contracts, and governance rules, which a structured review surfaces early in the lifecycle.

Related Skills

ouakarouakar

data-scientist

Document production AI/ML stack usage, LLM calls, pipelines, and cost drivers.

Community
Advanced
AbsolutelySkilledAbsolutelySkilled
207

data-quality

Enforces data quality with validation, lineage tracking, and contracts.

Official
Advanced
ginkidaginkida
20

data-pipeline

Build and operate data pipelines with Airflow, Spark, and dbt.

Community
Advanced
juanlamadrid20juanlamadrid20

databricks-unity-catalog

Query Unity Catalog system tables for lineage, audit, and usage observability.

Community
Advanced