senior-data-engineer

Data pipeline design with data contracts involves creating templates that capture schema, semantics, ownership, and SLAs to ensure reliable data ingestion. This skill generates schema definitions, transformation plans, and data quality checklists to standardize pipeline governance and enforce data contracts.

A data contract in ETL pipeline design is a formal agreement capturing schema, semantics, ownership, SLAs, and change management rules. It ensures data pipelines remain predictable and observable by standardizing inputs and transformation expectations across data engineering teams.

Diagnosing a data quality incident involves running structured checklists for nulls, ranges, uniqueness, and schema drift. This skill helps data engineering teams produce operational runbooks, iterate schema evolution plans, and document recovery steps for freshness regressions.

Yes, you can profile a local CSV dataset for data quality checks using a lightweight CSV profiler. It provides local dataset inspection to identify nulls, ranges, and uniqueness issues before integrating datasets into broader ETL or ELT data pipelines.

The best way to plan schema evolution and handle pipeline failures is by generating structured operational runbooks and idempotent processing checklists. This skill produces backfill plans, failure handling steps, and schema evolution guidelines to keep data pipelines recoverable.

Data observability works by applying pragmatic checks for schema drift, lineage, and freshness metrics within data pipelines. This skill generates quality checklists and operational runbooks that make pipeline health predictable, observable, and recoverable for data engineering teams.