data-quality-pipeline

Data quality governance for tabular datasets covers encoding repair, tidying, cleaning, outlier detection, and merge discipline. This skill automates that workflow from raw ingestion to a provenance manifest, generating a four-axis quality score for auditable ship artifacts.

A provenance manifest is an auditable artifact that records the data cleaning and transformation history. This skill generates a manifest alongside profile and data quality JSON files, creating a portable packet that enforces a ship gate for non-hallucinated research results.

Outlier detection and merge discipline are automated stages within this data cleaning pipeline. It processes tabular datasets by applying tidy checks and four-axis scoring, ensuring standardized inputs and preventing join errors before generating the final provenance packet.

This data quality pipeline operates without external dependencies, using a portable workflow compatible with standard tabular datasets. It performs encoding repair automatically and outputs auditable artifacts like Parquet and JSON profile files without requiring prior environment setup.

Standardizing incoming research cohorts requires enforcing a manifest-driven ship gate with explicit references. This skill provides that best practice by automating encoding repair, cleaning, and four-axis quality scoring to produce auditable, non-hallucinated dataset artifacts.