auditing-source-provenance

What problem does it solve?

This Skill audits data ingest pipelines to ensure every record can be traced back to a specific upstream source, version, and pull time. It catches provenance gaps before they become unreproducible datasets, compliance risks, or broken audit trails.

Core Features & Use Cases

• Source inventorying: Identifies every upstream repo, URL, API, S3 object, or scrape touched by the pipeline.
• Provenance completeness checks: Verifies that each record carries the required source, version, hash, timestamp, adapter, license, and row locator fields.
• Transform survival auditing: Checks that provenance survives merges, joins, dedupes, groupbys, and derived-field lookups without being overwritten or dropped.
• Compliance and recovery guidance: Helps backfill partial provenance where possible and adds CI checks to prevent regressions.
• Use case: If you are combining multiple external registries into one corpus, this Skill will flag floating refs, missing hashes, and transform-time stamping before the dataset ships.

Quick Start

Ask the Skill to audit your ingest script and a sample output row for missing, partial, or lost provenance.