dataset-supply-chain-security

What problem does it solve?

Datasets can be poisoned, tampered, or misrepresented across public hubs and RAG pipelines, risking degraded model performance and security breaches.

Core Features & Use Cases

• Pinning and hashing: enforce commit/content hashes for all external datasets and verify on download.
• Vetting and provenance: formal checks for publishers, licenses, and provenance before ingestion.
• Ingestion governance: logging, internal mirroring, and sandboxing of new datasets.
• Use Case: A model team audits a HuggingFace repo to ensure the dataset version matches the audited snapshot and logs access through a central registry.

Quick Start

Pin all external datasets to immutable commit or content hashes and verify them at download time before ingestion.