Enforces data integrity and provenance across multi-source extraction pipelines.
npx skills add https://github.com/danbri/forgetmenot --skill data-quality-danbri
Please help me install this Agent Skill. Skill: data-quality Source: https://github.com/danbri/forgetmenot/tree/main/skills/data-quality Command: npx skills add https://github.com/danbri/forgetmenot --skill data-quality-danbri
This skill addresses the degradation of data corpora caused by noisy, multi-source ingestion, preventing the common pitfall of creating unreliable hybrid datasets that mask underlying source errors.
Use the data-quality skill to audit the current extraction pipeline by running the established anchor test suite against the latest data.
High-intent search queries and answers about installing and using this skill.
Data integrity during multi-source ingestion is validated through systematic cross-checking, anchor-case validation, and stable identifier usage to maintain high-confidence corpora. This framework prevents noisy ingestion from masking underlying source errors.
Diagnosing data extraction gaps requires categorizing them as crawl-misses, extractor-misses, or upstream source issues. This diagnostic discipline prevents superficial fixes by addressing the actual root cause of the API extraction failure.
Cross-extractor validation enforces independent extraction paths for the same source, such as HTML versus API, to identify and resolve discrepancies. This ensures reliable data extraction across Wikidata and gov.uk datasets.
Anchor-based testing implements known-good test cases to ensure pipeline stability and provide fast, noisy failure alerts. This system quickly exposes data quality regression when run against the latest extracted data.
Python is required to audit data provenance across UK Parliament APIs. The skill uses Python scripts to apply provenance standards and run established anchor test suites against complex data extraction tasks.
Hybrid datasets contain masked errors due to the degradation caused by noisy, multi-source ingestion. Applying rigorous provenance standards and cross-checking prevents the creation of unreliable hybrid datasets.
danbriExtract and query ministerial roles from GOV.UK content via SPARQL.
xjtulycBuild RDF knowledge graphs from research metadata and query them with SPARQL.
Timmy6942025Extract structured KYC fields from investor onboarding documents into JSON records.
okgoogle13Run sequential diagnostic scripts and report project health summaries.
