Validate datasets and analyses for publication readiness with Polars.
npx skills add https://github.com/lowmason/agent-skills --skill validate-data-lowmason
Please help me install this Agent Skill. Skill: validate-data Source: https://github.com/lowmason/agent-skills/tree/main/validate-data Command: npx skills add https://github.com/lowmason/agent-skills --skill validate-data-lowmason
Use when QA-ing a dataset or an analysis before it is shared, published, or fed downstream — the last gate before a number leaves your laptop. Covers two entry points: (1) dataset QA (schema/dtype contracts, unexpected nulls, key uniqueness, duplicate rows, cardinality and value-range sanity, parquet round-trip integrity) and (2) analysis QA (re-run determinism, benchmark reconciliation, units/scale/sign, whether the conclusions are actually supported by the data, and coverage/selection/survivorship/revision bias). Polars-first; tuned to BLS pipelines (QCEW/CES/JOLTS), NumPyro/PyMC nowcasts, and as-of/vintage correctness. Trigger on: "is this ready to publish", "sanity-check this dataset/parquet", "review my analysis", "do the numbers reconcile", "why doesn't this match the official total", validating an ETL output, pre-publish review, reproducibility / "I can't reproduce yesterday's run", silent cache or fallback masking a failure, a coverage ratio that looks too clean, a decomposition whose components don't add up, future leakage past a data's knowability boundary, or a claim that a result is "fine" without an independent check. Always consult before signing off on data or an analysis — these checks are the ones agents skip unprompted.
Run a one-time validation pass on your dataset or analysis and emit a structured validation report.
High-intent search queries and answers about installing and using this skill.
Dataset validation checks schema and dtype contracts, detects unexpected nulls, enforces key uniqueness, and guards against duplicate rows to ensure data is ready for publication.
Reproducibility failures often stem from silent cache fallbacks, unpinned data vintages, or non-deterministic execution. Analysis validation enforces strict prerequisites and re-run determinism to certify consistency.
Analysis QA evaluates methodology by checking coverage, selection, survivorship, and revision bias, while validating knowability boundaries to detect future leakage past a dataset's effective publication date.
Yes, the validation process is Polars-first and supports parquet round-trip integrity checks, verifying that dataframe serialization and deserialization preserve schema and data fidelity.
Benchmark reconciliation checks verify that decomposed components add up correctly and validate unit-consistency, scale, and sign to resolve mismatches against official totals from pipelines like QCEW or CES.
Run a one-time validation pass to emit a structured report summarizing schema integrity, null detection, uniqueness checks, and methodology bias evaluations for pre-publish guardrails.
djpTester44Generate Pydantic models from Parquet and YAML schemas and validate Polars DataFrames.
dfinityServe cryptographically verified query responses using Merkle trees and BLS signatures on the Internet Computer.
ovachieverBuild, fit, and evaluate Bayesian models with PyMC using MCMC sampling.
TheCraigHewittAudit domains for SEO issues and generate prioritized fix reports.
