skills.rest
platyplus
platyplusOFFICIAL
Explore all skills by this author →
GitHub

designing-tdd-ingest

Validate row counts, taxonomy, and geo-spatial constraints before and after bulk record insertion.

Updated Jul 23, 2026
Search Tags:#tdd#validation#ingestion#data-integrity#idempotency#cairn
One-click install
npx skills add https://github.com/platyplus/cairn-plugin --skill designing-tdd-ingest
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: designing-tdd-ingest
Source: https://github.com/platyplus/cairn-plugin/tree/main/cairn/skills/designing-tdd-ingest
Command: npx skills add https://github.com/platyplus/cairn-plugin --skill designing-tdd-ingest

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill solves the problem of unreliable data ingestion where silent failures, schema drift, or duplicate records compromise data integrity in humanitarian and field operations.

Core Features & Use Cases

  • Assertion-First Discipline: Define expected row counts, taxonomy, and geo-spatial constraints before any data is processed.
  • Idempotent Loading: Ensures that re-running imports does not result in duplicate records by using deterministic IDs derived from natural keys.
  • Verification Loop: Provides a structured workflow to dry-run, validate, land, and re-verify data against live system states.
  • Use Case: Use this when importing weekly health facility survey data to ensure that coordinate formats are correct and that no duplicate entries are created during pipeline re-runs.

Quick Start

Use the designing-tdd-ingest skill to set up an assertion-first import for my new facility survey data.

Frequently Asked Questions about designing-tdd-ingest

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I prevent silent pipeline failures during data ingestion?▼

Prevent silent data ingestion failures by defining assertions for expected row counts, taxonomy, and geo-spatial constraints before processing. This assertion-first discipline validates data against live system states to ensure integrity.

How do I make bulk record insertion idempotent to avoid duplicate entries?▼

Make bulk record insertion idempotent by using deterministic IDs derived from natural keys. This ensures re-running imports does not result in duplicate records in your schema-driven database.

What is the best way to validate geo-spatial constraints before importing survey data?▼

The best way to validate geo-spatial constraints is using an assertion-first workflow that dry-runs, lands, and re-verifies data. This verifies coordinate formats against expected constraints before final insertion.

How do I set up a dry-run and re-verify workflow for importing health facility data?▼

Set up a dry-run and re-verify workflow by coordinating the lifecycle of landing and validating data. Dry-run the import, validate row counts and constraints, land the records, then re-verify against live states.

Can I use assertion-first ingestion for schema-driven databases without extra dependencies?▼

Yes, assertion-first ingestion works for schema-driven databases without dependencies. It facilitates validation of row counts and taxonomy directly before and after bulk record insertion to prevent schema drift.

Why does re-running my data pipeline create duplicate records in my database?▼

Re-running data pipelines creates duplicates when imports lack idempotency. Prevent this by deriving deterministic IDs from natural keys, ensuring repeated bulk insertions maintain data integrity without duplicating entries.

Related Skills

Explore More Data & Analytics Skills→
agulliagulli
482

data-pipeline

Automate data pipeline definition, validation, and orchestration with schema contracts.

Community
Advanced
Datus-aiDatus-ai
1.5k

transfer-reconciliation

Verify post-transfer row count parity and target table accessibility.

Official
Intermediate
apokamoapokamo
1

series-create

Generate and validate sequential GitHub Issue series plans as YAML files.

Community
Advanced
cheetah-alocheetah-alo

data-analytics

Implement Python ETL pipelines with schema validation and idempotent execution.

Community
Intermediate