data-pipeline

Name: data-pipeline
Availability: InStock
Author: uabbasi

Automate charity evaluation data pipelines from extraction to narrative export.

Updated Feb 9, 2026

Search Tags:#debugging #versioning #pipeline #etl #data-processing #charity-evaluation

Ready to run

npx skills add https://github.com/uabbasi/good-measure-giving --skill data-pipeline-uabbasi

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: data-pipeline
Source: https://github.com/uabbasi/good-measure-giving/tree/main/.claude/skills/data-pipeline
Command: npx skills add https://github.com/uabbasi/good-measure-giving --skill data-pipeline-uabbasi

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Data pipeline for charity evaluation is complex and time-consuming; this Skill provides a modular, repeatable framework to extract, derive, reconcile, and export data with narrative generation and versioning, simplifying development and debugging workflows.

Core Features & Use Cases

End-to-end 4-stage V2 pipeline: crawl, process_data, process_baseline, process_rich with a common wrapper.
Deterministic data handling, source reconciliation, and versioned exports to enable reproducible analytics.
Guidance for developers and data engineers building, debugging, or extending collectors, scrapers, and database workflows.

Quick Start

Run the full V2 data-pipeline on pilot_charities.txt using the run_v2.sh script.

Frequently Asked Questions about data-pipeline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I orchestrate an end-to-end data pipeline for charity evaluation?▼

You orchestrate an end-to-end charity evaluation data pipeline by running the modular 4-stage V2 workflow, which automates extraction, derivation, reconciliation, and narrative export. The pipeline provides deterministic data handling and versioned outputs for reproducible analytics.

What are the stages of an ETL data processing pipeline for multi-source reconciliation?▼

The ETL data processing pipeline consists of four sequential stages: crawl, process_data, process_baseline, and process_rich. These stages automate extraction, derivation, multi-source reconciliation, and narrative generation through a common wrapper.

How do I debug and extend custom web scrapers within an ETL pipeline?▼

You debug and extend web scrapers within the ETL pipeline using its modular script structure and clear entry points. This framework simplifies development workflows for data engineers building or modifying collectors and database integrations.

How do I generate versioned exports with narrative generation from processed data?▼

You generate versioned exports with narrative generation by completing the pipeline's final processing stages. The workflow integrates versioning directly into the export process, ensuring deterministic data handling and enabling reproducible analytics.

Can I run an automated data pipeline without external dependencies?▼

Yes, you can run the automated data pipeline without external dependencies. It operates independently using a modular script structure and can be executed end-to-end on a pilot input file via a designated shell script entry point.

What is the best way to automate reproducible analytics workflows for charity data?▼

The best way to automate reproducible analytics for charity data is using a deterministic 4-stage pipeline that handles multi-source reconciliation and integrated versioning. This modular approach ensures consistent data handling from extraction to narrative export.