skills.rest
phamhoangtuan
phamhoangtuanCOMMUNITY
Explore all skills by this author →
GitHub

data-engineer

Enforces DAMA-DMBOK data engineering best practices for building Python pipelines.

Updated Feb 16, 2026
Search Tags:#data quality#data pipeline#data engineering#data governance#dama-dmbok#test-first development
One-click install
npx skills add https://github.com/phamhoangtuan/learn-de-opencode --skill data-engineer-phamhoangtuan
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-engineer
Source: https://github.com/phamhoangtuan/learn-de-opencode/tree/main/.opencode/skills/data-engineer
Command: npx skills add https://github.com/phamhoangtuan/learn-de-opencode --skill data-engineer-phamhoangtuan

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) and scripts (resource) components.

What problem does it solve?

This Skill enforces best practices for building robust, maintainable, and high-quality data pipelines, ensuring adherence to data governance, security, and testing standards.

Core Features & Use Cases

  • Enforces DAMA-DMBOK 2.0: Integrates data engineering principles for code quality, governance, security, and testing.
  • Test-First Development: Mandates writing tests before implementation to ensure reliability.
  • Data Quality & Governance: Implements validation across 6 quality dimensions and schema enforcement.
  • Metadata & Lineage: Captures technical, business, operational, and lineage metadata for traceability.
  • Use Case: Develop a new data pipeline for customer transaction data, ensuring all code is modular, well-tested, and adheres to data quality and security policies defined in the constitution.

Quick Start

Develop a new data pipeline for customer transaction data, ensuring all code is modular, well-tested, and adheres to data quality and security policies defined in the constitution.

Frequently Asked Questions about data-engineer

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I build data pipelines that enforce DAMA-DMBOK data governance and quality standards?▼

To build data pipelines with DAMA-DMBOK data governance, this Skill enforces best practices across six data quality dimensions, schema validation, and comprehensive metadata lineage capture to ensure robust and secure processing.

What is test-first development for data pipelines and how does it improve reliability?▼

Test-first development for data pipelines requires writing Pytest tests before implementation to validate logic early. This approach ensures robust pipeline creation by mandating strict adherence to data quality and governance standards.

Can I use Pandas or Polars with DuckDB for Parquet and JSON data pipeline processing?▼

Yes, you can use Pandas, Polars, and DuckDB for Parquet, JSON, and CSV data pipeline processing. The Skill integrates these Python 3.11+ frameworks to enforce schema validation and data quality standards.

What's the best way to capture lineage and metadata for customer transaction data pipelines?▼

The best way to capture lineage and metadata for transaction data pipelines is enforcing DAMA-DMBOK principles. This Skill automatically captures technical, business, operational, and lineage metadata to ensure full traceability.

Does this data pipeline approach support schema enforcement and data quality validation?▼

Yes, this data pipeline approach supports schema enforcement and validates data across six quality dimensions. It mandates test-first development with Pytest to ensure all code meets strict data governance and security policies.

Related Skills

Explore More Software Engineering Skills→
bettercallfanbettercallfan

dataset-onboarding-graph-preprocess

Convert raw phone network data files into structured graph tables.

Community
Advanced
CyapstayeCyapstaye
9

test-driven-development

Enforce the Red-Green-Refactor cycle with failing tests before implementation.

Community
Advanced
TeiNamTeiNam
12

python-data-analysis

Automate Python data analysis workflows with pandas, polars, and duckdb.

Community
Advanced
shafibabarshafibabar
1

data-classification

Design data-classification schemes mapping sensitivity levels to security controls.

Community
Advanced