skills.rest
JaviMontano
JaviMontanoCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

metodologia-data-engineering

Automate data platform architecture design for ingestion, storage, quality, and observability.

Updated Mar 31, 2026
Search Tags:#orchestration#ingestion#data-quality#data-engineering#lakehouse#lineage#cost-management
One-click install
npx skills add https://github.com/JaviMontano/metodologia-propuesta-agent-public --skill metodologia-data-engineering
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: metodologia-data-engineering
Source: https://github.com/JaviMontano/metodologia-propuesta-agent-public/tree/main/.claude/skills/data/data-engineering
Command: npx skills add https://github.com/JaviMontano/metodologia-propuesta-agent-public --skill metodologia-data-engineering

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Data engineering teams often struggle to design, communicate, and implement robust, end-to-end data platforms that span ingestion, orchestration, storage, quality, lineage, and cost management. This Skill provides a structured framework and reference patterns to architect data platforms that are scalable, auditable, and vendor-agnostic.

Core Features & Use Cases

  • Ingestion patterns and data contracts: defines ingestion strategies (batch, CDC, streaming) and formal data contracts to ensure data quality and clear ownership.
  • Orchestration and storage architecture: guides orchestration choices (Dagster, Airflow) and a lakehouse storage design (landing/curated/marts) with open formats like Iceberg or Delta.
  • Data quality, lineage, observability, and cost management: prescribes a unified approach to validation, lineage capture, monitoring, alerting, and cost attribution.
  • Broad coverage including edge cases: supports greenfield projects, legacy migrations, multi-cloud setups, real-time streaming, and compliance considerations.

Quick Start

Provide the system or project name as input to generate the end-to-end data platform architecture.

Frequently Asked Questions about metodologia-data-engineering

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I design a scalable data platform architecture for batch and streaming ingestion?▼

Designing scalable data platform architecture requires defining ingestion patterns like batch, CDC, and streaming alongside formal data contracts. This framework automates generating end-to-end architectures covering ingestion, storage, and quality across cloud environments.

What is the best way to implement a lakehouse storage design with open table formats?▼

Implementing a lakehouse storage design involves structuring data into landing, curated, and marts layers using open formats like Iceberg or Delta. This approach ensures scalable, auditable, and vendor-agnostic storage architecture.

How does OpenLineage lineage capture work within a data engineering platform?▼

OpenLineage lineage capture works by prescribing a unified approach to tracking data flow across your platform. This framework integrates lineage capture with data quality validation, monitoring, and alerting to ensure end-to-end observability.

Can I use this data engineering framework for legacy migrations and multi-cloud setups?▼

Yes, this data engineering framework supports greenfield projects, legacy migrations, multi-cloud setups, and real-time streaming. It provides reference patterns and compliance considerations to handle these specific edge cases.

How do I enforce data contracts and role-based access control in my data platform?▼

Enforcing data contracts and role-based access control ensures clear data ownership and security within your platform. This framework formalizes data contracts during ingestion and applies cost-aware guardrails with role-based access.

Related Skills

Explore More Data & Analytics Skills→
MakisuoMakisuo
1.6k

clickhousectl-local-dev

Install and configure a local ClickHouse environment with clickhousectl.

Community
Intermediate
davekim917davekim917
1

data-engineering

Design robust data pipelines with ETL/ELT, orchestration, and governance patterns.

Community
Advanced
AbsolutelySkilledAbsolutelySkilled
207

data-quality

Enforces data quality with validation, lineage tracking, and contracts.

Official
Advanced
vertivolatamvertivolatam
1

OVHCloud Backend Deployment

Deploy Flutter backends on OVHCloud with managed Kubernetes and PostgreSQL.

Official
Intermediate