skills.rest
wesleyosantos91
wesleyosantos91COMMUNITY
Explore all skills by this author →
GitHub

data-platform

Design AWS data platform architectures for ETL/ELT and data lake scenarios.

6|1|Updated Mar 30, 2026
Search Tags:#aws#streaming#etl#data-engineering#lakehouse#architecture-design#data-platform
One-click install
npx skills add https://github.com/wesleyosantos91/multi-agents --skill data-platform-wesleyosantos91
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-platform
Source: https://github.com/wesleyosantos91/multi-agents/tree/main/devin/.devin/skills/data-platform
Command: npx skills add https://github.com/wesleyosantos91/multi-agents --skill data-platform-wesleyosantos91

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Helps you turn a data engineering goal into a clear, AWS-aligned platform design by choosing the right services and pipeline approach.

Core Features & Use Cases

  • Pipeline architecture guidance: Recommends how to structure ETL/ELT pipelines, including batch vs streaming tradeoffs.
  • Platform pattern selection: Supports data lake and lakehouse design decisions and the flow of data across services.
  • AWS service recommendation: Helps decide among Glue, EMR Serverless, EMR on EC2, Lambda, and Step Functions, and how to integrate Kinesis, MSK, Athena, Redshift, and Lake Formation.

Quick Start

Ask the assistant to analyze your target dataset and propose a complete AWS data platform architecture with recommended services for your pipelines.

Frequently Asked Questions about data-platform

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I design an AWS data platform architecture for ETL pipelines?▼

Designing an AWS data platform architecture involves selecting appropriate compute and orchestration services like Glue, EMR, Lambda, and Step Functions to structure your ETL/ELT pipelines and manage data flow.

What is the best way to choose between batch and streaming approaches on AWS?▼

Choosing between batch and streaming approaches requires evaluating tradeoffs based on your target dataset context, which then dictates selecting AWS ingestion tools like Kinesis or MSK for your data platform.

Can I use AWS Lambda and Step Functions for data lake orchestration?▼

Yes, AWS Lambda and Step Functions can be used for data lake orchestration, serving as compute and orchestration options to process and coordinate ETL pipelines alongside other AWS analytics services.

How do I select AWS services for a lakehouse architecture?▼

Selecting AWS services for a lakehouse architecture involves combining storage and querying tools like Athena and Redshift with governance from Lake Formation, integrating them through a structured platform design.

When should I not use AWS Glue for data engineering pipelines?▼

You should evaluate alternatives to AWS Glue when your data engineering pipelines require specialized compute scaling or serverless processing that might be better handled by EMR Serverless or EMR on EC2.

Related Skills

Explore More Data & Analytics Skills→
KirillKlemKirillKlem
1

architecture-spec-writer

Generate architecture specification documents with goals, interfaces, and rollout plans.

Community
Advanced
AdawoduAdawodu

engineering-data-engineer

Design and implement ETL/ELT pipelines with Apache Spark and dbt.

Community
Advanced
AradotsoAradotso
5

terraform-data-engineering-infrastructure

Provision AWS data engineering infrastructure with Terraform.

Official
Advanced
HouseGarofaloHouseGarofalo
3

data-engineer-agent

Design, implement, and validate data pipelines with Spark, Airflow, and Kafka.

Official
Advanced