skills.rest
Adelie-Squad
Adelie-SquadOFFICIAL
Explore all skills by this author →
GitHub

Data Engineer Agent

Profile, clean, and transform raw datasets into analysis-ready tables with pandas.

17|1|Updated Feb 3, 2026
Search Tags:#pipeline#pandas#etl#data-transformation#data-quality#data-engineering#cleaning
One-click install
npx skills add https://github.com/Adelie-Squad/solopreneur-team-agents --skill data-engineer-agent-adelie-squad
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: Data Engineer Agent
Source: https://github.com/Adelie-Squad/solopreneur-team-agents/tree/main/assets/agents/engineering/data-engineer
Command: npx skills add https://github.com/Adelie-Squad/solopreneur-team-agents --skill data-engineer-agent-adelie-squad

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Takes raw, messy datasets and handles profiling, cleaning, transformation, and quality checking so teams avoid manual scripting and inconsistent ETL results.

Core Features & Use Cases

  • Profiling-first cleaning: summarize schema, remove duplicates, handle missing values, fix types, and standardize text with pandas utilities.
  • Transformations & aggregations: create derived datetime columns, categorize and encode dimensions, join external tables, and aggregate data for reporting or analytics needs.
  • Quality gating & handoff: run rule-based validators for non-null, uniqueness, and ranges, then document artifacts and pass cleaned data to analysts or backend engineers.

Quick Start

Ask the Data Engineer Agent to clean and transform your raw_data with the provided output requirements so it returns a validated dataset ready for analytics.

Frequently Asked Questions about Data Engineer Agent

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I clean and profile raw datasets for analysis readiness?▼

The Data Engineer Agent cleans raw datasets by profiling schemas, removing duplicates, handling missing values, fixing types, and standardizing text with pandas utilities. It outputs validated, analysis-ready tables.

Can I create derived features and aggregate data during ETL transformation?▼

Yes, ETL transformation includes creating derived datetime columns, categorizing and encoding dimensions, joining external tables, and aggregating data. These pandas-based transformations prepare datasets for reporting or analytics needs.

What is the best way to enforce data quality rules before handing off datasets?▼

Enforce data quality rules by running rule-based validators for non-null, uniqueness, and range constraints. This quality gating checks pandas dataframes, documents artifacts, and ensures clean handoff to analysts or backend engineers.

How do I handle type casting and duplicate management in pandas data pipelines?▼

Handle type casting and duplicate management in pandas data pipelines by applying built-in cleaners that enforce schema types and remove duplicate records. This standardizes raw inputs into consistent, validated tables for downstream engineering workflows.

Does this ETL pipeline approach work without external dependencies?▼

Yes, this ETL pipeline approach works without external dependencies. It relies entirely on internal pandas-based cleaners, transformers, and quality checkers to process raw datasets into handoff-ready tables for analytics, backend, or reporting tasks.

Related Skills

Explore More Data & Analytics Skills→
rar-filerar-file
42

Data Analyst

Analyze datasets to extract insights and present actionable findings.

Community
Intermediate
fatihkanfatihkan
7

data-analytics

Analyze datasets and generate analytics reports with visuals.

Community
Advanced
underPressueunderPressue

senior-data-engineer

Orchestrate, validate, and optimize ETL/ELT data pipelines in Python.

Community
Intermediate
l3ochol3ocho

data-integrity-audit

Assess data integrity across pipelines for schema validity and dbt compliance.

Community
Advanced