terraform-data-engineering-infrastructure

Provision AWS S3, EC2, and IAM resources for data pipelines using Terraform.

5|1|Updated May 16, 2026
Ready to run
npx skills add https://github.com/reason-machines/data-skills --skill terraform-data-engineering-infrastructure-reason-machines
💬 Prompt for ChatGPT / Claude Chat▼
Please help me install this Agent Skill.
Skill: terraform-data-engineering-infrastructure
Source: https://github.com/reason-machines/data-skills/tree/main/skills/terraform-data-engineering-infrastructure
Command: npx skills add https://github.com/reason-machines/data-skills --skill terraform-data-engineering-infrastructure-reason-machines

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Manually creating and configuring AWS infrastructure for data engineering workloads is error-prone, inconsistent across environments, and hard to reproduce. This Skill provides Infrastructure-as-Code patterns to declaratively provision and manage data platform resources with Terraform. ## Core Features & Use Cases - S3 Data Lake Provisioning: Create versioned, access-blocked S3 buckets organized into raw, processed, and curated layers with lifecycle policies. - EC2 and IAM Configuration: Deploy data processing instances with security groups, IAM roles, and instance profiles granting scoped S3 access. - Multi-Environment Management: Use variables, outputs, and remote state backends to replicate infrastructure across dev, staging, and prod. - Use Case: A data engineer needs a reproducible environment for an ETL pipeline. They define the S3 buckets, processing instance, and IAM policies in Terraform files, then run plan and apply to create identical infrastructure in every environment. ## Quick Start Ask the AI to generate a Terraform configuration that provisions an S3 data lake bucket and an EC2 instance with IAM permissions to read and write objects in that bucket.

Frequently Asked Questions about terraform-data-engineering-infrastructure

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I provision AWS infrastructure for data engineering with Terraform?▼

Define S3 buckets, EC2 instances, and IAM roles in .tf files, then run terraform init, plan, and apply. Terraform creates the resources declaratively and tracks them in a state file for repeatable updates and teardown.

How to create an S3 data lake with Terraform?▼

Define aws_s3_bucket resources for raw, processed, and curated layers, enable versioning, and block public access. Add aws_s3_bucket_lifecycle_configuration rules to transition old data to Glacier or expire it after a set number of days.

What IAM permissions does Terraform need for AWS provisioning?▼

The IAM user needs S3, EC2, and IAM access to create buckets, instances, and roles. For learning environments full access policies work, but production setups should use least-privilege policies scoped to required actions.

Why does Terraform fail with BucketAlreadyExists error?▼

S3 bucket names must be globally unique across all AWS accounts. Fix it by choosing a different name or appending a random suffix using the random_id resource to guarantee uniqueness.

How do I manage Terraform state for team environments?▼

Store state remotely in a versioned S3 bucket configured as a Terraform backend instead of keeping local terraform.tfstate files. This enables team collaboration, state locking, and recovery from accidental deletion.

Can Terraform import existing AWS resources?▼

Yes, use terraform import with the resource address and existing resource ID, such as a bucket name or EC2 instance ID. This brings unmanaged resources under Terraform control without recreating them.