skills.rest
ivanshamaev
ivanshamaevCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

infra-docker-best-practices

Apply multi-stage Dockerfile patterns and Trivy scanning to data engineering builds.

14|1|Updated May 5, 2026
Search Tags:#data engineering#docker#dockerfile#multi-stage builds#image hardening#ci security scanning#buildkit secrets
One-click install
npx skills add https://github.com/ivanshamaev/de-agent-skills --skill infra-docker-best-practices
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: infra-docker-best-practices
Source: https://github.com/ivanshamaev/de-agent-skills/tree/main/group_skills/infra_dataops_group_skills/infra_docker_best_practices
Command: npx skills add https://github.com/ivanshamaev/de-agent-skills --skill infra-docker-best-practices

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps you produce Docker images that are smaller, faster to build, and safer to run, reducing CI/CD time, registry costs, and exposure to common CVEs and container misconfigurations.

Core Features & Use Cases

  • Multi-stage builds to separate builder and runtime dependencies, keeping production images lean.
  • Layer cache optimization to speed up rebuilds by copying lockfiles/dependency manifests before source code.
  • Security hardening with non-root users, reduced attack surface (slim/distroless), read-only root recommendations, and secret-safe builds via BuildKit.
  • Operational best practices for data engineering containers, including examples for Python ETL, dbt adapters, and Spark submit images.
  • Quality gates in CI with image vulnerability scanning (Trivy) and basic size/layer audits (dive/dockers tools).
  • Anti-patterns to avoid such as cache-busting COPY order, separate apt-get layers, shell-form CMD/ENTRYPOINT, and shipping secrets/build context.

Quick Start

Use the skill to design your next production Dockerfile by asking: "Review my data engineering Dockerfile and propose a hardened multi-stage, cache-friendly, non-root build with CI image scanning."

Frequently Asked Questions about infra-docker-best-practices

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I make my Python ETL Docker images smaller and faster to build?▼

To make Python ETL Docker images smaller and faster, use multi-stage builds to separate builder dependencies from the runtime, and optimize layer caching by copying lockfiles before source code. This keeps production images lean and speeds up rebuilds.

What is the best way to harden a Dockerfile for data engineering pipelines?▼

Hardening a Dockerfile for data engineering involves using non-root users, reducing the attack surface with slim or distroless base images, and configuring read-only root filesystems. You should also use BuildKit secrets to prevent shipping credentials in the build context.

How do I scan Docker images for vulnerabilities in CI/CD?▼

To scan Docker images for vulnerabilities in CI/CD, integrate Trivy-based image scanning as a quality gate. You can also use tools like dive or dockers to audit image size and layers, ensuring containers remain secure and efficient before deployment.

Can I use multi-stage Docker builds for dbt adapters and Spark submit images?▼

Yes, multi-stage Docker builds are highly effective for dbt adapters and Spark submit images. They isolate heavy build dependencies from the runtime environment, ensuring operational best practices are maintained for data engineering workloads.

Why is my Docker build cache busted every time I change my source code?▼

Your Docker build cache is busted because of incorrect COPY order. To fix this cache-busting anti-pattern, copy dependency manifests like lockfiles before copying source code, ensuring dependency installation layers remain cached during code changes.

Do I need BuildKit to handle secrets during Docker image construction?▼

You need BuildKit to handle secrets securely during Docker image construction. BuildKit secrets allow you to pass credentials like API keys into the build process without risking them being saved in intermediate image layers or the final build context.

Related Skills

Explore More Software Engineering Skills→
Patasse97Patasse97

senior-data-engineer

Design and orchestrate data pipelines with Python, SQL, Spark, Airflow, and dbt.

Community
Advanced
phenobarbitalphenobarbital
13

production-dockerfile

Generate production-ready Dockerfiles for Python applications with multi-stage builds.

Community
Intermediate
ancolemanancoleman
503

managing-vulnerabilities

Automate multi-layer security scanning and SBOM generation in CI/CD pipelines.

Community
Advanced
weirdgmeweirdgme
4

devsecops

Automate security integration across CI/CD pipelines with policy-as-code enforcement.

Community
Advanced