What problem does it solve? Training and evaluation data on Kubernetes often lacks reproducibility: teams cannot answer which exact bytes trained a given checkpoint months later. This Skill guides the design of a lakehouse storage substrate — zone layout, table format selection, snapshot retention, and dataset-to-checkpoint lineage — so training inputs remain byte-reproducible and auditable. ## Core Features & Use Cases - Zone architecture: Defines medallion zones (landing, raw, cleansed, curated) plus research-specific feature/training and frozen eval zones with per-zone retention and access policies. - Table format decision: Compares Iceberg, Delta Lake, and Hudi across schema evolution, partition evolution, time travel, and ecosystem criteria, with an ADR-ready decision rule. - Versioning and lineage: Specifies dataset pinning conventions, immutable materialized-input manifests with content hashes, snapshot retention SLOs, and eval isolation with contamination gates. - Deployment planning: Covers MinIO/S3 on Kubernetes, Iceberg REST catalog HA, Spark and Trino compute, throughput budgeting, and lifecycle jobs. - Use Case: A research platform team needs to guarantee that any model checkpoint can be traced back to its exact training bytes a year later; the Skill produces the zone map, format ADR, lineage spec, and a golden-path reproducibility test as acceptance criteria. ## Quick Start Use the training-storage-architect skill to design a reproducible lakehouse storage layout for our training corpora on Kubernetes with MinIO and Iceberg.
