What problem does it solve? Data pipelines often grow into ad-hoc stacks where layers have no clear role, sources are chosen by habit, and dashboards drift from the tables they claim to show. This Skill provides a disciplined design method for building a medallion architecture (Bronze/Silver/Gold) where each layer has a single auditable role, the source is chosen by explicit criteria, and consumption (AI/ML/MCP/dashboards) reads the same tables the system writes. ## Core Features & Use Cases - Layer role definitions: Bronze is raw append-only audit storage, Silver handles dedup, type coercion, and explicit NULL policy, Gold answers a fixed business question as an aggregated Delta table or DuckDB query. - Source selection framework: Compare API, database, and SFTP sources on reliability, freshness, latency, quota, cost, and format before building, and record the decision. - Scenario dashboards: Turn Gold-layer queries into parameterized hypothetical scenarios rendered in dashboards via DuckDB delta_scan, measured at real production data volume. - Use Case: You ingest market data from an external API, store raw events in a Bronze Delta table, clean and dedup into Silver, aggregate edge-by-market into Gold, and serve a dashboard comparing conservative vs aggressive edge thresholds from the same parameterized query. ## Quick Start Ask the agent to design a medallion architecture for your data source, defining the Bronze, Silver, and Gold layers and a parameterized scenario dashboard on top of the Gold table.
