What problem does it solve? Data pipelines built by multi-agent SDLC workflows need automated data quality validation, but deriving null checks, duplicate detection, referential integrity, and freshness rules manually from data models and mapping documents is slow and error-prone. This Skill acts as Agent 9 of 15 in the Databricks Multi-Agent SDLC Platform, producing validated DQ rules and reports as structured artifacts. ## Core Features & Use Cases - DQ Rule Derivation: Generates null/duplicate checks, business-rule validation, referential integrity, completeness, and freshness expectations from upstream data model and mapping artifacts. - Framework-Aware Output: Emits Lakeflow/DLT-style expectations (@dlt.expect_*) or DQX/Great-Expectations-style rules files matching the workspace standard. - Contract-Validated Handoff: Returns JSON conforming to a strict output contract (dq_rules.json, dq_report.md), writes artifacts to Unity Catalog Volumes, and logs runs to state.agent_runs for audit. - Use Case: After the PySpark Development Agent and SQL Agent produce transformation code, this agent reads those artifacts from /Volumes/{catalog}/state/artifacts/{ticket}/, calls the Databricks Foundation Model API to generate DQ rules, executes them against a sample table via SQL warehouse, and hands a scored DQ report to the Code Review Agent. ## Quick Start Ask the agent to derive data quality rules and a DQ report from the PySpark code, SQL, and data model artifacts for the current ticket and write them to the ticket's artifact volume.
