dataops

Data validation belongs at the Bronze-to-Silver boundary, so Silver data satisfies a declared schema and invariants. Validating before Bronze landing is not recommended because Bronze must remain a faithful source copy to support replay for both testing validation logic and recovering from transformation bugs.

Test transformations by pinning a fixed input to a fixed output with one verification per test, using pytest.mark.parametrize for input matrices. Nothing is mocked; the transformation runs for real, which is why transformation logic must be separated from data-access code, typically by extracting it from notebooks into packages.

Data validation catches values that are simply wrong, such as out-of-range data, and triggers rerouting and repair. Data drift reflects genuine change in the world, such as shifting user preferences, and triggers model adaptation or retraining. The distinction comes from the ML Model Production Checklist.

No. ML unit tests deliberately leave some external calls unstubbed, making them closer to narrow integration tests. They check code quality, such as input and output shapes and whether weights change after one training epoch, not model accuracy or performance.

No. It generates code, assertions, and review guidance but does not execute pipelines, transformation engines, or telemetry backends. No upstream source prescribes drift thresholds or alerting policies, so the skill states that gap rather than inventing a number.