validate-stm

Validate Source-to-Target Mapping Excel workbooks for completeness and quality.

5|1|Updated Sep 23, 2025
One-click install
npx skills add https://github.com/RDEWAI/Redefining-DataEngineering-With-AI --skill validate-stm
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: validate-stm
Source: https://github.com/RDEWAI/Redefining-DataEngineering-With-AI/tree/main/chapter-6/mapping-analyst-plugin/skills/validate-stm
Command: npx skills add https://github.com/RDEWAI/Redefining-DataEngineering-With-AI --skill validate-stm

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires openpyxl, and includes scripts (resource) components.

What problem does it solve?

This Skill eliminates the risk of releasing incomplete or low-quality Source-to-Target Mapping (STM) Excel workbooks that cause broken data pipelines, missing data lineage traceability, and costly rework during data engineering implementations.

Core Features & Use Cases

  • Comprehensive STM Validation: Checks all 8 required STM sheets, standard column headers, transformation logic completeness, DMS reference coverage, and end-to-end lineage chains.
  • Severity-Ranked Issue Reporting: Flags problems as CRITICAL (must fix before the STM is usable), WARNING (address for production quality), or INFO (improvement suggestions) with actionable fix recommendations.
  • Use Case: A mapping analyst can run this validation immediately after creating a new STM workbook to catch missing sheets, placeholder values, or untraced DMS references before handing the document off to downstream development teams.

Quick Start

Use the validate-stm skill to validate the completeness and quality of your STM Excel workbook at 'outputs/stm/v2/stm-patient-data.xlsx' and receive a prioritized list of issues to resolve.

Frequently Asked Questions about validate-stm

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I validate a Source-to-Target Mapping Excel workbook before pipeline handoff?

To validate a Source-to-Target Mapping workbook, run automated checks using Python and openpyxl to verify required sheets, column headers, and transformation logic. This generates a severity-ranked report flagging missing lineage traceability and DMS reference gaps to resolve before downstream development.

What is STM validation and why is it needed for data engineering workflows?

STM validation is the process of checking Source-to-Target Mapping documents for completeness and quality. It is needed to prevent broken data pipelines, ensure accurate medallion layer transformations, and eliminate costly rework caused by incomplete mapping logic or missing lineage traceability.

Can I run automated transformation checks on STM Excel files in a CI/CD pipeline?

Yes, you can run automated transformation checks on STM Excel files in CI/CD pipelines. The validation executes via Python and openpyxl, allowing seamless integration into local development or automated workflows to continuously verify DMS references and lineage coverage.

Does openpyxl support checking DMS traceability and lineage coverage in Excel?

Yes, openpyxl supports checking DMS traceability and lineage coverage by reading the Excel workbook structure. The validation script uses openpyxl to inspect required STM sheets and column headers, ensuring end-to-end lineage chains and DMS references are completely mapped.

What are the limitations of using Python for Excel audit and STM validation?

A limitation of using Python for Excel audit is that validation depends on standardized workbook structures. If the STM workbook lacks the required sheets or expected column headers, the openpyxl-based script cannot accurately assess transformation logic or lineage coverage.

What's the best way to catch missing sheets and mapping gaps in STM documents?

The best way to catch mapping gaps is to run an automated STM validation script that checks all 8 required sheets and flags issues by severity. This provides actionable fix recommendations for CRITICAL errors and WARNINGs before handing the document to downstream teams.