skills.rest
github
githubOFFICIAL
Explore all skills by this author →
WebsiteGitHub

sql-server-table-reconciliation

Compare SQL Server tables across instances to detect missing rows, mismatches, and schema drift.

38.5k|4.9k|Updated Jun 11, 2025
Search Tags:#reconciliation#data migration#SQL Server#schema drift#Apache Arrow#mssql-python#etl validation
One-click install
npx skills add https://github.com/github/awesome-copilot --skill sql-server-table-reconciliation
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sql-server-table-reconciliation
Source: https://github.com/github/awesome-copilot/tree/main/skills/sql-server-table-reconciliation
Command: npx skills add https://github.com/github/awesome-copilot --skill sql-server-table-reconciliation

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires mssql-python, pyarrow, pandas, and includes scripts (resource) components.

What problem does it solve?

Validating that data matches between two SQL Server environments—such as production and staging, or before and after a migration—is slow and error-prone when done manually. This Skill automates cross-instance table comparison and produces a clear reconciliation report.

Core Features & Use Cases

  • Row-Level Comparison: Detects missing rows, extra rows, and column-level mismatches by joining on auto-detected primary keys.
  • Schema Drift Detection: Compares column names, data types, and nullability between source and target tables.
  • Scalable Performance: Uses Apache Arrow columnar transfer via the mssql-python driver, with chunked extraction and hash pre-checks for tables over 1M rows.
  • Use Case: After an ETL migration from a production database to staging, run the Skill against dbo.* to verify every table matches and export a CSV reconciliation report for audit sign-off.

Quick Start

Ask the AI to reconcile the dbo.Orders table between your production and staging SQL Server instances using Entra authentication and generate a console report.

Frequently Asked Questions about sql-server-table-reconciliation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I compare two SQL Server tables across different servers?▼

Run the reconcile.py script with source and target server, database, table names, and auth mode. It auto-detects primary keys, extracts data via Apache Arrow, and reports missing rows, extra rows, and column mismatches.

How to validate data migration between SQL Server databases?▼

Use table reconciliation to compare source and target after migration. The script supports wildcard specs like dbo.* to check all tables in a schema and generates a pass/fail summary report in console, CSV, or JSON format.

Does mssql-python support Azure AD authentication?▼

Yes, the script supports Entra (Azure AD) authentication using ActiveDirectoryDefault in the connection string. For SQL authentication, credentials are read from MSSQL_USER and MSSQL_PASSWORD environment variables, never hardcoded.

How do I compare large SQL Server tables with millions of rows?▼

For tables over 1M rows, use a hash pre-check with HASHBYTES to compare row hashes first, then fetch full rows only for mismatches. Chunked extraction with 100K-row batches also reduces memory pressure.

What happens if a table has no primary key during reconciliation?▼

The script auto-detects primary keys from sys.indexes metadata on both source and target. If no key is found on either side, the table is marked SKIPPED and you must supply one via the --primary-key argument.

Related Skills

Explore More Data & Analytics Skills→
MoonAxisMoonAxis

azure-postgres

Configure passwordless authentication for Azure Database for PostgreSQL Flexible Server using Microsoft Entra ID.

Official
Advanced
qa-amanqa-aman
19

qa-release

Automate QA release decisions and generate formal sign-off reports.

Community
Advanced
Datus-aiDatus-ai
1.5k

data-profiler

Profile data quality metrics and value distributions for tabular datasets.

Official
Basic
sandeepmvlsandeepmvl
21

cdc-debezium-rails

Configure Debezium CDC to stream Rails PostgreSQL changes to Kafka.

Community
Advanced