skills.rest
ThomazRossito
ThomazRossitoCOMMUNITY
Explore all skills by this author →
Website𝕏@thomazrossitoGitHub

teradata-to-databricks

Migrates Teradata data warehouses to Databricks using deterministic DDL generators and reconciliation scripts.

4|1|Updated May 22, 2026
Search Tags:#migration#reconciliation#data warehouse#databricks#teradata#delta lake#ddl conversion
One-click install
npx skills add https://github.com/ThomazRossito/ai-data-agents --skill teradata-to-databricks-thomazrossito
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: teradata-to-databricks
Source: https://github.com/ThomazRossito/ai-data-agents/tree/main/plugins/ai-data-agents/skills/teradata-to-databricks
Command: npx skills add https://github.com/ThomazRossito/ai-data-agents --skill teradata-to-databricks-thomazrossito

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve? Migrating a Teradata data warehouse to Databricks involves manual discovery (no dedicated Teradata MCP), risky hand-written DDL conversion, and error-prone validation. This Skill provides an 8-phase operational playbook that replaces hand-written SQL with deterministic generator scripts and enforces a human approval gate before any conversion. ## Core Features & Use Cases - Manual Discovery Playbook: Guides BTEQ/JDBC queries against DBC.* dictionary views (TablesV, TableSizeV, DBQLLogTbl, IndicesV) to inventory schemas, volumes, query logs, indexes, and TASM rules. - Deterministic DDL Conversion: Runs scripts/teradata_generate.py on concatenated SHOW TABLE output to emit Delta DDL with mapped types, CLUSTER BY from Primary Indexes, and a Snowflake anti-contamination gate that fails the build on tokens like VARIANT, IFF, or METADATA$*. - Two-Phase Reconciliation: Generates reconcile SQL via scripts/reconcile_generate.py with source_dialect=teradata, handling ANSI quoting, STDDEV/VARIANCE estimator matching, and snapshot-then-delta validation. - Use Case: A data team decommissioning a Teradata warehouse uses this Skill to score workloads into migration waves, produce a SPEC for human approval, convert DDL, choose WRITE_NOS/TPT/JDBC ingestion by volume, and execute a cutover runbook with rollback thresholds. ## Quick Start Ask the agent to start the Teradata to Databricks migration discovery for a given database by collecting SHOW TABLE output via BTEQ and producing a complexity-scored migration proposal.

Frequently Asked Questions about teradata-to-databricks

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I migrate a Teradata data warehouse to Databricks?▼

Follow the 8-phase flow: DISCOVER via BTEQ/JDBC queries on DBC.* views, ASSESS with a complexity scoring matrix, DESIGN a Medallion architecture, get human approval of the SPEC, then CONVERT with teradata_generate.py, INGEST by volume, add CDC, VALIDATE with reconcile_generate.py, and CUTOVER with a rollback runbook.

How to convert Teradata DDL to Delta Lake tables?▼

Concatenate SHOW TABLE output into a .sql file and run scripts/teradata_generate.py. It emits Delta CREATE TABLE statements with mapped types, CLUSTER BY from Primary Index columns, and flags PERIOD, INTERVAL, and ARRAY columns for manual review. Never hand-write the DDL.

Does Databricks support Teradata QUALIFY and other Teradata SQL functions?▼

QUALIFY is native in Databricks SQL since DBR 10.4 LTS and needs no translation. Other functions map directly: OREPLACE to REPLACE, INDEX to INSTR, ZEROIFNULL to COALESCE. OLAP functions like CSUM, MAVG, and RESET WHEN require manual mapping to Spark window functions.

Can I use Lakehouse Federation to migrate large Teradata tables?▼

No. Lakehouse Federation with CREATE CONNECTION TYPE teradata (DBR 16.1+, TD2 auth, port 1025) is only for discovery, profiling, and validation during transition. For large fact tables use WRITE_NOS export to Parquet with Auto Loader, or TPT for tables over 100GB.

Why does Teradata to Databricks reconciliation show STDDEV mismatches?▼

Teradata exposes explicit STDDEV_POP and STDDEV_SAMP, while Databricks STDDEV() equals STDDEV_SAMP. Comparing STDDEV_POP against STDDEV() diverges mathematically. Always match estimators: STDDEV_POP with stddev_pop, VAR_SAMP with variance, before comparing values.

What are the limitations of automated Teradata migration tools?▼

Join Indexes and Secondary Indexes have no 1:1 Delta equivalent and require redesign, not mechanical conversion. Teradata has no native Stream object for CDC, so watermark or third-party log-based CDC is needed. SET tables need explicit deduplication since Delta does not reject duplicates.

Related Skills

Explore More Data & Analytics Skills→
awsaws
2.5k

migrating-to-amazon-redshift

Guides end-to-end Teradata data warehouse migration to Amazon Redshift.

Official
Advanced
databricks-solutionsdatabricks-solutions
1.8k

spark-declarative-pipelines

Automate creation, configuration, and updating of Databricks Lakeflow Spark Declarative Pipelines.

Official
Advanced
vivekganavivekgana
4

access-management

Automate RBAC/ABAC access management with Python and SQL templates for Unity Catalog.

Community
Intermediate
fractaryfractary

log-auditor

Audit unmanaged logs and generate remediation specifications for fractary-logs integration.

Official
Advanced