databricks-iceberg

What problem does it solve?

This Skill provides a comprehensive solution for working with Apache Iceberg on Databricks, enabling seamless data access and processing across various use cases.

Core Features & Use Cases

• Managed Iceberg Tables: Create and manage native Iceberg tables with full read/write capabilities.
• UniForm: Make Delta tables readable as Iceberg for external engine access.
• Compatibility Mode: Enable UniForm for streaming tables and materialized views in SDP pipelines.
• Iceberg REST Catalog (IRC): Expose Databricks tables to external engines via a REST API.
• Snowflake Interop: Bidirectional data sharing between Databricks and Snowflake using Iceberg.
• External Engine Access: Connect PyIceberg, OSS Spark, and other external engines to Databricks tables.
• Quick Start: Create a managed Iceberg table and enable UniForm on an existing Delta table.

Quick Start

Create a managed Iceberg table using the following SQL command:

CREATE TABLE my_catalog.my_schema.events
USING ICEBERG
AS SELECT * FROM raw_events;

Enable UniForm on an existing Delta table with:

ALTER TABLE my_catalog.my_schema.customers
SET TBLPROPERTIES (
  'delta.columnMapping.mode' = 'name',
  'delta.enableIcebergCompatV2' = 'true',
  'delta.universalFormat.enabledFormats' = 'iceberg'
);