databricks-lakebase-autoscale

Manage Databricks Lakebase PostgreSQL autoscaling, branching, and scale-to-zero compute.

3|Updated Jan 2, 2025

Search Tags:#database #postgresql #branching #etl #autoscaling #databricks #oltp

Ready to run

npx skills add https://github.com/ScottHMcKean/genai_notebooks --skill databricks-lakebase-autoscale-scotthmckean

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: databricks-lakebase-autoscale
Source: https://github.com/ScottHMcKean/genai_notebooks/tree/main/.cursor/skills/databricks-lakebase-autoscale
Command: npx skills add https://github.com/ScottHMcKean/genai_notebooks --skill databricks-lakebase-autoscale-scotthmckean

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill simplifies managing PostgreSQL databases on Databricks by providing autoscaling compute, branching, and scale-to-zero capabilities, reducing operational overhead and optimizing costs.

Core Features & Use Cases

Autoscaling Compute: Dynamically adjusts compute resources (0.5-112 CU) based on load.
Branching: Create isolated database environments for development and testing.
Scale-to-Zero: Suspends compute after inactivity to save costs.
Reverse ETL: Sync data from Delta Lake to PostgreSQL.
Use Case: A data engineering team needs to provide a PostgreSQL database for a new application. They can use this Skill to set up a project with a production branch, configure autoscaling compute, and later create development branches for new features without impacting production.

Quick Start

Use the databricks-lakebase-autoscale skill to create a new project named 'my-app' with PostgreSQL version 17.

Frequently Asked Questions about databricks-lakebase-autoscale

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I configure autoscaling for PostgreSQL on Databricks?▼

To configure autoscaling for PostgreSQL on Databricks, you set up a managed project that dynamically adjusts compute resources between 0.5 and 112 compute units based on real-time load. This optimizes performance and reduces operational overhead.

Can I create isolated PostgreSQL database environments for development?▼

Yes, you can create isolated PostgreSQL database environments using Git-like branching. This feature allows you to instantly spin up development and testing branches from your production database without impacting active workloads.

Does Databricks managed PostgreSQL support scale-to-zero capabilities?▼

Databricks managed PostgreSQL supports scale-to-zero capabilities by automatically suspending compute resources after periods of inactivity. This functionality significantly reduces costs by ensuring you only pay for compute when the database is actively processing requests.

How do I sync data from Delta Lake to an OLTP PostgreSQL database?▼

You sync data from Delta Lake to an OLTP PostgreSQL database using the reverse ETL feature. This process syncs Delta Lake tables directly into your managed PostgreSQL instance via synced tables to support operational application workloads.

Can I use OAuth authentication for secure access to managed PostgreSQL?▼

Yes, you can use OAuth authentication for secure access to managed PostgreSQL. This provides a secure method to authenticate users and applications connecting to your Databricks Lakebase databases without managing static database passwords.

How does point-in-time restore work for Databricks managed PostgreSQL?▼

Point-in-time restore for Databricks managed PostgreSQL provides instant recovery by reverting the database to a specific previous state. This mechanism ensures data durability and allows quick rollback from accidental deletions or data corruption.