skills.rest
sakicodes
sakicodesCOMMUNITY
Explore all skills by this author →
GitHub

connecting-to-data-source

Create and test AWS Glue connections to JDBC databases, Redshift, Snowflake, and BigQuery.

Updated Jul 1, 2026
Search Tags:#snowflake#jdbc#redshift#aws glue#secrets manager#vpc networking#data source connection
One-click install
npx skills add https://github.com/sakicodes/BuildFestHackathon26 --skill connecting-to-data-source-sakicodes
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: connecting-to-data-source
Source: https://github.com/sakicodes/BuildFestHackathon26/tree/main/.agents/skills/connecting-to-data-source
Command: npx skills add https://github.com/sakicodes/BuildFestHackathon26 --skill connecting-to-data-source-sakicodes

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Registering an external data source with AWS Glue requires coordinating network configuration, credential storage, driver selection, and connection testing across many source types, and misconfigurations surface as opaque timeout or authentication errors. ## Core Features & Use Cases - Multi-source connection creation: Builds Glue connections for Oracle, SQL Server, PostgreSQL, MySQL, RDS/Aurora, Redshift, Snowflake, and BigQuery with the correct connection type and properties. - Credential and network setup: Registers credentials in AWS Secrets Manager or configures IAM database authentication, and configures VPC subnets, security groups, and S3 endpoints for private sources. - Discovery and two-phase testing: Finds existing Glue connections and candidate RDS/Redshift sources in the account, then validates with Glue TestConnection plus an engine-level smoke test. - Use Case: A data engineer needs to register a production PostgreSQL RDS instance so Glue ETL jobs can read from it; the skill discovers the instance, stores credentials in Secrets Manager, creates the JDBC connection with the right subnet and security group, and verifies it end-to-end. ## Quick Start Ask the assistant to connect Glue to your database, for example: set up a Glue connection to my PostgreSQL RDS instance in us-east-1.

Frequently Asked Questions about connecting-to-data-source

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create an AWS Glue connection to a JDBC database?▼

Create a Glue connection of type JDBC with the JDBC_CONNECTION_URL, a Secrets Manager SECRET_ID holding username and password, and PhysicalConnectionRequirements specifying the subnet, security group, and availability zone. Then run aws glue test-connection to validate network and credentials.

How do I connect AWS Glue to Snowflake?▼

Use Glue's native SNOWFLAKE connection type, not a JDBC connection, for Spark ETL jobs. Provide the account host, warehouse, role, and database, and store credentials in Secrets Manager with snowflakeUser and snowflakePassword keys. Password, key-pair, and OAuth authentication modes are supported.

Should I use Secrets Manager or IAM database authentication for Glue connections?▼

Prefer IAM database authentication where supported: Aurora MySQL/PostgreSQL, RDS MySQL/PostgreSQL, and Redshift. It avoids long-lived passwords and rotation. For Oracle, SQL Server, Snowflake, and BigQuery, store credentials in AWS Secrets Manager and reference them via SECRET_ID.

Why does my AWS Glue connection test fail with a timeout?▼

Timeouts are almost always network issues: missing route to the source VPC, security groups blocking the source port, a missing self-referencing rule on the Glue security group, or no S3 VPC endpoint or NAT gateway. Test reachability from an EC2 instance in the same subnet before blaming credentials.

Does AWS Glue need a VPC endpoint to connect to private databases?▼

Yes, private sources require PhysicalConnectionRequirements with a subnet in a VPC that can route to the source. The subnet also needs an S3 VPC gateway endpoint or NAT gateway, otherwise Glue jobs fail at startup with UnableToFindVpcEndpoint.

When should I not use a Glue connection for a data source?▼

DynamoDB is read directly by Glue without a connection, and local files belong in an ingestion workflow rather than a connection. SaaS sources like Salesforce, ServiceNow, SAP, MongoDB, and Kafka are also outside this skill's scope.

Related Skills

Explore More Data & Analytics Skills→
mreferremreferre

connecting-to-data-source

Create and validate AWS Glue connections to JDBC databases and native warehouses.

Community
Advanced
awsaws
2.5k

ingesting-into-data-lake

Import data from S3, JDBC databases, Snowflake, BigQuery, and DynamoDB into AWS data lake tables.

Official
Advanced
richardnromanrichardnroman

creating-data-lake-table

Creates managed Iceberg tables on Amazon S3 Tables with Glue catalog registration and IAM access control.

Community
Advanced
obispobrunoobispobruno
1

connecting-to-data-source

Create and test AWS Glue connections to JDBC databases, Redshift, Snowflake, and BigQuery.

Community
Advanced