skills.rest
mreferre
mreferreCOMMUNITY
Explore all skills by this author →
Website𝕏@mreferreGitHub

connecting-to-data-source

Create and validate AWS Glue connections to JDBC databases and native warehouses.

Updated May 12, 2026
Search Tags:#bigquery#snowflake#jdbc#aws glue#secrets manager#vpc networking#connection setup
One-click install
npx skills add https://github.com/mreferre/aws-agent-toolkit-skills --skill connecting-to-data-source
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: connecting-to-data-source
Source: https://github.com/mreferre/aws-agent-toolkit-skills/tree/main/.kiro/skills/connecting-to-data-source
Command: npx skills add https://github.com/mreferre/aws-agent-toolkit-skills --skill connecting-to-data-source

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill eliminates the trial-and-error involved in wiring AWS Glue to external databases and warehouses by creating a reusable, tested AWS Glue connection with correct networking and credentials references.

Core Features & Use Cases

  • Glue connection creation with validation: Creates an AWS Glue connection for JDBC sources (Oracle, SQL Server, PostgreSQL, MySQL, RDS, Aurora, Redshift) and native connections for Snowflake and BigQuery, then tests it before handoff.
  • Credential best practices: Guides selection of IAM DB auth when supported and strongly prefers Secrets Manager over plaintext credentials, including correct secret key conventions per source type.
  • Network + troubleshooting guardrails: Ensures VPC/Subnet/Security Group/S3 endpoint readiness for private sources and provides an ordered troubleshooting flow when tests fail.
  • Use Case: Register an existing RDS/PostgreSQL instance (or Snowflake/BigQuery project) as a named Glue connection so downstream ingestion jobs can reliably read from it without reconfiguring networking and auth each time.

Quick Start

Ask the AI to create a Glue connection for your PostgreSQL on RDS by running: connect to data source with source-type postgresql, connection name oracle-prod-sales, hostname <rds-endpoint>, and the secrets manager secret to use.

Frequently Asked Questions about connecting-to-data-source

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create and test an AWS Glue connection to a JDBC database?▼

To create an AWS Glue connection to JDBC databases like PostgreSQL or Redshift, you configure the connection type, register credentials in Secrets Manager, set VPC networking, and run a Glue test-connection to validate access before downstream jobs reuse it.

Can I set up AWS Glue connectivity to native Snowflake or BigQuery sources?▼

Yes, AWS Glue connectivity supports native connections for Snowflake and BigQuery sources. You select the specific native connection type, handle credential registration, and validate the setup using Glue test-connection and engine-level smoke testing.

Does AWS Glue connection setup require Secrets Manager for database credentials?▼

AWS Glue connection setup strongly prefers Secrets Manager over plaintext credentials for database access. It also guides selection of IAM DB auth when supported, ensuring correct secret key conventions per source type for secure connectivity.

What's the best way to troubleshoot AWS Glue connection failures to private databases?▼

The best way to troubleshoot AWS Glue connection failures is using an ordered flow for network, credential, driver, and SSL issues. This ensures VPC, Subnet, Security Group, and S3 endpoint readiness for private endpoints before testing connectivity.

Why does my Glue test-connection fail when connecting to RDS or Aurora?▼

Glue test-connection failures to RDS or Aurora often stem from incorrect VPC configuration, missing Security Group rules, or invalid Secrets Manager credentials. Ordered troubleshooting checks network readiness first, then credential and driver SSL configurations.

Do I need VPC configuration for AWS Glue ETL connectivity to private endpoints?▼

Yes, VPC configuration is required for AWS Glue ETL connectivity to private endpoints. You must ensure VPC, Subnet, Security Group, and S3 endpoint readiness to establish reliable access to private databases like RDS or Aurora.

Related Skills

Explore More Data & Analytics Skills→
AndreKuraitAndreKurait

configuring-vpc-endpoints-for-private-aws-service-access

Configure gateway and interface VPC endpoints for private AWS service access via PrivateLink.

Community
Advanced
simkeyursimkeyur
34

Data Transformation & ETL

Automate end-to-end ETL workflows with validation and audit trails.

Community
Advanced
itsmostafaitsmostafa
1.1k

secrets-manager

Store, retrieve, and rotate secrets with AWS Secrets Manager APIs.

Community
Advanced
mashharukimashharuki
2

connecting-to-data-source

Automate creation, testing, and troubleshooting of AWS Glue connections.

Community
Advanced