skills.rest
sakicodes
sakicodesCOMMUNITY
Explore all skills by this author →
GitHub

finding-data-lake-assets

Resolves data lake asset references across Glue Data Catalog, S3, S3 Tables, and Redshift.

Updated Jul 1, 2026
Search Tags:#s3#data catalog#redshift#reverse lookup#aws glue#data lake#table search
One-click install
npx skills add https://github.com/sakicodes/BuildFestHackathon26 --skill finding-data-lake-assets-sakicodes
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: finding-data-lake-assets
Source: https://github.com/sakicodes/BuildFestHackathon26/tree/main/.agents/skills/finding-data-lake-assets
Command: npx skills add https://github.com/sakicodes/BuildFestHackathon26 --skill finding-data-lake-assets-sakicodes

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires boto3, and includes references (resource) components.

What problem does it solve? Locating a specific table or dataset in a large AWS data lake is slow and error-prone when assets span Glue Data Catalog, S3 buckets, S3 Tables, and Redshift. This Skill resolves vague references like "the orders table" or an S3 path into concrete catalog entries with database, format, location, and partition keys. ## Core Features & Use Cases - Layered search with early stop: Searches Glue Data Catalog first via search-tables, then S3 reverse lookup, then Redshift, stopping at the first high-confidence match to minimize API calls. - S3 path reverse lookup: Converts an S3 path like s3://bucket/prefix/ into the corresponding Glue table registration. - S3 Tables and Redshift coverage: Enumerates S3 Tables via s3tables list-table-buckets and queries Redshift svv_all_tables for warehouse and lakehouse assets. - Experimental Glue Discovery support: Optionally queries customer-authored catalog context via the Glue Search/GetAsset APIs, with strict untrusted-data handling. - Use Case: A data analyst asks "which table has customer_id?" The Skill searches the Glue catalog, finds three candidate tables, and presents them ranked with format and match reasons. ## Quick Start Ask the assistant to find the table matching a name, keyword, column, or S3 path, for example: find the table containing customer_id in the us-west-2 region.

Frequently Asked Questions about finding-data-lake-assets

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find a table in the AWS Glue Data Catalog by name or keyword?▼

Use aws glue search-tables with a search text matching the table name, column name, or business keyword. It searches across all databases in one call, so you avoid looping over databases with get-tables.

How do I find which Glue table points to an S3 path?▼

Run a reverse lookup by passing a path fragment to aws glue search-tables, then verify candidates by checking StorageDescriptor.Location with get-table. Only list S3 objects directly if no catalog entry matches.

Does Glue search-tables find S3 Tables assets?▼

No, search-tables does not index the s3tablescatalog federated catalog. Enumerate S3 Tables with aws s3tables list-table-buckets and list-namespaces, which use a four-level catalog/table-bucket/namespace/table hierarchy.

How do I search for tables in Amazon Redshift?▼

Query the svv_all_tables system view with an ILIKE filter on the table name to cover both native and Spectrum external tables. Spectrum external tables also appear in Glue, so check the Glue catalog first to avoid duplicate work.

Why does aws glue search-tables return AccessDeniedException?▼

The caller lacks the glue:SearchTables IAM permission. Either request that permission or fall back to get-tables with a known database name and an Expression filter.

When should I not use this asset resolution skill?▼

Do not use it for full catalog audits, running queries against the data, or creating new tables. Those tasks belong to dedicated catalog exploration, query execution, and table creation workflows.

Related Skills

Explore More Data & Analytics Skills→
awsaws
2.5k

finding-data-lake-assets

Resolves data lake asset references across Glue Data Catalog, S3, S3 Tables, and Redshift.

Official
Advanced
awsaws
2.5k

querying-data-lake

Execute and manage Athena SQL queries across Glue, S3 Tables, and Redshift catalogs.

Official
Advanced
awsaws
2.5k

exploring-data-catalog

Inventory and audit AWS Glue Data Catalog assets across S3 Tables, Redshift-federated, and remote Iceberg catalogs.

Official
Advanced
obispobrunoobispobruno
1

finding-data-lake-assets

Resolves data lake asset references across Glue Data Catalog, S3, S3 Tables, and Redshift.

Community
Advanced