dpla-s3-and-aws

Coordinate S3 data synchronization and AWS operations for the DPLA ingestion pipeline.

35|12|Updated Jan 10, 2017

Search Tags:#automation #aws #infrastructure #scripts #s3 #data-ingestion

Ready to run

npx skills add https://github.com/dpla/ingestion3 --skill dpla-s3-and-aws

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: dpla-s3-and-aws
Source: https://github.com/dpla/ingestion3/tree/main/docs/ai-context/archive/cursor-skills/dpla-s3-and-aws
Command: npx skills add https://github.com/dpla/ingestion3 --skill dpla-s3-and-aws

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Automates and standardizes AWS S3 data synchronization and related AWS operations for the DPLA ingestion workflow, ensuring correct credentials, buckets, and consistent processes.

Core Features & Use Cases

S3 hub synchronization with anomaly detection and path handling
Check JSONL sync status against S3
Execute common AWS CLI actions with the dpla profile

Quick Start

Sync a hub to S3 using the project script and the dpla AWS profile.

Frequently Asked Questions about dpla-s3-and-aws

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I automate S3 data synchronization for hub-level ingestion?▼

Check JSONL sync status against S3 by executing the project's status scripts with the dpla AWS profile. This validates whether your JSONL files are successfully synchronized with the target S3 buckets during the ingestion workflow.

What AWS profile do I need to execute S3 bucket tasks for DPLA?▼

S3 hub synchronization handles path handling and anomaly detection during the data ingestion process. By applying the project scripts with the dpla profile, it identifies sync irregularities and manages file paths consistently across configured AWS buckets.

Can I run AWS CLI actions across multiple environments for data ingestion?▼

Run common AWS CLI actions across multiple environments by applying the dpla AWS profile within the project scripts. This standardizes AWS operations for data ingestion, ensuring consistent execution of S3 status checks and hub synchronization tasks.

What happens if AWS credentials and buckets are not configured for S3 sync?▼

If AWS credentials and buckets are not configured via the scripts and environment, S3 sync operations will fail. The automation requires the dpla AWS profile and properly configured buckets to execute data ingestion and status checks successfully.