skills.rest
dpla
dplaOFFICIAL
Explore all skills by this author →
WebsiteGitHub

dpla-ingest-debug

Diagnose and fix DPLA hub ingestion failures across pipeline stages.

35|12|Updated Jan 10, 2017
Search Tags:#debugging#logs#ingestion#orchestrator#harvest#remap#s3-sync
One-click install
npx skills add https://github.com/dpla/ingestion3 --skill dpla-ingest-debug
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: dpla-ingest-debug
Source: https://github.com/dpla/ingestion3/tree/main/.cursor/skills/dpla-ingest-debug
Command: npx skills add https://github.com/dpla/ingestion3 --skill dpla-ingest-debug

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

Debug and fix DPLA hub ingestion failures across harvest, mapping, enrichment, jsonl, s3-sync, and anomaly. Use when a hub fails, to debug an ingest failure, check an escalation report, or retry a failed hub/stage.

Core Features & Use Cases

  • Gather context and logs to identify where the pipeline failed.
  • Determine the correct runbook and rerun only the minimal steps (harvest, remap, s3-sync) for a hub.
  • Verify outputs via _SUCCESS markers and provide escalation-ready diagnostics.

Quick Start

Follow the fast path: identify the hub and stage from logs and run the minimal steps (harvest, remap, s3-sync) for that hub.

Frequently Asked Questions about dpla-ingest-debug

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I debug DPLA hub ingestion failures across harvest and s3-sync stages?▼

Debug DPLA hub ingestion failures by gathering context and logs to identify the failed pipeline stage, then running the minimal steps for harvest, remap, and s3-sync. This isolates the issue and enables targeted re-runs for the specific hub.

What's the best way to retry a failed DPLA hub stage without rerunning the entire pipeline?▼

Retry a failed DPLA hub stage by identifying the failing step from logs, selecting the correct runbook, and executing only the minimal required script like harvest.sh, remap.sh, or s3-sync.sh for that individual hub.

Why does DPLA ingestion fail during the mapping or enrichment workflow?▼

DPLA ingestion fails during mapping or enrichment due to data anomalies or processing errors in the pipeline. Diagnose these failures by checking escalation reports and logs to pinpoint the exact stage causing the breakdown.

Do I need environment setup to diagnose DPLA orchestrator failures?▼

You need environment setup to diagnose DPLA orchestrator failures, including sourcing .env or i3_conf files. Access to data/escalations, logs, and runbooks is required to run harvest, remap, and s3-sync scripts.

How do I verify outputs after rerunning a DPLA harvest or remap workflow?▼

Verify outputs after rerunning a DPLA harvest or remap workflow by checking for _SUCCESS markers. These markers confirm the process completed correctly and provide escalation-ready diagnostics for the hub.

When should I not use a minimal re-run approach for DPLA ingestion debugging?▼

Avoid a minimal re-run approach for DPLA ingestion debugging when the failure spans multiple interconnected stages or requires a full pipeline reset. Use it only for isolated stage failures like single harvest, remap, or s3-sync issues.

Related Skills

Explore More Software Engineering Skills→
aaron-he-zhuaaron-he-zhu
142

memory-management

Manage two-layer memory with hot cache and cold storage for SEO projects.

Community
Basic
dpladpla
35

dpla-run-ingest

Automate hub ingest runs by selecting runbooks and verifying output artifacts.

Official
Intermediate
amadeus-dlcamadeus-dlc
5

amadeus-infrastructure-design

Run the infrastructure-design stage in isolation without advancing the main workflow.

Official
Intermediate
tneztnez

ci-health-check

Monitor GitHub Actions workflow runs and diagnose failures.

Community
Intermediate