skills.rest
dpla
dplaOFFICIAL
Explore all skills by this author →
WebsiteGitHub

dpla-monitor-ingest-remap

Monitor IngestRemap pipeline progress across hubs via _SUCCESS markers.

35|12|Updated Jan 10, 2017
Search Tags:#status#mapping#jsonl#enrichment#ingest#monitor#remap
One-click install
npx skills add https://github.com/dpla/ingestion3 --skill dpla-monitor-ingest-remap
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: dpla-monitor-ingest-remap
Source: https://github.com/dpla/ingestion3/tree/main/.cursor/skills/dpla-monitor-ingest-remap
Command: npx skills add https://github.com/dpla/ingestion3 --skill dpla-monitor-ingest-remap

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Monitors the progress of IngestRemap pipelines (remap.sh / ingest.sh step 2) to provide real-time visibility into mapping, enrichment, and jsonl outputs.

Core Features & Use Cases

  • Orchestrator status monitoring: Prefer orchestrator status when available; otherwise watch latest mapping/enrichment/jsonl outputs.
  • Hub-scoped watch: Monitor a single hub or all hubs to track progress and _SUCCESS markers.
  • Status interpretation and fallbacks: Detect running vs completed stages and gracefully handle missing outputs.

Quick Start

Run the status watcher to observe the latest remap progress for your hub.

Frequently Asked Questions about dpla-monitor-ingest-remap

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I monitor IngestRemap pipeline progress across mapping and enrichment steps?▼

To monitor IngestRemap pipeline progress, you can track mapping, enrichment, and jsonl outputs by polling latest step directories and checking _SUCCESS markers to determine stage completion. This applies to both orchestrator-based and manual remap runs.

What is the best way to track remap status for a specific hub?▼

Tracking remap status for a specific hub involves using a hub-scoped watch to monitor mapping, enrichment, and jsonl outputs. The watcher interprets running versus completed stages and provides fallbacks when outputs are missing.

How does status interpretation work when mapping or enrichment outputs are missing?▼

Status interpretation detects running versus completed stages by checking _SUCCESS markers and polling latest step directories. It gracefully handles missing outputs by providing clear fallbacks rather than failing the monitoring process.

Can I use this to monitor both orchestrator-based runs and manual remap runs?▼

Yes, you can monitor both orchestrator-based runs and manual remap runs. The watcher prefers orchestrator status when available and falls back to watching latest mapping, enrichment, and jsonl outputs across all hubs.

Why does my ingest step 2 jsonl output show as incomplete during remap monitoring?▼

Your jsonl output may show as incomplete because the _SUCCESS marker has not been generated yet. The watcher polls latest step directories to detect running stages and relies on these markers to confirm completion.

Do I need an orchestrator to monitor ingest and remap progress across all hubs?▼

No, you do not need an orchestrator to monitor ingest and remap progress. The watcher prefers orchestrator status when available but can independently watch latest mapping, enrichment, and jsonl outputs across all hubs.

Related Skills

Explore More Software Engineering Skills→
dpladpla
35

dpla-ingest-debug

Diagnose and fix DPLA hub ingestion failures across pipeline stages.

Official
Advanced
MichaelsEngineeringMichaelsEngineering

gpd-export

Export completed GPD research phases to HTML, LaTeX, and ZIP packages.

Community
Advanced
mafvietnammafvietnam

ck:kanban

Visualize AI agent tasks on a centralized kanban board with progress tracking.

Community
Intermediate
dpladpla
35

dpla-run-ingest

Automate hub ingest runs by selecting runbooks and verifying output artifacts.

Official
Intermediate