shepherd

Monitor snakemake and nextflow pipelines, detect errors, and restart failed jobs.

3|3|Updated Dec 26, 2025

Search Tags:#automation #monitoring #error handling #bioinformatics #pipeline #nextflow #snakemake

Ready to run

npx skills add https://github.com/genomewalker/cc-soul --skill shepherd-genomewalker

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: shepherd
Source: https://github.com/genomewalker/cc-soul/tree/main/skills/shepherd
Command: npx skills add https://github.com/genomewalker/cc-soul --skill shepherd-genomewalker

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill automates the monitoring and management of long-running bioinformatics pipelines, preventing failures and ensuring continuous progress without constant human oversight.

Core Features & Use Cases

Error Detection & Auto-Fix: Identifies common pipeline errors and attempts to apply learned fixes from memory.
Automated Restarts: Restarts failed pipeline jobs up to a configurable limit.
Session Management: Can run pipelines in isolated Zellij sessions for robustness.
Use Case: Monitor a multi-day snakemake pipeline for genomic analysis, automatically restarting failed steps due to transient network issues or temporary resource unavailability, and notifying you only if manual intervention is required after multiple failures.

Quick Start

Start monitoring a snakemake pipeline with a 60-second check interval and a maximum of 3 restarts.

Frequently Asked Questions about shepherd

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I automate monitoring and restart failed snakemake or nextflow jobs?▼

Automate pipeline monitoring by running an autonomous sense-think-act loop that detects errors, recalls fixes from memory, and restarts failed jobs up to a configurable limit. It prevents long-running bioinformatics pipelines from stalling due to transient network or resource issues.

What is the best way to handle transient network errors in long-running bioinformatics pipelines?▼

Handling transient pipeline errors involves detecting failures and automatically applying learned fixes from memory before restarting the job. This approach ensures continuous progress for multi-day genomic analyses without requiring constant human oversight.

Can I run nextflow pipelines in isolated sessions for enhanced reliability?▼

Yes, you can run pipelines in isolated Zellij sessions for enhanced reliability. Session management ensures robust task execution and isolation, preventing pipeline interruptions from affecting other operations.

Does autonomous pipeline monitoring work with both snakemake and nextflow?▼

Yes, autonomous monitoring works with both snakemake and nextflow pipelines. It uses a sense-think-act loop to detect errors, recall previous fixes, and restart failed jobs within isolated Zellij sessions.

How do I configure restart limits and check intervals for automated pipeline monitoring?▼

Configure automated pipeline monitoring by setting a specific check interval, such as 60 seconds, and defining a maximum number of restart attempts. The monitor will restart failed jobs automatically up to this limit before requesting manual intervention.

Why do I need isolated Zellij sessions for bioinformatics pipeline management?▼

Isolated Zellij sessions provide enhanced reliability for pipeline management by ensuring robust execution environments. Utilizing chitta and zellij-mcp tools for task execution prevents pipeline failures from impacting the broader system.