skills.rest
liyanqing1987
liyanqing1987COMMUNITY
Explore all skills by this author →
GitHub

lsf_job_fail

Diagnose LSF job EXIT failures by mapping exit codes and TERM_* reasons.

66|20|Updated Jul 7, 2020
Search Tags:#oom#lsf#log triage#job failure diagnosis#exit code#term termination#hpc monitoring
One-click install
npx skills add https://github.com/liyanqing1987/lsfMonitor --skill lsf-job-fail
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: lsf_job_fail
Source: https://github.com/liyanqing1987/lsfMonitor/tree/main/monitor/conf/skills/lsf_job_fail
Command: npx skills add https://github.com/liyanqing1987/lsfMonitor --skill lsf-job-fail

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill helps you diagnose why an LSF/OpenLava/Volclava job EXITed or failed by translating exit codes and TERM_* termination reasons into likely root causes, separating application errors from system kills.

Core Features & Use Cases

  • Exit Code to Root-Cause Mapping: Interprets common exit code ranges (1–127 for application/script issues, >=128 for signal-driven termination) to guide the next investigation step.
  • TERM_ Termination Reason Diagnosis*: Uses TERM_MEMLIMIT, TERM_CPULIMIT, TERM_RUNLIMIT, TERM_* resource limits, and admin/owner signals to identify what actually stopped the job.
  • Actionable Investigation Workflow: Collects job details via bjobs/bhist, extracts required fields (exit code, TERM_*, peak usage vs limits, CWD, command, stdout/stderr), and outputs a structured diagnosis report with immediate remediation steps.

Quick Start

Ask: “I have an LSF job failure—what caused my job to EXIT? Please diagnose using the job’s exit code, TERM_* reason, and stdout/stderr logs.”

Frequently Asked Questions about lsf_job_fail

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I diagnose LSF job failures from exit codes and TERM reasons?▼

Diagnose OpenLava job EXIT failures by mapping exit codes to root causes: 1-127 indicates application errors, while >=128 signifies signal-driven termination. Extract bhist data to separate script issues from system kills.

What does exit code 137 mean in LSF job failure diagnosis?▼

Exit code 137 in LSF signifies signal-driven termination, typically an out-of-memory or OOM kill. It maps to SIGKILL, separating system resource kills from standard application errors.

How do I investigate TERM_MEMLIMIT and TERM_CPULIMIT reasons in OpenLava?▼

Investigate TERM_MEMLIMIT and TERM_CPULIMIT by extracting peak usage and limit thresholds via bjobs/bhist. This workflow identifies whether resource limits or admin/owner signals actually stopped the OpenLava job.

Does LSF job failure diagnosis work with OpenLava and Volclava?▼

Yes, LSF job failure diagnosis applies to OpenLava and Volclava. It interprets exit codes and TERM_* termination reasons across these HPC monitoring platforms to determine the most likely root cause.

Why did my HPC job EXIT with a segfault or admin kill?▼

Your HPC job EXITed with a segfault or admin kill due to signal-driven termination. The diagnosis translates exit codes >=128 and TERM_* reasons to identify OOM, segfaults, or admin signals.

Related Skills

Explore More Software Engineering Skills→
raas-devraas-dev
19

verification-before-completion

Run fresh verification commands and provide output evidence before completion claims.

Official
Basic
haposofthaposoft
54

hapo:hotfix

Guide structured diagnosis and evidence-based verification for code issue resolution.

Official
Advanced
mapachekurtmapachekurt
1

token-budget-monitor

Monitor Claude conversation token usage and issue warnings at critical thresholds.

Community
Basic
openeuler-mirroropeneuler-mirror
3

root-cause-localization

Correlate logs, trace spans, and dependency graphs to output structured JSON root cause reports.

Official
Advanced