skills.rest
marin-community
marin-communityOFFICIAL
Explore all skills by this author →
WebsiteGitHub

zephyr-debugger

Diagnose Zephyr pipeline issues by analyzing logs, profiling, and resource data.

1.2k|146|Updated Mar 22, 2024
Search Tags:#performance#debugging#pipeline#workers#zephyr#stragglers
One-click install
npx skills add https://github.com/marin-community/marin --skill zephyr-debugger
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: zephyr-debugger
Source: https://github.com/marin-community/marin/tree/main/.agents/skills/zephyr-debugger
Command: npx skills add https://github.com/marin-community/marin --skill zephyr-debugger

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill helps diagnose and resolve issues within Zephyr pipeline executions, specifically addressing problems like stuck stages, slow-performing tasks (stragglers), and idle worker processes.

Core Features & Use Cases

  • Pipeline Monitoring: Provides tools to inspect the state and logs of Zephyr coordinator and worker processes.
  • Straggler Detection: Identifies tasks that are taking an unusually long time to complete, hindering pipeline progress.
  • Resource Analysis: Offers methods to profile CPU, memory, and thread usage on workers and tasks to pinpoint performance bottlenecks.
  • Use Case: When a Zephyr data processing pipeline is running much slower than expected or has completely stalled, this Skill can be used to identify which specific stage or task is causing the delay and why.

Quick Start

Use the zephyr-debugger skill to get the job logs for job ID 'example-job-123'.

Frequently Asked Questions about zephyr-debugger

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I debug a Zephyr pipeline that is stuck or has idle workers?▼

To debug a stuck Zephyr pipeline, inspect the state and logs of coordinator and worker processes. Analyzing observability data from dashboards and CLIs helps identify idle workers and resolve execution stalls.

How do I find stragglers slowing down my Zephyr pipeline execution?▼

Find stragglers in a Zephyr pipeline by identifying tasks taking unusually long to complete. Examine task logs and thread stacks to pinpoint which specific stage is hindering pipeline progress.

How do I profile memory and CPU usage on Zephyr workers to resolve data skew?▼

Profile CPU and memory usage on Zephyr workers using on-demand tools like py-spy and memray. Examining memory distribution and thread stacks pinpoints resource bottlenecks causing data skew.

What is the best way to diagnose Zephyr worker failures and performance bottlenecks?▼

The best way to diagnose Zephyr worker failures is to analyze pipeline architecture and on-demand profiling data. Examining task logs, memory distribution, and thread stacks reveals the root cause of performance bottlenecks.

Can I retrieve specific job logs from a stalled Zephyr data processing pipeline?▼

Yes, you can retrieve specific job logs from a stalled Zephyr pipeline. Requesting logs for a specific job ID allows you to inspect the state of coordinator and worker processes to identify the delay.

Related Skills

Explore More Software Engineering Skills→
lygavlygav

otel-observability

Instrument federated squad agents with OpenTelemetry to collect traces, metrics, and logs.

Community
Advanced
pluginagentmarketplacepluginagentmarketplace
30

memory-management

Implements object pooling, memory budgets, and profiling for cross-platform game projects.

Community
Advanced
wfukatsuwfukatsu
12

design-observability

Design observability systems with SLI/SLOs, distributed tracing, and log aggregation.

Community
Advanced
aztr0nutzsaztr0nutzs
2

codex-orchestration

Orchestrate parallel Codex AI worker agents via background PTY terminals.

Community
Advanced