skills.rest
netdata
netdataOFFICIAL
Explore all skills by this author →
WebsiteGitHub

query-agent-events

Diagnose Netdata agent crashes and fatal errors by analyzing systemd-journal events across the infrastructure fleet.

80.4k|6.6k|Updated Jun 17, 2013
Search Tags:#Debugging#log-analysis#crash-triage#Netdata#Observability#systemd-journal
One-click install
npx skills add https://github.com/netdata/netdata --skill query-agent-events
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: query-agent-events
Source: https://github.com/netdata/netdata/tree/main/.agents/skills/query-agent-events
Command: npx skills add https://github.com/netdata/netdata --skill query-agent-events

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires jq, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill addresses the difficulty of triaging thousands of daily agent crashes by providing a structured, index-friendly method to filter, cluster, and analyze agent-events logs.

Core Features & Use Cases

  • Structured Triage: Filter events by crash class, exit cause, version, or signal code using index-friendly facets.
  • Automated Analysis: Group crash data by function, signal, or architecture to identify regression patterns and environment-specific bugs.
  • Use Case: When a new version causes a spike in SIGSEGV crashes, use this Skill to isolate the affected function and correlate it with specific OS distributions or hardware architectures.

Quick Start

Use the query-agent-events skill to fetch signal crashes from the last 24 hours and analyze them by fatal function.

Frequently Asked Questions about query-agent-events

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find recent signal crashes for Netdata agents?▼

Use the get-events.sh script with the --health crash flag to fetch recent signal-related events. Then, pipe the output to analyze-events.sh --by signal to see the distribution of SIGSEGV, SIGBUS, and other fatal signals.

What is the difference between a fatal exit and a signal crash?▼

A signal crash occurs when the kernel delivers a fatal signal like SIGSEGV or SIGBUS to the agent. A fatal exit is a deliberate shutdown initiated by the agent itself due to conditions like OOM, disk full, or assertion failures.

Why are my queries for agent-events slow?▼

Queries are slow when using full-text search (FTS) over a wide time window without structured filters. Always include at least one index-friendly field like AE_AGENT_HEALTH or AE_EXIT_CAUSE in your selections to narrow the dataset before applying FTS.

Can I use this skill to investigate regressions across versions?▼

Yes, use the --versions auto flag to compare crash counts between stable releases and nightlies. The analyze-events.sh script can group these results by version to help you pinpoint when a specific crash signature first appeared.

What are the limitations of the agent-events dataset?▼

The dataset is not real-time because agents only POST status events upon restart. Additionally, client-side deduplication suppresses identical crash signatures from the same agent within a 23-hour window, ensuring one record per agent per event-class per day.

Related Skills

Explore More Software Engineering Skills→
netdatanetdata
1

netdata-mcp-integration

Query live Netdata metrics and alerts via MCP tools.

Official
Intermediate
getsentrygetsentry
20

sentry-fix-issues

Identify and fix Sentry production issues with validated code changes.

Official
Advanced
virtualrusselvirtualrussel

dt-obs-hosts

Inventory and monitor hosts, processes, and containers across cloud and on-prem environments.

Community
Advanced
Soul-Brews-StudioSoul-Brews-Studio
116

warp

Manage interactive SSH and tmux sessions on remote Oracle nodes.

Official
Advanced