triage-rule-hits

Classify Osprey rule hits into true positive, false positive, novel, or uncertain categories.

6|Updated Feb 21, 2026
One-click install
npx skills add https://github.com/skywatch-bsky/skywatch-agent-skills --skill triage-rule-hits
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: triage-rule-hits
Source: https://github.com/skywatch-bsky/skywatch-agent-skills/tree/main/claude-skills/plugins/skywatch-investigations/skills/triage-rule-hits
Command: npx skills add https://github.com/skywatch-bsky/skywatch-agent-skills --skill triage-rule-hits

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill addresses the challenge of rule drift and maintenance by providing a structured methodology to evaluate Osprey rule performance, identify false positives, and surface novel detection patterns.

Core Features & Use Cases

  • Stratified Sampling: Automatically samples rule hits across a specified time window to ensure a representative view of rule behavior.
  • Classification Schema: Provides a standardized framework to categorize hits as true positives, false positives, novel patterns, or uncertain.
  • Actionable Reporting: Generates aggregate health assessments and specific recommendations for rule refinement or keyword adjustments.
  • Use Case: Use this during Phase 5 of an investigation to determine if a rule is drifting due to emerging satire or historical references, allowing for precise rule tuning.

Quick Start

Run the triage-rule-hits skill for the rule election_misinfo_en over the last 30 days with a sample size of 50.

Frequently Asked Questions about triage-rule-hits

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I detect and triage false positives in Osprey rule execution results?

To triage false positives in Osprey rule execution results, you sample hits across a specified time window and classify them into true positive, false positive, novel, or uncertain categories to identify rule drift and actionable adjustments.

What is rule drift and how do I validate detection rules against emerging patterns?

Rule drift occurs when detection rules lose accuracy due to emerging satire or historical references. You validate rules by sampling execution results and classifying hits to surface novel detection patterns and recommend keyword adjustments.

How do I run stratified sampling on Osprey rules for a moderation investigation?

You run stratified sampling on Osprey rules by specifying a rule name, time window, and sample size. This automatically samples rule hits to ensure a representative view of rule behavior for your moderation investigation.

What data access is required to perform statistical aggregation on Osprey rule hits?

Performing statistical aggregation on Osprey rule hits requires access to osprey_execution_results and data-analyst dispatch capabilities to classify hits and generate aggregate health assessments.

When should I use automated rule validation during a data investigation?

You should use automated rule validation during Phase 5 of a data investigation to determine if a rule is drifting, allowing for precise rule tuning and keyword adjustments based on aggregate health assessments.