skills.rest
lukefitzg
lukefitzgCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

tinysocs-research-methodology

Guides enable, disable, and tuning decisions for TinySocs detection rules using evidence-based judgment.

Updated Oct 12, 2025
Search Tags:#threat detection#false positives#opensearch#siem#rule tuning#detection engineering#windows events
One-click install
npx skills add https://github.com/lukefitzg/tinysocs --skill tinysocs-research-methodology-lukefitzg
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: tinysocs-research-methodology
Source: https://github.com/lukefitzg/tinysocs/tree/main/.claude/skills/tinysocs-research-methodology
Command: npx skills add https://github.com/lukefitzg/tinysocs --skill tinysocs-research-methodology-lukefitzg

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Deciding whether a detection rule should fire in a real small-business environment is a judgment call, not a mechanical task — a rule that false-positives on day one destroys customer trust. This Skill provides the decision framework for enabling, disabling, tuning, or retiring TinySocs detection rules, grounded in the repo's documented 2026-07-04 pilot cut that disabled 17 of 39 rules. ## Core Features & Use Cases - Three-class evidence taxonomy: Classify failing rules as will-alarm-on-normal-behaviour (disable, needs schema capability), mislabeled (fix with a field_match filter), or dead/unverifiable (needs plumbing, not a filter), each with a distinct fix path. - Hypothesis-before-testing discipline: Write down expected event counts versus thresholds before running a harness test, so tests that pass "for free" (like the TS-002 18-vs-20 non-test) are caught as non-evidence. - Rule lifecycle state machine: Move rules through candidate, test-authored, harness-validated, and pilot-pack-enabled states, with disabled rules retained as documented assets rather than silently deleted. - FP-tuning judgment patterns: Apply recurring reasoning patterns (threshold-as-human-vs-automation detector, the RMM/encoded-PowerShell hazard, alert-on-scan framing) plus an OpenSearch terms-aggregation query to pull real event volume before deciding. - Use Case: When asked "why is TS-NNN disabled" or "should I flip enabled: true on this new rule", load this Skill to evaluate the rule against the pilot evidence bar, check its event source, and write the inline pilot: rationale. ## Quick Start Ask the assistant to evaluate whether a specific TinySocs detection rule should be enabled, disabled, or tuned, and to justify the call using the pilot evidence bar and lifecycle state machine.

Frequently Asked Questions about tinysocs-research-methodology

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I decide whether a detection rule should be enabled or disabled?▼

Apply the three-question evidence bar: low false positives in a normal SMB estate, high signal when it fires, and explainable in one sentence. When in doubt, keep the rule disabled — a week-one false positive kills customer trust while a false negative is invisible.

How do I fix a detection rule that fires on normal behavior?▼

First classify the failure: if the description promises a narrow condition but the YAML lacks a field_match filter, add one (the mislabeled class). If the event source itself is wide open, like Sysmon FileCreate, the rule needs schema capability that does not exist yet and should stay disabled.

Why did my detection rule test pass but prove nothing?▼

A test passes vacuously when the input never reaches the rule's threshold, filter, or group key — like firing 18 events against a threshold of 20. Before running, state the expected count-versus-threshold relationship and what a broken implementation would produce instead.

What is the difference between a noisy rule and a dead rule?▼

Ask whether a perfect textbook attack would fire the rule. If no, because the event never reaches the engine (empty Sysmon include list, missing emitter, unpopulated group-by field), it is dead and needs a plumbing fix. If yes but routine software also fires it, it is noisy and needs filtering or schema capability.

When should a disabled detection rule be deleted from rules.yml?▼

Never. Disabled rules are retained assets documenting a decision, kept with an inline pilot: reason explaining why they are off. Silent deletion loses the institutional knowledge of why the rule failed the evidence bar.

How do I check real event volume before tuning a rule threshold?▼

Query the tinysocs-winlog-* index on OpenSearch port 9201 with a terms aggregation over the rule's group_by field for a representative window, such as seven days. Read the top-N buckets, not just totals, to identify what a field_match exclusion should target.

Related Skills

Explore More Software Engineering Skills→
Yenn503Yenn503
1

performing-threat-emulation-with-atomic-red-team

Executes Atomic Red Team tests to validate MITRE ATT&CK detection coverage.

Community
Intermediate
abrahamgreysonabrahamgreyson

document-release

Updates README, CHANGELOG, and related docs to match shipped code changes.

Community
Advanced
alvarovillalbaaalvarovillalbaa

pentest

Validate suspected vulnerabilities through authorized offensive testing on staging targets.

Community
Advanced
xalgordxalgord
954

building-detection-rules-with-sigma

Create vendor-agnostic Sigma detection rules and convert them to Splunk, Elastic, and Sentinel queries.

Community
Advanced