tinysocs-validation-and-qa

A rule is harness-validated only if it fired in an Atomic Red Team run under a rule definition equivalent to today's. Check tests/atomic-results.json for live attack-to-alert evidence, and confirm the xUnit tests in DetectionEngineTests.cs prove it fires on synthetic events.

xUnit tests fire synthetic AgentEvent objects at the C# DetectionEngine to prove a rule can fire and stay silent off-target. The Atomic harness runs real attack techniques on a Windows VM against live OpenSearch to prove the full pipeline produces an alert.

Same-week harness re-runs silently clobber run_id because normalize_validation_run.py defaults the sequence to 001. The public dashboard can go stale, showing an older unfiltered rule set while atomic-results.json reflects the newer curated run.

No. The March 100% figure predates the test-fidelity overhaul and is banned outright, and 57.1% is a raw first-run number that miscounted deliberately deferred rules as misses. Only the curated 88.9% (8 of 9 executed enabled-rule techniques) is quotable.

PowerShell 5.1's Schannel TLS handshake fails against the OpenSearch endpoint, and Invoke-RestMethod swallowed the exception into empty results, turning real detections into silent false misses. This is deliberate scar-tissue engineering that must not be simplified.

Do not use it for running the publication pipeline, authoring a new atomic test's fallback command, deciding whether a candidate rule is worth adding, or choosing externally safe efficacy figures. Those belong to the publication-campaign, validation-toolkit, research-methodology, and external-positioning skills respectively.