skills.rest
lukefitzg
lukefitzgCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

tinysocs-validation-publication-campaign

Repairs and republishes TinySocs validation pipeline results across dashboards, packs, and weekly runs.

Updated Oct 12, 2025
Search Tags:#validation#security testing#dashboard#opensearch#detection rules#siem#ci pipeline
One-click install
npx skills add https://github.com/lukefitzg/tinysocs --skill tinysocs-validation-publication-campaign-lukefitzg
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: tinysocs-validation-publication-campaign
Source: https://github.com/lukefitzg/tinysocs/tree/main/.claude/skills/tinysocs-validation-publication-campaign
Command: npx skills add https://github.com/lukefitzg/tinysocs --skill tinysocs-validation-publication-campaign-lukefitzg

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? The TinySocs public validation story is stale and misleading: the dashboard shows an old 53.85% pass rate with disabled rules counted as PASS, the weekly validation run has been dead for weeks, and signed packs ship with null validation metadata. This Skill provides a phased, multi-session campaign to fix the attribution bug, normalize the latest run, rebuild the dashboard, restart the cadence, and stamp real validation data into signed artifacts. ## Core Features & Use Cases - Status-assessment preamble: Runs a battery of checks (git state, grep probes, JSON diffs) to determine which of the numbered phases (P0-P7) are already complete before resuming work. - Phased pipeline repair: Fixes the noise-blind rule attribution bug in validation_lib.py, decides how pilot_status:deferred techniques render, normalizes the 2026-07-08 run into results/, and rebuilds site/validation/data/summary.json. - Cadence and provenance restoration: Restarts the weekly Windows VM validation run, adds a freshness check script, and stamps validation metadata into the signed pack before re-signing. - Use Case: A maintainer returning after weeks away asks an AI to "fix the validation dashboard and publish the real numbers"; the Skill assesses current state, then executes only the incomplete phases in dependency order with change-control gates. ## Quick Start Ask the AI to run the status-assessment preamble and resume the TinySocs validation publication campaign from whichever phase is incomplete.

Frequently Asked Questions about tinysocs-validation-publication-campaign

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fix a stale validation dashboard that disagrees with internal results?▼

Rebuild the dashboard summary by running build_validation_summary.py with --results-dir results, --rules packaging/detection/rules.yml, and --output site/validation/data/summary.json. Never hand-edit summary.json since it is a generated artifact overwritten by CI.

How do I resume a multi-phase validation repair campaign in a new session?▼

Run the status-assessment preamble first: check git state, grep validation_lib.py for the attribution fix and deferred category, inspect results/ for recent runs, and diff results/latest.json against the site summary. The check table maps each result to the correct starting phase.

Why does a detection test pass when the wrong rule fired?▼

The harness queries OpenSearch with a should-clause over all expected_rules and marks the technique DETECTED if any co-listed rule fires. The fix adds a primary_rule field and requires it in detected_rules for a PASS, implemented in validation_lib.py's categorize() and normalize_result().

Does normalizing a validation run overwrite previous weekly results?▼

Yes. The normalizer names output files by ISO week only, so two runs in the same week silently overwrite each other even with different --run-seq values. Preserving multiple runs per week requires changing the output filename to include run_seq.

How do I restart a dead weekly validation cadence on a Windows VM?▼

Check Get-ScheduledTaskInfo for the TinySocs Weekly Validation task, verify the SSH deploy key still authenticates to GitHub, then trigger run_weekly_validation.ps1 manually to re-prove the chain. Add a freshness check script that fails when results/latest.json is over 8 days old.

When should validation metadata be stamped into a signed pack?▼

Stamp the metadata.validation block before signing, because pack_sign.py's canonical_bytes() covers that block. Stamping after signing produces a signature that does not cover the validation data, so the stamp must land first and the pack re-signed.

Related Skills

Explore More Software Engineering Skills→
browseros-aibrowseros-ai
13.4k

ask-internal

Answers internal BrowserOS questions by searching internal docs and codebase with file:line citations.

Official
Intermediate
slgoodrichslgoodrich
121

competitive-analysis-templates

Create competitive intelligence templates and battle cards for sales enablement.

Community
Basic
jaktestowacjaktestowac
113

test-driven-development

Drives implementation through red-green-refactor cycles with failing tests written before production code.

Community
Advanced
RohithdgrrRohithdgrr

ci-cd-pipeline-builder

Generate GitHub Actions or GitLab CI pipelines from detected repository stack signals.

Community
Intermediate