skills.rest
Overlord-Z
Overlord-ZCOMMUNITY
Explore all skills by this author →
GitHub

evaluator

Collect anonymous usage telemetry for ClaudeShack skills via GitHub Issues.

1|Updated Nov 19, 2025
Search Tags:#automation#feedback#telemetry#skill improvement#privacy#usage analytics#github integration
One-click install
npx skills add https://github.com/Overlord-Z/ClaudeShack --skill evaluator
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: evaluator
Source: https://github.com/Overlord-Z/ClaudeShack/tree/main/skills/evaluator
Command: npx skills add https://github.com/Overlord-Z/ClaudeShack --skill evaluator

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

This skill addresses the challenge of improving AI tools without compromising user privacy. It provides a transparent, opt-in framework for collecting anonymous usage data and feedback, enabling developers to enhance skills based on real-world interactions and user needs. It eliminates guesswork in feature prioritization and bug fixing, ensuring that AI works better for everyone.

Core Features & Use Cases

  • Privacy-First Telemetry: Collects anonymous usage events (skill invoked, success/failure, duration) with no PII, ensuring user trust and compliance.
  • GitHub-Native Feedback: Leverages GitHub Issues and Projects for structured feedback, bug reports, and improvement suggestions, streamlining the feedback loop.
  • Skill-Specific Metrics: Tracks detailed metrics for integrated skills like Oracle (query success rate), Guardian (suggestion acceptance), and Summoner (subagent spawn frequency), providing actionable insights.
  • Use Case: A skill developer wants to understand which features of their new Claude Skill are most used and where users encounter errors. They integrate Evaluator to collect anonymous data, which then informs their development roadmap and helps prioritize bug fixes, leading to a more robust and user-centric skill that truly serves its users.

Quick Start

To enable anonymous telemetry for your project: /evaluator enable To submit feedback on a skill: /evaluator feedback

Frequently Asked Questions about evaluator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I collect anonymous usage data for my Claude skills without compromising privacy?▼

Anonymous usage telemetry lets you track skill invocations, performance, and errors while preserving privacy through opt-in collection, no PII storage, and daily-rotating anonymous IDs. Enable it with /evaluator enable to gather actionable metrics without user data exposure.

Can I track skill performance metrics like success rates and error patterns?▼

Yes, skill-specific metrics capture invocation success/failure, execution duration, and error details. Evaluator integrates with GitHub Issues and Projects to log and visualize these patterns, enabling data-driven prioritization of fixes and features.

How do I submit and manage feedback on skill improvements?▼

Use /evaluator feedback to route user suggestions and bug reports directly into GitHub Issues and Projects. This centralizes feedback collection in a structured format, streamlining the loop between telemetry insights and development action.

Does evaluator comply with privacy regulations for telemetry collection?▼

Evaluator satisfies privacy requirements through opt-in consent, zero PII collection, local data control, configurable exclusions, and user transparency. These safeguards ensure compliance with privacy standards while enabling aggregate reporting.

What skills can integrate evaluator for tracking usage insights?▼

Evaluator applies across all ClaudeShack skills. It tracks specialized metrics for skills like Oracle (query success), Guardian (suggestion acceptance), and Summoner (subagent spawn frequency), adapting to each skill's specific performance indicators.

Why should I use GitHub-native feedback instead of other feedback channels?▼

GitHub integration centralizes feedback alongside issues and project planning, eliminating data silos and letting developers act directly on telemetry. This keeps skill improvement tightly coupled to your existing development workflow and version control.

Related Skills

Explore More Product & Management Skills→
JeremiahEllingtonJeremiahEllington

canary

Monitor deployed URLs for console errors, performance regressions, and page failures against baselines.

Community
Advanced
olaservoolaservo
10

github-issue-analyzer

Analyze GitHub issues and generate a prioritized markdown report with HIGH, MEDIUM, LOW, and Good First Issues categories.

Community
Intermediate
shogo-labsshogo-labs
12

dev-activity-track

Aggregate GitHub commits, PRs, and reviews into per-developer dashboards.

Official
Advanced
simotasimota
68

Pulse

Design metrics frameworks for product analytics with event schemas and dashboards.

Community
Advanced