skills.rest
PostHog
PostHogOFFICIAL
Explore all skills by this author →
Website𝕏@PostHogGitHub

exploring-mcp-tool-quality

Analyze error rates, latency, and reach of PostHog MCP tool calls.

713|118|Updated Aug 11, 2020
Search Tags:#mcp#latency#Analytics#PostHog#error rate#HogQL#tool quality
One-click install
npx skills add https://github.com/PostHog/posthog-foss --skill exploring-mcp-tool-quality
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: exploring-mcp-tool-quality
Source: https://github.com/PostHog/posthog-foss/tree/main/products/mcp_analytics/skills/exploring-mcp-tool-quality
Command: npx skills add https://github.com/PostHog/posthog-foss --skill exploring-mcp-tool-quality

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

When MCP servers are instrumented with PostHog's MCP analytics SDK, teams need to know which tools are failing, which are slow, and how reliable each tool is. This Skill turns raw $mcp_tool_call events into ranked error-rate tables, latency percentiles, and failure breakdowns so you can pinpoint problematic tools without hand-writing every query.

Core Features & Use Cases

  • Cross-tool error ranking: Rank all MCP tools by error rate with a call-volume floor to avoid small-sample noise, using HogQL via posthog:execute-sql.
  • Per-tool drill-down: Use typed tools like posthog:query-mcp-tool-stats, posthog:query-mcp-tool-failures, and posthog:query-mcp-tool-daily-stats for calls, errors, p50/p95 latency, users, and failure buckets of a single tool.
  • Tool-quality matrix and UI links: Build a one-row-per-tool matrix of error rate, latency, and reach, and generate PostHog dashboard and tool-quality URLs for visual verification.
  • Use Case: Ask "which MCP tool has the highest error rate this month?" and get a ranked table of tools with error rates and call volumes, then drill into the worst tool's top failure messages grouped by harness.

Quick Start

Ask which MCP tool has the highest error rate over the last 30 days and request a ranked breakdown with call volumes.

Frequently Asked Questions about exploring-mcp-tool-quality

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find which MCP tool has the highest error rate?▼

Run a HogQL query over $mcp_tool_call events grouping by the effective tool name, computing countIf(toBool($mcp_is_error)) divided by total calls. Add a HAVING floor on call volume, such as 20 calls, so low-traffic tools do not top the ranking spuriously.

How do I investigate why a specific MCP tool is failing?▼

Call posthog:query-mcp-tool-failures with the tool name to get top failure buckets grouped by $mcp_error_type and HTTP status. Then pass a bucket's error_type and error_status to posthog:query-mcp-tool-failure-occurrences to see individual errored calls with messages, sessions, and harness.

Why should I use the effective tool name instead of $mcp_tool_name?▼

New-SDK events wrap the real tool in a single-exec call, so grouping on raw $mcp_tool_name collapses everything under the wrapper. Use coalesce(nullIf(toString($mcp_exec_tool_call_name), ''), toString($mcp_tool_name)) to group by the actual tool.

When should I use typed MCP tools versus hand-written HogQL?▼

Use typed tools like posthog:query-mcp-tool-stats and posthog:query-mcp-tool-failures for single-tool questions since they mirror the UI and need no SQL. Use posthog:execute-sql for cross-tool questions like error-rate rankings, which have no typed equivalent.

Does MCP analytics data live in a dedicated ClickHouse table?▼

No, there is no dedicated table. Every MCP metric is an aggregation over $mcp_tool_call events on the shared events table, with all fields stored as $mcp_* properties, so queries must cast string properties with toBool and toFloat.

Related Skills

Explore More Data & Analytics Skills→
PostHogPostHog
58

integration-javascript_node

Instrument Node.js applications with PostHog to capture events and identify users.

Official
Advanced
FrekiManagarmFrekiManagarm

exploring-llm-clusters

Analyze PostHog LLM clustering runs to compute cost, latency, and error metrics.

Community
Intermediate
harnessprotocolharnessprotocol
9

harness-sync

Synchronize AI coding tool configurations across Claude Code, Cursor, and GitHub Copilot.

Official
Advanced
DojoCodingLabsDojoCodingLabs
2

posthog-analytics

Create PostHog dashboards and run HogQL queries for analytics, funnel, and cohort reporting.

Official
Intermediate