revyl-mcp-analyze

Classify failed Revyl MCP test executions into bug, flaky, infra, or improvement categories.

502|24|Updated Aug 5, 2024

Search Tags:#automation #mcp #classification #quality-assurance #analyze #test-executions

Ready to run

npx skills add https://github.com/RevylAI/revyl-cli --skill revyl-mcp-analyze

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: revyl-mcp-analyze
Source: https://github.com/RevylAI/revyl-cli/tree/main/skills/revyl-mcp-analyze
Command: npx skills add https://github.com/RevylAI/revyl-cli --skill revyl-mcp-analyze

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Identify failed Revyl MCP test executions and classify them into REAL BUG, FLAKY TEST, INFRA ISSUE, or TEST IMPROVEMENT, enabling faster triage and targeted fixes.

Core Features & Use Cases

Automated classification of MCP test failures based on status payloads and observed outcomes.
Generates actionable next steps and recommended actions for developers, SREs, and QA.
Use Case: When an MCP test run fails, run this skill to determine whether the issue is with the app, the test environment, or the test design, and to capture evidence for bug filing.

Quick Start

Run this skill after a failing MCP test to obtain a structured classification and recommended next actions.

Frequently Asked Questions about revyl-mcp-analyze

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I classify failed test executions to determine if they are real bugs or flaky tests?▼

Classify failed test executions by analyzing status payloads and failing steps to categorize them as REAL BUG, FLAKY TEST, INFRA ISSUE, or TEST IMPROVEMENT. This produces a structured triage output with confidence levels and evidence for targeted fixes.

What is the best way to automate triage for failed MCP test runs across CI and cloud devices?▼

Automate MCP test triage by running a classification skill on failing task runs to analyze status payloads. It generates structured outputs with classification, confidence, evidence, and next actions suitable for automated workflows and dashboards across CI and cloud-device scenarios.

Can I use automated test failure classification for local debugging scenarios?▼

Yes, automated test failure classification applies to local debugging scenarios alongside CI and cloud-device runs. It analyzes failing steps to determine whether the root cause is a real bug, flaky test, infrastructure issue, or test design improvement.

How does test failure classification integrate into automated developer workflows?▼

Test failure classification integrates into automated workflows by producing structured output containing Task, Classification, Confidence, Evidence, and Next action fields. Developers, SREs, and QA teams can use these actionable next steps directly in dashboards and CI pipelines.

Why do my automated tests fail and how can I tell if it is an infrastructure issue?▼

Automated tests fail due to real bugs, flaky tests, infrastructure issues, or test design flaws. Analyzing the status payload and failing steps classifies the failure as an INFRA ISSUE with supporting evidence and confidence levels, enabling targeted resolution.

Do I need to provide status payloads to classify failing MCP test steps accurately?▼

Yes, status payloads and failing step data are required to classify MCP test failures accurately. The classification mechanism analyzes these inputs to produce clear triage outputs with evidence and recommended next actions for bug filing and fixes.