skills.rest
zhuangfanupup
zhuangfanupupCOMMUNITY
Explore all skills by this author →
GitHub

qa-tester

Automate QA execution and failure diagnosis for the Modular RAG MCP Server.

Updated Apr 1, 2026
Search Tags:#testing#qa#cli#mcp#dashboard#observability#regression
One-click install
npx skills add https://github.com/zhuangfanupup/modular-rag-mcp-server --skill qa-tester-zhuangfanupup
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: qa-tester
Source: https://github.com/zhuangfanupup/modular-rag-mcp-server/tree/main/.claude/skills/qa-tester
Command: npx skills add https://github.com/zhuangfanupup/modular-rag-mcp-server --skill qa-tester-zhuangfanupup

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pyyaml, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill automates end-to-end QA for the Modular RAG MCP Server, reducing the manual effort required to validate dashboards, CLI workflows, MCP tools, and data lifecycle behavior.

Core Features & Use Cases

  • Dashboard validation: Exercises Streamlit dashboard pages headlessly and checks rendered metrics, controls, and state-dependent views.
  • CLI workflow testing: Runs ingestion, querying, evaluation, and failure scenarios from the terminal with concrete output verification.
  • MCP protocol checks: Sends JSON-RPC requests to confirm server startup, tool availability, query behavior, and protocol stability.
  • Configuration and recovery scenarios: Switches provider profiles, injects invalid config states, and verifies fallback or error handling.
  • Multi-step lifecycle coverage: Tests ingest, delete, query, and collection-isolation flows while recording progress for traceable regression testing.

Quick Start

Use the qa-tester skill to run the next pending QA test, capture real command output, and record the verified result in QA_TEST_PROGRESS.md.

Frequently Asked Questions about qa-tester

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate QA testing for MCP server workflows and dashboard UI?▼

Automating QA testing for MCP server workflows and dashboard UI involves executing headless Streamlit checks, running CLI ingestion scenarios, and sending JSON-RPC protocol calls to validate server behavior. This Skill automates that end-to-end execution, capturing real output to verify results deterministically.

What is the best way to run regression tests for RAG ingestion and data lifecycle scenarios?▼

Running regression tests for RAG ingestion and data lifecycle scenarios requires multi-step execution covering ingest, delete, query, and collection-isolation flows. This process records progress in a tracking file, ensuring traceable validation of state bootstrapping and data lifecycle behavior for deterministic regression coverage.

How do I test MCP protocol stability using JSON-RPC requests?▼

Testing MCP protocol stability using JSON-RPC requests involves sending calls to confirm server startup, verify tool availability, and validate query behavior. This Skill automates these protocol checks to ensure the server responds correctly and maintains stability under various testing scenarios.

Can I validate Streamlit dashboard pages headlessly and check rendered metrics?▼

Validating Streamlit dashboard pages headlessly to check rendered metrics is fully supported. This Skill exercises dashboard UI pages without a browser, verifying rendered metrics, controls, and state-dependent views to ensure the interface behaves correctly during automated testing workflows.

Does this QA testing approach handle configuration switches and invalid config injection?▼

This QA testing approach handles configuration switches and invalid config injection by switching provider profiles and injecting invalid config states. It verifies fallback or error handling behavior, ensuring the RAG server maintains stability and proper recovery when configuration issues occur.

Why do my automated CLI workflow tests fail to capture deterministic output?▼

Automated CLI workflow tests fail to capture deterministic output when lacking serialized execution and evidence-backed notes. This Skill enforces serialized execution, state bootstrapping, and records verified results in QA_TEST_PROGRESS.md to validate outputs deterministically and prevent flaky test regressions.

Related Skills

Explore More Software Engineering Skills→
Aarong365Aarong365

qa-tester

Automates QA tests for Modular RAG MCP Server via predefined cases.

Community
Advanced
starlink-awakenstarlink-awaken

AnnualReports

Aggregate annual security reports and synthesize threat intelligence trends.

Community
Advanced
ljk486ljk486

qa-tester

Automate QA test execution, verification, and result logging for a modular RAG application.

Community
Advanced
mitul-bhatiamitul-bhatia

python-testing

Design and execute pytest-based unit and integration tests with TDD workflows.

Community
Intermediate