skills.rest
promptranks
promptranksOFFICIAL
Explore all skills by this author →
GitHub

prk-psv-qa

Detects and validates problematic patterns in PSV samples, generating actionable-quality reports and optionally correcting them via git commit.

Updated Apr 12, 2026
Search Tags:#validation#quality#calibration#ground-truth#prompt-evaluation#psv
One-click install
npx skills add https://github.com/promptranks/prk-psv-flow --skill prk-psv-qa
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: prk-psv-qa
Source: https://github.com/promptranks/prk-psv-flow/tree/main/plugins/prk-psv-flow/skills/prk-psv-qa
Command: npx skills add https://github.com/promptranks/prk-psv-flow --skill prk-psv-qa

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

PSV samples often lack consistent quality and realism, making validation and calibration difficult. This skill provides structured quality checks to ensure prompts, contexts, and ground-truth ratings meet defined standards.

Core Features & Use Cases

  • Quality validation of PSV sample prompts, contexts, and outputs.
  • Ground-truth calibration checks to ensure ratings align with PECAM levels.
  • Reporting & revision: generates qa_status, qa_score, and feedback; updates YAML and produces a validation report.

Quick Start

Validate a batch of PSV samples using the default YAML file and review the validation report.

Frequently Asked Questions about prk-psv-qa

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I validate PSV sample quality and realism before export?▼

To validate PSV sample quality, apply structured checks across levels L1-L5 to assess task context, prompt authenticity, and ground truth accuracy. The process updates qa_status and qa_score, then outputs a validation report with revision feedback.

What is ground-truth calibration in PSV datasets?▼

Ground-truth calibration ensures that sample ratings align consistently with PECAM levels across all PSV datasets. This mechanism verifies that quality dimensions meet defined standards, preventing inconsistent or unrealistic prompt evaluations.

How do I run quality checks on a YAML file of PSV samples?▼

To run quality checks on PSV samples, validate the batch using your default YAML file. The system enforces checks on prompts, contexts, and outputs, updates the YAML with qa_status, and produces a validation report.

Can I assess PSV quality dimensions across multiple levels?▼

Yes, you can assess PSV quality dimensions across levels L1-L5, including pillars and task context. This ensures comprehensive coverage of prompt authenticity and ground truth accuracy before finalizing dataset export.

Why does my PSV dataset lack consistent prompt evaluation?▼

PSV datasets lack consistent prompt evaluation when structured quality checks are missing. Applying validation to task context, prompt authenticity, and ground truth calibration enforces defined standards and resolves rating inconsistencies.

Related Skills

Explore More Data & Analytics Skills→
Amar1404Amar1404
1

question-router

Classify questions into L1-L5 complexity levels and route them to response paths.

Community
Basic
promptrankspromptranks

prk-psv-export

Export validated PSV samples to SQL or JSON for database import.

Official
Intermediate
stanahstanah

skill-eval

Run unit tests and ground truth evaluations for the doc-code-sync skill.

Community
Advanced
whichguywhichguy

review-fix-bench

Benchmark code-reviewer agent prompts with parallel test runs and F1 metrics.

Community
Advanced