skills.rest
ptreezh
ptreezhCOMMUNITY
Explore all skills by this author →
GitHub

validity-reliability

Compute reliability and validity metrics for research instruments with JSON output.

24|7|Updated Nov 15, 2025
Search Tags:#reliability#validity#psychometrics#scale development#item analysis#content validity#construct validity
One-click install
npx skills add https://github.com/ptreezh/sscisubagent-skills --skill validity-reliability
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: validity-reliability
Source: https://github.com/ptreezh/sscisubagent-skills/tree/main/archive/skills/validity-reliability
Command: npx skills add https://github.com/ptreezh/sscisubagent-skills --skill validity-reliability

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires numpy, scipy, pandas, and includes scripts (resource) and references (resource) components.

What problem does it solve?

It helps researchers evaluate whether a survey/scale instrument produces consistent (reliable) and meaningful (valid) measurements, reducing the risk of using poor-quality measurement tools in social science research.

Core Features & Use Cases

  • Reliability analysis: Compute internal consistency (Cronbach’s Alpha), split-half reliability, test-retest reliability, and inter-rater agreement to judge measurement stability and consistency.
  • Validity analysis: Estimate content validity, construct validity, criterion validity, and convergent/discriminant validity to verify whether the instrument measures the intended constructs.
  • Measurement quality assessment: Provide item-level diagnostics (e.g., item-total relatedness), factor analysis support, and model-based reporting guidance for improving instrument design.

Use case: You developed a new Likert scale for organizational research and need a defensible report covering internal consistency, expert-judged content validity, and evidence from construct/criterion relationships before publishing.

Quick Start

Use the validity-reliability skill to analyze your dataset for reliability (Cronbach’s Alpha, split-half, and test-retest/inter-rater when applicable) and validity (content, construct, criterion, and convergent/discriminant) and return a structured JSON report plus an interpretation-ready summary.

Frequently Asked Questions about validity-reliability

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I check Cronbach's Alpha and internal consistency for my Likert scale?▼

To check internal consistency for a Likert scale, you compute Cronbach's Alpha alongside split-half reliability and item-total statistics. This validity-reliability analysis processes structured survey inputs and outputs a JSON report with interpretable thresholds for measurement stability.

How do I run construct validity and content validity analysis for a questionnaire?▼

Construct and content validity analysis verifies whether a questionnaire measures its intended theoretical constructs. The process accepts structured item inputs to compute factor analysis support, expert-judged content validity, and convergent/discriminant evidence, returning a JSON summary with measurement quality diagnostics.

Can I use Python with pandas and scipy for psychometric scale development?▼

Yes, you can use Python with pandas, numpy, and scipy for psychometric scale development. This analysis environment supports item-level diagnostics, inter-rater agreement calculations, and factor analysis to evaluate survey instruments and output machine-readable JSON results.

What is the best way to assess inter-rater agreement and test-retest reliability?▼

The best way to assess inter-rater agreement and test-retest reliability is to compute stability coefficients from structured response data. This evaluates measurement consistency across different raters and time points, providing JSON results with statistical metrics to judge instrument quality.

Does item analysis include item-total relatedness diagnostics for survey development?▼

Yes, item analysis includes item-total relatedness diagnostics for survey development. It provides item-level metrics to identify poorly performing questions, supporting factor analysis and model-based reporting guidance to help you improve overall instrument design.

Related Skills

Explore More Education & Research Skills→
xjtulycxjtulyc
33

psychometrics

Run end-to-end psychometric validation on questionnaire item response datasets.

Community
Advanced
stratagastrataga

referral-program

Design referral and affiliate programs with incentive models and measurement plans.

Community
Basic
Ritabrata-ChakrabortyRitabrata-Chakraborty

exploratory-data-analysis

Automate exploratory data analysis on scientific data files and generate markdown reports.

Community
Advanced
rocopolasrocopolas
25

matematicas

Solve polynomial equations, derivatives, and statistical problems.

Community
Advanced