skills.rest
mims-harvard
mims-harvardOFFICIAL
Explore all skills by this author →
Website𝕏@marinkazitnikGitHub

tooluniverse-diagnostic-test-evaluation

Evaluate diagnostic test accuracy with 2x2 metrics, ROC analysis, and Bayesian post-test probability.

1.7k|254|Updated Mar 3, 2025
Search Tags:#epidemiology#specificity#sensitivity#biomarker#AUC#diagnostic test#roc analysis
One-click install
npx skills add https://github.com/mims-harvard/ToolUniverse --skill tooluniverse-diagnostic-test-evaluation
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: tooluniverse-diagnostic-test-evaluation
Source: https://github.com/mims-harvard/ToolUniverse/tree/main/plugins/tooluniverse/skills/tooluniverse-diagnostic-test-evaluation
Command: npx skills add https://github.com/mims-harvard/ToolUniverse --skill tooluniverse-diagnostic-test-evaluation

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires numpy, scikit-learn, and includes scripts (resource) components.

What problem does it solve?

Clinicians and researchers need to judge how well a diagnostic test or biomarker discriminates disease, choose an operating threshold, and translate a test result into a real probability of disease, but PPV/NPV are often misquoted without accounting for prevalence.

Core Features & Use Cases

  • Fixed-cutoff 2x2 metrics: Compute sensitivity, specificity, PPV, NPV, accuracy, and likelihood ratios from TP/FP/TN/FN counts via the Epidemiology_diagnostic tool.
  • ROC / AUC / optimal cutoff: Analyze a continuous biomarker score against true labels with AUC, bootstrap 95% CI, and the Youden-optimal cutoff using the ROC_analysis tool or the bundled Python script.
  • Post-test probability via Bayes: Convert pre-test probability plus sensitivity/specificity into a real-world post-test probability with Epidemiology_bayesian.
  • Use Case: Given a CSV of biomarker scores and disease labels, compute the AUC with confidence interval, pick a cutoff, build the 2x2 table at that cutoff, and report the PPV at the true population prevalence.

Quick Start

Ask the agent to evaluate a diagnostic test from a 2x2 table with 90 true positives, 10 false positives, 180 true negatives, and 20 false negatives, including sensitivity, specificity, and likelihood ratios.

Frequently Asked Questions about tooluniverse-diagnostic-test-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I calculate sensitivity and specificity from a 2x2 table?▼

Provide the TP, FP, TN, and FN counts to the Epidemiology_diagnostic tool, which returns sensitivity, specificity, PPV, NPV, accuracy, and positive and negative likelihood ratios in one call.

How do I find the optimal cutoff for a continuous biomarker?▼

Run ROC analysis on the biomarker scores and true labels to get the AUC with a bootstrap 95% CI and the Youden-optimal cutoff that maximizes sensitivity plus specificity minus one, along with its sens/spec.

Why is PPV low even when sensitivity and specificity are high?▼

PPV and NPV depend on disease prevalence in the tested population, unlike sensitivity and specificity. In low-prevalence screening settings, even a 90%/95% test yields a post-positive probability of only about 67% at 10% prevalence.

What input format does the ROC analysis script accept?▼

The script reads a CSV with columns label (1 for disease, 0 for healthy) and score (continuous biomarker value). It requires both positive and negative labels and uses numpy and scikit-learn for AUC and ROC computation.

What are the limitations of AUC for evaluating a diagnostic test?▼

AUC summarizes discrimination across all thresholds but ignores the operating point, so report sensitivity and specificity at the chosen cutoff too. Comparing two tests on the same patients requires a paired test like DeLong, which is not included.

Related Skills

Explore More Data & Analytics Skills→
writerwriter
6

care-gap-identification

Identify missing or overdue care steps against HEDIS, STAR, and USPSTF measures.

Official
Advanced
choxoschoxos
9

meta-analysis

Compute Bayesian meta-analysis with Stan and JAGS across fixed and random effects.

Community
Advanced
mrl2013mrl2013

logistic-regression-basics

Fit logistic regression models and interpret odds ratios for binary outcomes.

Community
Intermediate
obinopaulobinopaul
2

ml-pipeline

Develop and optimize scikit-learn machine learning pipelines for model training, evaluation, and selection.

Community
Intermediate