skills.rest
awslabs
awslabsOFFICIAL
Explore all skills by this author →
WebsiteGitHub

ml-researcher

Design and audit machine learning experiments for healthcare and life sciences.

13|5|Updated May 4, 2026
Search Tags:#regulatory#biomedical#machine-learning#experiment-design#clinical-validation#hcls
One-click install
npx skills add https://github.com/awslabs/hcls-agent-skills --skill ml-researcher-awslabs
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ml-researcher
Source: https://github.com/awslabs/hcls-agent-skills/tree/main/skills/ml-researcher
Command: npx skills add https://github.com/awslabs/hcls-agent-skills --skill ml-researcher-awslabs

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill addresses the high failure rate in healthcare and life sciences machine learning by enforcing rigorous methodology, preventing data leakage, and ensuring models are evaluated in ways that mirror clinical deployment.

Core Features & Use Cases

  • Methodological Guardrails: Provides decision frameworks for model selection, cross-validation strategies, and handling class imbalance to avoid common pitfalls like temporal leakage or overfitting.
  • Regulatory & Fairness Auditing: Guides the user through reporting standards like TRIPOD+AI and PROBAST, while ensuring fairness and clinical utility are assessed via decision curve analysis.
  • Use Case: Use this skill to critique a proposed ML pipeline for EHR-based mortality prediction, ensuring the index time is correctly defined and the evaluation strategy accounts for site-level generalization.

Quick Start

Use the ml-researcher skill to design a validation strategy for a clinical prediction model that accounts for temporal leakage and site-level generalization.

Frequently Asked Questions about ml-researcher

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I design a validation strategy for a clinical prediction model that prevents data leakage?▼

Designing a validation strategy for clinical prediction models requires applying methodological guardrails like nested cross-validation and site-level holdouts to prevent temporal leakage and ensure generalization. Proper index time definition is also critical.

What is TRIPOD+AI and how does it apply to healthcare machine learning experiments?▼

TRIPOD+AI is a reporting standard for healthcare machine learning experiments. It ensures scientific rigor by guiding regulatory and fairness auditing, helping researchers assess clinical utility via decision curve analysis and document model performance transparently.

How do I evaluate an ML pipeline for EHR-based mortality prediction?▼

Evaluating an ML pipeline for EHR-based mortality prediction involves ensuring the index time is correctly defined and applying robust evaluation strategies. The process requires accounting for site-level generalization and avoiding temporal leakage.

Can I use nested cross-validation for class imbalance in medical imaging datasets?▼

Nested cross-validation is highly suitable for handling class imbalance in medical imaging datasets. It provides methodological guardrails during model selection, ensuring robust evaluation that mirrors clinical deployment while preventing overfitting.

What's the best way to audit a clinical prediction model for regulatory compliance?▼

Auditing a clinical prediction model for regulatory compliance requires using frameworks like PROBAST and TRIPOD+AI. This ensures fairness and clinical utility are properly assessed through decision curve analysis and robust validation techniques.

When do I need site-level holdouts for genomics and drug discovery workflows?▼

Site-level holdouts are needed for genomics and drug discovery workflows when evaluating models for clinical deployment. They ensure the evaluation strategy mirrors real-world clinical environments and confirms site-level generalization.

Related Skills

Explore More Education & Research Skills→
jackspacejackspace
16

scikit-learn

Build, evaluate, and deploy classical machine learning models with scikit-learn.

Community
Advanced
awslabsawslabs
13

biomarker-discovery

Audit biomarker study designs for data leakage and validation rigor.

Official
Advanced
travisjneumantravisjneuman
88

statistics-verifier

Verify statistical claims by auditing methodology, significance testing, and bias.

Community
Advanced
andikarachmanandikarachman
14

target-leakage-detection

Detect temporal, direct, and group data leakage in machine-learning feature sets.

Community
Advanced