scientific-critical-thinking

Evaluate research methodology, statistical validity, biases, and evidence quality using GRADE and Cochrane frameworks.

1|Updated May 6, 2026
One-click install
npx skills add https://github.com/surfingalien/FinSurfing --skill scientific-critical-thinking-surfingalien
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: scientific-critical-thinking
Source: https://github.com/surfingalien/FinSurfing/tree/main/.claude/skills/scientific-critical-thinking
Command: npx skills add https://github.com/surfingalien/FinSurfing --skill scientific-critical-thinking-surfingalien

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Assessing whether scientific claims and research papers are trustworthy requires systematic evaluation of methodology, statistics, and bias—work that is easy to do superficially and hard to do rigorously. This Skill provides structured frameworks for critiquing research rigor so conclusions are proportional to actual evidence quality. ## Core Features & Use Cases - Methodology & Design Critique: Assess internal, external, construct, and statistical conclusion validity, including AI/ML-specific concerns like benchmark contamination and compute fairness. - Bias & Fallacy Detection: Identify cognitive, selection, measurement, and analysis biases plus 40 named logical fallacies with detection and mitigation strategies. - Evidence Quality Grading: Apply GRADE, Cochrane ROB, and evidence hierarchy standards, with a 20-point quick triage checklist for rapid paper screening. - Use Case: When reviewing a preprint claiming a new model beats baselines, use this Skill to check for cherry-picked benchmarks, missing ablations, p-hacking, and whether causal language is justified by the study design. ## Quick Start Ask the AI to critically evaluate the methodology and statistical validity of an attached research paper using the scientific-critical-thinking skill.

Frequently Asked Questions about scientific-critical-thinking

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I critically evaluate a research paper's methodology?

Assess study design appropriateness, internal validity (randomization, blinding, confounding control), external validity (sample representativeness), and statistical conclusion validity (power, test assumptions). The Skill provides a structured framework and a 20-point quick checklist for triage.

What is the GRADE system for evidence quality?

GRADE rates evidence as high, moderate, low, or very low quality. RCTs start high and are downgraded for risk of bias, inconsistency, indirectness, imprecision, or publication bias; observational studies can be upgraded for large effects or dose-response relationships.

How do I detect p-hacking and HARKing in published studies?

Check for preregistration on OSF or ClinicalTrials.gov and compare planned versus reported outcomes. Red flags include p-values clustering just below .05, undisclosed subgroup analyses, and hypotheses presented as a priori without registration.

What biases should I check when reviewing AI/ML papers?

Check for benchmark contamination (test data in pretraining), cherry-picked qualitative examples, unfair compute comparisons, missing code or weights for reproducibility, and inconsistent evaluation harnesses across compared models.

Can correlational studies ever support causal claims?

Rarely on their own, but strong observational evidence with large effects, dose-response relationships, temporal precedence, and controlled confounding can approach causal inference. Bradford Hill criteria and natural experiments strengthen such claims.

When should I distrust a statistically significant result?

Be cautious when studies are underpowered, multiple comparisons were uncorrected, effect sizes are unreported, or results come from small samples with implausibly large effects. Significance alone does not imply practical importance or validity.