skills.rest
linardsb
linardsbCOMMUNITY
Explore all skills by this author →
GitHub

data-and-experimentation

Designs A/B tests and analyzes them with Python-based statistical tools.

Updated Apr 17, 2026
Search Tags:#A/B testing#data science#machine learning#experimentation#statistical analysis#LLM evaluation
One-click install
npx skills add https://github.com/linardsb/fredis --skill data-and-experimentation
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-and-experimentation
Source: https://github.com/linardsb/fredis/tree/main/.claude/skills/data-and-experimentation
Command: npx skills add https://github.com/linardsb/fredis --skill data-and-experimentation

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires numpy, scipy, pandas, scikit-learn, xgboost, shap, mlflow, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill provides a comprehensive suite of tools for data science and experimentation, enabling users to design, execute, and analyze experiments, as well as perform advanced statistical analysis.

Core Features & Use Cases

  • Experiment Design: Offers guidance on designing A/B tests, multivariate tests, and holdout tests, including sample size estimation and ICE scoring.
  • Statistical Analysis: Provides tools for hypothesis testing, sample sizing, effect sizes, confidence intervals, and statistical significance.
  • Machine Learning: Assists with building and evaluating prediction models using XGBoost, SHAP, MLflow, and causal inference techniques.
  • LLM Evaluation: Offers a framework for evaluating LLMs through fixture-driven, judge-based, and regression-guarded approaches.
  • Use Case: Imagine you want to run an A/B test to measure the impact of a new feature on user engagement. Use this Skill to design the test, estimate the sample size, analyze the results, and draw conclusions based on statistical significance and effect sizes.

Quick Start

Use the data-and-experimentation skill to design an A/B test for the new feature 'Feature X' on the 'User Engagement' metric.

Frequently Asked Questions about data-and-experimentation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I design an A/B test and estimate sample size for a new product feature?▼

A/B test design involves defining control and treatment groups, estimating sample size based on expected effect sizes, and calculating statistical significance. This skill guides you through experiment design, ICE scoring, and holdout test setup.

What's the best way to evaluate LLM outputs using statistical analysis?▼

LLM evaluation uses fixture-driven, judge-based, and regression-guarded approaches to assess model outputs. This framework integrates statistical analysis principles to validate LLM performance and detect regressions.

Can I use XGBoost and SHAP for causal inference in marketing workflows?▼

Yes, machine learning models built with XGBoost can be combined with SHAP for feature importance analysis and causal inference techniques. This supports marketing and product development workflows by evaluating prediction models.

Do I need numpy and scipy installed to run statistical significance calculations?▼

Yes, numpy and scipy are required dependencies for performing statistical analysis including hypothesis testing, confidence intervals, and effect size calculations. These libraries provide the computational foundation for experiment evaluation.

When should I use multivariate testing instead of standard A/B testing?▼

Multivariate testing is appropriate when you need to evaluate multiple variables simultaneously and understand their interactions, whereas standard A/B testing compares two versions. This skill supports both approaches with sample sizing and ICE scoring.

Related Skills

Explore More Data & Analytics Skills→
4lerman4lerman

senior-data-scientist

Design, implement, and evaluate AI/ML/Data systems with statistical modeling and predictive analytics.

Community
Advanced
Nuwanda04Nuwanda04

senior-data-scientist

Design experiments, build predictive models, and perform causal inference with Python and R.

Community
Advanced
Ritabrata-ChakrabortyRitabrata-Chakraborty

shap

Computes SHAP values to interpret ML model predictions and feature importance.

Community
Advanced
amplitudeamplitude
142

craft-experiment-readout

Summarize A/B test outcomes and recommend next steps for stakeholders.

Official
Intermediate