skills.rest
mdnaimul22
mdnaimul22COMMUNITY
Explore all skills by this author →
GitHub

senior-data-scientist

Design and analyze A/B tests and causal estimates with statistical methods.

2|Updated Apr 15, 2026
Search Tags:#feature engineering#model evaluation#causal inference#mlflow#ab testing#xgboost#difference-in-differences
One-click install
npx skills add https://github.com/mdnaimul22/human-skills --skill senior-data-scientist-mdnaimul22
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: senior-data-scientist
Source: https://github.com/mdnaimul22/human-skills/tree/main/skills/senior-data-scientist
Command: npx skills add https://github.com/mdnaimul22/human-skills --skill senior-data-scientist-mdnaimul22

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

It helps you run trustworthy experiments and build production-grade predictive models, while also drawing defensible causal conclusions from data.

Core Features & Use Cases

  • Design an A/B test: sample sizing, two-proportion z-tests, confidence intervals, and Bonferroni correction for multiple metrics.
  • Build a feature engineering pipeline: imputation, scaling, one-hot encoding, cyclical time features, and leakage-avoidant train/test handling.
  • Train, evaluate, and select models: cross-validated evaluation using ROC-AUC and PR-AUC, overfitting diagnostics, and MLflow logging.
  • Conduct causal inference (DiD): difference-in-differences estimation with robust standard errors and confidence intervals, including checks like parallel trends.
  • Use cases: designing controlled launches, improving classification/regression performance on tabular data, and estimating treatment effects from observational/panel data.

Quick Start

Use the senior-data-scientist skill to design and analyze an A/B test by asking: "Estimate the required sample size for a baseline 10% conversion with a 5% relative lift, then analyze control vs treatment conversions with a two-proportion z-test and report lift, p-value, and a 95% confidence interval."

Frequently Asked Questions about senior-data-scientist

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I calculate sample size and run a two-proportion z-test for an A/B test?▼

To run an A/B test, calculate sample sizing for baseline conversions and analyze results using a two-proportion z-test. This process reports lift, p-value, and 95% confidence intervals, applying Bonferroni correction for multiple metrics.

What is the best way to prevent data leakage during feature engineering?▼

Prevent data leakage during feature engineering by applying leakage-avoidant train/test handling. This ensures imputation, scaling, one-hot encoding, and cyclical time features are fitted only on training data before transforming test sets.

How do I estimate causal effects using difference-in-differences on observational data?▼

Estimate causal effects with difference-in-differences (DiD) by running robust OLS interaction terms on observational panel data. This provides treatment effect estimates with robust standard errors, confidence intervals, and parallel trends checks.

Can I use MLflow to track cross-validated model evaluation with ROC-AUC and PR-AUC?▼

Yes, you can use MLflow to log cross-validated model evaluation. This tracks ROC-AUC and PR-AUC metrics, captures overfitting diagnostics, and records SHAP interpretation outputs for tabular predictive modeling.

How do I evaluate classification models and interpret predictions using SHAP?▼

Evaluate classification models using cross-validated ROC-AUC and PR-AUC metrics to diagnose overfitting. Interpret predictions by generating SHAP values to explain feature importance and model behavior across the tabular dataset.

When should I apply Bonferroni correction when analyzing multiple A/B test metrics?▼

Apply Bonferroni correction when analyzing multiple metrics in A/B testing to control the family-wise error rate. This adjusts confidence intervals and p-values, ensuring defensible statistical conclusions across simultaneous two-proportion z-tests.

Related Skills

Explore More Data & Analytics Skills→
brainbrewlabsbrainbrewlabs
7

data-cleaning

Profile and clean tabular datasets, then write a cleaned file with a change report.

Official
Intermediate
zadnan2002zadnan2002

evaluate-model

Calculate model metrics and generate accuracy, calibration, and stability visualizations.

Community
Intermediate
idanmoradarthasidanmoradarthas
37

ds-utils-metrics

Analyze and visualize sklearn-compatible model performance with metrics and plots.

Community
Intermediate
AMindToThinkAMindToThink
4

bayesian-stats

Convert frequentist statistical tests into Bayesian equivalents with Python implementation guidance.

Community
Intermediate