online-experimentation
CommunityOptimize model rollouts and A/B testing with advanced statistical methods.
Software Engineering#A/B testing#model deployment#statistical testing#online experimentation#bandits
Authorhung-phan
Version1.0.0
Installs0
System Documentation
What problem does it solve?
Online experimentation helps bridge the gap between offline model evaluation and actual business impact, tackling distribution shifts, novelty effects, and metric mismatches.
Core Features & Use Cases
- Online Evaluation: Measure business KPIs and live metrics with statistical rigor.
- A/B Testing: Conduct experiments to determine the effectiveness of new models.
- Bandits: Maximize cumulative rewards through sequential decision-making.
- Use Case: When deploying a new model, use online experimentation to understand its impact on retention and revenue, without relying solely on offline metrics.
Quick Start
Use the online-experimentation skill to perform a Welch's t-test on your A/B test results with the 'welch_ttest' script.
Dependency Matrix
Required Modules
scipynumpy
Components
scriptsreferences
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: online-experimentation Download link: https://github.com/hung-phan/ml-skills/archive/main.zip#online-experimentation Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 620,000+ vetted skills library on demand.