skills.rest
bolun-ben-ship
bolun-ben-shipCOMMUNITY
Explore all skills by this author →
GitHub

ab-test-loop

Automate PostHog A/B test monitoring, winner declaration, and variant rollout.

Updated Mar 20, 2026
Search Tags:#data-analysis#posthog#feature-flags#ab-testing#conversion-optimization#webflow
One-click install
npx skills add https://github.com/bolun-ben-ship/RightClickAI-seo-workspace --skill ab-test-loop
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: ab-test-loop
Source: https://github.com/bolun-ben-ship/RightClickAI-seo-workspace/tree/main/seo-workflow/ab-test-loop
Command: npx skills add https://github.com/bolun-ben-ship/RightClickAI-seo-workspace --skill ab-test-loop

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Autonomous A/B test iteration loop for PostHog experiments. Monitors all running experiments for Bayesian significance (>95%), declares winners, rolls out the winning variant to 100% via PostHog feature flags, generates the next copy variants informed by brand voice, and launches the next experiment — without manual intervention. Supports CTA button text, hero section headlines, and any other PostHog experiment type. Also handles one-time scroll depth tracking setup (25/50/75/90%) via Webflow Pages API. Use when user says "ab-test-loop", "check my a/b tests", "run the ab loop", "check experiment results", "declare ab test winner", "launch next variant", "set up scroll tracking", or "automate ab testing".

Core Features & Use Cases

  • Autonomous monitoring of all running PostHog experiments and automatic significance checks.
  • Bayesian-based decisioning to identify significant winners (≥ 95% probability).
  • 100% rollout of winning variant via feature flags; 0% for losers to ensure clean transitions.
  • Automated variant generation aligned with brand voice to inform next test iterations.
  • Sequential experiment management: generate and launch subsequent experiments with updated copy.
  • Phase S one-time scroll depth tracking setup for Webflow pages to capture engagement signals.
  • Runs from the client workspace (clients/{domain}/) to keep context anchored to the relevant site.

Quick Start

Invoke ab-test-loop to automatically read all running PostHog experiments and begin monitoring, winner declaration, and variant generation.

Frequently Asked Questions about ab-test-loop

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate A/B testing workflows in PostHog?▼

You can automate A/B testing workflows by monitoring running PostHog experiments, checking for Bayesian significance above 95%, rolling out winning variants via feature flags, and launching subsequent tests without manual intervention.

How do I set up scroll depth tracking in Webflow for A/B testing?▼

You can set up scroll depth tracking in Webflow by configuring one-time event tracking for 25, 50, 75, and 90 percent scroll thresholds via the Webflow Pages API to capture engagement signals for experiments.

How does Bayesian significance determine an A/B test winner?▼

Bayesian significance determines an A/B test winner by calculating the probability that one variant outperforms the other, requiring a minimum 95 percent probability threshold to declare a winner and roll it out to 100 percent.

Can I automatically generate new A/B test variants based on my brand voice?▼

Yes, you can automatically generate new A/B test variants for elements like CTA buttons and hero headlines by using your client context and tone guide to ensure the new copy aligns with your brand voice.

Do I need PostHog feature flags to automate experiment rollouts?▼

Yes, PostHog feature flags are required to automate experiment rollouts, enabling the system to set winning variants to 100 percent and losing variants to 0 percent for clean test transitions.

What is the best way to manage sequential A/B test iterations?▼

The best way to manage sequential A/B test iterations is to autonomously monitor active experiments, declare significant winners, generate updated copy variants, and launch the next experiment automatically from the client workspace.

Related Skills

Explore More Marketing & Sales Skills→
djinilabsdjinilabs
8

PostHog Feature Flags

Manage PostHog feature flags for A/B testing and gradual rollouts.

Official
Intermediate
samuelross99samuelross99
1

page-cro

Identify conversion rate drivers on marketing pages and output a prioritized action list.

Community
Basic
EthGlobalBangkokEthGlobalBangkok
1

integration-nextjs-app-router

Integrate PostHog analytics into Next.js App Router with client and server instrumentation.

Official
Intermediate
dkyazzentwatwadkyazzentwatwa
86

ab-test-calculator

Calculate statistical significance for A/B tests using Chi-square, Z-test, or T-test.

Community
Intermediate