skills.rest
oliver-kriska
oliver-kriskaCOMMUNITY
Explore all skills by this author →
Website𝕏@quatermain32GitHub

codex-ab

Runs an A/B experiment comparing holistic codex review against three focused dimension passes on a branch diff.

537|38|Updated Feb 12, 2026
Search Tags:#security review#a/b testing#Elixir#experiment#CODEX#Git diff#Code review
One-click install
npx skills add https://github.com/oliver-kriska/claude-elixir-phoenix --skill codex-ab
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: codex-ab
Source: https://github.com/oliver-kriska/claude-elixir-phoenix/tree/main/.claude/skills/codex-ab
Command: npx skills add https://github.com/oliver-kriska/claude-elixir-phoenix --skill codex-ab

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

When deciding whether a multi-pass focused code review strategy is worth its cost, teams lack hard evidence. This Skill answers one question with data: do dimension-focused codex passes (security, ecto, liveview) find real issues that a single holistic codex exec review misses on the same branch diff?

Core Features & Use Cases

  • Parallel A/B Harness: Runs 1 holistic codex exec review plus 3 focused codex exec workers (security, ecto, liveview) in parallel against the same diff, with all streams redirected to log files.
  • Finding Classification: Guides classification of every focused finding as DUPLICATE, REAL MISS, or FALSE POSITIVE, with mandatory code verification at the cited file:line before counting a miss.
  • Verdict Persistence: Produces a verdict table written to .claude/reviews/codex-ab-{date}/VERDICT.md so the experiment outcome is recorded across runs.
  • Use Case: Before building a --codex-panel mode into your review workflow, run this on 2-3 fresh branches to check whether real misses outnumber false positives.

Quick Start

Ask the AI to run the codex A/B experiment on the current fresh branch against main and report the verdict table.

Frequently Asked Questions about codex-ab

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I compare focused code review passes against a holistic review?▼

Run the codex-ab skill on a fresh branch, which executes one holistic codex review plus three focused passes (security, ecto, liveview) in parallel on the same diff. Then classify each focused finding as duplicate, real miss, or false positive against the holistic output.

How do I run the codex A/B review experiment?▼

Invoke /codex-ab with an optional base branch, defaulting to main. The script runs four codex executions in the background over roughly five minutes, then you read the four findings files and record a verdict table.

Why must the branch be fresh before running codex-ab?▼

A branch already reviewed by codex returns no findings everywhere, which proves nothing and wastes quota. The skill asks you to confirm no prior codex review ran on the branch and stops if it has.

What are the limitations of the codex panel A/B approach?▼

The experiment costs four codex runs per branch and needs two to three fresh branches for a reliable verdict. The maintainers already concluded the panel was killed after real misses did not outnumber false positives, keeping it only as contributor tooling.

Does codex-ab modify my code or fix findings?▼

No. All four codex runs use read-only or review modes and only produce findings files. It is a measurement instrument that decides whether a panel review mode is worth building, not a fixing tool.

Related Skills

Explore More Software Engineering Skills→
boxpboxp

codex-review

Review Git-managed code changes using the codex CLI.

Community
Advanced
enkira-aienkira-ai
2

codex-review

Review uncommitted changes, branch diffs, or commits using the codex CLI.

Official
Advanced
camilooscargbaptistacamilooscargbaptista
7

elixir-review

Review Elixir and Phoenix code for OTP, GenServer, LiveView, and concurrency issues.

Community
Intermediate
jkfeejkfee

auto-review-loop

Automate multi-round review loops for research artifacts with configurable thresholds.

Community
Advanced