ultraqa

Run adversarial E2E QA scenarios with strict evidence and cleanup gates.

3|Updated May 24, 2026

Search Tags:#evidence #e2e testing #state tracking #cli verification #regression hardening #adversarial qa

Ready to run

npx skills add https://github.com/shayne-snap/oh-my-antigravity --skill ultraqa-shayne-snap

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: ultraqa
Source: https://github.com/shayne-snap/oh-my-antigravity/tree/main/plugin/skills/ultraqa
Command: npx skills add https://github.com/shayne-snap/oh-my-antigravity --skill ultraqa-shayne-snap

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

UltraQA prevents “false confidence” by running adversarial, hostile, and edge-case E2E scenarios and only declaring success when baseline verification and adversarial results both meet strict evidence and cleanup gates.

Core Features & Use Cases

Scenario matrix-driven QA: Requires a maintained matrix including intent, attacker/user model, setup, commands, expected vs actual signals, fixes applied, evidence, and cleanup status.
Fix/retest loop with guardrails: Iterates through diagnose and precise fixes while preserving unrelated work, then cleans up temporary artifacts and state.
Omagy state-backed traceability: Tracks progress under .omagy/state/ultraqa-state.json and records each scenario row for durable auditability.

Quick Start

Start an UltraQA run by asking to plan adversarial QA for your goal and begin state-tracked scenario execution using the UltraQA CLI entrypoint.

Frequently Asked Questions about ultraqa

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I run adversarial E2E testing to catch prompt injection and malformed inputs?▼

Adversarial E2E testing for prompt injection and malformed inputs requires running hostile dynamic scenarios that target CLI workflows and interactive services. This approach enforces scenario-matrix persistence and strict completion gates using evidence and cleanup verification to prevent false confidence.

What is the best way to harden CLI workflows against stale state and dirty worktrees?▼

Hardening CLI workflows against stale state and dirty worktrees involves executing bounded, non-destructive operations with state tracking. A fix and retest loop iterates through precise diagnoses while preserving unrelated work, subsequently cleaning up temporary artifacts and verifying state.

How does state tracking work during regression hardening for edge cases?▼

State tracking during regression hardening records each scenario row for durable auditability under a dedicated state file. This mechanism maintains a detailed matrix including intent, attacker model, setup, commands, expected versus actual signals, fixes applied, evidence, and cleanup status.

Can I use scenario-matrix-driven QA for cancellation and resume edge cases?▼

Scenario-matrix-driven QA applies directly to cancellation and resume edge cases by modeling hostile user interactions and verifying strict completion gates. It requires maintaining a matrix that tracks setup, commands, and actual signals to safely execute bounded, non-destructive operations.

Why does my E2E testing show misleading success outputs and how do I verify them?▼

Misleading success outputs occur when E2E testing lacks strict evidence and cleanup gates. To verify results, run adversarial scenarios that require baseline verification alongside adversarial results, ensuring both meet strict completion criteria before declaring success.

Does UltraQA work without external testing dependencies for CLI verification?▼

UltraQA operates without external dependencies to perform CLI verification and adversarial QA. It applies hostile dynamic testing beyond linting or unit tests using internal state tracking to manage scenario execution, fix loops, and cleanup verification.