phoenix-skills-audit

Audit recent Phoenix client, CLI, and API changes and patch stale agent skill documentation.

11.3k|1.1k|Updated Nov 9, 2022
One-click install
npx skills add https://github.com/Arize-ai/phoenix --skill phoenix-skills-audit
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: phoenix-skills-audit
Source: https://github.com/Arize-ai/phoenix/tree/main/.agents/skills/phoenix-skills-audit
Command: npx skills add https://github.com/Arize-ai/phoenix --skill phoenix-skills-audit

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Agent-facing skill files for Phoenix (phoenix-tracing, phoenix-cli, phoenix-evals) drift out of sync with the actual Python clients, TypeScript clients, CLI, and REST/GraphQL APIs as new code ships, teaching every future agent the wrong API. This Skill audits recent commits on origin/main and applies corrective edits directly to the skill files.

Core Features & Use Cases

  • Commit triage: Fetches commits from origin/main over a configurable window (default 7 days) and classifies them into audit candidates, skips, and unclear items requiring diff inspection.
  • Source-to-skill mapping: Maps changes in phoenix-otel, phoenix-evals, phoenix-client, phoenix-cli, and server APIs to the correct target skill by exposed feature, including Python/TypeScript mirror checks.
  • In-place patching: Edits SKILL.md files and references/*.md files with verified signatures, runnable examples, and updated cross-links, then emits a structured run summary for PR bodies.
  • Use Case: After a week of merges adding a new CLI subcommand and a new evaluator class, run the audit to update phoenix-cli/SKILL.md and phoenix-evals/references/evaluators-pre-built.md with verbatim signatures and verified examples.

Quick Start

Ask the agent to audit the phoenix-tracing, phoenix-cli, and phoenix-evals skills against the last 7 days of changes on origin/main and patch anything stale in place.

Frequently Asked Questions about phoenix-skills-audit

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I keep agent skill documentation in sync with recent code changes?

Run the audit against origin/main over a chosen window (default 7 days). It triages commits affecting Python clients, TypeScript clients, CLI, and APIs, then edits the phoenix-tracing, phoenix-cli, and phoenix-evals skill files in place with verified signatures and examples.

How do I audit Phoenix skills for stale API references?

Ask for a skills audit or skill drift check. The workflow fetches origin/main, lists commits with git log, reads the actual diffs rather than trusting commit messages, and patches any skill snippet whose signature or import path no longer matches the tree.

Does the audit cover both Python and TypeScript client changes?

Yes. The workflow pairs each Python change with its TypeScript mirror by grepping the equivalent package for the same symbol family. If a feature is genuinely Python-only or TS-only, that is flagged in the run summary as a finding.

Which skills and files is the audit allowed to edit?

Only phoenix-tracing, phoenix-cli, and phoenix-evals, including their SKILL.md files and references directories. Changes belonging to other skills like phoenix-server or phoenix-frontend are noted in the run summary but never edited.

Why does the audit use origin/main instead of the local main branch?

Local main is routinely stale by dozens of commits, so auditing it silently misses everything shipped after the last git pull. The workflow always runs git fetch origin main first and records the audited SHA in the run summary for reproducibility.

What changes are intentionally skipped during the audit?

Internal refactors with no public surface, dependency bumps, test-only or CI changes, frontend-only changes, release bookkeeping commits, and feature flags such as *_DANGEROUSLY_* or *_EXPERIMENTAL_* env vars are skipped and listed in the run summary for reviewer confirmation.