auditing-llm-gateway-parity

Audits Python and Go LLM gateway implementations and updates the parity record.

713|118|Updated Aug 11, 2020
One-click install
npx skills add https://github.com/PostHog/posthog-foss --skill auditing-llm-gateway-parity
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: auditing-llm-gateway-parity
Source: https://github.com/PostHog/posthog-foss/tree/main/.agents/skills/auditing-llm-gateway-parity
Command: npx skills add https://github.com/PostHog/posthog-foss --skill auditing-llm-gateway-parity

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Keeping two LLM gateway implementations (a Python service and a Go service) in sync is error-prone, and teams need an evidence-based record of which features are supported, blocked, or need verification before migrating callers.

Core Features & Use Cases

  • Dual-gateway inspection: Compares routes, auth, billing, rate limiting, providers, models, and event attribution across the Python services/llm-gateway and the Go PostHog/ai-gateway codebases.
  • Evidence classification: Labels each contract as Supported, Blocking, or Verify based on actual implementation code rather than READMEs or existing tables.
  • Parity record maintenance: Updates services/llm-gateway/PARITY.md with source SHAs, verification dates, and migration-relevant contract changes.
  • Use Case: When a Go gateway PR changes authentication or billing behavior, run this audit to determine which Python caller classes become eligible or blocked for migration.

Quick Start

Audit the current LLM gateway implementations against each other and refresh the parity record in services/llm-gateway/PARITY.md.

Frequently Asked Questions about auditing-llm-gateway-parity

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I audit parity between two LLM gateway implementations?

Inspect the actual implementation code of both gateways covering routes, auth, billing, rate limits, providers, and event attribution, then classify each contract as supported, blocking, or verify. Record source SHAs from both repositories and update the parity document with the evidence.

When should the LLM gateway parity record be updated?

Update it whenever either gateway changes auth, attribution, billing, endpoints, providers, models, routing, or metadata, or when reviewing a Python gateway change. In-flight changes should be audited from the working tree or PR branch, not just from master.

Does matching route names prove gateway parity?

No. Matching route names do not prove parity for headers, streaming, errors, timeouts, retries, billing, or emitted events. Both request and response behavior must be checked in the implementation code.

Is an unbilled Python workload automatically blocked from migrating to the Go gateway?

No. An internal workload can migrate with a PostHog-owned team credential when debiting that wallet is the intended attribution method. It is blocked only when it must preserve customer-specific billing policy or debit no wallet.

Does this audit migrate callers to the Go gateway?

No. The audit only updates the parity record and identifies caller classes newly eligible to migrate. Caller migration is a separate task handled by the migrating-llm-gateway-callers skill.