auditing-warehouse-source-coverage

Audit implemented data warehouse sources for missing vendor API endpoints and tables.

713|118|Updated Aug 11, 2020
One-click install
npx skills add https://github.com/PostHog/posthog-foss --skill auditing-warehouse-source-coverage
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: auditing-warehouse-source-coverage
Source: https://github.com/PostHog/posthog-foss/tree/main/.agents/skills/auditing-warehouse-source-coverage
Command: npx skills add https://github.com/PostHog/posthog-foss --skill auditing-warehouse-source-coverage

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires django, and includes scripts (resource) components.

What problem does it solve?

Data warehouse import sources often ship with only a subset of the endpoints a vendor's API actually offers, leaving users unable to sync data they need. This Skill provides a repeatable method to find those gaps across hundreds of implemented sources without guessing.

Core Features & Use Cases

  • Credential-free inventory dump: Runs a script against the source registry to list every source's real tables via get_documented_tables(), with no secrets or vendor API calls required.
  • Adoption-ranked prioritization: Ranks sources by production connection counts so audit effort goes where users actually are, then diffs top sources against vendor OpenAPI or GraphQL specs.
  • Structured gap recording: Writes findings into COVERAGE_GAPS.md with spec-verified status, adoption tiers, and cross-source pattern themes like missing lookup tables.
  • Use Case: A maintainer suspects the HubSpot source is missing endpoints. Use this Skill to dump the current table inventory, fetch HubSpot's API surface, diff the two, and record verified gaps ordered by user value.

Quick Start

Audit the Stripe warehouse source for endpoints the vendor API offers that we have not wired up, and update COVERAGE_GAPS.md with the verified findings.

Frequently Asked Questions about auditing-warehouse-source-coverage

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I audit a data warehouse source for missing API endpoints?

Dump the source's real table inventory with the registry-based script, fetch the vendor's OpenAPI or GraphQL spec, and set-difference the vendor's collection GET paths against your tables. Record verified gaps in COVERAGE_GAPS.md ordered by user value.

How do I find which warehouse sources have the biggest coverage gaps?

Rank sources by production adoption using distinct teams with live connections, then join that ranking to table counts from the inventory dump. Low table count combined with high adoption signals where a small amount of work reaches the most users.

Does the endpoint inventory script need vendor API credentials?

No, the dump script builds a placeholder config through get_documented_tables(), so it needs no secrets and makes no vendor API calls. Sources that raise while listing record the error and are skipped rather than aborting the dump.

Why should I fetch vendor specs with curl instead of WebFetch?

WebFetch summarizes pages with a small model and reliably drops most of an API reference, missing resource lists entirely. Fetching the raw OpenAPI or GraphQL spec and parsing it yourself gives a complete, auditable endpoint list.

What are the limitations of an endpoint coverage audit?

Table coverage does not imply column coverage, since tables can miss vendor fields through sparse fieldsets or allowlists. Specs also are not entitlements: endpoints may be deprecated, plan-gated, or absent on the pinned API version, and nothing is validated against a live account.

When should I not use this coverage audit skill?

Do not use it to implement a new source, add a vendor API version, or write source documentation; those have dedicated skills. It also excludes SQL and file sources like Postgres or Google Sheets, which introspect user schemas and have no fixed endpoint set to audit.