adding-ingestion-warnings

Registers and emits new ingestion warning types across nodejs and Rust pipelines.

713|118|Updated Aug 11, 2020
One-click install
npx skills add https://github.com/PostHog/posthog-foss --skill adding-ingestion-warnings
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: adding-ingestion-warnings
Source: https://github.com/PostHog/posthog-foss/tree/main/.agents/skills/adding-ingestion-warnings
Command: npx skills add https://github.com/PostHog/posthog-foss --skill adding-ingestion-warnings

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Adding a new ingestion warning type in PostHog touches many surfaces: the nodejs type registry, ClickHouse v2 column conventions, debouncing, the generated Rust warning enum, deploy ordering, and downstream UI/docs. This Skill codifies that entire procedure so a new warning type is registered, emitted, and synchronized correctly without drift between nodejs and Rust.

Core Features & Use Cases

  • Registry-first workflow: Add the type to INGESTION_WARNING_TYPES with the correct category and severity, making unregistered types a compile error.
  • Emission guidance: Covers pipeline-step warnings (ok/drop results), direct emit helpers, debouncing keys, and the exact details keys ClickHouse v2 materializes into columns.
  • Rust codegen sync: Explains regenerating warning_types.generated.json, choosing between from_tag and DIRECT_EMIT routes, and the direct-row transport for team-aware services.
  • Use Case: You need to warn customers when events are dropped due to oversized payloads. Register the type, emit it from the pipeline step, regenerate the Rust artifact, and update the v1 UI map and resolution skill in one guided pass.

Quick Start

Ask the agent to add a new ingestion warning type for your scenario and have it register the type, wire up emission, and update all downstream surfaces.

Frequently Asked Questions about adding-ingestion-warnings

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add a new ingestion warning type in PostHog?

Register the type in the INGESTION_WARNING_TYPES registry in nodejs/src/ingestion/common/ingestion-warning-types.ts with a category and severity, then emit it from a pipeline step or via emitIngestionWarning. Finally update downstream surfaces like the v1 UI map and docs.

How do I emit ingestion warnings from Rust capture services?

Rust services emit a synthetic $$client_ingestion_warning CapturedEvent via WarningEmitter, which the nodejs clientwarnings consumer resolves to a team and writes as a v2 row. Team-aware services can instead produce the terminal row directly to the clickhouse_ingestion_warnings topic.

Why does my typecheck fail with 'not assignable to IngestionWarningType'?

IngestionWarningType is the key union of the INGESTION_WARNING_TYPES registry, so any unregistered string is a compile error. Add your new type to the registry first, choosing an appropriate category and severity.

Does adding a warning type require regenerating Rust code?

Yes. Every edit to INGESTION_WARNING_TYPES requires running pnpm --filter=@posthog/nodejs gen:ingestion-warning-types and committing the updated warning_types.generated.json. A no-drift test fails CI if the committed artifact diverges from the generator output.

What details keys does ClickHouse v2 materialize into columns?

ingestion_warnings_v2 derives columns from four exact JSON keys: eventUuid, distinctId, personId, and groupKey. Never set category, severity, or pipelineStep in details yourself, since the serializer appends them and structured fields are spread last.

What deploy order is required for a new capture-produced warning type?

Deploy the nodejs consumer before the Rust capture producer so the consumer recognizes the new type. If capture ships first, unknown types fall back to the generic client_ingestion_warning type and structured details are lost until nodejs catches up.