datadog-query-recipes

Queries Langfuse production telemetry in Datadog across regions, services, queues, and tenants.

34.0k|3.7k|Updated May 18, 2023
One-click install
npx skills add https://github.com/langfuse/langfuse --skill datadog-query-recipes
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: datadog-query-recipes
Source: https://github.com/langfuse/langfuse/tree/main/.agents/skills/datadog-query-recipes
Command: npx skills add https://github.com/langfuse/langfuse --skill datadog-query-recipes

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Investigating Langfuse production behavior requires knowing which Datadog site, environment, service, and query shape to use, and this Skill provides reusable query recipes so engineers find the right telemetry without guessing.

Core Features & Use Cases

  • Multi-Region Coverage: Covers prod-us, prod-eu, prod-hipaa, and prod-jp with correct Datadog site routing and cross-site verification rules.
  • Tenant and Endpoint Analysis: Correlates api-auth-verify spans with request root spans to measure per-tenant public API usage, including legacy and migrated endpoints.
  • Queue and Consumer Diagnostics: Provides the full queue inventory, consumer feature gates, BullMQ span queries, and langfuse.queue.* metric patterns for depth, rate, and latency checks.
  • Use Case: An engineer asks whether a specific org still calls the legacy /api/public/traces endpoint; the Skill aggregates auth spans by tenant, samples traces, and reports endpoint usage with Datadog links across all production regions.

Quick Start

Use the datadog-query-recipes skill to check whether project X is still calling the legacy /api/public/observations endpoint across all production environments.

Frequently Asked Questions about datadog-query-recipes

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I check if a tenant uses a specific Langfuse API endpoint in Datadog?

Aggregate api-auth-verify spans filtered by @langfuse.org.id or @langfuse.project.id, then fetch representative traces and read the root span's http.path_group or http.route. Tenant tags and route tags live on different spans, so single-span queries combining both usually fail.

How do I check if a Langfuse queue consumer is running in production?

Check langfuse.queue.<name>.depth and .rate metrics for waiting, active, and request counts, then search BullMQ processor spans on worker and worker-cpu services. If all signals are empty, verify the QUEUE_CONSUMER_*_IS_ENABLED gate in worker/src/app.ts.

Which Datadog site should I query for Langfuse production environments?

prod-us and prod-hipaa default to the US site (datadoghq.com), while prod-eu and prod-jp default to the EU site (datadoghq.eu). Always verify with a small count or facet query on both sites before concluding an environment has no data.

Why does my Datadog query return no data for a Langfuse environment?

Zero results usually come from wrong time windows, misspelled env or service filters, or querying the wrong Datadog site. Run facet queries for env and service, then repeat a small count query on the other site before reporting no measurements.

When should I use debug-issue-with-datadog instead of query recipes?

Use debug-issue-with-datadog when a Linear issue, GitHub issue, incident, or monitor needs root-cause analysis and patch recommendations. The query recipes skill is for telemetry research and measurement, not diagnosis and remediation.