nemo-relay-plugin-observability

Configure NeMo Relay observability exporters for ATOF, ATIF, OpenTelemetry, and OpenInference telemetry.

3.2k|370|Updated Feb 25, 2026
One-click install
npx skills add https://github.com/NVIDIA/skills --skill nemo-relay-plugin-observability
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: nemo-relay-plugin-observability
Source: https://github.com/NVIDIA/skills/tree/main/skills/nemo-relay-plugin-observability
Command: npx skills add https://github.com/NVIDIA/skills --skill nemo-relay-plugin-observability

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Choosing and wiring observability for NeMo Relay agent runtimes is error-prone: versions 0.6 and 0.7 use different configuration schemas and exporter classes, and each output format (raw events, trajectories, OTLP traces) has its own lifecycle and semantics. This Skill guides agents through selecting the right exporter, configuring it correctly for the installed Relay version, and verifying the output.

Core Features & Use Cases

  • Version-aware configuration: Distinguishes NeMo Relay 0.6 (config version 2, separate OpenTelemetry/OpenInference exporters) from 0.7 (config version 3, unified typed OpenTelemetry projections).
  • Multiple export outputs: Covers raw ATOF JSONL events, ATIF v1.7 trajectories, OTLP tracing via OpenTelemetry, and OpenInference-aware backends such as Arize Phoenix.
  • Lifecycle and safety guidance: Enforces the create-register-run-flush-deregister-shutdown lifecycle, secret handling via environment variables, and sanitization before production export.
  • Use Case: A team deploying a NeMo Relay service on Kubernetes needs spans in Grafana Tempo and replayable trajectories for their ML team; the Skill walks through registering an OpenTelemetry subscriber and an AtifExporter on the same event stream.

Quick Start

Ask your agent to set up observability for your NeMo Relay application, for example by requesting OTLP trace export to a local OpenTelemetry Collector for your Relay 0.7 Python service.

Frequently Asked Questions about nemo-relay-plugin-observability

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I set up observability for a NeMo Relay application?

Start with one exporter managed by the built-in Observability plugin, choosing the output that matches your inspection target: ATOF for raw events, ATIF for trajectories, or OpenTelemetry for OTLP tracing. Verify that single output before layering additional telemetry destinations.

What is the difference between NeMo Relay 0.6 and 0.7 observability configuration?

Relay 0.6 uses observability configuration version 2 with separate OpenTelemetryConfig/OpenTelemetrySubscriber and OpenInferenceConfig/OpenInferenceSubscriber exporters. Relay 0.7 uses configuration version 3 with one typed OpenTelemetry exporter offering full, gen_ai, and openinference projections.

How do I export NeMo Relay execution traces as ATIF trajectories?

Create an AtifExporter with session and agent metadata, register it before instrumented work, run scoped tool and LLM activity, then call export() or export_json(). Use one exporter per run or call clear() between runs, and confirm schema_version is ATIF-v1.7 before evaluation use.

Can I send NeMo Relay traces to Jaeger, Tempo, or Arize Phoenix?

Yes. Use the OpenTelemetry exporter with http_binary transport and an OTLP endpoint for Jaeger, Tempo, or a collector; for Phoenix, use the OpenInference path (standalone exporter in 0.6, the openinference typed projection in 0.7). Multiple subscribers can observe the same event stream concurrently.

Why are no spans showing up in my OTLP backend from NeMo Relay?

Common causes include the subscriber not being registered before scoped work, missing flush or shutdown calls, wrong transport for the binding, or events emitted outside active scopes. In 0.6, grpc also requires an active Tokio runtime; check construction logs and collector traffic first.

When should I use manual subscriber APIs instead of the observability plugin?

Use manual subscriber or exporter APIs only when a test, script, or application needs direct control over registration names, collection windows, or flush timing. The plugin-managed configuration remains the default for reusable process setup, and both paths consume the same canonical event stream.