observability-and-instrumentation

Structured logging emits JSON objects with a stable event name and machine-readable fields instead of interpolated strings. Attach a child logger with a request ID per request so every line can be correlated, and never log secrets or full request bodies.

Use the RED method: rate, errors, and duration for every endpoint and external dependency. Track latency as a histogram with p50/p95/p99 percentiles rather than averages, and keep label values from small fixed sets to avoid cardinality explosions.

Initialize the OpenTelemetry NodeSDK with auto-instrumentations before importing anything else, which covers HTTP, gRPC, and common database clients. Add manual spans around meaningful internal work and propagate context across HTTP headers and queue metadata.

Symptom-based alerts fire exactly when users are hurt, such as error rate above 1% or p99 latency over 2 seconds, regardless of cause. Cause-based alerts like CPU thresholds fire when nothing is wrong and miss failures you did not predict.

Do not use it to diagnose an active failure; use a debugging skill for that. It also does not cover performance profiling of measured slowness or launch-day monitoring checklists, which belong to performance-optimization and shipping workflows.