skills.rest
pvliesdonk
pvliesdonkCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

observability-patterns

Configures LuaSnip snippets with lazy loading, custom c?loaders?, and per-filetype settings from a single source file.

3|Updated Feb 15, 2026
Search Tags:#observability#cost tracking#prometheus#opentelemetry#structlog#llm monitoring#pii redaction
One-click install
npx skills add https://github.com/pvliesdonk/agents.md --skill observability-patterns-pvliesdonk
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: observability-patterns
Source: https://github.com/pvliesdonk/agents.md/tree/main/skills/observability-patterns
Command: npx skills add https://github.com/pvliesdonk/agents.md --skill observability-patterns-pvliesdonk

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill provides a comprehensive framework for monitoring Large Language Model (LLM) applications, addressing the challenges of understanding performance, cost, and potential issues in complex AI pipelines.

Core Features & Use Cases

  • Structured Logging: Implement robust logging with structlog, including PII redaction and context propagation.
  • LLM Monitoring: Track token usage, estimate costs, and log model fallbacks and selections.
  • Observability Tools: Integrate with OpenTelemetry for tracing and Prometheus for metrics.
  • Error Tracking: Utilize Sentry for capturing and analyzing LLM-related errors.
  • Use Case: Monitor the performance and cost of your LLM-powered chatbot, identify which prompts are most expensive, and set up alerts for model fallbacks or high error rates.

Quick Start

Configure structlog with PII redaction and JSON rendering for your application logs.

Frequently Asked Questions about observability-patterns

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I monitor LLM application performance and token costs in production?▼

You can monitor LLM application performance and costs by implementing structured logging with token tracking and cost estimation, then visualizing the data through Prometheus metrics and OpenTelemetry traces.

How do I set up structured logging with PII redaction for my LLM pipelines?▼

Structured logging with PII redaction is set up using the structlog library, which configures context propagation and JSON rendering to ensure sensitive user data is scrubbed before logs are generated.

Does this observability pattern integrate with OpenTelemetry and Prometheus?▼

Yes, the observability pattern integrates directly with OpenTelemetry for distributed tracing and Prometheus for collecting metrics, providing a comprehensive view of LLM resource utilization and errors.

What's the best way to track model fallbacks and error rates in LLM apps?▼

The best way to track model fallbacks and error rates is by combining Sentry for capturing LLM-related errors with Prometheus metrics to set up alerts for high error rates or fallback conditions.

When do I need observability patterns for LLM monitoring?▼

You need observability patterns for LLM monitoring when your AI pipelines reach production scale and you must understand performance bottlenecks, control prompt costs, and trace complex model selections.

Can I identify which prompts are most expensive using these observability tools?▼

Yes, you can identify expensive prompts by utilizing the token tracking and cost estimation features to log model selections and calculate the exact resource utilization of each LLM invocation.

Related Skills

Explore More Software Engineering Skills→
civictechdccivictechdc
1

python-observability

Instrument Python applications with structured logging, metrics, and distributed tracing.

Official
Advanced
allthingslinuxallthingslinux
1

monitoring-expert

Implement observability with OpenTelemetry, Prometheus, Grafana, Loki, and Jaeger.

Official
Advanced
hopeoverturehopeoverture
6

sentry-and-otel-setup

Configure Sentry and OpenTelemetry error tracking and performance monitoring for Next.js applications.

Community
Advanced
timequitytimequity
6

monitoring-setup

Set up Prometheus, Grafana, Loki, and Jaeger for Kubernetes observability.

Official
Advanced