skills.rest
LiorVainer
LiorVainerCOMMUNITY
Explore all skills by this author →
GitHub

sentry-setup-ai-monitoring

Configure Sentry AI Agent Monitoring to trace LLM calls, agent executions, and token usage.

130|14|Updated Jan 12, 2026
Search Tags:#tracing#instrumentation#token usage#openai#sentry#llm monitoring#ai observability
One-click install
npx skills add https://github.com/LiorVainer/data-israel --skill sentry-setup-ai-monitoring-liorvainer
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sentry-setup-ai-monitoring
Source: https://github.com/LiorVainer/data-israel/tree/main/.agents/skills/sentry-setup-ai-monitoring
Command: npx skills add https://github.com/LiorVainer/data-israel --skill sentry-setup-ai-monitoring-liorvainer

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Setting up observability for AI applications is error-prone: each LLM SDK (OpenAI, Anthropic, Vercel AI, LangChain, Google GenAI, Pydantic AI) requires different Sentry integrations, minimum SDK versions, and configuration patterns. This Skill detects installed AI SDKs and applies the correct Sentry AI Agent Monitoring setup so you can trace model calls, agent lifecycles, tool usage, and token consumption without guesswork. ## Core Features & Use Cases - SDK Detection: Scans package.json or requirements.txt to identify installed AI libraries before configuring anything. - Framework-Specific Setup: Provides correct integration code for JavaScript (Node.js, browser, Next.js, Vercel AI, LangChain/LangGraph) and Python (OpenAI, Anthropic, LangChain, Pydantic AI, LiteLLM, MCP). - Manual Instrumentation Fallback: Defines gen_ai.* span operations and attributes for unsupported SDKs, including token usage and model metadata. - Privacy Guardrails: Warns about PII capture in prompt/output recording and requires explicit user consent before enabling recordInputs or include_prompts. - Use Case: A developer adds AI features with the Vercel AI SDK and asks to track token costs; the Skill detects the SDK, adds vercelAIIntegration to the Sentry config, and enables experimental_telemetry per call. ## Quick Start Ask the assistant to set up Sentry AI monitoring for your project so it detects your installed AI SDKs and configures the right integrations.

Frequently Asked Questions about sentry-setup-ai-monitoring

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I monitor OpenAI API calls with Sentry?▼

Enable tracing with tracesSampleRate in Sentry.init and the openAIIntegration auto-enables in Node.js when the openai package is installed (Sentry SDK 10.28.0+). In browser or Next.js code, wrap the client manually with Sentry.instrumentOpenAiClient.

How do I track token usage for the Vercel AI SDK?▼

Add Sentry.vercelAIIntegration() to your Sentry config (including sentry.edge.config.ts for Edge runtime) and set experimental_telemetry: { isEnabled: true } on each generateText call. Token counts then appear as gen_ai.usage attributes on spans.

Does Sentry AI monitoring work with LangChain and LangGraph?▼

Yes, langChainIntegration and langGraphIntegration auto-enable in Node.js when the packages are installed (Sentry SDK 10.28.0+). In Python, LangChain and LangGraph integrations also auto-enable with no explicit registration needed.

Why are my AI spans not appearing in Sentry?▼

AI spans require tracing enabled, so verify tracesSampleRate is greater than 0 and your Sentry SDK meets the minimum version for your AI package. For Vercel AI, also confirm experimental_telemetry is set on each call.

Should I enable prompt and response capture in Sentry?▼

Prompt and output recording captures user content that is likely PII, so it is disabled by default. Only enable recordInputs/recordOutputs (JS) or include_prompts/send_default_pii (Python) after confirming your privacy policy and regulatory obligations permit it.

What if my AI SDK is not supported by Sentry integrations?▼

Use manual instrumentation with Sentry.startSpan and gen_ai.* operations such as gen_ai.request, gen_ai.invoke_agent, and gen_ai.execute_tool. Set attributes like gen_ai.request.model and gen_ai.usage.input_tokens to record model and token metadata.

Related Skills

Explore More Software Engineering Skills→
ksciuskscius
2

adding-error-tracking

Instruments web apps with Sentry for error tracking, performance metrics, and source maps.

Community
Advanced
achmfachmf

sentry-setup-ai-monitoring

Configure Sentry tracing for AI SDKs in JavaScript and Python environments.

Community
Advanced
TMHSDigitalTMHSDigital
2

mobile-app-monitoring

Instrument Sentry, Datadog RUM, or Instabug in Expo and Flutter apps for performance monitoring.

Community
Advanced
coreyonecoreyone
1

observability-telemetry

Design OpenTelemetry instrumentation, structured logging, distributed tracing, and SLO alert rules.

Community
Intermediate