skills.rest
imrishuroy
imrishuroyCOMMUNITY
Explore all skills by this author →
𝕏@imrishuroyGitHub

sentry-setup-ai-monitoring

Configure Sentry AI agent monitoring for LLM calls, agents, and token usage.

Updated Apr 22, 2026
Search Tags:#tracing#instrumentation#token usage#openai#sentry#llm monitoring#ai observability
One-click install
npx skills add https://github.com/imrishuroy/algopatterns --skill sentry-setup-ai-monitoring-imrishuroy
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sentry-setup-ai-monitoring
Source: https://github.com/imrishuroy/algopatterns/tree/main/.agents/skills/sentry-setup-ai-monitoring
Command: npx skills add https://github.com/imrishuroy/algopatterns --skill sentry-setup-ai-monitoring-imrishuroy

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Setting up observability for AI applications is error-prone: developers must detect which AI SDKs are installed, enable tracing, configure sampling so agent spans are not dropped, and handle PII concerns around prompt capture. This Skill guides the full setup of Sentry AI Agent Monitoring so LLM calls, agent executions, tool usage, and token costs become visible in Sentry. ## Core Features & Use Cases - SDK Detection and Auto-Configuration: Detects installed AI SDKs (OpenAI, Anthropic, Vercel AI, LangChain, Google GenAI, Pydantic AI) in JavaScript or Python projects and applies the correct Sentry integrations. - Sampling and Cost Guidance: Configures tracesSampler so gen_ai spans are kept at 100% while other traffic uses a lower rate, and explains correct cached/reasoning token accounting to avoid negative costs. - Conversations View Setup: Sets conversation IDs and user attribution so multi-turn chats are grouped into a readable timeline in Sentry. - Use Case: A team shipping a Next.js chatbot using the Vercel AI SDK asks to track model latency and token spend; the Skill detects the SDK, adds vercelAIIntegration with per-call telemetry, and verifies spans appear in the Sentry Traces dashboard. ## Quick Start Ask the assistant to set up Sentry AI monitoring for the LLM calls in this project and verify the spans appear in the Sentry dashboard.

Frequently Asked Questions about sentry-setup-ai-monitoring

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I monitor OpenAI or Anthropic API calls with Sentry?▼

Enable tracing in Sentry.init with tracesSampleRate above zero; the OpenAI and Anthropic integrations auto-enable in Node.js and Python when those packages are installed. In browser or Next.js client code, wrap the client manually with Sentry.instrumentOpenAiClient.

How to track token usage and LLM costs in Sentry?▼

Set gen_ai.usage.input_tokens and gen_ai.usage.output_tokens span attributes, and Sentry calculates model costs from them. Cached and reasoning tokens are subsets of the totals, so never report a cached or reasoning count larger than its total or costs go negative.

Does Sentry AI monitoring work with the Vercel AI SDK?▼

Yes, add Sentry.vercelAIIntegration() to your Sentry config (including sentry.edge.config.ts for Edge runtime) with SDK version 10.53.0 or higher. You must also pass experimental_telemetry with isEnabled true on each generateText call.

Why are my gen_ai spans missing from Sentry traces?▼

Sampling is decided at the root span, so if the parent HTTP transaction is dropped, all child gen_ai spans are lost. Use a tracesSampler that returns 1.0 for AI routes like /api/chat while keeping a lower baseline rate for other traffic.

Why is the Sentry Conversations view empty?▼

Conversations requires sendDefaultPii enabled so input and output messages are captured, plus a conversation ID set via Sentry.setConversationId or sentry_sdk.ai.set_conversation_id. Gen AI span streaming must also be on, which is the default since JS SDK 10.61.0 and Python SDK 2.64.0.

Related Skills

Explore More Software Engineering Skills→
Valor-mmmValor-mmm

sentry-browser-sdk

Install and initialize @sentry/browser for error monitoring and tracing.

Community
Advanced
achmfachmf

sentry-setup-ai-monitoring

Configure Sentry tracing for AI SDKs in JavaScript and Python environments.

Community
Advanced
achmfachmf

sentry-tanstack-start-sdk

Integrate Sentry error monitoring, tracing, and session replay into TanStack Start applications.

Community
Advanced
BagelHoleBagelHole
46

opentelemetry

Instrument applications and infrastructure with OpenTelemetry for unified traces, metrics, and logs.

Community
Intermediate