Inspect and manage Grafana Agent Observability conversations, evaluators, rules, and scores via gcx.
npx skills add https://github.com/grafana/gcx --skill agento11y
Please help me install this Agent Skill. Skill: agento11y Source: https://github.com/grafana/gcx/tree/main/claude-plugin/skills/agento11y Command: npx skills add https://github.com/grafana/gcx --skill agento11y
Teams running LLM-powered applications in production lack visibility into what their agents actually do and whether their outputs are good. This Skill lets you inspect recorded LLM conversations and generations, build quality evaluators, bind them to production traffic with rules, and review the resulting scores — all through the gcx CLI.
Ask the agent to list recent Agent Observability conversations for your agent and set up an online evaluation rule with a helpfulness evaluator.
High-intent search queries and answers about installing and using this skill.
Pick a built-in template with gcx agento11y templates list, copy its fields into your own evaluator YAML, upsert it, test against a real generation, then create a rule binding the evaluator to traffic with a selector and sample rate. Verify with rules list and inspect scores via rules list-scores.
Use gcx agento11y conversations search with space-separated filters like agent = "my-agent" status = "error". Supported keys include model, provider, agent, status, error.type, duration, and eval.score, with operators =, !=, >, <, >=, <=, and =~ for regex.
Use llm_judge for subjective quality like helpfulness or groundedness, json_schema for output format validation, regex for presence or absence of specific content, and heuristic for deterministic checks like non-empty or minimum length. Each kind has specific config constraints documented in the reference examples.
The selector likely does not match your agent's shape. The user_visible_turn selector only matches user-facing turns, so multi-agent pipelines or single-shot agents need all_assistant_generations scoped with match.agent_name. Check eval_summary.total_scores to confirm.
There is no update command; re-run evaluators upsert with the same evaluator_id and a new version value. Re-using an existing version is rejected with a 409 error, and IDs accept only letters, digits, underscores, and dots.
grafanaConfigure gcx contexts for Grafana Cloud and on-prem instances.
adamatdevopsAnalyze LLM pipeline traces to prioritize failure modes by root cause.
berksunduriInstrument Go services with structured logging, Prometheus metrics, OpenTelemetry tracing, and pprof profiling.
langwatchCreates and runs LangWatch batch experiments to test agents against datasets with evaluators.
