inngest-agent-evals

To evaluate AI agent performance in production, you integrate scoring, variant experimentation, and session tracking directly into your durable workflow executions to measure actual agent behavior and outcomes.

Implementing LLM-as-judge scoring involves using direct and deferred evaluation primitives within your durable workflows to capture quality signals and user feedback after function execution.

Running AI prompt experiments against real traffic requires using group experiments with variant attribution, allowing you to compare different prompts, models, or tools across live agent sessions.

No, the Inngest agent evaluation skill does not work without TypeScript SDK v4, which is required alongside experimental middleware to enable the core scoring and experiment primitives.

Yes, you can track user feedback sessions across durable function executions by leveraging built-in observability features that correlate agent outcomes with specific user flows and historical data.