langgraph-testing-evaluation

What problem does it solve?

This Skill streamlines the process of testing and evaluating LangGraph and LangChain agents, ensuring their quality, reliability, and performance.

Core Features & Use Cases

• Automated Testing: Generate unit and integration test scaffolds for Python and JavaScript/TypeScript agents.
• Trajectory Evaluation: Assess multi-step agent behavior using methods like trajectory matching or LLM-as-judge.
• LangSmith Integration: Run evaluations against datasets stored in LangSmith for robust quality tracking.
• A/B Testing: Compare different agent versions offline to validate improvements before deployment.
• Use Case: Before deploying a new version of your customer support chatbot, use this Skill to automatically generate tests, run it against a dataset of common queries, evaluate its response quality and latency, and compare it against the current production version.

Quick Start

Use the langgraph-testing-evaluation skill to generate Python pytest test scaffolding for your agent defined in my_agent:graph and output it to the tests/ directory.