Evaluate agent outputs with LangSmith evaluators and run functions.
npx skills add https://github.com/lguibr/fh --skill langsmith-evaluator-lguibr
Please help me install this Agent Skill. Skill: langsmith-evaluator Source: https://github.com/lguibr/fh/tree/main/.agents/skills/langsmith-evaluator Command: npx skills add https://github.com/lguibr/fh --skill langsmith-evaluator-lguibr
Streamlines evaluation of agent outputs using LangSmith by providing evaluators, run functions, and evaluation workflows.
Configure LangSmith credentials, implement a simple run function and evaluator, then run your agent against a dataset or project to begin automatic evaluation.
High-intent search queries and answers about installing and using this skill.
To evaluate agent trajectories using LangSmith, you can define custom evaluators or use LLM-as-Judge to assess outputs and execution paths. This Skill streamlines the process by providing run functions that capture trajectories for reliable scoring.
The best way to run offline dataset evaluations is using the local evaluate() function. This Skill provides workflows that structure your agent outputs and traces, allowing you to automatically score results against your prepared datasets.
Yes, LangSmith evaluators support both Python and TypeScript. This Skill includes examples for both languages and integrates with the LangSmith CLI to help you orchestrate and run evaluations seamlessly across your projects.
You capture agent execution traces by defining run functions within your evaluation pipeline. These run functions record agent outputs and execution trajectories, ensuring your LangSmith evaluation results are fully auditable and reproducible.
Yes, LangSmith supports online project-based checks for real-time evaluation. This Skill helps you configure evaluation workflows to continuously monitor and score agent outputs directly within your active LangSmith projects.
Yes, you need to structure outputs and traces properly for reliable scoring by LLM-as-Judge evaluators. This Skill defines the required components and clarifies how to format your agent trajectories and outputs for accurate automated evaluation.
jxtngxCreate and deploy LLM agent evaluation pipelines within the LangSmith ecosystem.
tavvaQuery Langfuse traces, sessions, observations, scores, metrics, and datasets via CLI.
lguibrInstrument applications to generate LangSmith traces for LLM observability.
lucasgaldinosAutomate agent output evaluation with rubric-based scoring and self-critique.
