skills.rest
anukkrit149
anukkrit149COMMUNITY
Explore all skills by this author →
Website𝕏@anukkritGitHub

langsmith-dataset

Create and manage LangSmith evaluation datasets using the langsmith CLI.

Updated May 26, 2026
Search Tags:#rag#examples#langsmith#llm testing#dataset upload#cli management#evaluation datasets
One-click install
npx skills add https://github.com/anukkrit149/anukkrit-skills --skill langsmith-dataset-anukkrit149
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: langsmith-dataset
Source: https://github.com/anukkrit149/anukkrit-skills/tree/main/cloud/skills/langsmith-dataset
Command: npx skills add https://github.com/anukkrit149/anukkrit-skills --skill langsmith-dataset-anukkrit149

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps you create, manage, and upload evaluation datasets to LangSmith so you can test and validate LLM applications with consistent, reusable examples.

Core Features & Use Cases

  • Dataset lifecycle management: list, create, get, export, and delete datasets via the LangSmith CLI.
  • Example and experiment control: manage dataset examples and view experiment results tied to datasets.
  • Multi-type evaluation coverage: supports common evaluation dataset structures such as final_response, single_step, trajectory, and RAG (as defined in the skill guidance).
  • Auth and project scoping: uses LANGSMITH_API_KEY and expects LANGSMITH_PROJECT to ensure actions target the correct project/trace context.

Quick Start

Ask the AI to guide you to create a new LangSmith evaluation dataset, add examples, and upload the resulting JSON file using the langsmith CLI with your LANGSMITH_API_KEY.

Frequently Asked Questions about langsmith-dataset

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create and upload evaluation datasets to LangSmith for LLM testing?▼

You create LangSmith evaluation datasets by structuring examples for final_response, single_step, trajectory, or RAG scenarios, then uploading them via the langsmith CLI using your LANGSMITH_API_KEY for model validation.

What types of evaluation datasets does LangSmith support for RAG and trajectory testing?▼

LangSmith supports common evaluation dataset structures including final_response, single_step, trajectory, and RAG scenarios, allowing you to validate model workflows across different testing patterns.

Do I need a LANGSMITH_API_KEY to manage datasets via the CLI?▼

Yes, LangSmith CLI management requires a LANGSMITH_API_KEY for authentication. You can also optionally use LANGSMITH_WORKSPACE_ID for scoping actions to the correct project and trace context.

Can I list, export, and delete existing LangSmith datasets using CLI commands?▼

Yes, LangSmith dataset lifecycle management allows you to list, create, get, export, and delete datasets via the CLI, alongside managing dataset examples and viewing experiment results tied to those datasets.

What is the best way to structure RAG examples for LangSmith evaluation uploads?▼

The best way to structure RAG examples for LangSmith is using SDK-oriented dataset and example creation patterns that align with your RAG evaluation scenarios, ensuring consistent and reusable test data for model validation.

Related Skills

Explore More Data & Analytics Skills→
dotlab-hqdotlab-hq

langsmith-trace

Instrument applications and query LangSmith traces to debug LLM workflows.

Official
Intermediate
Harmeet10000Harmeet10000
7

langsmith-dataset

Create, upload, and manage LangSmith evaluation datasets via CLI and SDK.

Community
Advanced
IgorGanapolskyIgorGanapolsky
38

LangSmith Monitor Skill

Monitor LangSmith traces, runs, and performance metrics via Python API.

Community
Advanced
trueratruera
3.5k

trulens-dataset-curation

Create and curate ground-truth evaluation datasets for TruLens.

Official
Intermediate