skills.rest
dhar174
dhar174COMMUNITY
Explore all skills by this author →
WebsiteGitHub

langsmith-dataset

Create and manage LangSmith evaluation datasets via CLI and SDKs.

1|Updated Dec 22, 2025
Search Tags:#testing#llm#rag#evaluation#data management#dataset#langsmith
One-click install
npx skills add https://github.com/dhar174/langgraph_system_generator --skill langsmith-dataset-dhar174
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: langsmith-dataset
Source: https://github.com/dhar174/langgraph_system_generator/tree/main/.github/skills/langsmith-dataset
Command: npx skills add https://github.com/dhar174/langgraph_system_generator --skill langsmith-dataset-dhar174

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill streamlines the creation, management, and uploading of evaluation datasets to LangSmith, crucial for testing and validating AI models and systems.

Core Features & Use Cases

  • Dataset Management: Create, list, get, export, and delete datasets via the LangSmith CLI.
  • Example Management: Add, list, and delete individual examples within datasets.
  • Dataset Types: Supports various formats like final_response, single_step, trajectory, and rag for comprehensive testing.
  • Use Case: After running a batch of tests, you can use this Skill to export the traces, convert them into a structured dataset format, and upload it back to LangSmith for future regression testing or performance analysis.

Quick Start

Use the langsmith CLI to upload a local JSON file named 'evaluation_data.json' as a new dataset called 'My Test Dataset'.

Frequently Asked Questions about langsmith-dataset

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create LangSmith evaluation datasets from exported traces?▼

Yes, you can manage LangSmith datasets using the CLI to create, list, get, export, and delete datasets, as well as add, list, and delete individual examples within those datasets.

What dataset types does LangSmith support for LLM testing?▼

LangSmith supports final_response, single_step, trajectory, and rag dataset types to provide comprehensive evaluation coverage for various LLM and RAG testing scenarios.

Do I need a LangSmith API key to upload evaluation datasets?▼

Yes, uploading evaluation datasets requires a LangSmith API key and project configuration to authenticate and interact with the platform via the CLI or SDKs.

How do I upload a local JSON file as a LangSmith dataset?▼

To upload a local JSON file as a LangSmith dataset, use the langsmith CLI to push the file directly to a new dataset, enabling immediate use for LLM evaluation.

Can I manage individual examples within a LangSmith dataset?▼

Yes, you can manage individual examples within LangSmith datasets by using the CLI to add new examples, list existing ones, or delete outdated entries.

Related Skills

Explore More Software Engineering Skills→
lguibrlguibr

langsmith-trace

Instrument applications to generate LangSmith traces for LLM observability.

Community
Advanced
EntelligentsiaEntelligentsia
1

evaluation-harness

Evaluate AI model outputs against predefined criteria using golden datasets and LLM-as-judge.

Official
Advanced
jxtngxjxtngx
2

langsmith-evaluator

Create and deploy LLM agent evaluation pipelines within the LangSmith ecosystem.

Community
Advanced
joonlabjoonlab
8

langsmith-evaluator

Create and deploy LangSmith evaluation pipelines for LLM agents.

Community
Advanced