langsmith-observability

Set LANGSMITH_TRACING=true and LANGSMITH_API_KEY, then decorate functions with @traceable or wrap the OpenAI client using langsmith.wrappers.wrap_openai. All calls are automatically captured with inputs, outputs, and latency.

Create a dataset with client.create_dataset and add examples, then call langsmith.evaluate with your model function and custom or built-in evaluators. Results include per-example scores and aggregate metrics for comparison across experiments.

LangSmith focuses on LLM tracing, prompt debugging, and dataset-based evaluation, while Weights & Biases targets deep learning experiment tracking and model training. Choose LangSmith for LLM application observability and W&B for training workflows.

Yes, LangChain integration is automatic once LANGSMITH_TRACING=true is set. All chain, prompt, and LLM runs are traced without code changes, and LangChain string evaluators can be used directly in evaluations.

Traces fail to appear when LANGSMITH_TRACING is not set to true, the API key is missing or invalid, or the project name is misconfigured. Verify connectivity with Client().list_projects() and call client.flush() to send pending traces.

Enable background batching with Client(auto_batch_tracing=True), set LANGSMITH_TRACING_SAMPLING_RATE to sample a fraction of traces, and use process_inputs to strip large or sensitive fields from recorded payloads.