PlatformPlatform (Python / JS)

LangSmith

See exactly what your AI app did on every request, and run evals on saved datasets.

What it is

LangSmith is a hosted platform from the LangChain team. It records a trace of every step your AI app takes (the prompt, the documents it fetched, the model reply), and lets you save example inputs as datasets and run evals against them from a web dashboard.

Why a QA engineer cares

When an AI answer looks wrong, the trace shows you why, step by step, like a network tab for AI. It is strong for debugging and for watching quality in production, not just in a test file.

Get started

Install it, then run the example below.

Install

pip install langsmith   # set LANGCHAIN_API_KEY

Quickstart

import os
from langsmith import traceable

os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGCHAIN_API_KEY"] = "your-key"

@traceable                       # every call is now traced
def my_app(question: str) -> str:
    ...

my_app("What is the refund policy?")
# Open the trace + create datasets and evals in the LangSmith dashboard

What you get

✓Full step-by-step trace of every AI request
✓Save datasets and run evals from the web UI
✓Watch quality, cost and latency in production
✓Works with any stack, not only LangChain

Best for

Debugging AI appsProduction monitoringTeam dashboards

Where it fits

Pair with DeepEval or Ragas for scoring; LangSmith is the tracing and dashboard layer.

Official docs for LangSmith →

Other AI-testing tools

promptfoo

Run evals and safety tests on prompts and models, works in CI

DeepEval

Pytest-style AI evals (correct, relevant, bias)

Ragas

Check RAG bots: did they find and use the right source

Giskard

Scan AI models for weak spots and unsafe answers

Confident AI

Dashboards to track eval scores over time

Put it to work

See where LangSmith fits in the full picture, or follow the step-by-step AI for QA roadmap.

The AI testing guide The AI for QA roadmap