Integration · Developer tools

Connect Claude & ChatGPT to LangSmith

Ask your AI how a tracing project is doing, and it pulls run counts, latency, tokens, cost and errors from LangSmith, LangChain's platform for tracing and evaluation. It can add examples to datasets, record feedback on runs, read and save prompt versions, and find the annotation queues assigned to you.

Connect LangChain (LangSmith) free

Free plan, no credit card. Takes about a minute.

Try asking

“How many runs errored in the support-bot project in the last 24 hours, and what did they cost?”
The AI gets run statistics for the project in that time range: counts, errors, latency, tokens and cost.
“Add this question and the expected answer to the regression dataset.”
It finds the dataset and adds an example with the inputs, the expected output and a split if you name one.
“Show me the latest version of our summarizer prompt.”
It reads the prompt's content at the latest commit. It can also list earlier versions so you can compare.

What your AI can do

Tracing projects and feedback

Get run statistics per project, preview the runs in a conversation thread, and create, list and update feedback.

Datasets and examples

Search, create and update datasets, add, edit and delete examples, and export a dataset as CSV.

Prompts and annotation queues

Search the prompt hub, read and save prompt versions, create prompts, and create or look up annotation queues.

“Mark run 4f2a as incorrect and add a comment on why.”
It records feedback on the run with the key correctness, a score and your comment.
“Export the eval dataset as CSV.”
It downloads the dataset's examples as CSV, or as the dataset looked on an earlier date.
“Which annotation queues are assigned to me?”
It searches annotation queues assigned to you and can open one to show its rubric and settings.

139 tools for LangChain (LangSmith)

These tools are switched on when you connect. You can switch any of them off, or require your approval before it runs.

create_annotation_queueChanges dataCreate an annotation queue with a `name`, optional description, rubric instructions and reviewer settings.
create_batchVisible to othersCreate a new batch. Has effects visible outside this system. API summary: Ingest Runs (Batch JSON).
create_bulkVisible to othersCreate a new bulk. Required: items. Has effects visible outside this system. API summary: Create Examples.
create_bulk_validate_examples_api_v1_examples_validate_bulk_postVisible to othersCreate a new bulk. Has effects visible outside this system. API summary: Validate Examples.
create_comparativeChanges dataCreate a new comparative. Required: experiment ids. API summary: Create Comparative Experiment.
create_configChanges dataCreate a new config. Required: name or config. API summary: [Beta] Create Insights Job Config.
create_datasetChanges dataCreate a dataset with a `name`, optional `description` and `data_type` (kv, llm or chat).
create_dataset_generateChanges dataCreate a new generate for a dataset. Required: num examples.
create_exampleChanges dataAdd an example to a dataset: `dataset_id` plus `inputs`, optional `outputs`, `metadata` and `split`.
create_experiment_evaluateChanges dataCreate a new evaluate for a experiment. Required: rule id. API summary: Evaluate Experiment Adhoc.
create_feedbackChanges dataRecord feedback on a run: a feedback `key` (e.g. correctness) with `score` or `value`, optional `comment` and `run_id`.
create_feedback_configChanges dataCreate a new feedback config. Required: feedback key or feedback config. API summary: Create Feedback Config Endpoint.
create_generateChanges dataCreate a new generate. Required: user context. API summary: [Beta] Auto-Generate Insights Job Config.
create_generate_queryChanges dataCreate a new generate query. Required: query. API summary: Generate Query For Runs.
create_groupChanges dataCreate a new group. Required: session id or group by. API summary: Group Runs.
create_invoke_promptChanges dataCreate a new invoke prompt. Required: messages, template format or inputs.
create_logChanges dataCreate a new project for a project. Required: log type or message. API summary: Create Log.
create_message_likeChanges dataCreate a new like for a message. API summary: Like Comment.
create_multipartChanges dataCreate a new multipart. API summary: Ingest Runs (Multipart).
create_populateChanges dataCreate a new populate. Required: queue id or session ids. API summary: Populate Annotation Queue.
create_projectChanges dataCreate a new project. Required: algorithm, config or prompt name. API summary: Optimize Prompt Job.
create_promptChanges dataCreate a new prompt by `repo_handle` and `is_public`, with optional description, readme and tags.
create_repo_forkChanges dataCreate a new fork for a repo. Required: repo handle. API summary: Fork Repo.
create_repo_ownerChanges dataCreate a new owner for a repo. Required: email. API summary: Add Repo Owner.
create_repo_projectChanges dataCreate a new project for a repo. Required: algorithm or config. API summary: Create Job.
create_repo_tagChanges dataCreate a new tag for a repo. Required: tag name or commit id. Search tags first to avoid duplicates. API summary: Create Tag.
create_ruleChanges dataCreate a new rule. Required: display name or sampling rate. API summary: Create Rule.
create_rule_triggerChanges dataCreate a new trigger for a rule. API summary: Trigger Rule.
create_runChanges dataCreate a new run. API summary: Create a Run.
create_session_dashboardChanges dataCreate a new dashboard for a session. API summary: Get Tracing Project Prebuilt Dashboard.
create_session_insightChanges dataCreate a new insight for a session. API summary: [Beta] Create Insights Job.
create_session_viewChanges dataCreate a new view for a session. Required: display name. Search views first to avoid duplicates. API summary: Create Filter View.
create_stat_stats_group_runs_api_v1_runs_group_stats_postChanges dataCreate a new stat. Required: session id or group by. API summary: Stats Group Runs.
create_studio_experimentChanges dataCreate a new studio experiment. Required: project name or dataset id.
create_sub_commentChanges dataUpdate an existing message identified by parent comment id. API summary: Create Sub Comment.
create_tokenDeletesCreate a new token. Required: body. Destructive action; may require approval. API summary: Create Feedback Ingest Token.
create_triggerChanges dataCreate a new trigger. API summary: Trigger Rules.
delete_annotation_queueDeletesPermanently delete a annotation queue by queue id. Destructive action; may require approval. API summary: Delete Annotation Queue.
delete_annotation_queuesMoves moneyPermanently delete a annotation queue. Financial or irreversible action; may require approval. API summary: Delete Annotation Queues.
delete_comparativeDeletesPermanently delete a comparative by comparative experiment id. Destructive action; may require approval. API summary: Delete Comparative Experiment.
delete_configDeletesPermanently delete a config by config id. Destructive action; may require approval. API summary: [Beta] Delete Insights Job Config.
delete_datasetDeletesPermanently delete a dataset by dataset id. Destructive action; may require approval. API summary: Delete Dataset.
delete_dataset_sharesMoves moneyPermanently delete a share by dataset id. Financial or irreversible action; may require approval. API summary: Unshare Dataset.
delete_datasetsMoves moneyPermanently delete a dataset. Financial or irreversible action; may require approval. API summary: Delete Datasets.
delete_deletesDeletesPermanently delete a delete. Destructive action; may require approval. API summary: Delete Runs.
delete_exampleDeletesPermanently delete one dataset example by `example_id`. Cannot be undone.
delete_examplesMoves moneyPermanently delete a example. Financial or irreversible action; may require approval. API summary: Delete Examples.
delete_feedbackDeletesPermanently delete a feedback by feedback id. Destructive action; may require approval. API summary: Delete Feedback.
delete_feedback_configsMoves moneyPermanently delete a feedback config. Financial or irreversible action; may require approval. API summary: Delete Feedback Config Endpoint.
delete_logDeletesPermanently delete a project by project id. Destructive action; may require approval. API summary: Delete Log.
delete_message_likesMoves moneyPermanently delete a like by parent comment id. Financial or irreversible action; may require approval. API summary: Unlike Comment.
delete_projectDeletesPermanently delete a project by job id. Destructive action; may require approval. API summary: Delete Job.
delete_repoDeletesPermanently delete a repo by repo. Destructive action; may require approval. API summary: Delete Repo.
delete_repo_ownersMoves moneyPermanently delete a owner by repo. Financial or irreversible action; may require approval. API summary: Remove Repo Owner.
delete_reposMoves moneyPermanently delete a repo. Financial or irreversible action; may require approval. API summary: Delete Repos.
delete_ruleDeletesPermanently delete a rule by rule id. Destructive action; may require approval. API summary: Delete Rule.
delete_sessionDeletesPermanently delete a session by session id. Destructive action; may require approval. API summary: Delete Tracer Session.
delete_session_insightDeletesPermanently delete a insight by job id. Destructive action; may require approval. API summary: [Beta] Delete Insights Job.
delete_session_viewDeletesPermanently delete a view by view id. Destructive action; may require approval. API summary: Delete Filter View.
delete_tagDeletesPermanently delete a tag by tag name. Destructive action; may require approval. API summary: Delete Tag.
delete_tracesDeletesPermanently delete a trace. Destructive action; may require approval. API summary: Delete Runs Abac.
download_dataset_csvDownload a dataset's examples as CSV by `dataset_id`, optionally as of a past version (`as_of`).
download_jsonlDownload the document for a jsonl. API summary: Download Dataset Jsonl.
download_openaiDownload the document for a openai. API summary: Download Dataset Openai.
download_openai_ftDownload the document for a openai ft. API summary: Download Dataset Openai Ft.
export_annotation_queuesChanges dataExport annotation queues. API summary: Export Annotation Queue Archived Runs.
get_annotation_queueGet one annotation queue by `queue_id`, including its rubric and settings.
get_datasetGet one dataset by `dataset_id`, including its description and example count.
get_exampleGet one dataset example by `example_id`, including its inputs and outputs.
get_example_2Get one example. API summary: Count Examples.
get_feedbackGet one feedback entry by `feedback_id`.
get_insight_clusterGet one cluster for a insight by cluster id. API summary: [Beta] Get Run Cluster From Insights Job.
get_logGet one project for a project by project id. API summary: Get Log.
get_messageGet one message by repo. Results are paginated. API summary: Get Comments.
get_projectGet one project by job id. API summary: Get Job.
get_promptGet a prompt's details by `owner` (tenant handle, or '-' for your own) and `repo` (prompt handle).
get_prompt_versionGet the content (manifest) of one prompt version by `owner`, `repo` and `commit` (hash or 'latest').
get_run_statsChanges dataGet aggregate run statistics (counts, latency, tokens, cost, errors) for one or more tracing projects in `session`, with optional time range and filters.
get_session_insightGet one insight for a session by job id. API summary: [Beta] Get Insights Job.
get_session_viewGet one view for a session by view id. API summary: Read Filter View.
get_sub_commentsGet one message by parent comment id. Results are paginated. API summary: Get Sub Comments.
get_tagGet one tag by tag name. API summary: Get Tag.
get_threadPreview a conversation thread's runs by `thread_id`.
get_tracing_projectGet a tracing project by `session_id`, optionally with run statistics (`include_stats`).
list_annotation_queue_queuesList queues for a annotation queue. API summary: Get Annotation Queues For Run.
list_configsList configs. Optional filters: include prebuilts. API summary: [Beta] Get Insights Job Configs.
list_dataset_sharesList shares for a dataset. API summary: Read Dataset Share State.
list_dataset_splitsList splits for a dataset. Optional filters: as of. API summary: Get Dataset Splits.
list_dataset_versionsList versions for a dataset. Optional filters: as of or tag. API summary: Get Dataset Version.
list_diffsList diffs. Optional filters: from version or to version. API summary: Diff Dataset Versions.
list_feedbackList feedback, filtered by `run`, `source`, score or comment presence, date range and paging.
list_feedback_configsList feedback configs. Optional filters: name contains, sort by desc or read after write. Results are paginated. API summary: List Feedback Configs Endpoint.
list_insight_runsList runs for a insight. Optional filters: cluster id, attribute sort key or attribute sort order. Results are paginated. API summary: [Beta] Get Runs From Insights Job.
list_projectsList projects for a project. API summary: List Job Logs.
list_prompt_commitsList the saved versions (commits) of a prompt by `owner` and `repo`.
list_repo_ownersList owners for a repo. API summary: List Repo Owners.
list_repo_projectsList projects for a repo. API summary: List Jobs.
list_repo_tagsList tags for a repo. API summary: Get Tags.
list_rule_last_appliedsList last applieds for a rule. Optional filters: backfill. API summary: Get Last Applied Rule.
list_rule_logsList logs for a rule. Optional filters: start time or end time. Results are paginated. API summary: List Rule Logs.
list_rulesList rules. Optional filters: dataset id, type, name contains, id, evaluator id or tag value id.
list_session_insightsList insights for a session. Optional filters: config id or legacy. Results are paginated. API summary: [Beta] Get Insights Jobs.
list_session_metadataList metadata for a session. Optional filters: metadata keys, start time, k or root runs only. API summary: Read Tracer Sessions Runs Metadata.
list_tokensList tokens. Optional filters: run id. API summary: List Feedback Ingest Tokens.
list_v2sList v2s. Optional filters: backfill, start time or end time. Results are paginated. API summary: List Rule Logs V2.
save_prompt_versionChanges dataSave a new version of a prompt: `owner`, `repo` and the prompt `manifest`, optionally the `parent_commit`.
search_annotation_queuesSearch annotation queues by name, dataset or assignment (`assigned_to_me`).
search_dataset_versionsSearch versions for a dataset by search or example. Use this to find existing versions before creating a new version. Results are paginated. API summary: Get Dataset Versions.
search_datasetsSearch LangSmith datasets by `name_contains`, `name`, `id`, `data_type` or metadata, with limit/offset paging.
search_examplesList examples in a dataset (pass `dataset`), with optional `splits`, `metadata`, `full_text_contains` and paging.
search_promptsSearch prompts in the LangSmith prompt hub by `query`, tags, owner or visibility.
search_session_viewsSearch views for a session by type. Use this to find existing views before creating a new view. API summary: Read Filter Views.
search_streamsSearch streams by id, data type, name, name contains, metadata or sort by. Use this to find existing streams before creating a new stream. Results are paginated. API summary: Read Datasets Stream.
search_tagsSearch tags by tenant handle, tenant id, query, has commits, tags or is archived. Use this to find existing tags before creating a new tag. Results are paginated. API summary: List Repo Tags.
submit_datasetVisible to othersClone a dataset. Has effects visible outside this system. API summary: Clone Dataset.
submit_exampleVisible to othersValidate a example. Has effects visible outside this system. API summary: Validate Example.
submit_queryVisible to othersValidate a query. Has effects visible outside this system. API summary: Validate Runs Query.
submit_ruleVisible to othersValidate a rule. Has effects visible outside this system. API summary: Validate Rule.
update_annotation_queueChanges dataUpdate an existing annotation queue identified by queue id. Only the fields you pass are changed. API summary: Update Annotation Queue.
update_bulksVisible to othersUpdate an existing bulk. Only the fields you pass are changed. Has effects visible outside this system. API summary: Legacy Update Examples.
update_configChanges dataUpdate an existing config identified by config id. Only the fields you pass are changed. API summary: [Beta] Update Insights Job Config.
update_datasetChanges dataUpdate a dataset's `name`, `description` or metadata by `dataset_id`.
update_dataset_sharesVisible to othersUpdate an existing share identified by dataset id. Has effects visible outside this system. API summary: Share Dataset.
update_dataset_splitsVisible to othersUpdate an existing split identified by dataset id. Has effects visible outside this system. API summary: Update Dataset Splits.
update_dataset_tagsVisible to othersUpdate an existing tag identified by dataset id. Has effects visible outside this system. API summary: Update Dataset Version.
update_exampleChanges dataUpdate an example's `inputs`, `outputs`, `metadata` or `split` by `example_id`.
update_feedbackChanges dataUpdate a feedback entry's `score`, `value`, `comment` or `correction` by `feedback_id`.
update_feedback_configsVisible to othersUpdate an existing feedback config. Only the fields you pass are changed. Has effects visible outside this system. API summary: Update Feedback Config Endpoint.
update_messageChanges dataUpdate an existing message identified by repo. API summary: Create Comment.
update_projectChanges dataUpdate an existing project identified by job id. Only the fields you pass are changed. API summary: Update Job.
update_repoChanges dataUpdate an existing repo identified by repo. Only the fields you pass are changed. API summary: Update Repo.
update_ruleChanges dataUpdate an existing rule identified by rule id. Only the fields you pass are changed. API summary: Update Rule.
update_runChanges dataUpdate an existing run identified by run id. Only the fields you pass are changed. API summary: Update a Run.
update_sessionChanges dataUpdate an existing session identified by session id. Only the fields you pass are changed. API summary: Update Tracer Session.
update_session_insightChanges dataUpdate an existing insight identified by job id. Only the fields you pass are changed. API summary: [Beta] Update Insights Job.
update_session_viewChanges dataUpdate an existing view identified by view id. Only the fields you pass are changed. API summary: Update Filter View.
update_tagChanges dataUpdate an existing tag identified by tag name. Only the fields you pass are changed. API summary: Update Tag.
update_view_renamesVisible to othersUpdate an existing rename identified by view id. Only the fields you pass are changed. Has effects visible outside this system. API summary: Rename Filter View.
upload_dataset_experimentChanges dataUpload a dataset. API summary: Upload Experiment.

Set up in three steps

1
Pick the app
Create a free HeyMyApps account and choose LangChain (LangSmith) from the app list.
2
Paste your key
Paste a LangSmith API key. In LangSmith open Settings > API Keys and click Create API Key. You can use a Personal Access Token or a Service Key scoped to a workspace. Copy the key right away, it is shown only once.
3
Add the link to your AI
You get a personal MCP link. Add it to Claude, ChatGPT or Cursor:
1. Click your name, then Settings › Connectors › Add custom connector.
2. Paste your link as the Remote MCP server URL.
3. Switch it on from the + menu in a chat.

Questions

What can the AI do in LangSmith?

It can read run statistics for tracing projects, preview conversation threads, and read datasets, examples, feedback, prompts and annotation queues. It can create datasets, examples, feedback, prompts, prompt versions and annotation queues, update datasets, examples and feedback, and delete examples if you switch that tool on.

Does the AI see my LangSmith API key?

No. Your LangSmith API key is encrypted at rest and never shown to the AI. After you save it, it is not shown again, not even to you. The AI only sees the results of the tools it calls.

Can I control what the AI is allowed to do?

Yes. You choose which tools are switched on, so you can start read-only. Deleting a dataset example is permanent and can require your approval before it runs. Every tool call is logged.

Does it work with ChatGPT?

Yes. In ChatGPT go to Settings › Apps & Connectors › Advanced and turn on Developer mode, then add your HeyMyApps link. Developer mode needs a paid ChatGPT plan: Plus, Pro, Business or Enterprise. The same link also works in Claude (Settings › Connectors › Add custom connector), Cursor and other MCP clients.

What does it cost?

HeyMyApps has a free plan with no credit card required. Paid plans bill per completed task. You also need your own LangSmith account.

Should I use a Personal Access Token or a Service Key?

Both work. A Personal Access Token acts as you. A Service Key is scoped to a workspace, which suits a shared setup where the access should not depend on one person. You create either under Settings > API Keys in LangSmith.