Agent Skills › langwatch/langwatch › evaluations

evaluations

GitHub

LangWatch评估请求的路由技能，用于区分预部署实验与生产在线评估。通过意图分类将请求分发至experiments或online-evaluations子技能，若意图模糊则向用户提问确认，避免默认行为。

skills/_compiled/native/evaluations/SKILL.md langwatch/langwatch

Trigger Scenarios

用户请求进行模型或提示词评估但未明确是离线测试还是线上监控用户输入模糊的评估指令如'make me an eval' 需要路由评估任务到专用处理流程时

Install

npx skills add langwatch/langwatch --skill evaluations -g -y

More Options

Non-standard path

npx skills add https://github.com/langwatch/langwatch/tree/main/skills/_compiled/native/evaluations -g -y

Use without installing

npx skills use langwatch/langwatch@evaluations

指定 Agent (Claude Code)

npx skills add langwatch/langwatch --skill evaluations -a claude-code -g -y

安装 repo 全部 skill

npx skills add langwatch/langwatch --all -g -y

预览 repo 内 skill

npx skills add langwatch/langwatch --list

SKILL.md

Frontmatter

{
    "name": "evaluations",
    "license": "MIT",
    "description": "Compatibility router for LangWatch evaluation requests. Use only when the user asks for evaluations without making it clear whether they mean pre-deployment experiments or production online evaluations. Routes the request to the focused companion skill and does not implement either workflow itself.",
    "user-prompt": "Help me evaluate my agent",
    "compatibility": "Works with Claude Code and similar AI assistants. The `langwatch` CLI is the only interface for platform operations and documentation."
}

Route an Evaluation Request

This is a compatibility skill. Do not build an experiment, monitor, or guardrail from this skill.

Classify the user's intent:

Intent	Correct skill
Batch test a dataset, compare prompts or models, benchmark, create a CI quality gate	`experiments`
Score live traces or threads, monitor production quality, create a guardrail	`online-evaluations`

If the request remains ambiguous after inspecting context (a bare "make me an eval" that names neither a dataset nor live traffic), do not create anything yet. This choice picks what gets tested, so it is the user's to make, not a default's. Ask it as a question card and stop; the answer arrives as the next message.

In Langy, ask it with the question tool and stop; the options are the concrete alternatives, for example "A dataset, before deployment" and "Live production traffic". Elsewhere, ask the same question as one short line of prose.

A rejected field value is not this kind of choice. If a create later fails with a validation_error whose reason names the field and an expected list, correct that exact field from the list and retry once. Never turn a fixable slug into a question for the user.

Then hand off:

If the correct companion skill is available, load it and follow it instead of continuing here.
If experiments is missing, tell the user to install it with:
```
npx skills@1.5.19 add langwatch/skills/experiments
```
If online-evaluations is missing, tell the user to install it with:
```
npx skills@1.5.19 add langwatch/skills/online-evaluations
```

Do not recreate the companion skill's instructions from memory. Load the focused skill so its current workflow, safety checks, and verification steps are used.