Agent Skills › langchain-ai/design-agent › design-agent-evals

design-agent-evals

GitHub

用于评估设计代理变更效果,通过运行前后对比测试、分析报告及将真实故障转化为测试任务,量化模型表现与成本。

.agents/skills/design-agent-evals/SKILL.md langchain-ai/design-agent

Trigger Scenarios

评估指令、技能或模型的变更 询问代理性能如何时 发现真实请求失败并需添加测试用例时

Install

npx skills add langchain-ai/design-agent --skill design-agent-evals -g -y
More Options

Non-standard path

npx skills add https://github.com/langchain-ai/design-agent/tree/main/.agents/skills/design-agent-evals -g -y

Use without installing

npx skills use langchain-ai/design-agent@design-agent-evals

指定 Agent (Claude Code)

npx skills add langchain-ai/design-agent --skill design-agent-evals -a claude-code -g -y

安装 repo 全部 skill

npx skills add langchain-ai/design-agent --all -g -y

预览 repo 内 skill

npx skills add langchain-ai/design-agent --list

SKILL.md

Frontmatter
{
    "name": "design-agent-evals",
    "description": "Find out whether a change made Design Agent better or worse. Run the eval set or a few tasks before and after against the same model, read the reports, look at the files, and turn a real request that went wrong into a task. Use after changing instructions, skills, checks or models, or when someone asks how good it is."
}

Evaluate a change

Eval runs spend model and media money. eval run prints the budget: show it to the person and get a yes. The whole set costs about $120 per model; a few tasks cost a few dollars.

  1. uv run design-agent eval list shows every task with its dollar and minute caps.
  2. Pick the tasks the change touches, plus one behavior task: still tasks for a stills skill, the teaser and film for motion, exact-copy for copy rules.
  3. Run before and after, same model, same tasks: uv run design-agent eval run --task <id> --task <id> --out workspace/out/evals/<label>.
  4. Read the reports. Each folder has report.json, README.md (failed checks per task, spend, time) and files/<task>/. Open the files and look at them. The graders check outputs, sizes, copy and behavior, never taste: a person judges taste.
  5. Report numbers from the reports: checks passed, spend and time, before against after. Do not summarize a run you did not read.
  6. Turn a real failure into a task. When a thread shows something going wrong, add an entry to tests/eval/tasks.json modelled on a similar task (the request, expected outputs and sizes, exact copy lines, budgets), then uv run pytest -q tests/unit/test_evals.py.

Graders and reports: Evals.

Version History

  • 8134d7c Current 2026-11-20 21:40

Same Skill Collection

.agents/skills/brand-onboarding/SKILL.md
.agents/skills/deep-agents-core/SKILL.md
.agents/skills/deep-agents-memory/SKILL.md
.agents/skills/deep-agents-orchestration/SKILL.md
.agents/skills/design-agent-customize/SKILL.md
.agents/skills/design-agent-deploy/SKILL.md
.agents/skills/design-agent-onboarding/SKILL.md
.agents/skills/design-agent-troubleshoot/SKILL.md
.agents/skills/design-agent-update/SKILL.md
.agents/skills/langsmith-evaluator/SKILL.md
.agents/skills/vendor-sync/SKILL.md
skills/brand-design/SKILL.md
skills/content-brief/SKILL.md
skills/data-viz/SKILL.md
skills/design-wiki/SKILL.md
skills/faceless-explainer/SKILL.md
skills/film-breakdown/SKILL.md
skills/general-video/SKILL.md
skills/gif-edit/SKILL.md
skills/hyperframes-core/SKILL.md
skills/hyperframes-creative/SKILL.md
skills/hyperframes-keyframes/SKILL.md
skills/image-generation/SKILL.md
skills/image-ops/SKILL.md
skills/launch-film/SKILL.md
skills/media-production/SKILL.md
skills/motion-direction/SKILL.md
skills/music-to-video/SKILL.md
skills/paper-design/SKILL.md
skills/pr-to-video/SKILL.md
skills/reference-study/SKILL.md
skills/remotion-to-hyperframes/SKILL.md
skills/slideshow/SKILL.md
skills/sound/SKILL.md
skills/stills/SKILL.md
skills/storyboard/SKILL.md
skills/thumbnails/SKILL.md
skills/visual-verify/SKILL.md
.agents/skills/managed-deep-agents/SKILL.md
skills/embedded-captions/SKILL.md
skills/hyperframes-animation/SKILL.md
skills/hyperframes-audio/SKILL.md
skills/hyperframes-cli/SKILL.md
skills/hyperframes-registry/SKILL.md
skills/hyperframes/SKILL.md
skills/media-use/SKILL.md
skills/motion-graphics/SKILL.md

Metadata

Files
0
Version
8134d7c
Hash
884c8efb
Indexed
2026-11-20 21:40

Главная - Вики-сайт
Copyright © 2011-2026 iteam. Current version is 2.156.0. UTC+08:00, 2026-11-20 22:28
浙ICP备14020137号-1