Agent Skills › meituan-longcat/WBench › wbench-generate

wbench-generate

GitHub

为注册模型生成 WBench 视频。支持文本、相机及动作模型，按类型筛选测试用例（文本全量，相机/动作仅导航）。提供单例冒烟测试与后台断点续跑功能，确保输出多轮拼接视频。

.claude/skills/wbench-generate/SKILL.md meituan-longcat/WBench

Trigger Scenarios

生成模型视频运行视频生成任务用 wan/kling 生成 case

Install

npx skills add meituan-longcat/WBench --skill wbench-generate -g -y

More Options

Non-standard path

npx skills add https://github.com/meituan-longcat/WBench/tree/main/.claude/skills/wbench-generate -g -y

Use without installing

npx skills use meituan-longcat/WBench@wbench-generate

指定 Agent (Claude Code)

npx skills add meituan-longcat/WBench --skill wbench-generate -a claude-code -g -y

安装 repo 全部 skill

npx skills add meituan-longcat/WBench --all -g -y

预览 repo 内 skill

npx skills add meituan-longcat/WBench --list

SKILL.md

Frontmatter

{
    "name": "wbench-generate",
    "description": "Generate WBench videos for a model. Use when the user asks to generate \/ produce videos for a registered model (e.g. \"generate kling videos\", \"用 wan 生成全部 case\", \"跑 camera_preview 的 navi case\"). Drives generate.py over data\/cases and writes work_dirs\/<model>\/videos\/case_<id>_combined.mp4."
}

WBench Video Generation

Run a registered model over the dataset cases and produce one combined multi-turn clip per case at work_dirs/<model>/videos/case_<id>_combined.mp4.

Entry point: generate.py (repo root). Paths are relative to the repo root, so cd into the checkout first.

Registered models

wan, kling, seedance (text-conditioned), camera_preview, action_preview (reference camera/action demos). List them anytime:

python -c "from src.models import list_models; print(list_models())"

To add your own, register_model(name, cls) in src/models/__init__.py; subclass ConditionedVideoModel (camera/action) or BaseVideoModel (text). See src/models/{camera,action}/example_model.py.

Which cases to cover (by model type)

Type	Cases	Count
text	all	289
camera / action	navigation only	158

A case is "navigation" if it has ≥1 W/A/S/D/arrow action. Camera/action models must be restricted to the 158 navi cases — passing all 289 wastes compute and produces videos for cases the model can't be scored on.

Workflow

1. Pick GPU (if the model needs one) and set API creds (text models)

Text models call a video API:

export VIDEO_API_URL="https://your-video-api.com"
export VIDEO_API_KEY="your-key"

2. Smoke test on one case first

python generate.py --model <model> --cases data/cases/case_1.json

Confirm work_dirs/<model>/videos/case_1_combined.mp4 exists and plays before launching the full run.

3. Full run (background for anything > a few minutes)

mkdir -p logs
nohup python generate.py --model <model> --resume \
  > logs/generate_<model>.log 2>&1 &

--resume skips cases that already have a video — safe to re-run after an interruption.
--limit N caps the number of cases (quick sanity passes).
--cases f1.json f2.json ... restricts to specific cases.
For camera/action models, pass only the navi cases via --cases (glob the navi id list from data/cases/).

4. Verify output

ls work_dirs/<model>/videos/ | wc -l          # expected count (289 or 158)
python -c "import cv2,glob; \
  [print(p, int(cv2.VideoCapture(p).get(7))) for p in glob.glob('work_dirs/<model>/videos/*.mp4')[:3]]"

Gotchas

The combined clip must contain all turns concatenated in order — the multi-turn logic (build prompt → infer → take last frame → next turn) is handled by generate_multi_turn; don't emit per-turn files.
Video filename uses the JSON id field, not the source filename (e.g. case_210_scratch.json whose JSON id is 211 → case_211_combined.mp4).
Frames-per-turn differ per model and matter for submission turns.json — see the wbench-submit skill.