Agent Skills › meituan-longcat/WBench › wbench-generate

wbench-generate

GitHub

驱动generate.py为注册视频模型生成测试视频，支持文本或相机/动作条件模型。需配置API凭证，先单用例冒烟测试，再全量运行并断点续跑，输出合并多轮视频文件至指定目录。

.claude/skills/wbench-generate/SKILL.md meituan-longcat/WBench

Trigger Scenarios

用户要求生成模型视频提及使用wan/kling等模型跑case 请求执行视频生成任务

Install

npx skills add meituan-longcat/WBench --skill wbench-generate -g -y

More Options

Non-standard path

npx skills add https://github.com/meituan-longcat/WBench/tree/main/.claude/skills/wbench-generate -g -y

Use without installing

npx skills use meituan-longcat/WBench@wbench-generate

指定 Agent (Claude Code)

npx skills add meituan-longcat/WBench --skill wbench-generate -a claude-code -g -y

安装 repo 全部 skill

npx skills add meituan-longcat/WBench --all -g -y

预览 repo 内 skill

npx skills add meituan-longcat/WBench --list

SKILL.md

Frontmatter

{
    "name": "wbench-generate",
    "description": "Generate WBench videos for a model. Use when the user asks to generate \/ produce videos for a registered model (e.g. \"generate kling videos\", \"用 wan 生成全部 case\", \"跑 camera_preview 的 navi case\"). Drives generate.py over data\/cases and writes work_dirs\/<model>\/videos\/case_<id>_combined.mp4."
}

WBench Video Generation

Run a registered model over the dataset cases and produce one combined multi-turn clip per case at work_dirs/<model>/videos/case_<id>_combined.mp4.

Entry point: generate.py (repo root). Paths are relative to the repo root, so cd into the checkout first.

Registered models

wan, kling, seedance (text-conditioned), camera_preview, action_preview (reference camera/action demos). List them anytime:

python -c "from src.models import list_models; print(list_models())"

To add your own, register_model(name, cls) in src/models/__init__.py; subclass ConditionedVideoModel (camera/action) or BaseVideoModel (text). See src/models/{camera,action}/example_model.py.

Which cases to cover (by model type)

Type	Cases	Count
text	all	289
camera / action	navigation only	158

A case is "navigation" if it has ≥1 W/A/S/D/arrow action. Camera/action models must be restricted to the 158 navi cases — passing all 289 wastes compute and produces videos for cases the model can't be scored on.

Workflow

1. Pick GPU (if the model needs one) and set API creds (text models)

Text models call a video API:

export VIDEO_API_URL="https://your-video-api.com"
export VIDEO_API_KEY="your-key"

2. Smoke test on one case first

python generate.py --model <model> --cases data/cases/case_1.json

Confirm work_dirs/<model>/videos/case_1_combined.mp4 exists and plays before launching the full run.

3. Full run (background for anything > a few minutes)

mkdir -p logs
nohup python generate.py --model <model> --resume \
  > logs/generate_<model>.log 2>&1 &

--resume skips cases that already have a video — safe to re-run after an interruption.
--limit N caps the number of cases (quick sanity passes).
--cases f1.json f2.json ... restricts to specific cases.
For camera/action models, pass only the navi cases via --cases (glob the navi id list from data/cases/).

4. Verify output

ls work_dirs/<model>/videos/ | wc -l          # expected count (289 or 158)
python -c "import cv2,glob; \
  [print(p, int(cv2.VideoCapture(p).get(7))) for p in glob.glob('work_dirs/<model>/videos/*.mp4')[:3]]"

Gotchas

The combined clip must contain all turns concatenated in order — the multi-turn logic (build prompt → infer → take last frame → next turn) is handled by generate_multi_turn; don't emit per-turn files.
Video filename uses the JSON id field, not the source filename (e.g. case_210_scratch.json whose JSON id is 211 → case_211_combined.mp4).
Frames-per-turn differ per model and matter for submission turns.json — see the wbench-submit skill.