Agent Skills › meituan-longcat/WBench › wbench-generate

wbench-generate

GitHub

用于为注册的视频生成模型批量运行数据集案例并输出合并后的多轮视频片段。支持文本条件及摄像头/动作预览模型，涵盖环境配置、单例冒烟测试、后台全量执行及结果验证流程。

.claude/skills/wbench-generate/SKILL.md meituan-longcat/WBench

Trigger Scenarios

用户要求为特定模型生成视频需要批量处理数据集以制作评估视频询问如何运行视频生成脚本

Install

npx skills add meituan-longcat/WBench --skill wbench-generate -g -y

More Options

Non-standard path

npx skills add https://github.com/meituan-longcat/WBench/tree/main/.claude/skills/wbench-generate -g -y

Use without installing

npx skills use meituan-longcat/WBench@wbench-generate

指定 Agent (Claude Code)

npx skills add meituan-longcat/WBench --skill wbench-generate -a claude-code -g -y

安装 repo 全部 skill

npx skills add meituan-longcat/WBench --all -g -y

预览 repo 内 skill

npx skills add meituan-longcat/WBench --list

SKILL.md

Frontmatter

{
    "name": "wbench-generate",
    "description": "Generate WBench videos for a model. Use when the user asks to generate \/ produce videos for a registered model (e.g. \"generate kling videos\", \"用 wan 生成全部 case\", \"跑 camera_preview 的 navi case\"). Drives generate.py over data\/cases and writes work_dirs\/<model>\/videos\/case_<id>_combined.mp4."
}

WBench Video Generation

Run a registered model over the dataset cases and produce one combined multi-turn clip per case at work_dirs/<model>/videos/case_<id>_combined.mp4.

Entry point: generate.py (repo root). Paths are relative to the repo root, so cd into the checkout first.

Registered models

wan, kling, seedance (text-conditioned), camera_preview, action_preview (reference camera/action demos). List them anytime:

python -c "from src.models import list_models; print(list_models())"

To add your own, register_model(name, cls) in src/models/__init__.py; subclass ConditionedVideoModel (camera/action) or BaseVideoModel (text). See src/models/{camera,action}/example_model.py.

Which cases to cover (by model type)

Type	Cases	Count
text	all	289
camera / action	navigation only	158

A case is "navigation" if it has ≥1 W/A/S/D/arrow action. Camera/action models must be restricted to the 158 navi cases — passing all 289 wastes compute and produces videos for cases the model can't be scored on.

Workflow

1. Pick GPU (if the model needs one) and set API creds (text models)

Text models call a video API:

export VIDEO_API_URL="https://your-video-api.com"
export VIDEO_API_KEY="your-key"

2. Smoke test on one case first

python generate.py --model <model> --cases data/cases/case_1.json

Confirm work_dirs/<model>/videos/case_1_combined.mp4 exists and plays before launching the full run.

3. Full run (background for anything > a few minutes)

mkdir -p logs
nohup python generate.py --model <model> --resume \
  > logs/generate_<model>.log 2>&1 &

--resume skips cases that already have a video — safe to re-run after an interruption.
--limit N caps the number of cases (quick sanity passes).
--cases f1.json f2.json ... restricts to specific cases.
For camera/action models, pass only the navi cases via --cases (glob the navi id list from data/cases/).

4. Verify output

ls work_dirs/<model>/videos/ | wc -l          # expected count (289 or 158)
python -c "import cv2,glob; \
  [print(p, int(cv2.VideoCapture(p).get(7))) for p in glob.glob('work_dirs/<model>/videos/*.mp4')[:3]]"

Gotchas

The combined clip must contain all turns concatenated in order — the multi-turn logic (build prompt → infer → take last frame → next turn) is handled by generate_multi_turn; don't emit per-turn files.
Video filename uses the JSON id field, not the source filename (e.g. case_210_scratch.json whose JSON id is 211 → case_211_combined.mp4).
Frames-per-turn differ per model and matter for submission turns.json — see the wbench-submit skill.