Agent Skills › vllm-project/vllm-omni › diffusion-accuracy-test

diffusion-accuracy-test

GitHub

为vLLM-Omni扩散模型及加速特性添加数值精度测试与Buildkite CI覆盖,确保原生与参考实现的一致性。

.claude/skills/diffusion-accuracy-test/SKILL.md vllm-project/vllm-omni

触发场景

需要验证扩散模型或加速特性的数值准确性 建立新的模型精度测试用例 扩展CI覆盖率以检查生成结果对齐

安装

npx skills add vllm-project/vllm-omni --skill diffusion-accuracy-test -g -y
更多选项

非标准路径

npx skills add https://github.com/vllm-project/vllm-omni/tree/main/.claude/skills/diffusion-accuracy-test -g -y

不安装直接使用

npx skills use vllm-project/vllm-omni@diffusion-accuracy-test

指定 Agent (Claude Code)

npx skills add vllm-project/vllm-omni --skill diffusion-accuracy-test -a claude-code -g -y

安装 repo 全部 skill

npx skills add vllm-project/vllm-omni --all -g -y

预览 repo 内 skill

npx skills add vllm-project/vllm-omni --list

SKILL.md

Frontmatter
{
    "name": "diffusion-accuracy-test",
    "description": "Add or extend numerical accuracy tests and Buildkite CI coverage for existing vLLM-Omni diffusion image and video models, using aligned reference outputs and scoped tolerances. Use for native\/reference parity or accuracy checks for acceleration features; generation smoke tests and performance benchmarking are separate workflows."
}

Add diffusion accuracy tests

Add reproducible accuracy coverage for a working diffusion model or acceleration feature, using existing helpers, reference runners, and Buildkite conventions. Default to one representative real-weight comparison, including the reference and candidate runs it needs. Prefer a supported workload that exercises more of the model's input and output paths, such as IT2I over T2I for a model with image conditioning. Expand the workload/hardware matrix when the request, a distinct uncovered path, or a failure requires it. Tiny random weights do not prove model accuracy.

For a model that is not yet integrated, first use add-diffusion-model to establish a working runtime, then return here for the requested numerical coverage. An already working model can use this skill directly.

Workflow

  1. Choose extension or first accuracy case. For an existing case, locate its reference, metrics/threshold evidence, and CI step; reuse them within their validated scope and inspect only the requested change and affected dependencies. Reopen reference research or calibration when that scope changes. For a first case, inspect the model's registry/pipeline, recipe, task/output contract, and native vs Diffusers backend, then choose a reference via reference-contract.md. Select the representative task using its path coverage guidance and record the covered paths, remaining gaps, and any reason for choosing a simpler task. A changed task needs its own aligned reference and gate scope. First accuracy coverage does not imply porting a new model.
  2. Preflight before model loading. Use the preflight entry in the actual reference environment, then check required weights, inputs, and comparison tools. Resolve failures before starting either model/server. Reuse a successful preflight only while its environment and assets match.
  3. Check effective comparison conditions. Use the short comparison checklist for this case's relevant fields. Inspect how arguments reach the pipeline and what settings actually take effect; matching names alone are insufficient. Cover text/vision encoders as well as the denoiser and decoder. Resolve unexplained drift with the diagnostic order before using the result to calibrate a gate. For acceleration, keep unrelated settings fixed against a trusted baseline with the feature disabled. Online/offline agreement proves path consistency.
  4. Implement the modality case. For image generation, editing, or layered outputs, read image-accuracy.md. For T2V/I2V and video with audio, read video-accuracy.md. Follow the model directory rule for tests, reference runners, and model-specific support files. Reuse accuracy helpers and fixtures. Add a shared helper only for a missing reusable operation. Generate the required reference/candidate artifacts within a self-contained test or explicit fixture dependency, without relying on another test running first or an old output directory.
  5. Recommend an evidenced gate. Follow reference-contract.md. Reuse existing thresholds only within their evidenced scope; calibrate a first case or changed numerical contract. Check output structure before metrics, and save both outputs and metric summaries before assertions. Use the shared device threshold resolver for SSIM/PSNR tables and record the resolved profile and bounds. Deliver a threshold recommendation with observed variability and headroom; the user selects new or adjusted thresholds after reviewing that evidence.
  6. Wire execution into CI. Read ci-wiring.md. Match hardware marks, real-checkpoint runtime, explicit file/node selection, source dependencies, tools/reference assets, and artifact retention in the appropriate image/video Accuracy step. Choose nightly or weekly based on the case's resource cost and intended coverage. A marker alone does not make Buildkite execute the test.
  7. Validate, diagnose, and stop. Collect the intended nodes with the exact CI selector, then run the representative comparison on suitable hardware. On failure, follow the diagnostic order before rerunning. Use the completion and rerun rules to finish once evidence is sufficient and to rerun only affected checks. Report passed, failed, skipped, deselected, and unvalidated separately. Without suitable hardware, deliver feasible edits and exact commands with GPU execution/calibration explicitly unvalidated; collection is not a pass.

Read vllm-omni-test and its test routing reference when fixture, marker, runtime, or CI mechanics need more detail. For this accuracy workflow, use the specific Accuracy lane and actual files described here rather than applying a functional *_expansion.py recipe automatically.

Deliver when invoked for a model

  • The minimal accuracy pytest case/reference adapter and CI edits requested.
  • Reference provenance, the aligned request contract, and a threshold recommendation with headroom evidence, including its calibration limits and user-selection status.
  • Why the representative task was selected, which input/output paths it exercises, and which supported paths remain unvalidated.
  • Copy-paste local single-case and CI-equivalent commands, with hardware, checkpoint cache/access, reference environment, and media-tool prerequisites.
  • Locations of reference/candidate outputs, metrics and logs, plus execution results and the precise scope of the accuracy claim. Follow the run comparison rules when settings changed or diagnostic hooks were used.

版本历史

  • 71f0fa4 当前 2026-10-19 11:49

同 Skill 集合

.agents/skills/3d-animation-short-generator/SKILL.md
.agents/skills/brand-promo-video-generator/SKILL.md
.agents/skills/co-op-game-intro-generator/SKILL.md
.agents/skills/h3-prompt-writing/SKILL.md
.agents/skills/handdrawn-live-video-generator/SKILL.md
.agents/skills/minimalist-product-ad-generator/SKILL.md
.agents/skills/music-video-subtitle-generator/SKILL.md
.agents/skills/paper-collage-explainer-generator/SKILL.md
.agents/skills/papercraft-stop-motion-explainer/SKILL.md
.claude/skills/add-diffusion-model/SKILL.md
.claude/skills/add-recipe/SKILL.md
.claude/skills/add-tts-model/SKILL.md
.claude/skills/diffusion-perf-opt/SKILL.md
.claude/skills/diffusion-perf-test/SKILL.md
.claude/skills/find-simplifications/SKILL.md
.claude/skills/precheck-pr/SKILL.md
.claude/skills/quantization/SKILL.md
.claude/skills/vllm-omni-npu-upgrade/SKILL.md
.claude/skills/vllm-omni-test/SKILL.md
.claude/skills/production-add-diffusion-model/SKILL.md
.claude/skills/review-pr/SKILL.md

元信息

文件数
0
版本
7afd00e
Hash
fb21d1e5
收录时间
2026-10-19 11:49

首页 - Wiki
Copyright © 2011-2026 iteam. Current version is 2.155.2. UTC+08:00, 2026-11-04 09:16
浙ICP备14020137号-1