Agent Skills
› openai/plugins
› metric-pack-designer
metric-pack-designer
GitHub指导用户为plugin-eval设计自定义评估指标包，包括定义检查项、生成清单及脚本，并运行分析以验证本地评估标准。
Trigger Scenarios
需要扩展plugin-eval的本地评估标准
希望添加自定义评估规则或可视化指标
Install
npx skills add openai/plugins --skill metric-pack-designer -g -y
SKILL.md
Frontmatter
{
"name": "metric-pack-designer",
"description": "Design custom metric packs for plugin-eval so teams can add local evaluation rubrics that emit schema-compatible checks and metrics. Use when the user wants their own evaluation criteria or visualizations."
}
Metric Pack Designer
Use this skill when the user wants to extend plugin-eval with a local rubric.
Workflow
- Clarify the custom rubric categories and target kinds.
- Define the smallest useful
checks[]andmetrics[]payload. - Create a metric-pack manifest plus a script that prints JSON to stdout.
- Run the pack through
plugin-eval analyze <path> --metric-pack <manifest.json>.
Design Rules
- Keep IDs stable across runs so comparisons stay meaningful.
- Emit only
checks[],metrics[], and optionalartifacts[]. - Do not try to overwrite the core score or summary.
- Prefer deterministic local signals over subjective text generation.
Reference
../../references/metric-pack-manifest.md
Version History
- 11c74d6 Current 2026-07-19 09:43
