Agent Skills
› uw-syfi/vibesys
› results-against-base
results-against-base
GitHub分析实验结果与基准及历史最佳值的对比，判断变更是否真实有效。用于评估运行表现、比较方案优劣及验证增益可信度，依据统计噪声和判定结论提供决策建议。
Trigger Scenarios
询问当前运行效果如何
比较哪个想法最好
验证增益是否可信
Install
npx skills add uw-syfi/vibesys --skill results-against-base -g -y
SKILL.md
Frontmatter
{
"name": "results-against-base",
"description": "Read a run's results against the base and the best so far, and judge whether a change is real. Use when asked how well the run is doing, which idea is best, or whether a gain can be trusted."
}
Results against the base
The base is the unchanged starting code. Every result is meaningful only against it, so say results as a ratio to the base (for example 1.42x faster) and state the direction: for a metric where lower is better, a ratio above 1 means the candidate is worse.
- The best is the strongest kept result so far. An idea is kept when its result beats the base by more than noise; otherwise it is not better, or it failed.
- A step's value comes from the task's evaluation. Compare steps measured the same way: a value from a different workload or machine is not comparable.
- A small gain inside the run's noise is not evidence. When a step reports a comparison, its
verdict is
better,within_noise,worseorinconclusive(too few runs to estimate the noise: not noise, wait for more runs); trust the verdict over the raw numbers. - Results flow from
get_ideaandlist_steps(value, unit, commit). Do not average across units or invent a baseline the tools did not show.
When the operator asks what to do next, ground it in the pattern of results: what was tried and kept, what repeated failures have in common, what no idea has tried yet.
Version History
- 3a50e8b Current 2026-11-02 13:56
