Agent Skills
› NeverSight/learn-skills.dev
› benchmark-optimization-loop
benchmark-optimization-loop
GitHub用于将性能优化请求转化为可测量的迭代循环。通过设定基线、生成变体、严格验证正确性与安全性，并追踪递归优化结果，确保以最小成本获得最佳实测性能提升。
Trigger Scenarios
用户要求提高系统速度或延迟
需要对比多种实现方案的基准测试
执行递归优化或超参数搜索
评估吞吐量、成本或资源消耗
Install
npx skills add NeverSight/learn-skills.dev --skill benchmark-optimization-loop -g -y
SKILL.md
Frontmatter
{
"name": "benchmark-optimization-loop",
"tools": "Read, Write, Edit, Bash, Grep, Glob",
"origin": "ECC",
"description": "Use when the user asks to make something faster, try many variants, run recursive optimization, benchmark latency\/throughput\/cost, or choose the best implementation by repeated measured tests."
}
Benchmark Optimization Loop
Use this skill to convert "make it 20x faster" or "try 50 recursive optimizations" into a bounded measured loop that can actually improve a system.
Required Baseline
Do not optimize until these exist:
- the operation being optimized;
- the correctness gate that must stay green;
- the metric: wall time, p95 latency, rows/sec, cost/run, memory, error rate;
- the current baseline;
- the search budget: max variants, max time, max spend, max data impact.
If the user asks for an unrealistic target, keep the ambition but make the loop bounded and measurable.
Loop
- Measure the baseline.
- Identify bottlenecks from evidence.
- Generate variants that test one hypothesis each.
- Run variants with the same input shape.
- Reject variants that fail correctness, safety, or reproducibility.
- Promote the fastest safe variant.
- Codify the winning path in a script, command, test, config, or doc.
- Rerun the baseline and winner to confirm the delta.
Variant Table
Track variants like this:
Variant | Hypothesis | Command | Time | Correct? | Notes
baseline | current path | npm run job | 120s | yes | stable
batch-500 | fewer round trips | npm run job -- --batch 500 | 42s | yes | winner
parallel-8 | more workers | npm run job -- --workers 8 | 31s | no | rate limited
Recursive Search
For recursive or hyperparameter work:
- persist every run to a ledger;
- compare against the prior accepted winner, not only the previous run;
- keep a holdout or replay check;
- stop when improvement is within noise, correctness fails, cost exceeds the budget, or the search starts changing more variables than it can explain.
Use phrases like "best measured safe variant" instead of "global optimum" unless the search space was actually exhaustive.
Promotion Gate
A variant cannot become the new default until:
- correctness tests pass;
- the performance delta is repeated or explained;
- rollback is obvious;
- the change is encoded in source control or a durable runbook;
- the final summary includes exact commands and measurements.
Version History
- e0220ca Current 2026-07-05 23:51
