Agent Skills › Owl-Listener/designer-skills › a-b-test-design

a-b-test-design

GitHub

指导设计严谨的A/B测试,涵盖假设构建、变量控制、核心指标选择及样本量计算。确保实验具备统计学效力,规避常见陷阱如提前窥探结果或样本不足,适用于可量化评估的大规模变更场景。

prototyping-testing/skills/a-b-test-design/SKILL.md Owl-Listener/designer-skills

触发场景

需要设计A/B测试方案 确定实验假设和核心指标 计算统计显著性所需的样本量

安装

npx skills add Owl-Listener/designer-skills --skill a-b-test-design -g -y
更多选项

非标准路径

npx skills add https://github.com/Owl-Listener/designer-skills/tree/main/prototyping-testing/skills/a-b-test-design -g -y

不安装直接使用

npx skills use Owl-Listener/designer-skills@a-b-test-design

指定 Agent (Claude Code)

npx skills add Owl-Listener/designer-skills --skill a-b-test-design -a claude-code -g -y

安装 repo 全部 skill

npx skills add Owl-Listener/designer-skills --all -g -y

预览 repo 内 skill

npx skills add Owl-Listener/designer-skills --list

SKILL.md

Frontmatter
{
    "name": "a-b-test-design",
    "description": "Design an A\/B experiment — hypothesis, variants, primary metric, and sample size. Use when a change can be measured quantitatively at scale. For observing behaviour qualitatively, use `test-scenario`."
}

A/B Test Design

You are an expert in designing rigorous A/B experiments that produce actionable results.

What You Do

You design A/B tests with clear hypotheses, controlled variants, appropriate metrics, and statistical rigor.

Test Structure

1. Hypothesis

Structured as: 'If we [change], then [outcome] will [improve/decrease] because [rationale].'

2. Variants

  • Control (A): current design
  • Treatment (B): proposed change
  • Keep changes isolated — test one variable at a time

3. Primary Metric

The single most important measure of success. Must be measurable, relevant, and sensitive to the change.

4. Secondary Metrics

Supporting measures and guardrail metrics to detect unintended consequences.

5. Sample Size

Based on: minimum detectable effect, baseline conversion rate, statistical significance level (typically 95%), and power (typically 80%).

6. Duration

Run until sample size is reached. Account for weekly cycles (run in full weeks). Minimum 1-2 weeks typically.

Common Pitfalls

  • Peeking at results before completion
  • Too many variants at once
  • Metric not sensitive enough to detect change
  • Sample size too small
  • Not accounting for novelty effects
  • Ignoring segmentation effects

When Not to A/B Test

  • Very low traffic (insufficient sample)
  • Ethical concerns with withholding improvement
  • Foundational changes that affect everything
  • When qualitative insight is more valuable

Best Practices

  • One hypothesis per test
  • Document everything before starting
  • Don't stop early on positive results
  • Analyze segments after overall results
  • Share learnings broadly regardless of outcome

版本历史

  • 20e34c4 当前 2026-08-19 23:22

    优化技能描述,增加使用触发条件和边界说明,解决与其他技能的重叠问题,确保描述包含'Use when'句式并限制长度。

  • acc3e57 2026-07-25 05:16

同 Skill 集合

design-ops/skills/design-critique/SKILL.md
design-ops/skills/design-debt-audit/SKILL.md
design-ops/skills/design-impact-reporting/SKILL.md
design-ops/skills/design-qa-checklist/SKILL.md
design-ops/skills/design-review-process/SKILL.md
design-ops/skills/design-sprint-plan/SKILL.md
design-ops/skills/handoff-spec/SKILL.md
design-ops/skills/team-workflow/SKILL.md
design-ops/skills/version-control-strategy/SKILL.md
design-research/skills/affinity-diagram/SKILL.md
design-research/skills/behavioural-analytics/SKILL.md
design-research/skills/card-sort-analysis/SKILL.md
design-research/skills/diary-study-plan/SKILL.md
design-research/skills/empathy-map/SKILL.md
design-research/skills/interview-script/SKILL.md
design-research/skills/jobs-to-be-done/SKILL.md
design-research/skills/journey-map/SKILL.md
design-research/skills/qual-quant-triangulation/SKILL.md
design-research/skills/research-repository/SKILL.md
design-research/skills/summarize-interview/SKILL.md
design-research/skills/survey-design/SKILL.md
design-research/skills/usability-test-plan/SKILL.md
design-research/skills/user-persona/SKILL.md
design-systems/skills/accessibility-audit/SKILL.md
design-systems/skills/component-spec/SKILL.md
design-systems/skills/design-system-governance/SKILL.md
design-systems/skills/design-token/SKILL.md
design-systems/skills/documentation-template/SKILL.md
design-systems/skills/icon-system/SKILL.md
design-systems/skills/localization-design/SKILL.md
design-systems/skills/motion-system/SKILL.md
design-systems/skills/naming-convention/SKILL.md
design-systems/skills/pattern-library/SKILL.md
design-systems/skills/theming-system/SKILL.md
designer-toolkit/skills/case-study/SKILL.md
designer-toolkit/skills/design-negotiation/SKILL.md
designer-toolkit/skills/design-rationale/SKILL.md
designer-toolkit/skills/design-system-adoption/SKILL.md
designer-toolkit/skills/design-token-audit/SKILL.md
designer-toolkit/skills/presentation-deck/SKILL.md
designer-toolkit/skills/ux-writing/SKILL.md
interaction-design/skills/animation-principles/SKILL.md
interaction-design/skills/conversational-ux/SKILL.md
interaction-design/skills/doherty-threshold/SKILL.md
interaction-design/skills/error-handling-ux/SKILL.md
interaction-design/skills/feedback-patterns/SKILL.md
interaction-design/skills/fitts-law/SKILL.md
interaction-design/skills/form-design/SKILL.md
interaction-design/skills/gesture-patterns/SKILL.md
interaction-design/skills/hicks-law/SKILL.md

元信息

文件数
0
版本
9a6930c
Hash
2ca225aa
收录时间
2026-07-25 05:16

首页 - Wiki
Copyright © 2011-2026 iteam. Current version is 2.156.0. UTC+08:00, 2026-11-20 03:46
浙ICP备14020137号-1