circleci-smarter-testing
GitHub用于配置CircleCI Smarter Testing，生成test-suites.yml并集成至CI。通过严格使用--doctor命令验证YAML、JUnit及测试集错误，支持TIA、动态分片等功能，确保Beta阶段配置正确且合规。
Trigger Scenarios
Install
npx skills add fanfan-de/anybox --skill circleci-smarter-testing -g -y
SKILL.md
Frontmatter
{
"name": "circleci-smarter-testing",
"description": "Onboard onto CircleCI Smarter Testing (testsuite) with `.circleci\/test-suites.yml`, config wiring, `circleci run testsuite`, and `--doctor` until checks pass. Use for Smarter Testing, testsuite, test-suites YAML, test impact analysis, dynamic test splitting, auto rerun failed tests, or migrating raw test commands."
}
CircleCI Smarter Testing
Overview
Beta (Discuss). Create or adjust .circleci/test-suites.yml, wire it into CI, and validate with --doctor until checks pass. Let doctor diagnose YAML, command, JUnit, and testsuite errors before adding extra guidance.
Doctor command (required)
The testsuite CLI uses Bubble Tea and must run with a TTY. When iterating with --doctor, the agent must always run this exact command (substitute the suite name only):
script -q /dev/null circleci run testsuite "<suite name>" --doctor
Do not modify this command: no pipes, tail, redirects, backgrounding, or truncation.
Scope: testsuite setup and validation. Not legacy circleci tests run / timings-only JUnit tuning → circleci-config (test-results-and-splitting.md). Primary doc: Getting started.
Lazy references: read only what the task needs—do not load every reference file up front.
- Runner YAML → references/runners.md (matching section only)
- CI /
store_test_results→ references/ci-and-junit.md - TIA, splitting, auto-rerun → references/optional-features.md
Inputs To Gather
- Runner, test root, existing
.circleci/config.yml - Optional: TIA, dynamic splitting, auto-rerun limits
- Hand off:
circleci-cli(install/auth/plugin) |circleci-config(JUnit/circleci tests runwithout testsuite) |circleci-builds(CI fails after doctor-clean config)
Workflow
- Inspect — runner, layout, existing CI; reuse documented test commands; run testsuite from the test root (no
cdin YAML). - Baseline —
.circleci/test-suites.ymlwithname,discover,run,outputs.junit(see runners.md). - Doctor — run the exact command above; apply its guidance; repeat until pass.
- Local vs CI — Local needs CLI + testsuite plugin (
brew install circleci/tap/circleci-testsuiteon macOS; binaries on Linux/WSL). CI usescircleci run testsuiteon executors + ci-and-junit.md. - Optional features — only if asked; see optional-features.md; doctor after each change.
- Verify — full
circleci run testsuite "<suite name>"; optional TIA/rerun checks per docs.
Guardrails
- Beta unless confirmed otherwise; never skip doctor after editing
test-suites.yml. - No secrets in YAML—use contexts/env vars; document variable names only.
- No local impact JSON in
.circleci/; smallest change that passes doctor. - Do not swap a working testsuite for legacy
circleci tests split/circleci tests rununless the user requests migration.
Reference map
- runners.md —
test-suites.ymltemplate, Vitest, pytest, pointers for other runners - ci-and-junit.md — full
config.yml, JUnit directory upload - optional-features.md — TIA, splitting, auto-rerun
- test-results-and-splitting.md — classic test metadata path
Output Contract
Provide:
- Runner, working directory, files changed, commands run (
--doctorand follow-ups). - What to commit and open items (beta access, blocked prerequisites).
Version History
- 08dc189 Current 2026-07-05 18:56
