Auto Review

Run the bundled structured review helper as a closeout check. This is code review, not Guardian auto_review approval routing.

Codex review is the default when no engine is set. It uses gpt-5.5 by default, usually delivers the best review results, and should remain the normal final closeout engine. Claude review is optional and uses claude-fable-5 by default.

Use when:

• user asks for Codex review / Claude review / Pi review / Droid review / OpenCode review / autoreview / second-model review
• after non-trivial code edits, before final/commit/ship
• reviewing a local branch or PR branch after fixes

Contract

• Treat review output as advisory. Never blindly apply it.
• Verify every finding by reading the real code path and adjacent files.
• Read dependency docs/source/types when the finding depends on external behavior.
• Reject unrealistic edge cases, speculative risks, broad rewrites, and fixes that over-complicate the codebase.
• Prefer small fixes at the right ownership boundary; no refactor unless it clearly improves the bug class.
• When an accepted finding shows a bug class or repeated pattern, inspect the current PR scope for sibling instances before fixing.
• Fix the scoped bug class at once when practical; stop at touched surfaces, owner boundaries, and clear follow-up territory.
• Keep going until structured review returns no accepted/actionable findings only while the work remains inside the original task scope.
• If a review-triggered fix changes code, rerun focused tests and rerun the structured review helper.
• For security-audit suppression changes, verify accepted findings remain auditable: suppressed findings stay in structured output, active output keeps an unsuppressible suppression notice, and aggregate findings cannot hide unrelated active risk.
• Never switch or override the requested review engine/model. If the review hits model capacity, retry the same command a few times with the same engine/model.
• Be patient with large bundles. Structured review can take up to 30 minutes while the model call is active, especially with Codex tools or web search.
• Treat heartbeat lines like review still running: ... elapsed=... pid=... as healthy progress, not a hang. Let the helper continue while heartbeats are advancing. Pass --stream-engine-output when live engine text is useful; Codex and Claude filter tool/file chatter, other engines pass raw output through.
• Do not kill a review just because it has been quiet for 2-5 minutes, or because it is still running under the 30-minute window. Inspect the process only after missing multiple expected heartbeats, after 30 minutes, or after an obviously failed subprocess; prefer letting the same helper command finish.
• Tools are useful in review mode. The helper allows read-only inspection tools and web search by default so reviewers can check dependency contracts, upstream docs, and current behavior.
• Security perspective is always included, but it should not cripple legitimate functionality. Report security findings only when the change creates a concrete, actionable risk or removes an important safety check.
• For regression provenance, keep roles separate: blamed code author, blamed PR author, PR merger/committer, current PR author, and PR/date. If no blamed PR is traceable, use the blamed commit as the provenance: commit SHA, date, and author username. Do not guess a merger or frame missing PR metadata as a separate finding.
• If the blamed PR was merged by clawsweeper[bot] or another automation, identify the human trigger when practical. Check timeline/comments first; if rate-limited, use gitcrawl/cache or public PR HTML. Look for maintainer commands such as @clawsweeper automerge, /landpr, or labels/status comments that armed automerge. Report automerge triggered by @login; if not found, say trigger unknown.
• Do not invoke built-in codex review, nested reviewers, or reviewer panels from inside the review. The helper builds one bundle, calls one selected engine, validates one structured result, and stops.
• Stop as soon as the helper exits 0 with no accepted/actionable findings. Do not run an extra review just to get a nicer "clean" line, a second opinion, or clearer closeout wording.
• Treat the helper's successful exit plus absence of actionable findings as the clean review result, even if the underlying Codex CLI output is terse.
• Multi-reviewer panels are opt-in only. Use them when explicitly requested or when risk justifies the extra spend; the main agent still verifies every accepted finding before fixing.
• If rejecting a finding as intentional/not worth fixing, add a brief inline code comment only when it explains a real invariant or ownership decision that future reviewers should know.
• If gh/Gitcrawl reports database disk image is malformed, run gitcrawl doctor --json once to let the portable cache repair before retrying review; do not bypass the shim unless repair fails and freshness requires live GitHub.
• If Gitcrawl reports a portable manifest mismatch, source/runtime DB health error, or stale portable-store checkout, run gitcrawl doctor --json and inspect source_db_health, runtime_db_health, and portable_store_status before falling back to live GitHub.
• Do not push just to review. Push only when the user requested push/ship/PR update.

Scope Governor

Autoreview is a closeout gate, not permission to rewrite the task.

Before the first review, freeze a scope baseline: original request or issue, target branch, intended behavior, owner boundary, changed files, and non-test LOC. For inherited or already-bloated branches, use the intended PR diff as the baseline rather than accepting all existing branch drift.

Before patching a finding, classify it:

• In-scope blocker: the finding is introduced by the current diff, affects the same owner boundary, and can be fixed without changing the task's contract.
• Follow-up: the finding is real but belongs to an adjacent bug class, sibling surface, cleanup, or broader hardening track.
• Stop-and-escalate: the finding requires a new protocol/config/storage/public API contract, a different owner boundary, a release-process change, or a design choice outside the original request.

Stop patching and report the scope break instead of continuing when:

• a narrow PR turns into an architecture change, protocol change, migration, or release-process change;
• the diff grows past 2x the original files or non-test LOC without explicit approval to expand scope;
• two review-triggered patch cycles have not converged; pause and reclassify every remaining finding before another edit;
• the best fix is "define the canonical contract first" rather than another local inference layer;
• fixing the accepted finding would make the PR no longer describe the same behavior, issue, or owner boundary.

After the two-cycle pause, continue only when every remaining accepted finding is still an in-scope blocker. Otherwise preserve the useful analysis, identify the smallest safe landed subset if one exists, and open or request a follow-up for the larger fix. Do not keep committing speculative fixes just to satisfy the reviewer.

Do not stack or push review-triggered fix commits while scope classification or focused proof is unresolved. Keep exploratory edits local until the cycle is proven in scope; if scope breaks, remove them from the landing lane instead of preserving them as branch history.

Critical exceptions must be explicit: active data loss, crash, broken install/upgrade, release blocker, or concrete security exposure. If the exception is not one of those, it is not critical enough to blow up scope.

Release Branches And Release Process

On release, beta, stable, hotfix, signing, notarization, appcast, package-publish, or release-check work, use freeze discipline even when the branch name is not release-like:

• Fix only release blockers, failed release infrastructure, exact backports, install/upgrade breakage, data loss, crashes, or concrete security exposure.
• Treat non-blocking autoreview findings as follow-ups for main, not reasons to broaden the release branch.
• Do not introduce new product behavior, config surface, protocol shape, migration, plugin ownership, docs narrative, or process policy unless it directly unblocks the release.
• Keep proof tied to the release target: exact branch/ref, failing check or shipped-risk reason, smallest command/proof, and whether the fix must also forward-port to main.
• If review discovers a real but non-critical design problem during release closeout, stop with a follow-up issue/PR plan; do not use the release branch as the refactor lane.

Skill Path (set once)

Set the skill script paths once, then use "$AUTOREVIEW" and "$AUTOREVIEW_HARNESS" in the examples below.

Choose one:

# Project-local skill in the current repo:
export AUTOREVIEW=".agents/skills/autoreview/scripts/autoreview"
export AUTOREVIEW_HARNESS=".agents/skills/autoreview/scripts/test-review-harness"

# Source checkout of openclaw/agent-skills:
export AUTOREVIEW="skills/autoreview/scripts/autoreview"
export AUTOREVIEW_HARNESS="skills/autoreview/scripts/test-review-harness"

# Global skill:
export AGENTS_HOME="${AGENTS_HOME:-$HOME/.agents}"
export AUTOREVIEW="$AGENTS_HOME/skills/autoreview/scripts/autoreview"
export AUTOREVIEW_HARNESS="$AGENTS_HOME/skills/autoreview/scripts/test-review-harness"

When using Claude Code, set AGENTS_HOME="$HOME/.claude" for global skills. Project-local skills live under .claude/skills/ in the current repo.

Pick Target

Dirty local work:

"$AUTOREVIEW" --mode local

Use this only when the patch is actually unstaged/staged/untracked in the current checkout. --mode uncommitted is accepted as an alias for --mode local. For committed, pushed, or PR work, point the helper at the commit or branch diff instead; do not force dirty modes just because the helper docs mention dirty work first. A clean local review only proves there is no local patch.

Branch/PR work:

"$AUTOREVIEW" --mode branch --base origin/main

Optional review context is first-class. Prompt files and datasets must be repo-relative so review bundles cannot pull arbitrary host files:

"$AUTOREVIEW" --mode branch --base origin/main --prompt-file review-notes.md --dataset evidence.json

If an open PR exists, use its actual base:

base=$(gh pr view --json baseRefName --jq .baseRefName)
"$AUTOREVIEW" --mode branch --base "origin/$base"

Committed single change:

"$AUTOREVIEW" --mode commit --commit HEAD

Use commit review for already-landed or already-pushed work on main. Reviewing clean main against origin/main is usually an empty diff after push. For a small stack, review each commit explicitly or review the branch before merging with --base.

Parallel Closeout

Format first if formatting can change line locations. Then it is OK to run tests and review in parallel:

"$AUTOREVIEW" --parallel-tests "<focused test command>"

On Windows, the default --parallel-tests shell preserves the platform cmd.exe semantics used by Python shell=True. Use --parallel-tests-shell powershell or --parallel-tests-shell pwsh when the focused test command is PowerShell-specific.

Tradeoff: tests may force code changes that stale the review. If tests or review lead to code edits, rerun the affected tests and rerun review until no accepted/actionable findings remain. Once that rerun exits cleanly, stop; do not spend another long review cycle on redundant confirmation.

Review Panels

Run multiple reviewers against one frozen bundle:

"$AUTOREVIEW" --reviewers codex,claude,pi,droid

--panel is shorthand for Codex plus Claude unless --engine changes the first reviewer:

"$AUTOREVIEW" --panel

Set reviewer models and thinking/effort explicitly:

"$AUTOREVIEW" --reviewers codex,claude --model codex=gpt-5.5 --thinking codex=high --model claude=claude-fable-5 --thinking claude=max

Inline syntax is also supported for simple model IDs:

"$AUTOREVIEW" --reviewers codex:gpt-5.5:high,claude:claude-fable-5:max

For models with slashes or extra colons, prefer keyed form:

"$AUTOREVIEW" --engine pi --model anthropic/claude-sonnet-4 --thinking high
"$AUTOREVIEW" --engine opencode --model opencode/north-mini-code-free --thinking high
"$AUTOREVIEW" --engine droid --model claude-opus-4-8 --thinking low
"$AUTOREVIEW" --reviewers codex,pi --model codex=gpt-5.5 --model pi=anthropic/claude-sonnet-4
"$AUTOREVIEW" --reviewers codex,opencode --model codex=gpt-5.5 --model opencode=opencode/north-mini-code-free
"$AUTOREVIEW" --reviewers codex,droid --model codex=gpt-5.5 --model droid=claude-opus-4-8

Models and thinking

The helper accepts --model globally or per engine (engine=model) and --thinking globally or per engine (engine=level). Repeat either flag for multiple reviewers.

Recommended model defaults:

CLI flags and environment variables override these defaults. Droid, Copilot, Pi, and OpenCode do not get built-in model defaults here because their provider catalogs are external to the Codex/Claude closeout path and may vary by installation.

Claude also supports --fallback-model a,b for availability-based fallback chains (model-config). Current Claude docs note that auth, billing, rate-limit, request-size, and transport errors do not trigger fallback, and the changelog documents interactive-session support in v2.1.166.

Examples matching current main behavior:

# Codex with explicit model and reasoning
"$AUTOREVIEW" --engine codex --model gpt-5.5 --thinking high

# Claude Code aliases or full model names, with optional availability fallback
"$AUTOREVIEW" --engine claude --model claude-fable-5 --thinking max
"$AUTOREVIEW" --engine claude --model claude-fable-5 --fallback-model claude-opus-4-8,claude-sonnet-4-6

# Factory Droid with explicit model and reasoning effort
"$AUTOREVIEW" --engine droid --model claude-opus-4-8 --thinking low

# GitHub Copilot (model only; no thinking knob)
"$AUTOREVIEW" --engine copilot --model gpt-5.2

# Pi with explicit model and thinking level
"$AUTOREVIEW" --engine pi --model anthropic/claude-sonnet-4 --thinking high --pi-bin pi

# OpenCode with explicit provider/model and variant
"$AUTOREVIEW" --engine opencode --model opencode/north-mini-code-free --thinking high

Environment defaults

CLI flags take precedence over environment variables.

Codex maps thinking to model_reasoning_effort. Claude maps thinking to --effort. Droid maps thinking to -r, --reasoning-effort. Pi maps thinking to --thinking. OpenCode maps thinking to --variant. Copilot rejects --thinking. Only Claude accepts --fallback-model; global CLI/env fallback requires at least one Claude reviewer, and engine-specific fallback overrides require that reviewer to be selected. Non-Claude fallback overrides, including AUTOREVIEW_<NONCLAUDE>_FALLBACK_MODEL, fail closed instead of being silently ignored.

Review engine isolation

When autoreview runs inside the repository under review, external reviewer CLIs must not load project-local trust or configuration that the branch controls.

Codex --ignore-user-config skips config loading for the exec run. Autoreview reconstructs only the documented cli_auth_credentials_store, forced_login_method, and forced_chatgpt_workspace_id settings from CODEX_HOME/config.toml, keeping authentication and workspace restrictions usable without forwarding unrelated user configuration. The explicit repo trust override and zero project-doc budget keep reviewed-repo AGENTS.md and .codex/ trust surfaces out of the review prompt. --ignore-rules skips user/project execpolicy rules. Claude --safe-mode disables project hooks, skills, plugins, MCP servers, and CLAUDE.md while preserving normal authentication, model selection, built-in tools, and permissions; managed settings policy can still apply. --setting-sources user avoids project/local settings from the reviewed checkout, and current Claude Code docs note the project-skill blocking behavior was fixed in v2.1.69. --strict-mcp-config and --disallowedTools mcp__* keep MCP unavailable to the review run. --bare is not used here because Claude's headless docs say it skips OAuth and keychain reads. Pi --no-approve ignores project-local files for one run; the helper requires Pi v0.79.0+ plus help output that advertises every required isolation flag because older legacy binaries can ignore unknown flags. The current package is @earendil-works/pi-coding-agent; deprecated @mariozechner/pi-coding-agent 0.73.x is intentionally rejected. Pi version/help probes and the review command run from neutral temporary directories, not the reviewed repo. Pi --no-context-files removes AGENTS.md/CLAUDE.md, the resource-disable flags keep .pi extensions, skills, prompts, and themes out of the run, --no-session avoids writing review sessions, and the read-only allowlist omits bash, edit, and write. OpenCode starts from a neutral temporary directory, points at the reviewed repo with --dir, disables project config through OPENCODE_DISABLE_PROJECT_CONFIG=1, and injects OPENCODE_CONFIG_CONTENT; permissions default to deny, allow read/grep/glob, preserve OpenCode's .env ask rules, and gate websearch/webfetch with --no-web-search. The injected config also clears command/instruction/plugin arrays and disables write/edit/bash/task/skill/todowrite tools without changing user auth storage. The helper sends the review prompt over stdin rather than argv and extracts the final structured JSON from type: "text" events. OpenCode rejects --no-tools.

Context Efficiency

Run the helper directly so target selection, engine choice, structured validation, and exit status all stay in one path. If output is noisy, summarize the completed helper output after it returns; do not ask another agent or reviewer to rerun the review.

Helper

After setting AUTOREVIEW and AUTOREVIEW_HARNESS above:

"$AUTOREVIEW" --help

The smoke harness has thin shell wrappers over a shared Python implementation:

"$AUTOREVIEW_HARNESS" --fixture benign --engine codex

On native Windows, invoke the extensionless Python helper through Python:

python skills\autoreview\scripts\autoreview --help

and the smoke harness:

skills\autoreview\scripts\test-review-harness.ps1 -Fixture benign -Engine codex

The helper:

• chooses dirty local changes first
• accepts --mode uncommitted as an alias for --mode local
• otherwise uses current PR base if gh pr view works
• otherwise uses origin/main for non-main branches
• does not fetch automatically during branch review; the selected base ref must already resolve locally
• supports --engine codex, claude, droid, copilot, pi, and opencode; default is AUTOREVIEW_ENGINE or codex; Codex should remain the default when nothing is set
• resolves bare git, gh, reviewer, and PowerShell shell commands from absolute PATH entries only, never from the reviewed checkout; explicit relative --*-bin paths are resolved from the reviewed repository root
• use --mode commit --commit <ref> for already-committed work, especially clean main after landing
• should be left in --mode auto or forced to --mode branch for PR/branch work; do not force --mode local after committing
• writes only to stdout unless --output, --json-output, or live streamed engine stderr is set
• supports --dry-run, --parallel-tests, --parallel-tests-shell, --prompt, repo-relative --prompt-file, repo-relative --dataset, --no-tools, --no-web-search, and commit refs
• supports --stream-engine-output or AUTOREVIEW_STREAM_ENGINE_OUTPUT=1 for live engine text while preserving structured validation; Codex and Claude hide tool/file event details, emit compact activity summaries, and report usage at turn completion
• supports opt-in review panels with --panel / --reviewers, plus per-engine --model, --thinking, and Claude --fallback-model
• uses built-in model defaults codex=gpt-5.5 and claude=claude-fable-5; honors AUTOREVIEW_MODEL, AUTOREVIEW_THINKING, AUTOREVIEW_FALLBACK_MODEL, and per-engine AUTOREVIEW_<ENGINE>_MODEL / AUTOREVIEW_<ENGINE>_THINKING environment overrides when CLI flags are omitted
• allows read-only tools and web search by default where the selected CLI supports them; forbids nested review in the prompt; Codex is run through codex exec with auth-only user settings, read-only sandbox, reviewed-repo instruction/config/rule isolation flags, and structured output
• runs Claude with --safe-mode (v2.1.169+), --setting-sources user, MCP disabled, explicit allowed tools, and --fallback-model when set, so reviewed-repo hooks/skills/MCP do not affect the review run while normal auth still works; managed settings policy can still apply
• runs Droid with droid exec in read-only mode, forwards --model and -r, --reasoning-effort, and switches --output-format to stream-json when streaming is enabled
• runs Pi v0.79.0+ from neutral temporary directories with --no-approve, --no-session, disabled Pi context/resource loading, and built-in read-only tools (read,grep,find,ls) when tools are enabled
• runs OpenCode with opencode run --dir <repo> --pure --format json from a neutral temporary directory, forwards --model and --variant, injects deny-by-default permissions, disables project config loading, and passes the review prompt over stdin
• prints review still running: <engine> elapsed=<seconds>s pid=<pid> to stderr at long-running intervals while waiting for the selected review engine, unless streamed output or compact Codex activity has been visible recently
• prints autoreview clean: no accepted/actionable findings reported when the selected review command exits 0
• exits nonzero when accepted/actionable findings are present

Final Report

Include:

• review command used
• tests/proof run
• findings accepted/rejected, briefly why
• the clean review result from the final helper/review run, or why a remaining finding was consciously rejected

Do not run another review solely to improve the final report wording. If the final helper run exited 0 and produced no accepted/actionable findings, report that exact run as clean.

Crabbox

Use Crabbox when a project needs remote proof, larger cloud capacity, a fresh PR checkout, a reusable warmed box, GitHub Actions-style setup, durable run logs/results, UI proof artifacts, or sync from a dirty local checkout.

Source Of Truth

• Run Crabbox from the repository root; sync mirrors the current checkout.
• Treat repo-local crabbox.yaml or .crabbox.yaml as executable project automation. Review it before remote runs, especially provider, actions, jobs, profiles, env.allow, artifacts, and cleanup policy.
• Verify the installed binary before relying on examples: command -v crabbox && crabbox --version && crabbox --help | sed -n '1,120p'.
• Use crabbox providers or crabbox providers --json for the current provider/capability matrix; provider docs can lag the compiled binary.
• Use crabbox doctor for live readiness checks and crabbox config show to inspect merged config without printing secrets.
• Prefer local targeted tests for tight edit loops. Move to Crabbox for broad suites, package-heavy checks, Docker/E2E/live-provider proof, cross-OS proof, UI proof, or commands that bog down the local machine.

Auth And Config

Brokered operation needs a coordinator URL and token. First login usually needs an explicit broker URL:

crabbox login --url <broker-url>
crabbox whoami
crabbox doctor

After broker.url is configured, crabbox login can reuse it. Trusted operator automation can store a shared token without putting it on argv:

printf '%s' "$CRABBOX_COORDINATOR_TOKEN" |
  crabbox login --url <broker-url> --provider aws --token-stdin

Config precedence is flags > env > repo config > user config > defaults. Default user config is ~/Library/Application Support/crabbox/config.yaml on macOS, ~/.config/crabbox/config.yaml on Linux, or $XDG_CONFIG_HOME/crabbox/config.yaml when set. crabbox config path prints the active user config path.

Keep provider and broker tokens out of repo config and command arguments. Use environment variables, a credential store, coordinator-managed secrets, or a short-lived token command.

Choose The Remote Surface

• crabbox run -- <command>: one command on a fresh or reused box.
• crabbox warmup: create a reusable lease and run commands later with --id.
• crabbox prewarm: warm a reusable lease and hydrate it from configured GitHub Actions.
• crabbox job run <name>: use a repo-local named flow that expands to warmup, optional hydration, run, and stop.
• crabbox run --pool <key>: borrow a hydrated broker ready-pool lease, run, then return/drain/release it according to --pool-return.
• crabbox run --fresh-pr ...: ignore local sync and check out a GitHub PR on the remote; add --apply-local-patch to test local uncommitted changes on top of that PR.
• crabbox run --provider ssh: use an existing macOS, Linux, or Windows host.
• crabbox warmup --desktop --browser: provision a visible desktop/browser for UI testing, WebVNC, screenshots, and artifacts.

If remote proof is blocked, name the missing capability precisely: auth, coordinator, capacity, provider support, target OS, hydration, secret access, artifact storage, desktop support, or a delegated-provider limitation.

Common Remote Proof

One-shot command:

crabbox run --preflight --timing-json -- pnpm test

Warm and reuse a lease:

crabbox warmup --class beast --idle-timeout 90m
crabbox status --id <cbx_id-or-slug> --wait
crabbox run --id <cbx_id-or-slug> -- pnpm test:changed
crabbox run --id <cbx_id-or-slug> --full-resync -- pnpm test:changed
crabbox stop <cbx_id-or-slug>

Use a repo-local job when configured:

crabbox job list
crabbox job run --dry-run <job-name>
crabbox job run <job-name>
crabbox job run --id <cbx_id-or-slug> <job-name>

Use a ready-pool lease when the coordinator has hydrated pool capacity:

crabbox pool ready
crabbox run --pool <pool-key> -- pnpm test
crabbox run --pool <pool-key> --pool-return drain -- pnpm test:flaky

Use GitHub Actions hydration when the repository already owns setup in CI:

crabbox warmup --idle-timeout 90m
crabbox actions hydrate --id <cbx_id-or-slug>
crabbox run --id <cbx_id-or-slug> -- pnpm test

Use --github-runner only when the workflow needs full GitHub Actions semantics such as repository secrets, OIDC, service containers, job containers, or unsupported uses: steps:

crabbox actions hydrate --github-runner --id <cbx_id-or-slug>

Sync And Fresh Checkouts

Normal sync transfers tracked files plus non-ignored untracked files, excludes ignored dependency/build/cache output, honors .crabboxignore and sync.exclude, seeds the remote checkout from origin when possible, and skips rsync when the sync fingerprint matches.

Use crabbox sync-plan before large runs. Unexpected counts usually mean local generated churn; update .crabboxignore or sync.exclude instead of forcing huge uploads.

crabbox sync-plan
crabbox run --debug --timing-json -- pnpm test
crabbox run --full-resync -- pnpm test

Use fresh PR checkout when local dependency churn or dirty sync would confuse the result:

crabbox run --fresh-pr example-org/my-app#123 --script ./scripts/e2e-smoke.sh
crabbox run --fresh-pr 123 --apply-local-patch -- pnpm test

--fresh-pr accepts owner/repo#number, GitHub PR URLs, or a numeric PR from the current GitHub origin. Non-GitHub hosts are rejected. Fresh PR checkout is an SSH-run sync feature; delegated providers reject it. Native Windows SSH targets are supported.

When a warm lease smells stale, prefer --full-resync (alias --fresh-sync) to reset the remote workdir, skip the sync fingerprint fast path, reseed Git when possible, and upload the checkout from scratch.

Scripts, Shells, And Windows Targets

Use plain argv after -- for one executable. Use --shell for multi-statement shell snippets, pipes, or shell expansion:

crabbox run --id <lease> -- go test ./...
crabbox run --id <lease> --shell 'corepack enable && pnpm install --frozen-lockfile && pnpm test'

Prefer uploaded scripts for multi-line commands. Scripts are included in failure bundles and avoid brittle quoted shell strings:

crabbox run --script ./scripts/e2e-smoke.sh --timing-json
printf '%s\n' 'echo CRABBOX_PHASE:test' 'pnpm test' | crabbox run --script-stdin

Native Windows targets use PowerShell and tar-based manifest sync. Prefer plain argv for one executable such as dotnet test; use --shell for multi-statement PowerShell and --script <file.ps1> for longer scripts.

Secrets And Environment Forwarding

Crabbox does not forward the whole local environment. Forwarding is name-based: only allowlisted names that are actually set locally or in an allowed profile cross the boundary. Avoid allowlisting secret-shaped names unless the run is an explicit live-secret smoke.

crabbox run --allow-env CI,NODE_OPTIONS -- pnpm test
crabbox run \
  --env-from-profile ~/.project-live.profile \
  --allow-env API_TOKEN \
  --preflight \
  --script ./scripts/live-smoke.sh

--env-from-profile parses simple export NAME=value and NAME=value lines without executing the profile. Crabbox prints redacted presence/length metadata, not values. POSIX SSH leases can persist a helper for later commands on a lease you control:

crabbox run \
  --id <lease> \
  --env-from-profile ~/.project-live.profile \
  --allow-env API_TOKEN \
  --env-helper live \
  -- true
crabbox run --id <lease> -- ./.crabbox/env/live ./scripts/live-smoke.sh

The generated helper and matching secret profile remain in the remote workdir until cleanup, lease reset, or --full-resync; do not persist helpers on shared or untrusted leases.

Profiles, Presets, Proof, And Results

Repo config can define profiles, presets, doctor requirements, artifact globs, required artifacts, and proof templates. Use them for stable validation lanes instead of encoding project knowledge in agent prompts.

crabbox run \
  --profile live-qa \
  --preset qa-live \
  --scenario login-regression \
  --emit-proof /tmp/proof.md \
  --stop-after success

Use --preflight for a target capability snapshot before the command, not as an installer. Use --preflight-tools to tune probes:

crabbox run --preflight --preflight-tools node,bun,docker -- bun test
crabbox run --preflight --preflight-tools default,uv -- node --test

Attach structured results and proof artifacts when the command emits them:

crabbox run --junit reports/junit.xml -- ./scripts/test-with-junit.sh
crabbox run --artifact-glob 'reports/**' --require-artifact reports/summary.json -- pnpm test:e2e
crabbox run --download reports/summary.json=.crabbox/logs/summary.json -- pnpm test:e2e

--require-artifact fails the run if the remote command exits 0 but the proof file is missing. Keep required artifacts bounded and scrubbed; do not collect raw datasets, secrets, credentials, signed URLs, or unredacted customer rows.

Run Handles And Observability

Coordinator-backed runs print a durable run_... handle before leasing starts. Keep that run ID in status updates and PR notes.

crabbox history --limit 20
crabbox history --lease <cbx_id-or-slug> --limit 20
crabbox attach <run_id>
crabbox attach <run_id> --after <seq>
crabbox events <run_id> --after <seq> --limit 100
crabbox events <run_id> --json
crabbox logs <run_id>
crabbox results <run_id>

Use --timing-json on run, warmup, and actions hydrate when a stable machine-readable timing record is needed. Commands can mark subphases by printing markers on stdout or stderr:

echo CRABBOX_PHASE:install
pnpm install --frozen-lockfile
echo CRABBOX_PHASE:test
pnpm test

Output events are capped previews. Use logs for retained output tails and results for parsed test summaries.

Desktop, WebVNC, And UI Proof

Create desktop/browser leases for visual QA, headed browser automation, or UI proof:

crabbox warmup --desktop --browser
crabbox warmup --provider aws --os ubuntu:26.04 --desktop --browser --desktop-env wayland
crabbox warmup --provider aws --os ubuntu:26.04 --desktop --browser --desktop-env gnome

ubuntu:26.04 is the default portable Linux OS selector where the provider catalog supports it. Use --os ubuntu:24.04 only when a test must stay on the previous LTS. Explicit provider image flags still win over --os.

For human demos, prefer WebVNC over native VNC because crabbox webvnc --open preloads the lease password in the browser fragment:

crabbox webvnc --id <lease> --open --take-control
crabbox webvnc status --id <lease>
crabbox webvnc reset --id <lease> --open --take-control
crabbox vnc --id <lease> --open

For input automation, use first-class helpers instead of hand-written xdotool:

crabbox desktop doctor --id <lease>
crabbox desktop launch --id <lease> --browser --url https://example.com --webvnc --open --take-control
crabbox desktop click --id <lease> --x 640 --y 420
crabbox desktop paste --id <lease> --text "user@example.com"
printf 'user@example.com' | crabbox desktop paste --id <lease>
crabbox desktop type --id <lease> --text "user+qa@example.com"
crabbox desktop key --id <lease> ctrl+l
crabbox screenshot --id <lease> --output desktop.png

When desktop/WebVNC hangs, trust the inline rescue output first: problem: and rescue: lines usually name exact next commands such as webvnc status/reset, desktop doctor, or native vnc --open.

Use artifacts for UI QA proof instead of committing screenshots or videos to a product repo branch:

crabbox artifacts collect --id <lease> --all --output artifacts/<slug>
crabbox artifacts publish --dir artifacts/<slug> --pr <number>
crabbox artifacts list <artifact-manifest-url-or-dir>
crabbox artifacts pull <artifact-manifest-url-or-dir> --output /tmp/<slug>-proof

artifacts publish uses brokered storage when configured, or explicit S3/R2 / Cloudflare/local hosting flags. Use --dry-run before public PR comments when reviewing generated Markdown or storage commands.

Provider Boundaries

SSH-lease providers support the full sync/run surface when the target supports it: scripts, fresh PR checkouts, captures, downloads, Actions hydration, SSH, ports, WebVNC, code-server, desktop, browser, cache volumes, and cleanup.

Delegated-run providers own command transport. Expect them to reject SSH-run features such as --capture-stdout, --capture-stderr, --capture-on-fail, --script, --script-stdin, --fresh-pr, local captures, --download, --full-resync, and --env-helper unless crabbox providers --json and the provider docs advertise the matching capability. --keep-on-failure is still useful for one-shot delegated providers that Crabbox would otherwise stop after a failed command.

Module-runtime delegated providers, such as Cloudflare Dynamic Workers, run source modules rather than Linux shell commands. Use --script <file> or --script-stdin for module source; trailing -- <command>, SSH, rsync, ports, Actions hydration, desktop, browser, and code-server do not apply unless the provider explicitly documents them.

Use --market spot|on-demand on AWS warmup or one-shot run when account quota or capacity testing needs a temporary market override. An explicit --type means exact type; Crabbox reports quota/capacity/policy failures instead of silently falling back.

Local And Static Targets

Use local-container for fast local proof when the host has Docker or Podman. warmup creates a container but does not sync; for an interactive synced container, use run --keep --sync-only:

crabbox run --provider local-container --keep --slug local-smoke --sync-only
eval "$(crabbox ssh --provider local-container --id local-smoke)"

Pass --local-container-runtime docker or --local-container-runtime podman when the engine matters, and keep that flag on reused lease commands such as run --id, ssh, status, and stop. crabbox ssh prints an SSH command; use eval "$(crabbox ssh ...)" to connect. After login, cd into the workdir printed by run --sync-only.

Use static SSH for existing machines:

crabbox run --provider ssh --target macos --static-host mac.example.com -- xcodebuild test
crabbox run --provider ssh --target windows --windows-mode normal --static-host win.example.com -- dotnet test

Static hosts are host-managed: Crabbox does not provision or delete them.

Useful Commands

crabbox providers
crabbox providers --json
crabbox doctor
crabbox config path
crabbox config show
crabbox whoami
crabbox sync-plan
crabbox warmup --class beast
crabbox prewarm
crabbox status --id <lease> --wait
crabbox inspect --id <lease> --json
crabbox run --id <lease> --preflight --timing-json -- pnpm test
crabbox job list
crabbox job run --dry-run <job-name>
crabbox pool ready
crabbox history --lease <lease>
crabbox events <run_id> --json
crabbox attach <run_id>
crabbox logs <run_id>
crabbox results <run_id>
crabbox cache stats --id <lease>
crabbox cache volumes
crabbox ssh --id <lease>
crabbox connect <lease>
crabbox ports --id <lease> --publish 8080
crabbox cp --id <lease> ./coverage.xml SANDBOX:/tmp/coverage.xml
crabbox webvnc --id <lease> --open
crabbox code --id <lease> --open
crabbox egress start --id <lease> --profile discord --daemon
crabbox desktop doctor --id <lease>
crabbox desktop proof --id <lease> --output artifacts/<slug>-proof -- ./scripts/visual-smoke.sh
crabbox artifacts collect --id <lease> --all --output artifacts/<slug>
crabbox artifacts publish --dir artifacts/<slug> --pr <number> --dry-run
crabbox usage --scope org
crabbox pause <lease>
crabbox resume <lease>
crabbox stop <lease>

Failure Triage

• Provider missing or old CLI: verify crabbox --help and crabbox providers list the provider, then rebuild or install a current binary.
• Bad local config: compare crabbox config show, pass --provider ... explicitly, and run crabbox doctor.
• Sync surprise: run crabbox sync-plan, add excludes, then retry with --debug --timing-json or --full-resync.
• Raw box missing Node/pnpm/Docker: use --preflight; hydrate first if the repo has an Actions workflow, or include setup in the command/script.
• Command failed: keep the run_... handle, inspect results, then rerun the focused failing shard/file before a full suite.
• Desktop unhealthy: run desktop doctor, then follow problem: / rescue: output from webvnc status or webvnc reset.
• Cleanup uncertain: use crabbox list, crabbox inspect --json, and only stop leases or provider resources you created.
• Broker/auth confusion: use crabbox doctor, crabbox whoami, and crabbox config show before asking for cloud credentials.

Cleanup

Brokered leases have coordinator-owned idle expiry and local lease claims. Default idle timeout is 30 minutes unless config or flags set a different value. Still stop boxes you created when done:

crabbox stop <cbx_id-or-slug>

When crabbox list prints orphan=no-active-lease, treat it as an operator review hint: verify the provider machine is not referenced by an active coordinator lease before deleting anything, especially if keep=true is set.