browser-skill

Drive the user's real Chromium browser (with their logins and cookies) through the bsk CLI. The extension opens an isolated Agent Window for automation; the user's normal windows stay protected unless you explicitly borrow a tab.

When to use

• Open pages, read titles/text, scrape structured data from sites the user can already access
• Fill forms, click through multi-step flows, smoke-test a UI change
• Understand pages with bsk snapshot first; use bsk get-html or bsk screenshot only when the snapshot is insufficient
• Operate on a specific user tab they point you at (after bsk tab borrow)

When NOT to use

• Tasks with no browser involved (files, APIs, databases only)
• Installing or configuring the extension (point the user to setup docs instead)
• Credential harvesting — never run bsk evaluate on banking, SSO, or password-manager pages to extract tokens, cookies, or secrets
• Long-lived control of a user's personal login window — borrow only for the immediate step, then bsk tab return or end the session
• Replacing the user's manual browsing when they only wanted an explanation

Prerequisites

1. bsk on PATH (Rust CLI from browser-skill)
2. browser-skill extension loaded in Chromium and connected (popup shows green)
3. Any bsk command auto-starts background services as needed; use bsk doctor if anything fails

Mandatory workflow

Every automation task must follow this lifecycle. Do not rely on idle timeouts (default session idle is 5 minutes).

1. bsk session start              → capture the 4-letter session id printed on stdout
2. … every tool command …        → always pass --session <id>
3. bsk session stop <id>          → REQUIRED when done (even on error paths)

Optional: bsk session start --browser <instance-id-or-label> when multiple browsers are connected (bsk browsers / error output lists them).

Emergency cleanup: bsk session stop --all or the Agent Window overlay Stop all.

Stop when the goal is met

Every task is a bounded goal, not open-ended browsing. The goal may come from the user's request, a recorded trace.json, or both.

1. Define success first — one concrete, observable condition derived from the user's words, purpose, or the last meaningful step in a trace (e.g. "form submitted", "item added to cart", "playback started").
2. Take the shortest path — snapshot → act → at most one check. Do not wander, re-try unrelated actions, or stack exploratory steps.
3. Stop as soon as success is reached — run bsk session stop <id> immediately unless the user explicitly asked to keep the session open (e.g. "don't close yet", "keep browsing").
4. No post-success work — once the goal is met, do not click, refresh, navigate, re-search, switch tabs, or "double-check" that it worked. Further verification is a new task.
5. When blocked, pause — do not brute-force — if the page requires human input (login, captcha, OTP, payment confirmation) or an action fails twice with no progress, call bsk request-help instead of retrying blindly. See Ask the human for help below.
6. When unsure — at most one extra bsk snapshot. If success looks met, stop. If not, ask the user; do not keep clicking.

With a trace: replay steps in order using target role/name/tag and raw value/selection fields. After the last step (or when its effect.navigated_to / success hint is satisfied), apply rules 3–4 immediately. The trace guides execution; it does not extend control beyond the goal.

Without a trace: the user's request is the success condition. Satisfying it ends the task — same stop rules apply.

Core interaction loop

Write operations only affect tabs in the Agent Window (or tabs you borrowed into it).

bsk navigate <url> --session <id>
bsk snapshot --session <id>          → aria tree with @e1, @e2, … refs
bsk observe --session <id>           → semantic VOM view; may reveal hover/focus surfaces
bsk click @e3 --session <id>          → or bsk fill, bsk select, bsk press
bsk snapshot --session <id>            → again after navigation / DOM change

Refs invalidate after navigation — always re-snapshot before clicking, filling, or selecting on a new page.

Prefer @eN refs from the latest snapshot over raw CSS selectors. Use --ref / --selector when ambiguous (bsk click --help).

Observation priority

Start with bsk snapshot to understand page structure, text, controls, and element refs. Use bsk observe when semantic VOM output or conditional hover/focus surfaces would materially help. Only escalate to raw HTML or screenshots when the latest observation cannot answer the question:

1. bsk snapshot — strict static page understanding and interaction planning
2. bsk observe — semantic VOM observation; may run bounded perception probes such as hover-surface discovery
3. bsk get-html — when hidden DOM, metadata, or markup details are required
4. bsk screenshot — when visual layout, canvas/image content, or styling cannot be inferred from the observation. Use --ref @eN (from the latest snapshot/observe) to crop to one element; omit --ref for the full visible tab.

Do not call bsk get-html or bsk screenshot first just to inspect a page.

Sandbox rules

Global flags

Command-specific flags (timeouts, --tab-id, --wait-until, …): bsk <cmd> --help

CLI command reference (one line each)

Details and flags: bsk <cmd> --help

Diagnostics

Session

Tabs (require --session <id>)

Observation (require --session unless noted)

Navigation

(bsk navigate back / bsk navigate forward are equivalent subcommands.)

Interaction

Scripting & timing

Ask the human for help — bsk request-help

When a step needs a human (captcha, login, OTP) or you want the user to confirm an important action, pause and ask:

bsk request-help --session <id> --prompt "Solve the captcha, then click Done only after the site accepts it" \
  --title "Captcha required" --target @e7 --target "#submit" --timeout 5m

• --prompt (required): what the user should do.
• --title (optional): custom title for the overlay panel. When omitted, the extension shows its default localized title.
• --target (repeatable): a snapshot ref (@e7) or CSS selector (#submit) to scroll to and flash-highlight. Strongly recommended — whenever the prompt refers to a concrete element (a button to click, a field to fill, a checkbox to toggle), pass its @eN ref / selector so the user is guided straight to the right spot instead of hunting for it. For interaction scenarios, always include the relevant target(s); reserve a prompt with no --target for cases where there is genuinely no specific element to point at (e.g. "wait for the page to finish loading").
• --timeout (default 5m): how long to wait.
• --completion-criteria (optional): JSON success detector. Use it only when there is a concrete post-help success signal, e.g. {"any":[{"url_contains":"/dashboard"},{"selector_exists":"[data-testid='account-menu']"}],"stable_for_ms":1000}.

The target tab is brought to the foreground; the page stays interactive while the agent control mask is hidden. The call blocks until the user explicitly acts, the timeout expires, cancellation arrives, or explicit completion criteria match. Page reloads, SPA route changes, and captcha refreshes do not return control by themselves. The result outcome is one of:

• continued — the user finished and clicked Done / return control (treat as confirm).
• cancelled — the user clicked Cancel (treat as reject/abort).
• timed_out — nobody acted within the timeout.
• completed — the explicit --completion-criteria matched while the user had control.
• navigated — deprecated legacy outcome. Do not rely on navigation as a completion signal.

note carries any text the user typed back. resolved_targets reports which refs/selectors matched a live element.

request-help does not refresh the page model after the user returns control. After a continued or completed result, issue a separate observation tool call (usually bsk snapshot --session <id>) before using new refs or reasoning about the post-help page state.

Disabling request-help (unattended mode)

Set BSK_REQUEST_HELP=off on unattended servers: bsk request-help then returns immediately with outcome="disabled" (no overlay, no waiting, exit 0). Any other value keeps it enabled. If you get disabled, do not retry — complete the task autonomously or stop gracefully.

Recording — bsk record

Capture the user's own actions in the Agent Window to a trace.json, for later LLM-driven automation:

bsk record start --browser <instance-id-or-label> [--url https://…] [--purpose "publish a wiki doc"] [--output trace.json]
# `--url` is optional; default https://example.com/ when omitted (must be http(s)).
# Blocks until the user clicks Finish in the recording panel, then writes ./trace.json and closes the window.

bsk record stop [--output trace.json]   # terminal fallback if the browser panel is unavailable

• The trace is a record-only action log (a pages[] dictionary + navigate/click/fill/select/press steps with target descriptors). It records what the user did; deciding which inputs are variable is left to the executing agent.
• --purpose is optional context metadata; it does not change what gets captured.
• There is no bsk replay — to redo a flow, read the trace and reuse the existing session / snapshot / @eN / click / fill tools. Follow Stop when the goal is met.
• Do not record on banking/SSO/password-manager pages; passwords are redacted but traces may still contain sensitive text.

Error handling

Exit codes (echo $? after bsk …)

Human errors print error: + hint: on stderr; --json includes code, message, hint, exit_code.

When to run diagnostics

Always bsk session stop <id> in a finally-style path so the Agent Window closes and borrowed tabs return.

Red lines

1. No token theft — do not bsk evaluate on sensitive sites to read localStorage, cookies, or auth headers for exfiltration.
2. No long borrow — do not leave a user's personal tab in the Agent Window across unrelated tasks.
3. No skip stop — always bsk session stop <id>; never assume idle timeout will clean up.
4. No post-success control — once the user’s goal (or last trace step) is met, do not keep operating the page; stop the session unless they asked to keep it open.
5. No raw observe escalation before snapshot/observe — use bsk snapshot first; use bsk observe when VOM semantics or conditional surfaces help. Only use bsk get-html or bsk screenshot when snapshot/observe is insufficient. Element screenshots (--ref @eN) still require a fresh snapshot/observe ref — never skip observation just to grab a visual.
6. evaluate is powerful and risky — use only when snapshot + click/fill/select cannot suffice; never on credential surfaces.

More detail for any command: bsk <cmd> --help