background-computer-use
GitHub用于启动并连接本地 macOS BackgroundComputerUse 运行时，通过其自文档化 API 控制应用、窗口、截图及辅助功能。支持自动安装、权限检查及动态路由发现，实现可视化交互与操作。
Trigger Scenarios
Install
npx skills add NeverSight/learn-skills.dev --skill background-computer-use -g -y
SKILL.md
Frontmatter
{
"name": "background-computer-use",
"description": "Launch and use the local BackgroundComputerUse macOS runtime through its self-documenting loopback API. Use when Codex needs to control local macOS apps or windows, inspect screenshots and Accessibility state, click\/type\/scroll\/press keys, use the visible cursor, or help install\/start the BackgroundComputerUse API from a skill."
}
Background Computer Use
Use this skill to start or connect to the local macOS BackgroundComputerUse runtime, then rely on the runtime's own documentation endpoints instead of memorizing route schemas.
Workflow
-
Start or discover the runtime:
bash "$SKILL_DIR/scripts/ensure-runtime.sh"If
$SKILL_DIRis not already set by the host, derive it from this skill folder path before running scripts. -
Read the runtime manifest from:
$TMPDIR/background-computer-use/runtime-manifest.jsonNever assume a fixed port.
-
Call
GET /v1/bootstrap.- If
instructions.readyis false, report the returned permission steps to the user. - Do not continue with actions until Accessibility and Screen Recording are ready.
- If
-
Call
GET /v1/routes.- Treat this as the source of truth for available routes, request fields, response fields, examples, and error codes.
- Do not assume browser routes exist; use them only when
/v1/routesadvertises them.
-
For visual tasks, request screenshots with
imageMode: "path"and inspect the returned image files when useful. -
For action routes, reuse
stateTokenfrom the state you inspected. Reuse the samecursor.idwhen the user wants one continuous visible cursor.
Helpers
scripts/ensure-runtime.sh: find, install, launch, and bootstrap the runtime.scripts/install-runtime.sh: installBackgroundComputerUse.appfrom an app zip or release URL.scripts/bcu-request.py: call runtime endpoints from the manifest base URL.
Examples:
python3 "$SKILL_DIR/scripts/bcu-request.py" GET /v1/bootstrap
python3 "$SKILL_DIR/scripts/bcu-request.py" GET /v1/routes
python3 "$SKILL_DIR/scripts/bcu-request.py" POST /v1/list_apps '{}'
Local Development
When working from a source checkout instead of an installed release:
BCU_SOURCE_DIR=/path/to/background-computer-use bash "$SKILL_DIR/scripts/ensure-runtime.sh"
That runs the repo's script/start.sh, which builds, signs, installs, launches, and bootstraps the app.
More Detail
Read references/runtime.md only when you need install modes, release packaging notes, or the permission/debugging checklist.
Version History
- e0220ca Current 2026-07-05 23:03
