Agent Skills › viralcode/openwhale › Vision Sandbox

Vision Sandbox

GitHub

利用 Gemini 代码执行沙箱进行高精度图像分析，支持空间定位、视觉数学计算及 UI 审计。通过提取 UI 元数据辅助自动化编码代理生成或修复前端样式。

skills/vision-sandbox/SKILL.md viralcode/openwhale

Trigger Scenarios

需要识别图片中特定元素坐标基于图像内容进行计数或计算检查 UI 布局重叠或可读性为自动化编码代理提供视觉 grounding

Install

npx skills add viralcode/openwhale --skill Vision Sandbox -g -y

More Options

Use without installing

npx skills use viralcode/openwhale@Vision Sandbox

指定 Agent (Claude Code)

npx skills add viralcode/openwhale --skill Vision Sandbox -a claude-code -g -y

安装 repo 全部 skill

npx skills add viralcode/openwhale --all -g -y

预览 repo 内 skill

npx skills add viralcode/openwhale --list

SKILL.md

Frontmatter

{
    "name": "Vision Sandbox",
    "slug": "vision-sandbox",
    "version": "1.1.0",
    "metadata": {
        "openclaw": {
            "emoji": "🔭",
            "requires": {
                "env": [
                    "GEMINI_API_KEY"
                ],
                "bins": [
                    "uv"
                ]
            },
            "primaryEnv": "GEMINI_API_KEY"
        }
    },
    "description": "Agentic Vision via Gemini's native Code Execution sandbox. Use for spatial grounding, visual math, and UI auditing."
}

Vision Sandbox 🔭

Leverage Gemini's native code execution to analyze images with high precision. The model writes and runs Python code in a Google-hosted sandbox to verify visual data, perfect for UI auditing, spatial grounding, and visual reasoning.

Installation

clawhub install vision-sandbox

Usage

uv run vision-sandbox --image "path/to/image.png" --prompt "Identify all buttons and provide [x, y] coordinates."

Pattern Library

📍 Spatial Grounding

Ask the model to find specific items and return coordinates.

Prompt: "Locate the 'Submit' button in this screenshot. Use code execution to verify its center point and return the [x, y] coordinates in a [0, 1000] scale."

🧮 Visual Math

Ask the model to count or calculate based on the image.

Prompt: "Count the number of items in the list. Use Python to sum their values if prices are visible."

🖥️ UI Audit

Check layout and readability.

Prompt: "Check if the header text overlaps with any icons. Use the sandbox to calculate the bounding box intersections."

🖐️ Counting & Logic

Solve visual counting tasks with code verification.

Prompt: "Count the number of fingers on this hand. Use code execution to identify the bounding box for each finger and return the total count."

Integration with OpenCode

This skill is designed to provide Visual Grounding for automated coding agents like OpenCode.

Step 1: Use vision-sandbox to extract UI metadata (coordinates, sizes, colors).
Step 2: Pass the JSON output to OpenCode to generate or fix CSS/HTML.