Agent Skills › viralcode/openwhale › Vision Sandbox

Vision Sandbox

GitHub

利用Gemini代码执行沙箱分析图像，提供空间定位、视觉数学计算及UI审计功能。通过编写Python代码验证视觉数据，提取UI元数据以辅助自动化编码和前端修复。

skills/vision-sandbox/SKILL.md viralcode/openwhale

Trigger Scenarios

需要识别图像中元素坐标进行UI布局或可读性审计基于图片进行计数或视觉计算

Install

npx skills add viralcode/openwhale --skill Vision Sandbox -g -y

More Options

Use without installing

npx skills use viralcode/openwhale@Vision Sandbox

指定 Agent (Claude Code)

npx skills add viralcode/openwhale --skill Vision Sandbox -a claude-code -g -y

安装 repo 全部 skill

npx skills add viralcode/openwhale --all -g -y

预览 repo 内 skill

npx skills add viralcode/openwhale --list

SKILL.md

Frontmatter

{
    "name": "Vision Sandbox",
    "slug": "vision-sandbox",
    "version": "1.1.0",
    "metadata": {
        "openclaw": {
            "emoji": "🔭",
            "requires": {
                "env": [
                    "GEMINI_API_KEY"
                ],
                "bins": [
                    "uv"
                ]
            },
            "primaryEnv": "GEMINI_API_KEY"
        }
    },
    "description": "Agentic Vision via Gemini's native Code Execution sandbox. Use for spatial grounding, visual math, and UI auditing."
}

Vision Sandbox 🔭

Leverage Gemini's native code execution to analyze images with high precision. The model writes and runs Python code in a Google-hosted sandbox to verify visual data, perfect for UI auditing, spatial grounding, and visual reasoning.

Installation

clawhub install vision-sandbox

Usage

uv run vision-sandbox --image "path/to/image.png" --prompt "Identify all buttons and provide [x, y] coordinates."

Pattern Library

📍 Spatial Grounding

Ask the model to find specific items and return coordinates.

Prompt: "Locate the 'Submit' button in this screenshot. Use code execution to verify its center point and return the [x, y] coordinates in a [0, 1000] scale."

🧮 Visual Math

Ask the model to count or calculate based on the image.

Prompt: "Count the number of items in the list. Use Python to sum their values if prices are visible."

🖥️ UI Audit

Check layout and readability.

Prompt: "Check if the header text overlaps with any icons. Use the sandbox to calculate the bounding box intersections."

🖐️ Counting & Logic

Solve visual counting tasks with code verification.

Prompt: "Count the number of fingers on this hand. Use code execution to identify the bounding box for each finger and return the total count."

Integration with OpenCode

This skill is designed to provide Visual Grounding for automated coding agents like OpenCode.

Step 1: Use vision-sandbox to extract UI metadata (coordinates, sizes, colors).
Step 2: Pass the JSON output to OpenCode to generate or fix CSS/HTML.