Isaac Sim Validator

WARNING: Level 3 executes the target script with your full user privileges. There is no container or sandbox isolation. Only use --confirm-execute on scripts you fully trust.

Purpose

Reject common delivery defects: black frames, deprecated imports, hardcoded user paths, missing lights, wrong render modes, and undersized SimulationApp configurations.

Levels 1–2 are static checks only. Level 3 runs the target Python script under timeout with the calling user's privileges. Level 3 requires an explicit --confirm-execute (or CONFIRM_EXECUTE=1) opt-in; it is not a sandbox.

Prerequisites

• Built Isaac Sim ($ISAAC_SIM_DIR or _build/linux-x86_64/release).
• NVIDIA GPU with a current driver (nvidia-smi).
• Shell env contract from isaac-sim-orchestrator: $ISAAC_SIM_DIR, $ISAAC_LAB_DIR, $WORKSPACE_DIR.

Limitations

• Levels 1–2 cannot prove runtime correctness for every workload.
• Level 3 executes untrusted script content only after --confirm-execute; there is no container/firejail isolation.
• Thresholds target common demo/SDG failures, not all product features.

Troubleshooting

Final gate before delivery. Reject outputs that fail the checks below; do not soften results.

Available Scripts

Running scripts

From agent runtimes that expose skill execution helpers, invoke helpers with run_script():

run_script("scripts/validate_sim.sh", args=[])

From a built Isaac Sim tree, run the same file with ./python.sh (Linux) or python.bat (Windows) from _build/*/release, or execute shell helpers directly when they do not require the simulator.

Universal checklist (apply to every output)

File structure validation (USD scenes)

Render validation (frame analysis)

Video validation

For any delivered MP4/GIF, extract and inspect at least start/middle/end frames. Reject or recut the presentation copy if the first frames show render warm-up, the subject is occluded, or the camera loses the task. Keep the complete source render and metrics when a shorter presentation cut is made.

RL training validation

Asset & shell validation

Decision tree

graph TD
    A[Received Output] --> B{Has script?}
    B -->|No| C[Reject: "No script provided"]
    B -->|Yes| D{File size >1KB?}
    D -->|No| E[Reject: "Script empty or corrupted"]
    D -->|Yes| F{Python?}
    F -->|Yes| G{isaacsim.* imports?}
    G -->|No| H[Reject: "Use isaacsim.* not omni.isaac.core"]
    G -->|Yes| I{SimulationApp or --exec?}
    I -->|Neither| J[Reject: "Must use SimulationApp with window args OR isaac-sim.sh --exec"]
    I -->|SimulationApp| K{width and height set?}
    K -->|No| L[Reject: "SimulationApp requires explicit width and height"]
    K -->|Yes| M{Dome + Distant lights?}
    M -->|No| N[Reject: "Add DomeLight and DistantLight (min intensity 100 and 500)"]
    M -->|Yes| O{RayTracedLighting?}
    O -->|No| P[Reject: "Use RayTracedLighting for iteration"]
    O -->|Yes| Q{Render output exists?}
    Q -->|No| R[Reject: "No render output"]
    Q -->|Yes| S{>= 150 KB and mean_RGB > 30?}
    S -->|No| T[Reject: "Render is black, too dark, or corrupted"]
    S -->|Yes| U{Hardcoded user paths or bare IPs?}
    U -->|Yes| V[Reject: "Replace hardcoded user paths/IPs with env vars or args"]
    U -->|No| Y[VALID - deliver]

Training output

Reply format

• Pass: Valid: <one-line summary>
• Fail: Invalid: <list of failed checks>
• Suggest (only when otherwise valid): Could be improved: <tip>

Do not hedge ("I think", "maybe"). Do not deliver a black frame.

Input sources

• isaac-sim-orchestrator, isaac-sim-rendering, other producer skills, and sub-agent outputs.
• Invoked on Isaac Sim deliverables: "validate this Isaac Sim script", "check this Isaac Sim render", "review this sim output".

Automated validation

Run the checks above from the CLI with scripts/validate_sim.sh:

# Levels 1–2: static only
bash scripts/validate_sim.sh --level <1|2> <script.py> [--output-dir <dir>] [--timeout <seconds>]

# Level 3: EXECUTES the script — requires explicit opt-in
bash scripts/validate_sim.sh --level 3 --confirm-execute <script.py> [--output-dir <dir>] [--timeout <seconds>]
# or: CONFIRM_EXECUTE=1 bash scripts/validate_sim.sh --level 3 <script.py> ...

Feedback loop

• Rejections feed back into the producing skill's failure-mode catalog.
• Recurring failures become new validator rules.
• Standing rubric: references/quality-criteria.md. Record each iteration in references/feedback-log.md.