Calibrate Placed Cameras

Purpose

Produce a Metropolis-format calibration.json — per-camera intrinsics, extrinsics, homography, and field-of-view polygons — plus a top-view reference image, using CameraCalibrationManager from isaacsim.sensors.rtx.calibration.

Prerequisites

• Isaac Sim 6 / Kit 110 running with the Python server (isaacsim.code_editor.python_server, port 8226) — see isaac-sim-remote.
• isaacsim.sensors.rtx.calibration resolvable. It is pinned by isaacsim.exp.action_and_event_data_generation.base.kit; launch that app, or --enable isaacsim.sensors.rtx.calibration.
• Cameras already placed under the camera parent prim (default /World/Cameras) — use place-camera-max-coverage or place-camera-aim-at first.
• A local output folder. Nucleus / cloud URIs are rejected.
• Shell env contract from isaac-sim-orchestrator: $ISAAC_SIM_DIR, $ISAAC_LAB_DIR, $WORKSPACE_DIR.

Limitations

• This repo does not build the extension. isaacsim.sensors.rtx.calibration is authored in the metrosim repo and consumed here as an exact registry pin declared in source/apps/isaacsim.exp.action_and_event_data_generation.base.kit (read the version there rather than from this skill). This skill drives it; API changes belong upstream.
• Needs a live Kit session — the top view is a real orthographic render, so there is no offline path.
• Every camera must fall inside the top-view frustum. One camera outside it aborts the whole calibration.
• The top-view camera path is stored in a non-persistent setting, so the top-view stage must be re-run each session before calibration.
• Calibration describes cameras as they are now. Moving or re-placing a camera invalidates the file; re-run.

Troubleshooting

When to use this skill

The pipeline

Four stages, strictly ordered — each one depends on the previous having finished:

1. Top-view camera. An orthographic camera framed on scene_root. Sets top_view_camera_path, which every later stage validates.
2. Calibration dots. dot_count reference points per camera under /World/Calibration_Dots, used to solve the world↔image mapping.
3. Calibration file. Per-camera intrinsics, extrinsics, homography, and FOV polygons → calibration.json.
4. Top-view image. Orthographic render → Top.png plus imageMetadata.json.

Parameter semantics and settings are in references/calibration-api.md; the output schema and async/ordering rules are in references/calibration-runtime-notes.md.

Available Scripts

Running scripts

The script is a python_server payload, not a standalone program. From agent runtimes that expose skill execution helpers, invoke it with run_script():

run_script("scripts/calibrate_camera.py", args=["action=preflight"])

From a built tree, send it through the isaac-sim-remote client:

python skills/isaac-sim-remote/scripts/isaacsim_send.py \
  --file skills/calibrate-metropolis-camera/scripts/calibrate_camera.py \
  --arg action=calibrate --arg output_dir=/tmp/calib

Workflow

1. Preflight. action=preflight (the default) reports extension, stage, camera count, place_info, scene_root, and output folder without touching the stage. It never raises.

   run_script("scripts/calibrate_camera.py", args=["action=preflight"])
   

   Expect CALIBRATE_PREFLIGHT: ext=enabled stage=ok cameras=5 place_info=ok scene_root=ok output_dir=.... A cameras=0 here is the usual blocker — place cameras first.

2. Calibrate. Runs all four stages and verifies the files landed.

   run_script("scripts/calibrate_camera.py", args=["action=calibrate", "output_dir=/tmp/calib"])
   

   Expect CALIBRATE_RESULT: action=calibrate cameras=5 top_camera=... output=/tmp/calib followed by a wrote <file> (<n> bytes) line per artifact.

3. Verify. Confirm calibration.json has one sensors[] entry per camera and that Top.png is not black — a black top view means the orthographic camera framed empty space, and the homographies derived from it are meaningless.

4. Re-run after any camera moves. The file is a snapshot of the current poses.

Use action=topview to refresh only the top-view camera and image without redoing the dots and calibration.

Calling the API directly

from isaacsim.sensors.rtx.calibration import CameraCalibrationManager, CameraCalibrationSettings

CameraCalibrationSettings.camera_calibration_output_folder_path = "/tmp/calib"
CameraCalibrationSettings.place_info = "room=warehouse"
CameraCalibrationSettings.scene_bounding_box_path = "/World"

manager = CameraCalibrationManager.get_instance()
manager.create_top_view_camera(root_prim_path="/World")
await manager.generate_calibration_dot_prim_async()

# NOT `if manager.is_ready_to_calibrate():` -- see below.
if manager.is_ready_to_calibrate() is False:
    raise RuntimeError("stage is not ready; see the carb log")

await manager.generate_calibration_async()
await manager.capture_and_process_camera_view_image()

Three traps worth stating up front:

• is_ready_to_calibrate() returns None on success. It has three return False branches and no return True, so it falls off the end. Testing it for truthiness means the guard never passes. Compare is not False.
• The sync entry points are fire-and-forget. generate_calibration() wraps asyncio.ensure_future(...) and returns immediately, so the next stage races it. Await the _async variants; python_server payloads support top-level await.
• top_view_camera_path is non-persistent. It is only set by create_top_view_camera(), so stage 1 must run every session even if the top-view camera prim still exists on the stage.

Related skills

• place-camera-max-coverage / place-camera-aim-at — produce the cameras this skill calibrates.
• isaac-sim-remote — the transport this skill's script rides on.
• isaac-camera — per-camera intrinsics, render products, AOVs, distortion.
• isaac-sim-sensor — attaching annotators and writers once cameras exist.