orx-compute

Each run uses an immutable snapshot of the experiment branch's recorded commit. Remote backends receive that snapshot; Tinker extracts it for a local controller whose SDK sends model operations remotely.

orx exp status <expId>                 # branch, parent, run command, latest run + commit
orx compute                            # browse GPU offers across all providers
orx compute --gpu H100_SXM --count 1   # filter GPU offers
orx compute --cpu                      # browse CPU-only offers
orx exp run <expId>                    # launch on the configured default
orx exp cancel <expId>                 # cancel the in-flight run

Inspect configuration and custom instructions first

Before configuring compute or constructing a launch command:

orx compute status --json
orx compute instructions show --json

instructions show returns the machine-wide CUSTOM.md content, absolute path, and revision. Missing or blank content means there are no custom instructions. Read the section for the selected backend/host, then its backend guide below. For configuration, connection tests, login, or instruction updates, read references/configuration.md (orx skill compute/configuration is the fallback).

Usually leave CUSTOM.md empty. Add verified, consistently repeated bespoke compute instructions only when standard CLI settings and committed project scripts cannot express the workflow and the agent truly needs extra steps. Keep any such guidance short: working commands, environment paths, cluster constraints, and documentation links. Use orx compute instructions set --file - --expected-revision <revision from show>; never write the file directly with file-edit tools or shell redirects. Preserve the user's guidance; replace obsolete steps instead of appending run logs. Never store credentials or invent cluster commands. Read linked cluster instructions before using unfamiliar scheduler options. Instructions do not change the configured backend or authorize bypassing the launch contract.

Universal launch contract

• Launch all experiment compute with orx exp run. Never invoke provider CLIs, schedulers, raw SSH, or the training command directly. The worktree is for editing, Git, orchestration, and lightweight checks; direct jobs are untracked and may run code other than the recorded commit.
• Keep the run command fixed. Set it once on the baseline and vary code or configuration on child branches. If none exists, use orx project edit <projectId> --run-command '<cmd>' before launching.
• Commit before launching. Every backend runs the recorded commit's immutable source snapshot. Uncommitted files are excluded; no backend needs a GitHub push.
• orx exp run queues the run and returns immediately. Follow it with orx runs, orx logs, orx exp wait, or orx exp wake.
• --force permits a deliberate concurrent run on the same experiment. Without it, orx rejects a launch while that node already has a run in flight.
• Every run gets ORX_RUN_ID; use it as the run name in metric trackers. Saved environment variables reach every backend except hf. If TRACKIO_SERVER_URL is saved, log with trackio.init(project=os.environ.get("TRACKIO_PROJECT", <name>), name=os.environ.get("ORX_RUN_ID")); the client reads the URL and TRACKIO_WRITE_TOKEN itself. A localhost URL only works on the local backend. Never print the token.
• local, tinker, ssh and slurm runs also get TENSORBOARD_LOGDIR, a per-run directory; pass it as the trainer's logging_dir/log_dir. orx tensorboard <runId> prints the command that opens it (not for SSH container runs).

Resolve the backend, then read one guide

The session playbook states the configured default. A bare orx exp run <expId> uses it. Use another backend only when the user names one; a connected credential is not a signal to switch.

Before constructing the launch command, read exactly one reference relative to this SKILL.md:

Do not read guides for backends you are not using. Kubernetes manifest work always requires references/k8s.md before creating or editing the manifest. If the installed reference cannot be read, orx skill compute/<backend> prints the same canonical document.

For Slurm, --cpus and --mem request #SBATCH --cpus-per-task and #SBATCH --mem; omitting either uses the configured Slurm default, then the partition default. Read references/slurm.md before choosing values.

Waiting on runs — orx exp wait

Block until a run changes state when you want to act as soon as it finishes:

orx exp wait <expId>                    # wait for this experiment's latest run
orx exp wait --project <projectId>      # return on the first project completion
orx exp wait <expId> --interval 10 --timeout 3600

• Pass exactly one of <expId> or --project.
• --project is the budget-loop primitive: it returns on the first completion, not on starts or queued-to-running transitions. Reissue it once per loop tick.
• A wait is a sleep-until-change signal, not the source of truth. After every return, read orx runs <projectId> and reconcile all newly terminal runs.
• When nothing is in flight, project wait returns drained: no runs in flight.
• The default interval is 5 seconds and timeout is 1800 seconds. Timeout exits non-zero and means nothing changed yet, not that the run failed.
• Cancel only a run that is broken or no longer wanted; after a wait timeout, wait again instead of cancelling.
• Failed runs include a reason: line. Provider-capacity failures are often retryable; failures after startup require reading the file located by orx logs <runId>.
• A failed run is not a new node. Repair and relaunch the same experiment as described by orx-experiment-tree.

Going idle instead — orx exp wake

After launching, use orx exp wake <expId> when you want to end the turn and resume after that run succeeds or fails. Wake-up is opt-in, fires only for done or failed, and waits behind queued user messages. If orx cannot monitor a live Slurm run for over a minute, it tells you once per outage; the final wake-up comes once orx can see the run finish. Use either wait or wake for a run, not both.

Sizing compute

• Decide GPU versus CPU first. API-driven evaluation and data preparation often run more cheaply on CPU.
• Pick the smallest shape that fits the model and a minimal batch.
• Escalate after a real OOM or hopelessly slow run instead of starting with the largest accelerator.
• Raise the timeout only for genuinely long runs.