Isaac Sim robot workbench

A frame from the capture camera. Works headless; no GUI or viewport needed.

Stop the sim daemon; its container removes itself.

Aim the capture camera so the whole robot fills the frame. direction points from the robot toward the camera, e.g. [1, 0, 0.3] for a front view slightly from above (default: the last direction used). margin > 1 leaves room around the robot. raise_frac lifts it in the frame (fraction of its size), e.g. 0.15 to clear recording captions. Then sim_capture or sim_record_start.

Joint positions (rad/m) and velocities, plus driven joints' normalized positions and limits.

Per-joint travel and driven-joint mean tracking error since the last reset. Travel on a passive joint proves a linkage transmits: with zero stiffness it moves only if its constraint moves it.

List a USD's articulation DOFs, joints excluded from the articulation (loop closures), and roots. Reads the file, not the live scene, so it shows edits saved from the Isaac GUI. Defaults to the loaded USD.

Every spawned prop with its current world pose (pos in m, rot as w, x, y, z).

Named poses from the robot config, as {pose: {joint_pattern: 0..1}}.

Run the sim continuously in real time (True) or pause it (False).

Drive every driven joint from its lower limit toward `target`, report the fraction reached. A joint below 0.9 is blocked: self-collision, a binding linkage, or too little effort for the load. reset=True first puts every joint at its lower limit and verifies it. Start the daemon with ground=False if the robot can reach the floor, or the test measures the floor.

Change the caption drawn on frames recorded from now on ('' for none).

Start recording the capture camera: one frame every `every` physics steps. Everything that steps the sim is recorded (sim_step, sim_wave, sim_range_test). caption is drawn along the bottom of each frame; change it with sim_record_caption.

Stop recording and write an animated GIF; returns its path on the host. At the default every=2 and dt=1/120, fps=30 plays back at half speed; 60 is real time.

Restart the daemon to change a spawn property: robot, USD, solver iterations, self-collision. One Kit start rather than one per experiment; gains stay live via sim_set_params. Omitted robot/usd fall back to the defaults, not to what was loaded before.

Delete a spawned prop.

Capture the viewport as an image. Requires the daemon started with gui=True.

Put a plain panel (RGB 0..1) behind the robot, facing the capture camera; no color removes it. For clean footage, also start the daemon with ground=False.

Point the viewport camera, e.g. eye=[1.2, 1.2, 1.0] target=[0, 0, 0.3].

Place the capture camera by hand (sim_frame_robot does it automatically).

Change software-coupling ratios live, keyed by follower joint. 0 releases a follower. Couplings are declared in the robot config; a follower is driven to ratio x its leader's target.

Change the driven joints' PD gains and effort limit on the live articulation.

Command a named pose. amount blends toward it: 1 = as authored, 0.5 = halfway. Halfway from the lower limits by default, or from the current targets with from_current=True. Follow with sim_step, then sim_screenshot to verify.

Command the driven joints, normalized 0..1 (lower..upper limit). Pass `unit` as a full vector in driven order (see sim_status), or `joints` to set individual joints by name. Targets persist; advance with sim_step.

Add a prop to the live scene: a primitive or a USD file, with collision. shape: cuboid (size=[x,y,z]), sphere (radius), cylinder / capsule / cone (radius, height), or usd (usd=<path>, optional scale). Units are meters and kg. static=True: a fixed collider (a table, a wall). kinematic=True: a rigid body that contact cannot move. rot is a (w, x, y, z) quaternion. color is RGB 0..1. Spawns at /World/Objects/<name>; sim_step to let it fall and settle.

Report whether the sim daemon is up, its robot, driven joints, poses and parameters.

Advance the simulation n physics steps (dt from sim_status, default 1/120 s).

Compare several values of one live parameter inside the single session. param: stiffness, damping, effort, or "coupling:<follower>". For each value it applies the parameter and runs `test` ("wave" or "range") for `steps`. The original value is restored afterwards, even on error. A diverging solver is recorded as a result, not raised.

Start the daemon container and wait until it answers. Idempotent. robot: a robot config JSON, path relative to the repo root (e.g. examples/robots/franka.json); defaults to $ISAAC_MCP_ROBOT. usd: a USD path instead of / overriding the config's (every joint driven if no config). gui=True is needed for sim_screenshot; the host must have run `xhost +local:docker` once. ground=False spawns without a floor (use for range tests). extra_args go to scripts/simd.py verbatim (e.g. ["--pos-iters", "64"]). The first start downloads Nucleus assets and builds shader caches: minutes.

Sweep every driven joint through its range for n steps; return travel stats.

List a run's saved checkpoints with step and size.

List training runs: live or recent containers, and log directories holding checkpoints.

Tail a running training container's stdout/stderr.

TensorBoard scalars for a run: no tag lists tag names; a tag returns its series, downsampled.

Launch a headless Isaac Lab training run in its own container; returns at once. task is a registered gym id (e.g. Isaac-Cartpole-v0). run_name defaults to '<task>_<timestamp>'; keep it to find the run later. extra_args go to the training script verbatim (e.g. Hydra overrides). device pins one GPU index. Independent of the daemon and of this MCP session.

Container state, checkpoints, and latest TensorBoard scalar values for one run.

Stop a training run's container.