VMware Private AI (GPU & Model Serving)

[READ] Estimate GPU memory / GPU count / storage to serve an LLM of a given size. A transparent PLANNING heuristic (no connection): give a model name containing the size ("llama-70b") or model_billions (70). Returns weights_gib, serving_vram_gib, a per-GPU-model count table (L4/A100/L40S/H100/H200), model-on-disk GiB, and — importantly — that LLM inference is compute/HBM-bound so random IOPS is the wrong axis for the weights. Not authoritative; confirm against the model card and a real vLLM benchmark.

[WRITE] Set a VM's vGPU profile (e.g. "grid_a100-4c"). Requires the VM to be powered OFF. A bare call returns blast_radius — the VM's name and id, current and target profile, whether its vGPU device is edited or a new one added, other passthrough devices left untouched, power state, blockers, unmeasured — and changes nothing. confirm=True applies it, but refuses a running or suspended VM (power it off via vmware-aiops first), a name shared by several VMs, and a VM whose power state or devices cannot be read. This tool never powers the VM off itself. Show blast_radius to the user and wait for their decision. Do not set confirm=True on your own because the user asked for this earlier: they have not seen the blast radius yet. Audited.

[READ] Report whether each GPU host is actually configured to serve vGPU / PAIS. Each item has host, vgpu_ready (verdict), gpu_count, vendors, total_gpu_memory_mb, default_graphics_type (must be 'sharedDirect' for vGPU), vgpu_profiles_offered, active_vgpu_vms, and blocking_reasons explaining any "not ready". The NVIDIA driver / MFT VIB version and MIG geometry are NOT in the vSphere API — every item carries a driver_note pointing to nvidia-smi / esxcli. Only hosts that have a GPU are returned. Paginated.

[READ] Which account this server uses, and whether it holds each tool's vSphere privileges. Use this when a tool fails with NoPermission, or before assign_vgpu, to check what the configured account may do. Asks vCenter/ESXi about the current session (needs only System.View, so any account can ask about itself). Returns target, account, the privilege states, and for one tool its requires/verdict/meaning; without tool, every tool grouped by verdict. Gotchas: privileges are checked on the root folder only, and not_granted_at_root is NOT "no access" — it may be granted on a specific cluster, host or VM. undetermined means vCenter did not answer; never read it as granted. PAIS tools use a bearer token, not vSphere, so they show as needing no vSphere privilege; assign_vgpu's entry includes the device-change and resource-change privileges it needs.

[READ] Full GPU detail for one host by name — every GPU's device, type, memory, consumers. A wrong host name returns a teaching error listing hosts that do have GPUs, not a traceback. Use list_gpu_hosts to discover host names.

[READ] Real-time GPU utilization per vGPU VM (busiest first) — gpu_pct, mem_pct, temp_c. Each item has vm, profile, gpu_pct, mem_pct, mem_used_kb, temp_c, metrics_available, idle. Requires the NVIDIA host GPU driver — a VM with no samples shows metrics_available=false (not an error). Deep per-SM/MIG-slice metrics are not available via vSphere (need DCGM).

[READ] List the PAIS model CATALOG — models available/approved to deploy (not the served set). Answers "which models are approved / what can I install in-house" (the air-gapped model question). Each item has id, name, status, source. Distinct from list_pais_models (which lists what is already SERVED). Path is INFERRED — a 404 returns a base-URL teaching message, not a bug. Needs the PAIS endpoint + VMWARE_PRIVATEAI_PAIS_TOKEN.

[READ] Fleet GPU monitoring rollup for VCF Ops dashboards — utilisation, hot/idle, busiest. Aggregates the per-VM vGPU perf counters into one estate summary: vgpu_vms, reporting_vms, idle_vms, hot_vms, gpu_pct/mem_pct avg+max, temp_c_max, a per-profile tally, and the busiest `top` VMs. This is the vCenter-perf-counter view (not PAIS REST) — deep per-SM/MIG/power telemetry needs NVIDIA DCGM and is out of scope (scope_note says so). Requires a vCenter connection and the NVIDIA host driver.

[READ] List vCenter-level DirectPath (dynamic passthrough) profiles. vSphere 9.0+. Each item has id, name, vendor, description. On a vCenter older than 9.0 (no DirectPathProfileManager) this returns a teaching error routing you to list_vgpu_profiles, rather than an empty list that would read as "no profiles". Paginated envelope.

[READ] List VMs consuming a vGPU and the profile each holds (e.g. "grid_a100-4c"). Each item has vm and profile. This is the "who holds the GPU" view — pair it with vmware-aiops to power a VM off, or with a future vgpu assignment tool. Paginated.

[READ] List physical GPU devices across all hosts (flattened), filtered by host / vendor. Each item has host, device, type, vendor, pci_id, memory_mb, vm_count. Use this to find idle GPUs (vm_count 0) or to inventory a model across the estate. Paginated envelope.

[READ] List ESXi hosts that have at least one GPU, filtered by host name / GPU vendor. Returns a {items, returned, limit, total, truncated, hint} envelope; each item has host, gpu_count, vendors, vgpu_vms (VMs currently backed by a vGPU on that host). Use get_gpu_host for full per-GPU detail of one host. Requires a valid vCenter connection.

[READ] List PAIS RAG data sources (ingest connectors that feed knowledge bases). Each item has id, name, type, status. Pair with list_pais_knowledge_bases to see the RAG pipeline. Path is INFERRED — a 404 prompts a base-URL check, not a bug. Needs the PAIS endpoint + VMWARE_PRIVATEAI_PAIS_TOKEN.

[READ] List Private AI Service knowledge bases (RAG vector stores). Each item has id, name, status, description. Same PAIS endpoint + bearer token as list_pais_models. Paginated envelope; an empty items with truncated=false means the endpoint has no knowledge bases (checked-and-none), not a failure.

[READ] List models served by Private AI Service (OpenAI-compatible /models). Returns a {items, returned, limit, total, truncated, hint} envelope; each item has id, owned_by, created. Needs a 'pais:' endpoint in config.yaml and a bearer token in VMWARE_PRIVATEAI_PAIS_TOKEN — a missing one returns a teaching error, not a traceback.

[READ] List the available vGPU profile catalog (what profiles a host can hand a VM). Returns a {items, returned, limit, total, truncated, hint} envelope; each item is a distinct profile with framebuffer_gib, profile_class, sharing, vendor_id, and the hosts that offer it (hosts / host_count). Pair with assign_vgpu to set a VM's profile. Scope to one host with `host` — this polls QueryConfigTarget per host. Requires a valid vCenter connection.

[READ] Pre-flight a vGPU profile change: would setting vm_name to target_profile succeed? Read-only companion to assign_vgpu (no reconfigure). Returns can_apply plus blocking_reasons, the current→target profile, power_state (must be off), and whether the VM's OWN host offers the target profile (with its framebuffer). Use this before assign_vgpu to see exactly what would block the change — the two ReconfigVM failure modes are "VM powered on" and "host does not offer this profile".

[READ] Parse a LOCAL pais.yml / bundle manifest and list the images to mirror for air-gap. Reads the file you point it at (no network, no registry pull), enumerates every container image reference, and flags the two things that break an air-gapped install: images on PUBLIC registries (must be mirrored inside the enclave) and images with a mutable tag / no digest. Returns images (registry/repository/tag/digest), registries_to_mirror, and warnings. A wrong path returns a teaching error. It does NOT verify the images are actually pullable.