AI Process Manager (AIPM)

File or folder status: exists, size, last modified, locked?, locking process. Ideal for checking if a render/export finished without opening Explorer.

Check if a process is running on the user's Windows machine — no screenshot. Filter by executable name fragment (e.g. "python", "ffmpeg") and/or window title (e.g. "RENDER VIDEO 02"). Returns PID, CPU%, RAM, title, command line, start time. Command lines are truncated to 120 chars unless full_command_line=true. Results are capped at 10 matches, windowed processes first.

Bring a window to the foreground. REQUIRES action tier + app allowlist.

Query what AIPM learned locally about an app: successful (role, name) -> action recipes, plus common_failures (targets that keep failing — do not retry them). Call BEFORE acting in an app. Even with known=false you may get ui_shape: the shape of the app's UI tree (counts, roles, depth) learned passively from earlier reads — useful cold-start map. Pass the process name as shown in task manager (e.g. "notepad.exe", "chrome.exe").

Recent API audit log: which agents called which endpoints and when. Works even when the API is paused. For compliance and debugging agent behavior. Secrets and typed text (api_key=, value=) are masked before being recorded.

Local economy metrics: tokens and time saved by structured state vs screenshots, queries served, task/action success rates, top apps, 24h/7d trends. Two independent views of savings — economy_measured_by_api (measured here) and economy_reported_by_agents (self-reported): never add them together. The store block states which local database the totals came from.

Last 24h of PC events: process_started, process_ended (with duration), window_focused, file_changed. Useful for "what happened while I was away" (e.g. when did the render finish?).

Machine snapshot: CPU%, RAM used/total, GPU name and usage, disk free space per drive, network, uptime, active displays.

Human view of the taskbar: pinned apps and running apps grouped with window titles.

UI Automation tree of a window (roles, names, states) — the accessibility snapshot of a native Windows app. depth: default 4, max 30. max_nodes: default 200, max 1000 — max_nodes is the cost brake, NOT depth. Win32 apps expose content within 4-6 levels. Chromium/Electron apps (VS Code, Slack, Discord, Claude Desktop, Teams) bury real content under ~10 levels of Pane/Group wrappers: at low depth you get only empty Panes and conclude, wrongly, that the window is empty. Measured on Claude Desktop: depth=8 -> 15 useless nodes; depth=17 -> 115 nodes, 87 of them named (~8 KB) — the tree saturates at 17. So for Chromium/Electron ask for depth=15-20 and cap cost with max_nodes. The response reports depth, max_nodes, nodes, depth_reached and truncated; when truncated=true the tree was cut (read next_action) and you should repeat with a higher depth and/or max_nodes before concluding anything about the window. For finding a clickable target prefer ui_find (already filtered to interactive elements).

Ping the AI Process Manager backend. Returns version, API URL, and whether the local service is reachable. Call this first in a new session.

List running processes sorted by CPU. Use "top" to limit (default 25, max 200). Prefer check_process when searching for one process. Command lines truncated unless full_command_line=true.

List open desktop windows: title, process, minimized/maximized state, position, Z-order, and which has focus. Replaces screenshots for "what is open".

Read TEXT from inside a window via UI Automation — no screenshot. Works with consoles (cmd, PowerShell, Windows Terminal), editors, and most native apps. Typical use: read render progress from a console ("frame 4812/5000"). Identify the window by title_contains or hwnd from list_windows.

Report one UI action outcome (invoke/set_value/focus). Feeds per-app recipes in get_app_knowledge. Metadata only — never screen content or typed text. Report FAILURES too (ok=false plus reason): they become common_failures and stop the next agent from repeating the same dead end.

Report task outcome to local telemetry (metadata only — never screen content). Call at END of a computer-use task. Feeds get_app_knowledge and get_economy_stats.

List interactive UI elements in a window (buttons, fields, menus first; grid cells last). Each item has a stable global element_index plus can_invoke/can_set_value flags. Paginated (total/offset/limit/has_more). Read-only — no action tier required. Pass element_index to ui_invoke/ui_set_value.

Semantically CLICK a UI element via UI Automation (button, menu item, checkbox). Target by name_contains or element_index from ui_find. REQUIRES user-enabled action tier + app allowlist; otherwise returns action_denied. Confirm with the user before acting.

Set text in an editable field via UI Automation (no key simulation). REQUIRES action tier + allowlist. Check can_set_value in ui_find; modern apps (e.g. Win11 Notepad) may not expose fields. Confirm with the user first.

Long-poll until a condition is met (or timeout). Replaces polling loops. Conditions: process_ended, process_started, file_stable (render/download done), file_exists, window_title_contains. Max 50s per call; if satisfied=false, call again to keep waiting.