io.github.Savvy-Tech-Sphere/usable-browser-agent

Go back in the active tab's history.

Disarm previously-armed HTTP basic-auth answers (all domains, or specific ones).

Click an element by its ref from the latest snapshot. Supports right/middle button, double-click, and modifier keys (ctrl/shift/alt/meta) for context menus, multi-select, and open-in-new-tab.

Read text from the system clipboard. Use after the page put something on the clipboard (e.g. a 'Copy' button) or to move data between pages and apps.

Read an image from the system clipboard (e.g. after a 'Copy image' action) and return it as a PNG/JPEG. Errors if the clipboard holds no image.

Write text to the system clipboard.

Put a local image file onto the system clipboard so it can be pasted into a page.

Close a tab by id (defaults to the active tab).

List native JS dialogs (alert/confirm/prompt) the page tried to open and how they were auto-answered. Use this to see if the page asked something.

Download a file to disk WITHOUT the OS save dialog, and return its local path so you can read it. Use for PDFs, exports, attachments — the file lands in the browser's downloads folder.

List recent downloads with their on-disk paths and state.

Drag one element onto another (kanban cards, sortable lists, file tiles, sliders). Fires both HTML5 drag-and-drop and a pointer-drag sequence so native and JS-library targets both respond.

Run JavaScript in the page and return the result. Disabled under hardened mode, which is the default; the operator enables it by setting UBA_STRICT_SECRETS=0 in the server environment. On Firefox the code runs in the content-script sandbox; on Chrome it runs in the page's own realm, where the page can observe it and its CSP applies. Use only when no dedicated tool fits.

Fill multiple fields in one call. Pass fields as an array of {ref, text}. Faster and more reliable than many browser_type calls for forms.

Fill a username or password field from the server-side credential broker. Pass only a secret alias; secret values are never accepted as tool input or returned.

Bring the connected browser (Firefox or Chrome, per the bridge connection) to the foreground so OS-level input lands in it. macOS only.

Go forward in the active tab's history.

Get the current value/text of a field or element by ref.

Hover the pointer over an element by ref.

Arm answers for HTTP basic-auth prompts (the native username/password dialog) from a server-side secret alias, scoped to that secret's domains. Pass only the alias; the credential value is never accepted as input or returned. Call before navigating to the protected URL.

Get the active tab's url, title, load state, scroll position, and short workflow-memory hints when available.

List all open tabs with their ids, urls, and titles.

Atomically fill username/password fields from a server-side secret alias and submit the login form. Pass only refs and the secret alias; secret values are never accepted as tool input or returned.

Make navigator.geolocation return a fixed position (so location-gated tasks proceed without the native permission prompt). Pass clear=true to stop mocking. Applies to the current page; re-set after navigation.

Inspect or operate a NATIVE browser dialog that no extension can touch — the file picker, print dialog, basic-auth popup, or 'Leave page?'. action='list' shows the dialog's buttons; 'click_button' presses one by name (e.g. 'Open', 'Save', 'Don't Save'); 'set_text' types into the focused dialog field. macOS only; needs Accessibility permission.

Report which OS-level input capabilities are available (trusted keystrokes, native-dialog control, raw clicks) and any setup needed. Check this before relying on the browser_os_* / browser_native_dialog tools.

Navigate the active tab to a URL and wait for it to finish loading.

Inspect recent network requests (method, URL, status, type, errors) the browser made. Use to diagnose why a button 'did nothing', find an API endpoint, or confirm a request succeeded.

Open a new tab, optionally at a URL, and make it active.

Click at absolute screen coordinates with a REAL OS mouse event. Prefer browser_click; use this only for trusted clicks or native chrome. macOS only; raw clicks need `brew install cliclick`.

Click a page element by ref with a REAL, OS-trusted mouse event (for elements that ignore synthetic clicks). Computes the element's on-screen position via the connected browser, raises its window, then clicks. On Firefox coordinates are exact; on Chrome they are approximate (refused unless allowApproximate is set) and iframe elements are not supported. macOS only; needs cliclick + Accessibility permission.

Press a REAL OS key (Enter, Tab, Escape, ArrowDown, F1-F8, or a single char), optionally with modifiers. For native UI and isTrusted-checking pages. macOS only.

Type text using REAL OS keystrokes into whatever currently has focus. Use only when in-page typing fails because the element checks event.isTrusted (some payment fields, editors, games). Focus the field first (e.g. browser_click), then call this. macOS only; needs Accessibility permission.

Paste into an element by ref as a real paste event — rich editors (Google Docs, Notion, contenteditable) handle this better than typing. Pastes the given text, or the current clipboard contents when text is omitted.

Press a key (Enter, Tab, Escape, ArrowDown, etc.). Optionally target an element by ref; otherwise the focused element receives it.

Get the visible text of the page (or of one element by ref). Use for reading article/content; use browser_snapshot for interactive structure.

Reload the active tab.

Hand control to the human for something automation must not or cannot do — CAPTCHAs, anti-bot 'verify you're human' walls, 2FA approvals, or a deliberate judgment call. Sends a desktop notification and returns a message you should relay to the user, then STOP and wait for them to act before continuing. Do not attempt to bypass CAPTCHAs yourself.

Save the current tab as a PDF to the downloads folder (dialog-free). Useful to capture a rendered page. If your Firefox build still shows a save dialog, use browser_native_dialog to confirm it.

Capture a PNG screenshot of the visible part of the active tab. Use only when the snapshot is insufficient (e.g. canvas, charts, visual layout).

Scroll the page or a specific scrollable container. Pass a ref to scroll that container (inner panes, virtualized/infinite lists); omit it to scroll the window. Use to='bottom' to jump to the end (e.g. trigger infinite-scroll loading).

Choose an option in a <select> dropdown by visible label or value.

Make a tab active by id (the agent acts on the active tab).

Set how native JS dialogs are auto-answered so they never block the agent. accept=true confirms (default), promptText sets what prompt() returns, suppress=true neutralizes 'Leave page?' beforeunload guards. Applies to the current page; re-set after navigation if needed.

Capture an accessibility/DOM snapshot of the active Firefox tab as a structured tree. Each interactive element gets a stable [ref=eN] handle. USE THIS to understand the page before acting, and re-run it after the page changes — refs become stale after navigation or DOM updates. If workflow-memory hints appear, call browser_workflow_recall before continuing.

Check the bridge connection and the current active tab. If the active tab has workflow memories, this returns short hints.

Focus a text field by ref and type text into it. Set submit=true to press Enter afterward (e.g. to submit a search).

Attach local file(s) to the page WITHOUT opening the OS file picker (the picker cannot be automated — never click elements that open it). Pass the ref of a file input (hidden ones appear in snapshots as 'file-input ... [hidden]'), or of an upload button/dropzone near it, plus absolute local file paths. Sets the input's files directly, or simulates a drag-and-drop when no file input is found.

Wait a fixed number of milliseconds (use sparingly).

Wait until the given text appears anywhere on the page (or timeout).