io.github.matthewalexong/neo-vision

Click an element on the page at exact pixel coordinates. Use the click_center.x and click_center.y values from a spatial_snapshot element. Returns an updated spatial map reflecting the page state after the click (including any navigation, modals, or DOM changes triggered by the click). You do NOT need to call spatial_snapshot again after clicking. Requires an active session — call spatial_snapshot first to open a page.

Filter the current spatial map without re-loading the page. Search for elements by ARIA role, HTML tag, label text, bounding box region, or actionability. Much faster than taking a new snapshot — use this when the page hasn't changed and you need to find specific elements. Requires a prior spatial_snapshot call (uses the cached map).

Scroll the page or a specific scrollable container. Use delta_y with positive values to scroll down, negative to scroll up. Optionally target a specific scrollable element by passing its x/y coordinates. Returns an updated spatial map reflecting the new scroll position and any newly visible elements. Requires an active session — call spatial_snapshot first.

Navigate to a URL and return a spatial map of the page. Launches a real Chrome browser automatically — no setup required. **Output modes** (controlled by output_format parameter): - 'compact' (default): Returns every visible element with pixel coordinates, ARIA roles, accessible labels, and actionability flags. Use the click_center coordinates from each element to target it with spatial_click or spatial_type. - 'agent': Optimized for AI agent context windows on text-dense pages (Wikipedia, Amazon, long articles). Returns: * 'page_text': deduplicated readable text from the current viewport (overlapping DOM nodes collapsed by position via 2-pass dedup: 10px grid + substring elimination, JS code stripped, text limited to 120 chars) * 'interactive_elements': only actionable elements within the viewport, with their click_center coordinates. Elements are strictly confined to viewport bounds — wide-layout pages (multi-column) won't leak off-screen elements. * 'viewport_info': scroll position, page height, percent_seen, percent_remaining — tells you how much more content is below * '_navigation': a hint with the exact spatial_scroll delta to use to advance to the next viewport **When to use agent mode**: On any page that returns 1000+ elements in compact mode, or when you need the readable text content for research/summarization tasks. After calling spatial_scroll(delta_y=...), re-call spatial_snapshot with output_format='agent' to get the next viewport's content. **Note**: The full DOM map is always cached server-side. You do NOT need to re-snapshot after clicks, typing, or scrolling — every action returns an updated map automatically. Re-snapshot with the same URL only when you want to reload or switch URLs. The browser session persists across calls. Calling this again with a different URL navigates to that URL. Calling with the same URL reloads it.

Type text into an element. If x/y coordinates are provided, clicks that position first to focus the element, then types. Otherwise types into whatever element currently has focus. Set clear_first=true to replace existing text (selects all + deletes before typing). Set press_enter=true to submit after typing (useful for search boxes). Returns an updated spatial map. Requires an active session — call spatial_snapshot first.