refactor: make browser actions ref-only

This commit is contained in:
Peter Steinberger
2025-12-26 19:02:19 +00:00
parent f50f18f65a
commit 1236c4dafb
15 changed files with 131 additions and 183 deletions

View File

@@ -112,4 +112,4 @@ git commit -m "Add Clawd workspace"
- Canvas UI runs full-screen with native overlays. Avoid placing critical controls in the top-left/top-right/bottom edges; add explicit gutters in the layout and dont rely on safe-area insets.
- For browser-driven verification, use `clawdis browser` (tabs/status/screenshot) with the clawd-managed Chrome profile.
- For DOM inspection, use `clawdis browser eval|query|dom|snapshot` (and `--json`/`--out` when you need machine output).
- For interactions, use `clawdis browser click|type|hover|drag|select|upload|press|wait|navigate|back|evaluate|run` (click/type accept `--selector`).
- For interactions, use `clawdis browser click|type|hover|drag|select|upload|press|wait|navigate|back|evaluate|run` (click/type require snapshot refs; use `evaluate` for CSS selectors).

View File

@@ -167,16 +167,14 @@ Inspection:
- `clawdis browser screenshot`
- `clawdis browser screenshot --full-page`
- `clawdis browser screenshot --ref 12`
- `clawdis browser snapshot`
- `clawdis browser snapshot --format aria --limit 200`
- `clawdis browser snapshot --format ai`
Actions:
- `clawdis browser navigate https://example.com`
- `clawdis browser resize 1280 720`
- `clawdis browser click 12 --double`
- `clawdis browser click --selector 'button.save'`
- `clawdis browser type 23 "hello" --submit`
- `clawdis browser type --selector "input[name=q]" "hello"`
- `clawdis browser press Enter`
- `clawdis browser hover 44`
- `clawdis browser drag 10 11`
@@ -186,14 +184,15 @@ Actions:
- `clawdis browser dialog --accept`
- `clawdis browser wait --text "Done"`
- `clawdis browser evaluate --fn '(el) => el.textContent' --ref 7`
- `clawdis browser evaluate --fn "document.querySelector('.my-class').click()"`
- `clawdis browser console --level error`
- `clawdis browser pdf`
Notes:
- `upload` and `dialog` are **arming** calls; run them before the click/press that triggers the chooser/dialog.
- The arm default timeout is **2 minutes** (clamped to max 2 minutes); pass `timeoutMs` if you need shorter.
- `snapshot --format ai` returns AI snapshot markup used for ref-based actions.
- `click`/`type` accept `--selector` to target CSS selectors instead of AI refs.
- `snapshot` defaults to `ai`; `aria` returns an accessibility tree for debugging.
- `click`/`type` require `ref` from `snapshot --format ai`; use `evaluate` for rare CSS selector one-offs.
## Security & privacy notes

View File

@@ -49,7 +49,8 @@ Core actions:
Notes:
- Requires `browser.enabled=true` in `~/.clawdis/clawdis.json`.
- Uses `browser.controlUrl` unless `controlUrl` is passed explicitly.
- `act` supports CSS selectors for `click`/`type` via `selector` (use `ref` for AI snapshot targets).
- `snapshot` defaults to `ai`; use `aria` for the accessibility tree.
- `act` requires `ref` from `snapshot --format ai`; use `evaluate` for rare CSS selector needs.
### `clawdis_canvas`
Drive the node Canvas (present, eval, snapshot, A2UI).