let5see/clawdbot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Docs: document canvas.* node.invoke commands

Browse Source
This commit is contained in:
Peter Steinberger
2025-12-18 01:17:27 +00:00
parent 21a27e3b65
commit 272015c701
5 changed files with 16 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/android/connect.md
View File

@@ -102,7 +102,7 @@ The Android node’s Chat sheet uses the gateway’s **primary session key** (`m
## 7) Canvas + camera
Canvas commands (foreground only):
- `screen.eval`, `screen.snapshot`, `screen.navigate`, `screen.setMode`
- `canvas.eval`, `canvas.snapshot`, `canvas.navigate`, `canvas.setMode`
Camera commands (foreground only; permission-gated):
- `camera.snap` (jpg)

8
docs/ios/connect.md
View File

@@ -122,19 +122,19 @@ Iris runs a WKWebView “Canvas” scaffold which exposes:
- `window.__clawdis.ctx` (2D context)
- `window.__clawdis.setStatus(title, subtitle)`
### Draw with `screen.eval`
### Draw with `canvas.eval`
```bash
clawdis nodes invoke --node "iOS Node" --command screen.eval --params "$(cat <<'JSON'
clawdis nodes invoke --node "iOS Node" --command canvas.eval --params "$(cat <<'JSON'
{"javaScript":"(() => { const {ctx,setStatus} = window.__clawdis; setStatus('Drawing','…'); ctx.clearRect(0,0,innerWidth,innerHeight); ctx.lineWidth=6; ctx.strokeStyle='#ff2d55'; ctx.beginPath(); ctx.moveTo(40,40); ctx.lineTo(innerWidth-40, innerHeight-40); ctx.stroke(); setStatus(null,null); return 'ok'; })()"}
JSON
)"
```
### Snapshot with `screen.snapshot`
### Snapshot with `canvas.snapshot`
```bash
clawdis nodes invoke --node 192.168.0.88 --command screen.snapshot --params '{"maxWidth":900}'
clawdis nodes invoke --node 192.168.0.88 --command canvas.snapshot --params '{"maxWidth":900}'
```
The response includes `base64` PNG data (for debugging/verification).

21
docs/ios/spec.md
View File

@@ -117,14 +117,11 @@ Add to `src/gateway/protocol/schema.ts` (and regenerate Swift models):
### Node command set (screen-focused)
These are values for `node.invoke.command`:
- `screen.show` / `screen.hide`
- `screen.navigate` with `{ url }` (Canvas URL or https URL)
- `screen.eval` with `{ javaScript }`
- `screen.snapshot` with `{ maxWidth?, quality?, format? }`
- `screen.setMode` with `{ mode: "canvas" | "web" }`
Alias:
- `canvas.*` is accepted as a synonym for `screen.*` (e.g. `canvas.eval` → `screen.eval`).
- `canvas.show` / `canvas.hide`
- `canvas.navigate` with `{ url }` (Canvas URL or https URL)
- `canvas.eval` with `{ javaScript }`
- `canvas.snapshot` with `{ maxWidth?, quality?, format? }`
- `canvas.setMode` with `{ mode: "canvas" | "web" }`
Result pattern:
- Request is a standard `req/res` with `ok` / `error`.
@@ -135,13 +132,13 @@ As of 2025-12-13, the Gateway supports `node.invoke` for bridge-connected nodes.
Example: draw a diagonal line on the iOS Canvas:
```bash
clawdis nodes invoke --node ios-node --command screen.eval --params '{"javaScript":"(() => { const {ctx} = window.__clawdis; ctx.clearRect(0,0,innerWidth,innerHeight); ctx.lineWidth=6; ctx.strokeStyle=\"#ff2d55\"; ctx.beginPath(); ctx.moveTo(40,40); ctx.lineTo(innerWidth-40, innerHeight-40); ctx.stroke(); return \"ok\"; })()"}'
clawdis nodes invoke --node ios-node --command canvas.eval --params '{"javaScript":"(() => { const {ctx} = window.__clawdis; ctx.clearRect(0,0,innerWidth,innerHeight); ctx.lineWidth=6; ctx.strokeStyle=\"#ff2d55\"; ctx.beginPath(); ctx.moveTo(40,40); ctx.lineTo(innerWidth-40, innerHeight-40); ctx.stroke(); return \"ok\"; })()"}'
```
### Background behavior requirement
When iOS is backgrounded:
- Voice may still be active (subject to iOS suspension).
- **All `screen.*` commands must fail** with a stable error code, e.g.:
- **All `canvas.*` commands must fail** with a stable error code, e.g.:
- `NODE_BACKGROUND_UNAVAILABLE`
- Include `retryable: true` and `retryAfterMs` if we want the agent to wait.
@@ -222,8 +219,8 @@ open Clawdis.xcodeproj
- Implement bridge routing + ACLs
4) **iOS screen/canvas**
- WKWebView screen surface
- `screen.navigate/eval/snapshot`
- Background fast-fail for `screen.*`
- `canvas.navigate/eval/snapshot`
- Background fast-fail for `canvas.*`
5) **Unify mac Canvas under the same node.invoke**
- Keep existing implementation, but expose it through the unified protocol path so the agent uses one API.