diff --git a/CHANGELOG.md b/CHANGELOG.md index 52f331b14..f6d793cee 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -31,7 +31,7 @@ First Clawdis release post rebrand. This is a semver-major because we dropped le ### iOS node (Iris) - New iOS companion app that pairs to the Gateway bridge, reports presence as a node, and exposes a WKWebView “Canvas” for agent-driven UI. -- `clawdis nodes invoke` supports `screen.eval` and `screen.snapshot` to drive and verify the iOS Canvas (fails fast when Iris is backgrounded). +- `clawdis nodes invoke` supports `canvas.eval` and `canvas.snapshot` to drive and verify the iOS Canvas (fails fast when Iris is backgrounded). - Voice wake words are configurable in-app; Iris reconnects to the last bridge when credentials are still present in Keychain. ### WhatsApp & agent experience diff --git a/README.md b/README.md index 98546183a..76f1dc597 100644 --- a/README.md +++ b/README.md @@ -123,7 +123,7 @@ Build/run the mac app with `./scripts/restart-mac.sh` (packages, installs, and l Iris is an internal/prototype iOS app that connects as a **remote node**: - **Voice trigger:** forwards transcripts into the Gateway (agent runs + wakeups). -- **Canvas screen:** a WKWebView + `` surface the agent can control (via `screen.eval` / `screen.snapshot` over `node.invoke`). +- **Canvas screen:** a WKWebView + `` surface the agent can control (via `canvas.eval` / `canvas.snapshot` over `node.invoke`). - **Discovery + pairing:** finds the bridge via Bonjour (`_clawdis-bridge._tcp`) and uses Gateway-owned pairing (`clawdis nodes pending|approve`); `clawdis nodes status` shows paired nodes + capabilities. Runbook: `docs/ios/connect.md` diff --git a/docs/android/connect.md b/docs/android/connect.md index fde107f4a..7affd49a8 100644 --- a/docs/android/connect.md +++ b/docs/android/connect.md @@ -102,7 +102,7 @@ The Android node’s Chat sheet uses the gateway’s **primary session key** (`m ## 7) Canvas + camera Canvas commands (foreground only): -- `screen.eval`, `screen.snapshot`, `screen.navigate`, `screen.setMode` +- `canvas.eval`, `canvas.snapshot`, `canvas.navigate`, `canvas.setMode` Camera commands (foreground only; permission-gated): - `camera.snap` (jpg) diff --git a/docs/ios/connect.md b/docs/ios/connect.md index 575e162fd..0f3f30c01 100644 --- a/docs/ios/connect.md +++ b/docs/ios/connect.md @@ -122,19 +122,19 @@ Iris runs a WKWebView “Canvas” scaffold which exposes: - `window.__clawdis.ctx` (2D context) - `window.__clawdis.setStatus(title, subtitle)` -### Draw with `screen.eval` +### Draw with `canvas.eval` ```bash -clawdis nodes invoke --node "iOS Node" --command screen.eval --params "$(cat <<'JSON' +clawdis nodes invoke --node "iOS Node" --command canvas.eval --params "$(cat <<'JSON' {"javaScript":"(() => { const {ctx,setStatus} = window.__clawdis; setStatus('Drawing','…'); ctx.clearRect(0,0,innerWidth,innerHeight); ctx.lineWidth=6; ctx.strokeStyle='#ff2d55'; ctx.beginPath(); ctx.moveTo(40,40); ctx.lineTo(innerWidth-40, innerHeight-40); ctx.stroke(); setStatus(null,null); return 'ok'; })()"} JSON )" ``` -### Snapshot with `screen.snapshot` +### Snapshot with `canvas.snapshot` ```bash -clawdis nodes invoke --node 192.168.0.88 --command screen.snapshot --params '{"maxWidth":900}' +clawdis nodes invoke --node 192.168.0.88 --command canvas.snapshot --params '{"maxWidth":900}' ``` The response includes `base64` PNG data (for debugging/verification). diff --git a/docs/ios/spec.md b/docs/ios/spec.md index 936f390d0..c8a1ae741 100644 --- a/docs/ios/spec.md +++ b/docs/ios/spec.md @@ -117,14 +117,11 @@ Add to `src/gateway/protocol/schema.ts` (and regenerate Swift models): ### Node command set (screen-focused) These are values for `node.invoke.command`: -- `screen.show` / `screen.hide` -- `screen.navigate` with `{ url }` (Canvas URL or https URL) -- `screen.eval` with `{ javaScript }` -- `screen.snapshot` with `{ maxWidth?, quality?, format? }` -- `screen.setMode` with `{ mode: "canvas" | "web" }` - -Alias: -- `canvas.*` is accepted as a synonym for `screen.*` (e.g. `canvas.eval` → `screen.eval`). +- `canvas.show` / `canvas.hide` +- `canvas.navigate` with `{ url }` (Canvas URL or https URL) +- `canvas.eval` with `{ javaScript }` +- `canvas.snapshot` with `{ maxWidth?, quality?, format? }` +- `canvas.setMode` with `{ mode: "canvas" | "web" }` Result pattern: - Request is a standard `req/res` with `ok` / `error`. @@ -135,13 +132,13 @@ As of 2025-12-13, the Gateway supports `node.invoke` for bridge-connected nodes. Example: draw a diagonal line on the iOS Canvas: ```bash -clawdis nodes invoke --node ios-node --command screen.eval --params '{"javaScript":"(() => { const {ctx} = window.__clawdis; ctx.clearRect(0,0,innerWidth,innerHeight); ctx.lineWidth=6; ctx.strokeStyle=\"#ff2d55\"; ctx.beginPath(); ctx.moveTo(40,40); ctx.lineTo(innerWidth-40, innerHeight-40); ctx.stroke(); return \"ok\"; })()"}' +clawdis nodes invoke --node ios-node --command canvas.eval --params '{"javaScript":"(() => { const {ctx} = window.__clawdis; ctx.clearRect(0,0,innerWidth,innerHeight); ctx.lineWidth=6; ctx.strokeStyle=\"#ff2d55\"; ctx.beginPath(); ctx.moveTo(40,40); ctx.lineTo(innerWidth-40, innerHeight-40); ctx.stroke(); return \"ok\"; })()"}' ``` ### Background behavior requirement When iOS is backgrounded: - Voice may still be active (subject to iOS suspension). -- **All `screen.*` commands must fail** with a stable error code, e.g.: +- **All `canvas.*` commands must fail** with a stable error code, e.g.: - `NODE_BACKGROUND_UNAVAILABLE` - Include `retryable: true` and `retryAfterMs` if we want the agent to wait. @@ -222,8 +219,8 @@ open Clawdis.xcodeproj - Implement bridge routing + ACLs 4) **iOS screen/canvas** - WKWebView screen surface - - `screen.navigate/eval/snapshot` - - Background fast-fail for `screen.*` + - `canvas.navigate/eval/snapshot` + - Background fast-fail for `canvas.*` 5) **Unify mac Canvas under the same node.invoke** - Keep existing implementation, but expose it through the unified protocol path so the agent uses one API.