Docs: add canvas host usage

This commit is contained in:
Peter Steinberger
2025-12-18 11:39:30 +01:00
parent e1f320276e
commit 256f0fc765
10 changed files with 56 additions and 0 deletions

View File

@@ -101,6 +101,28 @@ The Android nodes Chat sheet uses the gateways **primary session key** (`m
## 7) Canvas + camera
### Gateway Canvas Host (recommended for web content)
If you want the node to show real HTML/CSS/JS that the agent can edit on disk, enable the Gateway canvas host and point the node at it.
1) On the gateway host, enable `canvasHost` in `~/.clawdis/clawdis.json`:
```json5
{
canvasHost: { enabled: true, root: "~/clawd/canvas", port: 18793, bind: "lan" }
}
```
2) Create `~/clawd/canvas/index.html`.
3) Navigate the node to it (LAN):
```bash
clawdis nodes invoke --node "<Android Node>" --command canvas.navigate --params '{"url":"http://<gateway-hostname>.local:18793/"}'
```
This server injects a live-reload client into HTML and reloads on file changes.
Canvas commands (foreground only):
- `canvas.eval`, `canvas.snapshot`, `canvas.navigate` (switches to web mode), `canvas.setMode` (use `"canvas"` to return)
- A2UI: `canvas.a2ui.push`, `canvas.a2ui.reset` (`canvas.a2ui.pushJSONL` legacy alias)

View File

@@ -11,6 +11,7 @@ Last updated: 2025-12-09
- A single long-lived **Gateway** process owns all messaging surfaces (WhatsApp via Baileys, Telegram when enabled) and the control/event plane.
- All clients (macOS app, CLI, web UI, automations) connect to the Gateway over one transport: **WebSocket on 127.0.0.1:18789** (tunnel or VPN for remote).
- One Gateway per host; it is the only place that is allowed to open a WhatsApp session. All sends/agent runs go through it.
- Optional: the Gateway can expose a LAN/tailnet HTTP canvas host (`canvasHost`) to serve `~/clawd/canvas` to node WebViews (live-reloads on file changes).
## Components and flows
- **Gateway (daemon)**

View File

@@ -93,6 +93,7 @@ The Gateway advertises small non-secret hints to make UI flows convenient:
- `sshPort=<port>` (defaults to 22 when not overridden)
- `gatewayPort=<port>` (informational; the Gateway WS is typically loopback-only)
- `bridgePort=<port>` (only when bridge is enabled)
- `canvasPort=<port>` (only when the optional canvas host is enabled; default `18793`)
- `tailnetDns=<magicdns>` (optional hint; may be absent)
## Debugging on macOS

View File

@@ -55,6 +55,7 @@ Troubleshooting and beacon details: `docs/bonjour.md`.
- `sshPort=22` (or whatever is advertised)
- `gatewayPort=18789` (loopback WS port; informational)
- `bridgePort=18790` (when bridge is enabled)
- `canvasPort=18793` (when the optional canvas host is enabled)
- `tailnetDns=<magicdns>` (optional hint)
Disable/override:

View File

@@ -21,6 +21,7 @@ pnpm clawdis gateway --port 18789 --verbose
pnpm clawdis gateway --force
```
- Binds WebSocket control plane to `127.0.0.1:<port>` (default 18789).
- Optionally starts a LAN/tailnet Canvas file server when `canvasHost.enabled` is `true` (default `http://0.0.0.0:18793`, serves `~/clawd/canvas`).
- Logs to stdout; use launchd/systemd to keep it alive and rotate logs.
- Pass `--verbose` to mirror debug logging (handshakes, req/res, events) from the log file into stdio when troubleshooting.
- `--force` uses `lsof` to find listeners on the chosen port, sends SIGTERM, logs what it killed, then starts the gateway (fails fast if `lsof` is missing).

View File

@@ -35,6 +35,7 @@ WhatsApp / Telegram
┌──────────────────────────┐
│ Gateway │ ws://127.0.0.1:18789 (loopback-only)
│ (single source) │ tcp://0.0.0.0:18790 (optional Bridge)
│ │ http://0.0.0.0:18793 (optional Canvas host)
└───────────┬───────────────┘
├─ Pi agent (RPC)
@@ -51,6 +52,7 @@ Most operations flow through the **Gateway** (`clawdis gateway`), a single long-
- **One Gateway per host**: it is the only process allowed to own the WhatsApp Web session.
- **Loopback-first**: Gateway WS is `ws://127.0.0.1:18789` (not exposed on the LAN).
- **Bridge for nodes**: optional LAN/tailnet-facing bridge on `tcp://0.0.0.0:18790` for paired nodes (Bonjour-discoverable).
- **Canvas host (optional)**: LAN/tailnet HTTP file server (default `18793`) for node WebViews; see `docs/configuration.md` (`canvasHost`).
- **Remote use**: SSH tunnel or tailnet/VPN; see `docs/remote.md` and `docs/discovery.md`.
## Features (high level)

View File

@@ -122,6 +122,30 @@ Iris runs a WKWebView “Canvas” scaffold which exposes:
- `window.__clawdis.ctx` (2D context)
- `window.__clawdis.setStatus(title, subtitle)`
### Gateway Canvas Host (recommended for web content)
If you want Iris to show real HTML/CSS/JS that the agent can edit on disk, enable the Gateway canvas host and point Iris at it.
1) On the gateway host, enable `canvasHost` in `~/.clawdis/clawdis.json`:
```json5
{
canvasHost: { enabled: true, root: "~/clawd/canvas", port: 18793, bind: "lan" }
}
```
2) Create `~/clawd/canvas/index.html`.
3) Navigate Iris to it (LAN):
```bash
clawdis nodes invoke --node "iOS Node" --command canvas.navigate --params '{"url":"http://<gateway-hostname>.local:18793/"}'
```
Notes:
- The server injects a live-reload client into HTML and reloads on file changes.
- iOS may require App Transport Security allowances to load plain `http://` URLs; if it fails to load, prefer HTTPS or adjust the iOS apps ATS config.
### Draw with `canvas.eval`
```bash

View File

@@ -30,6 +30,7 @@ Non-goals (v1):
## Current repo reality (constraints we respect)
- The Gateway WebSocket server binds to `127.0.0.1:18789` (`src/gateway/server.ts`) with an optional `CLAWDIS_GATEWAY_TOKEN`.
- The Gateway can optionally expose a LAN/tailnet Canvas file server (`canvasHost`) so nodes can `canvas.navigate` to `http://<lanHost>:<canvasPort>/` and auto-reload when files change (`docs/configuration.md`).
- macOS “Canvas” exists today, but is **mac-only** and controlled via mac app IPC (`clawdis-mac canvas ...`) rather than the Gateway protocol (`docs/mac/canvas.md`).
- Voice wake forwards via `GatewayChannel` to Gateway `agent` (mac app: `VoiceWakeForwarder``GatewayConnection.sendAgent`).

View File

@@ -42,6 +42,7 @@ Clawdis already uses:
For the clawd browser-control server, use “family” ports:
- Browser control HTTP API: `18791` (bridge + 1)
- Browser CDP/debugging port: `18792` (control + 1)
- Canvas host HTTP (optional): `18793` (next free port; see `docs/configuration.md`)
The user usually only configures the **control URL** (port `18791`). CDP is an internal detail.

View File

@@ -10,6 +10,8 @@ read_when:
Status: draft spec · Date: 2025-12-12
Note: for iOS/Android nodes that should render agent-edited HTML/CSS/JS over the network, prefer the Gateway `canvasHost` (serves `~/clawd/canvas` over LAN/tailnet with live reload). This doc focuses on the macOS in-app canvas panel. See `docs/configuration.md`.
Clawdis can embed an agent-controlled “visual workspace” panel (“Canvas”) inside the macOS app using `WKWebView`, served via a **custom URL scheme** (no loopback HTTP port required).
This is designed for: