Files
clawdbot/docs/nodes/index.md
2026-01-11 03:30:09 +00:00

175 lines
6.0 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
summary: "Nodes: pairing, capabilities, permissions, and CLI helpers for canvas/camera/screen/system"
read_when:
- Pairing iOS/Android nodes to a gateway
- Using node canvas/camera for agent context
- Adding new node commands or CLI helpers
---
# Nodes
A **node** is a companion device (iOS/Android today) that connects to the Gateway over the **Bridge** and exposes a command surface (e.g. `canvas.*`, `camera.*`, `system.*`) via `node.invoke`.
macOS can also run in **node mode**: the menubar app connects to the Gateways bridge and exposes its local canvas/camera commands as a node (so `clawdbot nodes …` works against this Mac).
Notes:
- Nodes are **peripherals**, not gateways. They dont run the gateway daemon.
- Telegram/WhatsApp/etc. messages land on the **gateway**, not on nodes.
## Pairing + status
Pairing is gateway-owned and approval-based. See [Gateway pairing](/gateway/pairing) for the full flow.
Quick CLI:
```bash
clawdbot nodes pending
clawdbot nodes approve <requestId>
clawdbot nodes reject <requestId>
clawdbot nodes status
clawdbot nodes describe --node <idOrNameOrIp>
clawdbot nodes rename --node <idOrNameOrIp> --name "Kitchen iPad"
```
Notes:
- `nodes rename` stores a display name override in the gateway pairing store.
## Invoking commands
Low-level (raw RPC):
```bash
clawdbot nodes invoke --node <idOrNameOrIp> --command canvas.eval --params '{"javaScript":"location.href"}'
```
Higher-level helpers exist for the common “give the agent a MEDIA attachment” workflows.
## Screenshots (canvas snapshots)
If the node is showing the Canvas (WebView), `canvas.snapshot` returns `{ format, base64 }`.
CLI helper (writes to a temp file and prints `MEDIA:<path>`):
```bash
clawdbot nodes canvas snapshot --node <idOrNameOrIp> --format png
clawdbot nodes canvas snapshot --node <idOrNameOrIp> --format jpg --max-width 1200 --quality 0.9
```
### Canvas controls
```bash
clawdbot nodes canvas present --node <idOrNameOrIp> --target https://example.com
clawdbot nodes canvas hide --node <idOrNameOrIp>
clawdbot nodes canvas navigate https://example.com --node <idOrNameOrIp>
clawdbot nodes canvas eval --node <idOrNameOrIp> --js "document.title"
```
Notes:
- `canvas present` accepts URLs or local file paths (`--target`), plus optional `--x/--y/--width/--height` for positioning.
- `canvas eval` accepts inline JS (`--js`) or a positional arg.
### A2UI (Canvas)
```bash
clawdbot nodes canvas a2ui push --node <idOrNameOrIp> --text "Hello"
clawdbot nodes canvas a2ui push --node <idOrNameOrIp> --jsonl ./payload.jsonl
clawdbot nodes canvas a2ui reset --node <idOrNameOrIp>
```
Notes:
- Only A2UI v0.8 JSONL is supported (v0.9/createSurface is rejected).
## Photos + videos (node camera)
Photos (`jpg`):
```bash
clawdbot nodes camera list --node <idOrNameOrIp>
clawdbot nodes camera snap --node <idOrNameOrIp> # default: both facings (2 MEDIA lines)
clawdbot nodes camera snap --node <idOrNameOrIp> --facing front
```
Video clips (`mp4`):
```bash
clawdbot nodes camera clip --node <idOrNameOrIp> --duration 10s
clawdbot nodes camera clip --node <idOrNameOrIp> --duration 3000 --no-audio
```
Notes:
- The node must be **foregrounded** for `canvas.*` and `camera.*` (background calls return `NODE_BACKGROUND_UNAVAILABLE`).
- Clip duration is clamped (currently `<= 60s`) to avoid oversized base64 payloads.
- Android will prompt for `CAMERA`/`RECORD_AUDIO` permissions when possible; denied permissions fail with `*_PERMISSION_REQUIRED`.
## Screen recordings (nodes)
Nodes expose `screen.record` (mp4). Example:
```bash
clawdbot nodes screen record --node <idOrNameOrIp> --duration 10s --fps 10
clawdbot nodes screen record --node <idOrNameOrIp> --duration 10s --fps 10 --no-audio
```
Notes:
- `screen.record` requires the node app to be foregrounded.
- Android will show the system screen-capture prompt before recording.
- Screen recordings are clamped to `<= 60s`.
- `--no-audio` disables microphone capture (supported on iOS/Android; macOS uses system capture audio).
- Use `--screen <index>` to select a display when multiple screens are available.
## Location (nodes)
Nodes expose `location.get` when Location is enabled in settings.
CLI helper:
```bash
clawdbot nodes location get --node <idOrNameOrIp>
clawdbot nodes location get --node <idOrNameOrIp> --accuracy precise --max-age 15000 --location-timeout 10000
```
Notes:
- Location is **off by default**.
- “Always” requires system permission; background fetch is best-effort.
- The response includes lat/lon, accuracy (meters), and timestamp.
## SMS (Android nodes)
Android nodes can expose `sms.send` when the user grants **SMS** permission and the device supports telephony.
Low-level invoke:
```bash
clawdbot nodes invoke --node <idOrNameOrIp> --command sms.send --params '{"to":"+15555550123","message":"Hello from Clawdbot"}'
```
Notes:
- The permission prompt must be accepted on the Android device before the capability is advertised.
- Wi-Fi-only devices without telephony will not advertise `sms.send`.
## System commands (mac node)
The macOS node exposes `system.run` and `system.notify`.
Examples:
```bash
clawdbot nodes run --node <idOrNameOrIp> -- echo "Hello from mac node"
clawdbot nodes notify --node <idOrNameOrIp> --title "Ping" --body "Gateway ready"
```
Notes:
- `system.run` returns stdout/stderr/exit code in the payload.
- `system.notify` respects notification permission state on the macOS app.
- `system.run` supports `--cwd`, `--env KEY=VAL`, `--command-timeout`, and `--needs-screen-recording`.
- `system.notify` supports `--priority <passive|active|timeSensitive>` and `--delivery <system|overlay|auto>`.
## Permissions map
Nodes may include a `permissions` map in `node.list` / `node.describe`, keyed by permission name (e.g. `screenRecording`, `accessibility`) with boolean values (`true` = granted).
## Mac node mode
- The macOS menubar app connects to the Gateway bridge as a node (so `clawdbot nodes …` works against this Mac).
- In remote mode, the app opens an SSH tunnel for the bridge port and connects to `localhost`.