docs: reorganize documentation structure

This commit is contained in:
Peter Steinberger
2026-01-07 00:41:31 +01:00
parent b8db8502aa
commit db4d0b8e75
126 changed files with 881 additions and 270 deletions

157
docs/nodes/index.md Normal file
View File

@@ -0,0 +1,157 @@
---
summary: "Nodes: pairing, capabilities, permissions, and CLI helpers for canvas/camera/screen/system"
read_when:
- Pairing iOS/Android nodes to a gateway
- Using node canvas/camera for agent context
- Adding new node commands or CLI helpers
---
# Nodes
A **node** is a companion device (iOS/Android today) that connects to the Gateway over the **Bridge** and exposes a command surface (e.g. `canvas.*`, `camera.*`, `system.*`) via `node.invoke`.
macOS can also run in **node mode**: the menubar app connects to the Gateways bridge and exposes its local canvas/camera commands as a node (so `clawdbot nodes …` works against this Mac).
## Pairing + status
Pairing is gateway-owned and approval-based. See [`docs/gateway/pairing.md`](/gateway/pairing) for the full flow.
Quick CLI:
```bash
clawdbot nodes pending
clawdbot nodes approve <requestId>
clawdbot nodes reject <requestId>
clawdbot nodes status
clawdbot nodes describe --node <idOrNameOrIp>
clawdbot nodes rename --node <idOrNameOrIp> --name "Kitchen iPad"
```
Notes:
- `nodes rename` stores a display name override in the gateway pairing store.
## Invoking commands
Low-level (raw RPC):
```bash
clawdbot nodes invoke --node <idOrNameOrIp> --command canvas.eval --params '{"javaScript":"location.href"}'
```
Higher-level helpers exist for the common “give the agent a MEDIA attachment” workflows.
## Screenshots (canvas snapshots)
If the node is showing the Canvas (WebView), `canvas.snapshot` returns `{ format, base64 }`.
CLI helper (writes to a temp file and prints `MEDIA:<path>`):
```bash
clawdbot nodes canvas snapshot --node <idOrNameOrIp> --format png
clawdbot nodes canvas snapshot --node <idOrNameOrIp> --format jpg --max-width 1200 --quality 0.9
```
Simple shortcut (auto-picks a single connected node if possible):
```bash
clawdbot canvas snapshot --format png
clawdbot canvas snapshot --format jpg --max-width 1200 --quality 0.9
```
## Photos + videos (node camera)
Photos (`jpg`):
```bash
clawdbot nodes camera snap --node <idOrNameOrIp> # default: both facings (2 MEDIA lines)
clawdbot nodes camera snap --node <idOrNameOrIp> --facing front
```
Video clips (`mp4`):
```bash
clawdbot nodes camera clip --node <idOrNameOrIp> --duration 10s
clawdbot nodes camera clip --node <idOrNameOrIp> --duration 3000 --no-audio
```
Notes:
- The node must be **foregrounded** for `canvas.*` and `camera.*` (background calls return `NODE_BACKGROUND_UNAVAILABLE`).
- Clip duration is clamped (currently `<= 60s`) to avoid oversized base64 payloads.
- Android will prompt for `CAMERA`/`RECORD_AUDIO` permissions when possible; denied permissions fail with `*_PERMISSION_REQUIRED`.
## Screen recordings (nodes)
Nodes expose `screen.record` (mp4). Example:
```bash
clawdbot nodes screen record --node <idOrNameOrIp> --duration 10s --fps 10
clawdbot nodes screen record --node <idOrNameOrIp> --duration 10s --fps 10 --no-audio
```
Notes:
- `screen.record` requires the node app to be foregrounded.
- Android will show the system screen-capture prompt before recording.
- Screen recordings are clamped to `<= 60s`.
- `--no-audio` disables microphone capture (supported on iOS/Android; macOS uses system capture audio).
## Location (nodes)
Nodes expose `location.get` when Location is enabled in settings.
CLI helper:
```bash
clawdbot nodes location get --node <idOrNameOrIp>
clawdbot nodes location get --node <idOrNameOrIp> --accuracy precise --max-age 15000 --location-timeout 10000
```
Notes:
- Location is **off by default**.
- “Always” requires system permission; background fetch is best-effort.
- The response includes lat/lon, accuracy (meters), and timestamp.
## SMS (Android nodes)
Android nodes can expose `sms.send` when the user grants **SMS** permission and the device supports telephony.
Low-level invoke:
```bash
clawdbot nodes invoke --node <idOrNameOrIp> --command sms.send --params '{"to":"+15555550123","message":"Hello from Clawdbot"}'
```
Notes:
- The permission prompt must be accepted on the Android device before the capability is advertised.
- Wi-Fi-only devices without telephony will not advertise `sms.send`.
## System commands (mac node)
The macOS node exposes `system.run` and `system.notify`.
Examples:
```bash
clawdbot nodes run --node <idOrNameOrIp> -- echo "Hello from mac node"
clawdbot nodes notify --node <idOrNameOrIp> --title "Ping" --body "Gateway ready"
```
Notes:
- `system.run` returns stdout/stderr/exit code in the payload.
- `system.notify` respects notification permission state on the macOS app.
## Permissions map
Nodes may include a `permissions` map in `node.list` / `node.describe`, keyed by permission name (e.g. `screenRecording`, `accessibility`) with boolean values (`true` = granted).
## Mac node mode
- The macOS menubar app connects to the Gateway bridge as a node (so `clawdbot nodes …` works against this Mac).
- In remote mode, the app opens an SSH tunnel for the bridge port and connects to `localhost`.
## Where to look in code
- CLI wiring: [`src/cli/nodes-cli.ts`](https://github.com/clawdbot/clawdbot/blob/main/src/cli/nodes-cli.ts)
- Canvas snapshot decoding/temp paths: [`src/cli/nodes-canvas.ts`](https://github.com/clawdbot/clawdbot/blob/main/src/cli/nodes-canvas.ts)
- Duration parsing for CLI: [`src/cli/parse-duration.ts`](https://github.com/clawdbot/clawdbot/blob/main/src/cli/parse-duration.ts)
- iOS node commands: [`apps/ios/Sources/Model/NodeAppModel.swift`](https://github.com/clawdbot/clawdbot/blob/main/apps/ios/Sources/Model/NodeAppModel.swift)
- Android node commands: `apps/android/app/src/main/java/com/clawdbot/android/node/*`