Docs: add nodes overview

This commit is contained in:
Peter Steinberger
2025-12-19 00:29:42 +01:00
parent 3772a29557
commit d6c5ee86c5
3 changed files with 91 additions and 2 deletions

View File

@@ -96,7 +96,7 @@ Like `canvas.*`, the Android node only allows `camera.*` commands in the **foreg
The macOS companion app exposes a checkbox:
- **Settings → Debug → Camera → Allow Camera (agent)** (`clawdis.cameraEnabled`)
- **Settings → General → Allow Camera** (`clawdis.cameraEnabled`)
- Default: **off**
- When off: camera requests return “Camera disabled by user”.
@@ -109,7 +109,8 @@ Examples:
```bash
clawdis-mac camera snap # prints MEDIA:<path>
clawdis-mac camera snap --max-width 1280
clawdis-mac camera clip --duration-ms 3000 # prints MEDIA:<path>
clawdis-mac camera clip --duration 10s # prints MEDIA:<path>
clawdis-mac camera clip --duration-ms 3000 # prints MEDIA:<path> (legacy flag)
clawdis-mac camera clip --no-audio
```
@@ -120,3 +121,14 @@ Notes:
- Camera and microphone access trigger the usual OS permission prompts (and require usage strings in Info.plist).
- Video clips are capped (currently `<= 60s`) to avoid oversized bridge payloads (base64 overhead + message limits).
## macOS screen video (OS-level)
For *screen* video (not camera), use the macOS companion:
```bash
clawdis-mac screen record --duration 10s --fps 15 # prints MEDIA:<path>
```
Notes:
- Requires macOS **Screen Recording** permission (TCC).

View File

@@ -118,6 +118,7 @@ Example:
- [Clawd personal assistant setup](./clawd.md)
- [AGENTS.md template (default)](./AGENTS.default.md)
- [Gateway runbook](./gateway.md)
- [Nodes (iOS/Android)](./nodes.md)
- [Web surfaces (Control UI)](./web.md)
- [Discovery + transports](./discovery.md)
- [Remote access](./remote.md)

76
docs/nodes.md Normal file
View File

@@ -0,0 +1,76 @@
---
summary: "Nodes: pairing, capabilities (canvas/camera), and the CLI helpers for screenshots + clips"
read_when:
- Pairing iOS/Android nodes to a gateway
- Using node canvas/camera for agent context
- Adding new node commands or CLI helpers
---
# Nodes
A **node** is a companion device (iOS/Android today) that connects to the Gateway over the **Bridge** and exposes a small command surface (e.g. `canvas.*`, `camera.*`) via `node.invoke`.
## Pairing + status
Pairing is gateway-owned and approval-based. See `docs/gateway/pairing.md` for the full flow.
Quick CLI:
```bash
clawdis nodes pending
clawdis nodes approve <requestId>
clawdis nodes reject <requestId>
clawdis nodes status
clawdis nodes describe --node <idOrNameOrIp>
```
## Invoking commands
Low-level (raw RPC):
```bash
clawdis nodes invoke --node <idOrNameOrIp> --command canvas.eval --params '{"javaScript":"location.href"}'
```
Higher-level helpers exist for the common “give the agent a MEDIA attachment” workflows.
## Screenshots (canvas snapshots)
If the node is showing the Canvas (WebView), `canvas.snapshot` returns `{ format, base64 }`.
CLI helper (writes to a temp file and prints `MEDIA:<path>`):
```bash
clawdis nodes canvas snapshot --node <idOrNameOrIp> --format png
clawdis nodes canvas snapshot --node <idOrNameOrIp> --format jpg --max-width 1200 --quality 0.9
```
## Photos + videos (node camera)
Photos (`jpg`):
```bash
clawdis nodes camera snap --node <idOrNameOrIp> # default: both facings (2 MEDIA lines)
clawdis nodes camera snap --node <idOrNameOrIp> --facing front
```
Video clips (`mp4`):
```bash
clawdis nodes camera clip --node <idOrNameOrIp> --duration 10s
clawdis nodes camera clip --node <idOrNameOrIp> --duration 3000 --no-audio
```
Notes:
- The node must be **foregrounded** for `canvas.*` and `camera.*` (background calls return `NODE_BACKGROUND_UNAVAILABLE`).
- Clip duration is clamped (currently `<= 60s`) to avoid oversized base64 payloads.
- Android will prompt for `CAMERA`/`RECORD_AUDIO` permissions when possible; denied permissions fail with `*_PERMISSION_REQUIRED`.
## Where to look in code
- CLI wiring: `src/cli/nodes-cli.ts`
- Canvas snapshot decoding/temp paths: `src/cli/nodes-canvas.ts`
- Duration parsing for CLI: `src/cli/parse-duration.ts`
- iOS node commands: `apps/ios/Sources/Model/NodeAppModel.swift`
- Android node commands: `apps/android/app/src/main/java/com/steipete/clawdis/node/node/*`