diff --git a/docs/camera.md b/docs/camera.md index 0bda1aba5..8e9f499b4 100644 --- a/docs/camera.md +++ b/docs/camera.md @@ -96,7 +96,7 @@ Like `canvas.*`, the Android node only allows `camera.*` commands in the **foreg The macOS companion app exposes a checkbox: -- **Settings → Debug → Camera → Allow Camera (agent)** (`clawdis.cameraEnabled`) +- **Settings → General → Allow Camera** (`clawdis.cameraEnabled`) - Default: **off** - When off: camera requests return “Camera disabled by user”. @@ -109,7 +109,8 @@ Examples: ```bash clawdis-mac camera snap # prints MEDIA: clawdis-mac camera snap --max-width 1280 -clawdis-mac camera clip --duration-ms 3000 # prints MEDIA: +clawdis-mac camera clip --duration 10s # prints MEDIA: +clawdis-mac camera clip --duration-ms 3000 # prints MEDIA: (legacy flag) clawdis-mac camera clip --no-audio ``` @@ -120,3 +121,14 @@ Notes: - Camera and microphone access trigger the usual OS permission prompts (and require usage strings in Info.plist). - Video clips are capped (currently `<= 60s`) to avoid oversized bridge payloads (base64 overhead + message limits). + +## macOS screen video (OS-level) + +For *screen* video (not camera), use the macOS companion: + +```bash +clawdis-mac screen record --duration 10s --fps 15 # prints MEDIA: +``` + +Notes: +- Requires macOS **Screen Recording** permission (TCC). diff --git a/docs/index.md b/docs/index.md index be4cf6434..6d93c40ac 100644 --- a/docs/index.md +++ b/docs/index.md @@ -118,6 +118,7 @@ Example: - [Clawd personal assistant setup](./clawd.md) - [AGENTS.md template (default)](./AGENTS.default.md) - [Gateway runbook](./gateway.md) + - [Nodes (iOS/Android)](./nodes.md) - [Web surfaces (Control UI)](./web.md) - [Discovery + transports](./discovery.md) - [Remote access](./remote.md) diff --git a/docs/nodes.md b/docs/nodes.md new file mode 100644 index 000000000..2a078ea0d --- /dev/null +++ b/docs/nodes.md @@ -0,0 +1,76 @@ +--- +summary: "Nodes: pairing, capabilities (canvas/camera), and the CLI helpers for screenshots + clips" +read_when: + - Pairing iOS/Android nodes to a gateway + - Using node canvas/camera for agent context + - Adding new node commands or CLI helpers +--- + +# Nodes + +A **node** is a companion device (iOS/Android today) that connects to the Gateway over the **Bridge** and exposes a small command surface (e.g. `canvas.*`, `camera.*`) via `node.invoke`. + +## Pairing + status + +Pairing is gateway-owned and approval-based. See `docs/gateway/pairing.md` for the full flow. + +Quick CLI: + +```bash +clawdis nodes pending +clawdis nodes approve +clawdis nodes reject +clawdis nodes status +clawdis nodes describe --node +``` + +## Invoking commands + +Low-level (raw RPC): + +```bash +clawdis nodes invoke --node --command canvas.eval --params '{"javaScript":"location.href"}' +``` + +Higher-level helpers exist for the common “give the agent a MEDIA attachment” workflows. + +## Screenshots (canvas snapshots) + +If the node is showing the Canvas (WebView), `canvas.snapshot` returns `{ format, base64 }`. + +CLI helper (writes to a temp file and prints `MEDIA:`): + +```bash +clawdis nodes canvas snapshot --node --format png +clawdis nodes canvas snapshot --node --format jpg --max-width 1200 --quality 0.9 +``` + +## Photos + videos (node camera) + +Photos (`jpg`): + +```bash +clawdis nodes camera snap --node # default: both facings (2 MEDIA lines) +clawdis nodes camera snap --node --facing front +``` + +Video clips (`mp4`): + +```bash +clawdis nodes camera clip --node --duration 10s +clawdis nodes camera clip --node --duration 3000 --no-audio +``` + +Notes: +- The node must be **foregrounded** for `canvas.*` and `camera.*` (background calls return `NODE_BACKGROUND_UNAVAILABLE`). +- Clip duration is clamped (currently `<= 60s`) to avoid oversized base64 payloads. +- Android will prompt for `CAMERA`/`RECORD_AUDIO` permissions when possible; denied permissions fail with `*_PERMISSION_REQUIRED`. + +## Where to look in code + +- CLI wiring: `src/cli/nodes-cli.ts` +- Canvas snapshot decoding/temp paths: `src/cli/nodes-canvas.ts` +- Duration parsing for CLI: `src/cli/parse-duration.ts` +- iOS node commands: `apps/ios/Sources/Model/NodeAppModel.swift` +- Android node commands: `apps/android/app/src/main/java/com/steipete/clawdis/node/node/*` +