CLI/docs: expose node metadata and commands
This commit is contained in:
@@ -51,6 +51,10 @@ pnpm clawdis gateway --force
|
|||||||
- `system-event` — post a presence/system note (structured).
|
- `system-event` — post a presence/system note (structured).
|
||||||
- `send` — send a message via the active provider(s).
|
- `send` — send a message via the active provider(s).
|
||||||
- `agent` — run an agent turn (streams events back on same connection).
|
- `agent` — run an agent turn (streams events back on same connection).
|
||||||
|
- `node.list` — list paired + currently-connected bridge nodes (includes `caps`, `deviceFamily`, `modelIdentifier`, `paired`, `connected`, and advertised `commands`).
|
||||||
|
- `node.describe` — describe a node (capabilities + supported `node.invoke` commands; works for paired nodes and for currently-connected unpaired nodes).
|
||||||
|
- `node.invoke` — invoke a command on a node (e.g. `canvas.*`, `camera.*`).
|
||||||
|
- `node.pair.*` — pairing lifecycle (`request`, `list`, `approve`, `reject`, `verify`).
|
||||||
|
|
||||||
See also: `docs/presence.md` for how presence is produced/deduped and why `instanceId` matters.
|
See also: `docs/presence.md` for how presence is produced/deduped and why `instanceId` matters.
|
||||||
|
|
||||||
|
|||||||
@@ -1,11 +1,11 @@
|
|||||||
---
|
---
|
||||||
summary: "Plan for an iOS voice + screen (Canvas) node that connects via a secure Bonjour-discovered macOS bridge"
|
summary: "Plan for an iOS voice + canvas node that connects via a secure Bonjour-discovered macOS bridge"
|
||||||
read_when:
|
read_when:
|
||||||
- Designing iOS node + gateway integration
|
- Designing iOS node + gateway integration
|
||||||
- Extending the Gateway protocol for node/screen commands
|
- Extending the Gateway protocol for node/canvas commands
|
||||||
- Implementing Bonjour pairing or transport security
|
- Implementing Bonjour pairing or transport security
|
||||||
---
|
---
|
||||||
# iOS Node (internal) — Voice Trigger + Screen/Canvas
|
# iOS Node (internal) — Voice Trigger + Canvas
|
||||||
|
|
||||||
Status: prototype implemented (internal) · Date: 2025-12-13
|
Status: prototype implemented (internal) · Date: 2025-12-13
|
||||||
|
|
||||||
@@ -14,13 +14,13 @@ Runbook (how to connect/pair + drive Canvas): `docs/ios/connect.md`
|
|||||||
## Goals
|
## Goals
|
||||||
- Build an **iOS app** that acts as a **remote node** for Clawdis:
|
- Build an **iOS app** that acts as a **remote node** for Clawdis:
|
||||||
- **Voice trigger** (wake-word / always-listening intent) that forwards transcripts to the Gateway `agent` method.
|
- **Voice trigger** (wake-word / always-listening intent) that forwards transcripts to the Gateway `agent` method.
|
||||||
- **Screen/Canvas** surface that the agent can control: navigate, draw/render, evaluate JS, snapshot.
|
- **Canvas** surface that the agent can control: navigate, draw/render, evaluate JS, snapshot.
|
||||||
- **Dead-simple setup**:
|
- **Dead-simple setup**:
|
||||||
- Auto-discover the host on the local network via **Bonjour**.
|
- Auto-discover the host on the local network via **Bonjour**.
|
||||||
- One-tap pairing with an approval prompt on the Mac.
|
- One-tap pairing with an approval prompt on the Mac.
|
||||||
- iOS is **never** a local gateway; it is always a remote node.
|
- iOS is **never** a local gateway; it is always a remote node.
|
||||||
- Operational clarity:
|
- Operational clarity:
|
||||||
- When iOS is backgrounded, voice may still run; **screen/canvas commands must fail fast** with a structured error.
|
- When iOS is backgrounded, voice may still run; **canvas commands must fail fast** with a structured error.
|
||||||
- Provide **settings**: node display name, enable/disable voice wake, pairing status.
|
- Provide **settings**: node display name, enable/disable voice wake, pairing status.
|
||||||
|
|
||||||
Non-goals (v1):
|
Non-goals (v1):
|
||||||
@@ -41,7 +41,7 @@ Keep the Node gateway loopback-only; expose a dedicated **gateway-owned bridge**
|
|||||||
Why:
|
Why:
|
||||||
- Preserves current threat model: Gateway remains local-only.
|
- Preserves current threat model: Gateway remains local-only.
|
||||||
- Centralizes auth, rate limiting, and allowlisting in the bridge.
|
- Centralizes auth, rate limiting, and allowlisting in the bridge.
|
||||||
- Lets us unify “screen node” semantics across mac + iOS without exposing raw gateway methods.
|
- Lets us unify “canvas node” semantics across mac + iOS without exposing raw gateway methods.
|
||||||
|
|
||||||
## Security plan (internal, but still robust)
|
## Security plan (internal, but still robust)
|
||||||
### Transport
|
### Transport
|
||||||
@@ -84,7 +84,7 @@ The bridge must not be a raw proxy to every gateway method.
|
|||||||
- Allow by default:
|
- Allow by default:
|
||||||
- `agent` (with guardrails; idempotency required)
|
- `agent` (with guardrails; idempotency required)
|
||||||
- minimal `system-event` beacons (presence updates for the node)
|
- minimal `system-event` beacons (presence updates for the node)
|
||||||
- node/screen methods defined below (new protocol surface)
|
- node/canvas methods defined below (new protocol surface)
|
||||||
- Deny by default:
|
- Deny by default:
|
||||||
- anything that widens control without explicit intent (future “shell”, “files”, etc.)
|
- anything that widens control without explicit intent (future “shell”, “files”, etc.)
|
||||||
- Rate limit:
|
- Rate limit:
|
||||||
@@ -92,7 +92,7 @@ The bridge must not be a raw proxy to every gateway method.
|
|||||||
- voice forwards per minute
|
- voice forwards per minute
|
||||||
- snapshot frequency / payload size
|
- snapshot frequency / payload size
|
||||||
|
|
||||||
## Protocol unification: add “node/screen” to Gateway protocol
|
## Protocol unification: add “node/canvas” to Gateway protocol
|
||||||
### Principle
|
### Principle
|
||||||
Unify mac Canvas + iOS Canvas under a single conceptual surface:
|
Unify mac Canvas + iOS Canvas under a single conceptual surface:
|
||||||
- The agent talks to the Gateway using a stable method set (typed protocol).
|
- The agent talks to the Gateway using a stable method set (typed protocol).
|
||||||
@@ -108,14 +108,15 @@ Add to `src/gateway/protocol/schema.ts` (and regenerate Swift models):
|
|||||||
|
|
||||||
**Methods**
|
**Methods**
|
||||||
- `node.list` → list paired/connected nodes + capabilities
|
- `node.list` → list paired/connected nodes + capabilities
|
||||||
|
- `node.describe` → describe a node (capabilities + supported `node.invoke` commands)
|
||||||
- `node.invoke` → send a command to a specific node
|
- `node.invoke` → send a command to a specific node
|
||||||
- Params: `{ nodeId, command, params?, timeoutMs? }`
|
- Params: `{ nodeId, command, params?, timeoutMs? }`
|
||||||
|
|
||||||
**Events**
|
**Events**
|
||||||
- `node.event` → async node status/errors
|
- `node.event` → async node status/errors
|
||||||
- e.g. background/foreground transitions, voice availability, screen availability
|
- e.g. background/foreground transitions, voice availability, canvas availability
|
||||||
|
|
||||||
### Node command set (screen-focused)
|
### Node command set (canvas)
|
||||||
These are values for `node.invoke.command`:
|
These are values for `node.invoke.command`:
|
||||||
- `canvas.show` / `canvas.hide`
|
- `canvas.show` / `canvas.hide`
|
||||||
- `canvas.navigate` with `{ url }` (Canvas URL or https URL)
|
- `canvas.navigate` with `{ url }` (Canvas URL or https URL)
|
||||||
@@ -153,8 +154,8 @@ When iOS is backgrounded:
|
|||||||
- `BridgeConnection`: TCP session + pairing handshake + reconnect (TLS planned)
|
- `BridgeConnection`: TCP session + pairing handshake + reconnect (TLS planned)
|
||||||
- `NodeRuntime`:
|
- `NodeRuntime`:
|
||||||
- Voice pipeline (wake-word + capture + forward)
|
- Voice pipeline (wake-word + capture + forward)
|
||||||
- Screen pipeline (WKWebView controller + snapshot + eval)
|
- Canvas pipeline (WKWebView controller + snapshot + eval)
|
||||||
- Background state tracking; enforces “screen unavailable in background”
|
- Background state tracking; enforces “canvas unavailable in background”
|
||||||
|
|
||||||
### Voice in background (internal)
|
### Voice in background (internal)
|
||||||
- Enable background audio mode (and required session configuration) so the mic pipeline can keep running when the user switches apps.
|
- Enable background audio mode (and required session configuration) so the mic pipeline can keep running when the user switches apps.
|
||||||
@@ -164,7 +165,7 @@ When iOS is backgrounded:
|
|||||||
Create/expand SwiftPM targets so both apps share:
|
Create/expand SwiftPM targets so both apps share:
|
||||||
- `ClawdisProtocol` (generated models; platform-neutral)
|
- `ClawdisProtocol` (generated models; platform-neutral)
|
||||||
- `ClawdisGatewayClient` (shared WS framing + connect/req/res + seq-gap handling)
|
- `ClawdisGatewayClient` (shared WS framing + connect/req/res + seq-gap handling)
|
||||||
- `ClawdisKit` (node/screen command types + deep links + shared utilities)
|
- `ClawdisKit` (node/canvas command types + deep links + shared utilities)
|
||||||
|
|
||||||
macOS continues to own:
|
macOS continues to own:
|
||||||
- local Canvas implementation details (custom scheme handler serving on-disk HTML, window/panel presentation)
|
- local Canvas implementation details (custom scheme handler serving on-disk HTML, window/panel presentation)
|
||||||
@@ -217,8 +218,8 @@ open Clawdis.xcodeproj
|
|||||||
3) **Protocol additions for nodes**
|
3) **Protocol additions for nodes**
|
||||||
- Add `node.list` / `node.invoke` / `node.event` to Gateway
|
- Add `node.list` / `node.invoke` / `node.event` to Gateway
|
||||||
- Implement bridge routing + ACLs
|
- Implement bridge routing + ACLs
|
||||||
4) **iOS screen/canvas**
|
4) **iOS canvas**
|
||||||
- WKWebView screen surface
|
- WKWebView canvas surface
|
||||||
- `canvas.navigate/eval/snapshot`
|
- `canvas.navigate/eval/snapshot`
|
||||||
- Background fast-fail for `canvas.*`
|
- Background fast-fail for `canvas.*`
|
||||||
5) **Unify mac Canvas under the same node.invoke**
|
5) **Unify mac Canvas under the same node.invoke**
|
||||||
@@ -226,5 +227,5 @@ open Clawdis.xcodeproj
|
|||||||
|
|
||||||
## Open questions
|
## Open questions
|
||||||
- Should `connect.params.client.mode` be `"node"` with `platform="ios ..."` or a distinct mode `"ios-node"`? (Presence filtering currently excludes `"cli"` only.)
|
- Should `connect.params.client.mode` be `"node"` with `platform="ios ..."` or a distinct mode `"ios-node"`? (Presence filtering currently excludes `"cli"` only.)
|
||||||
- Do we want a “permissions” model per node (voice only vs voice+screen) at pairing time?
|
- Do we want a “permissions” model per node (voice only vs voice+canvas) at pairing time?
|
||||||
- Should “website mode” allow arbitrary https, or enforce an allowlist to reduce risk?
|
- Should “website mode” allow arbitrary https, or enforce an allowlist to reduce risk?
|
||||||
|
|||||||
@@ -35,6 +35,8 @@ type NodeListNode = {
|
|||||||
deviceFamily?: string;
|
deviceFamily?: string;
|
||||||
modelIdentifier?: string;
|
modelIdentifier?: string;
|
||||||
caps?: string[];
|
caps?: string[];
|
||||||
|
commands?: string[];
|
||||||
|
paired?: boolean;
|
||||||
connected?: boolean;
|
connected?: boolean;
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -183,18 +185,25 @@ export function registerNodesCli(program: Command) {
|
|||||||
nodesCallOpts(
|
nodesCallOpts(
|
||||||
nodes
|
nodes
|
||||||
.command("status")
|
.command("status")
|
||||||
.description("List paired nodes with connection status and capabilities")
|
.description("List known nodes with connection status and capabilities")
|
||||||
.action(async (opts: NodesRpcOpts) => {
|
.action(async (opts: NodesRpcOpts) => {
|
||||||
try {
|
try {
|
||||||
const result = (await callGatewayCli("node.list", opts, {})) as unknown;
|
const result = (await callGatewayCli(
|
||||||
|
"node.list",
|
||||||
|
opts,
|
||||||
|
{},
|
||||||
|
)) as unknown;
|
||||||
if (opts.json) {
|
if (opts.json) {
|
||||||
defaultRuntime.log(JSON.stringify(result, null, 2));
|
defaultRuntime.log(JSON.stringify(result, null, 2));
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
const nodes = parseNodeList(result);
|
const nodes = parseNodeList(result);
|
||||||
const connectedCount = nodes.filter((n) => Boolean(n.connected)).length;
|
const pairedCount = nodes.filter((n) => Boolean(n.paired)).length;
|
||||||
|
const connectedCount = nodes.filter((n) =>
|
||||||
|
Boolean(n.connected),
|
||||||
|
).length;
|
||||||
defaultRuntime.log(
|
defaultRuntime.log(
|
||||||
`Paired: ${nodes.length} · Connected: ${connectedCount}`,
|
`Known: ${nodes.length} · Paired: ${pairedCount} · Connected: ${connectedCount}`,
|
||||||
);
|
);
|
||||||
for (const n of nodes) {
|
for (const n of nodes) {
|
||||||
const name = n.displayName || n.nodeId;
|
const name = n.displayName || n.nodeId;
|
||||||
@@ -207,8 +216,9 @@ export function registerNodesCli(program: Command) {
|
|||||||
: Array.isArray(n.caps)
|
: Array.isArray(n.caps)
|
||||||
? "[]"
|
? "[]"
|
||||||
: "?";
|
: "?";
|
||||||
|
const pairing = n.paired ? "paired" : "unpaired";
|
||||||
defaultRuntime.log(
|
defaultRuntime.log(
|
||||||
`- ${name} · ${n.nodeId}${ip}${device}${hw} · ${n.connected ? "connected" : "disconnected"} · caps: ${caps}`,
|
`- ${name} · ${n.nodeId}${ip}${device}${hw} · ${pairing} · ${n.connected ? "connected" : "disconnected"} · caps: ${caps}`,
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
@@ -218,6 +228,63 @@ export function registerNodesCli(program: Command) {
|
|||||||
}),
|
}),
|
||||||
);
|
);
|
||||||
|
|
||||||
|
nodesCallOpts(
|
||||||
|
nodes
|
||||||
|
.command("describe")
|
||||||
|
.description("Describe a node (capabilities + supported invoke commands)")
|
||||||
|
.requiredOption("--node <idOrNameOrIp>", "Node id, name, or IP")
|
||||||
|
.action(async (opts: NodesRpcOpts) => {
|
||||||
|
try {
|
||||||
|
const nodeId = await resolveNodeId(opts, String(opts.node ?? ""));
|
||||||
|
const result = (await callGatewayCli("node.describe", opts, {
|
||||||
|
nodeId,
|
||||||
|
})) as unknown;
|
||||||
|
if (opts.json) {
|
||||||
|
defaultRuntime.log(JSON.stringify(result, null, 2));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const obj =
|
||||||
|
typeof result === "object" && result !== null
|
||||||
|
? (result as Record<string, unknown>)
|
||||||
|
: {};
|
||||||
|
const displayName =
|
||||||
|
typeof obj.displayName === "string" ? obj.displayName : nodeId;
|
||||||
|
const connected = Boolean(obj.connected);
|
||||||
|
const caps = Array.isArray(obj.caps)
|
||||||
|
? obj.caps.map(String).filter(Boolean).sort()
|
||||||
|
: null;
|
||||||
|
const commands = Array.isArray(obj.commands)
|
||||||
|
? obj.commands.map(String).filter(Boolean).sort()
|
||||||
|
: [];
|
||||||
|
const family =
|
||||||
|
typeof obj.deviceFamily === "string" ? obj.deviceFamily : null;
|
||||||
|
const model =
|
||||||
|
typeof obj.modelIdentifier === "string"
|
||||||
|
? obj.modelIdentifier
|
||||||
|
: null;
|
||||||
|
const ip = typeof obj.remoteIp === "string" ? obj.remoteIp : null;
|
||||||
|
|
||||||
|
const parts: string[] = ["Node:", displayName, nodeId];
|
||||||
|
if (ip) parts.push(ip);
|
||||||
|
if (family) parts.push(`device: ${family}`);
|
||||||
|
if (model) parts.push(`hw: ${model}`);
|
||||||
|
parts.push(connected ? "connected" : "disconnected");
|
||||||
|
parts.push(`caps: ${caps ? `[${caps.join(",")}]` : "?"}`);
|
||||||
|
defaultRuntime.log(parts.join(" · "));
|
||||||
|
defaultRuntime.log("Commands:");
|
||||||
|
if (commands.length === 0) {
|
||||||
|
defaultRuntime.log("- (none reported)");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
for (const c of commands) defaultRuntime.log(`- ${c}`);
|
||||||
|
} catch (err) {
|
||||||
|
defaultRuntime.error(`nodes describe failed: ${String(err)}`);
|
||||||
|
defaultRuntime.exit(1);
|
||||||
|
}
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
|
||||||
nodesCallOpts(
|
nodesCallOpts(
|
||||||
nodes
|
nodes
|
||||||
.command("list")
|
.command("list")
|
||||||
@@ -345,10 +412,7 @@ export function registerNodesCli(program: Command) {
|
|||||||
.command("invoke")
|
.command("invoke")
|
||||||
.description("Invoke a command on a paired node")
|
.description("Invoke a command on a paired node")
|
||||||
.requiredOption("--node <idOrNameOrIp>", "Node id, name, or IP")
|
.requiredOption("--node <idOrNameOrIp>", "Node id, name, or IP")
|
||||||
.requiredOption(
|
.requiredOption("--command <command>", "Command (e.g. canvas.eval)")
|
||||||
"--command <command>",
|
|
||||||
"Command (e.g. canvas.eval)",
|
|
||||||
)
|
|
||||||
.option("--params <json>", "JSON object string for params", "{}")
|
.option("--params <json>", "JSON object string for params", "{}")
|
||||||
.option(
|
.option(
|
||||||
"--invoke-timeout <ms>",
|
"--invoke-timeout <ms>",
|
||||||
|
|||||||
@@ -73,6 +73,7 @@ describe("cli program", () => {
|
|||||||
deviceFamily: "iPad",
|
deviceFamily: "iPad",
|
||||||
modelIdentifier: "iPad16,6",
|
modelIdentifier: "iPad16,6",
|
||||||
caps: ["canvas", "camera"],
|
caps: ["canvas", "camera"],
|
||||||
|
paired: true,
|
||||||
connected: true,
|
connected: true,
|
||||||
},
|
},
|
||||||
],
|
],
|
||||||
@@ -85,14 +86,64 @@ describe("cli program", () => {
|
|||||||
expect.objectContaining({ method: "node.list", params: {} }),
|
expect.objectContaining({ method: "node.list", params: {} }),
|
||||||
);
|
);
|
||||||
|
|
||||||
const output = runtime.log.mock.calls.map((c) => String(c[0] ?? "")).join("\n");
|
const output = runtime.log.mock.calls
|
||||||
expect(output).toContain("Paired: 1 · Connected: 1");
|
.map((c) => String(c[0] ?? ""))
|
||||||
|
.join("\n");
|
||||||
|
expect(output).toContain("Known: 1 · Paired: 1 · Connected: 1");
|
||||||
expect(output).toContain("iOS Node");
|
expect(output).toContain("iOS Node");
|
||||||
expect(output).toContain("device: iPad");
|
expect(output).toContain("device: iPad");
|
||||||
expect(output).toContain("hw: iPad16,6");
|
expect(output).toContain("hw: iPad16,6");
|
||||||
|
expect(output).toContain("paired");
|
||||||
expect(output).toContain("caps: [camera,canvas]");
|
expect(output).toContain("caps: [camera,canvas]");
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("runs nodes describe and calls node.describe", async () => {
|
||||||
|
callGateway
|
||||||
|
.mockResolvedValueOnce({
|
||||||
|
ts: Date.now(),
|
||||||
|
nodes: [
|
||||||
|
{
|
||||||
|
nodeId: "ios-node",
|
||||||
|
displayName: "iOS Node",
|
||||||
|
remoteIp: "192.168.0.88",
|
||||||
|
connected: true,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
})
|
||||||
|
.mockResolvedValueOnce({
|
||||||
|
ts: Date.now(),
|
||||||
|
nodeId: "ios-node",
|
||||||
|
displayName: "iOS Node",
|
||||||
|
caps: ["canvas", "camera"],
|
||||||
|
commands: ["canvas.eval", "canvas.snapshot", "camera.snap"],
|
||||||
|
connected: true,
|
||||||
|
});
|
||||||
|
|
||||||
|
const program = buildProgram();
|
||||||
|
runtime.log.mockClear();
|
||||||
|
await program.parseAsync(["nodes", "describe", "--node", "ios-node"], {
|
||||||
|
from: "user",
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(callGateway).toHaveBeenNthCalledWith(
|
||||||
|
1,
|
||||||
|
expect.objectContaining({ method: "node.list", params: {} }),
|
||||||
|
);
|
||||||
|
expect(callGateway).toHaveBeenNthCalledWith(
|
||||||
|
2,
|
||||||
|
expect.objectContaining({
|
||||||
|
method: "node.describe",
|
||||||
|
params: { nodeId: "ios-node" },
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
|
||||||
|
const out = runtime.log.mock.calls
|
||||||
|
.map((c) => String(c[0] ?? ""))
|
||||||
|
.join("\n");
|
||||||
|
expect(out).toContain("Commands:");
|
||||||
|
expect(out).toContain("canvas.eval");
|
||||||
|
});
|
||||||
|
|
||||||
it("runs nodes approve and calls node.pair.approve", async () => {
|
it("runs nodes approve and calls node.pair.approve", async () => {
|
||||||
callGateway.mockResolvedValue({
|
callGateway.mockResolvedValue({
|
||||||
requestId: "r1",
|
requestId: "r1",
|
||||||
|
|||||||
Reference in New Issue
Block a user