Merge branch 'main' into feat/mattermost-channel

This commit is contained in:
Dominic Damoah
2026-01-22 18:17:40 -05:00
committed by GitHub
149 changed files with 5513 additions and 672 deletions

View File

@@ -7,7 +7,7 @@ read_when:
# Bonjour / mDNS discovery
Clawdbot uses Bonjour (mDNS / DNSSD) as a **LANonly convenience** to discover
an active Gateway bridge. It is besteffort and does **not** replace SSH or
an active Gateway (WebSocket endpoint). It is besteffort and does **not** replace SSH or
Tailnet-based connectivity.
## Widearea Bonjour (Unicast DNSSD) over Tailscale
@@ -31,7 +31,7 @@ browse both `local.` and `clawdbot.internal.` automatically.
```json5
{
bridge: { bind: "tailnet" }, // tailnet-only (recommended)
gateway: { bind: "tailnet" }, // tailnet-only (recommended)
discovery: { wideArea: { enabled: true } } // enables clawdbot.internal DNS-SD publishing
}
```
@@ -63,13 +63,13 @@ In the Tailscale admin console:
Once clients accept tailnet DNS, iOS nodes can browse
`_clawdbot-gw._tcp` in `clawdbot.internal.` without multicast.
### Bridge listener security (recommended)
### Gateway listener security (recommended)
The bridge port (default `18790`) is a plain TCP service. By default it binds to
`0.0.0.0`, which makes it reachable from any interface on the gateway host.
The Gateway WS port (default `18789`) binds to loopback by default. For LAN/tailnet
access, bind explicitly and keep auth enabled.
For tailnetonly setups:
- Set `bridge.bind: "tailnet"` in `~/.clawdbot/clawdbot.json`.
- Set `gateway.bind: "tailnet"` in `~/.clawdbot/clawdbot.json`.
- Restart the Gateway (or restart the macOS menubar app).
## What advertises
@@ -87,11 +87,12 @@ The Gateway advertises small nonsecret hints to make UI flows convenient:
- `role=gateway`
- `displayName=<friendly name>`
- `lanHost=<hostname>.local`
- `gatewayPort=<port>` (informational; Gateway WS is usually loopbackonly)
- `bridgePort=<port>` (only when bridge is enabled)
- `gatewayPort=<port>` (Gateway WS + HTTP)
- `gatewayTls=1` (only when TLS is enabled)
- `gatewayTlsSha256=<sha256>` (only when TLS is enabled and fingerprint is available)
- `canvasPort=<port>` (only when the canvas host is enabled; default `18793`)
- `sshPort=<port>` (defaults to 22 when not overridden)
- `transport=bridge`
- `transport=gateway`
- `cliPath=<path>` (optional; absolute path to a runnable `clawdbot` entrypoint)
- `tailnetDns=<magicdns>` (optional hint when Tailnet is available)
@@ -125,8 +126,8 @@ The Gateway writes a rolling log file (printed on startup as
The iOS node uses `NWBrowser` to discover `_clawdbot-gw._tcp`.
To capture logs:
- Settings → Bridge → Advanced → **Discovery Debug Logs**
- Settings → Bridge → Advanced → **Discovery Logs** → reproduce → **Copy**
- Settings → Gateway → Advanced → **Discovery Debug Logs**
- Settings → Gateway → Advanced → **Discovery Logs** → reproduce → **Copy**
The log includes browser state transitions and resultset changes.
@@ -136,7 +137,7 @@ The log includes browser state transitions and resultset changes.
- **Multicast blocked**: some WiFi networks disable mDNS.
- **Sleep / interface churn**: macOS may temporarily drop mDNS results; retry.
- **Browse works but resolve fails**: keep machine names simple (avoid emojis or
punctuation), then restart the Gateway. The bridge instance name derives from
punctuation), then restart the Gateway. The service instance name derives from
the host name, so overly complex names can confuse some resolvers.
## Escaped instance names (`\032`)
@@ -150,9 +151,7 @@ sequences (e.g. spaces become `\032`).
## Disabling / configuration
- `CLAWDBOT_DISABLE_BONJOUR=1` disables advertising.
- `CLAWDBOT_BRIDGE_ENABLED=0` disables the bridge listener (and the bridge beacon).
- `bridge.bind` / `bridge.port` in `~/.clawdbot/clawdbot.json` control bridge bind/port.
- `CLAWDBOT_BRIDGE_HOST` / `CLAWDBOT_BRIDGE_PORT` still work as backcompat overrides.
- `gateway.bind` in `~/.clawdbot/clawdbot.json` controls the Gateway bind mode.
- `CLAWDBOT_SSH_PORT` overrides the SSH port advertised in TXT.
- `CLAWDBOT_TAILNET_DNS` publishes a MagicDNS hint in TXT.
- `CLAWDBOT_CLI_PATH` overrides the advertised CLI path.

View File

@@ -14,6 +14,9 @@ should use the unified Gateway WebSocket protocol instead.
If you are building an operator or node client, use the
[Gateway protocol](/gateway/protocol).
**Note:** Current Clawdbot builds no longer ship the TCP bridge listener; this document is kept for historical reference.
Legacy `bridge.*` config keys are no longer part of the config schema.
## Why we have both
- **Security boundary**: the bridge exposes a small allowlist instead of the
@@ -28,7 +31,7 @@ If you are building an operator or node client, use the
- TCP, one JSON object per line (JSONL).
- Optional TLS (when `bridge.tls.enabled` is true).
- Gateway owns the listener (default `18790`).
- Legacy default listener port was `18790` (current builds do not start a TCP bridge).
When TLS is enabled, discovery TXT records include `bridgeTls=1` plus
`bridgeTlsSha256` so nodes can pin the certificate.
@@ -54,7 +57,7 @@ Gateway → Client:
- `event`: chat updates for subscribed sessions
- `ping` / `pong`: keepalive
Exact allowlist is enforced in `src/gateway/server-bridge.ts`.
Legacy allowlist enforcement lived in `src/gateway/server-bridge.ts` (removed).
## Exec lifecycle events

View File

@@ -568,5 +568,5 @@ Save to `~/.clawdbot/clawdbot.json` and you can DM the bot from that number.
- If you set `dmPolicy: "open"`, the matching `allowFrom` list must include `"*"`.
- Provider IDs differ (phone numbers, user IDs, channel IDs). Use the provider docs to confirm the format.
- Optional sections to add later: `web`, `browser`, `ui`, `bridge`, `discovery`, `canvasHost`, `talk`, `signal`, `imessage`.
- Optional sections to add later: `web`, `browser`, `ui`, `discovery`, `canvasHost`, `talk`, `signal`, `imessage`.
- See [Providers](/channels/whatsapp) and [Troubleshooting](/gateway/troubleshooting) for deeper setup notes.

View File

@@ -1719,7 +1719,7 @@ auto-compaction, instructing the model to store durable memories on disk (e.g.
`memory/YYYY-MM-DD.md`). It triggers when the session token estimate crosses a
soft threshold below the compaction limit.
Defaults:
Legacy defaults:
- `memoryFlush.enabled`: `true`
- `memoryFlush.softThresholdTokens`: `4000`
- `memoryFlush.prompt` / `memoryFlush.systemPrompt`: built-in defaults with `NO_REPLY`
@@ -2814,7 +2814,7 @@ Hot-applied (no full gateway restart):
Requires full Gateway restart:
- `gateway` (port/bind/auth/control UI/tailscale)
- `bridge`
- `bridge` (legacy)
- `discovery`
- `canvasHost`
- `plugins`
@@ -2832,7 +2832,7 @@ Convenience flags (CLI):
- `clawdbot --dev …` → uses `~/.clawdbot-dev` + shifts ports from base `19001`
- `clawdbot --profile <name> …` → uses `~/.clawdbot-<name>` (port via config/env/flags)
See [Gateway runbook](/gateway) for the derived port mapping (gateway/bridge/browser/canvas).
See [Gateway runbook](/gateway) for the derived port mapping (gateway/browser/canvas).
See [Multiple gateways](/gateway/multiple-gateways) for browser/CDP port isolation details.
Example:
@@ -2951,7 +2951,7 @@ The Gateway serves a directory of HTML/CSS/JS over HTTP so iOS/Android nodes can
Default root: `~/clawd/canvas`
Default port: `18793` (chosen to avoid the clawd browser CDP port `18792`)
The server listens on the **bridge bind host** (LAN or Tailnet) so nodes can reach it.
The server listens on the **gateway bind host** (LAN or Tailnet) so nodes can reach it.
The server:
- serves files under `canvasHost.root`
@@ -2980,9 +2980,13 @@ Disable with:
- config: `canvasHost: { enabled: false }`
- env: `CLAWDBOT_SKIP_CANVAS_HOST=1`
### `bridge` (node bridge server)
### `bridge` (legacy TCP bridge, removed)
The Gateway can expose a simple TCP bridge for nodes (iOS/Android), typically on port `18790`.
Current builds no longer include the TCP bridge listener; `bridge.*` config keys are ignored.
Nodes connect over the Gateway WebSocket. This section is kept for historical reference.
Legacy behavior:
- The Gateway could expose a simple TCP bridge for nodes (iOS/Android), typically on port `18790`.
Defaults:
- enabled: `true`

View File

@@ -3,7 +3,7 @@ summary: "Node discovery and transports (Bonjour, Tailscale, SSH) for finding th
read_when:
- Implementing or changing Bonjour discovery/advertising
- Adjusting remote connection modes (direct vs SSH)
- Designing bridge + pairing for remote nodes
- Designing node discovery + pairing for remote nodes
---
# Discovery & transports
@@ -17,17 +17,18 @@ The design goal is to keep all network discovery/advertising in the **Node Gatew
## Terms
- **Gateway**: a single long-running gateway process that owns state (sessions, pairing, node registry) and runs channels. Most setups use one per host; isolated multi-gateway setups are possible.
- **Gateway WS (loopback)**: the existing gateway WebSocket control endpoint on `127.0.0.1:18789`.
- **Bridge (direct transport)**: a LAN/tailnet-facing endpoint owned by the gateway that allows authenticated clients/nodes to call a scoped subset of gateway methods. The bridge exists so the gateway can remain loopback-only.
- **Gateway WS (control plane)**: the WebSocket endpoint on `127.0.0.1:18789` by default; can be bound to LAN/tailnet via `gateway.bind`.
- **Direct WS transport**: a LAN/tailnet-facing Gateway WS endpoint (no SSH).
- **SSH transport (fallback)**: remote control by forwarding `127.0.0.1:18789` over SSH.
- **Legacy TCP bridge (deprecated/removed)**: older node transport (see [Bridge protocol](/gateway/bridge-protocol)); no longer advertised for discovery.
Protocol details:
- [Gateway protocol](/gateway/protocol)
- [Bridge protocol](/gateway/bridge-protocol)
- [Bridge protocol (legacy)](/gateway/bridge-protocol)
## Why we keep both “direct” and SSH
- **Direct bridge** is the best UX on the same network and within a tailnet:
- **Direct WS** is the best UX on the same network and within a tailnet:
- auto-discovery on LAN via Bonjour
- pairing tokens + ACLs owned by the gateway
- no shell access required; protocol surface can stay tight and auditable
@@ -43,7 +44,7 @@ Protocol details:
Bonjour is best-effort and does not cross networks. It is only used for “same LAN” convenience.
Target direction:
- The **gateway** advertises its bridge via Bonjour.
- The **gateway** advertises its WS endpoint via Bonjour.
- Clients browse and show a “pick a gateway” list, then store the chosen endpoint.
Troubleshooting and beacon details: [Bonjour](/gateway/bonjour).
@@ -56,19 +57,19 @@ Troubleshooting and beacon details: [Bonjour](/gateway/bonjour).
- `role=gateway`
- `lanHost=<hostname>.local`
- `sshPort=22` (or whatever is advertised)
- `gatewayPort=18789` (loopback WS port; informational)
- `bridgePort=18790` (when bridge is enabled)
- `gatewayPort=18789` (Gateway WS + HTTP)
- `gatewayTls=1` (only when TLS is enabled)
- `gatewayTlsSha256=<sha256>` (only when TLS is enabled and fingerprint is available)
- `canvasPort=18793` (default canvas host port; serves `/__clawdbot__/canvas/`)
- `cliPath=<path>` (optional; absolute path to a runnable `clawdbot` entrypoint or binary)
- `tailnetDns=<magicdns>` (optional hint; auto-detected when Tailscale is available)
Disable/override:
- `CLAWDBOT_DISABLE_BONJOUR=1` disables advertising.
- `CLAWDBOT_BRIDGE_ENABLED=0` disables the bridge listener.
- `bridge.bind` / `bridge.port` in `~/.clawdbot/clawdbot.json` control bridge bind/port (preferred).
- `CLAWDBOT_BRIDGE_HOST` / `CLAWDBOT_BRIDGE_PORT` still work as a back-compat override when `bridge.bind` / `bridge.port` are not set.
- `CLAWDBOT_SSH_PORT` overrides the SSH port advertised in the bridge beacon (defaults to 22).
- `CLAWDBOT_TAILNET_DNS` publishes a `tailnetDns` hint (MagicDNS) in the bridge beacon (auto-detected if unset).
- `gateway.bind` in `~/.clawdbot/clawdbot.json` controls the Gateway bind mode.
- `CLAWDBOT_SSH_PORT` overrides the SSH port advertised in TXT (defaults to 22).
- `CLAWDBOT_TAILNET_DNS` publishes a `tailnetDns` hint (MagicDNS).
- `CLAWDBOT_CLI_PATH` overrides the advertised CLI path.
### 2) Tailnet (cross-network)
@@ -97,13 +98,13 @@ Recommended client behavior:
The gateway is the source of truth for node/client admission.
- Pairing requests are created/approved/rejected in the gateway (see [Gateway pairing](/gateway/pairing)).
- The bridge enforces:
- The gateway enforces:
- auth (token / keypair)
- scopes/ACLs (bridge is not a raw proxy to every gateway method)
- scopes/ACLs (the gateway is not a raw proxy to every method)
- rate limits
## Responsibilities by component
- **Gateway**: advertises discovery beacons, owns pairing decisions, runs the bridge listener.
- **Gateway**: advertises discovery beacons, owns pairing decisions, and hosts the WS endpoint.
- **macOS app**: helps you pick a gateway, shows pairing prompts, and uses SSH only as a fallback.
- **iOS/Android nodes**: browse Bonjour as a convenience and connect via the paired bridge.
- **iOS/Android nodes**: browse Bonjour as a convenience and connect to the paired Gateway WS.

View File

@@ -82,14 +82,12 @@ Defaults (can be overridden via env/flags/config):
- `CLAWDBOT_STATE_DIR=~/.clawdbot-dev`
- `CLAWDBOT_CONFIG_PATH=~/.clawdbot-dev/clawdbot.json`
- `CLAWDBOT_GATEWAY_PORT=19001` (Gateway WS + HTTP)
- `bridge.port=19002` (derived: `gateway.port+1`)
- `browser.controlUrl=http://127.0.0.1:19003` (derived: `gateway.port+2`)
- `canvasHost.port=19005` (derived: `gateway.port+4`)
- `agents.defaults.workspace` default becomes `~/clawd-dev` when you run `setup`/`onboard` under `--dev`.
Derived ports (rules of thumb):
- Base port = `gateway.port` (or `CLAWDBOT_GATEWAY_PORT` / `--port`)
- `bridge.port = base + 1` (or `CLAWDBOT_BRIDGE_PORT` / config override)
- `browser.controlUrl port = base + 2` (or `CLAWDBOT_BROWSER_CONTROL_URL` / config override)
- `canvasHost.port = base + 4` (or `CLAWDBOT_CANVAS_HOST_PORT` / config override)
- Browser profile CDP ports auto-allocate from `browser.controlPort + 9 .. + 108` (persisted per profile).
@@ -114,7 +112,7 @@ CLAWDBOT_CONFIG_PATH=~/.clawdbot/b.json CLAWDBOT_STATE_DIR=~/.clawdbot-b clawdbo
```
## Protocol (operator view)
- Full docs: [Gateway protocol](/gateway/protocol) and [Bridge protocol](/gateway/bridge-protocol).
- Full docs: [Gateway protocol](/gateway/protocol) and [Bridge protocol (legacy)](/gateway/bridge-protocol).
- Mandatory first frame from client: `req {type:"req", id, method:"connect", params:{minProtocol,maxProtocol,client:{id,displayName?,version,platform,deviceFamily?,modelIdentifier?,mode,instanceId?}, caps, auth?, locale?, userAgent? } }`.
- Gateway replies `res {type:"res", id, ok:true, payload:hello-ok }` (or `ok:false` with an error, then closes).
- After handshake:
@@ -130,7 +128,7 @@ CLAWDBOT_CONFIG_PATH=~/.clawdbot/b.json CLAWDBOT_STATE_DIR=~/.clawdbot-b clawdbo
- `system-event` — post a presence/system note (structured).
- `send` — send a message via the active channel(s).
- `agent` — run an agent turn (streams events back on same connection).
- `node.list` — list paired + currently-connected bridge nodes (includes `caps`, `deviceFamily`, `modelIdentifier`, `paired`, `connected`, and advertised `commands`).
- `node.list` — list paired + currently-connected nodes (includes `caps`, `deviceFamily`, `modelIdentifier`, `paired`, `connected`, and advertised `commands`).
- `node.describe` — describe a node (capabilities + supported `node.invoke` commands; works for paired nodes and for currently-connected unpaired nodes).
- `node.invoke` — invoke a command on a node (e.g. `canvas.*`, `camera.*`).
- `node.pair.*` — pairing lifecycle (`request`, `list`, `approve`, `reject`, `verify`).

View File

@@ -13,7 +13,7 @@ Most setups should use one Gateway because a single Gateway can handle multiple
- `CLAWDBOT_STATE_DIR` — per-instance sessions, creds, caches
- `agents.defaults.workspace` — per-instance workspace root
- `gateway.port` (or `--port`) — unique per instance
- Derived ports (bridge/browser/canvas) must not overlap
- Derived ports (browser/canvas) must not overlap
If these are shared, you will hit config races and port conflicts.
@@ -47,7 +47,7 @@ Run a second Gateway on the same host with its own:
This keeps the rescue bot isolated from the main bot so it can debug or apply config changes if the primary bot is down.
Port spacing: leave at least 20 ports between base ports so the derived bridge/browser/canvas/CDP ports never collide.
Port spacing: leave at least 20 ports between base ports so the derived browser/canvas/CDP ports never collide.
### How to install (rescue bot)
@@ -73,7 +73,6 @@ clawdbot --profile rescue gateway install
Base port = `gateway.port` (or `CLAWDBOT_GATEWAY_PORT` / `--port`).
- `bridge.port = base + 1`
- `browser.controlUrl port = base + 2`
- `canvasHost.port = base + 4`
- Browser profile CDP ports auto-allocate from `browser.controlPort + 9 .. + 108`

View File

@@ -11,16 +11,20 @@ In Gateway-owned pairing, the **Gateway** is the source of truth for which nodes
are allowed to join. UIs (macOS app, future clients) are just frontends that
approve or reject pending requests.
**Important:** WS nodes use **device pairing** (role `node`) during `connect`.
`node.pair.*` is a separate pairing store and does **not** gate the WS handshake.
Only clients that explicitly call `node.pair.*` use this flow.
## Concepts
- **Pending request**: a node asked to join; requires approval.
- **Paired node**: approved node with an issued auth token.
- **Bridge**: transport endpoint only; it forwards requests but does not decide
membership.
- **Transport**: the Gateway WS endpoint forwards requests but does not decide
membership. (Legacy TCP bridge support is deprecated/removed.)
## How pairing works
1. A node connects to the bridge and requests pairing.
1. A node connects to the Gateway WS and requests pairing.
2. The Gateway stores a **pending request** and emits `node.pair.requested`.
3. You approve or reject the request (CLI or UI).
4. On approval, the Gateway issues a **new token** (tokens are rotated on repair).
@@ -81,9 +85,8 @@ Security notes:
- Tokens are secrets; treat `paired.json` as sensitive.
- Rotating a token requires re-approval (or deleting the node entry).
## Bridge behavior
## Transport behavior
- The bridge is **transport only**; it does not store membership.
- The transport is **stateless**; it does not store membership.
- If the Gateway is offline or pairing is disabled, nodes cannot pair.
- If the bridge is running but the Gateway is in remote mode, pairing still
happens against the remote Gateways store.
- If the Gateway is in remote mode, pairing still happens against the remote Gateways store.

View File

@@ -94,8 +94,9 @@ clawdbot gateway --tailscale funnel --auth password
or `tailscale funnel` configuration on shutdown.
- `gateway.bind: "tailnet"` is a direct Tailnet bind (no HTTPS, no Serve/Funnel).
- `gateway.bind: "auto"` prefers loopback; use `tailnet` if you want Tailnet-only.
- Serve/Funnel only expose the **Gateway control UI + WS**. Node **bridge** traffic
uses the separate bridge port (default `18790`) and is **not** proxied by Serve.
- Serve/Funnel only expose the **Gateway control UI + WS**. Nodes connect over
the same Gateway WS endpoint, so Serve can work for node access. (Legacy TCP
bridge traffic is not proxied by Serve.)
## Browser control server (remote Gateway + local browser)