let5see/clawdbot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add ws chat attachments

Browse Source
This commit is contained in:
Peter Steinberger
2025-12-09 23:16:57 +01:00
parent e80e5b0801
commit 1dd5c97ae0
25 changed files with 987 additions and 882 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/mac/webchat.md
View File

@@ -14,8 +14,8 @@ The macOS menu bar app opens the gateway’s loopback web chat server in a WKWeb
- WK logs: navigation lifecycle, readyState, js location, and JS errors/unhandled rejections are mirrored to OSLog for easier diagnosis.
## How it’s wired
- Assets: `apps/macos/Sources/Clawdis/Resources/WebChat/` contains the `pi-web-ui` dist plus a local import map pointing at bundled vendor modules and a tiny `pi-ai` stub. Everything is served from the gateway at `/` (legacy `/webchat/*` still works).
- Bridge: none. The web UI calls `/webchat/rpc` directly; Swift no longer proxies messages. RPC is handled in-process inside the gateway (no CLI spawn/PATH dependency).
- Assets: `apps/macos/Sources/Clawdis/Resources/WebChat/` contains the `pi-web-ui` dist plus a local import map pointing at bundled vendor modules and a tiny `pi-ai` stub. Everything is served from the static host at `/` (legacy `/webchat/*` still works).
- Bridge: none. The web UI connects directly to the Gateway WebSocket (default 18789) and uses `chat.history`/`chat.send` plus `chat/presence/tick/health` events. No `/rpc` or file-watcher socket path remains.
- Session: always primary; multiple transports (WhatsApp/Telegram/Desktop) share the same session key so context is unified.
## Security / surface area

39
docs/refactor/webagent-session.md Normal file
View File

@@ -0,0 +1,39 @@
# WebAgent session migration (WS-only)
Context: web chat currently lives in a WKWebView that loads the pi-web bundle. Sends go over HTTP `/rpc` to the webchat server, and updates come from `/socket` snapshots based on session JSONL file changes. The Gateway itself already speaks WebSocket to the webchat server, and Pi writes the session JSONL files. This doc tracks the plan to move WebChat to a single Gateway WebSocket and drop the HTTP shim/file-watching.
## Target state
- Gateway WS adds methods:
- `chat.history { sessionKey }` → `{ sessionKey, messages[], thinkingLevel }` (reads the existing JSONL + sessions.json).
- `chat.send { sessionKey, message, attachments?, thinking?, deliver?, timeoutMs<=30000, idempotencyKey }` → `res { runId, status:"accepted" }` or `res ok:false` on validation/timeout.
- Gateway WS emits `chat` events `{ runId, sessionKey, seq, state:"delta"|"final"|"error", message?, errorMessage?, usage?, stopReason? }`. Streaming is optional; minimum is a single `state:"final"` per send.
- Client consumes only WS: bootstrap via `chat.history`, send via `chat.send`, live updates via `chat` events. No file watchers.
- Health gate: client subscribes to `health` and blocks send when health is not OK; 30s client-side timeout for sends.
- Tunneling: only the Gateway WS port needs to be forwarded; HTTP server remains for static assets but no RPC endpoints.
## Server work (Node)
- Implement `chat.history` and `chat.send` handlers in `src/gateway/server.ts`; update protocol schemas/tests.
- Emit `chat` events by plumbing `agentCommand`/`emitAgentEvent` outputs; include assistant text/tool results.
- Remove `/rpc` and `/socket` routes + file-watch broadcast from `src/webchat/server.ts`; leave static host only.
## Client work (pi-web bundle)
- Replace `NativeTransport` with a Gateway WS client:
- `hello` → `chat.history` for initial state.
- Listen to `chat/presence/tick/health`; update UI from events only.
- Send via `chat.send`; mark pending until `chat state:final|error`.
- Enforce health gate + 30s timeout.
- Remove reliance on session file snapshots and `/rpc`.
## Persistence
- Keep passing `--session <.../.clawdis/sessions/{{SessionId}}.jsonl>` to Pi so it continues writing JSONL. The WS history reader uses the same file; no new store introduced.
## Docs to update when shipping
- `docs/webchat.md` (WS-only flow, methods/events, health gate, tunnel WS port).
- `docs/mac/webchat.md` (WKWebView now talks Gateway WS; `/rpc`/file-watch removed).
- `docs/architecture.md` / `typebox.md` if protocol methods are listed.
- Optional: add a concise Gateway chat protocol appendix if needed.
## Open decisions
- Streaming granularity: start with `state:"final"` only, or include token/tool deltas immediately?
- Attachments over WS: text-only initially is OK; confirm before wiring binary/upload path.
- Error shape: use `res ok:false` for validation/timeout, `chat state:"error"` for model/runtime failures.

26
docs/webchat.md
View File

@@ -1,5 +1,5 @@
---
summary: "Loopback WebChat server and SSH tunnel usage for chat UI"
summary: "Loopback WebChat static host and Gateway WS usage for chat UI"
read_when:
- Debugging or configuring WebChat access
---
@@ -8,23 +8,21 @@ read_when:
Updated: 2025-12-09
## What it is
- A local web UI for chatting with the Gateway.
- A local web UI for chatting with the Gateway, now WS-only for data.
- Static assets served by the WebChat HTTP server (default port **18788**, configurable).
- The WebChat backend holds a single WebSocket connection to the Gateway (`ws://127.0.0.1:18789` by default) for all control/data: history fetch, sends, agent runs, presence.
- The browser/WebView connects directly to the Gateway WebSocket (`ws://127.0.0.1:18789` by default) for history, sends, and events. No file watching or HTTP RPC.
- Trust model: access is granted by being on localhost or inside your SSH/Tailscale tunnel. No additional auth prompts once you can reach the box.
- `webchat.gatewayPort` config can point at a non-default Gateway port if needed.
## Endpoints
- UI is now served at the root: `http://127.0.0.1:<port>/` (legacy `/webchat/` still works).
- `GET /webchat/info?session=<key>` (alias `/info`) → `{ port, sessionId, initialMessages, basePath }` plus history from the Gateway session store.
- `GET /` (or `/webchat/*`) → static assets.
- `POST /webchat/rpc` (alias `/rpc`) → proxies a chat/agent action through the Gateway connection and returns `{ ok, payloads?, error? }`.
- UI is served at the root: `http://127.0.0.1:<port>/` (legacy `/webchat/` still works).
- `GET /` (or `/webchat/*`) → static assets only. No RPC endpoints.
- Data plane is entirely on the Gateway WS (`ws://127.0.0.1:<gatewayPort>`): methods `chat.history`, `chat.send`; events `chat`, `presence`, `tick`, `health`.
## How it connects
- On startup, the WebChat server dials the Gateway WebSocket and performs the mandatory `hello` handshake; the `hello-ok` snapshot seeds presence + health immediately.
- All outgoing sends/agent calls are requests on that WS; streamed events (`agent`, `presence`, `tick`) are forwarded to the browser client.
- If a seq gap is detected in Gateway events, WebChat auto-refreshes health + presence and broadcasts a `gateway-refresh` to connected browsers.
- If the Gateway WS is unavailable, WebChat fails fast and surfaces the error in the UI.
- Browser/WebView performs Gateway WS `hello`, then calls `chat.history` for bootstrap and `chat.send` for sends; listens to `chat/presence/tick/health` events.
- No session file watching. History comes from the Gateway via `chat.history`.
- If Gateway WS is unavailable, the UI surfaces the error and blocks send.
## Remote use
- SSH tunnel example: `ssh -N -L 18788:127.0.0.1:18788 -L 18789:127.0.0.1:18789 user@host`.
@@ -36,10 +34,10 @@ Updated: 2025-12-09
- Gateway WS port is set by `clawdis gateway --port`; WebChat expects it at 18789 unless overridden.
## Failure handling
- Clear UI error when the Gateway handshake fails or the WS drops.
- UI errors when the Gateway handshake fails or the WS drops; no HTTP fallback.
- WebChat does not attempt fallback transports; the Gateway WS is required.
## Dev notes
- Assets live in `apps/macos/Sources/Clawdis/Resources/WebChat`.
- Server implementation: `src/webchat/server.ts`.
- macOS glue: `WebChatWindow.swift` + `WebChatTunnel` for SSH -L helpers.
- Static host: `src/webchat/server.ts` (loopback-only HTTP).
- macOS glue: `WebChatWindow.swift` + `WebChatTunnel` for SSH -L helpers; WKWebView talks directly to Gateway WS.