diff --git a/CHANGELOG.md b/CHANGELOG.md index c9f56d3a5..2314e771c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,19 +2,36 @@ ## 2.0.0 — Unreleased -First Clawdis release post rebrand. This is a semver-major because we dropped legacy providers/agents and moved defaults to new paths while adding a full macOS companion app. +_No changes since 2.0.0-beta1._ + +## 2.0.0-beta1 — 2025-12-14 + +First Clawdis release post rebrand. This is a semver-major because we dropped legacy providers/agents and moved defaults to new paths while adding a full macOS companion app, a WebSocket Gateway, and an iOS node (Iris). ### Breaking - Renamed to **Clawdis**: defaults now live under `~/.clawdis` (sessions in `~/.clawdis/sessions/`, IPC at `~/.clawdis/clawdis.sock`, logs in `/tmp/clawdis`). Launchd labels and config filenames follow the new name; legacy stores are copied forward on first run. - Pi only: `inbound.reply.agent.kind` accepts only `"pi"`, and the agent CLI/CLI flags for Claude/Codex/Gemini were removed. The Pi CLI runs in RPC mode with a persistent worker. - WhatsApp Web is the only transport; Twilio support and related CLI flags/tests were removed. - Direct chats now collapse into a single `main` session by default (no config needed); groups stay isolated as `group:`. -- Gateway background helpers were removed; run `clawdis gateway --verbose` under your supervisor of choice if you want it detached. +- Gateway is now a loopback-only WebSocket daemon (`ws://127.0.0.1:18789`) that owns all providers/state; clients (CLI, WebChat, macOS app, nodes) connect to it. Start it explicitly (`clawdis gateway …`) or via Clawdis.app; helper subcommands no longer auto-spawn a gateway. + +### Gateway, nodes, and automation +- New typed Gateway WS protocol (JSON schema validated) with `clawdis gateway {health,status,send,agent,call}` helpers and structured presence/instance updates for all clients. +- Optional LAN-facing bridge (`tcp://0.0.0.0:18790`) keeps the Gateway loopback-only while enabling direct Bonjour-discovered connections for paired nodes. +- Node pairing + management via `clawdis nodes {pending,approve,reject,invoke}` (used by the iOS node “Iris” and future remote nodes). +- Cron jobs are Gateway-owned (`clawdis cron …`) with run history stored as JSONL and support for “isolated summary” posting into the main session. ### macOS companion app - **Clawdis.app menu bar companion**: packaged, signed bundle with gateway start/stop, launchd toggle, project-root and pnpm/node auto-resolution, live log shortcut, restart button, and status/recipient table plus badges/dimming for attention and paused states. - **On-device Voice Wake**: Apple speech recognizer with wake-word table, language picker, live mic meter, “hold until silence,” animated ears/legs, and main-session routing that replies on the **last used surface** (WhatsApp/Telegram/WebChat). Delivery failures are logged, and the run remains visible via WebChat/session logs. - **WebChat & Debugging**: bundled WebChat UI, Debug tab with heartbeat sliders, session-store picker, log opener (`clawlog`), gateway restart, health probes, and scrollable settings panes. +- **Browser control**: manage clawd’s dedicated Chrome/Chromium with tab listing/open/focus/close, screenshots, DOM query/dump, and “AI snapshots” (aria/domSnapshot/ai) via `clawdis browser …` and UI controls. +- **Remote gateway control**: Bonjour discovery for local masters plus SSH-tunnel fallback for remote control when multicast is unavailable. + +### iOS node (Iris) +- New iOS companion app that pairs to the Gateway bridge, reports presence as a node, and exposes a WKWebView “Canvas” for agent-driven UI. +- `clawdis nodes invoke` supports `screen.eval` and `screen.snapshot` to drive and verify the iOS Canvas (fails fast when Iris is backgrounded). +- Voice wake words are configurable in-app; Iris reconnects to the last bridge when credentials are still present in Keychain. ### WhatsApp & agent experience - Group chats fully supported: mention-gated triggers (including media-only captions), sender attribution, session primer with subject/member roster, allowlist bypass when you’re @‑mentioned, and safer handling of view-once/ephemeral media. @@ -36,7 +53,7 @@ First Clawdis release post rebrand. This is a semver-major because we dropped le ### Docs - Added `docs/telegram.md` outlining the Telegram Bot API provider (grammY) and how it shares the `main` session. Default grammY throttler keeps Bot API calls under rate limits. -- CLI gateway now auto-starts WhatsApp and Telegram when configured (single `gateway` command with `--provider` selector); text/media sends still use `--provider telegram`; webhook/proxy options documented. +- Gateway can run WhatsApp + Telegram together when configured; `clawdis send --provider telegram …` sends via the Telegram bot (webhook/proxy options documented). ## 1.5.0 — 2025-12-05 diff --git a/Peekaboo b/Peekaboo index 4807e6fdf..35d495e28 160000 --- a/Peekaboo +++ b/Peekaboo @@ -1 +1 @@ -Subproject commit 4807e6fdf016f013dba8a32fa4b2528182539fad +Subproject commit 35d495e28cba317681734015d61834c30869a7de diff --git a/README.md b/README.md index 644162868..3c449958c 100644 --- a/README.md +++ b/README.md @@ -10,18 +10,27 @@

CI status - npm version + GitHub release MIT License

-**CLAWDIS** is a WhatsApp- and Telegram-to-AI gateway. Send a message, get an AI response. It's like having a genius lobster in your pocket 24/7. +**CLAWDIS** is a TypeScript/Node gateway that bridges WhatsApp (Web/Baileys) and Telegram (Bot API/grammY) to a local coding agent (**Pi**). +It’s like having a genius lobster in your pocket 24/7 — but with a real control plane, companion apps, and a network model that won’t corrupt sessions. ``` -┌─────────────┐ ┌──────────┐ ┌─────────────┐ -│ WhatsApp │ ───▶ │ CLAWDIS │ ───▶ │ AI Agent │ -│ Telegram │ ───▶ │ 🦞⏱️💙 │ ◀─── │ (Pi) │ -│ (You) │ ◀─── │ │ │ │ -└─────────────┘ └──────────┘ └─────────────┘ +WhatsApp / Telegram + │ + ▼ + ┌──────────────────────────┐ + │ Gateway │ ws://127.0.0.1:18789 (loopback-only) + │ (single source) │ tcp://0.0.0.0:18790 (optional Bridge) + └───────────┬───────────────┘ + │ + ├─ Pi agent (RPC) + ├─ CLI (clawdis …) + ├─ WebChat (loopback UI) + ├─ macOS app (Clawdis.app) + └─ iOS node (Iris) via Bridge + pairing ``` ## Why "CLAWDIS"? @@ -34,52 +43,66 @@ Because every space lobster needs a time-and-space machine. The Doctor has a TAR - 📱 **WhatsApp Integration** — Personal WhatsApp Web (Baileys) - ✈️ **Telegram (Bot API)** — DMs and groups via grammY -- 🤖 **AI Agent Gateway** — Pi only (Pi CLI in RPC mode) -- 💬 **Session Management** — Per-sender conversation context +- 🛰️ **Gateway control plane** — One long-lived gateway owns provider state; clients connect over WebSocket +- 🤖 **Agent runtime** — Pi only (Pi CLI in RPC mode), with tool streaming +- 💬 **Sessions** — Direct chats collapse into `main` by default; groups are isolated - 🔔 **Heartbeats** — Periodic check-ins for proactive AI - 🧭 **Clawd Browser** — Dedicated Chrome/Chromium profile with tabs + screenshot control (no interference with your daily browser) - 👥 **Group Chat Support** — Mention-based triggering - 📎 **Media Support** — Images, audio, documents, voice notes -- 🎤 **Voice Transcription** — Whisper integration +- 🎤 **Voice & transcription hooks** — Voice Wake (macOS/iOS) + optional transcription pipeline - 🔧 **Tool Streaming** — Real-time display (💻📄✍️📝) -- 🖥️ **macOS Companion (Clawdis.app)** — Menu bar controls, on-device Voice Wake, model/config editor +- 🖥️ **macOS Companion (Clawdis.app)** — Menu bar controls, Voice Wake, WebChat, onboarding, remote gateway control +- 📱 **iOS Node (Iris)** — Pairs as a node, exposes a Canvas surface, forwards voice wake transcripts Only the Pi CLI is supported now; legacy Claude/Codex/Gemini paths have been removed. +## Network model (the “new reality”) + +- **One Gateway per host**. The Gateway is the only process allowed to own the WhatsApp Web session. +- **Loopback-first**: the Gateway WebSocket listens on `ws://127.0.0.1:18789` and is not exposed on the LAN. +- **Bridge for nodes**: when enabled, the Gateway also exposes a LAN/tailnet-facing bridge on `tcp://0.0.0.0:18790` for paired nodes (Bonjour-discoverable). +- **Remote control**: use a VPN/tailnet or an SSH tunnel (`ssh -N -L 18789:127.0.0.1:18789 user@host`). The macOS app can drive this flow. + +## Codebase + +- **TypeScript (ESM)**: CLI + Gateway live in `src/` and run on Node ≥ 22. +- **macOS app (Swift)**: menu bar companion lives in `apps/macos/`. +- **iOS app (Swift)**: Iris node prototype lives in `apps/ios/`. + ## Quick Start -Mac signing tip: set `SIGN_IDENTITY="Apple Development: Your Name (TEAMID)"` in your shell profile so `scripts/restart-mac.sh` signs with your cert (defaults to ad-hoc). Debug bundle ID remains `com.steipete.clawdis.debug`. Runtime requirement: **Node ≥22.0.0** (not bundled). The macOS app and CLI both use the host runtime; install via Homebrew or official installers before running `clawdis`. ```bash -# Install -npm install -g clawdis +# From source (recommended while the npm package is still settling) +pnpm install +pnpm build -# Link your WhatsApp -clawdis login - -# Send a message -clawdis send --to +1234567890 --message "Hello from the CLAWDIS!" - -# Talk directly to the agent (no WhatsApp send) -clawdis agent --to +1234567890 --message "Ship checklist" --thinking high +# Link your WhatsApp (stores creds under ~/.clawdis/credentials) +pnpm clawdis login # Start the gateway (WebSocket control plane) -clawdis gateway --port 18789 --verbose +pnpm clawdis gateway --port 18789 --verbose + +# Send a WhatsApp message (WhatsApp sends go through the Gateway) +pnpm clawdis send --to +1234567890 --message "Hello from the CLAWDIS!" + +# Talk to the agent (optionally deliver back to WhatsApp/Telegram) +pnpm clawdis agent --message "Ship checklist" --thinking high # If the port is busy, force-kill listeners then start -clawdis gateway --force +pnpm clawdis gateway --force ``` ## Companion Apps ### macOS Companion (Clawdis.app) -- **On-device Voice Wake:** listens for wake words (e.g. “Claude”) using Apple’s on-device speech recognizer (macOS 26+). macOS still shows the standard Speech/Mic permissions prompt, but audio stays on device. -- **Push-to-talk (Right Option hold):** hold right Option to speak; the voice overlay shows live partials and sends when you release. -- **Config tab:** pick the model from your local Pi model catalog (`pi-mono/packages/ai/src/models.generated.ts`), or enter a custom model ID; edit session store path and context tokens. -- **Voice settings:** language + additional languages, mic picker, live level meter, trigger-word table, and a built-in test harness. -- **Menu bar toggle:** enable/disable Voice Wake from the menu bar; respects Dock-icon preference. +- A menu bar app that can start/stop the Gateway, show health/presence, and provide a local ops UI. +- **Voice Wake** (on-device speech recognition) and Push-to-talk overlay. +- **WebChat** embed + debug tooling (logs, status, heartbeats, sessions). +- Hosts **PeekabooBridge** for UI automation brokering (for clawd workflows). ### Voice Wake reply routing @@ -97,9 +120,9 @@ Build/run the mac app with `./scripts/restart-mac.sh` (packages, installs, and l Iris is an internal/prototype iOS app that connects as a **remote node**: -- **Voice trigger:** forwards transcripts into the Gateway `agent` method. +- **Voice trigger:** forwards transcripts into the Gateway (agent runs + wakeups). - **Canvas screen:** a WKWebView + `` surface the agent can control (via `screen.eval` / `screen.snapshot` over `node.invoke`). -- **Discovery + pairing:** finds the gateway bridge via Bonjour (`_clawdis-bridge._tcp`) and uses Gateway-owned pairing (`clawdis nodes pending|approve`). +- **Discovery + pairing:** finds the bridge via Bonjour (`_clawdis-bridge._tcp`) and uses Gateway-owned pairing (`clawdis nodes pending|approve`). Runbook: `docs/ios/connect.md` @@ -110,16 +133,7 @@ Create `~/.clawdis/clawdis.json`: ```json5 { inbound: { - allowFrom: ["+1234567890"], - reply: { - mode: "command", - command: ["pi", "--mode", "rpc", "{{BodyStripped}}"], - session: { - scope: "per-sender", - idleMinutes: 1440 - }, - heartbeatMinutes: 10 - } + allowFrom: ["+1234567890"] } } ``` @@ -139,12 +153,16 @@ Optional: enable/configure clawd’s dedicated browser control (defaults are alr ## Documentation - [Configuration Guide](./docs/configuration.md) +- [Gateway runbook](./docs/gateway.md) +- [Discovery + transports](./docs/discovery.md) - [Agent Integration](./docs/agents.md) - [Group Chats](./docs/group-messages.md) - [Security](./docs/security.md) - [Troubleshooting](./docs/troubleshooting.md) - [The Lore](./docs/lore.md) 🦞 - [Telegram (Bot API)](./docs/telegram.md) +- [iOS node runbook (Iris)](./docs/ios/connect.md) +- [macOS app spec](./docs/clawdis-mac.md) ## Clawd @@ -157,21 +175,23 @@ CLAWDIS was built for **Clawd**, a space lobster AI assistant. See the full setu ## Provider +If you’re running from source, use `pnpm clawdis …` instead of `clawdis …`. + ### WhatsApp Web ```bash -clawdis login # Scan QR code -clawdis gateway # Start listening (WS on 127.0.0.1:18789) +clawdis login # scan QR, store creds +clawdis gateway # run Gateway (WS on 127.0.0.1:18789) ``` ### Telegram (Bot API) -Bot-mode support (grammY only) shares the same `main` session as WhatsApp/WebChat, with groups kept isolated. Text and media send work via `clawdis send --provider telegram`. The unified `clawdis gateway` starts WhatsApp and, when `TELEGRAM_BOT_TOKEN` or `telegram.botToken` is set, Telegram too (use `--provider` to force web|telegram|all). Webhook mode: `--webhook --port … --webhook-secret … --webhook-url …` (or register via BotFather). See `docs/telegram.md` for setup and limits. +Bot-mode support (grammY only) shares the same `main` session as WhatsApp/WebChat, with groups kept isolated. Text/media sends work via `clawdis send --provider telegram` (reads `TELEGRAM_BOT_TOKEN` or `telegram.botToken`). Webhook mode is supported; see `docs/telegram.md` for setup and limits. ## Commands | Command | Description | |---------|-------------| | `clawdis login` | Link WhatsApp Web via QR | -| `clawdis send` | Send a message (WhatsApp default; `--provider telegram` for bot mode). Always uses the Gateway WS; requires a running gateway. | +| `clawdis send` | Send a message (WhatsApp default; `--provider telegram` for bot mode). WhatsApp sends go via the Gateway WS; Telegram sends are direct. | | `clawdis agent` | Talk directly to the agent (no WhatsApp send) | | `clawdis browser ...` | Manage clawd’s dedicated browser (status/tabs/open/screenshot). | | `clawdis gateway` | Start the Gateway server (WS control plane). Params: `--port`, `--token`, `--force`, `--verbose`. | @@ -202,7 +222,7 @@ In chat, send `/status` to see if the agent is reachable, how much context the s ### Sessions, surfaces, and WebChat - Direct chats now share a canonical session key `main` by default (configurable via `inbound.reply.session.mainKey`). Groups stay isolated as `group:`. -- WebChat always attaches to the `main` session and hydrates the full session history from `~/.clawdis/sessions/.jsonl`, so desktop view mirrors WhatsApp/Telegram turns. +- WebChat attaches to `main` and hydrates history from `~/.clawdis/sessions/.jsonl`, so desktop view mirrors WhatsApp/Telegram turns. - Inbound contexts carry a `Surface` hint (e.g., `whatsapp`, `webchat`, `telegram`) for logging; replies still go back to the originating surface deterministically. - Every inbound message is wrapped for the agent as `[Surface FROM HOST/IP TIMESTAMP] body`: - WhatsApp: `[WhatsApp +15551234567 2025-12-09 12:34] …` diff --git a/apps/ios/Sources/Bridge/BridgeConnectionController.swift b/apps/ios/Sources/Bridge/BridgeConnectionController.swift new file mode 100644 index 000000000..6361093de --- /dev/null +++ b/apps/ios/Sources/Bridge/BridgeConnectionController.swift @@ -0,0 +1,94 @@ +import ClawdisKit +import Combine +import Foundation +import Network +import SwiftUI + +@MainActor +final class BridgeConnectionController: ObservableObject { + @Published private(set) var bridges: [BridgeDiscoveryModel.DiscoveredBridge] = [] + @Published private(set) var discoveryStatusText: String = "Idle" + + private let discovery = BridgeDiscoveryModel() + private weak var appModel: NodeAppModel? + private var cancellables = Set() + private var didAutoConnect = false + + init(appModel: NodeAppModel) { + self.appModel = appModel + + BridgeSettingsStore.bootstrapPersistence() + + self.discovery.$bridges + .sink { [weak self] newValue in + guard let self else { return } + self.bridges = newValue + self.maybeAutoConnect() + } + .store(in: &self.cancellables) + + self.discovery.$statusText + .assign(to: &self.$discoveryStatusText) + + self.discovery.start() + } + + func setScenePhase(_ phase: ScenePhase) { + switch phase { + case .background: + self.discovery.stop() + case .active, .inactive: + self.discovery.start() + @unknown default: + self.discovery.start() + } + } + + private func maybeAutoConnect() { + guard !self.didAutoConnect else { return } + guard let appModel = self.appModel else { return } + guard appModel.bridgeServerName == nil else { return } + + let defaults = UserDefaults.standard + let preferredStableID = defaults.string(forKey: "bridge.preferredStableID")? + .trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + guard !preferredStableID.isEmpty else { return } + + let instanceId = defaults.string(forKey: "node.instanceId")? + .trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + guard !instanceId.isEmpty else { return } + + let token = KeychainStore.loadString( + service: "com.steipete.clawdis.bridge", + account: "bridge-token.\(instanceId)")? + .trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + guard !token.isEmpty else { return } + + guard let target = self.bridges.first(where: { $0.stableID == preferredStableID }) else { return } + + self.didAutoConnect = true + appModel.connectToBridge(endpoint: target.endpoint, hello: self.makeHello(token: token)) + } + + private func makeHello(token: String) -> BridgeHello { + let defaults = UserDefaults.standard + let nodeId = defaults.string(forKey: "node.instanceId") ?? "ios-node" + let displayName = defaults.string(forKey: "node.displayName") ?? "iOS Node" + + return BridgeHello( + nodeId: nodeId, + displayName: displayName, + token: token, + platform: self.platformString(), + version: self.appVersion()) + } + + private func platformString() -> String { + let v = ProcessInfo.processInfo.operatingSystemVersion + return "iOS \(v.majorVersion).\(v.minorVersion).\(v.patchVersion)" + } + + private func appVersion() -> String { + Bundle.main.infoDictionary?["CFBundleShortVersionString"] as? String ?? "dev" + } +} diff --git a/apps/ios/Sources/Bridge/BridgeSettingsStore.swift b/apps/ios/Sources/Bridge/BridgeSettingsStore.swift new file mode 100644 index 000000000..653d56280 --- /dev/null +++ b/apps/ios/Sources/Bridge/BridgeSettingsStore.swift @@ -0,0 +1,79 @@ +import Foundation + +enum BridgeSettingsStore { + private static let bridgeService = "com.steipete.clawdis.bridge" + private static let nodeService = "com.steipete.clawdis.node" + + private static let instanceIdDefaultsKey = "node.instanceId" + private static let preferredBridgeStableIDDefaultsKey = "bridge.preferredStableID" + + private static let instanceIdAccount = "instanceId" + private static let preferredBridgeStableIDAccount = "preferredStableID" + + static func bootstrapPersistence() { + self.ensureStableInstanceID() + self.ensurePreferredBridgeStableID() + } + + static func loadStableInstanceID() -> String? { + KeychainStore.loadString(service: self.nodeService, account: self.instanceIdAccount)? + .trimmingCharacters(in: .whitespacesAndNewlines) + } + + static func saveStableInstanceID(_ instanceId: String) { + _ = KeychainStore.saveString(instanceId, service: self.nodeService, account: self.instanceIdAccount) + } + + static func loadPreferredBridgeStableID() -> String? { + KeychainStore.loadString(service: self.bridgeService, account: self.preferredBridgeStableIDAccount)? + .trimmingCharacters(in: .whitespacesAndNewlines) + } + + static func savePreferredBridgeStableID(_ stableID: String) { + _ = KeychainStore.saveString( + stableID, + service: self.bridgeService, + account: self.preferredBridgeStableIDAccount) + } + + private static func ensureStableInstanceID() { + let defaults = UserDefaults.standard + + if let existing = defaults.string(forKey: self.instanceIdDefaultsKey)? + .trimmingCharacters(in: .whitespacesAndNewlines), + !existing.isEmpty + { + if self.loadStableInstanceID() == nil { + self.saveStableInstanceID(existing) + } + return + } + + if let stored = self.loadStableInstanceID(), !stored.isEmpty { + defaults.set(stored, forKey: self.instanceIdDefaultsKey) + return + } + + let fresh = UUID().uuidString + self.saveStableInstanceID(fresh) + defaults.set(fresh, forKey: self.instanceIdDefaultsKey) + } + + private static func ensurePreferredBridgeStableID() { + let defaults = UserDefaults.standard + + if let existing = defaults.string(forKey: self.preferredBridgeStableIDDefaultsKey)? + .trimmingCharacters(in: .whitespacesAndNewlines), + !existing.isEmpty + { + if self.loadPreferredBridgeStableID() == nil { + self.savePreferredBridgeStableID(existing) + } + return + } + + if let stored = self.loadPreferredBridgeStableID(), !stored.isEmpty { + defaults.set(stored, forKey: self.preferredBridgeStableIDDefaultsKey) + } + } +} diff --git a/apps/ios/Sources/Bridge/KeychainStore.swift b/apps/ios/Sources/Bridge/KeychainStore.swift index 6f6189da3..1377d8517 100644 --- a/apps/ios/Sources/Bridge/KeychainStore.swift +++ b/apps/ios/Sources/Bridge/KeychainStore.swift @@ -3,14 +3,13 @@ import Security enum KeychainStore { static func loadString(service: String, account: String) -> String? { - var query: [String: Any] = [ + let query: [String: Any] = [ kSecClass as String: kSecClassGenericPassword, kSecAttrService as String: service, kSecAttrAccount as String: account, kSecReturnData as String: true, kSecMatchLimit as String: kSecMatchLimitOne, ] - query[kSecAttrAccessible as String] = kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly var item: CFTypeRef? let status = SecItemCopyMatching(query as CFDictionary, &item) @@ -20,20 +19,20 @@ enum KeychainStore { static func saveString(_ value: String, service: String, account: String) -> Bool { let data = Data(value.utf8) - let base: [String: Any] = [ + let query: [String: Any] = [ kSecClass as String: kSecClassGenericPassword, kSecAttrService as String: service, kSecAttrAccount as String: account, - kSecAttrAccessible as String: kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly, ] let update: [String: Any] = [kSecValueData as String: data] - let status = SecItemUpdate(base as CFDictionary, update as CFDictionary) + let status = SecItemUpdate(query as CFDictionary, update as CFDictionary) if status == errSecSuccess { return true } if status != errSecItemNotFound { return false } - var insert = base + var insert = query insert[kSecValueData as String] = data + insert[kSecAttrAccessible as String] = kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly return SecItemAdd(insert as CFDictionary, nil) == errSecSuccess } diff --git a/apps/ios/Sources/ClawdisApp.swift b/apps/ios/Sources/ClawdisApp.swift index c71b3ff8e..89bf736df 100644 --- a/apps/ios/Sources/ClawdisApp.swift +++ b/apps/ios/Sources/ClawdisApp.swift @@ -2,19 +2,29 @@ import SwiftUI @main struct ClawdisApp: App { - @StateObject private var appModel = NodeAppModel() + @StateObject private var appModel: NodeAppModel + @StateObject private var bridgeController: BridgeConnectionController @Environment(\.scenePhase) private var scenePhase + init() { + BridgeSettingsStore.bootstrapPersistence() + let appModel = NodeAppModel() + _appModel = StateObject(wrappedValue: appModel) + _bridgeController = StateObject(wrappedValue: BridgeConnectionController(appModel: appModel)) + } + var body: some Scene { WindowGroup { RootCanvas() .environmentObject(self.appModel) .environmentObject(self.appModel.voiceWake) + .environmentObject(self.bridgeController) .onOpenURL { url in Task { await self.appModel.handleDeepLink(url: url) } } .onChange(of: self.scenePhase) { _, newValue in self.appModel.setScenePhase(newValue) + self.bridgeController.setScenePhase(newValue) } } } diff --git a/apps/ios/Sources/RootTabs.swift b/apps/ios/Sources/RootTabs.swift index 95174f63d..c87392974 100644 --- a/apps/ios/Sources/RootTabs.swift +++ b/apps/ios/Sources/RootTabs.swift @@ -1,6 +1,8 @@ import SwiftUI struct RootTabs: View { + @EnvironmentObject private var appModel: NodeAppModel + var body: some View { TabView { ScreenTab() @@ -10,7 +12,71 @@ struct RootTabs: View { .tabItem { Label("Voice", systemImage: "mic") } SettingsTab() - .tabItem { Label("Settings", systemImage: "gearshape") } + .tabItem { + VStack { + ZStack(alignment: .topTrailing) { + Image(systemName: "gearshape") + Circle() + .fill(self.settingsIndicatorColor) + .frame(width: 9, height: 9) + .overlay( + Circle() + .stroke(.black.opacity(0.2), lineWidth: 0.5)) + .shadow( + color: self.settingsIndicatorGlowColor, + radius: self.settingsIndicatorGlowRadius, + x: 0, + y: 0) + .offset(x: 7, y: -2) + } + Text("Settings") + } + } + } + } + + private enum BridgeIndicatorState { + case connected + case connecting + case disconnected + } + + private var bridgeIndicatorState: BridgeIndicatorState { + if self.appModel.bridgeServerName != nil { return .connected } + if self.appModel.bridgeStatusText.localizedCaseInsensitiveContains("connecting") { return .connecting } + return .disconnected + } + + private var settingsIndicatorColor: Color { + switch self.bridgeIndicatorState { + case .connected: + Color.green + case .connecting: + Color.yellow + case .disconnected: + Color.red + } + } + + private var settingsIndicatorGlowColor: Color { + switch self.bridgeIndicatorState { + case .connected: + Color.green.opacity(0.75) + case .connecting: + Color.yellow.opacity(0.6) + case .disconnected: + Color.clear + } + } + + private var settingsIndicatorGlowRadius: CGFloat { + switch self.bridgeIndicatorState { + case .connected: + 6 + case .connecting: + 4 + case .disconnected: + 0 } } } diff --git a/apps/ios/Sources/Settings/SettingsTab.swift b/apps/ios/Sources/Settings/SettingsTab.swift index e10d57505..d5304d698 100644 --- a/apps/ios/Sources/Settings/SettingsTab.swift +++ b/apps/ios/Sources/Settings/SettingsTab.swift @@ -12,15 +12,15 @@ extension ConnectStatusStore: @unchecked Sendable {} struct SettingsTab: View { @EnvironmentObject private var appModel: NodeAppModel @EnvironmentObject private var voiceWake: VoiceWakeManager + @EnvironmentObject private var bridgeController: BridgeConnectionController @Environment(\.dismiss) private var dismiss @AppStorage("node.displayName") private var displayName: String = "iOS Node" @AppStorage("node.instanceId") private var instanceId: String = UUID().uuidString @AppStorage("voiceWake.enabled") private var voiceWakeEnabled: Bool = false + @AppStorage("camera.enabled") private var cameraEnabled: Bool = true @AppStorage("bridge.preferredStableID") private var preferredBridgeStableID: String = "" - @StateObject private var discovery = BridgeDiscoveryModel() @StateObject private var connectStatus = ConnectStatusStore() @State private var connectingBridgeID: String? - @State private var didAutoConnect = false @State private var localIPAddress: String? var body: some View { @@ -58,8 +58,15 @@ struct SettingsTab: View { } } + Section("Camera") { + Toggle("Allow Camera", isOn: self.$cameraEnabled) + Text("Allows the bridge to request photos or short video clips (foreground only).") + .font(.footnote) + .foregroundStyle(.secondary) + } + Section("Bridge") { - LabeledContent("Discovery", value: self.discovery.statusText) + LabeledContent("Discovery", value: self.bridgeController.discoveryStatusText) LabeledContent("Status", value: self.appModel.bridgeStatusText) if let serverName = self.appModel.bridgeServerName { LabeledContent("Server", value: serverName) @@ -120,31 +127,12 @@ struct SettingsTab: View { } } .onAppear { - self.discovery.start() self.localIPAddress = Self.primaryIPv4Address() } - .onDisappear { self.discovery.stop() } - .onChange(of: self.discovery.bridges) { _, newValue in - if self.didAutoConnect { return } - if self.appModel.bridgeServerName != nil { return } - - let existing = KeychainStore.loadString( - service: "com.steipete.clawdis.bridge", - account: self.keychainAccount()) - guard let existing, !existing.isEmpty else { return } - guard let target = self.pickAutoConnectBridge(from: newValue) else { return } - - self.didAutoConnect = true - self.preferredBridgeStableID = target.stableID - self.appModel.connectToBridge( - endpoint: target.endpoint, - hello: BridgeHello( - nodeId: self.instanceId, - displayName: self.displayName, - token: existing, - platform: self.platformString(), - version: self.appVersion())) - self.connectStatus.text = nil + .onChange(of: self.preferredBridgeStableID) { _, newValue in + let trimmed = newValue.trimmingCharacters(in: .whitespacesAndNewlines) + guard !trimmed.isEmpty else { return } + BridgeSettingsStore.savePreferredBridgeStableID(trimmed) } .onChange(of: self.appModel.bridgeServerName) { _, _ in self.connectStatus.text = nil @@ -154,12 +142,12 @@ struct SettingsTab: View { @ViewBuilder private func bridgeList(showing: BridgeListMode) -> some View { - if self.discovery.bridges.isEmpty { + if self.bridgeController.bridges.isEmpty { Text("No bridges found yet.") .foregroundStyle(.secondary) } else { let connectedID = self.appModel.connectedBridgeID - let rows = self.discovery.bridges.filter { bridge in + let rows = self.bridgeController.bridges.filter { bridge in let isConnected = bridge.stableID == connectedID switch showing { case .all: @@ -218,6 +206,7 @@ struct SettingsTab: View { private func connect(_ bridge: BridgeDiscoveryModel.DiscoveredBridge) async { self.connectingBridgeID = bridge.id self.preferredBridgeStableID = bridge.stableID + BridgeSettingsStore.savePreferredBridgeStableID(bridge.stableID) defer { self.connectingBridgeID = nil } do { @@ -265,16 +254,6 @@ struct SettingsTab: View { } } - private func pickAutoConnectBridge(from bridges: [BridgeDiscoveryModel.DiscoveredBridge]) -> BridgeDiscoveryModel - .DiscoveredBridge? { - if !self.preferredBridgeStableID.isEmpty, - let match = bridges.first(where: { $0.stableID == self.preferredBridgeStableID }) - { - return match - } - return bridges.first - } - private static func primaryIPv4Address() -> String? { var addrList: UnsafeMutablePointer? guard getifaddrs(&addrList) == 0, let first = addrList else { return nil } diff --git a/apps/macos/Sources/Clawdis/Bridge/BridgeServer.swift b/apps/macos/Sources/Clawdis/Bridge/BridgeServer.swift index 63e630678..9295d94e1 100644 --- a/apps/macos/Sources/Clawdis/Bridge/BridgeServer.swift +++ b/apps/macos/Sources/Clawdis/Bridge/BridgeServer.swift @@ -1,6 +1,6 @@ import AppKit -import ClawdisProtocol import ClawdisKit +import ClawdisProtocol import Foundation import Network import OSLog @@ -174,6 +174,7 @@ actor BridgeServer { deliver: false, to: nil, channel: "last") + case "agent.request": guard let json = evt.payloadJSON, let data = json.data(using: .utf8) else { return @@ -199,6 +200,7 @@ actor BridgeServer { deliver: link.deliver, to: to, channel: channel ?? "last") + default: break } @@ -234,7 +236,7 @@ actor BridgeServer { } do { - let data = try await GatewayConnection.shared.request(method: req.method, params: params, timeoutMs: 30_000) + let data = try await GatewayConnection.shared.request(method: req.method, params: params, timeoutMs: 30000) guard let json = String(data: data, encoding: .utf8) else { return BridgeRPCResponse( id: req.id, @@ -303,7 +305,8 @@ actor BridgeServer { let payloadJSON = payloadData.flatMap { String(data: $0, encoding: .utf8) } struct MinimalChat: Codable { var sessionKey: String } - let sessionKey = payloadData.flatMap { try? JSONDecoder().decode(MinimalChat.self, from: $0) }?.sessionKey + let sessionKey = payloadData.flatMap { try? JSONDecoder().decode(MinimalChat.self, from: $0) }? + .sessionKey if let sessionKey { for nodeId in subscribedNodes { guard self.chatSubscriptions[nodeId]?.contains(sessionKey) == true else { continue } diff --git a/apps/macos/Sources/Clawdis/WebChatSwiftUI.swift b/apps/macos/Sources/Clawdis/WebChatSwiftUI.swift index 212c38798..fa81c19ca 100644 --- a/apps/macos/Sources/Clawdis/WebChatSwiftUI.swift +++ b/apps/macos/Sources/Clawdis/WebChatSwiftUI.swift @@ -33,7 +33,7 @@ struct MacGatewayChatTransport: ClawdisChatTransport, Sendable { "message": AnyCodable(message), "thinking": AnyCodable(thinking), "idempotencyKey": AnyCodable(idempotencyKey), - "timeoutMs": AnyCodable(30_000), + "timeoutMs": AnyCodable(30000), ] if !attachments.isEmpty { diff --git a/apps/macos/Sources/ClawdisProtocol/GatewayModels.swift b/apps/macos/Sources/ClawdisProtocol/GatewayModels.swift index e9cfcdfb6..873f8498e 100644 --- a/apps/macos/Sources/ClawdisProtocol/GatewayModels.swift +++ b/apps/macos/Sources/ClawdisProtocol/GatewayModels.swift @@ -482,28 +482,35 @@ public struct NodePairVerifyParams: Codable { } } +public struct NodeListParams: Codable { +} + public struct NodeInvokeParams: Codable { public let nodeid: String public let command: String public let params: AnyCodable? public let timeoutms: Int? + public let idempotencykey: String public init( nodeid: String, command: String, params: AnyCodable?, - timeoutms: Int? + timeoutms: Int?, + idempotencykey: String ) { self.nodeid = nodeid self.command = command self.params = params self.timeoutms = timeoutms + self.idempotencykey = idempotencykey } private enum CodingKeys: String, CodingKey { case nodeid = "nodeId" case command case params case timeoutms = "timeoutMs" + case idempotencykey = "idempotencyKey" } } diff --git a/dist/protocol.schema.json b/dist/protocol.schema.json index 9a2eb0361..3bd16b52d 100644 --- a/dist/protocol.schema.json +++ b/dist/protocol.schema.json @@ -181,6 +181,10 @@ "minLength": 1, "type": "string" }, + "platform": { + "minLength": 1, + "type": "string" + }, "mode": { "minLength": 1, "type": "string" @@ -530,6 +534,10 @@ "minLength": 1, "type": "string" }, + "platform": { + "minLength": 1, + "type": "string" + }, "mode": { "minLength": 1, "type": "string" @@ -605,6 +613,10 @@ "minLength": 1, "type": "string" }, + "platform": { + "minLength": 1, + "type": "string" + }, "mode": { "minLength": 1, "type": "string" @@ -906,6 +918,39 @@ "token" ] }, + "NodeListParams": { + "additionalProperties": false, + "type": "object", + "properties": {} + }, + "NodeInvokeParams": { + "additionalProperties": false, + "type": "object", + "properties": { + "nodeId": { + "minLength": 1, + "type": "string" + }, + "command": { + "minLength": 1, + "type": "string" + }, + "params": {}, + "timeoutMs": { + "minimum": 0, + "type": "integer" + }, + "idempotencyKey": { + "minLength": 1, + "type": "string" + } + }, + "required": [ + "nodeId", + "command", + "idempotencyKey" + ] + }, "SessionsListParams": { "additionalProperties": false, "type": "object", diff --git a/docs/ios/connect.md b/docs/ios/connect.md index e2a46ca18..8917a3b25 100644 --- a/docs/ios/connect.md +++ b/docs/ios/connect.md @@ -54,6 +54,14 @@ More debugging notes: `docs/bonjour.md`. In Iris: - Pick the discovered bridge (or hit refresh). - If not paired yet, Iris will initiate pairing automatically. +- After the first successful pairing, Iris will auto-reconnect to the **last bridge** on launch (including after reinstall), as long as the iOS Keychain entry is still present. + +### Connection indicator (always visible) + +The Settings tab icon shows a small status dot: +- **Green**: connected to the bridge +- **Yellow**: connecting +- **Red**: not connected / error ## 4) Approve pairing (CLI) @@ -119,7 +127,8 @@ The response includes `base64` PNG data (for debugging/verification). - **iOS in background:** all `screen.*` commands fail fast with `NODE_BACKGROUND_UNAVAILABLE` (bring Iris to foreground). - **mDNS blocked:** some networks block multicast; use a different LAN or plan a tailnet-capable bridge (see `docs/discovery.md`). - **Wrong node selector:** `--node` can be the node id (UUID), display name (e.g. `iOS Node`), IP, or an unambiguous prefix. If it’s ambiguous, the CLI will tell you. -- **Stale pairing:** if the token is lost, Iris must pair again; approve a new pending request. +- **Stale pairing / Keychain cleared:** if the pairing token is missing (or iOS Keychain was wiped), Iris must pair again; approve a new pending request. +- **App reinstall but no reconnect:** Iris restores `instanceId` + last bridge preference from Keychain; if it still comes up “unpaired”, verify Keychain persistence on your device/simulator and re-pair once. ## Related docs