From 336048441c7295aea8daccd55e0896a5d557c74f Mon Sep 17 00:00:00 2001 From: Peter Steinberger Date: Fri, 2 Jan 2026 01:19:22 +0100 Subject: [PATCH] docs: add imessage rpc and groups docs --- CHANGELOG.md | 1 + README.md | 4 +-- docs/agent-send.md | 2 +- docs/configuration.md | 2 +- docs/cron.md | 4 +-- docs/groups.md | 56 ++++++++++++++++++++++++++++++++++++++ docs/imessage.md | 63 +++++++++++++++++++++++++++++++++++++++++++ docs/index.md | 10 ++++--- docs/rpc.md | 35 ++++++++++++++++++++++++ docs/surface.md | 4 +-- docs/wizard.md | 5 ++-- 11 files changed, 173 insertions(+), 13 deletions(-) create mode 100644 docs/groups.md create mode 100644 docs/imessage.md create mode 100644 docs/rpc.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 7c79479de..6c2e8c912 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,7 @@ - Nix mode: opt-in declarative config + read-only settings UI when `CLAWDIS_NIX_MODE=1` (thanks @joshp123 for the persistence — earned my trust; I'll merge these going forward). - Agent runtime: accept legacy `Z_AI_API_KEY` for Z.AI provider auth (maps to `ZAI_API_KEY`). - Signal: add `signal-cli` JSON-RPC support for send/receive via the Signal provider. +- iMessage: add imsg JSON-RPC integration (stdio), chat_id routing, and group chat support. - Chat UI: add recent-session dropdown switcher (main first) in macOS/iOS/Android + Control UI. - Discord: allow agent-triggered reactions via `clawdis_discord` when enabled, and surface message ids in context. - Tests: add a Z.AI live test gate for smoke validation when keys are present. diff --git a/README.md b/README.md index b1bd9c6ac..a8cb7900b 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@

**Clawdis** is a *personal AI assistant* you run on your own devices. -It answers you on the surfaces you already use (WhatsApp, Telegram, Discord, WebChat), can speak and listen on macOS/iOS, and can render a live Canvas you control. The Gateway is just the control plane — the product is the assistant. +It answers you on the surfaces you already use (WhatsApp, Telegram, Discord, iMessage, WebChat), can speak and listen on macOS/iOS, and can render a live Canvas you control. The Gateway is just the control plane — the product is the assistant. If you want a private, single-user assistant that feels local, fast, and always-on, this is it. @@ -45,7 +45,7 @@ Your surfaces ## What Clawdis does - **Personal assistant** — one user, one identity, one memory surface. -- **Multi-surface inbox** — WhatsApp, Telegram, Discord, WebChat, macOS, iOS. Signal support via `signal-cli` (see `docs/signal.md`). +- **Multi-surface inbox** — WhatsApp, Telegram, Discord, iMessage, WebChat, macOS, iOS. Signal support via `signal-cli` (see `docs/signal.md`). iMessage uses `imsg` (see `docs/imessage.md`). - **Voice wake + push-to-talk** — local speech recognition on macOS/iOS. - **Canvas** — a live visual workspace you can drive from the agent. - **Automation-ready** — browser control, media handling, and tool streaming. diff --git a/docs/agent-send.md b/docs/agent-send.md index 097f72d18..08f0124ec 100644 --- a/docs/agent-send.md +++ b/docs/agent-send.md @@ -18,4 +18,4 @@ read_when: - Output: - Default: prints text (and `MEDIA:` lines) to stdout. - `--json`: prints structured payloads + meta. -- Optional: `--deliver` sends the reply back to the selected provider (requires `--to` for WhatsApp). +- Optional: `--deliver` sends the reply back to the selected provider (`whatsapp`, `telegram`, `discord`, `signal`, `imessage`). diff --git a/docs/configuration.md b/docs/configuration.md index b23df2acc..421bf6a04 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -88,7 +88,7 @@ Allowlist of E.164 phone numbers that may trigger auto-replies. ### `routing.groupChat` -Group messages default to **require mention** (either metadata mention or regex patterns). +Group messages default to **require mention** (either metadata mention or regex patterns). Applies to WhatsApp, Telegram, Discord, and iMessage group chats. ```json5 { diff --git a/docs/cron.md b/docs/cron.md index eeb7b152a..f225f09e8 100644 --- a/docs/cron.md +++ b/docs/cron.md @@ -75,7 +75,7 @@ Each job is a JSON object with stable keys (unknown keys ignored for forward com - For `sessionTarget:"main"`, `wakeMode` controls whether we trigger the heartbeat immediately or just enqueue and wait. - `payload` (one of) - `{"kind":"systemEvent","text":string}` (enqueue as `System:`) - - `{"kind":"agentTurn","message":string,"deliver"?:boolean,"channel"?: "last"|"whatsapp"|"telegram","to"?:string,"timeoutSeconds"?:number}` + - `{"kind":"agentTurn","message":string,"deliver"?:boolean,"channel"?: "last"|"whatsapp"|"telegram"|"discord"|"signal"|"imessage","to"?:string,"timeoutSeconds"?:number}` - `isolation` (optional; only meaningful for isolated jobs) - `{"postToMainPrefix"?: string}` - `runtime` (optional) @@ -264,7 +264,7 @@ Add a `cron` command group (all commands should also support `--json` where sens - `--wake now|next-heartbeat` - payload flags (choose one): - `--system-event ""` - - `--message "" [--deliver] [--channel last|whatsapp|telegram|discord] [--to ]` + - `--message "" [--deliver] [--channel last|whatsapp|telegram|discord|signal|imessage] [--to ]` - `clawdis cron edit ...` (patch-by-flags, non-interactive) - `clawdis cron rm ` diff --git a/docs/groups.md b/docs/groups.md new file mode 100644 index 000000000..bc5aa96eb --- /dev/null +++ b/docs/groups.md @@ -0,0 +1,56 @@ +--- +summary: "Group chat behavior across surfaces (WhatsApp/Telegram/Discord/iMessage)" +read_when: + - Changing group chat behavior or mention gating +--- +# Groups + +Clawdis treats group chats consistently across surfaces: WhatsApp, Telegram, Discord, iMessage. + +## Session keys +- Group sessions use `group:` in `ctx.From`. +- Direct chats use the main session (or per-sender if configured). +- Heartbeats are skipped for group sessions. + +## Mention gating (default) +Group messages require a mention unless overridden per group. + +```json5 +{ + routing: { + groupChat: { + requireMention: true, + mentionPatterns: ["@clawd", "clawdbot", "\\+15555550123"], + historyLimit: 50 + } + } +} +``` + +Notes: +- `mentionPatterns` are case-insensitive regexes. +- Surfaces that provide explicit mentions still pass; patterns are a fallback. + +## Activation (owner-only) +Group owners can toggle per-group activation: +- `/activation mention` +- `/activation always` + +Owner is determined by `routing.allowFrom` (or the bot’s default identity when unset). + +## Context fields +Group inbound payloads set: +- `ChatType=group` +- `GroupSubject` (if known) +- `GroupMembers` (if known) +- `WasMentioned` (mention gating result) + +The agent system prompt includes a group intro on the first turn of a new group session. + +## iMessage specifics +- Prefer `chat_id:` when routing or allowlisting. +- List chats: `imsg chats --limit 20`. +- Group replies always go back to the same `chat_id`. + +## WhatsApp specifics +See `docs/group-messages.md` for WhatsApp-only behavior (history injection, mention handling details). diff --git a/docs/imessage.md b/docs/imessage.md new file mode 100644 index 000000000..ff0cf9e23 --- /dev/null +++ b/docs/imessage.md @@ -0,0 +1,63 @@ +--- +summary: "iMessage support via imsg (JSON-RPC over stdio), setup, and chat_id routing" +read_when: + - Setting up iMessage support + - Debugging iMessage send/receive +--- +# iMessage (imsg) + +Status: external CLI integration. No daemon. + +## Model +- Clawdis spawns `imsg rpc` as a child process. +- JSON-RPC runs over stdin/stdout (one JSON object per line). +- Gateway owns the process; no TCP port needed. + +## Requirements +- macOS with Messages signed in. +- Full Disk Access for Clawdis + the `imsg` binary (Messages DB access). +- Automation permission for Messages when sending. + +## Config + +```json5 +{ + imessage: { + enabled: true, + cliPath: "imsg", + dbPath: "~/Library/Messages/chat.db", + allowFrom: ["+15555550123", "user@example.com", "chat_id:123"], + includeAttachments: false, + mediaMaxMb: 16, + service: "auto", + region: "US" + } +} +``` + +Notes: +- `allowFrom` accepts handles (phone/email) or `chat_id:` entries. +- `service` defaults to `auto` (use `imessage` or `sms` to pin). +- `region` is only used for SMS targeting. + +## Addressing / targets + +Prefer `chat_id` for stable routing: +- `chat_id:123` (preferred) +- `chat_guid:...` (fallback) +- `chat_identifier:...` (fallback) +- direct handles: `imessage:+1555` / `sms:+1555` / `user@example.com` + +List chats: +``` +imsg chats --limit 20 +``` + +## Group chat behavior +- Group messages set `ChatType=group`, `GroupSubject`, and `GroupMembers`. +- Group activation respects `routing.groupChat.requireMention` and `mentionPatterns`. +- Replies go back to the same `chat_id` (group or direct). + +## Troubleshooting +- `clawdis gateway call providers.status --params '{"probe":true}'` +- Verify `imsg` is on PATH and has access to Messages DB. diff --git a/docs/index.md b/docs/index.md index 98bbc0e71..72ede08e6 100644 --- a/docs/index.md +++ b/docs/index.md @@ -13,7 +13,7 @@ read_when:

- WhatsApp + Telegram + Discord gateway for AI agents (Pi).
+ WhatsApp + Telegram + Discord + iMessage gateway for AI agents (Pi).
Send a message, get an agent response — from your pocket.

@@ -23,7 +23,7 @@ read_when: Clawd setup

-CLAWDIS bridges WhatsApp (via WhatsApp Web / Baileys), Telegram (Bot API / grammY), and Discord (Bot API / discord.js) to coding agents like [Pi](https://github.com/badlogic/pi-mono). +CLAWDIS bridges WhatsApp (via WhatsApp Web / Baileys), Telegram (Bot API / grammY), Discord (Bot API / discord.js), and iMessage (imsg CLI) to coding agents like [Pi](https://github.com/badlogic/pi-mono). It’s built for [Clawd](https://clawd.me), a space lobster who needed a TARDIS. ## How it works @@ -61,6 +61,7 @@ Most operations flow through the **Gateway** (`clawdis gateway`), a single long- - 📱 **WhatsApp Integration** — Uses Baileys for WhatsApp Web protocol - ✈️ **Telegram Bot** — DMs + groups via grammY - 🎮 **Discord Bot** — DMs + guild channels via discord.js +- 💬 **iMessage** — Local imsg CLI integration (macOS) - 🤖 **Agent bridge** — Pi (RPC mode) with tool streaming - 💬 **Sessions** — Direct chats collapse into shared `main` (default); groups are isolated - 👥 **Group Chat Support** — Mention-based by default; owner can toggle `/activation always|mention` @@ -121,6 +122,7 @@ Example: - [Skills](./skills.md) - [Skills config](./skills-config.md) - [Workspace templates](./templates/AGENTS.md) + - [RPC adapters](./rpc.md) - [Gateway runbook](./gateway.md) - [Nodes (iOS/Android)](./nodes.md) - [Web surfaces (Control UI)](./web.md) @@ -131,7 +133,9 @@ Example: - [Control UI (browser)](./control-ui.md) - [Telegram](./telegram.md) - [Discord](./discord.md) - - [Group messages](./group-messages.md) + - [iMessage](./imessage.md) + - [Groups](./groups.md) + - [WhatsApp group messages](./group-messages.md) - [Media: images](./images.md) - [Media: audio](./audio.md) - Ops and safety: diff --git a/docs/rpc.md b/docs/rpc.md new file mode 100644 index 000000000..08ff5abd4 --- /dev/null +++ b/docs/rpc.md @@ -0,0 +1,35 @@ +--- +summary: "RPC adapters for external CLIs (signal-cli, imsg) and gateway patterns" +read_when: + - Adding or changing external CLI integrations + - Debugging RPC adapters (signal-cli, imsg) +--- +# RPC adapters + +Clawdis integrates external CLIs via JSON-RPC. Two patterns are used today. + +## Pattern A: HTTP daemon (signal-cli) +- `signal-cli` runs as a daemon with JSON-RPC over HTTP. +- Event stream is SSE (`/api/v1/events`). +- Health probe: `/api/v1/check`. +- Clawdis owns lifecycle when `signal.autoStart=true`. + +See `docs/signal.md` for setup and endpoints. + +## Pattern B: stdio child process (imsg) +- Clawdis spawns `imsg rpc` as a child process. +- JSON-RPC is line-delimited over stdin/stdout (one JSON object per line). +- No TCP port, no daemon required. + +Core methods used: +- `watch.subscribe` → notifications (`method: "message"`) +- `watch.unsubscribe` +- `send` +- `chats.list` (probe/diagnostics) + +See `docs/imessage.md` for setup and addressing (`chat_id` preferred). + +## Adapter guidelines +- Gateway owns the process (start/stop tied to provider lifecycle). +- Keep RPC clients resilient: timeouts, restart on exit. +- Prefer stable IDs (e.g., `chat_id`) over display strings. diff --git a/docs/surface.md b/docs/surface.md index 2967d9afc..9135e80bc 100644 --- a/docs/surface.md +++ b/docs/surface.md @@ -9,12 +9,12 @@ Updated: 2025-12-07 Goal: make replies deterministic per channel while keeping one shared context for direct chats. -- **Surfaces** (channel labels): `whatsapp`, `webchat`, `telegram`, `discord`, `voice`, etc. Add `Surface` to inbound `MsgContext` so templates/agents can log which channel a turn came from. Routing is fixed: replies go back to the origin surface; the model doesn’t choose. +- **Surfaces** (channel labels): `whatsapp`, `webchat`, `telegram`, `discord`, `imessage`, `voice`, etc. Add `Surface` to inbound `MsgContext` so templates/agents can log which channel a turn came from. Routing is fixed: replies go back to the origin surface; the model doesn’t choose. - **Reply context:** inbound replies include `ReplyToId`, `ReplyToBody`, and `ReplyToSender`, and the quoted context is appended to `Body` as a `[Replying to ...]` block. - **Canonical direct session:** All direct chats collapse into the single `main` session by default (no config needed). Groups stay `group:`, so they remain isolated. - **Session store:** Keys are resolved via `resolveSessionKey(scope, ctx, mainKey)`; the agent JSONL path lives under `~/.clawdis/sessions/.jsonl`. - **WebChat:** Always attaches to `main`, loads the full session transcript so desktop reflects cross-surface history, and writes new turns back to the same session. - **Implementation hints:** - - Set `Surface` in each ingress (WhatsApp gateway, WebChat bridge, Telegram, Discord). + - Set `Surface` in each ingress (WhatsApp gateway, WebChat bridge, Telegram, Discord, iMessage). - Keep routing deterministic: originate → same surface. Use the gateway WebSocket for sends; avoid side channels. - Do not let the agent emit “send to X” decisions; keep that policy in the host code. diff --git a/docs/wizard.md b/docs/wizard.md index cccd74bda..3e319fd6d 100644 --- a/docs/wizard.md +++ b/docs/wizard.md @@ -65,6 +65,7 @@ It does **not** install or change anything on the remote host. - Telegram: bot token. - Discord: bot token. - Signal: optional `signal-cli` install + account config. + - iMessage: local `imsg` CLI path + DB access. 6) **Daemon install** - macOS: LaunchAgent @@ -131,7 +132,7 @@ Typical fields in `~/.clawdis/clawdis.json`: - `agent.workspace` - `agent.model` / `models.providers` (if Minimax chosen) - `gateway.*` (mode, bind, auth, tailscale) -- `telegram.botToken`, `discord.token`, `signal.*` +- `telegram.botToken`, `discord.token`, `signal.*`, `imessage.*` - `skills.install.nodeManager` - `wizard.lastRunAt` - `wizard.lastRunVersion` @@ -146,5 +147,5 @@ Sessions are stored under `~/.clawdis/sessions/`. - macOS app onboarding: `docs/onboarding.md` - Config reference: `docs/configuration.md` -- Providers: `docs/whatsapp.md`, `docs/telegram.md`, `docs/discord.md`, `docs/signal.md` +- Providers: `docs/whatsapp.md`, `docs/telegram.md`, `docs/discord.md`, `docs/signal.md`, `docs/imessage.md` - Skills: `docs/skills.md`, `docs/skills-config.md`