diff --git a/docs/compaction.md b/docs/compaction.md deleted file mode 100644 index c09082de6..000000000 --- a/docs/compaction.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -summary: "Compaction modes and configuration" -read_when: - - You want to configure compaction summarization behavior - - You are tuning compaction settings in clawdbot.json ---- -# Compaction - -Compaction summarizes older session history so the conversation stays within the model context window. The summary is stored in the session JSONL history and combined with the most recent messages. - -## Modes - -`agents.defaults.compaction.mode` controls how summaries are generated. - -- `default` (default): use the built-in compaction summarizer. -- `safeguard`: uses a chunked summarization pass to avoid context overflow for very long histories. If chunked summarization fails, Clawdbot falls back to a minimal summary plus file-operation metadata. - -## Configuration - -```json5 -{ - agents: { - defaults: { - compaction: { - mode: "safeguard", - reserveTokensFloor: 20000, - memoryFlush: { - enabled: true, - softThresholdTokens: 4000 - } - } - } - } -} -``` - -## Related docs - -- [Context window + compaction behavior](/concepts/compaction) -- [Gateway configuration reference](/gateway/configuration) diff --git a/docs/docs.json b/docs/docs.json index 17df5cf75..0e15268a1 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -53,6 +53,14 @@ "source": "/context/", "destination": "/concepts/context" }, + { + "source": "/compaction", + "destination": "/concepts/compaction" + }, + { + "source": "/compaction/", + "destination": "/concepts/compaction" + }, { "source": "/minimax", "destination": "/providers/minimax" @@ -557,14 +565,6 @@ "source": "/concepts/provider-routing/", "destination": "/concepts/channel-routing" }, - { - "source": "/refactor/provider-plugin", - "destination": "/refactor/channel-plugin" - }, - { - "source": "/refactor/provider-plugin/", - "destination": "/refactor/channel-plugin" - }, { "source": "/queue", "destination": "/concepts/queue" @@ -758,6 +758,7 @@ "install/index", "install/installer", "install/updating", + "install/uninstall", "install/ansible", "install/nix", "install/docker", @@ -797,6 +798,7 @@ "concepts/channel-routing", "concepts/messages", "concepts/streaming", + "concepts/markdown-formatting", "concepts/groups", "concepts/group-messages", "concepts/typing-indicators", @@ -823,11 +825,14 @@ "gateway/configuration-examples", "gateway/authentication", "gateway/openai-http-api", + "gateway/cli-backends", + "gateway/local-models", "gateway/background-process", "gateway/health", "gateway/heartbeat", "gateway/doctor", "gateway/logging", + "logging", "gateway/security", "gateway/sandbox-vs-tool-policy-vs-elevated", "gateway/sandboxing", @@ -875,9 +880,11 @@ "providers/models", "providers/openai", "providers/anthropic", + "bedrock", "providers/moonshot", "providers/minimax", "providers/openrouter", + "providers/synthetic", "providers/opencode", "providers/glm", "providers/zai" @@ -898,11 +905,13 @@ "pages": [ "tools", "plugin", + "plugins/voice-call", "tools/exec", "tools/web", "tools/apply-patch", "tools/elevated", "tools/browser", + "tools/browser-login", "tools/chrome-extension", "tools/browser-linux-troubleshooting", "tools/slash-commands", diff --git a/docs/experiments/proposals/hooks-gmail-model.md b/docs/experiments/proposals/hooks-gmail-model.md deleted file mode 100644 index a33c25c0a..000000000 --- a/docs/experiments/proposals/hooks-gmail-model.md +++ /dev/null @@ -1,306 +0,0 @@ ---- -summary: "Spec for hooks.gmail.model - cheaper model for Gmail PubSub processing" -read_when: - - Implementing hooks.gmail.model feature - - Modifying Gmail hook processing - - Working on hook model selection ---- -# hooks.gmail.model: Cheaper Model for Gmail PubSub Processing - -## Problem - -Gmail PubSub hook processing (`/gmail-pubsub`) currently uses the session's primary model (`agents.defaults.model.primary`), which may be an expensive model like `claude-opus-4-5`. For automated email processing that doesn't require the most capable model, this wastes tokens/cost. - -## Solution - -Add `hooks.gmail.model` config option to specify an optional cheaper model for Gmail PubSub processing, with intelligent fallback to the primary model on auth/rate-limit/timeout failures. - -## Config Structure - -```json5 -{ - hooks: { - gmail: { - account: "user@gmail.com", - // ... existing gmail config ... - - // NEW: Optional model override for Gmail hook processing - model: "openrouter/meta-llama/llama-3.3-70b-instruct:free", - - // NEW: Optional thinking level override - thinking: "off" - } - } -} -``` - -### Fields - -| Field | Type | Default | Description | -|-------|------|---------|-------------| -| `hooks.gmail.model` | `string` | (none) | Model to use for Gmail hook processing. Accepts `provider/model` refs or aliases from `agents.defaults.models`. | -| `hooks.gmail.thinking` | `string` | (inherited) | Thinking level override (`off`, `minimal`, `low`, `medium`, `high`, `xhigh`; GPT-5.2 + Codex models only). If unset, inherits from `agents.defaults.thinkingDefault` or model's default. | - -### Alias Support - -`hooks.gmail.model` accepts: -- Full refs: `"openrouter/meta-llama/llama-3.3-70b-instruct:free"` -- Aliases from `agents.defaults.models`: `"Opus"`, `"Sonnet"`, `"GLM"` - -Resolution uses `buildModelAliasIndex()` from `model-selection.ts`. - -## Fallback Behavior - -### Fallback Triggers - -Auth, rate-limit, and timeout errors trigger fallback: -- `401 Unauthorized` -- `403 Forbidden` -- `429 Too Many Requests` -- Timeouts (provider hangs / network timeouts) - -Other errors (500s, content errors) fail in place. - -### Fallback Chain - -``` -hooks.gmail.model (if set) - ↓ (on auth/rate-limit/timeout) -agents.defaults.model.fallbacks[0..n] - ↓ (exhausted) -agents.defaults.model.primary -``` - -### Uncatalogued Model - -If `hooks.gmail.model` is set but not found in the model catalog or allowlist: -- **Config load**: Log warning (surfaced in `clawdbot doctor`) -- **Allowlist**: If `agents.defaults.models` is set and the model isn't listed, the hook falls back to primary. - -### Cooldown Integration - -Uses existing model-failover cooldown from `model-failover.ts`: -- After auth/rate-limit failure, model enters cooldown -- Next hook invocation respects cooldown before retrying -- Integrates with auth profile rotation - -## Implementation - -### Type Changes - -```typescript -// src/config/types.ts -export type HooksGmailConfig = { - account?: string; - label?: string; - // ... existing fields ... - - /** Optional model override for Gmail hook processing (provider/model or alias). */ - model?: string; - /** Optional thinking level override for Gmail hook processing. */ - thinking?: "off" | "minimal" | "low" | "medium" | "high"; -}; -``` - -### Model Resolution - -New function in `src/cron/isolated-agent.ts` or `src/agents/model-selection.ts`: - -```typescript -export function resolveHooksGmailModel(params: { - cfg: ClawdbotConfig; - defaultProvider: string; - defaultModel: string; -}): { provider: string; model: string; isHooksOverride: boolean } | null { - const hooksModel = params.cfg.hooks?.gmail?.model; - if (!hooksModel) return null; - - const aliasIndex = buildModelAliasIndex({ - cfg: params.cfg, - defaultProvider: params.defaultProvider, - }); - - const resolved = resolveModelRefFromString({ - raw: hooksModel, - defaultProvider: params.defaultProvider, - aliasIndex, - }); - - if (!resolved) return null; - return { - provider: resolved.ref.provider, - model: resolved.ref.model, - isHooksOverride: true, - }; -} -``` - -### Processing Flow - -In `runCronIsolatedAgentTurn()` (or new wrapper for hooks): - -```typescript -// Resolve model - prefer hooks.gmail.model for Gmail hooks -const isGmailHook = params.sessionKey.startsWith("hook:gmail:"); -const hooksModelRef = isGmailHook - ? resolveHooksGmailModel({ cfg, defaultProvider, defaultModel }) - : null; - -const { provider, model } = hooksModelRef ?? resolveConfiguredModelRef({ - cfg: params.cfg, - defaultProvider: DEFAULT_PROVIDER, - defaultModel: DEFAULT_MODEL, -}); - -// Run with fallback - on auth/rate-limit/timeout, fall through to agents.defaults.model.fallbacks -const fallbackResult = await runWithModelFallback({ - cfg: params.cfg, - provider, - model, - hooksOverride: hooksModelRef?.isHooksOverride, - run: (providerOverride, modelOverride) => runEmbeddedPiAgent({ - // ... existing params ... - }), -}); -``` - -### Fallback Detection - -Extend `runWithModelFallback()` to detect auth/rate-limit: - -```typescript -function isAuthRateLimitError(err: unknown): boolean { - if (err instanceof ApiError) { - return [401, 403, 429].includes(err.status); - } - // Check for common patterns in error messages - const msg = String(err).toLowerCase(); - return msg.includes("unauthorized") - || msg.includes("rate limit") - || msg.includes("quota exceeded"); -} -``` - -## Validation - -### Config Load Time - -In config validation (for `clawdbot doctor`): - -```typescript -if (cfg.hooks?.gmail?.model) { - const resolved = resolveHooksGmailModel({ cfg, defaultProvider, defaultModel }); - if (!resolved) { - issues.push({ - path: "hooks.gmail.model", - message: `Model "${cfg.hooks.gmail.model}" could not be resolved`, - }); - } else { - const catalog = await loadModelCatalog({ config: cfg }); - const key = modelKey(resolved.provider, resolved.model); - const inCatalog = catalog.some(e => modelKey(e.provider, e.id) === key); - if (!inCatalog) { - issues.push({ - path: "hooks.gmail.model", - message: `Model "${key}" not found in agents.defaults.models catalog (will fall back to primary)`, - }); - } - } -} -``` - -### Runtime - -At hook invocation time, validate and fall back: -- If model not in catalog → log warning, use primary -- If model auth fails → log warning, enter cooldown, fall back - -## Observability - -### Log Messages - -``` -[hooks] Gmail hook: using model openrouter/meta-llama/llama-3.3-70b-instruct:free -[hooks] Gmail hook: model llama auth failed (429), falling back to claude-opus-4-5 -``` - -### Hook Event Summary - -Include fallback info in the hook summary sent to session: - -``` -Hook Gmail (fallback:llama→opus): -``` - -## Hot Reload - -`hooks.gmail.model` and `hooks.gmail.thinking` are hot-reloadable: -- Changes apply to the next hook invocation -- No gateway restart required -- Hooks config is already in the hot-reload matrix - -## Test Plan - -### Unit Tests - -1. **Model resolution** (`model-selection.test.ts`): - - `resolveHooksGmailModel()` with valid ref - - `resolveHooksGmailModel()` with alias - - `resolveHooksGmailModel()` with invalid input → null - -2. **Config validation** (`config.test.ts`): - - Warning on uncatalogued model - - No warning on valid model - - Graceful handling of missing hooks.gmail section - -3. **Fallback triggers** (`model-fallback.test.ts`): - - 401/403/429 → triggers fallback - - timeouts → triggers fallback - - 500/content error → no fallback - - Content error → no fallback - -### Integration Tests - -1. **Hook processing** (`server.hooks.test.ts`): - - Gmail hook uses `hooks.gmail.model` when set - - Fallback to primary on auth failure - - Thinking level override applied - -2. **Hot reload** (`config-reload.test.ts`): - - Change `hooks.gmail.model` → next hook uses new model - -## Documentation - -Update `docs/gateway/configuration.md`: - -```json5 -{ - hooks: { - gmail: { - account: "user@gmail.com", - topic: "projects/my-project/topics/gmail-watch", - // ... existing config ... - - // Optional: Use a cheaper model for Gmail processing - // Falls back to agents.defaults.model.primary on auth/rate-limit errors - model: "openrouter/meta-llama/llama-3.3-70b-instruct:free", - - // Optional: Override thinking level for Gmail processing - thinking: "off" - } - } -} -``` - -## Scope Limitation - -This PR is Gmail-specific. Future hooks (`hooks.github.model`, etc.) would follow the same pattern but are out of scope. - -## Changelog Entry - -``` -- feat: add hooks.gmail.model for cheaper Gmail PubSub processing (#XXX) - - Falls back to agents.defaults.model.primary on auth/rate-limit/timeouts (401/403/429) - - Supports aliases from agents.defaults.models - - Add hooks.gmail.thinking override -``` diff --git a/docs/refactor/channel-plugin.md b/docs/refactor/channel-plugin.md deleted file mode 100644 index c5cd8bc1d..000000000 --- a/docs/refactor/channel-plugin.md +++ /dev/null @@ -1,110 +0,0 @@ ---- -summary: "Channel plugin refactor implementation notes (registry, status, gateway/runtime)" -read_when: - - Adding or refactoring channel plugin wiring - - Moving channel-specific behavior into plugin hooks ---- - -# Channel Plugins — Implementation Notes - -Goal: make chat channels (iMessage, Discord, etc.) pluggable with minimal wiring and shared UX/state paths. - -## Architecture Overview -- Registry: `src/channels/plugins/index.ts` owns the plugin list. -- Channel dock: `src/channels/dock.ts` owns lightweight channel metadata used by shared flows (reply, command auth, block streaming) without importing full plugins. -- IDs/aliases: `src/channels/registry.ts` owns stable channel ids + input aliases. -- Shape: `src/channels/plugins/types.ts` defines the plugin contract. -- Gateway: `src/gateway/server-channels.ts` drives start/stop + runtime snapshots via plugins. -- Outbound: `src/infra/outbound/deliver.ts` routes through plugin outbound when present. -- Outbound delivery loads **outbound adapters** on-demand via `src/channels/plugins/outbound/load.ts` (avoid importing heavy channel plugins on hot paths). -- Reload: `src/gateway/config-reload.ts` uses plugin `reload.configPrefixes` lazily (avoid init cycles). -- CLI: `src/commands/channels/*` uses plugin list for add/remove/status/list. -- Protocol: `src/gateway/protocol/schema.ts` keeps `channels.status` schema-light (maps keyed by channel id). - -## Plugin Contract (high-level) -Each `ChannelPlugin` bundles: -- `meta`: id/labels/docs/sort order. -- `capabilities`: chatTypes + optional features (polls, media, nativeCommands, etc.). -- `config`: list/resolve/default/isConfigured/describeAccount + isEnabled + (un)configured reasons + `resolveAllowFrom` + `formatAllowFrom`. -- `outbound`: deliveryMode + chunker + resolveTarget (mode-aware) + sendText/sendMedia/sendPoll + pollMaxOptions. -- `status`: defaultRuntime + probe/audit/buildAccountSnapshot + buildChannelSummary + logSelfId + collectStatusIssues. -- `gateway`: startAccount/stopAccount with runtime context (`getStatus`/`setStatus`), plus optional `loginWithQrStart/loginWithQrWait` for gateway-owned QR login flows. -- `security`: dmPolicy + allowFrom hints used by `doctor security`. -- `heartbeat`: optional readiness checks + heartbeat recipient resolution when channels own targeting. -- `auth`: optional login hook used by `clawdbot channels login`. -- `reload`: `configPrefixes` that map to hot restarts. -- `onboarding`: optional CLI onboarding adapter (wizard UI hooks per channel). -- `agentTools`: optional channel-owned agent tools (ex: QR login). - -## Key Integration Notes -- `listChannelPlugins()` is the runtime source of truth for channel UX and wiring. -- Avoid importing `src/channels/plugins/index.ts` from shared modules (reply flow, command auth, sandbox explain). It’s intentionally “heavy” (channels may pull web login / monitor code). Use `getChannelDock()` + `normalizeChannelId()` (from `src/channels/registry.ts`) for cheap metadata, and only `getChannelPlugin()` at execution boundaries (ex: `src/auto-reply/reply/route-reply.ts`). -- WhatsApp plugin keeps Baileys-heavy login bits behind lazy imports; cheap auth file checks live in `src/web/auth-store.ts` (so outbound routing doesn’t pay Baileys import cost). -- `routeReply` delegates sending to plugin `outbound` adapters via a lazy import of `src/infra/outbound/deliver.ts` (so adding a channel is “just implement outbound adapter”, no router switches). -- Avoid static imports of channel monitors inside plugin modules. Monitors typically import the reply pipeline, which can create ESM cycles (and break Vite/Vitest SSR with TDZ errors). Prefer lazy imports inside `gateway.startAccount`. -- Debug cycle leaks quickly with: `npx -y madge --circular src/channels/plugins/index.ts`. -- Protocol v3: `channels.status` is map-based and schema-light so new channels can ship without protocol updates. -- `DEFAULT_CHAT_CHANNEL` lives in `src/channels/registry.ts` and is used anywhere we need a fallback delivery surface. -- Channel reload rules are computed lazily to avoid static init cycles in tests. -- Signal/iMessage media size limits are resolved inside their plugins. -- `normalizeChannelId()` handles aliases (ex: `imsg`, `teams`) so CLI and API inputs stay stable. -- Gateway runtime snapshot (`getRuntimeSnapshot`) is map-based: `{ channels, channelAccounts }` (no `${id}Accounts` keys). -- `channels.status` response keys (v3): - - `channelOrder: string[]` - - `channelLabels: Record` - - `channels: Record` (channel summary objects, plugin-defined) - - `channelAccounts: Record` - - `channelDefaultAccountId: Record` -- `channels.status` summary objects come from `status.buildChannelSummary` (no per-channel branching in the handler). -- `channels.status` warnings flow through `status.collectStatusIssues` per plugin. -- CLI list uses `meta.showConfigured` to decide whether to show configured state. -- CLI channel options and prompt channel lists are generated from `listChannelPlugins()` (avoid hardcoded arrays). -- Channel selection (`resolveMessageChannelSelection`) inspects `config.isEnabled` + `config.isConfigured` per plugin instead of hardcoded checks. -- Pairing flows (CLI + store) use `plugin.pairing` (`idLabel`, `normalizeAllowEntry`, `notifyApproval`) via `src/channels/plugins/pairing.ts`. -- CLI channel remove/disable delegates to `config.setAccountEnabled` + `config.deleteAccount` per plugin. -- CLI channel add delegates to `plugin.setup` for account validation, naming, and config writes (no hardcoded checks). -- Agent channel status entries are built from plugin config/status (`status.resolveAccountState` for custom state labels). -- Agent binding defaults use `meta.forceAccountBinding` to avoid hardcoded checks. -- Onboarding quickstart allowlist uses `meta.quickstartAllowFrom` to avoid hardcoded channel lists. -- `resolveChannelDefaultAccountId()` is the shared helper for picking default accounts from `accountIds` + plugin config. -- `routeReply` uses plugin outbound senders; `ChannelOutboundContext` supports `replyToId` + `threadId` and outbound delivery supports `abortSignal` for cooperative cancellation. -- Outbound target resolution (`resolveOutboundTarget`) delegates to `plugin.outbound.resolveTarget` (mode-aware, uses config allowlists when present). -- Outbound delivery results accept `meta` for channel-specific fields to avoid core type churn in new plugins. -- Agent gateway routing sets `deliveryTargetMode` and uses `resolveOutboundTarget` for implicit fallback targets when `to` is missing. -- Elevated tool allowlists (`tools.elevated.allowFrom`) are a record keyed by channel id (no schema update needed when adding channels). -- Block streaming defaults live on the plugin (`capabilities.blockStreaming`, `streaming.blockStreamingCoalesceDefaults`) instead of hardcoded channel checks. -- Channel logout routes through `channels.logout` using `gateway.logoutAccount` on each plugin (clients should call the generic method). -- Gateway message-channel normalization uses `src/channels/registry.ts` for cheap validation/normalization without plugin init cycles. -- Group mention gating now flows through `plugin.groups.resolveRequireMention` (Discord/Slack/Telegram/WhatsApp/iMessage) instead of branching in reply handlers. -- Command authorization uses `config.resolveAllowFrom` + `config.formatAllowFrom`, with `commands.enforceOwnerForCommands` and `commands.skipWhenConfigEmpty` driving channel-specific behavior. -- Security warnings (`doctor security`) use `plugin.security.resolveDmPolicy` + `plugin.security.collectWarnings`; supply `policyPath` + `allowFromPath` for accurate config hints. -- Reply threading uses `plugin.threading.resolveReplyToMode` and `plugin.threading.allowTagsWhenOff` rather than channel switches in reply helpers. -- Tool auto-threading context flows through `plugin.threading.buildToolContext` (e.g., Slack threadTs injection). -- Messaging tool dedupe now relies on `plugin.messaging.normalizeTarget` for channel-specific target normalization. -- Message tool + CLI action dispatch now use `plugin.actions.listActions` + `plugin.actions.handleAction`; use `plugin.actions.supportsAction` for dispatch-only gating when you still want fallback send/poll. -- Session announce targets can opt into `meta.preferSessionLookupForAnnounceTarget` when session keys are insufficient (e.g., WhatsApp). -- Onboarding channel setup is delegated to adapter modules under `src/channels/plugins/onboarding/*`, keeping `setupChannels` channel-agnostic. -- Onboarding registry now reads `plugin.onboarding` from each channel (no standalone onboarding map). -- Channel login flows (`clawdbot channels login`) route through `plugin.auth.login` when available. -- `clawdbot status` reports `linkChannel` (derived from `status.buildChannelSummary().linked`) instead of a hardcoded `web` field. -- Gateway `web.login.*` methods use `plugin.gatewayMethods` ownership to pick the channel (no hardcoded `normalizeChannelId("web")` in the handler). - -## CLI Commands (inline references) -- Add/remove channels: `clawdbot channels add ` / `clawdbot channels remove `. -- Inspect channel state: `clawdbot channels list`, `clawdbot channels status`. -- Link/unlink channels: `clawdbot channels login --channel ` / `clawdbot channels logout --channel `. -- Pairing approvals: `clawdbot pairing list `, `clawdbot pairing approve `. - -## Adding a Channel (checklist) -1) Create `src/channels/plugins/.ts` exporting `ChannelPlugin`. -2) Register in `src/channels/plugins/index.ts` and update `src/channels/registry.ts` (ids/aliases/meta) if needed. -3) Add a dock entry in `src/channels/dock.ts` for any shared behavior (capabilities, allowFrom format/resolve, mention stripping, threading, streaming chunk defaults). -4) Add `reload.configPrefixes` for hot reload when config changes. -5) Delegate to existing channel modules (send/probe/monitor) or create them. -6) If you changed the gateway protocol: run `pnpm protocol:check` (updates `dist/protocol.schema.json` + `apps/macos/Sources/ClawdbotProtocol/GatewayModels.swift`). -7) Update docs/tests for any behavior changes. - -## Cleanup Expectations -- Keep plugin files small; move heavy logic into channel modules. -- Prefer shared helpers over V2 copies. -- Update docs when behavior/inputs change. diff --git a/docs/refactor/vector-memory.md b/docs/refactor/vector-memory.md deleted file mode 100644 index e4c8e545e..000000000 --- a/docs/refactor/vector-memory.md +++ /dev/null @@ -1,127 +0,0 @@ ---- -summary: "Vector memory search design plan (per-agent, watch/lazy sync, storage)" -read_when: - - Designing or implementing vector memory search - - Adding embedding providers or sync behavior ---- - -# Vector Memory Search — Design Plan - -Goal: semantic search over **agent memory files** only, with minimal deps and -good UX defaults. Default enabled. Per-agent overrides. - -## Scope -- Sources: `MEMORY.md` + `memory/YYYY-MM-DD.md` inside the agent workspace. -- No indexing outside the workspace. No hidden paths. -- No QMD-style query expansion or rerank in v1. - -## Config Shape -Location: `agents.defaults.memorySearch` + `agents.list[].memorySearch`. - -```json5 -agents: { - defaults: { - memorySearch: { - enabled: true, - provider: "openai", // "openai" | "local" - fallback: "openai", // "openai" | "none" - model: "text-embedding-3-small", - store: { - driver: "sqlite", - path: "~/.clawdbot/memory/{agentId}.sqlite" - }, - chunking: { - tokens: 400, - overlap: 80 - }, - sync: { - onSessionStart: true, - onSearch: true, // LazySync - watch: true, // default on - watchDebounceMs: 1500, - intervalMinutes: 0 - }, - query: { - maxResults: 6, - minScore: 0.35 - } - } - }, - list: [ - { id: "peter", memorySearch: { provider: "local", sync: { watch: false } } } - ] -} -``` - -## Storage -Per-agent DB (default): `~/.clawdbot/memory/{agentId}.sqlite`. - -Tables (v1): -- `files(path PRIMARY KEY, hash, mtime, size)` -- `chunks(id PRIMARY KEY, path, start_line, end_line, hash, text, embedding, updated_at)` - -Notes: -- `hash` = content hash of chunk text. -- `embedding` stored as float[] (sqlite vec extension optional); if not using vec, - store as JSON and do linear scan in memory for small corpora. - -## Embedding Providers -Interface (core): -- `embedQuery(text): number[]` -- `embedBatch(texts[]): number[][]` - -Providers: -- `openai` (default): OpenAI embeddings via existing keys. -- `local` (optional): node-llama-cpp (GGUF). -- Fallback: when `provider: "local"` fails, fallback to OpenAI unless `fallback: "none"`. - -## Index Pipeline -1) Resolve memory file list (workspace only). -2) Read file, compute file hash/mtime. -3) Chunk by headings + token cap (overlap). -4) Embed only changed chunks (hash compare). -5) Upsert `chunks` rows, prune deleted files. - -Chunking: -- Prefer heading-aware splits. -- Max tokens + overlap; keep line ranges for snippets. - -## Sync Strategy -Default: **watch + lazy + session-start** -- `watch`: chokidar on `MEMORY.md` + `memory/**/*.md` (debounced). -- `onSearch`: if dirty, sync before search (LazySync). -- `onSessionStart`: warm index once per session. -- `intervalMinutes`: optional for long-lived sessions. - -If workspace access is read-only or missing: disable writes; return “not indexed”. - -## Query Flow -1) Embed query. -2) Cosine similarity over all chunk embeddings. -3) Return top K with `{path, startLine, endLine, snippet, score}`. -4) Model may call `memory_get` when full context needed. - -Optional v2: add FTS5 + RRF merge (FTS + vector) for quality. - -## Tool + CLI -Tools: -- `memory_search { query, maxResults?, minScore? }` -- `memory_get { path, from?, lines? }` - -CLI (optional): -- `clawdbot memory index|search|status` - -## Security + Permissions -- Indexer reads only memory files in workspace. -- No scanning outside workspace; no “sneak” reads. -- Respect sandbox `workspaceAccess` (ro = read-only; none = disabled). - -## Tests -- Chunking boundaries + line ranges. -- Hash-based incremental updates. -- Search ranking (cosine). -- Watcher debounce (fake fs). - -## Rollout -- Default enabled; if no memory files, index is empty (silent). -- No migration needed. diff --git a/docs/web/tui.md b/docs/web/tui.md deleted file mode 100644 index 155f2ba89..000000000 --- a/docs/web/tui.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -summary: "Terminal UI (TUI) for Clawdbot via the Gateway" -read_when: - - You want a terminal UI that connects to the Gateway from any machine - - You are debugging the TUI client or Gateway chat stream ---- -# TUI (Gateway chat client) - - -## What it is -- A terminal UI that connects to the Gateway WebSocket and speaks the same chat APIs as WebChat. -- Uses Gateway agent events for tool cards while streaming responses. -- Works locally (loopback) or remotely (Tailscale/SSH tunnel) without running a separate agent process. - -## Run -```bash -clawdbot tui -``` - -### Remote -```bash -clawdbot tui --url ws://127.0.0.1:18789 --token -``` -Use SSH tunneling or Tailscale to reach the Gateway WS. - -## Options -- `--url `: Gateway WebSocket URL (defaults to config `gateway.remote.url` or `ws://127.0.0.1:18789`). -- `--token `: Gateway token (if required). -- `--password `: Gateway password (if required). -- `--session `: Session key (default: `main`, or `global` when scope is global). -- `--deliver`: Deliver assistant replies to the channel (default off). -- `--thinking `: Override thinking level for sends. -- `--timeout-ms `: Agent timeout in ms (defaults to `agents.defaults.timeoutSeconds`). -- `--history-limit `: History entries to load (default 200). - -## Controls -- Enter: send message -- Esc: abort active run -- Ctrl+C: clear input (press twice to exit) -- Ctrl+D: exit -- Ctrl+L: model picker -- Ctrl+G: agent picker -- Ctrl+P: session picker -- Ctrl+O: toggle tool output expansion -- Ctrl+T: toggle thinking visibility - -## Slash commands -- `/help` -- `/status` -- `/agent ` (or `/agents`) -- `/session ` (or `/sessions`) -- `/model ` (or `/model list`, `/models`) -- `/think ` (GPT-5.2 + Codex models only) -- `/verbose ` -- `/reasoning ` (stream = Telegram draft only) -- `/cost ` -- `/elevated ` -- `/elev ` -- `/activation ` -- `/deliver ` -- `/new` or `/reset` -- `/compact [instructions]` -- `/abort` -- `/settings` -- `/exit` - -## Notes -- The TUI shows Gateway chat deltas (`event: chat`) and agent tool events. -- It registers as a Gateway client with `mode: "tui"` for presence and debugging. -- The TUI uses the **Gateway’s** model/auth config and environment. If a model - token works in your shell but not in TUI, put it in `~/.clawdbot/.env` or - enable `env.shellEnv.enabled`, then restart the Gateway.