docs: prune internal notes and doc aliases
This commit is contained in:
Peter Steinberger
@@ -1,40 +0,0 @@
|
|||||||
---
|
|
||||||
summary: "Compaction modes and configuration"
|
|
||||||
read_when:
|
|
||||||
- You want to configure compaction summarization behavior
|
|
||||||
- You are tuning compaction settings in clawdbot.json
|
|
||||||
---
|
|
||||||
# Compaction
|
|
||||||
|
|
||||||
Compaction summarizes older session history so the conversation stays within the model context window. The summary is stored in the session JSONL history and combined with the most recent messages.
|
|
||||||
|
|
||||||
## Modes
|
|
||||||
|
|
||||||
`agents.defaults.compaction.mode` controls how summaries are generated.
|
|
||||||
|
|
||||||
- `default` (default): use the built-in compaction summarizer.
|
|
||||||
- `safeguard`: uses a chunked summarization pass to avoid context overflow for very long histories. If chunked summarization fails, Clawdbot falls back to a minimal summary plus file-operation metadata.
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
```json5
|
|
||||||
{
|
|
||||||
agents: {
|
|
||||||
defaults: {
|
|
||||||
compaction: {
|
|
||||||
mode: "safeguard",
|
|
||||||
reserveTokensFloor: 20000,
|
|
||||||
memoryFlush: {
|
|
||||||
enabled: true,
|
|
||||||
softThresholdTokens: 4000
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Related docs
|
|
||||||
|
|
||||||
- [Context window + compaction behavior](/concepts/compaction)
|
|
||||||
- [Gateway configuration reference](/gateway/configuration)
|
|
||||||
@@ -53,6 +53,14 @@
|
|||||||
"source": "/context/",
|
"source": "/context/",
|
||||||
"destination": "/concepts/context"
|
"destination": "/concepts/context"
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
"source": "/compaction",
|
||||||
|
"destination": "/concepts/compaction"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"source": "/compaction/",
|
||||||
|
"destination": "/concepts/compaction"
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"source": "/minimax",
|
"source": "/minimax",
|
||||||
"destination": "/providers/minimax"
|
"destination": "/providers/minimax"
|
||||||
@@ -557,14 +565,6 @@
|
|||||||
"source": "/concepts/provider-routing/",
|
"source": "/concepts/provider-routing/",
|
||||||
"destination": "/concepts/channel-routing"
|
"destination": "/concepts/channel-routing"
|
||||||
},
|
},
|
||||||
{
|
|
||||||
"source": "/refactor/provider-plugin",
|
|
||||||
"destination": "/refactor/channel-plugin"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"source": "/refactor/provider-plugin/",
|
|
||||||
"destination": "/refactor/channel-plugin"
|
|
||||||
},
|
|
||||||
{
|
{
|
||||||
"source": "/queue",
|
"source": "/queue",
|
||||||
"destination": "/concepts/queue"
|
"destination": "/concepts/queue"
|
||||||
@@ -758,6 +758,7 @@
|
|||||||
"install/index",
|
"install/index",
|
||||||
"install/installer",
|
"install/installer",
|
||||||
"install/updating",
|
"install/updating",
|
||||||
|
"install/uninstall",
|
||||||
"install/ansible",
|
"install/ansible",
|
||||||
"install/nix",
|
"install/nix",
|
||||||
"install/docker",
|
"install/docker",
|
||||||
@@ -797,6 +798,7 @@
|
|||||||
"concepts/channel-routing",
|
"concepts/channel-routing",
|
||||||
"concepts/messages",
|
"concepts/messages",
|
||||||
"concepts/streaming",
|
"concepts/streaming",
|
||||||
|
"concepts/markdown-formatting",
|
||||||
"concepts/groups",
|
"concepts/groups",
|
||||||
"concepts/group-messages",
|
"concepts/group-messages",
|
||||||
"concepts/typing-indicators",
|
"concepts/typing-indicators",
|
||||||
@@ -823,11 +825,14 @@
|
|||||||
"gateway/configuration-examples",
|
"gateway/configuration-examples",
|
||||||
"gateway/authentication",
|
"gateway/authentication",
|
||||||
"gateway/openai-http-api",
|
"gateway/openai-http-api",
|
||||||
|
"gateway/cli-backends",
|
||||||
|
"gateway/local-models",
|
||||||
"gateway/background-process",
|
"gateway/background-process",
|
||||||
"gateway/health",
|
"gateway/health",
|
||||||
"gateway/heartbeat",
|
"gateway/heartbeat",
|
||||||
"gateway/doctor",
|
"gateway/doctor",
|
||||||
"gateway/logging",
|
"gateway/logging",
|
||||||
|
"logging",
|
||||||
"gateway/security",
|
"gateway/security",
|
||||||
"gateway/sandbox-vs-tool-policy-vs-elevated",
|
"gateway/sandbox-vs-tool-policy-vs-elevated",
|
||||||
"gateway/sandboxing",
|
"gateway/sandboxing",
|
||||||
@@ -875,9 +880,11 @@
|
|||||||
"providers/models",
|
"providers/models",
|
||||||
"providers/openai",
|
"providers/openai",
|
||||||
"providers/anthropic",
|
"providers/anthropic",
|
||||||
|
"bedrock",
|
||||||
"providers/moonshot",
|
"providers/moonshot",
|
||||||
"providers/minimax",
|
"providers/minimax",
|
||||||
"providers/openrouter",
|
"providers/openrouter",
|
||||||
|
"providers/synthetic",
|
||||||
"providers/opencode",
|
"providers/opencode",
|
||||||
"providers/glm",
|
"providers/glm",
|
||||||
"providers/zai"
|
"providers/zai"
|
||||||
@@ -898,11 +905,13 @@
|
|||||||
"pages": [
|
"pages": [
|
||||||
"tools",
|
"tools",
|
||||||
"plugin",
|
"plugin",
|
||||||
|
"plugins/voice-call",
|
||||||
"tools/exec",
|
"tools/exec",
|
||||||
"tools/web",
|
"tools/web",
|
||||||
"tools/apply-patch",
|
"tools/apply-patch",
|
||||||
"tools/elevated",
|
"tools/elevated",
|
||||||
"tools/browser",
|
"tools/browser",
|
||||||
|
"tools/browser-login",
|
||||||
"tools/chrome-extension",
|
"tools/chrome-extension",
|
||||||
"tools/browser-linux-troubleshooting",
|
"tools/browser-linux-troubleshooting",
|
||||||
"tools/slash-commands",
|
"tools/slash-commands",
|
||||||
|
|||||||
@@ -1,306 +0,0 @@
|
|||||||
---
|
|
||||||
summary: "Spec for hooks.gmail.model - cheaper model for Gmail PubSub processing"
|
|
||||||
read_when:
|
|
||||||
- Implementing hooks.gmail.model feature
|
|
||||||
- Modifying Gmail hook processing
|
|
||||||
- Working on hook model selection
|
|
||||||
---
|
|
||||||
# hooks.gmail.model: Cheaper Model for Gmail PubSub Processing
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
Gmail PubSub hook processing (`/gmail-pubsub`) currently uses the session's primary model (`agents.defaults.model.primary`), which may be an expensive model like `claude-opus-4-5`. For automated email processing that doesn't require the most capable model, this wastes tokens/cost.
|
|
||||||
|
|
||||||
## Solution
|
|
||||||
|
|
||||||
Add `hooks.gmail.model` config option to specify an optional cheaper model for Gmail PubSub processing, with intelligent fallback to the primary model on auth/rate-limit/timeout failures.
|
|
||||||
|
|
||||||
## Config Structure
|
|
||||||
|
|
||||||
```json5
|
|
||||||
{
|
|
||||||
hooks: {
|
|
||||||
gmail: {
|
|
||||||
account: "user@gmail.com",
|
|
||||||
// ... existing gmail config ...
|
|
||||||
|
|
||||||
// NEW: Optional model override for Gmail hook processing
|
|
||||||
model: "openrouter/meta-llama/llama-3.3-70b-instruct:free",
|
|
||||||
|
|
||||||
// NEW: Optional thinking level override
|
|
||||||
thinking: "off"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Fields
|
|
||||||
|
|
||||||
| Field | Type | Default | Description |
|
|
||||||
|-------|------|---------|-------------|
|
|
||||||
| `hooks.gmail.model` | `string` | (none) | Model to use for Gmail hook processing. Accepts `provider/model` refs or aliases from `agents.defaults.models`. |
|
|
||||||
| `hooks.gmail.thinking` | `string` | (inherited) | Thinking level override (`off`, `minimal`, `low`, `medium`, `high`, `xhigh`; GPT-5.2 + Codex models only). If unset, inherits from `agents.defaults.thinkingDefault` or model's default. |
|
|
||||||
|
|
||||||
### Alias Support
|
|
||||||
|
|
||||||
`hooks.gmail.model` accepts:
|
|
||||||
- Full refs: `"openrouter/meta-llama/llama-3.3-70b-instruct:free"`
|
|
||||||
- Aliases from `agents.defaults.models`: `"Opus"`, `"Sonnet"`, `"GLM"`
|
|
||||||
|
|
||||||
Resolution uses `buildModelAliasIndex()` from `model-selection.ts`.
|
|
||||||
|
|
||||||
## Fallback Behavior
|
|
||||||
|
|
||||||
### Fallback Triggers
|
|
||||||
|
|
||||||
Auth, rate-limit, and timeout errors trigger fallback:
|
|
||||||
- `401 Unauthorized`
|
|
||||||
- `403 Forbidden`
|
|
||||||
- `429 Too Many Requests`
|
|
||||||
- Timeouts (provider hangs / network timeouts)
|
|
||||||
|
|
||||||
Other errors (500s, content errors) fail in place.
|
|
||||||
|
|
||||||
### Fallback Chain
|
|
||||||
|
|
||||||
```
|
|
||||||
hooks.gmail.model (if set)
|
|
||||||
↓ (on auth/rate-limit/timeout)
|
|
||||||
agents.defaults.model.fallbacks[0..n]
|
|
||||||
↓ (exhausted)
|
|
||||||
agents.defaults.model.primary
|
|
||||||
```
|
|
||||||
|
|
||||||
### Uncatalogued Model
|
|
||||||
|
|
||||||
If `hooks.gmail.model` is set but not found in the model catalog or allowlist:
|
|
||||||
- **Config load**: Log warning (surfaced in `clawdbot doctor`)
|
|
||||||
- **Allowlist**: If `agents.defaults.models` is set and the model isn't listed, the hook falls back to primary.
|
|
||||||
|
|
||||||
### Cooldown Integration
|
|
||||||
|
|
||||||
Uses existing model-failover cooldown from `model-failover.ts`:
|
|
||||||
- After auth/rate-limit failure, model enters cooldown
|
|
||||||
- Next hook invocation respects cooldown before retrying
|
|
||||||
- Integrates with auth profile rotation
|
|
||||||
|
|
||||||
## Implementation
|
|
||||||
|
|
||||||
### Type Changes
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
// src/config/types.ts
|
|
||||||
export type HooksGmailConfig = {
|
|
||||||
account?: string;
|
|
||||||
label?: string;
|
|
||||||
// ... existing fields ...
|
|
||||||
|
|
||||||
/** Optional model override for Gmail hook processing (provider/model or alias). */
|
|
||||||
model?: string;
|
|
||||||
/** Optional thinking level override for Gmail hook processing. */
|
|
||||||
thinking?: "off" | "minimal" | "low" | "medium" | "high";
|
|
||||||
};
|
|
||||||
```
|
|
||||||
|
|
||||||
### Model Resolution
|
|
||||||
|
|
||||||
New function in `src/cron/isolated-agent.ts` or `src/agents/model-selection.ts`:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
export function resolveHooksGmailModel(params: {
|
|
||||||
cfg: ClawdbotConfig;
|
|
||||||
defaultProvider: string;
|
|
||||||
defaultModel: string;
|
|
||||||
}): { provider: string; model: string; isHooksOverride: boolean } | null {
|
|
||||||
const hooksModel = params.cfg.hooks?.gmail?.model;
|
|
||||||
if (!hooksModel) return null;
|
|
||||||
|
|
||||||
const aliasIndex = buildModelAliasIndex({
|
|
||||||
cfg: params.cfg,
|
|
||||||
defaultProvider: params.defaultProvider,
|
|
||||||
});
|
|
||||||
|
|
||||||
const resolved = resolveModelRefFromString({
|
|
||||||
raw: hooksModel,
|
|
||||||
defaultProvider: params.defaultProvider,
|
|
||||||
aliasIndex,
|
|
||||||
});
|
|
||||||
|
|
||||||
if (!resolved) return null;
|
|
||||||
return {
|
|
||||||
provider: resolved.ref.provider,
|
|
||||||
model: resolved.ref.model,
|
|
||||||
isHooksOverride: true,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Processing Flow
|
|
||||||
|
|
||||||
In `runCronIsolatedAgentTurn()` (or new wrapper for hooks):
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
// Resolve model - prefer hooks.gmail.model for Gmail hooks
|
|
||||||
const isGmailHook = params.sessionKey.startsWith("hook:gmail:");
|
|
||||||
const hooksModelRef = isGmailHook
|
|
||||||
? resolveHooksGmailModel({ cfg, defaultProvider, defaultModel })
|
|
||||||
: null;
|
|
||||||
|
|
||||||
const { provider, model } = hooksModelRef ?? resolveConfiguredModelRef({
|
|
||||||
cfg: params.cfg,
|
|
||||||
defaultProvider: DEFAULT_PROVIDER,
|
|
||||||
defaultModel: DEFAULT_MODEL,
|
|
||||||
});
|
|
||||||
|
|
||||||
// Run with fallback - on auth/rate-limit/timeout, fall through to agents.defaults.model.fallbacks
|
|
||||||
const fallbackResult = await runWithModelFallback({
|
|
||||||
cfg: params.cfg,
|
|
||||||
provider,
|
|
||||||
model,
|
|
||||||
hooksOverride: hooksModelRef?.isHooksOverride,
|
|
||||||
run: (providerOverride, modelOverride) => runEmbeddedPiAgent({
|
|
||||||
// ... existing params ...
|
|
||||||
}),
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
### Fallback Detection
|
|
||||||
|
|
||||||
Extend `runWithModelFallback()` to detect auth/rate-limit:
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
function isAuthRateLimitError(err: unknown): boolean {
|
|
||||||
if (err instanceof ApiError) {
|
|
||||||
return [401, 403, 429].includes(err.status);
|
|
||||||
}
|
|
||||||
// Check for common patterns in error messages
|
|
||||||
const msg = String(err).toLowerCase();
|
|
||||||
return msg.includes("unauthorized")
|
|
||||||
|| msg.includes("rate limit")
|
|
||||||
|| msg.includes("quota exceeded");
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Validation
|
|
||||||
|
|
||||||
### Config Load Time
|
|
||||||
|
|
||||||
In config validation (for `clawdbot doctor`):
|
|
||||||
|
|
||||||
```typescript
|
|
||||||
if (cfg.hooks?.gmail?.model) {
|
|
||||||
const resolved = resolveHooksGmailModel({ cfg, defaultProvider, defaultModel });
|
|
||||||
if (!resolved) {
|
|
||||||
issues.push({
|
|
||||||
path: "hooks.gmail.model",
|
|
||||||
message: `Model "${cfg.hooks.gmail.model}" could not be resolved`,
|
|
||||||
});
|
|
||||||
} else {
|
|
||||||
const catalog = await loadModelCatalog({ config: cfg });
|
|
||||||
const key = modelKey(resolved.provider, resolved.model);
|
|
||||||
const inCatalog = catalog.some(e => modelKey(e.provider, e.id) === key);
|
|
||||||
if (!inCatalog) {
|
|
||||||
issues.push({
|
|
||||||
path: "hooks.gmail.model",
|
|
||||||
message: `Model "${key}" not found in agents.defaults.models catalog (will fall back to primary)`,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Runtime
|
|
||||||
|
|
||||||
At hook invocation time, validate and fall back:
|
|
||||||
- If model not in catalog → log warning, use primary
|
|
||||||
- If model auth fails → log warning, enter cooldown, fall back
|
|
||||||
|
|
||||||
## Observability
|
|
||||||
|
|
||||||
### Log Messages
|
|
||||||
|
|
||||||
```
|
|
||||||
[hooks] Gmail hook: using model openrouter/meta-llama/llama-3.3-70b-instruct:free
|
|
||||||
[hooks] Gmail hook: model llama auth failed (429), falling back to claude-opus-4-5
|
|
||||||
```
|
|
||||||
|
|
||||||
### Hook Event Summary
|
|
||||||
|
|
||||||
Include fallback info in the hook summary sent to session:
|
|
||||||
|
|
||||||
```
|
|
||||||
Hook Gmail (fallback:llama→opus): <summary>
|
|
||||||
```
|
|
||||||
|
|
||||||
## Hot Reload
|
|
||||||
|
|
||||||
`hooks.gmail.model` and `hooks.gmail.thinking` are hot-reloadable:
|
|
||||||
- Changes apply to the next hook invocation
|
|
||||||
- No gateway restart required
|
|
||||||
- Hooks config is already in the hot-reload matrix
|
|
||||||
|
|
||||||
## Test Plan
|
|
||||||
|
|
||||||
### Unit Tests
|
|
||||||
|
|
||||||
1. **Model resolution** (`model-selection.test.ts`):
|
|
||||||
- `resolveHooksGmailModel()` with valid ref
|
|
||||||
- `resolveHooksGmailModel()` with alias
|
|
||||||
- `resolveHooksGmailModel()` with invalid input → null
|
|
||||||
|
|
||||||
2. **Config validation** (`config.test.ts`):
|
|
||||||
- Warning on uncatalogued model
|
|
||||||
- No warning on valid model
|
|
||||||
- Graceful handling of missing hooks.gmail section
|
|
||||||
|
|
||||||
3. **Fallback triggers** (`model-fallback.test.ts`):
|
|
||||||
- 401/403/429 → triggers fallback
|
|
||||||
- timeouts → triggers fallback
|
|
||||||
- 500/content error → no fallback
|
|
||||||
- Content error → no fallback
|
|
||||||
|
|
||||||
### Integration Tests
|
|
||||||
|
|
||||||
1. **Hook processing** (`server.hooks.test.ts`):
|
|
||||||
- Gmail hook uses `hooks.gmail.model` when set
|
|
||||||
- Fallback to primary on auth failure
|
|
||||||
- Thinking level override applied
|
|
||||||
|
|
||||||
2. **Hot reload** (`config-reload.test.ts`):
|
|
||||||
- Change `hooks.gmail.model` → next hook uses new model
|
|
||||||
|
|
||||||
## Documentation
|
|
||||||
|
|
||||||
Update `docs/gateway/configuration.md`:
|
|
||||||
|
|
||||||
```json5
|
|
||||||
{
|
|
||||||
hooks: {
|
|
||||||
gmail: {
|
|
||||||
account: "user@gmail.com",
|
|
||||||
topic: "projects/my-project/topics/gmail-watch",
|
|
||||||
// ... existing config ...
|
|
||||||
|
|
||||||
// Optional: Use a cheaper model for Gmail processing
|
|
||||||
// Falls back to agents.defaults.model.primary on auth/rate-limit errors
|
|
||||||
model: "openrouter/meta-llama/llama-3.3-70b-instruct:free",
|
|
||||||
|
|
||||||
// Optional: Override thinking level for Gmail processing
|
|
||||||
thinking: "off"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Scope Limitation
|
|
||||||
|
|
||||||
This PR is Gmail-specific. Future hooks (`hooks.github.model`, etc.) would follow the same pattern but are out of scope.
|
|
||||||
|
|
||||||
## Changelog Entry
|
|
||||||
|
|
||||||
```
|
|
||||||
- feat: add hooks.gmail.model for cheaper Gmail PubSub processing (#XXX)
|
|
||||||
- Falls back to agents.defaults.model.primary on auth/rate-limit/timeouts (401/403/429)
|
|
||||||
- Supports aliases from agents.defaults.models
|
|
||||||
- Add hooks.gmail.thinking override
|
|
||||||
```
|
|
||||||
@@ -1,110 +0,0 @@
|
|||||||
---
|
|
||||||
summary: "Channel plugin refactor implementation notes (registry, status, gateway/runtime)"
|
|
||||||
read_when:
|
|
||||||
- Adding or refactoring channel plugin wiring
|
|
||||||
- Moving channel-specific behavior into plugin hooks
|
|
||||||
---
|
|
||||||
|
|
||||||
# Channel Plugins — Implementation Notes
|
|
||||||
|
|
||||||
Goal: make chat channels (iMessage, Discord, etc.) pluggable with minimal wiring and shared UX/state paths.
|
|
||||||
|
|
||||||
## Architecture Overview
|
|
||||||
- Registry: `src/channels/plugins/index.ts` owns the plugin list.
|
|
||||||
- Channel dock: `src/channels/dock.ts` owns lightweight channel metadata used by shared flows (reply, command auth, block streaming) without importing full plugins.
|
|
||||||
- IDs/aliases: `src/channels/registry.ts` owns stable channel ids + input aliases.
|
|
||||||
- Shape: `src/channels/plugins/types.ts` defines the plugin contract.
|
|
||||||
- Gateway: `src/gateway/server-channels.ts` drives start/stop + runtime snapshots via plugins.
|
|
||||||
- Outbound: `src/infra/outbound/deliver.ts` routes through plugin outbound when present.
|
|
||||||
- Outbound delivery loads **outbound adapters** on-demand via `src/channels/plugins/outbound/load.ts` (avoid importing heavy channel plugins on hot paths).
|
|
||||||
- Reload: `src/gateway/config-reload.ts` uses plugin `reload.configPrefixes` lazily (avoid init cycles).
|
|
||||||
- CLI: `src/commands/channels/*` uses plugin list for add/remove/status/list.
|
|
||||||
- Protocol: `src/gateway/protocol/schema.ts` keeps `channels.status` schema-light (maps keyed by channel id).
|
|
||||||
|
|
||||||
## Plugin Contract (high-level)
|
|
||||||
Each `ChannelPlugin` bundles:
|
|
||||||
- `meta`: id/labels/docs/sort order.
|
|
||||||
- `capabilities`: chatTypes + optional features (polls, media, nativeCommands, etc.).
|
|
||||||
- `config`: list/resolve/default/isConfigured/describeAccount + isEnabled + (un)configured reasons + `resolveAllowFrom` + `formatAllowFrom`.
|
|
||||||
- `outbound`: deliveryMode + chunker + resolveTarget (mode-aware) + sendText/sendMedia/sendPoll + pollMaxOptions.
|
|
||||||
- `status`: defaultRuntime + probe/audit/buildAccountSnapshot + buildChannelSummary + logSelfId + collectStatusIssues.
|
|
||||||
- `gateway`: startAccount/stopAccount with runtime context (`getStatus`/`setStatus`), plus optional `loginWithQrStart/loginWithQrWait` for gateway-owned QR login flows.
|
|
||||||
- `security`: dmPolicy + allowFrom hints used by `doctor security`.
|
|
||||||
- `heartbeat`: optional readiness checks + heartbeat recipient resolution when channels own targeting.
|
|
||||||
- `auth`: optional login hook used by `clawdbot channels login`.
|
|
||||||
- `reload`: `configPrefixes` that map to hot restarts.
|
|
||||||
- `onboarding`: optional CLI onboarding adapter (wizard UI hooks per channel).
|
|
||||||
- `agentTools`: optional channel-owned agent tools (ex: QR login).
|
|
||||||
|
|
||||||
## Key Integration Notes
|
|
||||||
- `listChannelPlugins()` is the runtime source of truth for channel UX and wiring.
|
|
||||||
- Avoid importing `src/channels/plugins/index.ts` from shared modules (reply flow, command auth, sandbox explain). It’s intentionally “heavy” (channels may pull web login / monitor code). Use `getChannelDock()` + `normalizeChannelId()` (from `src/channels/registry.ts`) for cheap metadata, and only `getChannelPlugin()` at execution boundaries (ex: `src/auto-reply/reply/route-reply.ts`).
|
|
||||||
- WhatsApp plugin keeps Baileys-heavy login bits behind lazy imports; cheap auth file checks live in `src/web/auth-store.ts` (so outbound routing doesn’t pay Baileys import cost).
|
|
||||||
- `routeReply` delegates sending to plugin `outbound` adapters via a lazy import of `src/infra/outbound/deliver.ts` (so adding a channel is “just implement outbound adapter”, no router switches).
|
|
||||||
- Avoid static imports of channel monitors inside plugin modules. Monitors typically import the reply pipeline, which can create ESM cycles (and break Vite/Vitest SSR with TDZ errors). Prefer lazy imports inside `gateway.startAccount`.
|
|
||||||
- Debug cycle leaks quickly with: `npx -y madge --circular src/channels/plugins/index.ts`.
|
|
||||||
- Protocol v3: `channels.status` is map-based and schema-light so new channels can ship without protocol updates.
|
|
||||||
- `DEFAULT_CHAT_CHANNEL` lives in `src/channels/registry.ts` and is used anywhere we need a fallback delivery surface.
|
|
||||||
- Channel reload rules are computed lazily to avoid static init cycles in tests.
|
|
||||||
- Signal/iMessage media size limits are resolved inside their plugins.
|
|
||||||
- `normalizeChannelId()` handles aliases (ex: `imsg`, `teams`) so CLI and API inputs stay stable.
|
|
||||||
- Gateway runtime snapshot (`getRuntimeSnapshot`) is map-based: `{ channels, channelAccounts }` (no `${id}Accounts` keys).
|
|
||||||
- `channels.status` response keys (v3):
|
|
||||||
- `channelOrder: string[]`
|
|
||||||
- `channelLabels: Record<string, string>`
|
|
||||||
- `channels: Record<string, unknown>` (channel summary objects, plugin-defined)
|
|
||||||
- `channelAccounts: Record<string, ChannelAccountSnapshot[]>`
|
|
||||||
- `channelDefaultAccountId: Record<string, string>`
|
|
||||||
- `channels.status` summary objects come from `status.buildChannelSummary` (no per-channel branching in the handler).
|
|
||||||
- `channels.status` warnings flow through `status.collectStatusIssues` per plugin.
|
|
||||||
- CLI list uses `meta.showConfigured` to decide whether to show configured state.
|
|
||||||
- CLI channel options and prompt channel lists are generated from `listChannelPlugins()` (avoid hardcoded arrays).
|
|
||||||
- Channel selection (`resolveMessageChannelSelection`) inspects `config.isEnabled` + `config.isConfigured` per plugin instead of hardcoded checks.
|
|
||||||
- Pairing flows (CLI + store) use `plugin.pairing` (`idLabel`, `normalizeAllowEntry`, `notifyApproval`) via `src/channels/plugins/pairing.ts`.
|
|
||||||
- CLI channel remove/disable delegates to `config.setAccountEnabled` + `config.deleteAccount` per plugin.
|
|
||||||
- CLI channel add delegates to `plugin.setup` for account validation, naming, and config writes (no hardcoded checks).
|
|
||||||
- Agent channel status entries are built from plugin config/status (`status.resolveAccountState` for custom state labels).
|
|
||||||
- Agent binding defaults use `meta.forceAccountBinding` to avoid hardcoded checks.
|
|
||||||
- Onboarding quickstart allowlist uses `meta.quickstartAllowFrom` to avoid hardcoded channel lists.
|
|
||||||
- `resolveChannelDefaultAccountId()` is the shared helper for picking default accounts from `accountIds` + plugin config.
|
|
||||||
- `routeReply` uses plugin outbound senders; `ChannelOutboundContext` supports `replyToId` + `threadId` and outbound delivery supports `abortSignal` for cooperative cancellation.
|
|
||||||
- Outbound target resolution (`resolveOutboundTarget`) delegates to `plugin.outbound.resolveTarget` (mode-aware, uses config allowlists when present).
|
|
||||||
- Outbound delivery results accept `meta` for channel-specific fields to avoid core type churn in new plugins.
|
|
||||||
- Agent gateway routing sets `deliveryTargetMode` and uses `resolveOutboundTarget` for implicit fallback targets when `to` is missing.
|
|
||||||
- Elevated tool allowlists (`tools.elevated.allowFrom`) are a record keyed by channel id (no schema update needed when adding channels).
|
|
||||||
- Block streaming defaults live on the plugin (`capabilities.blockStreaming`, `streaming.blockStreamingCoalesceDefaults`) instead of hardcoded channel checks.
|
|
||||||
- Channel logout routes through `channels.logout` using `gateway.logoutAccount` on each plugin (clients should call the generic method).
|
|
||||||
- Gateway message-channel normalization uses `src/channels/registry.ts` for cheap validation/normalization without plugin init cycles.
|
|
||||||
- Group mention gating now flows through `plugin.groups.resolveRequireMention` (Discord/Slack/Telegram/WhatsApp/iMessage) instead of branching in reply handlers.
|
|
||||||
- Command authorization uses `config.resolveAllowFrom` + `config.formatAllowFrom`, with `commands.enforceOwnerForCommands` and `commands.skipWhenConfigEmpty` driving channel-specific behavior.
|
|
||||||
- Security warnings (`doctor security`) use `plugin.security.resolveDmPolicy` + `plugin.security.collectWarnings`; supply `policyPath` + `allowFromPath` for accurate config hints.
|
|
||||||
- Reply threading uses `plugin.threading.resolveReplyToMode` and `plugin.threading.allowTagsWhenOff` rather than channel switches in reply helpers.
|
|
||||||
- Tool auto-threading context flows through `plugin.threading.buildToolContext` (e.g., Slack threadTs injection).
|
|
||||||
- Messaging tool dedupe now relies on `plugin.messaging.normalizeTarget` for channel-specific target normalization.
|
|
||||||
- Message tool + CLI action dispatch now use `plugin.actions.listActions` + `plugin.actions.handleAction`; use `plugin.actions.supportsAction` for dispatch-only gating when you still want fallback send/poll.
|
|
||||||
- Session announce targets can opt into `meta.preferSessionLookupForAnnounceTarget` when session keys are insufficient (e.g., WhatsApp).
|
|
||||||
- Onboarding channel setup is delegated to adapter modules under `src/channels/plugins/onboarding/*`, keeping `setupChannels` channel-agnostic.
|
|
||||||
- Onboarding registry now reads `plugin.onboarding` from each channel (no standalone onboarding map).
|
|
||||||
- Channel login flows (`clawdbot channels login`) route through `plugin.auth.login` when available.
|
|
||||||
- `clawdbot status` reports `linkChannel` (derived from `status.buildChannelSummary().linked`) instead of a hardcoded `web` field.
|
|
||||||
- Gateway `web.login.*` methods use `plugin.gatewayMethods` ownership to pick the channel (no hardcoded `normalizeChannelId("web")` in the handler).
|
|
||||||
|
|
||||||
## CLI Commands (inline references)
|
|
||||||
- Add/remove channels: `clawdbot channels add <channel>` / `clawdbot channels remove <channel>`.
|
|
||||||
- Inspect channel state: `clawdbot channels list`, `clawdbot channels status`.
|
|
||||||
- Link/unlink channels: `clawdbot channels login --channel <channel>` / `clawdbot channels logout --channel <channel>`.
|
|
||||||
- Pairing approvals: `clawdbot pairing list <channel>`, `clawdbot pairing approve <channel> <code>`.
|
|
||||||
|
|
||||||
## Adding a Channel (checklist)
|
|
||||||
1) Create `src/channels/plugins/<id>.ts` exporting `ChannelPlugin`.
|
|
||||||
2) Register in `src/channels/plugins/index.ts` and update `src/channels/registry.ts` (ids/aliases/meta) if needed.
|
|
||||||
3) Add a dock entry in `src/channels/dock.ts` for any shared behavior (capabilities, allowFrom format/resolve, mention stripping, threading, streaming chunk defaults).
|
|
||||||
4) Add `reload.configPrefixes` for hot reload when config changes.
|
|
||||||
5) Delegate to existing channel modules (send/probe/monitor) or create them.
|
|
||||||
6) If you changed the gateway protocol: run `pnpm protocol:check` (updates `dist/protocol.schema.json` + `apps/macos/Sources/ClawdbotProtocol/GatewayModels.swift`).
|
|
||||||
7) Update docs/tests for any behavior changes.
|
|
||||||
|
|
||||||
## Cleanup Expectations
|
|
||||||
- Keep plugin files small; move heavy logic into channel modules.
|
|
||||||
- Prefer shared helpers over V2 copies.
|
|
||||||
- Update docs when behavior/inputs change.
|
|
||||||
@@ -1,127 +0,0 @@
|
|||||||
---
|
|
||||||
summary: "Vector memory search design plan (per-agent, watch/lazy sync, storage)"
|
|
||||||
read_when:
|
|
||||||
- Designing or implementing vector memory search
|
|
||||||
- Adding embedding providers or sync behavior
|
|
||||||
---
|
|
||||||
|
|
||||||
# Vector Memory Search — Design Plan
|
|
||||||
|
|
||||||
Goal: semantic search over **agent memory files** only, with minimal deps and
|
|
||||||
good UX defaults. Default enabled. Per-agent overrides.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
- Sources: `MEMORY.md` + `memory/YYYY-MM-DD.md` inside the agent workspace.
|
|
||||||
- No indexing outside the workspace. No hidden paths.
|
|
||||||
- No QMD-style query expansion or rerank in v1.
|
|
||||||
|
|
||||||
## Config Shape
|
|
||||||
Location: `agents.defaults.memorySearch` + `agents.list[].memorySearch`.
|
|
||||||
|
|
||||||
```json5
|
|
||||||
agents: {
|
|
||||||
defaults: {
|
|
||||||
memorySearch: {
|
|
||||||
enabled: true,
|
|
||||||
provider: "openai", // "openai" | "local"
|
|
||||||
fallback: "openai", // "openai" | "none"
|
|
||||||
model: "text-embedding-3-small",
|
|
||||||
store: {
|
|
||||||
driver: "sqlite",
|
|
||||||
path: "~/.clawdbot/memory/{agentId}.sqlite"
|
|
||||||
},
|
|
||||||
chunking: {
|
|
||||||
tokens: 400,
|
|
||||||
overlap: 80
|
|
||||||
},
|
|
||||||
sync: {
|
|
||||||
onSessionStart: true,
|
|
||||||
onSearch: true, // LazySync
|
|
||||||
watch: true, // default on
|
|
||||||
watchDebounceMs: 1500,
|
|
||||||
intervalMinutes: 0
|
|
||||||
},
|
|
||||||
query: {
|
|
||||||
maxResults: 6,
|
|
||||||
minScore: 0.35
|
|
||||||
}
|
|
||||||
}
|
|
||||||
},
|
|
||||||
list: [
|
|
||||||
{ id: "peter", memorySearch: { provider: "local", sync: { watch: false } } }
|
|
||||||
]
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Storage
|
|
||||||
Per-agent DB (default): `~/.clawdbot/memory/{agentId}.sqlite`.
|
|
||||||
|
|
||||||
Tables (v1):
|
|
||||||
- `files(path PRIMARY KEY, hash, mtime, size)`
|
|
||||||
- `chunks(id PRIMARY KEY, path, start_line, end_line, hash, text, embedding, updated_at)`
|
|
||||||
|
|
||||||
Notes:
|
|
||||||
- `hash` = content hash of chunk text.
|
|
||||||
- `embedding` stored as float[] (sqlite vec extension optional); if not using vec,
|
|
||||||
store as JSON and do linear scan in memory for small corpora.
|
|
||||||
|
|
||||||
## Embedding Providers
|
|
||||||
Interface (core):
|
|
||||||
- `embedQuery(text): number[]`
|
|
||||||
- `embedBatch(texts[]): number[][]`
|
|
||||||
|
|
||||||
Providers:
|
|
||||||
- `openai` (default): OpenAI embeddings via existing keys.
|
|
||||||
- `local` (optional): node-llama-cpp (GGUF).
|
|
||||||
- Fallback: when `provider: "local"` fails, fallback to OpenAI unless `fallback: "none"`.
|
|
||||||
|
|
||||||
## Index Pipeline
|
|
||||||
1) Resolve memory file list (workspace only).
|
|
||||||
2) Read file, compute file hash/mtime.
|
|
||||||
3) Chunk by headings + token cap (overlap).
|
|
||||||
4) Embed only changed chunks (hash compare).
|
|
||||||
5) Upsert `chunks` rows, prune deleted files.
|
|
||||||
|
|
||||||
Chunking:
|
|
||||||
- Prefer heading-aware splits.
|
|
||||||
- Max tokens + overlap; keep line ranges for snippets.
|
|
||||||
|
|
||||||
## Sync Strategy
|
|
||||||
Default: **watch + lazy + session-start**
|
|
||||||
- `watch`: chokidar on `MEMORY.md` + `memory/**/*.md` (debounced).
|
|
||||||
- `onSearch`: if dirty, sync before search (LazySync).
|
|
||||||
- `onSessionStart`: warm index once per session.
|
|
||||||
- `intervalMinutes`: optional for long-lived sessions.
|
|
||||||
|
|
||||||
If workspace access is read-only or missing: disable writes; return “not indexed”.
|
|
||||||
|
|
||||||
## Query Flow
|
|
||||||
1) Embed query.
|
|
||||||
2) Cosine similarity over all chunk embeddings.
|
|
||||||
3) Return top K with `{path, startLine, endLine, snippet, score}`.
|
|
||||||
4) Model may call `memory_get` when full context needed.
|
|
||||||
|
|
||||||
Optional v2: add FTS5 + RRF merge (FTS + vector) for quality.
|
|
||||||
|
|
||||||
## Tool + CLI
|
|
||||||
Tools:
|
|
||||||
- `memory_search { query, maxResults?, minScore? }`
|
|
||||||
- `memory_get { path, from?, lines? }`
|
|
||||||
|
|
||||||
CLI (optional):
|
|
||||||
- `clawdbot memory index|search|status`
|
|
||||||
|
|
||||||
## Security + Permissions
|
|
||||||
- Indexer reads only memory files in workspace.
|
|
||||||
- No scanning outside workspace; no “sneak” reads.
|
|
||||||
- Respect sandbox `workspaceAccess` (ro = read-only; none = disabled).
|
|
||||||
|
|
||||||
## Tests
|
|
||||||
- Chunking boundaries + line ranges.
|
|
||||||
- Hash-based incremental updates.
|
|
||||||
- Search ranking (cosine).
|
|
||||||
- Watcher debounce (fake fs).
|
|
||||||
|
|
||||||
## Rollout
|
|
||||||
- Default enabled; if no memory files, index is empty (silent).
|
|
||||||
- No migration needed.
|
|
||||||
@@ -1,72 +0,0 @@
|
|||||||
---
|
|
||||||
summary: "Terminal UI (TUI) for Clawdbot via the Gateway"
|
|
||||||
read_when:
|
|
||||||
- You want a terminal UI that connects to the Gateway from any machine
|
|
||||||
- You are debugging the TUI client or Gateway chat stream
|
|
||||||
---
|
|
||||||
# TUI (Gateway chat client)
|
|
||||||
|
|
||||||
|
|
||||||
## What it is
|
|
||||||
- A terminal UI that connects to the Gateway WebSocket and speaks the same chat APIs as WebChat.
|
|
||||||
- Uses Gateway agent events for tool cards while streaming responses.
|
|
||||||
- Works locally (loopback) or remotely (Tailscale/SSH tunnel) without running a separate agent process.
|
|
||||||
|
|
||||||
## Run
|
|
||||||
```bash
|
|
||||||
clawdbot tui
|
|
||||||
```
|
|
||||||
|
|
||||||
### Remote
|
|
||||||
```bash
|
|
||||||
clawdbot tui --url ws://127.0.0.1:18789 --token <gateway-token>
|
|
||||||
```
|
|
||||||
Use SSH tunneling or Tailscale to reach the Gateway WS.
|
|
||||||
|
|
||||||
## Options
|
|
||||||
- `--url <url>`: Gateway WebSocket URL (defaults to config `gateway.remote.url` or `ws://127.0.0.1:18789`).
|
|
||||||
- `--token <token>`: Gateway token (if required).
|
|
||||||
- `--password <password>`: Gateway password (if required).
|
|
||||||
- `--session <key>`: Session key (default: `main`, or `global` when scope is global).
|
|
||||||
- `--deliver`: Deliver assistant replies to the channel (default off).
|
|
||||||
- `--thinking <level>`: Override thinking level for sends.
|
|
||||||
- `--timeout-ms <ms>`: Agent timeout in ms (defaults to `agents.defaults.timeoutSeconds`).
|
|
||||||
- `--history-limit <n>`: History entries to load (default 200).
|
|
||||||
|
|
||||||
## Controls
|
|
||||||
- Enter: send message
|
|
||||||
- Esc: abort active run
|
|
||||||
- Ctrl+C: clear input (press twice to exit)
|
|
||||||
- Ctrl+D: exit
|
|
||||||
- Ctrl+L: model picker
|
|
||||||
- Ctrl+G: agent picker
|
|
||||||
- Ctrl+P: session picker
|
|
||||||
- Ctrl+O: toggle tool output expansion
|
|
||||||
- Ctrl+T: toggle thinking visibility
|
|
||||||
|
|
||||||
## Slash commands
|
|
||||||
- `/help`
|
|
||||||
- `/status`
|
|
||||||
- `/agent <id>` (or `/agents`)
|
|
||||||
- `/session <key>` (or `/sessions`)
|
|
||||||
- `/model <provider/model>` (or `/model list`, `/models`)
|
|
||||||
- `/think <off|minimal|low|medium|high|xhigh>` (GPT-5.2 + Codex models only)
|
|
||||||
- `/verbose <on|off>`
|
|
||||||
- `/reasoning <on|off|stream>` (stream = Telegram draft only)
|
|
||||||
- `/cost <on|off>`
|
|
||||||
- `/elevated <on|off>`
|
|
||||||
- `/elev <on|off>`
|
|
||||||
- `/activation <mention|always>`
|
|
||||||
- `/deliver <on|off>`
|
|
||||||
- `/new` or `/reset`
|
|
||||||
- `/compact [instructions]`
|
|
||||||
- `/abort`
|
|
||||||
- `/settings`
|
|
||||||
- `/exit`
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
- The TUI shows Gateway chat deltas (`event: chat`) and agent tool events.
|
|
||||||
- It registers as a Gateway client with `mode: "tui"` for presence and debugging.
|
|
||||||
- The TUI uses the **Gateway’s** model/auth config and environment. If a model
|
|
||||||
token works in your shell but not in TUI, put it in `~/.clawdbot/.env` or
|
|
||||||
enable `env.shellEnv.enabled`, then restart the Gateway.
|
|
||||||
Reference in New Issue
Block a user