* refactor: introduce provider plugin registry * refactor: move provider CLI to plugins * docs: add provider plugin implementation notes * refactor: shift provider runtime logic into plugins * refactor: add plugin defaults and summaries * docs: update provider plugin notes * feat(commands): add /commands slash list * Auto-reply: tidy help message * Auto-reply: fix status command lint * Tests: align google shared expectations * Auto-reply: tidy help message * Auto-reply: fix status command lint * refactor: move provider routing into plugins * test: align agent routing expectations * docs: update provider plugin notes * refactor: route replies via provider plugins * docs: note route-reply plugin hooks * refactor: extend provider plugin contract * refactor: derive provider status from plugins * refactor: unify gateway provider control * refactor: use plugin metadata in auto-reply * fix: parenthesize cron target selection * refactor: derive gateway methods from plugins * refactor: generalize provider logout * refactor: route provider logout through plugins * refactor: move WhatsApp web login methods into plugin * refactor: generalize provider log prefixes * refactor: centralize default chat provider * refactor: derive provider lists from registry * refactor: move provider reload noops into plugins * refactor: resolve web login provider via alias * refactor: derive CLI provider options from plugins * refactor: derive prompt provider list from plugins * style: apply biome lint fixes * fix: resolve provider routing edge cases * docs: update provider plugin refactor notes * fix(gateway): harden agent provider routing * refactor: move provider routing into plugins * refactor: move provider CLI to plugins * refactor: derive provider lists from registry * fix: restore slash command parsing * refactor: align provider ids for schema * refactor: unify outbound target resolution * fix: keep outbound labels stable * feat: add msteams to cron surfaces * fix: clean up lint build issues * refactor: localize chat provider alias normalization * refactor: drive gateway provider lists from plugins * docs: update provider plugin notes * style: format message-provider * fix: avoid provider registry init cycles * style: sort message-provider imports * fix: relax provider alias map typing * refactor: move provider routing into plugins * refactor: add plugin pairing/config adapters * refactor: route pairing and provider removal via plugins * refactor: align auto-reply provider typing * test: stabilize telegram media mocks * docs: update provider plugin refactor notes * refactor: pluginize outbound targets * refactor: pluginize provider selection * refactor: generalize text chunk limits * docs: update provider plugin notes * refactor: generalize group session/config * fix: normalize provider id for room detection * fix: avoid provider init in system prompt * style: formatting cleanup * refactor: normalize agent delivery targets * test: update outbound delivery labels * chore: fix lint regressions * refactor: extend provider plugin adapters * refactor: move elevated/block streaming defaults to plugins * refactor: defer outbound send deps to plugins * docs: note plugin-driven streaming/elevated defaults * refactor: centralize webchat provider constant * refactor: add provider setup adapters * refactor: delegate provider add config to plugins * docs: document plugin-driven provider add * refactor: add plugin state/binding metadata * refactor: build agent provider status from plugins * docs: note plugin-driven agent bindings * refactor: centralize internal provider constant usage * fix: normalize WhatsApp targets for groups and E.164 (#631) (thanks @imfing) * refactor: centralize default chat provider * refactor: centralize WhatsApp target normalization * refactor: move provider routing into plugins * refactor: normalize agent delivery targets * chore: fix lint regressions * fix: normalize WhatsApp targets for groups and E.164 (#631) (thanks @imfing) * feat: expand provider plugin adapters * refactor: route auto-reply via provider plugins * fix: align WhatsApp target normalization * fix: normalize WhatsApp targets for groups and E.164 (#631) (thanks @imfing) * refactor: centralize WhatsApp target normalization * feat: add /config chat config updates * docs: add /config get alias * feat(commands): add /commands slash list * refactor: centralize default chat provider * style: apply biome lint fixes * chore: fix lint regressions * fix: clean up whatsapp allowlist typing * style: format config command helpers * refactor: pluginize tool threading context * refactor: normalize session announce targets * docs: note new plugin threading and announce hooks * refactor: pluginize message actions * docs: update provider plugin actions notes * fix: align provider action adapters * refactor: centralize webchat checks * style: format message provider helpers * refactor: move provider onboarding into adapters * docs: note onboarding provider adapters * feat: add msteams onboarding adapter * style: organize onboarding imports * fix: normalize msteams allowFrom types * feat: add plugin text chunk limits * refactor: use plugin chunk limit fallbacks * feat: add provider mention stripping hooks * style: organize provider plugin type imports * refactor: generalize health snapshots * refactor: update macOS health snapshot handling * docs: refresh health snapshot notes * style: format health snapshot updates * refactor: drive security warnings via plugins * docs: note provider security adapter * style: format provider security adapters * refactor: centralize provider account defaults * refactor: type gateway client identity constants * chore: regen gateway protocol swift * fix: degrade health on failed provider probe * refactor: centralize pairing approve hint * docs: add plugin CLI command references * refactor: route auth and tool sends through plugins * docs: expand provider plugin hooks * refactor: document provider docking touchpoints * refactor: normalize internal provider defaults * refactor: streamline outbound delivery wiring * refactor: make provider onboarding plugin-owned * refactor: support provider-owned agent tools * refactor: move telegram draft chunking into telegram module * refactor: infer provider tool sends via extractToolSend * fix: repair plugin onboarding imports * refactor: de-dup outbound target normalization * style: tidy plugin and agent imports * refactor: data-drive provider selection line * fix: satisfy lint after provider plugin rebase * test: deflake gateway-cli coverage * style: format gateway-cli coverage test * refactor(provider-plugins): simplify provider ids * test(pairing-cli): avoid provider-specific ternary * style(macos): swiftformat HealthStore * refactor(sandbox): derive provider tool denylist * fix(sandbox): avoid plugin init in defaults * refactor(provider-plugins): centralize provider aliases * style(test): satisfy biome * refactor(protocol): v3 providers.status maps * refactor(ui): adapt to protocol v3 * refactor(macos): adapt to protocol v3 * test: update providers.status v3 fixtures * refactor(gateway): map provider runtime snapshot * test(gateway): update reload runtime snapshot * refactor(whatsapp): normalize heartbeat provider id * docs(refactor): update provider plugin notes * style: satisfy biome after rebase * fix: describe sandboxed elevated in prompt * feat(gateway): add agent image attachments + live probe * refactor: derive CLI provider options from plugins * fix(gateway): harden agent provider routing * fix(gateway): harden agent provider routing * refactor: align provider ids for schema * fix(protocol): keep agent provider string * fix(gateway): harden agent provider routing * fix(protocol): keep agent provider string * refactor: normalize agent delivery targets * refactor: support provider-owned agent tools * refactor(config): provider-keyed elevated allowFrom * style: satisfy biome * fix(gateway): appease provider narrowing * style: satisfy biome * refactor(reply): move group intro hints into plugin * fix(reply): avoid plugin registry init cycle * refactor(providers): add lightweight provider dock * refactor(gateway): use typed client id in connect * refactor(providers): document docks and avoid init cycles * refactor(providers): make media limit helper generic * fix(providers): break plugin registry import cycles * style: satisfy biome * refactor(status-all): build providers table from plugins * refactor(gateway): delegate web login to provider plugin * refactor(provider): drop web alias * refactor(provider): lazy-load monitors * style: satisfy lint/format * style: format status-all providers table * style: swiftformat gateway discovery model * test: make reload plan plugin-driven * fix: avoid token stringification in status-all * refactor: make provider IDs explicit in status * feat: warn on signal/imessage provider runtime errors * test: cover gateway provider runtime warnings in status * fix: add runtime kind to provider status issues * test: cover health degradation on probe failure * fix: keep routeReply lightweight * style: organize routeReply imports * refactor(web): extract auth-store helpers * refactor(whatsapp): lazy login imports * refactor(outbound): route replies via plugin outbound * docs: update provider plugin notes * style: format provider status issues * fix: make sandbox scope warning wrap-safe * refactor: load outbound adapters from provider plugins * docs: update provider plugin outbound notes * style(macos): fix swiftformat lint * docs: changelog for provider plugins * fix(macos): satisfy swiftformat * fix(macos): open settings via menu action * style: format after rebase * fix(macos): open Settings via menu action --------- Co-authored-by: LK <luke@kyohere.com> Co-authored-by: Luke K (pr-0f3t) <2609441+lc0rp@users.noreply.github.com> Co-authored-by: Xin <xin@imfing.com>
114 lines
11 KiB
Markdown
114 lines
11 KiB
Markdown
---
|
||
summary: "Provider plugin refactor implementation notes (registry, status, gateway/runtime)"
|
||
read_when:
|
||
- Adding or refactoring provider plugin wiring
|
||
- Moving provider-specific behavior into plugin hooks
|
||
---
|
||
|
||
# Provider Plugin Refactor — Implementation Notes
|
||
|
||
Goal: make providers (iMessage, Discord, etc.) pluggable with minimal wiring and shared UX/state paths.
|
||
|
||
## Architecture Overview
|
||
- Registry: `src/providers/plugins/index.ts` owns the plugin list.
|
||
- Provider dock: `src/providers/dock.ts` owns lightweight provider metadata used by shared flows (reply, command auth, block streaming) without importing full plugins.
|
||
- IDs/aliases: `src/providers/registry.ts` owns stable provider ids + input aliases.
|
||
- Shape: `src/providers/plugins/types.ts` defines the plugin contract.
|
||
- Gateway: `src/gateway/server-providers.ts` drives start/stop + runtime snapshots via plugins.
|
||
- Outbound: `src/infra/outbound/deliver.ts` routes through plugin outbound when present.
|
||
- Outbound delivery loads **outbound adapters** on-demand via `src/providers/plugins/outbound/load.ts` (avoid importing heavy provider plugins on hot paths).
|
||
- Reload: `src/gateway/config-reload.ts` uses plugin `reload.configPrefixes` lazily (avoid init cycles).
|
||
- CLI: `src/commands/providers/*` uses plugin list for add/remove/status/list.
|
||
- Protocol: `src/gateway/protocol/schema.ts` (v3) makes provider-shaped responses container-generic (maps keyed by provider id).
|
||
|
||
## Plugin Contract (high-level)
|
||
Each `ProviderPlugin` bundles:
|
||
- `meta`: id/labels/docs/sort order.
|
||
- `capabilities`: chatTypes + optional features (polls, media, nativeCommands, etc.).
|
||
- `config`: list/resolve/default/isConfigured/describeAccount + isEnabled + (un)configured reasons + `resolveAllowFrom` + `formatAllowFrom`.
|
||
- `outbound`: deliveryMode + chunker + resolveTarget (mode-aware) + sendText/sendMedia/sendPoll + pollMaxOptions.
|
||
- `status`: defaultRuntime + probe/audit/buildAccountSnapshot + buildProviderSummary + logSelfId + collectStatusIssues.
|
||
- `gateway`: startAccount/stopAccount with runtime context (`getStatus`/`setStatus`), plus optional `loginWithQrStart/loginWithQrWait` for gateway-owned QR login flows.
|
||
- `security`: dmPolicy + allowFrom hints used by `doctor security`.
|
||
- `heartbeat`: optional readiness checks + heartbeat recipient resolution when providers own targeting.
|
||
- `auth`: optional login hook used by `clawdbot providers login`.
|
||
- `reload`: `configPrefixes` that map to hot restarts.
|
||
- `onboarding`: optional CLI onboarding adapter (wizard UI hooks per provider).
|
||
- `agentTools`: optional provider-owned agent tools (ex: QR login).
|
||
|
||
## Key Integration Notes
|
||
- `listProviderPlugins()` is the runtime source of truth for provider UX and wiring.
|
||
- Avoid importing `src/providers/plugins/index.ts` from shared modules (reply flow, command auth, sandbox explain). It’s intentionally “heavy” (providers may pull web login / monitor code). Use `getProviderDock()` + `normalizeProviderId()` for cheap metadata, and only `getProviderPlugin()` at execution boundaries (ex: `src/auto-reply/reply/route-reply.ts`).
|
||
- WhatsApp plugin keeps Baileys-heavy login bits behind lazy imports; cheap auth file checks live in `src/web/auth-store.ts` (so outbound routing doesn’t pay Baileys import cost).
|
||
- `routeReply` delegates sending to plugin `outbound` adapters via a lazy import of `src/infra/outbound/deliver.ts` (so adding a provider is “just implement outbound adapter”, no router switches).
|
||
- Avoid static imports of provider monitors inside plugin modules. Monitors typically import the reply pipeline, which can create ESM cycles (and break Vite/Vitest SSR with TDZ errors). Prefer lazy imports inside `gateway.startAccount`.
|
||
- Debug cycle leaks quickly with: `npx -y madge --circular src/providers/plugins/index.ts`.
|
||
- Gateway protocol schema keeps provider selection as an open-ended string (no provider enum / static list) to avoid init cycles and so new plugins don’t require protocol changes.
|
||
- Protocol v3: no more per-provider fields in `providers.status`; consumers must read map entries by provider id.
|
||
- `DEFAULT_CHAT_PROVIDER` lives in `src/providers/registry.ts` and is used anywhere we need a fallback delivery surface.
|
||
- Provider reload rules are computed lazily to avoid static init cycles in tests.
|
||
- Signal/iMessage media size limits are now resolved inside their plugins.
|
||
- `normalizeProviderId()` handles aliases (ex: `imsg`, `teams`) so CLI and API inputs stay stable.
|
||
- `ProviderId` is `ChatProviderId` (no extra special-cased provider IDs in shared code).
|
||
- Gateway runtime defaults (`status.defaultRuntime`) replace the old per-provider runtime map.
|
||
- Gateway runtime snapshot (`getRuntimeSnapshot`) is map-based: `{ providers, providerAccounts }` (no `${id}Accounts` keys).
|
||
- `providers.status` response keys (v3):
|
||
- `providerOrder: string[]`
|
||
- `providerLabels: Record<string, string>`
|
||
- `providers: Record<string, unknown>` (provider summary objects, plugin-defined)
|
||
- `providerAccounts: Record<string, ProviderAccountSnapshot[]>`
|
||
- `providerDefaultAccountId: Record<string, string>`
|
||
- `providers.status` summary objects come from `status.buildProviderSummary` (no per-provider branching in the handler).
|
||
- `providers.status` warnings now flow through `status.collectStatusIssues` per plugin.
|
||
- CLI list uses `meta.showConfigured` to decide whether to show configured state.
|
||
- CLI provider options and prompt provider lists are generated from `listProviderPlugins()` (avoid hardcoded arrays).
|
||
- Provider selection (`resolveMessageProviderSelection`) now inspects `config.isEnabled` + `config.isConfigured` per plugin instead of hardcoded provider checks.
|
||
- Pairing flows (CLI + store) now use `plugin.pairing` (`idLabel`, `normalizeAllowEntry`, `notifyApproval`) via `src/providers/plugins/pairing.ts`.
|
||
- CLI provider remove/disable delegates to `config.setAccountEnabled` + `config.deleteAccount` per plugin.
|
||
- CLI provider add now delegates to `plugin.setup` for account validation, naming, and config writes (no hardcoded provider checks).
|
||
- Agent provider status entries are now built from plugin config/status (`status.resolveAccountState` for custom state labels).
|
||
- Agent binding defaults use `meta.forceAccountBinding` to avoid hardcoded provider checks.
|
||
- Onboarding quickstart allowlist uses `meta.quickstartAllowFrom` to avoid hardcoded provider lists.
|
||
- `resolveProviderDefaultAccountId()` is the shared helper for picking default accounts from `accountIds` + plugin config.
|
||
- `routeReply` uses plugin outbound senders; `ProviderOutboundContext` supports `replyToId` + `threadId` and outbound delivery supports `abortSignal` for cooperative cancellation.
|
||
- Outbound target resolution (`resolveOutboundTarget`) now delegates to `plugin.outbound.resolveTarget` (mode-aware, uses config allowlists when present).
|
||
- Outbound delivery results accept `meta` for provider-specific fields to avoid core type churn in new plugins.
|
||
- Agent gateway routing sets `deliveryTargetMode` and uses `resolveOutboundTarget` for implicit fallback targets when `to` is missing.
|
||
- Elevated tool allowlists (`tools.elevated.allowFrom`) are a record keyed by provider id (no schema update needed when adding providers).
|
||
- Block streaming defaults live on the plugin (`capabilities.blockStreaming`, `streaming.blockStreamingCoalesceDefaults`) instead of hardcoded provider checks.
|
||
- Provider logout now routes through `providers.logout` using `gateway.logoutAccount` on each plugin (clients should call the generic method).
|
||
- Gateway message-provider normalization uses `src/providers/registry.ts` for cheap validation/normalization without plugin init cycles.
|
||
- Group mention gating now flows through `plugin.groups.resolveRequireMention` (Discord/Slack/Telegram/WhatsApp/iMessage) instead of branching in reply handlers.
|
||
- Command authorization uses `config.resolveAllowFrom` + `config.formatAllowFrom`, with `commands.enforceOwnerForCommands` and `commands.skipWhenConfigEmpty` driving provider-specific behavior.
|
||
- Security warnings (`doctor security`) use `plugin.security.resolveDmPolicy` + `plugin.security.collectWarnings`; supply `policyPath` + `allowFromPath` for accurate config hints.
|
||
- Reply threading uses `plugin.threading.resolveReplyToMode` and `plugin.threading.allowTagsWhenOff` rather than provider switches in reply helpers.
|
||
- Tool auto-threading context flows through `plugin.threading.buildToolContext` (e.g., Slack threadTs injection).
|
||
- Messaging tool dedupe now relies on `plugin.messaging.normalizeTarget` for provider-specific target normalization.
|
||
- Message tool + CLI action dispatch now use `plugin.actions.listActions` + `plugin.actions.handleAction`; use `plugin.actions.supportsAction` for dispatch-only gating when you still want fallback send/poll.
|
||
- Session announce targets can opt into `meta.preferSessionLookupForAnnounceTarget` when session keys are insufficient (e.g., WhatsApp).
|
||
- Onboarding provider setup is delegated to adapter modules under `src/providers/plugins/onboarding/*`, keeping `setupProviders` provider-agnostic.
|
||
- Onboarding registry now reads `plugin.onboarding` from each provider (no standalone onboarding map).
|
||
- Provider login flows (`clawdbot providers login`) route through `plugin.auth.login` when available.
|
||
- `clawdbot status` reports `linkProvider` (derived from `status.buildProviderSummary().linked`) instead of a hardcoded `web` provider field.
|
||
- Gateway `web.login.*` methods use `plugin.gatewayMethods` ownership to pick the provider (no hardcoded `normalizeProviderId("web")` in the handler).
|
||
|
||
## CLI Commands (inline references)
|
||
- Add/remove providers: `clawdbot providers add <provider>` / `clawdbot providers remove <provider>`.
|
||
- Inspect provider state: `clawdbot providers list`, `clawdbot providers status`.
|
||
- Link/unlink providers: `clawdbot providers login --provider <provider>` / `clawdbot providers logout --provider <provider>`.
|
||
- Pairing approvals: `clawdbot pairing list <provider>`, `clawdbot pairing approve <provider> <code>`.
|
||
|
||
## Adding a Provider (checklist)
|
||
1) Create `src/providers/plugins/<id>.ts` exporting `ProviderPlugin`.
|
||
2) Register in `src/providers/plugins/index.ts` and update `src/providers/registry.ts` (ids/aliases/meta) if needed.
|
||
3) Add a dock entry in `src/providers/dock.ts` for any shared behavior (capabilities, allowFrom format/resolve, mention stripping, threading, streaming chunk defaults).
|
||
4) Add `reload.configPrefixes` for hot reload when config changes.
|
||
5) Delegate to existing provider modules (send/probe/monitor) or create them.
|
||
6) If you changed the gateway protocol: run `pnpm protocol:check` (updates `dist/protocol.schema.json` + `apps/macos/Sources/ClawdbotProtocol/GatewayModels.swift`).
|
||
7) Update docs/tests for any behavior changes.
|
||
|
||
## Cleanup Expectations
|
||
- Keep plugin files small; move heavy logic into provider modules.
|
||
- Prefer shared helpers over V2 copies.
|
||
- Update docs when behavior/inputs change.
|