Files
clawdbot/docs/refactor/provider-plugin.md
Peter Steinberger 7acd26a2fc Move provider to a plugin-architecture (#661)
* refactor: introduce provider plugin registry

* refactor: move provider CLI to plugins

* docs: add provider plugin implementation notes

* refactor: shift provider runtime logic into plugins

* refactor: add plugin defaults and summaries

* docs: update provider plugin notes

* feat(commands): add /commands slash list

* Auto-reply: tidy help message

* Auto-reply: fix status command lint

* Tests: align google shared expectations

* Auto-reply: tidy help message

* Auto-reply: fix status command lint

* refactor: move provider routing into plugins

* test: align agent routing expectations

* docs: update provider plugin notes

* refactor: route replies via provider plugins

* docs: note route-reply plugin hooks

* refactor: extend provider plugin contract

* refactor: derive provider status from plugins

* refactor: unify gateway provider control

* refactor: use plugin metadata in auto-reply

* fix: parenthesize cron target selection

* refactor: derive gateway methods from plugins

* refactor: generalize provider logout

* refactor: route provider logout through plugins

* refactor: move WhatsApp web login methods into plugin

* refactor: generalize provider log prefixes

* refactor: centralize default chat provider

* refactor: derive provider lists from registry

* refactor: move provider reload noops into plugins

* refactor: resolve web login provider via alias

* refactor: derive CLI provider options from plugins

* refactor: derive prompt provider list from plugins

* style: apply biome lint fixes

* fix: resolve provider routing edge cases

* docs: update provider plugin refactor notes

* fix(gateway): harden agent provider routing

* refactor: move provider routing into plugins

* refactor: move provider CLI to plugins

* refactor: derive provider lists from registry

* fix: restore slash command parsing

* refactor: align provider ids for schema

* refactor: unify outbound target resolution

* fix: keep outbound labels stable

* feat: add msteams to cron surfaces

* fix: clean up lint build issues

* refactor: localize chat provider alias normalization

* refactor: drive gateway provider lists from plugins

* docs: update provider plugin notes

* style: format message-provider

* fix: avoid provider registry init cycles

* style: sort message-provider imports

* fix: relax provider alias map typing

* refactor: move provider routing into plugins

* refactor: add plugin pairing/config adapters

* refactor: route pairing and provider removal via plugins

* refactor: align auto-reply provider typing

* test: stabilize telegram media mocks

* docs: update provider plugin refactor notes

* refactor: pluginize outbound targets

* refactor: pluginize provider selection

* refactor: generalize text chunk limits

* docs: update provider plugin notes

* refactor: generalize group session/config

* fix: normalize provider id for room detection

* fix: avoid provider init in system prompt

* style: formatting cleanup

* refactor: normalize agent delivery targets

* test: update outbound delivery labels

* chore: fix lint regressions

* refactor: extend provider plugin adapters

* refactor: move elevated/block streaming defaults to plugins

* refactor: defer outbound send deps to plugins

* docs: note plugin-driven streaming/elevated defaults

* refactor: centralize webchat provider constant

* refactor: add provider setup adapters

* refactor: delegate provider add config to plugins

* docs: document plugin-driven provider add

* refactor: add plugin state/binding metadata

* refactor: build agent provider status from plugins

* docs: note plugin-driven agent bindings

* refactor: centralize internal provider constant usage

* fix: normalize WhatsApp targets for groups and E.164 (#631) (thanks @imfing)

* refactor: centralize default chat provider

* refactor: centralize WhatsApp target normalization

* refactor: move provider routing into plugins

* refactor: normalize agent delivery targets

* chore: fix lint regressions

* fix: normalize WhatsApp targets for groups and E.164 (#631) (thanks @imfing)

* feat: expand provider plugin adapters

* refactor: route auto-reply via provider plugins

* fix: align WhatsApp target normalization

* fix: normalize WhatsApp targets for groups and E.164 (#631) (thanks @imfing)

* refactor: centralize WhatsApp target normalization

* feat: add /config chat config updates

* docs: add /config get alias

* feat(commands): add /commands slash list

* refactor: centralize default chat provider

* style: apply biome lint fixes

* chore: fix lint regressions

* fix: clean up whatsapp allowlist typing

* style: format config command helpers

* refactor: pluginize tool threading context

* refactor: normalize session announce targets

* docs: note new plugin threading and announce hooks

* refactor: pluginize message actions

* docs: update provider plugin actions notes

* fix: align provider action adapters

* refactor: centralize webchat checks

* style: format message provider helpers

* refactor: move provider onboarding into adapters

* docs: note onboarding provider adapters

* feat: add msteams onboarding adapter

* style: organize onboarding imports

* fix: normalize msteams allowFrom types

* feat: add plugin text chunk limits

* refactor: use plugin chunk limit fallbacks

* feat: add provider mention stripping hooks

* style: organize provider plugin type imports

* refactor: generalize health snapshots

* refactor: update macOS health snapshot handling

* docs: refresh health snapshot notes

* style: format health snapshot updates

* refactor: drive security warnings via plugins

* docs: note provider security adapter

* style: format provider security adapters

* refactor: centralize provider account defaults

* refactor: type gateway client identity constants

* chore: regen gateway protocol swift

* fix: degrade health on failed provider probe

* refactor: centralize pairing approve hint

* docs: add plugin CLI command references

* refactor: route auth and tool sends through plugins

* docs: expand provider plugin hooks

* refactor: document provider docking touchpoints

* refactor: normalize internal provider defaults

* refactor: streamline outbound delivery wiring

* refactor: make provider onboarding plugin-owned

* refactor: support provider-owned agent tools

* refactor: move telegram draft chunking into telegram module

* refactor: infer provider tool sends via extractToolSend

* fix: repair plugin onboarding imports

* refactor: de-dup outbound target normalization

* style: tidy plugin and agent imports

* refactor: data-drive provider selection line

* fix: satisfy lint after provider plugin rebase

* test: deflake gateway-cli coverage

* style: format gateway-cli coverage test

* refactor(provider-plugins): simplify provider ids

* test(pairing-cli): avoid provider-specific ternary

* style(macos): swiftformat HealthStore

* refactor(sandbox): derive provider tool denylist

* fix(sandbox): avoid plugin init in defaults

* refactor(provider-plugins): centralize provider aliases

* style(test): satisfy biome

* refactor(protocol): v3 providers.status maps

* refactor(ui): adapt to protocol v3

* refactor(macos): adapt to protocol v3

* test: update providers.status v3 fixtures

* refactor(gateway): map provider runtime snapshot

* test(gateway): update reload runtime snapshot

* refactor(whatsapp): normalize heartbeat provider id

* docs(refactor): update provider plugin notes

* style: satisfy biome after rebase

* fix: describe sandboxed elevated in prompt

* feat(gateway): add agent image attachments + live probe

* refactor: derive CLI provider options from plugins

* fix(gateway): harden agent provider routing

* fix(gateway): harden agent provider routing

* refactor: align provider ids for schema

* fix(protocol): keep agent provider string

* fix(gateway): harden agent provider routing

* fix(protocol): keep agent provider string

* refactor: normalize agent delivery targets

* refactor: support provider-owned agent tools

* refactor(config): provider-keyed elevated allowFrom

* style: satisfy biome

* fix(gateway): appease provider narrowing

* style: satisfy biome

* refactor(reply): move group intro hints into plugin

* fix(reply): avoid plugin registry init cycle

* refactor(providers): add lightweight provider dock

* refactor(gateway): use typed client id in connect

* refactor(providers): document docks and avoid init cycles

* refactor(providers): make media limit helper generic

* fix(providers): break plugin registry import cycles

* style: satisfy biome

* refactor(status-all): build providers table from plugins

* refactor(gateway): delegate web login to provider plugin

* refactor(provider): drop web alias

* refactor(provider): lazy-load monitors

* style: satisfy lint/format

* style: format status-all providers table

* style: swiftformat gateway discovery model

* test: make reload plan plugin-driven

* fix: avoid token stringification in status-all

* refactor: make provider IDs explicit in status

* feat: warn on signal/imessage provider runtime errors

* test: cover gateway provider runtime warnings in status

* fix: add runtime kind to provider status issues

* test: cover health degradation on probe failure

* fix: keep routeReply lightweight

* style: organize routeReply imports

* refactor(web): extract auth-store helpers

* refactor(whatsapp): lazy login imports

* refactor(outbound): route replies via plugin outbound

* docs: update provider plugin notes

* style: format provider status issues

* fix: make sandbox scope warning wrap-safe

* refactor: load outbound adapters from provider plugins

* docs: update provider plugin outbound notes

* style(macos): fix swiftformat lint

* docs: changelog for provider plugins

* fix(macos): satisfy swiftformat

* fix(macos): open settings via menu action

* style: format after rebase

* fix(macos): open Settings via menu action

---------

Co-authored-by: LK <luke@kyohere.com>
Co-authored-by: Luke K (pr-0f3t) <2609441+lc0rp@users.noreply.github.com>
Co-authored-by: Xin <xin@imfing.com>
2026-01-11 11:45:25 +00:00

114 lines
11 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
summary: "Provider plugin refactor implementation notes (registry, status, gateway/runtime)"
read_when:
- Adding or refactoring provider plugin wiring
- Moving provider-specific behavior into plugin hooks
---
# Provider Plugin Refactor — Implementation Notes
Goal: make providers (iMessage, Discord, etc.) pluggable with minimal wiring and shared UX/state paths.
## Architecture Overview
- Registry: `src/providers/plugins/index.ts` owns the plugin list.
- Provider dock: `src/providers/dock.ts` owns lightweight provider metadata used by shared flows (reply, command auth, block streaming) without importing full plugins.
- IDs/aliases: `src/providers/registry.ts` owns stable provider ids + input aliases.
- Shape: `src/providers/plugins/types.ts` defines the plugin contract.
- Gateway: `src/gateway/server-providers.ts` drives start/stop + runtime snapshots via plugins.
- Outbound: `src/infra/outbound/deliver.ts` routes through plugin outbound when present.
- Outbound delivery loads **outbound adapters** on-demand via `src/providers/plugins/outbound/load.ts` (avoid importing heavy provider plugins on hot paths).
- Reload: `src/gateway/config-reload.ts` uses plugin `reload.configPrefixes` lazily (avoid init cycles).
- CLI: `src/commands/providers/*` uses plugin list for add/remove/status/list.
- Protocol: `src/gateway/protocol/schema.ts` (v3) makes provider-shaped responses container-generic (maps keyed by provider id).
## Plugin Contract (high-level)
Each `ProviderPlugin` bundles:
- `meta`: id/labels/docs/sort order.
- `capabilities`: chatTypes + optional features (polls, media, nativeCommands, etc.).
- `config`: list/resolve/default/isConfigured/describeAccount + isEnabled + (un)configured reasons + `resolveAllowFrom` + `formatAllowFrom`.
- `outbound`: deliveryMode + chunker + resolveTarget (mode-aware) + sendText/sendMedia/sendPoll + pollMaxOptions.
- `status`: defaultRuntime + probe/audit/buildAccountSnapshot + buildProviderSummary + logSelfId + collectStatusIssues.
- `gateway`: startAccount/stopAccount with runtime context (`getStatus`/`setStatus`), plus optional `loginWithQrStart/loginWithQrWait` for gateway-owned QR login flows.
- `security`: dmPolicy + allowFrom hints used by `doctor security`.
- `heartbeat`: optional readiness checks + heartbeat recipient resolution when providers own targeting.
- `auth`: optional login hook used by `clawdbot providers login`.
- `reload`: `configPrefixes` that map to hot restarts.
- `onboarding`: optional CLI onboarding adapter (wizard UI hooks per provider).
- `agentTools`: optional provider-owned agent tools (ex: QR login).
## Key Integration Notes
- `listProviderPlugins()` is the runtime source of truth for provider UX and wiring.
- Avoid importing `src/providers/plugins/index.ts` from shared modules (reply flow, command auth, sandbox explain). Its intentionally “heavy” (providers may pull web login / monitor code). Use `getProviderDock()` + `normalizeProviderId()` for cheap metadata, and only `getProviderPlugin()` at execution boundaries (ex: `src/auto-reply/reply/route-reply.ts`).
- WhatsApp plugin keeps Baileys-heavy login bits behind lazy imports; cheap auth file checks live in `src/web/auth-store.ts` (so outbound routing doesnt pay Baileys import cost).
- `routeReply` delegates sending to plugin `outbound` adapters via a lazy import of `src/infra/outbound/deliver.ts` (so adding a provider is “just implement outbound adapter”, no router switches).
- Avoid static imports of provider monitors inside plugin modules. Monitors typically import the reply pipeline, which can create ESM cycles (and break Vite/Vitest SSR with TDZ errors). Prefer lazy imports inside `gateway.startAccount`.
- Debug cycle leaks quickly with: `npx -y madge --circular src/providers/plugins/index.ts`.
- Gateway protocol schema keeps provider selection as an open-ended string (no provider enum / static list) to avoid init cycles and so new plugins dont require protocol changes.
- Protocol v3: no more per-provider fields in `providers.status`; consumers must read map entries by provider id.
- `DEFAULT_CHAT_PROVIDER` lives in `src/providers/registry.ts` and is used anywhere we need a fallback delivery surface.
- Provider reload rules are computed lazily to avoid static init cycles in tests.
- Signal/iMessage media size limits are now resolved inside their plugins.
- `normalizeProviderId()` handles aliases (ex: `imsg`, `teams`) so CLI and API inputs stay stable.
- `ProviderId` is `ChatProviderId` (no extra special-cased provider IDs in shared code).
- Gateway runtime defaults (`status.defaultRuntime`) replace the old per-provider runtime map.
- Gateway runtime snapshot (`getRuntimeSnapshot`) is map-based: `{ providers, providerAccounts }` (no `${id}Accounts` keys).
- `providers.status` response keys (v3):
- `providerOrder: string[]`
- `providerLabels: Record<string, string>`
- `providers: Record<string, unknown>` (provider summary objects, plugin-defined)
- `providerAccounts: Record<string, ProviderAccountSnapshot[]>`
- `providerDefaultAccountId: Record<string, string>`
- `providers.status` summary objects come from `status.buildProviderSummary` (no per-provider branching in the handler).
- `providers.status` warnings now flow through `status.collectStatusIssues` per plugin.
- CLI list uses `meta.showConfigured` to decide whether to show configured state.
- CLI provider options and prompt provider lists are generated from `listProviderPlugins()` (avoid hardcoded arrays).
- Provider selection (`resolveMessageProviderSelection`) now inspects `config.isEnabled` + `config.isConfigured` per plugin instead of hardcoded provider checks.
- Pairing flows (CLI + store) now use `plugin.pairing` (`idLabel`, `normalizeAllowEntry`, `notifyApproval`) via `src/providers/plugins/pairing.ts`.
- CLI provider remove/disable delegates to `config.setAccountEnabled` + `config.deleteAccount` per plugin.
- CLI provider add now delegates to `plugin.setup` for account validation, naming, and config writes (no hardcoded provider checks).
- Agent provider status entries are now built from plugin config/status (`status.resolveAccountState` for custom state labels).
- Agent binding defaults use `meta.forceAccountBinding` to avoid hardcoded provider checks.
- Onboarding quickstart allowlist uses `meta.quickstartAllowFrom` to avoid hardcoded provider lists.
- `resolveProviderDefaultAccountId()` is the shared helper for picking default accounts from `accountIds` + plugin config.
- `routeReply` uses plugin outbound senders; `ProviderOutboundContext` supports `replyToId` + `threadId` and outbound delivery supports `abortSignal` for cooperative cancellation.
- Outbound target resolution (`resolveOutboundTarget`) now delegates to `plugin.outbound.resolveTarget` (mode-aware, uses config allowlists when present).
- Outbound delivery results accept `meta` for provider-specific fields to avoid core type churn in new plugins.
- Agent gateway routing sets `deliveryTargetMode` and uses `resolveOutboundTarget` for implicit fallback targets when `to` is missing.
- Elevated tool allowlists (`tools.elevated.allowFrom`) are a record keyed by provider id (no schema update needed when adding providers).
- Block streaming defaults live on the plugin (`capabilities.blockStreaming`, `streaming.blockStreamingCoalesceDefaults`) instead of hardcoded provider checks.
- Provider logout now routes through `providers.logout` using `gateway.logoutAccount` on each plugin (clients should call the generic method).
- Gateway message-provider normalization uses `src/providers/registry.ts` for cheap validation/normalization without plugin init cycles.
- Group mention gating now flows through `plugin.groups.resolveRequireMention` (Discord/Slack/Telegram/WhatsApp/iMessage) instead of branching in reply handlers.
- Command authorization uses `config.resolveAllowFrom` + `config.formatAllowFrom`, with `commands.enforceOwnerForCommands` and `commands.skipWhenConfigEmpty` driving provider-specific behavior.
- Security warnings (`doctor security`) use `plugin.security.resolveDmPolicy` + `plugin.security.collectWarnings`; supply `policyPath` + `allowFromPath` for accurate config hints.
- Reply threading uses `plugin.threading.resolveReplyToMode` and `plugin.threading.allowTagsWhenOff` rather than provider switches in reply helpers.
- Tool auto-threading context flows through `plugin.threading.buildToolContext` (e.g., Slack threadTs injection).
- Messaging tool dedupe now relies on `plugin.messaging.normalizeTarget` for provider-specific target normalization.
- Message tool + CLI action dispatch now use `plugin.actions.listActions` + `plugin.actions.handleAction`; use `plugin.actions.supportsAction` for dispatch-only gating when you still want fallback send/poll.
- Session announce targets can opt into `meta.preferSessionLookupForAnnounceTarget` when session keys are insufficient (e.g., WhatsApp).
- Onboarding provider setup is delegated to adapter modules under `src/providers/plugins/onboarding/*`, keeping `setupProviders` provider-agnostic.
- Onboarding registry now reads `plugin.onboarding` from each provider (no standalone onboarding map).
- Provider login flows (`clawdbot providers login`) route through `plugin.auth.login` when available.
- `clawdbot status` reports `linkProvider` (derived from `status.buildProviderSummary().linked`) instead of a hardcoded `web` provider field.
- Gateway `web.login.*` methods use `plugin.gatewayMethods` ownership to pick the provider (no hardcoded `normalizeProviderId("web")` in the handler).
## CLI Commands (inline references)
- Add/remove providers: `clawdbot providers add <provider>` / `clawdbot providers remove <provider>`.
- Inspect provider state: `clawdbot providers list`, `clawdbot providers status`.
- Link/unlink providers: `clawdbot providers login --provider <provider>` / `clawdbot providers logout --provider <provider>`.
- Pairing approvals: `clawdbot pairing list <provider>`, `clawdbot pairing approve <provider> <code>`.
## Adding a Provider (checklist)
1) Create `src/providers/plugins/<id>.ts` exporting `ProviderPlugin`.
2) Register in `src/providers/plugins/index.ts` and update `src/providers/registry.ts` (ids/aliases/meta) if needed.
3) Add a dock entry in `src/providers/dock.ts` for any shared behavior (capabilities, allowFrom format/resolve, mention stripping, threading, streaming chunk defaults).
4) Add `reload.configPrefixes` for hot reload when config changes.
5) Delegate to existing provider modules (send/probe/monitor) or create them.
6) If you changed the gateway protocol: run `pnpm protocol:check` (updates `dist/protocol.schema.json` + `apps/macos/Sources/ClawdbotProtocol/GatewayModels.swift`).
7) Update docs/tests for any behavior changes.
## Cleanup Expectations
- Keep plugin files small; move heavy logic into provider modules.
- Prefer shared helpers over V2 copies.
- Update docs when behavior/inputs change.