diff --git a/CHANGELOG.md b/CHANGELOG.md index c53d0563b..dc8609673 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,10 +16,12 @@ - Groups: `whatsapp.groups`, `telegram.groups`, and `imessage.groups` now act as allowlists when set. Add `"*"` to keep allow-all behavior. ### Fixes +- Gateway/CLI: add daemon runtime selection (Node recommended; Bun optional) and document WhatsApp/Baileys Bun WebSocket instability on reconnect. - Heartbeat: default interval now 30m with a new default prompt + HEARTBEAT.md template. - Onboarding: write auth profiles to the multi-agent path (`~/.clawdbot/agents/main/agent/`) so the gateway finds credentials on first startup. Thanks @minghinmatthewlam for PR #327. - Docs: add missing `ui:install` setup step in the README. Thanks @hugobarauna for PR #300. - Docs: add ClawdHub guide and hubs link for browsing, install, and sync workflows. +- Docs: add FAQ for PNPM/Bun lockfile migration warning; link AgentSkills spec + ClawdHub from skills docs. - Build: import tool-display JSON as a module instead of runtime file reads. Thanks @mukhtharcm for PR #312. - Browser: fix `browser snapshot`/`browser act` timeouts under Bun by patching Playwright’s CDP WebSocket selection. Thanks @azade-c for PR #307. - Browser: add `--browser-profile` flag and honor profile in tabs routes + browser tool. Thanks @jamesgroat for PR #324. diff --git a/docs/android.md b/docs/android.md index b33760e98..0290639e1 100644 --- a/docs/android.md +++ b/docs/android.md @@ -28,7 +28,7 @@ The Gateway WebSocket stays loopback-only (`ws://127.0.0.1:18789`). Android talk Bridge is enabled by default (disable via `CLAWDBOT_BRIDGE_ENABLED=0`). ```bash -pnpm clawdbot gateway --port 18789 --verbose +clawdbot gateway --port 18789 --verbose ``` Confirm in logs you see something like: diff --git a/docs/faq.md b/docs/faq.md index 2614ebe36..e6677e4be 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -49,7 +49,7 @@ Some features are platform-specific: The gateway is just shuffling messages around. A Raspberry Pi 4 can run it. For the CLI, prefer the Node runtime (most stable): ```bash -pnpm clawdbot gateway +clawdbot gateway ``` ### How do I install on Linux without Homebrew? @@ -187,7 +187,7 @@ OAuth needs the callback to reach the machine running the CLI. Options: 5. **Start gateway:** ```bash - pnpm clawdbot gateway + clawdbot gateway ``` **Note:** WhatsApp may notice the IP change and require re-authentication. If so, run `pnpm clawdbot login` again. Stop the old instance before starting the new one to avoid conflicts. @@ -221,7 +221,7 @@ Key considerations: #!/bin/bash npm install -g pnpm cd /app -pnpm clawdbot gateway +clawdbot gateway ``` **Container command:** @@ -238,13 +238,19 @@ Yes! The terminal QR code login works fine over SSH. For long-running operation: - Use `pm2`, `systemd`, or a `launchd` plist to keep the gateway running. - Consider Tailscale for secure remote access. +### I'm seeing `InvalidPnpmLockfile: failed to migrate lockfile: 'pnpm-lock.yaml'` + +This can be ignored. This is simply a package manager warning when using PNPM and BUN. + +It often shows up when switching between `pnpm install` and `bun install`. If installs/build/tests work, you can safely ignore it. + ### bun binary vs Node runtime? Clawdbot can run as: - **bun binary (macOS app)** — Single executable, easy distribution, auto-restarts via launchd -- **Node runtime** (`pnpm clawdbot gateway`) — More stable for WhatsApp +- **Node runtime** (`clawdbot gateway`) — More stable for WhatsApp -If you see WebSocket errors like `ws.WebSocket 'upgrade' event is not implemented`, use Node instead of the bun binary. Bun's WebSocket implementation has edge cases that can break WhatsApp (Baileys). +If you see WebSocket errors like `ws.WebSocket 'upgrade' event is not implemented`, use Node instead of the bun binary. Bun's WebSocket implementation has edge cases that can break WhatsApp (Baileys) and can corrupt memory on reconnect. Baileys: https://github.com/WhiskeySockets/Baileys **For stability:** Use launchd (macOS) or the Clawdbot.app — they handle process supervision (auto-restart on crash). diff --git a/docs/gateway.md b/docs/gateway.md index 6fbf3aa0e..30fc41707 100644 --- a/docs/gateway.md +++ b/docs/gateway.md @@ -14,11 +14,11 @@ Last updated: 2025-12-09 ## How to run (local) ```bash -pnpm clawdbot gateway --port 18789 +clawdbot gateway --port 18789 # for full debug/trace logs in stdio: -pnpm clawdbot gateway --port 18789 --verbose +clawdbot gateway --port 18789 --verbose # if the port is busy, terminate listeners then start: -pnpm clawdbot gateway --force +clawdbot gateway --force # dev loop (auto-reload on TS changes): pnpm gateway:watch ``` diff --git a/docs/getting-started.md b/docs/getting-started.md index 3244784fb..06d9ac0a5 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -64,6 +64,7 @@ What you’ll choose: - **Auth**: Anthropic OAuth or OpenAI OAuth (recommended), API key (optional), or skip for now - **Providers**: WhatsApp QR login, bot tokens, etc. - **Daemon**: optional background install (launchd/systemd/Task Scheduler) + - **Runtime**: Node (recommended; required for WhatsApp) or Bun (faster, but incompatible with WhatsApp) Wizard doc: https://docs.clawd.bot/wizard @@ -79,11 +80,19 @@ Headless/server tip: do OAuth on a normal machine first, then copy `oauth.json` If the wizard didn’t start it for you: ```bash -bun run clawdbot gateway --port 18789 --verbose +# If you installed the CLI (npm/pnpm link --global): +clawdbot gateway --port 18789 --verbose +# From this repo: +node dist/entry.js gateway --port 18789 --verbose ``` Dashboard (local loopback): `http://127.0.0.1:18789/` +⚠️ **WhatsApp + Bun warning:** Baileys (WhatsApp Web library) uses a WebSocket +path that is currently incompatible with Bun and can cause memory corruption on +reconnect. If you use WhatsApp, run the Gateway with **Node** until this is +resolved. Baileys: https://github.com/WhiskeySockets/Baileys + ## 5) Pair + connect your first chat surface ### WhatsApp (QR login) diff --git a/docs/ios.md b/docs/ios.md index bd00a11a6..6c8d946a3 100644 --- a/docs/ios.md +++ b/docs/ios.md @@ -34,7 +34,7 @@ The Gateway WebSocket stays loopback-only (`ws://127.0.0.1:18789`). The iOS node Bridge is enabled by default (disable via `CLAWDBOT_BRIDGE_ENABLED=0`). ```bash -pnpm clawdbot gateway --port 18789 --verbose +clawdbot gateway --port 18789 --verbose ``` Confirm in logs you see something like: diff --git a/docs/session.md b/docs/session.md index b72222cec..957931bbd 100644 --- a/docs/session.md +++ b/docs/session.md @@ -76,7 +76,7 @@ Send these as standalone messages so they register. ## Inspecting - `pnpm clawdbot status` — shows store path and recent sessions. - `pnpm clawdbot sessions --json` — dumps every entry (filter with `--active `). -- `pnpm clawdbot gateway call sessions.list --params '{}'` — fetch sessions from the running gateway (use `--url`/`--token` for remote gateway access). +- `clawdbot gateway call sessions.list --params '{}'` — fetch sessions from the running gateway (use `--url`/`--token` for remote gateway access). - Send `/status` as a standalone message in chat to see whether the agent is reachable, how much of the session context is used, current thinking/verbose toggles, and when your WhatsApp web creds were last refreshed (helps spot relink needs). - Send `/compact` (optional instructions) as a standalone message to summarize older context and free up window space. - JSONL transcripts can be opened directly to review full turns. diff --git a/docs/updating.md b/docs/updating.md index 8c31091c3..b3de01043 100644 --- a/docs/updating.md +++ b/docs/updating.md @@ -121,7 +121,7 @@ Then reinstall deps + restart: ```bash pnpm install pnpm build -pnpm clawdbot gateway restart +clawdbot gateway restart ``` If you want to go back to latest later: diff --git a/docs/wizard.md b/docs/wizard.md index 76ac67534..c513eb377 100644 --- a/docs/wizard.md +++ b/docs/wizard.md @@ -81,6 +81,7 @@ It does **not** install or change anything on the remote host. - May prompt for sudo (writes `/var/lib/systemd/linger`); it tries without sudo first. - Windows: Scheduled Task - Runs on user logon; headless/system services are not configured by default. + - **Runtime selection:** Node (recommended; required for WhatsApp) or Bun (faster, but incompatible with WhatsApp). 7) **Health check** - Starts the Gateway (if needed) and runs `clawdbot health`. @@ -120,6 +121,8 @@ clawdbot onboard --non-interactive \ --anthropic-api-key "$ANTHROPIC_API_KEY" \ --gateway-port 18789 \ --gateway-bind loopback \ + --install-daemon \ + --daemon-runtime node \ --skip-skills ``` diff --git a/src/cli/program.ts b/src/cli/program.ts index 072339a8c..6f03c8492 100644 --- a/src/cli/program.ts +++ b/src/cli/program.ts @@ -228,6 +228,7 @@ export function buildProgram() { .option("--tailscale ", "Tailscale: off|serve|funnel") .option("--tailscale-reset-on-exit", "Reset tailscale serve/funnel on exit") .option("--install-daemon", "Install gateway daemon") + .option("--daemon-runtime ", "Daemon runtime: node|bun") .option("--skip-skills", "Skip skills setup") .option("--skip-health", "Skip health check") .option("--node-manager ", "Node manager for skills: npm|pnpm|bun") @@ -268,6 +269,7 @@ export function buildProgram() { tailscale: opts.tailscale as "off" | "serve" | "funnel" | undefined, tailscaleResetOnExit: Boolean(opts.tailscaleResetOnExit), installDaemon: Boolean(opts.installDaemon), + daemonRuntime: opts.daemonRuntime as "node" | "bun" | undefined, skipSkills: Boolean(opts.skipSkills), skipHealth: Boolean(opts.skipHealth), nodeManager: opts.nodeManager as "npm" | "pnpm" | "bun" | undefined, diff --git a/src/commands/configure.ts b/src/commands/configure.ts index 98eca3125..04be1bcd8 100644 --- a/src/commands/configure.ts +++ b/src/commands/configure.ts @@ -59,6 +59,11 @@ import { import { setupProviders } from "./onboard-providers.js"; import { promptRemoteGatewayConfig } from "./onboard-remote.js"; import { setupSkills } from "./onboard-skills.js"; +import { + DEFAULT_GATEWAY_DAEMON_RUNTIME, + GATEWAY_DAEMON_RUNTIME_OPTIONS, + type GatewayDaemonRuntime, +} from "./daemon-runtime.js"; import { applyOpenAICodexModelDefault, OPENAI_CODEX_DEFAULT_MODEL, @@ -502,11 +507,13 @@ async function maybeInstallDaemon(params: { runtime: RuntimeEnv; port: number; gatewayToken?: string; + daemonRuntime?: GatewayDaemonRuntime; }) { const service = resolveGatewayService(); const loaded = await service.isLoaded({ env: process.env }); let shouldCheckLinger = false; let shouldInstall = true; + let daemonRuntime = params.daemonRuntime ?? DEFAULT_GATEWAY_DAEMON_RUNTIME; if (loaded) { const action = guardCancel( await select({ @@ -531,11 +538,25 @@ async function maybeInstallDaemon(params: { } if (shouldInstall) { + if (!params.daemonRuntime) { + daemonRuntime = guardCancel( + await select({ + message: "Gateway daemon runtime", + options: GATEWAY_DAEMON_RUNTIME_OPTIONS, + initialValue: DEFAULT_GATEWAY_DAEMON_RUNTIME, + }), + params.runtime, + ) as GatewayDaemonRuntime; + } const devMode = process.argv[1]?.includes(`${path.sep}src${path.sep}`) && process.argv[1]?.endsWith(".ts"); const { programArguments, workingDirectory } = - await resolveGatewayProgramArguments({ port: params.port, dev: devMode }); + await resolveGatewayProgramArguments({ + port: params.port, + dev: devMode, + runtime: daemonRuntime, + }); const environment: Record = { PATH: process.env.PATH, CLAWDBOT_GATEWAY_TOKEN: params.gatewayToken, diff --git a/src/commands/daemon-runtime.ts b/src/commands/daemon-runtime.ts new file mode 100644 index 000000000..8c22ceac5 --- /dev/null +++ b/src/commands/daemon-runtime.ts @@ -0,0 +1,27 @@ +export type GatewayDaemonRuntime = "node" | "bun"; + +export const DEFAULT_GATEWAY_DAEMON_RUNTIME: GatewayDaemonRuntime = "node"; + +export const GATEWAY_DAEMON_RUNTIME_OPTIONS: Array<{ + value: GatewayDaemonRuntime; + label: string; + hint?: string; +}> = [ + { + value: "node", + label: "Node (recommended)", + hint: + "Required for WhatsApp (Baileys WebSocket + Bun can corrupt memory on reconnect)", + }, + { + value: "bun", + label: "Bun (faster)", + hint: "Use only when WhatsApp is disabled", + }, +]; + +export function isGatewayDaemonRuntime( + value: string | undefined, +): value is GatewayDaemonRuntime { + return value === "node" || value === "bun"; +} diff --git a/src/commands/doctor.ts b/src/commands/doctor.ts index e0e4b8bcc..707804e26 100644 --- a/src/commands/doctor.ts +++ b/src/commands/doctor.ts @@ -2,7 +2,7 @@ import fs from "node:fs"; import os from "node:os"; import path from "node:path"; -import { confirm, intro, note, outro } from "@clack/prompts"; +import { confirm, intro, note, outro, select } from "@clack/prompts"; import { DEFAULT_SANDBOX_BROWSER_IMAGE, @@ -45,6 +45,11 @@ import { guardCancel, printWizardHeader, } from "./onboard-helpers.js"; +import { + DEFAULT_GATEWAY_DAEMON_RUNTIME, + GATEWAY_DAEMON_RUNTIME_OPTIONS, + type GatewayDaemonRuntime, +} from "./daemon-runtime.js"; import { ensureSystemdUserLingerInteractive } from "./systemd-linger.js"; function resolveMode(cfg: ClawdbotConfig): "local" | "remote" { @@ -768,12 +773,24 @@ async function maybeMigrateLegacyGatewayService( ); if (!install) return; + const daemonRuntime = guardCancel( + await select({ + message: "Gateway daemon runtime", + options: GATEWAY_DAEMON_RUNTIME_OPTIONS, + initialValue: DEFAULT_GATEWAY_DAEMON_RUNTIME, + }), + runtime, + ) as GatewayDaemonRuntime; const devMode = process.argv[1]?.includes(`${path.sep}src${path.sep}`) && process.argv[1]?.endsWith(".ts"); const port = resolveGatewayPort(cfg, process.env); const { programArguments, workingDirectory } = - await resolveGatewayProgramArguments({ port, dev: devMode }); + await resolveGatewayProgramArguments({ + port, + dev: devMode, + runtime: daemonRuntime, + }); const environment: Record = { PATH: process.env.PATH, CLAWDBOT_GATEWAY_TOKEN: diff --git a/src/commands/onboard-non-interactive.ts b/src/commands/onboard-non-interactive.ts index 7b8127ddc..864de50ee 100644 --- a/src/commands/onboard-non-interactive.ts +++ b/src/commands/onboard-non-interactive.ts @@ -30,6 +30,10 @@ import type { OnboardMode, OnboardOptions, } from "./onboard-types.js"; +import { + DEFAULT_GATEWAY_DAEMON_RUNTIME, + isGatewayDaemonRuntime, +} from "./daemon-runtime.js"; import { ensureSystemdUserLingerNonInteractive } from "./systemd-linger.js"; export async function runNonInteractiveOnboarding( @@ -223,13 +227,24 @@ export async function runNonInteractiveOnboarding( skipBootstrap: Boolean(nextConfig.agent?.skipBootstrap), }); + const daemonRuntimeRaw = opts.daemonRuntime ?? DEFAULT_GATEWAY_DAEMON_RUNTIME; + if (opts.installDaemon) { + if (!isGatewayDaemonRuntime(daemonRuntimeRaw)) { + runtime.error("Invalid --daemon-runtime (use node or bun)"); + runtime.exit(1); + return; + } const service = resolveGatewayService(); const devMode = process.argv[1]?.includes(`${path.sep}src${path.sep}`) && process.argv[1]?.endsWith(".ts"); const { programArguments, workingDirectory } = - await resolveGatewayProgramArguments({ port, dev: devMode }); + await resolveGatewayProgramArguments({ + port, + dev: devMode, + runtime: daemonRuntimeRaw, + }); const environment: Record = { PATH: process.env.PATH, CLAWDBOT_GATEWAY_TOKEN: gatewayToken, @@ -260,6 +275,7 @@ export async function runNonInteractiveOnboarding( authChoice, gateway: { port, bind, authMode, tailscaleMode }, installDaemon: Boolean(opts.installDaemon), + daemonRuntime: opts.installDaemon ? daemonRuntimeRaw : undefined, skipSkills: Boolean(opts.skipSkills), skipHealth: Boolean(opts.skipHealth), }, diff --git a/src/commands/onboard-types.ts b/src/commands/onboard-types.ts index 42a5734df..9d334ea55 100644 --- a/src/commands/onboard-types.ts +++ b/src/commands/onboard-types.ts @@ -1,3 +1,5 @@ +import type { GatewayDaemonRuntime } from "./daemon-runtime.js"; + export type OnboardMode = "local" | "remote"; export type AuthChoice = | "oauth" @@ -33,6 +35,7 @@ export type OnboardOptions = { tailscale?: TailscaleMode; tailscaleResetOnExit?: boolean; installDaemon?: boolean; + daemonRuntime?: GatewayDaemonRuntime; skipSkills?: boolean; skipHealth?: boolean; nodeManager?: NodeManagerChoice; diff --git a/src/daemon/program-args.ts b/src/daemon/program-args.ts index bd530a789..d4175f224 100644 --- a/src/daemon/program-args.ts +++ b/src/daemon/program-args.ts @@ -6,6 +6,8 @@ type GatewayProgramArgs = { workingDirectory?: string; }; +type GatewayRuntimePreference = "auto" | "node" | "bun"; + function isNodeRuntime(execPath: string): boolean { const base = path.basename(execPath).toLowerCase(); return base === "node" || base === "node.exe"; @@ -114,23 +116,69 @@ function resolveRepoRootForDev(): string { } async function resolveBunPath(): Promise { - // Bun is expected to be in PATH, resolve via which/where + const bunPath = await resolveBinaryPath("bun"); + return bunPath; +} + +async function resolveNodePath(): Promise { + const nodePath = await resolveBinaryPath("node"); + return nodePath; +} + +async function resolveBinaryPath(binary: string): Promise { const { execSync } = await import("node:child_process"); + const cmd = process.platform === "win32" ? "where" : "which"; try { - const bunPath = execSync("which bun", { encoding: "utf8" }).trim(); - await fs.access(bunPath); - return bunPath; + const output = execSync(`${cmd} ${binary}`, { encoding: "utf8" }).trim(); + const resolved = output.split(/\r?\n/)[0]?.trim(); + if (!resolved) throw new Error("empty"); + await fs.access(resolved); + return resolved; } catch { - throw new Error("Bun not found in PATH. Install bun: https://bun.sh"); + if (binary === "bun") { + throw new Error("Bun not found in PATH. Install bun: https://bun.sh"); + } + throw new Error("Node not found in PATH. Install Node 22+."); } } export async function resolveGatewayProgramArguments(params: { port: number; dev?: boolean; + runtime?: GatewayRuntimePreference; }): Promise { const gatewayArgs = ["gateway-daemon", "--port", String(params.port)]; const execPath = process.execPath; + const runtime = params.runtime ?? "auto"; + + if (runtime === "node") { + const nodePath = isNodeRuntime(execPath) + ? execPath + : await resolveNodePath(); + const cliEntrypointPath = await resolveCliEntrypointPathForService(); + return { + programArguments: [nodePath, cliEntrypointPath, ...gatewayArgs], + }; + } + + if (runtime === "bun") { + if (params.dev) { + const repoRoot = resolveRepoRootForDev(); + const devCliPath = path.join(repoRoot, "src", "index.ts"); + await fs.access(devCliPath); + const bunPath = isBunRuntime(execPath) ? execPath : await resolveBunPath(); + return { + programArguments: [bunPath, devCliPath, ...gatewayArgs], + workingDirectory: repoRoot, + }; + } + + const bunPath = isBunRuntime(execPath) ? execPath : await resolveBunPath(); + const cliEntrypointPath = await resolveCliEntrypointPathForService(); + return { + programArguments: [bunPath, cliEntrypointPath, ...gatewayArgs], + }; + } if (!params.dev) { try { diff --git a/src/wizard/onboarding.ts b/src/wizard/onboarding.ts index 7536ab44b..2f38ce09e 100644 --- a/src/wizard/onboarding.ts +++ b/src/wizard/onboarding.ts @@ -52,6 +52,11 @@ import type { OnboardOptions, ResetScope, } from "../commands/onboard-types.js"; +import { + DEFAULT_GATEWAY_DAEMON_RUNTIME, + GATEWAY_DAEMON_RUNTIME_OPTIONS, + type GatewayDaemonRuntime, +} from "../commands/daemon-runtime.js"; import { applyOpenAICodexModelDefault, OPENAI_CODEX_DEFAULT_MODEL, @@ -629,6 +634,11 @@ export async function runOnboardingWizard( }); if (installDaemon) { + const daemonRuntime = (await prompter.select({ + message: "Gateway daemon runtime", + options: GATEWAY_DAEMON_RUNTIME_OPTIONS, + initialValue: opts.daemonRuntime ?? DEFAULT_GATEWAY_DAEMON_RUNTIME, + })) as GatewayDaemonRuntime; const service = resolveGatewayService(); const loaded = await service.isLoaded({ env: process.env }); if (loaded) { @@ -655,7 +665,11 @@ export async function runOnboardingWizard( process.argv[1]?.includes(`${path.sep}src${path.sep}`) && process.argv[1]?.endsWith(".ts"); const { programArguments, workingDirectory } = - await resolveGatewayProgramArguments({ port, dev: devMode }); + await resolveGatewayProgramArguments({ + port, + dev: devMode, + runtime: daemonRuntime, + }); const environment: Record = { PATH: process.env.PATH, CLAWDBOT_GATEWAY_TOKEN: gatewayToken,