docs: reorganize documentation structure
This commit is contained in:
Peter Steinberger
132
docs/automation/cron-jobs.md
Normal file
132
docs/automation/cron-jobs.md
Normal file
@@ -0,0 +1,132 @@
|
||||
---
|
||||
summary: "Cron jobs + wakeups for the Gateway scheduler"
|
||||
read_when:
|
||||
- Scheduling background jobs or wakeups
|
||||
- Wiring automation that should run with or alongside heartbeats
|
||||
---
|
||||
# Cron jobs (Gateway scheduler)
|
||||
|
||||
Cron runs inside the Gateway and schedules background work so Clawdbot can
|
||||
wake itself up, run isolated agent jobs, and deliver reminders on time.
|
||||
|
||||
## Update checklist (internal)
|
||||
- [x] Audit cron + heartbeat behavior in code
|
||||
- [x] Rewrite cron doc as user-facing feature
|
||||
- [x] Update heartbeat docs + templates
|
||||
- [x] Update cron links in docs
|
||||
- [x] Update changelog
|
||||
- [x] Run full gate (lint/build/test/docs)
|
||||
|
||||
## What cron is
|
||||
- **Gateway-owned scheduler** that persists jobs under `~/.clawdbot/cron/`.
|
||||
- **Two execution modes**:
|
||||
- **Main session jobs** enqueue `System:` events and rely on the heartbeat runner.
|
||||
- **Isolated jobs** run a dedicated agent turn in `cron:<jobId>` sessions.
|
||||
- **Wakeups** are first-class: a job can trigger the next heartbeat or run it now.
|
||||
|
||||
## When to use it
|
||||
- Recurring reminders: “every weekday at 7:30” or “every 2h.”
|
||||
- Background chores: summarize inboxes, check dashboards, watch logs.
|
||||
- Automation that should not pollute the main chat history.
|
||||
- Scheduled wakeups that drive the heartbeat pipeline.
|
||||
|
||||
## Schedules
|
||||
Cron supports three schedule kinds:
|
||||
- `at`: one-shot timestamp in ms.
|
||||
- `every`: fixed interval (ms).
|
||||
- `cron`: 5-field cron expression, optional IANA timezone.
|
||||
|
||||
Cron expressions use `croner` under the hood. If a timezone is omitted, the
|
||||
server’s local timezone is used.
|
||||
|
||||
## Job types
|
||||
|
||||
### Main session jobs
|
||||
Main jobs enqueue a system event and optionally wake the heartbeat runner.
|
||||
They **must** use `payload.kind = "systemEvent"`.
|
||||
|
||||
- **`wakeMode: "next-heartbeat"`** (default): the event waits for the next
|
||||
scheduled heartbeat.
|
||||
- **`wakeMode: "now"`**: the event triggers an immediate heartbeat run.
|
||||
|
||||
### Isolated jobs
|
||||
Isolated jobs run a dedicated agent turn in session `cron:<jobId>` and can
|
||||
optionally deliver a message.
|
||||
|
||||
Key behaviors:
|
||||
- Prompt is prefixed with `[cron:<jobId> <job name>]` for traceability.
|
||||
- A summary is posted to the main session with prefix `Cron` (or
|
||||
`isolation.postToMainPrefix`).
|
||||
- `wakeMode: "now"` triggers an immediate heartbeat after posting the summary.
|
||||
- `payload.deliver: true` sends output to a provider; otherwise it stays internal.
|
||||
|
||||
## Storage & history
|
||||
- Job store: `~/.clawdbot/cron/jobs.json` (JSON, Gateway-managed).
|
||||
- Run history: `~/.clawdbot/cron/runs/<jobId>.jsonl` (JSONL, auto-pruned).
|
||||
- Override store path: `cron.store` in config.
|
||||
|
||||
## Configuration
|
||||
|
||||
```json5
|
||||
{
|
||||
cron: {
|
||||
enabled: true, // default true
|
||||
store: "~/.clawdbot/cron/jobs.json",
|
||||
maxConcurrentRuns: 1 // default 1
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Disable cron entirely:
|
||||
- `cron.enabled: false` (config)
|
||||
- or `CLAWDBOT_SKIP_CRON=1` (env)
|
||||
|
||||
## CLI quickstart
|
||||
|
||||
One-shot reminder (main session, wake immediately):
|
||||
```bash
|
||||
clawdbot cron add \
|
||||
--name "Calendar check" \
|
||||
--at "20m" \
|
||||
--session main \
|
||||
--system-event "Next heartbeat: check calendar." \
|
||||
--wake now
|
||||
```
|
||||
|
||||
Recurring isolated job (deliver to WhatsApp):
|
||||
```bash
|
||||
clawdbot cron add \
|
||||
--name "Morning status" \
|
||||
--cron "0 7 * * *" \
|
||||
--tz "America/Los_Angeles" \
|
||||
--session isolated \
|
||||
--message "Summarize inbox + calendar for today." \
|
||||
--deliver \
|
||||
--provider whatsapp \
|
||||
--to "+15551234567"
|
||||
```
|
||||
|
||||
Manual run (debug):
|
||||
```bash
|
||||
clawdbot cron run <jobId> --force
|
||||
```
|
||||
|
||||
Run history:
|
||||
```bash
|
||||
clawdbot cron runs --id <jobId> --limit 50
|
||||
```
|
||||
|
||||
Immediate wake without creating a job:
|
||||
```bash
|
||||
clawdbot wake --mode now --text "Next heartbeat: check battery."
|
||||
```
|
||||
|
||||
## API surface (Gateway)
|
||||
- `cron.list`, `cron.status`, `cron.add`, `cron.update`, `cron.remove`
|
||||
- `cron.run` (force or due), `cron.runs`
|
||||
- `wake` (enqueue system event + optional heartbeat)
|
||||
|
||||
## Tips
|
||||
- Use **main session jobs** when you want the heartbeat prompt + existing context.
|
||||
- Use **isolated jobs** for noisy, frequent, or long-running work.
|
||||
- Keep messages short; cron turns are full agent runs and can burn tokens.
|
||||
194
docs/automation/gmail-pubsub.md
Normal file
194
docs/automation/gmail-pubsub.md
Normal file
@@ -0,0 +1,194 @@
|
||||
---
|
||||
summary: "Gmail Pub/Sub push wired into Clawdbot webhooks via gogcli"
|
||||
read_when:
|
||||
- Wiring Gmail inbox triggers to Clawdbot
|
||||
- Setting up Pub/Sub push for agent wake
|
||||
---
|
||||
|
||||
# Gmail Pub/Sub -> Clawdbot
|
||||
|
||||
Goal: Gmail watch -> Pub/Sub push -> `gog gmail watch serve` -> Clawdbot webhook.
|
||||
|
||||
## Prereqs
|
||||
|
||||
- `gcloud` installed and logged in ([install guide](https://docs.cloud.google.com/sdk/docs/install-sdk)).
|
||||
- `gog` (gogcli) installed and authorized for the Gmail account ([gogcli.sh](https://gogcli.sh/)).
|
||||
- Clawdbot hooks enabled (see [`docs/webhook.md`](https://docs.clawd.bot/automation/webhook)).
|
||||
- `tailscale` logged in ([tailscale.com](https://tailscale.com/)). Supported setup uses Tailscale Funnel for the public HTTPS endpoint.
|
||||
Other tunnel services can work, but are DIY/unsupported and require manual wiring.
|
||||
Right now, Tailscale is what we support.
|
||||
|
||||
Example hook config (enable Gmail preset mapping):
|
||||
|
||||
```json5
|
||||
{
|
||||
hooks: {
|
||||
enabled: true,
|
||||
token: "CLAWDBOT_HOOK_TOKEN",
|
||||
path: "/hooks",
|
||||
presets: ["gmail"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
To customize payload handling, add `hooks.mappings` or a JS/TS transform module
|
||||
under `hooks.transformsDir` (see [`docs/webhook.md`](https://docs.clawd.bot/automation/webhook)).
|
||||
|
||||
## Wizard (recommended)
|
||||
|
||||
Use the Clawdbot helper to wire everything together (installs deps on macOS via brew):
|
||||
|
||||
```bash
|
||||
clawdbot hooks gmail setup \
|
||||
--account clawdbot@gmail.com
|
||||
```
|
||||
|
||||
Defaults:
|
||||
- Uses Tailscale Funnel for the public push endpoint.
|
||||
- Writes `hooks.gmail` config for `clawdbot hooks gmail run`.
|
||||
- Enables the Gmail hook preset (`hooks.presets: ["gmail"]`).
|
||||
|
||||
Path note: when `tailscale.mode` is enabled, Clawdbot automatically sets
|
||||
`hooks.gmail.serve.path` to `/` and keeps the public path at
|
||||
`hooks.gmail.tailscale.path` (default `/gmail-pubsub`) because Tailscale
|
||||
strips the set-path prefix before proxying.
|
||||
|
||||
Want a custom endpoint? Use `--push-endpoint <url>` or `--tailscale off`.
|
||||
|
||||
Platform note: on macOS the wizard installs `gcloud`, `gogcli`, and `tailscale`
|
||||
via Homebrew; on Linux install them manually first.
|
||||
|
||||
Gateway auto-start (recommended):
|
||||
- When `hooks.enabled=true` and `hooks.gmail.account` is set, the Gateway starts
|
||||
`gog gmail watch serve` on boot and auto-renews the watch.
|
||||
- Set `CLAWDBOT_SKIP_GMAIL_WATCHER=1` to opt out (useful if you run the daemon yourself).
|
||||
- Do not run the manual daemon at the same time, or you will hit
|
||||
`listen tcp 127.0.0.1:8788: bind: address already in use`.
|
||||
|
||||
Manual daemon (starts `gog gmail watch serve` + auto-renew):
|
||||
|
||||
```bash
|
||||
clawdbot hooks gmail run
|
||||
```
|
||||
|
||||
## One-time setup
|
||||
|
||||
1) Select the GCP project **that owns the OAuth client** used by `gog`.
|
||||
|
||||
```bash
|
||||
gcloud auth login
|
||||
gcloud config set project <project-id>
|
||||
```
|
||||
|
||||
Note: Gmail watch requires the Pub/Sub topic to live in the same project as the OAuth client.
|
||||
|
||||
2) Enable APIs:
|
||||
|
||||
```bash
|
||||
gcloud services enable gmail.googleapis.com pubsub.googleapis.com
|
||||
```
|
||||
|
||||
3) Create a topic:
|
||||
|
||||
```bash
|
||||
gcloud pubsub topics create gog-gmail-watch
|
||||
```
|
||||
|
||||
4) Allow Gmail push to publish:
|
||||
|
||||
```bash
|
||||
gcloud pubsub topics add-iam-policy-binding gog-gmail-watch \
|
||||
--member=serviceAccount:gmail-api-push@system.gserviceaccount.com \
|
||||
--role=roles/pubsub.publisher
|
||||
```
|
||||
|
||||
## Start the watch
|
||||
|
||||
```bash
|
||||
gog gmail watch start \
|
||||
--account clawdbot@gmail.com \
|
||||
--label INBOX \
|
||||
--topic projects/<project-id>/topics/gog-gmail-watch
|
||||
```
|
||||
|
||||
Save the `history_id` from the output (for debugging).
|
||||
|
||||
## Run the push handler
|
||||
|
||||
Local example (shared token auth):
|
||||
|
||||
```bash
|
||||
gog gmail watch serve \
|
||||
--account clawdbot@gmail.com \
|
||||
--bind 127.0.0.1 \
|
||||
--port 8788 \
|
||||
--path /gmail-pubsub \
|
||||
--token <shared> \
|
||||
--hook-url http://127.0.0.1:18789/hooks/gmail \
|
||||
--hook-token CLAWDBOT_HOOK_TOKEN \
|
||||
--include-body \
|
||||
--max-bytes 20000
|
||||
```
|
||||
|
||||
Notes:
|
||||
- `--token` protects the push endpoint (`x-gog-token` or `?token=`).
|
||||
- `--hook-url` points to Clawdbot `/hooks/gmail` (mapped; isolated run + summary to main).
|
||||
- `--include-body` and `--max-bytes` control the body snippet sent to Clawdbot.
|
||||
|
||||
Recommended: `clawdbot hooks gmail run` wraps the same flow and auto-renews the watch.
|
||||
|
||||
## Expose the handler (dev, unsupported hack)
|
||||
|
||||
If you insist on a non-Tailscale tunnel, wire it manually and use the public URL in the push
|
||||
subscription (unsupported, no guardrails):
|
||||
|
||||
```bash
|
||||
cloudflared tunnel --url http://127.0.0.1:8788 --no-autoupdate
|
||||
```
|
||||
|
||||
Use the generated URL as the push endpoint:
|
||||
|
||||
```bash
|
||||
gcloud pubsub subscriptions create gog-gmail-watch-push \
|
||||
--topic gog-gmail-watch \
|
||||
--push-endpoint "https://<public-url>/gmail-pubsub?token=<shared>"
|
||||
```
|
||||
|
||||
Production: use a stable HTTPS endpoint and configure Pub/Sub OIDC JWT, then run:
|
||||
|
||||
```bash
|
||||
gog gmail watch serve --verify-oidc --oidc-email <svc@...>
|
||||
```
|
||||
|
||||
## Test
|
||||
|
||||
Send a message to the watched inbox:
|
||||
|
||||
```bash
|
||||
gog gmail send \
|
||||
--account clawdbot@gmail.com \
|
||||
--to clawdbot@gmail.com \
|
||||
--subject "watch test" \
|
||||
--body "ping"
|
||||
```
|
||||
|
||||
Check watch state and history:
|
||||
|
||||
```bash
|
||||
gog gmail watch status --account clawdbot@gmail.com
|
||||
gog gmail history --account clawdbot@gmail.com --since <historyId>
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- `Invalid topicName`: project mismatch (topic not in the OAuth client project).
|
||||
- `User not authorized`: missing `roles/pubsub.publisher` on the topic.
|
||||
- Empty messages: Gmail push only provides `historyId`; fetch via `gog gmail history`.
|
||||
|
||||
## Cleanup
|
||||
|
||||
```bash
|
||||
gog gmail watch stop --account clawdbot@gmail.com
|
||||
gcloud pubsub subscriptions delete gog-gmail-watch-push
|
||||
gcloud pubsub topics delete gog-gmail-watch
|
||||
```
|
||||
52
docs/automation/poll.md
Normal file
52
docs/automation/poll.md
Normal file
@@ -0,0 +1,52 @@
|
||||
---
|
||||
summary: "Poll sending via gateway + CLI"
|
||||
read_when:
|
||||
- Adding or modifying poll support
|
||||
- Debugging poll sends from the CLI or gateway
|
||||
---
|
||||
# Polls
|
||||
|
||||
Updated: 2026-01-06
|
||||
|
||||
## Supported providers
|
||||
- WhatsApp (web provider)
|
||||
- Discord
|
||||
|
||||
## CLI
|
||||
|
||||
```bash
|
||||
# WhatsApp
|
||||
clawdbot poll --to +15555550123 -q "Lunch today?" -o "Yes" -o "No" -o "Maybe"
|
||||
clawdbot poll --to 123456789@g.us -q "Meeting time?" -o "10am" -o "2pm" -o "4pm" -s 2
|
||||
|
||||
# Discord
|
||||
clawdbot poll --to channel:123456789 -q "Snack?" -o "Pizza" -o "Sushi" --provider discord
|
||||
clawdbot poll --to channel:123456789 -q "Plan?" -o "A" -o "B" --provider discord --duration-hours 48
|
||||
```
|
||||
|
||||
Options:
|
||||
- `--provider`: `whatsapp` (default) or `discord`
|
||||
- `--max-selections`: how many choices a voter can select (default: 1)
|
||||
- `--duration-hours`: Discord-only (defaults to 24 when omitted)
|
||||
|
||||
## Gateway RPC
|
||||
|
||||
Method: `poll`
|
||||
|
||||
Params:
|
||||
- `to` (string, required)
|
||||
- `question` (string, required)
|
||||
- `options` (string[], required)
|
||||
- `maxSelections` (number, optional)
|
||||
- `durationHours` (number, optional)
|
||||
- `provider` (string, optional, default: `whatsapp`)
|
||||
- `idempotencyKey` (string, required)
|
||||
|
||||
## Provider differences
|
||||
- WhatsApp: 2-12 options, `maxSelections` must be within option count, ignores `durationHours`.
|
||||
- Discord: 2-10 options, `durationHours` clamped to 1-768 hours (default 24). `maxSelections > 1` enables multi-select; Discord does not support a strict selection count.
|
||||
|
||||
## Agent tool (Discord)
|
||||
The Discord tool action `poll` still uses `question`, `answers`, optional `allowMultiselect`, `durationHours`, and `content`. The gateway/CLI poll model maps `allowMultiselect` to `maxSelections > 1`.
|
||||
|
||||
Note: Discord has no “pick exactly N” mode; `maxSelections` is treated as a boolean (`> 1` = multiselect).
|
||||
133
docs/automation/webhook.md
Normal file
133
docs/automation/webhook.md
Normal file
@@ -0,0 +1,133 @@
|
||||
---
|
||||
summary: "Webhook ingress for wake and isolated agent runs"
|
||||
read_when:
|
||||
- Adding or changing webhook endpoints
|
||||
- Wiring external systems into Clawdbot
|
||||
---
|
||||
|
||||
# Webhooks
|
||||
|
||||
Gateway can expose a small HTTP webhook endpoint for external triggers.
|
||||
|
||||
## Enable
|
||||
|
||||
```json5
|
||||
{
|
||||
hooks: {
|
||||
enabled: true,
|
||||
token: "shared-secret",
|
||||
path: "/hooks"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Notes:
|
||||
- `hooks.token` is required when `hooks.enabled=true`.
|
||||
- `hooks.path` defaults to `/hooks`.
|
||||
|
||||
## Auth
|
||||
|
||||
Every request must include the hook token:
|
||||
- `Authorization: Bearer <token>`
|
||||
- or `x-clawdbot-token: <token>`
|
||||
- or `?token=<token>`
|
||||
|
||||
## Endpoints
|
||||
|
||||
### `POST /hooks/wake`
|
||||
|
||||
Payload:
|
||||
```json
|
||||
{ "text": "System line", "mode": "now" }
|
||||
```
|
||||
|
||||
- `text` required (string)
|
||||
- `mode` optional: `now` | `next-heartbeat` (default `now`)
|
||||
|
||||
Effect:
|
||||
- Enqueues a system event for the **main** session
|
||||
- If `mode=now`, triggers an immediate heartbeat
|
||||
|
||||
### `POST /hooks/agent`
|
||||
|
||||
Payload:
|
||||
```json
|
||||
{
|
||||
"message": "Run this",
|
||||
"name": "Email",
|
||||
"sessionKey": "hook:email:msg-123",
|
||||
"wakeMode": "now",
|
||||
"deliver": false,
|
||||
"provider": "last",
|
||||
"to": "+15551234567",
|
||||
"thinking": "low",
|
||||
"timeoutSeconds": 120
|
||||
}
|
||||
```
|
||||
|
||||
- `message` required (string)
|
||||
- `name` optional (used in the summary prefix)
|
||||
- `sessionKey` optional (default random `hook:<uuid>`)
|
||||
- `wakeMode` optional: `now` | `next-heartbeat` (default `now`)
|
||||
- `deliver` optional (default `false`)
|
||||
- `provider` optional: `last` | `whatsapp` | `telegram`
|
||||
- `to` optional (provider-specific target)
|
||||
- `thinking` optional (override)
|
||||
- `timeoutSeconds` optional
|
||||
|
||||
Effect:
|
||||
- Runs an **isolated** agent turn (own session key)
|
||||
- Always posts a summary into the **main** session
|
||||
- If `wakeMode=now`, triggers an immediate heartbeat
|
||||
|
||||
### `POST /hooks/<name>` (mapped)
|
||||
|
||||
Custom hook names are resolved via `hooks.mappings` (see configuration). A mapping can
|
||||
turn arbitrary payloads into `wake` or `agent` actions, with optional templates or
|
||||
code transforms.
|
||||
|
||||
Mapping options (summary):
|
||||
- `hooks.presets: ["gmail"]` enables the built-in Gmail mapping.
|
||||
- `hooks.mappings` lets you define `match`, `action`, and templates in config.
|
||||
- `hooks.transformsDir` + `transform.module` loads a JS/TS module for custom logic.
|
||||
- Use `match.source` to keep a generic ingest endpoint (payload-driven routing).
|
||||
- TS transforms require a TS loader (e.g. `bun`) or precompiled `.js` at runtime.
|
||||
- `clawdbot hooks gmail setup` writes `hooks.gmail` config for `clawdbot hooks gmail run`.
|
||||
See [`docs/gmail-pubsub.md`](https://docs.clawd.bot/automation/gmail-pubsub) for the full Gmail watch flow.
|
||||
|
||||
## Responses
|
||||
|
||||
- `200` for `/hooks/wake`
|
||||
- `202` for `/hooks/agent` (async run started)
|
||||
- `401` on auth failure
|
||||
- `400` on invalid payload
|
||||
- `413` on oversized payloads
|
||||
|
||||
## Examples
|
||||
|
||||
```bash
|
||||
curl -X POST http://127.0.0.1:18789/hooks/wake \
|
||||
-H 'Authorization: Bearer SECRET' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"text":"New email received","mode":"now"}'
|
||||
```
|
||||
|
||||
```bash
|
||||
curl -X POST http://127.0.0.1:18789/hooks/agent \
|
||||
-H 'x-clawdbot-token: SECRET' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"message":"Summarize inbox","name":"Email","wakeMode":"next-heartbeat"}'
|
||||
```
|
||||
|
||||
```bash
|
||||
curl -X POST http://127.0.0.1:18789/hooks/gmail \
|
||||
-H 'Authorization: Bearer SECRET' \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"source":"gmail","messages":[{"from":"Ada","subject":"Hello","snippet":"Hi"}]}'
|
||||
```
|
||||
|
||||
## Security
|
||||
|
||||
- Keep hook endpoints behind loopback, tailnet, or trusted reverse proxy.
|
||||
- Use a dedicated hook token; do not reuse gateway auth tokens.
|
||||
- Avoid including sensitive raw payloads in webhook logs.
|
||||
Reference in New Issue
Block a user