docs: document testing kit

docs/testing.md

@@ -0,0 +1,91 @@
+---
+summary: "Testing kit: unit/e2e/live suites, Docker runners, and what each test covers"
+read_when:
+  - Running tests locally or in CI
+  - Adding regressions for model/provider bugs
+  - Debugging gateway + agent behavior
+---
+
+# Testing
+
+Clawdbot has three Vitest suites (unit, e2e, live) plus a couple Docker helpers for “run with my real keys” smoke checks.
+
+## Quick start
+
+- Full gate (what we expect before push): `pnpm lint && pnpm build && pnpm test`
+- Coverage gate: `pnpm test:coverage`
+- E2E suite: `pnpm test:e2e`
+- Live suite (opt-in): `LIVE=1 pnpm test:live`
+
+## Test suites (what runs where)
+
+### Unit / integration (default)
+
+- Command: `pnpm test`
+- Config: `vitest.config.ts`
+- Files: `src/**/*.test.ts`
+- Scope: pure unit tests + in-process integration tests (gateway server auth, routing, tooling, parsing, config).
+
+### E2E (gateway smoke)
+
+- Command: `pnpm test:e2e`
+- Config: `vitest.e2e.config.ts`
+- Files: `src/**/*.e2e.test.ts`
+- Scope: multi-instance gateway end-to-end behavior (WebSocket/HTTP/node pairing), heavier networking surface.
+
+### Live (real providers + real models)
+
+- Command: `pnpm test:live`
+- Config: `vitest.live.config.ts`
+- Files: `src/**/*.live.test.ts`
+- Default: **skipped** unless `LIVE=1` (or `CLAWDBOT_LIVE_TEST=1`)
+- Scope: “does this provider/model actually work today with real creds”.
+
+## Live: model smoke (profile keys)
+
+Two layers:
+
+1. Direct model completion (no gateway):
+   - Test: `src/agents/models.profiles.live.test.ts`
+   - Goal: enumerate discovered models, use `getApiKeyForModel` to pick ones you have creds for, then run a small completion.
+   - Selection:
+     - `CLAWDBOT_LIVE_ALL_MODELS=1` (required to run the suite)
+     - `CLAWDBOT_LIVE_MODELS=all` or comma allowlist (`openai/gpt-5.2,anthropic/claude-opus-4-5,...`)
+     - `CLAWDBOT_LIVE_REQUIRE_PROFILE_KEYS=1` to ensure creds come from the profile store (not ad-hoc env).
+   - Regression hook: OpenAI Responses tool-only → follow-up path (the `reasoning` replay class) is covered here.
+
+2. Gateway + dev agent smoke (what “@clawdbot” actually does):
+   - Test: `src/gateway/gateway-models.profiles.live.test.ts`
+   - Goal: spin up an in-process gateway, create/patch a `agent:dev:*` session, iterate models-with-keys, and assert “meaningful” responses.
+   - Selection:
+     - `CLAWDBOT_LIVE_GATEWAY=1`
+     - `CLAWDBOT_LIVE_GATEWAY_ALL_MODELS=1` (scan all discovered models with available keys)
+     - `CLAWDBOT_LIVE_GATEWAY_MODELS=all` or comma allowlist
+   - Extra regression: for OpenAI Responses/Codex Responses models, force a tool-call-only turn followed by a user question (the exact failure mode that produced `400 … reasoning … required following item`).
+
+## Credentials (never commit)
+
+Live tests discover credentials the same way the CLI does:
+
+- Profile store: `~/.clawdbot/credentials/` (preferred; what “profile keys” means in the tests)
+- Config: `~/.clawdbot/clawdbot.json` (or `CLAWDBOT_CONFIG_PATH`)
+
+If you want to rely on env keys (e.g. exported in your `~/.profile`), run local tests after `source ~/.profile`, or use the Docker runners below (they can mount `~/.profile` into the container).
+
+## Docker runners (optional “works in Linux” checks)
+
+These run `pnpm test:live` inside the repo Docker image, mounting your local config dir and workspace:
+
+- Direct models: `scripts/test-live-models-docker.sh`
+- Gateway + dev agent: `scripts/test-live-gateway-models-docker.sh`
+
+Useful env vars:
+
+- `CLAWDBOT_CONFIG_DIR=...` (default: `~/.clawdbot`) mounted to `/home/node/.clawdbot`
+- `CLAWDBOT_WORKSPACE_DIR=...` (default: `~/clawd`) mounted to `/home/node/clawd`
+- `CLAWDBOT_LIVE_GATEWAY_MODELS=...` / `CLAWDBOT_LIVE_MODELS=...` to narrow the run
+
+## Docs sanity
+
+Run docs checks after doc edits: `pnpm docs:list`.
+