Files
clawdbot/docs/concepts/models.md
2026-01-08 22:57:18 +01:00

8.3 KiB
Raw Blame History

summary, read_when
summary read_when
Plan for models CLI: scan, list, aliases, fallbacks, status
Adding or modifying models CLI (models list/set/scan/aliases/fallbacks)
Changing model fallback behavior or selection UX
Updating model scan probes (tools/images)

Models CLI plan

See docs/model-failover.md for how auth profiles rotate (OAuth vs API keys), cooldowns, and how that interacts with model fallbacks.

Goal: give clear model visibility + control (configured vs available), plus scan tooling that prefers tool-call + image-capable models and maintains ordered fallbacks.

How Clawdbot models work (quick explainer)

Clawdbot selects models in this order:

  1. The configured primary model (agent.model.primary).
  2. If it fails, fallbacks in agent.model.fallbacks (in order).
  3. Auth failover happens inside the provider first (see /concepts/model-failover).

Key pieces:

  • provider/model is the canonical model id (e.g. anthropic/claude-opus-4-5).
  • agent.models is the allowlist/catalog of models Clawdbot can use, with optional aliases and provider params.
  • agent.imageModel is only used when the primary model cant accept images.
  • models.providers lets you add custom providers + models (written to models.json).
  • /model <id> switches the active model for the current session; /model list shows whats allowed.

Related:

  • Context limits are model-specific; long sessions may trigger compaction. See /concepts/compaction.

Model recommendations

Suggested stacks:

  • Assistant-first: Opus 4.5 primary → Sonnet 4.5 fallback.
  • Agentic coding: Opus 4.5 primary → GPT-5.2-Codex for sub-agents → Sonnet 4.5 fallback.

Model discussions (community notes)

Anecdotal notes from the Discord thread on January 45, 2026. Treat as “reported by users,” not a benchmark.

Reported working well

  • Claude Opus 4.5: best overall quality in Clawdbot, especially for “assistant” work. Tradeoff is cost and hitting usage limits quickly.
  • Claude Sonnet 4.5: common fallback when Opus caps out. Similar behavior with fewer limit headaches.
  • Gemini 3 Pro: some users felt it maps well to Clawdbots structure. Vibe was “fits the framework” more than “best at everything.”
  • GLM: used successfully as a worker model under orchestration. Seen as strong for delegated/secondary tasks, not the primary brain.
  • MiniMax M2.1: “good enough” for grunt work or a cheap fallback. Community nickname was “Temu-Sonnet,” i.e. usable but not Sonnet-level polish.

Mixed / unclear

  • Antigravity (Claude Opus access): some reported extra Opus quota. Pricing/limits were unclear, so the value is hard to predict.

Reported weak in Clawdbot

  • GPT-5.2-Codex inside Clawdbot: reported as rough for conversation/assistant tasks when embedded. Same notes said Codex felt stronger via the Codex CLI than embedded use.
  • Grok: people tried it and then abandoned it. No strong upside showed up in the notes.

Theme

  • Token burn feels higher than expected in long sessions; people suspect context buildup + tool outputs. Pruning/compaction helps. Check session logs before blaming providers. See /concepts/session and /concepts/model-failover.

Want a tailored stack? Share whether youre using Clawdbot or Clawdis and your main workload (agentic coding vs “assistant” work), and we can suggest a primary + fallback set based on these reports.

Models CLI

See /cli for the full command tree and CLI flags.

CLI output (list + status)

clawdbot models list (default) prints a table with these columns:

  • Model: provider/model key (truncated in TTY).
  • Input: text or text+image.
  • Ctx: context window in K tokens (from the model registry).
  • Local: yes/no when the provider base URL is local.
  • Auth: yes/no when the provider has usable auth.
  • Tags: origin + role hints.

Common tags:

  • default — resolved default model.
  • fallback#Nagent.model.fallbacks order.
  • imageagent.imageModel.primary.
  • img-fallback#Nagent.imageModel.fallbacks order.
  • configured — present in agent.models.
  • alias:<name> — alias from agent.models.*.alias.
  • missing — referenced in config but not found in the registry.

Output formats:

  • --plain: prints only provider/model keys (one per line).
  • --json: { count, models: [{ key, name, input, contextWindow, local, available, tags, missing }] }.

clawdbot models status prints the resolved defaults, fallbacks, image model, aliases, and an Auth overview section showing which providers have profiles/env/models.json keys. --plain prints the resolved default model only; --json returns a structured object for tooling.

Config changes

  • agent.models (configured model catalog + aliases).
  • agent.models.*.params (provider-specific API params passed through to requests).
  • agent.model.primary + agent.model.fallbacks.
  • agent.imageModel.primary + agent.imageModel.fallbacks (optional).
  • auth.profiles + auth.order for per-provider auth failover.

Scan behavior (models scan)

Input

  • OpenRouter /models list (filter :free)
  • Requires OpenRouter API key from auth profiles or OPENROUTER_API_KEY (see /environment)
  • Optional filters: --max-age-days, --min-params, --provider, --max-candidates
  • Probe controls: --timeout, --concurrency

Probes (direct pi-ai complete)

  • Tool-call probe (required):
    • Provide a dummy tool, verify tool call emitted.
  • Image probe (preferred):
    • Prompt includes 1x1 PNG; success if no "unsupported image" error.

Scoring/selection

  • Prefer models passing tool + image for text/tool fallbacks.
  • Prefer image-only models for image tool fallback (even if tool probe fails).
  • Rank by: image ok, then lower tool latency, then larger context, then params.

Interactive selection (TTY)

  • Multiselect list with per-model stats:
    • model id, tool ok, image ok, median latency, context, inferred params.
  • Pre-select top N (default 6).
  • Non-TTY: auto-select; require --yes/--no-input to apply.

Output

  • Writes agent.model.fallbacks ordered.
  • Writes agent.imageModel.fallbacks ordered (image-capable models).
  • Ensures agent.models entries exist for selected models.
  • Optional --set-default to set agent.model.primary.
  • Optional --set-image to set agent.imageModel.primary.

Runtime fallback

  • On model failure: try agent.model.fallbacks in order.
  • Per-provider auth failover uses auth.order (or stored profile order) before moving to the next model.
  • Image routing uses agent.imageModel only when configured and the primary model lacks image input.
  • Persist last successful provider/model to session entry; auth profile success is global.
  • See docs/model-failover.md for auth profile rotation, cooldowns, and timeout handling.

Tests

  • Unit: scan selection ordering + probe classification.
  • CLI: list/aliases/fallbacks add/remove + scan writes config.
  • Status: shows last used model + fallbacks.

Docs

  • Update docs/configuration.md with agent.models + agent.model + agent.imageModel.
  • Keep this doc current when CLI surface or scan logic changes.
  • Note provider aliases like z.ai/* -> zai/* when relevant.
  • Provider ids in model refs are normalized to lowercase.