let5see/clawdbot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add Chrome extension browser relay

Browse Source
This commit is contained in:
Peter Steinberger
2026-01-15 04:50:11 +00:00
parent 5fdaef3646
commit ef78b198cb
40 changed files with 2467 additions and 49 deletions
Download Patch File Download Diff File Expand all files Collapse all files

130
docs/tools/browser.md
View File

@@ -106,6 +106,95 @@ Example:
Use `profiles.<name>.cdpUrl` for **remote CDP** if you want the Gateway to talk
directly to a Chrome instance without a remote control server.
### Running the control server on the browser machine
Run a standalone browser control server (recommended when your Gateway is remote):
```bash
# on the machine that runs Chrome
clawdbot browser serve --bind <browser-host> --port 18791 --token <token>
```
Then point your Gateway at it:
```json5
{
browser: {
enabled: true,
controlUrl: "http://<browser-host>:18791",
// Option A (recommended): keep token in env on the Gateway
// (avoid writing secrets into config files)
// controlToken: "<token>"
}
}
```
And set the auth token in the Gateway environment:
```bash
export CLAWDBOT_BROWSER_CONTROL_TOKEN="<token>"
```
Option B: store the token in the Gateway config instead (same shared token):
```json5
{
browser: {
enabled: true,
controlUrl: "http://<browser-host>:18791",
controlToken: "<token>"
}
}
```
## Security
This section covers the **browser control server** (`browser.controlUrl`) used for agent browser automation.
Key ideas:
- Treat the browser control server like an admin API: **private network only**.
- Use **token auth** always when the server is reachable off-machine.
- Prefer **Tailnet-only** connectivity over LAN exposure.
### Tokens (what is shared with what?)
- `browser.controlToken` / `CLAWDBOT_BROWSER_CONTROL_TOKEN` is **only** for authenticating browser control HTTP requests to `browser.controlUrl`.
- It is **not** the Gateway token (`gateway.auth.token`) and **not** a node pairing token.
- You *can* reuse the same string value, but it’s better to keep them separate to reduce blast radius.
### Binding (don’t expose to your LAN by accident)
Recommended:
- Keep `clawdbot browser serve` bound to loopback (`127.0.0.1`) and publish it via Tailscale.
- Or bind to a Tailnet IP only (never `0.0.0.0`) and require a token.
Avoid:
- `--bind 0.0.0.0` (LAN-visible). Even with token auth, traffic is plain HTTP unless you also add TLS.
### TLS / HTTPS (recommended approach: terminate in front)
Best practice here: keep `clawdbot browser serve` on HTTP and terminate TLS in front.
If you’re already using Tailscale, you have two good options:
1) **Tailnet-only, still HTTP** (transport is encrypted by Tailscale):
- Keep `controlUrl` as `http://…` but ensure it’s only reachable over your tailnet.
2) **Serve HTTPS via Tailscale** (nice UX: `https://…` URL):
```bash
# on the browser machine
clawdbot browser serve --bind 127.0.0.1 --port 18791 --token <token>
tailscale serve https / http://127.0.0.1:18791
```
Then set your Gateway config `browser.controlUrl` to the HTTPS URL (MagicDNS/ts.net) and keep using the same token.
Notes:
- Do **not** use Tailscale Funnel for this unless you explicitly want to make the endpoint public.
- For Tailnet setup/background, see [Gateway web surfaces](/web/index) and the [Gateway CLI](/cli/gateway).
## Profiles (multi-browser)
Clawdbot supports multiple named profiles. Each profile has its own:
@@ -120,6 +209,44 @@ Defaults:
All control endpoints accept `?profile=<name>`; the CLI uses `--browser-profile`.
## Chrome extension relay (use your existing Chrome)
Clawdbot can also drive **your existing Chrome tabs** (no separate “clawd” Chrome instance) via a local CDP relay + a Chrome extension.
Full guide: [Chrome extension](/tools/chrome-extension)
Flow:
- You run a **browser control server** (Gateway on the same machine, or `clawdbot browser serve`).
- A local **relay server** listens at a loopback `cdpUrl` (default: `http://127.0.0.1:18792`).
- You click the **Clawdbot Browser Relay** extension icon on a tab to attach.
- The agent controls that tab via the normal `browser` tool, by selecting the right profile.
### Setup
1) Create a profile that uses the extension driver:
```bash
clawdbot browser create-profile \
--name chrome \
--driver extension \
--cdp-url http://127.0.0.1:18792 \
--color "#00AA00"
```
2) Load the extension (dev/unpacked):
- Chrome → `chrome://extensions` → enable “Developer mode”
- `clawdbot browser extension install`
- “Load unpacked” → select the directory printed by `clawdbot browser extension path`
- Pin the extension, then click it on the tab you want to control (badge shows `ON`).
3) Use it:
- CLI: `clawdbot browser --browser-profile chrome tabs`
- Agent tool: `browser` with `profile="chrome"`
Notes:
- This mode relies on Playwright-on-CDP for most operations (screenshots/snapshots/actions).
- Detach by clicking the extension icon again.
## Isolation guarantees
- **Dedicated user data dir**: never touches your personal Chrome profile.
@@ -164,7 +291,8 @@ All endpoints accept `?profile=<name>`.
Some features (navigate/act/AI snapshot/role snapshot, element screenshots, PDF) require
Playwright. If Playwright isn’t installed, those endpoints return a clear 501
error. ARIA snapshots and basic screenshots still work.
error. ARIA snapshots and basic screenshots still work for clawd-managed Chrome.
For the Chrome extension relay driver, ARIA snapshots and screenshots require Playwright.
## How it works (internal)

121
docs/tools/chrome-extension.md Normal file
View File

@@ -0,0 +1,121 @@
---
summary: "Chrome extension: let Clawdbot drive your existing Chrome tab"
read_when:
- You want the agent to drive an existing Chrome tab (toolbar button)
- You need remote Gateway + local browser automation via Tailscale
- You want to understand the security implications of browser takeover
---
# Chrome extension (browser relay)
The Clawdbot Chrome extension lets the agent control your **existing Chrome tabs** (your normal Chrome window) instead of launching a separate clawd-managed Chrome profile.
Attach/detach happens via a **single Chrome toolbar button**.
## What it is (concept)
There are three parts:
- **Browser control server** (HTTP): the API the agent/tool calls (`browser.controlUrl`)
- **Local relay server** (loopback CDP): bridges between the control server and the extension (`http://127.0.0.1:18792` by default)
- **Chrome MV3 extension**: attaches to the active tab using `chrome.debugger` and pipes CDP messages to the relay
Clawdbot then controls the attached tab through the normal `browser` tool surface (selecting the right profile).
## Install / load (unpacked)
1) Install the extension to a stable local path:
```bash
clawdbot browser extension install
```
2) Print the installed extension directory path:
```bash
clawdbot browser extension path
```
3) Chrome → `chrome://extensions`
- Enable “Developer mode”
- “Load unpacked” → select the directory printed above
4) Pin the extension.
## Updates (no build step)
The extension ships inside the Clawdbot release (npm package) as static files. There is no separate “build” step.
After upgrading Clawdbot:
- Re-run `clawdbot browser extension install` to refresh the installed files under your Clawdbot state directory.
- Chrome → `chrome://extensions` → click “Reload” on the extension.
## Create a browser profile for the extension
```bash
clawdbot browser create-profile \
--name chrome \
--driver extension \
--cdp-url http://127.0.0.1:18792 \
--color "#00AA00"
```
Then target it:
- CLI: `clawdbot browser --browser-profile chrome tabs`
- Agent tool: `browser` with `profile="chrome"`
## Attach / detach (toolbar button)
- Open the tab you want Clawdbot to control.
- Click the extension icon.
- Badge shows `ON` when attached.
- Click again to detach.
## Remote Gateway (recommended: Tailscale Serve)
Goal: Gateway runs on one machine, but Chrome runs somewhere else.
On the **browser machine**:
```bash
clawdbot browser serve --bind 127.0.0.1 --port 18791 --token <token>
tailscale serve https / http://127.0.0.1:18791
```
On the **Gateway machine**:
- Set `browser.controlUrl` to the HTTPS Serve URL (MagicDNS/ts.net).
- Provide the token (prefer env):
```bash
export CLAWDBOT_BROWSER_CONTROL_TOKEN="<token>"
```
Then the agent can drive the browser by calling the remote `browser.controlUrl` API, while the extension + relay stay local on the browser machine.
## How “extension path” works
`clawdbot browser extension path` prints the **installed** on-disk directory containing the extension files.
The CLI intentionally does **not** print a `node_modules` path. Always run `clawdbot browser extension install` first to copy the extension to a stable location under your Clawdbot state directory.
If you move or delete that install directory, Chrome will mark the extension as broken until you reload it from a valid path.
## Security implications (read this)
This is powerful and risky. Treat it like giving the model “hands on your browser”.
- The extension uses Chrome’s debugger API (`chrome.debugger`). When attached, the model can:
- click/type/navigate in that tab
- read page content
- access whatever the tab’s logged-in session can access
- **This is not isolated** like the dedicated clawd-managed profile.
- If you attach to your daily-driver profile/tab, you’re granting access to that account state.
Recommendations:
- Prefer a dedicated Chrome profile (separate from your personal browsing) for extension relay usage.
- Keep the browser control server tailnet-only (Tailscale) and require a token.
- Avoid exposing browser control over LAN (`0.0.0.0`) and avoid Funnel (public).
Related:
- Browser tool overview: [Browser](/tools/browser)
- Security audit: [Security](/gateway/security)
- Tailscale setup: [Tailscale](/gateway/tailscale)