fix: harden web fetch SSRF and redirects
Co-authored-by: Eli <fogboots@users.noreply.github.com>
This commit is contained in:
@@ -1774,6 +1774,7 @@ Note: `applyPatch` is only under `tools.exec`.
|
||||
- `tools.web.fetch.maxChars` (default 50000)
|
||||
- `tools.web.fetch.timeoutSeconds` (default 30)
|
||||
- `tools.web.fetch.cacheTtlMinutes` (default 15)
|
||||
- `tools.web.fetch.maxRedirects` (default 3)
|
||||
- `tools.web.fetch.userAgent` (optional override)
|
||||
- `tools.web.fetch.readability` (default true; disable to use basic HTML cleanup only)
|
||||
- `tools.web.fetch.firecrawl.enabled` (default true when an API key is set)
|
||||
|
||||
@@ -215,6 +215,7 @@ Fetch a URL and extract readable content.
|
||||
maxChars: 50000,
|
||||
timeoutSeconds: 30,
|
||||
cacheTtlMinutes: 15,
|
||||
maxRedirects: 3,
|
||||
userAgent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 14_7_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36",
|
||||
readability: true,
|
||||
firecrawl: {
|
||||
@@ -241,6 +242,7 @@ Notes:
|
||||
- `web_fetch` uses Readability (main-content extraction) first, then Firecrawl (if configured). If both fail, the tool returns an error.
|
||||
- Firecrawl requests use bot-circumvention mode and cache results by default.
|
||||
- `web_fetch` sends a Chrome-like User-Agent and `Accept-Language` by default; override `userAgent` if needed.
|
||||
- `web_fetch` blocks private/internal hostnames and re-checks redirects (limit with `maxRedirects`).
|
||||
- `web_fetch` is best-effort extraction; some sites will need the browser tool.
|
||||
- See [Firecrawl](/tools/firecrawl) for key setup and service details.
|
||||
- Responses are cached (default 15 minutes) to reduce repeated fetches.
|
||||
|
||||
Reference in New Issue
Block a user