refactor(browser): remove model-backed page extraction (#120101)

* refactor(browser): remove model-backed extraction

* docs(browser): remove stale extract reference
This commit is contained in:
Peter Steinberger
2026-08-06 18:32:14 -07:00
committed by GitHub
parent 53006a289e
commit 01281ad9f8
34 changed files with 21 additions and 2162 deletions
+2 -32
View File
@@ -23,7 +23,7 @@ agent tools, but nothing listens on the loopback control port.
- Status/start/stop: `GET /`, `GET /doctor`, `POST /start`, `POST /stop`, `POST /reset-profile`
- Profiles: `GET /profiles`, `POST /profiles/create`, `DELETE /profiles/:name`
- Tabs: `GET /tabs`, `POST /tabs/open`, `POST /tabs/focus`, `DELETE /tabs/:targetId`, `POST /tabs/action`
- Snapshot/screenshot/extract: `GET /snapshot`, `POST /screenshot`, `POST /extract`
- Snapshot/screenshot: `GET /snapshot`, `POST /screenshot`
- Actions: `POST /navigate`, `POST /act`
- Hooks: `POST /hooks/file-chooser`, `POST /hooks/dialog`
- Downloads: `POST /download`, `POST /wait/download`
@@ -49,27 +49,6 @@ For tab endpoints, `targetId` is the compatibility field name. Prefer passing
handles such as `t1` are also accepted. Raw CDP target ids and unique raw
target-id prefixes still work, but they are volatile diagnostic handles.
### Page extraction
The agent tool accepts `action="extract"` with required `query` and optional
`targetId`, `timeoutMs`, `selector`, `ignoreSelectors`, and `schema`. `selector`
is a CSS selector that limits capture to matching subtrees; a no-match response
is an error and never falls back to the whole page. `ignoreSelectors` is an
array of CSS selectors removed from the captured subtree before readable text
conversion, so navigation, footers, ads, and banners do not consume the model
context window. The reported `chars` count reflects the scoped, converted text.
`schema` is a JSON Schema object for structured extraction. A successful result
stores the validated value in `details.json` and shows compact JSON in the
wrapped text block. Invalid JSON or a schema mismatch gets one correction retry;
if that also fails, retry without `schema` or adjust the schema. Without
`schema`, extraction keeps its free-text answer and `NOT_FOUND` behavior.
The CLI mirrors these fields with `--selector <css>`, repeatable
`--ignore-selector <css>`, and `--schema <json>`. The private `POST /extract`
capture route accepts `targetId`, `timeoutMs`, `selector`, and
`ignoreSelectors`; schema validation happens in the calling agent tool or CLI.
If shared-secret gateway auth is configured, browser HTTP routes require auth too:
- `Authorization: Bearer <gateway token>`
@@ -105,7 +84,7 @@ Other runtime failures may still return `{ "error": "<message>" }` without a
### Playwright requirement
Some features (navigate/act/AI snapshot/role snapshot, extract, element
Some features (navigate/act/AI snapshot/role snapshot, element
screenshots, PDF) require Playwright. If Playwright isn't installed, those endpoints return
a clear 501 error.
@@ -128,7 +107,6 @@ What still needs Playwright:
- AI snapshots that depend on Playwright's native AI snapshot format
- CSS-selector element screenshots (`--element`)
- full browser PDF export
- page-question extraction
Element screenshots also reject `--full-page`; the route returns `fullPage is
not supported for element screenshots`.
@@ -214,8 +192,6 @@ openclaw browser snapshot --urls
openclaw browser snapshot --selector "#main" --interactive
openclaw browser snapshot --frame "iframe#main" --interactive
openclaw browser snapshot --out snapshot.txt
openclaw browser extract "What is the page's main conclusion?"
openclaw browser extract "List the releases" --selector "main" --ignore-selector "nav" --schema '{"type":"array","items":{"type":"object"}}'
openclaw browser console --level error
openclaw browser errors --clear
openclaw browser requests --filter api --clear
@@ -283,12 +259,6 @@ openclaw browser set device "iPhone 14"
Notes:
- Use `browser extract "<question>"` or agent-tool `action="extract"` when you
need an answer from the current page but do not need interaction refs. It
sanitizes readable page content, caps it at 80,000 characters, runs one
model call, and returns only the wrapped answer. The overall timeout defaults
to 60 seconds and is clamped to 5120 seconds. If extraction fails, fall back
to `snapshot`; existing-session profiles do not support extraction.
- The agent-facing `browser` tool exposes `action=download` (required `ref` and
`path`) and `action=waitfordownload` (optional `path`). Both return the saved
download URL, suggested filename, and guarded local path. Explicit download
+4 -10
View File
@@ -102,16 +102,10 @@ Plugin-bundled skills are listed in the agent's available skills when the
plugin is enabled. The full skill instructions load on demand, so routine
turns do not pay the full token cost.
For “read this page and answer X,” use browser `action="extract"` with a
`query`. It sends sanitized, bounded readable text through one model call and
returns only the answer; keep `snapshot` for choosing actions and obtaining
refs. Extraction requires a Playwright-backed profile and falls back to a
snapshot workflow when it cannot complete.
On large pages, pass `selector` to capture only the relevant CSS subtree and
`ignoreSelectors` to remove repeated chrome before conversion. Pass a JSON
`schema` when the caller needs validated machine-usable fields in
`details.json`; without it, extraction remains a free-text answer.
For page text, use a selector-scoped snapshot or `act:evaluate` that returns
only the relevant text or structured data, then let the active agent model
reason over that bounded result. Use efficient snapshots for controls and
action discovery; they intentionally omit most non-interactive prose.
## Missing browser command or tool