diff --git a/skills/extract-structured-data/SKILL.md b/skills/extract-structured-data/SKILL.md index c5c2e59..e7d92b5 100644 --- a/skills/extract-structured-data/SKILL.md +++ b/skills/extract-structured-data/SKILL.md @@ -15,14 +15,18 @@ Use the dedicated `extract` tool, not `scrape`. It returns structured JSON direc 2. Choose the extraction mode: - `mode='auto'` (default): Zenrows decides the structure. Best when the user has not named exact fields. - `mode='css'` with `css_extractor`: named fields you can select, for example `css_extractor='{"title":"h1","price":".price"}'`. Required whenever `mode='css'`. - - `mode='autoparse'`: known page shapes (products, articles, listings) where the built-in parser applies. + - `mode='autoparse'`: deprecated. Use it only when the user asks for autoparse by name. 3. Add page-loading options only when the page needs them: `js_render=true` for SPAs, `mode_auto=true` or `premium_proxy=true` when a plain request does not load the page, `proxy_country` for geo-restricted content. `wait_for` and `wait` both require `js_render`. 4. Return the JSON formatted for the user's purpose: display, file export, or code. ## Two traps - **`mode` selects the extraction strategy** (`auto`, `autoparse`, `css`). It is not the adaptive request setting, which is the separate boolean `mode_auto` and is on by default. `scrape` has no `mode` parameter at all. -- **`extract` with `mode='auto'` works on any domain Zenrows has prepared, with any API key.** There is no per-account gate. On a domain that isn't prepared yet the call returns `REQS007` (or the older `AUTH010`). `fallback_autoparse` defaults to `true`, which retries once with autoparse. Leave it on unless the user wants the failure surfaced. A domain can be prepared on request: see https://docs.zenrows.com/extract/setup#domain-coverage. +- **`extract` with `mode='auto'` works on any domain Zenrows has prepared, with any API key.** There is no per-account gate. When it can't extract a page it returns `REQS007` (or the older `AUTH010`) with a `reason`, and nothing is charged: + - `domain_not_prepared`: call `prepare_domain` with the domain and 1 to 5 example URLs, one per kind of page the user wants (for example one product page and one listing page). It costs no credits and takes a few minutes. Poll `prepared_domain_status` about every 30 seconds until `status` is `ready`, then call `extract` again. Each new domain uses one slot of the account's monthly allowance, which `prepared_domain_status` shows. If the user can't wait, use `mode='css'` meanwhile. + - `page_not_recognised`: the domain is prepared, but this page has nothing Extract extracts (a home or search page, for example). Try a product, article or listing page on the same site. If this page should work, retry it once after a few minutes, not in a loop; otherwise use `mode='css'`. Don't call `prepare_domain`: it doesn't change this answer. + - `domain_not_enabled`: preparing doesn't help. Use `mode='css'`, or tell the user to contact Zenrows support to have the site enabled. +- **`fallback_autoparse` is deprecated and has no effect.** Extract no longer falls back to autoparse. Don't set it; if you do, the result only carries a note saying it was ignored. ## When to use `scrape` instead @@ -34,7 +38,7 @@ Use the dedicated `extract` tool, not `scrape`. It returns structured JSON direc |------|-----------|---------| | Let Zenrows infer structure | `mode` | `mode='auto'` (default) | | Named fields via CSS | `mode` + `css_extractor` | `mode='css'`, `css_extractor='{"title":"h1"}'` | -| Known page shape | `mode` | `mode='autoparse'` | +| Domain not prepared yet (`REQS007`, `domain_not_prepared`) | `prepare_domain`, then `prepared_domain_status` | see "Two traps" | | JS-rendered page | `js_render` | `js_render=true` | | Plain request does not load the page | `mode_auto` or `premium_proxy` | `mode_auto=true` | | Geo-restricted | `proxy_country` | `proxy_country='de'` (needs `premium_proxy` or `mode_auto`) | diff --git a/skills/using-zenrows/SKILL.md b/skills/using-zenrows/SKILL.md index 2a644fa..2dc889c 100644 --- a/skills/using-zenrows/SKILL.md +++ b/skills/using-zenrows/SKILL.md @@ -23,7 +23,9 @@ Key parameters: `url`, `response_type`, `proxy_country`, `css_extractor`, `autop ### `extract` -Structured extraction from one page. Returns JSON. Parameters: `url`, `mode` (`auto`, `autoparse` or `css`), `css_extractor` (required when `mode='css'`), `js_render`, `premium_proxy`, `mode_auto`, `proxy_country`, `wait_for`, `wait`, `fallback_autoparse`. +Structured extraction from one page. Returns JSON. Parameters: `url`, `mode` (`auto` by default, or `css`; `autoparse` is deprecated), `css_extractor` (required when `mode='css'`), `include_html`, `js_render`, `premium_proxy`, `mode_auto`, `proxy_country`, `wait_for`, `wait`. + +`prepare_domain` and `prepared_domain_status` prepare a domain for `extract` when it answers `REQS007` with reason `domain_not_prepared` (see the `extract-structured-data` skill). On `extract`, `mode` selects the extraction strategy. `scrape` has no `mode` parameter. @@ -79,7 +81,7 @@ Reads the connected account's plan and usage. It changes nothing. - `AUTH004` (the account's credit allowance is spent) or `AUTH014` (this API key reached its credit cap): a plan limit, not a sign-in problem. Reconnecting doesn't clear it. Tell the user; they can check their usage and plan in the Zenrows dashboard, or raise the key's cap. - `AUTH006`: too many requests are running at once. Wait for running requests to finish, then retry. - `413`: the response is too large. Narrow it with `css_extractor`. -- `REQS007` (or the older `AUTH010`) from `extract`: Extract hasn't prepared that domain yet. `fallback_autoparse` (on by default) retries with autoparse. Surface the error only if the user asked for strict extraction. +- `REQS007` (or the older `AUTH010`) from `extract`: read its `reason`. `domain_not_prepared`: prepare the domain with `prepare_domain` and wait for `prepared_domain_status` to say `ready`. `page_not_recognised`: try a product, article or listing page, or `mode='css'`; don't prepare the domain again. `domain_not_enabled`: use `mode='css'`. Not charged in any case. Details in the `extract-structured-data` skill. - Look up any other error code at https://docs.zenrows.com. ## Concurrency