Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions skills/extract-structured-data/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,18 @@ Use the dedicated `extract` tool, not `scrape`. It returns structured JSON direc
2. Choose the extraction mode:
- `mode='auto'` (default): Zenrows decides the structure. Best when the user has not named exact fields.
- `mode='css'` with `css_extractor`: named fields you can select, for example `css_extractor='{"title":"h1","price":".price"}'`. Required whenever `mode='css'`.
- `mode='autoparse'`: known page shapes (products, articles, listings) where the built-in parser applies.
- `mode='autoparse'`: deprecated. Use it only when the user asks for autoparse by name.
3. Add page-loading options only when the page needs them: `js_render=true` for SPAs, `mode_auto=true` or `premium_proxy=true` when a plain request does not load the page, `proxy_country` for geo-restricted content. `wait_for` and `wait` both require `js_render`.
4. Return the JSON formatted for the user's purpose: display, file export, or code.

## Two traps

- **`mode` selects the extraction strategy** (`auto`, `autoparse`, `css`). It is not the adaptive request setting, which is the separate boolean `mode_auto` and is on by default. `scrape` has no `mode` parameter at all.
- **`extract` with `mode='auto'` works on any domain Zenrows has prepared, with any API key.** There is no per-account gate. On a domain that isn't prepared yet the call returns `REQS007` (or the older `AUTH010`). `fallback_autoparse` defaults to `true`, which retries once with autoparse. Leave it on unless the user wants the failure surfaced. A domain can be prepared on request: see https://docs.zenrows.com/extract/setup#domain-coverage.
- **`extract` with `mode='auto'` works on any domain Zenrows has prepared, with any API key.** There is no per-account gate. When it can't extract a page it returns `REQS007` (or the older `AUTH010`) with a `reason`, and nothing is charged:
- `domain_not_prepared`: call `prepare_domain` with the domain and 1 to 5 example URLs, one per kind of page the user wants (for example one product page and one listing page). It costs no credits and takes a few minutes. Poll `prepared_domain_status` about every 30 seconds until `status` is `ready`, then call `extract` again. Each new domain uses one slot of the account's monthly allowance, which `prepared_domain_status` shows. If the user can't wait, use `mode='css'` meanwhile.
- `page_not_recognised`: the domain is prepared, but this page has nothing Extract extracts (a home or search page, for example). Try a product, article or listing page on the same site. If this page should work, retry it once after a few minutes, not in a loop; otherwise use `mode='css'`. Don't call `prepare_domain`: it doesn't change this answer.
- `domain_not_enabled`: preparing doesn't help. Use `mode='css'`, or tell the user to contact Zenrows support to have the site enabled.
- **`fallback_autoparse` is deprecated and has no effect.** Extract no longer falls back to autoparse. Don't set it; if you do, the result only carries a note saying it was ignored.

## When to use `scrape` instead

Expand All @@ -34,7 +38,7 @@ Use the dedicated `extract` tool, not `scrape`. It returns structured JSON direc
|------|-----------|---------|
| Let Zenrows infer structure | `mode` | `mode='auto'` (default) |
| Named fields via CSS | `mode` + `css_extractor` | `mode='css'`, `css_extractor='{"title":"h1"}'` |
| Known page shape | `mode` | `mode='autoparse'` |
| Domain not prepared yet (`REQS007`, `domain_not_prepared`) | `prepare_domain`, then `prepared_domain_status` | see "Two traps" |
| JS-rendered page | `js_render` | `js_render=true` |
| Plain request does not load the page | `mode_auto` or `premium_proxy` | `mode_auto=true` |
| Geo-restricted | `proxy_country` | `proxy_country='de'` (needs `premium_proxy` or `mode_auto`) |
6 changes: 4 additions & 2 deletions skills/using-zenrows/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,9 @@ Key parameters: `url`, `response_type`, `proxy_country`, `css_extractor`, `autop

### `extract`

Structured extraction from one page. Returns JSON. Parameters: `url`, `mode` (`auto`, `autoparse` or `css`), `css_extractor` (required when `mode='css'`), `js_render`, `premium_proxy`, `mode_auto`, `proxy_country`, `wait_for`, `wait`, `fallback_autoparse`.
Structured extraction from one page. Returns JSON. Parameters: `url`, `mode` (`auto` by default, or `css`; `autoparse` is deprecated), `css_extractor` (required when `mode='css'`), `include_html`, `js_render`, `premium_proxy`, `mode_auto`, `proxy_country`, `wait_for`, `wait`.

`prepare_domain` and `prepared_domain_status` prepare a domain for `extract` when it answers `REQS007` with reason `domain_not_prepared` (see the `extract-structured-data` skill).

On `extract`, `mode` selects the extraction strategy. `scrape` has no `mode` parameter.

Expand Down Expand Up @@ -79,7 +81,7 @@ Reads the connected account's plan and usage. It changes nothing.
- `AUTH004` (the account's credit allowance is spent) or `AUTH014` (this API key reached its credit cap): a plan limit, not a sign-in problem. Reconnecting doesn't clear it. Tell the user; they can check their usage and plan in the Zenrows dashboard, or raise the key's cap.
- `AUTH006`: too many requests are running at once. Wait for running requests to finish, then retry.
- `413`: the response is too large. Narrow it with `css_extractor`.
- `REQS007` (or the older `AUTH010`) from `extract`: Extract hasn't prepared that domain yet. `fallback_autoparse` (on by default) retries with autoparse. Surface the error only if the user asked for strict extraction.
- `REQS007` (or the older `AUTH010`) from `extract`: read its `reason`. `domain_not_prepared`: prepare the domain with `prepare_domain` and wait for `prepared_domain_status` to say `ready`. `page_not_recognised`: try a product, article or listing page, or `mode='css'`; don't prepare the domain again. `domain_not_enabled`: use `mode='css'`. Not charged in any case. Details in the `extract-structured-data` skill.
- Look up any other error code at https://docs.zenrows.com.

## Concurrency
Expand Down