# Fast AI data extraction from web pages | Spider

Describe the data you need and get it back as structured JSON. Spider fetches the page, runs the extraction and returns the result in one request.

Canonical: https://spider.cloud/ai/

## Describe the data. Get it fast.

AI extraction for the live web. Say what you need in plain language. Spider fetches the page, runs the extraction and hands back structured data, all in one request.

[Start free trial ](/ai/pricing/)[API reference](/docs/ai-studio/)

Add-on to your Spider account. From $6 a month, 7-day trial on Starter. Requests are metered against your credits.

[See the example](#example)

prompt "List every product with its price as a number and whether it is in stock."

source spider.cloud/examples/catalog/

| Item             | Price | Stock        |
| ---------------- | ----- | ------------ |
| Field notebook   | $18   | In stock     |
| Ruling pen       | $42   | In stock     |
| Cold-press paper | $27   | Out of stock |
| Sepia ink        | $12   | In stock     |

extracted_data 200

name Field notebook

price 18

in_stock true

name Ruling pen

price 42

in_stock true

name Cold-press paper

price 27

in_stock false

name Sepia ink

price 12

in_stock true

## One request. Structured data back.

Send a URL and a prompt. The page comes back as an array of page objects, and the data you asked for sits on `metadata.extracted_data`. Add an `extraction_schema` when the shape has to hold.

prompt "Return the headline, the publish date as YYYY-MM-DD, and a one sentence summary."

source spider.cloud/blog/spider-browser-stealth-benchmark/

Spider Browser stealth benchmark

March 22, 2026

Eighty tasks across the sites that block automation most, run against seven hosted browsers.

Spider completed 85 of 100 scored steps. The next best host completed 68.

metadata.extracted_data 200

headline Spider Browser stealth benchmark

published 2026-03-22

summary Eighty tasks on the sites that block automation most; Spider completed 85 scored steps, the next host 68.

Stored result from this request. Hover or focus a field to see where it came from.

```
curl -X POST https://api.spider.cloud/ai/scrape \
  -H "Authorization: Bearer $SPIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://spider.cloud/blog/spider-browser-stealth-benchmark/",
    "prompt": "Return the headline, the publish date as YYYY-MM-DD, and a one sentence summary.",
    "cleaning_intent": "extraction",
    "extraction_schema": {
      "name": "article",
      "description": "The article metadata.",
      "schema": "{\"type\":\"object\",\"properties\":{\"headline\":{\"type\":\"string\"},\"published\":{\"type\":\"string\",\"description\":\"YYYY-MM-DD\"},\"summary\":{\"type\":\"string\"}},\"required\":[\"headline\",\"published\",\"summary\"]}"
    },
    "return_format": "markdown",
    "metadata": true
  }'
```

```
import os

import requests

payload = {
  "url": "https://spider.cloud/blog/spider-browser-stealth-benchmark/",
  "prompt": "Return the headline, the publish date as YYYY-MM-DD, and a one sentence summary.",
  "cleaning_intent": "extraction",
  "extraction_schema": {
    "name": "article",
    "description": "The article metadata.",
    "schema": "{\"type\":\"object\",\"properties\":{\"headline\":{\"type\":\"string\"},\"published\":{\"type\":\"string\",\"description\":\"YYYY-MM-DD\"},\"summary\":{\"type\":\"string\"}},\"required\":[\"headline\",\"published\",\"summary\"]}"
  },
  "return_format": "markdown",
  "metadata": True
}

response = requests.post(
    "https://api.spider.cloud/ai/scrape",
    headers={
        "Authorization": f"Bearer {os.environ['SPIDER_API_KEY']}",
        "Content-Type": "application/json",
    },
    json=payload,
    timeout=120,
)

print(response.json())
```

```
import https from "node:https";

const payload = JSON.stringify({
  "url": "https://spider.cloud/blog/spider-browser-stealth-benchmark/",
  "prompt": "Return the headline, the publish date as YYYY-MM-DD, and a one sentence summary.",
  "cleaning_intent": "extraction",
  "extraction_schema": {
    "name": "article",
    "description": "The article metadata.",
    "schema": "{\"type\":\"object\",\"properties\":{\"headline\":{\"type\":\"string\"},\"published\":{\"type\":\"string\",\"description\":\"YYYY-MM-DD\"},\"summary\":{\"type\":\"string\"}},\"required\":[\"headline\",\"published\",\"summary\"]}"
  },
  "return_format": "markdown",
  "metadata": true
});

const options = {
  hostname: "api.spider.cloud",
  port: 443,
  path: "/ai/scrape",
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.SPIDER_API_KEY}`,
    "Content-Type": "application/json",
    "Content-Length": Buffer.byteLength(payload),
  },
};

const req = https.request(options, (res) => {
  let raw = "";
  res.on("data", (chunk) => {
    raw += chunk;
  });
  res.on("end", () => {
    console.log(JSON.parse(raw));
  });
});

req.on("error", (error) => {
  console.error(error);
});

req.write(payload);
req.end();
```

Set `SPIDER_API_KEY` to your key. The account needs an active AI Studio plan; without one the call returns 402.

response body · example

```
[
  {
    "url": "https://spider.cloud/blog/spider-browser-stealth-benchmark/",
    "error": null,
    "status": 200,
    "content": "# Spider Browser stealth benchmark\n\nEighty tasks across the sites that block automation most.",
    "metadata": {
      "title": "Spider Browser stealth benchmark",
      "extracted_data": {
        "headline": "Spider Browser stealth benchmark",
        "published": "2026-03-22",
        "summary": "Eighty tasks on the sites that block automation most; Spider completed 85 scored steps, the next host 68."
      }
    },
    "costs": null,
    "duration_elasped_ms": …
  }
]
```

One object per page. `costs` is null when a request bills nothing and a twelve-key breakdown otherwise; the timing is left out here because a stored example has none.

prompt "Keep only the links that go to API reference pages."

source spider.cloud/docs/

Quickstart

API reference

AI API reference

Guides

Blog

metadata.extracted_data 200

url https\://spider.cloud/docs/api/

url https\://spider.cloud/docs/ai-studio/

Stored result from this request. Hover or focus a field to see where it came from.

```
curl -X POST https://api.spider.cloud/ai/links \
  -H "Authorization: Bearer $SPIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://spider.cloud/docs/",
    "prompt": "Keep only the links that go to API reference pages.",
    "cleaning_intent": "general",
    "limit": 50,
    "return_format": "raw"
  }'
```

```
import os

import requests

payload = {
  "url": "https://spider.cloud/docs/",
  "prompt": "Keep only the links that go to API reference pages.",
  "cleaning_intent": "general",
  "limit": 50,
  "return_format": "raw"
}

response = requests.post(
    "https://api.spider.cloud/ai/links",
    headers={
        "Authorization": f"Bearer {os.environ['SPIDER_API_KEY']}",
        "Content-Type": "application/json",
    },
    json=payload,
    timeout=120,
)

print(response.json())
```

```
import https from "node:https";

const payload = JSON.stringify({
  "url": "https://spider.cloud/docs/",
  "prompt": "Keep only the links that go to API reference pages.",
  "cleaning_intent": "general",
  "limit": 50,
  "return_format": "raw"
});

const options = {
  hostname: "api.spider.cloud",
  port: 443,
  path: "/ai/links",
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.SPIDER_API_KEY}`,
    "Content-Type": "application/json",
    "Content-Length": Buffer.byteLength(payload),
  },
};

const req = https.request(options, (res) => {
  let raw = "";
  res.on("data", (chunk) => {
    raw += chunk;
  });
  res.on("end", () => {
    console.log(JSON.parse(raw));
  });
});

req.on("error", (error) => {
  console.error(error);
});

req.write(payload);
req.end();
```

Set `SPIDER_API_KEY` to your key. The account needs an active AI Studio plan; without one the call returns 402.

response body · example

```
[
  {
    "url": "https://spider.cloud/docs/",
    "error": null,
    "status": 200,
    "links": [
      "https://spider.cloud/docs/quickstart/",
      "https://spider.cloud/docs/api/",
      "https://spider.cloud/docs/ai-studio/",
      "https://spider.cloud/guides/",
      "https://spider.cloud/blog/"
    ],
    "content": "# Docs\n\nQuickstart, API reference, guides.",
    "metadata": {
      "title": "Spider docs",
      "extracted_data": {
        "links": [
          {
            "label": "API reference",
            "url": "https://spider.cloud/docs/api/"
          },
          {
            "label": "AI API reference",
            "url": "https://spider.cloud/docs/ai-studio/"
          }
        ]
      }
    },
    "costs": null,
    "duration_elasped_ms": …
  }
]
```

One object per page. `costs` is null when a request bills nothing and a twelve-key breakdown otherwise; the timing is left out here because a stored example has none.

prompt "List every product with its price as a number and whether it is in stock."

source spider.cloud/examples/catalog/

| Item             | Price | Stock        |
| ---------------- | ----- | ------------ |
| Field notebook   | $18   | In stock     |
| Ruling pen       | $42   | In stock     |
| Cold-press paper | $27   | Out of stock |
| Sepia ink        | $12   | In stock     |

metadata.extracted_data 200

name Field notebook

price 18

in_stock true

name Ruling pen

price 42

in_stock true

name Cold-press paper

price 27

in_stock false

name Sepia ink

price 12

in_stock true

Stored result from this request. Hover or focus a field to see where it came from.

```
curl -X POST https://api.spider.cloud/ai/scrape \
  -H "Authorization: Bearer $SPIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://spider.cloud/examples/catalog/",
    "prompt": "List every product with its price as a number and whether it is in stock.",
    "cleaning_intent": "extraction",
    "extraction_schema": {
      "name": "catalog",
      "description": "Every product on the page.",
      "schema": "{\"type\":\"object\",\"properties\":{\"products\":{\"type\":\"array\",\"items\":{\"type\":\"object\",\"properties\":{\"name\":{\"type\":\"string\"},\"price\":{\"type\":\"number\"},\"in_stock\":{\"type\":\"boolean\"}},\"required\":[\"name\",\"price\",\"in_stock\"]}}},\"required\":[\"products\"]}"
    },
    "return_format": "markdown",
    "metadata": true
  }'
```

```
import os

import requests

payload = {
  "url": "https://spider.cloud/examples/catalog/",
  "prompt": "List every product with its price as a number and whether it is in stock.",
  "cleaning_intent": "extraction",
  "extraction_schema": {
    "name": "catalog",
    "description": "Every product on the page.",
    "schema": "{\"type\":\"object\",\"properties\":{\"products\":{\"type\":\"array\",\"items\":{\"type\":\"object\",\"properties\":{\"name\":{\"type\":\"string\"},\"price\":{\"type\":\"number\"},\"in_stock\":{\"type\":\"boolean\"}},\"required\":[\"name\",\"price\",\"in_stock\"]}}},\"required\":[\"products\"]}"
  },
  "return_format": "markdown",
  "metadata": True
}

response = requests.post(
    "https://api.spider.cloud/ai/scrape",
    headers={
        "Authorization": f"Bearer {os.environ['SPIDER_API_KEY']}",
        "Content-Type": "application/json",
    },
    json=payload,
    timeout=120,
)

print(response.json())
```

```
import https from "node:https";

const payload = JSON.stringify({
  "url": "https://spider.cloud/examples/catalog/",
  "prompt": "List every product with its price as a number and whether it is in stock.",
  "cleaning_intent": "extraction",
  "extraction_schema": {
    "name": "catalog",
    "description": "Every product on the page.",
    "schema": "{\"type\":\"object\",\"properties\":{\"products\":{\"type\":\"array\",\"items\":{\"type\":\"object\",\"properties\":{\"name\":{\"type\":\"string\"},\"price\":{\"type\":\"number\"},\"in_stock\":{\"type\":\"boolean\"}},\"required\":[\"name\",\"price\",\"in_stock\"]}}},\"required\":[\"products\"]}"
  },
  "return_format": "markdown",
  "metadata": true
});

const options = {
  hostname: "api.spider.cloud",
  port: 443,
  path: "/ai/scrape",
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.SPIDER_API_KEY}`,
    "Content-Type": "application/json",
    "Content-Length": Buffer.byteLength(payload),
  },
};

const req = https.request(options, (res) => {
  let raw = "";
  res.on("data", (chunk) => {
    raw += chunk;
  });
  res.on("end", () => {
    console.log(JSON.parse(raw));
  });
});

req.on("error", (error) => {
  console.error(error);
});

req.write(payload);
req.end();
```

Set `SPIDER_API_KEY` to your key. The account needs an active AI Studio plan; without one the call returns 402.

response body · example

```
[
  {
    "url": "https://spider.cloud/examples/catalog/",
    "error": null,
    "status": 200,
    "content": "# Catalog\n\n| Item | Price | Stock |\n| --- | --- | --- |\n| Field notebook | $18 | In stock |",
    "metadata": {
      "title": "Catalog",
      "extracted_data": {
        "products": [
          {
            "name": "Field notebook",
            "price": 18,
            "in_stock": true
          },
          {
            "name": "Ruling pen",
            "price": 42,
            "in_stock": true
          },
          {
            "name": "Cold-press paper",
            "price": 27,
            "in_stock": false
          },
          {
            "name": "Sepia ink",
            "price": 12,
            "in_stock": true
          }
        ]
      }
    },
    "costs": null,
    "duration_elasped_ms": …
  }
]
```

One object per page. `costs` is null when a request bills nothing and a twelve-key breakdown otherwise; the timing is left out here because a stored example has none.

[Every response field ](/docs/ai-studio/#ai-response)[The catalog page](/examples/catalog/)

## Timed on one page.

One page and one prompt, sent 30 times in a row from a laptop, timed from the request leaving to the whole body arriving. The conditions are below and the script is in the repo, so you can run it on your own page.

10.23 s

median, request to result

15.17 s

p95, request to result

7.46 s

median server time per page

100%

requests with a page and no error

conditions

- Endpoint POST /ai/scrape
- Page spider.cloud/about/
- Prompt List each contact on the page with its label and email address.
- Sample 30 requests, one at a time, 3 warm-ups discarded, 1200 ms apart
- Client time fetch() start to the full JSON body parsed
- Server time the duration_elasped_ms field on the page object
- From a laptop on a residential connection outside AWS
- Success 100% of requests returned a page with status 200 and no error; 100% carried every schema field
- Date 2026-09-14

reading it

A single run. Time on a page depends on its size and how much of it the model has to read; a heavy page takes longer. Your plan's rate limit caps how many requests you can send a second and says nothing about how long one takes.

Client time includes the trip to and from the laptop the run was made on. Server time is the part Spider spent on the page.

[The script](https://github.com/spider-rs/spider-cloud-frontend/blob/main/scripts/bench/ai-scrape-latency.mjs)

## Five ways to ask.

Same headers, same key, same page object back. Pick the route by what you want done; each one takes every parameter its standard counterpart takes, plus the prompt.

[Full reference](/docs/ai-studio/)

[AI Crawl `POST /ai/crawl` Collect the same fields from every page of a site. "Collect every pricing plan with its monthly cost and included credits." ](/ai/crawl/)[AI Scrape `POST /ai/scrape` Pull fields from one page. "Pull the Pro plan name, its monthly cost, and whether annual billing is selected." ](/ai/scrape/)[AI Search `POST /ai/search` Search the web for a question and read the results. "Find recent benchmarks comparing Rust web crawlers on throughput." ](/ai/search/)[AI Browser `POST /ai/browser` Click, fill and wait in a real browser, then extract. "Switch the billing toggle to annual, then read back each plan name with its price." ](/ai/browser/)[AI Links `POST /ai/links` Keep only the links that match what you are after. "Find every documentation link under the API reference."](/ai/links/)

## Pick a rate limit.

AI Studio is an add-on to a Spider account. The subscription unlocks the five routes and sets how many requests a second you can send. The work each request does is metered against your credits, the same as the rest of the API.

[Compare plans](/ai/pricing/)

Starter 7-day trial

$ 6 /mo

1 requests per second

Lite or $300 a year

$ 30 /mo

5 requests per second

Standard or $1,250 a year

$ 125 /mo

10 requests per second

Scale or $6,000 a year

$ 600 /mo

25 requests per second

- Rate limit Requests per second on the AI routes. This is what the plan sets.
- Usage Each page is metered in the costs block against your pay as you go credits. The model cost passes through at vendor rate.
- Trial Seven days on Starter, monthly billing. Then $6 a month.
- Annual Two months free on any plan.
- Enterprise From $2,000 a month with custom limits, through sales.

[Start free trial ](/ai/pricing/)[Contact sales](mailto:sales@spider.cloud?subject=AI%20Studio%20Enterprise)

## Questions.

When do I use an AI route instead of the standard one?

When you want fields, not a document. /scrape returns the page as markdown or HTML for you to parse. /ai/scrape takes the same parameters plus a prompt and returns the fields you named on metadata.extracted_data. If you already have a parser that works, the standard route is cheaper; if the markup changes under you, the prompt does not.

How do I control the shape of the result?

Pass an extraction_schema with a name and a JSON schema. The result follows it, so a price comes back as a number and a missing field comes back as null rather than a guess. Without a schema the model picks the shape from the prompt.

What decides how long a request takes?

Fetching and rendering the page, cleaning it down to what the model needs to read, and the model call itself. A long page takes longer than a short one. Your plan sets a rate limit in requests per second, which caps how many you can send at once and has no effect on how long one takes.

What happens when the page does not have what I asked for?

The field is null and the page object still comes back with status 200. Each page carries its own status and error; a crawl can return 200 for the request while one page inside it failed, so read error on each object rather than the HTTP status alone.

How do I turn it on?

Subscribe on the pricing page. The AI routes then work with the API key you already have. Until then they answer 402 with code ai_subscription_required.

What do I pay?

The subscription, from $6 a month, sets your rate limit. Each request is then metered against your Spider credits and the breakdown is on the costs field of every page object. The model cost is the vendor's rate with nothing added.

## Start with one request.

Subscribe, keep your key, send a URL and a prompt.

[Start free trial ](/ai/pricing/)[API reference](/docs/ai-studio/)
