AI Studio · add-on

Describe the data. Get it fast.

AI extraction for the live web. Say what you need in plain language. Spider fetches the page, runs the extraction and hands back structured data, all in one request.

Start free trial API reference

Add-on to your Spider account. From $6 a month, 7-day trial on Starter. Requests are metered against your credits.

See the example

example POST /ai/scrape

prompt "List every product with its price as a number and whether it is in stock."

source spider.cloud/examples/catalog/

Item	Price	Stock
Field notebook	$18	In stock
Ruling pen	$42	In stock
Cold-press paper	$27	Out of stock
Sepia ink	$12	In stock

extracted_data 200

name Field notebook

price 18

in_stock true

name Ruling pen

price 42

in_stock true

name Cold-press paper

price 27

in_stock false

name Sepia ink

price 12

in_stock true

One request. Structured data back.

Send a URL and a prompt. The page comes back as an array of page objects, and the data you asked for sits on metadata.extracted_data. Add an extraction_schema when the shape has to hold.

example POST /ai/scrape

prompt "Return the headline, the author, the publish date as YYYY-MM-DD, and a one sentence summary."

source spider.cloud/blog/spider-browser-stealth-benchmark/

Spider Browser stealth benchmark

Jeff Mendez

March 22, 2026

Eighty tasks across the sites that block automation most, run against seven hosted browsers.

Spider completed 85 of 100 scored steps. The next best host completed 68.

metadata.extracted_data 200

headline Spider Browser stealth benchmark

author Jeff Mendez

published 2026-03-22

summary Eighty tasks on the sites that block automation most; Spider completed 85 scored steps, the next host 68.

Stored result from this request. Hover or focus a field to see where it came from.

curl -X POST https://api.spider.cloud/ai/scrape \
  -H "Authorization: Bearer $SPIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://spider.cloud/blog/spider-browser-stealth-benchmark/",
    "prompt": "Return the headline, the author, the publish date as YYYY-MM-DD, and a one sentence summary.",
    "cleaning_intent": "extraction",
    "extraction_schema": {
      "name": "article",
      "description": "The article metadata.",
      "schema": "{\"type\":\"object\",\"properties\":{\"headline\":{\"type\":\"string\"},\"author\":{\"type\":\"string\"},\"published\":{\"type\":\"string\",\"description\":\"YYYY-MM-DD\"},\"summary\":{\"type\":\"string\"}},\"required\":[\"headline\",\"author\",\"published\",\"summary\"]}"
    },
    "return_format": "markdown",
    "metadata": true
  }'

import os

import requests

payload = {
  "url": "https://spider.cloud/blog/spider-browser-stealth-benchmark/",
  "prompt": "Return the headline, the author, the publish date as YYYY-MM-DD, and a one sentence summary.",
  "cleaning_intent": "extraction",
  "extraction_schema": {
    "name": "article",
    "description": "The article metadata.",
    "schema": "{\"type\":\"object\",\"properties\":{\"headline\":{\"type\":\"string\"},\"author\":{\"type\":\"string\"},\"published\":{\"type\":\"string\",\"description\":\"YYYY-MM-DD\"},\"summary\":{\"type\":\"string\"}},\"required\":[\"headline\",\"author\",\"published\",\"summary\"]}"
  },
  "return_format": "markdown",
  "metadata": True
}

response = requests.post(
    "https://api.spider.cloud/ai/scrape",
    headers={
        "Authorization": f"Bearer {os.environ['SPIDER_API_KEY']}",
        "Content-Type": "application/json",
    },
    json=payload,
    timeout=120,
)

print(response.json())

import https from "node:https";

const payload = JSON.stringify({
  "url": "https://spider.cloud/blog/spider-browser-stealth-benchmark/",
  "prompt": "Return the headline, the author, the publish date as YYYY-MM-DD, and a one sentence summary.",
  "cleaning_intent": "extraction",
  "extraction_schema": {
    "name": "article",
    "description": "The article metadata.",
    "schema": "{\"type\":\"object\",\"properties\":{\"headline\":{\"type\":\"string\"},\"author\":{\"type\":\"string\"},\"published\":{\"type\":\"string\",\"description\":\"YYYY-MM-DD\"},\"summary\":{\"type\":\"string\"}},\"required\":[\"headline\",\"author\",\"published\",\"summary\"]}"
  },
  "return_format": "markdown",
  "metadata": true
});

const options = {
  hostname: "api.spider.cloud",
  port: 443,
  path: "/ai/scrape",
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.SPIDER_API_KEY}`,
    "Content-Type": "application/json",
    "Content-Length": Buffer.byteLength(payload),
  },
};

const req = https.request(options, (res) => {
  let raw = "";
  res.on("data", (chunk) => {
    raw += chunk;
  });
  res.on("end", () => {
    console.log(JSON.parse(raw));
  });
});

req.on("error", (error) => {
  console.error(error);
});

req.write(payload);
req.end();

Set SPIDER_API_KEY to your key. The account needs an active AI Studio plan; without one the call returns 402.

response body · example

[
  {
    "url": "https://spider.cloud/blog/spider-browser-stealth-benchmark/",
    "error": null,
    "status": 200,
    "content": "# Spider Browser stealth benchmark\n\nEighty tasks across the sites that block automation most.",
    "metadata": {
      "title": "Spider Browser stealth benchmark",
      "extracted_data": {
        "headline": "Spider Browser stealth benchmark",
        "author": "Jeff Mendez",
        "published": "2026-03-22",
        "summary": "Eighty tasks on the sites that block automation most; Spider completed 85 scored steps, the next host 68."
      }
    },
    "costs": null,
    "duration_elasped_ms": …
  }
]

One object per page. costs is null when a request bills nothing and a twelve-key breakdown otherwise; the timing is left out here because a stored example has none.

prompt "Keep only the links that go to API reference pages."

source spider.cloud/docs/

Quickstart

API reference

AI API reference

Guides

Blog

metadata.extracted_data 200

url https://spider.cloud/docs/api/

url https://spider.cloud/docs/ai-studio/

Stored result from this request. Hover or focus a field to see where it came from.

curl -X POST https://api.spider.cloud/ai/links \
  -H "Authorization: Bearer $SPIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://spider.cloud/docs/",
    "prompt": "Keep only the links that go to API reference pages.",
    "cleaning_intent": "general",
    "limit": 50,
    "return_format": "raw"
  }'

import os

import requests

payload = {
  "url": "https://spider.cloud/docs/",
  "prompt": "Keep only the links that go to API reference pages.",
  "cleaning_intent": "general",
  "limit": 50,
  "return_format": "raw"
}

response = requests.post(
    "https://api.spider.cloud/ai/links",
    headers={
        "Authorization": f"Bearer {os.environ['SPIDER_API_KEY']}",
        "Content-Type": "application/json",
    },
    json=payload,
    timeout=120,
)

print(response.json())

import https from "node:https";

const payload = JSON.stringify({
  "url": "https://spider.cloud/docs/",
  "prompt": "Keep only the links that go to API reference pages.",
  "cleaning_intent": "general",
  "limit": 50,
  "return_format": "raw"
});

const options = {
  hostname: "api.spider.cloud",
  port: 443,
  path: "/ai/links",
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.SPIDER_API_KEY}`,
    "Content-Type": "application/json",
    "Content-Length": Buffer.byteLength(payload),
  },
};

const req = https.request(options, (res) => {
  let raw = "";
  res.on("data", (chunk) => {
    raw += chunk;
  });
  res.on("end", () => {
    console.log(JSON.parse(raw));
  });
});

req.on("error", (error) => {
  console.error(error);
});

req.write(payload);
req.end();

Set SPIDER_API_KEY to your key. The account needs an active AI Studio plan; without one the call returns 402.

response body · example

[
  {
    "url": "https://spider.cloud/docs/",
    "error": null,
    "status": 200,
    "links": [
      "https://spider.cloud/docs/quickstart/",
      "https://spider.cloud/docs/api/",
      "https://spider.cloud/docs/ai-studio/",
      "https://spider.cloud/guides/",
      "https://spider.cloud/blog/"
    ],
    "content": "# Docs\n\nQuickstart, API reference, guides.",
    "metadata": {
      "title": "Spider docs",
      "extracted_data": {
        "links": [
          {
            "label": "API reference",
            "url": "https://spider.cloud/docs/api/"
          },
          {
            "label": "AI API reference",
            "url": "https://spider.cloud/docs/ai-studio/"
          }
        ]
      }
    },
    "costs": null,
    "duration_elasped_ms": …
  }
]

One object per page. costs is null when a request bills nothing and a twelve-key breakdown otherwise; the timing is left out here because a stored example has none.

prompt "List every product with its price as a number and whether it is in stock."

source spider.cloud/examples/catalog/

Item	Price	Stock
Field notebook	$18	In stock
Ruling pen	$42	In stock
Cold-press paper	$27	Out of stock
Sepia ink	$12	In stock

metadata.extracted_data 200

name Field notebook

price 18

in_stock true

name Ruling pen

price 42

in_stock true

name Cold-press paper

price 27

in_stock false

name Sepia ink

price 12

in_stock true

Stored result from this request. Hover or focus a field to see where it came from.

curl -X POST https://api.spider.cloud/ai/scrape \
  -H "Authorization: Bearer $SPIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://spider.cloud/examples/catalog/",
    "prompt": "List every product with its price as a number and whether it is in stock.",
    "cleaning_intent": "extraction",
    "extraction_schema": {
      "name": "catalog",
      "description": "Every product on the page.",
      "schema": "{\"type\":\"object\",\"properties\":{\"products\":{\"type\":\"array\",\"items\":{\"type\":\"object\",\"properties\":{\"name\":{\"type\":\"string\"},\"price\":{\"type\":\"number\"},\"in_stock\":{\"type\":\"boolean\"}},\"required\":[\"name\",\"price\",\"in_stock\"]}}},\"required\":[\"products\"]}"
    },
    "return_format": "markdown",
    "metadata": true
  }'

import os

import requests

payload = {
  "url": "https://spider.cloud/examples/catalog/",
  "prompt": "List every product with its price as a number and whether it is in stock.",
  "cleaning_intent": "extraction",
  "extraction_schema": {
    "name": "catalog",
    "description": "Every product on the page.",
    "schema": "{\"type\":\"object\",\"properties\":{\"products\":{\"type\":\"array\",\"items\":{\"type\":\"object\",\"properties\":{\"name\":{\"type\":\"string\"},\"price\":{\"type\":\"number\"},\"in_stock\":{\"type\":\"boolean\"}},\"required\":[\"name\",\"price\",\"in_stock\"]}}},\"required\":[\"products\"]}"
  },
  "return_format": "markdown",
  "metadata": True
}

response = requests.post(
    "https://api.spider.cloud/ai/scrape",
    headers={
        "Authorization": f"Bearer {os.environ['SPIDER_API_KEY']}",
        "Content-Type": "application/json",
    },
    json=payload,
    timeout=120,
)

print(response.json())

import https from "node:https";

const payload = JSON.stringify({
  "url": "https://spider.cloud/examples/catalog/",
  "prompt": "List every product with its price as a number and whether it is in stock.",
  "cleaning_intent": "extraction",
  "extraction_schema": {
    "name": "catalog",
    "description": "Every product on the page.",
    "schema": "{\"type\":\"object\",\"properties\":{\"products\":{\"type\":\"array\",\"items\":{\"type\":\"object\",\"properties\":{\"name\":{\"type\":\"string\"},\"price\":{\"type\":\"number\"},\"in_stock\":{\"type\":\"boolean\"}},\"required\":[\"name\",\"price\",\"in_stock\"]}}},\"required\":[\"products\"]}"
  },
  "return_format": "markdown",
  "metadata": true
});

const options = {
  hostname: "api.spider.cloud",
  port: 443,
  path: "/ai/scrape",
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.SPIDER_API_KEY}`,
    "Content-Type": "application/json",
    "Content-Length": Buffer.byteLength(payload),
  },
};

const req = https.request(options, (res) => {
  let raw = "";
  res.on("data", (chunk) => {
    raw += chunk;
  });
  res.on("end", () => {
    console.log(JSON.parse(raw));
  });
});

req.on("error", (error) => {
  console.error(error);
});

req.write(payload);
req.end();

Set SPIDER_API_KEY to your key. The account needs an active AI Studio plan; without one the call returns 402.

response body · example

[
  {
    "url": "https://spider.cloud/examples/catalog/",
    "error": null,
    "status": 200,
    "content": "# Catalog\n\n| Item | Price | Stock |\n| --- | --- | --- |\n| Field notebook | $18 | In stock |",
    "metadata": {
      "title": "Catalog",
      "extracted_data": {
        "products": [
          {
            "name": "Field notebook",
            "price": 18,
            "in_stock": true
          },
          {
            "name": "Ruling pen",
            "price": 42,
            "in_stock": true
          },
          {
            "name": "Cold-press paper",
            "price": 27,
            "in_stock": false
          },
          {
            "name": "Sepia ink",
            "price": 12,
            "in_stock": true
          }
        ]
      }
    },
    "costs": null,
    "duration_elasped_ms": …
  }
]

One object per page. costs is null when a request bills nothing and a twelve-key breakdown otherwise; the timing is left out here because a stored example has none.

Every response field The catalog page

Timed on one page.

One page and one prompt, sent 30 times in a row from a laptop, timed from the request leaving to the whole body arriving. The conditions are below and the script is in the repo, so you can run it on your own page.

10.23s

median, request to result

15.17s

p95, request to result

7.46s

median server time per page

100%

requests with a page and no error

conditions

Endpoint POST /ai/scrape
Page spider.cloud/about/
Prompt List each contact on the page with its label and email address.
Sample 30 requests, one at a time, 3 warm-ups discarded, 1200 ms apart
Client time fetch() start to the full JSON body parsed
Server time the duration_elasped_ms field on the page object
From a laptop on a residential connection outside AWS
Success 100% of requests returned a page with status 200 and no error; 100% carried every schema field
Date 2026-09-14

reading it

A single run. Time on a page depends on its size and how much of it the model has to read; a heavy page takes longer. Your plan's rate limit caps how many requests you can send a second and says nothing about how long one takes.

Client time includes the trip to and from the laptop the run was made on. Server time is the part Spider spent on the page.

The script

Six ways to ask.

Same headers, same key, same page object back. Pick the route by what you want done; each one takes every parameter its standard counterpart takes, plus the prompt.

Full reference

AI Crawl POST /ai/crawl Collect the same fields from every page of a site. "Collect every pricing tier with its monthly cost and included credits." AI Scrape POST /ai/scrape Pull fields from one page. "Pull the Pro plan name, its monthly cost, and whether annual billing is selected." AI Search POST /ai/search Search the web for a question and read the results. "Find recent benchmarks comparing Rust web crawlers on throughput." AI Browser POST /ai/browser Click, fill and wait in a real browser, then extract. "Switch the billing toggle to annual, then read back each plan name with its price." AI Links POST /ai/links Keep only the links that match what you are after. "Find every documentation link under the API reference." AI Unblocker POST /ai/unblocker Read a page that blocks a plain fetch. "Return the article headline and body text without navigation or ads."

Pick a rate limit.

AI Studio is an add-on to a Spider account. The subscription unlocks the six routes and sets how many requests a second you can send. The work each request does is metered against your credits, the same as the rest of the API.

Compare plans

7-day trial

$ 6 /mo

1 requests per second

or $300 a year

$ 30 /mo

5 requests per second

or $1,250 a year

$ 125 /mo

10 requests per second

or $6,000 a year

$ 600 /mo

25 requests per second

Rate limit Requests per second on the AI routes. This is what the tier sets.
Usage Each page is metered in the costs block against your pay as you go credits. The model cost passes through at vendor rate.
Trial Seven days on Starter, monthly billing. Then $6 a month.
Annual Two months free on any tier.
Enterprise From $2,000 a month with custom limits, through sales.

Start free trial Contact sales

Questions.

When do I use an AI route instead of the standard one?

When you want fields, not a document. /scrape returns the page as markdown or HTML for you to parse. /ai/scrape takes the same parameters plus a prompt and returns the fields you named on metadata.extracted_data. If you already have a parser that works, the standard route is cheaper; if the markup changes under you, the prompt does not.

How do I control the shape of the result?

Pass an extraction_schema with a name and a JSON schema. The result follows it, so a price comes back as a number and a missing field comes back as null rather than a guess. Without a schema the model picks the shape from the prompt.

What decides how long a request takes?

Fetching and rendering the page, cleaning it down to what the model needs to read, and the model call itself. A long page takes longer than a short one. Your plan sets a rate limit in requests per second, which caps how many you can send at once and has no effect on how long one takes.

What happens when the page does not have what I asked for?

The field is null and the page object still comes back with status 200. Each page carries its own status and error; a crawl can return 200 for the request while one page inside it failed, so read error on each object rather than the HTTP status alone.

How do I turn it on?

Subscribe on the pricing page. The AI routes then work with the API key you already have. Until then they answer 402 with code ai_subscription_required.

What do I pay?

The subscription, from $6 a month, sets your rate limit. Each request is then metered against your Spider credits and the breakdown is on the costs field of every page object. The model cost is the vendor's rate with nothing added.

Start with one request.

Subscribe, keep your key, send a URL and a prompt.

Start free trial API reference