Skip to main content
Web data infrastructure Crawler · Browser · Proxy network

Give an agent the web. All of it.

One request gets you a page, a whole site, or a search result, rendered in our own browser and routed through our own network. No scraper to maintain, no proxies to buy, no blocks to fight.

Get started free

Run requests before you buy. Or paste the setup prompt into your coding agent.

  • api.spider.cloud POST
    100K+ pages per second

    Crawl, scrape and search on one host. Whole sites stream back as markdown, in order.

  • browser.spider.cloud WS
    93.9% stealth bench v1

    Top score among 8 cloud browsers, across 71 anti-bot tasks. See the results

  • proxy.spider.cloud HTTP
    215M+ exits in 199 countries

    Residential and ISP addresses, rotated per request. The same network runs the crawler and the browser.

Framework integrations

Three products. One key.

Start with the API. The browser and the proxy network also work on their own. The trace below is one crawl through all three.

authentication

export SPIDER_API_KEY="<YOUR_API_KEY>"

Get a key

  1. The API

    Step one of an example trace, captured 2026-09-09. The API receives POST /crawl for doc.rust-lang.org/book with a limit of 30 and streams back 30 pages of markdown, 319 kilobytes in all, with status 200. Every page came back 200. demo-01 is an illustrative correlation id, not an API field.

    Scrape a page, crawl a site or search the web. Pages stream back as they land.

    See the API

  2. The browser

    Step two. The browser session was already warm. It opens a session for the API, loads each of the 30 pages from doc.rust-lang.org/book through a residential exit in Germany, and hands the markdown back as each page finishes. demo-01 is an illustrative correlation id, not an API field.

    Our custom browser opens in milliseconds. Stealth and CAPTCHA solving are on.

    See the browser

  3. The network

    Step three. The network routes the browser's requests through a residential exit in Germany, rotating the exit per request, and returns the 30 pages. The pool is 215M+ exits in 199 countries. demo-01 is an illustrative correlation id, not an API field.

    215M+ exits, 199 countries, one per request. 95%+ success on protected sites.

    See proxy locations

Use the whole web like a dataset.

Most websites have no API. Spider renders their pages and extracts the content for your RAG pipeline or agent.

  1. Renders like a user

    Engraving of an eighteenth century printing shop, the pressman at the press with a sheet on the tympan. From the Encyclopedie of Diderot and d'Alembert.

    Scripts run, lazy images load, and infinite scroll reaches the end. You get the page a person sees, not the one the server sends.

    See the browser

  2. Passes the checkpoint

    Engraving of the Porta Maggiore in Rome, two arches under an inscribed attic with the road passing through. From Piranesi's Vedute di Roma.

    The request handles bot walls, CAPTCHAs and geo checks on its own. What comes back is the page, with a 200.

    See the unblocker

  3. Extracts the content

    Woodcut of a papermaker lifting a fresh sheet from the vat on a mould, a boy carrying finished sheets beside him. From Jost Amman's Book of Trades.

    Navigation, ads and footers drop away. What is left is markdown or JSON that reads the way the page was written.

    See the formats

  4. Streams in order

    Engraving of the Marly machine in elevation, a water wheel driving a row of pumps on pilings. From the Encyclopedie of Diderot and d'Alembert.

    Each page in a crawl arrives as one JSONL line the moment it is done, in order. Nothing waits for the last URL.

    See streaming

Pay for usage, or for concurrency.

pay as you go

no minimum
$1 per GB

Plus $0.001 per CPU minute.

Bandwidth
$1 per GB
CPU time
$0.001 per minute
Blocked, timed out, failed
$0
Start free

unlimited

flat rate
from $40 a month

Priced by concurrency units. View unit pricing

Requests
unlimited
Starting allocation
2 concurrency units
Per concurrency unit
$20 down to $13
See unlimited plans

Make your first request.

Three steps, the same ones the dashboard shows after you sign up. Pick your client and paste.

01 Get a key

Create API key

Sign up and copy the key. New accounts start with a free balance.

export SPIDER_API_KEY="sk-..."

SDKs, the MCP server and every command read it.

02 Connect Claude Code

claude mcp add spider --transport http \
  https://mcp.spider.cloud/mcp \
  -H "Authorization: Bearer $SPIDER_API_KEY"
Read and follow https://spider.cloud/agent-skill/SKILL.md

Or skip MCP and let the agent onboard itself

Client configs in the docs

03 Read the data

example output example data after a scrape
[{
  "url": "https://example.com",
  "status": 200,
  "content": "# Example Domain\n\n..."
}]

The connection comes first. Data like this arrives once the client makes a request.

Markdown or JSON, streamed. Drop it into your RAG store, agent context or database.

Start with one URL.

Create a key. Make your first request.

Get started free