Skip to main content

Infrastructure for reading the public web.

One API to scrape, crawl, and search, backed by our own browser engine and network. Low cost, low latency, built to scale.

Get started free

Run requests before you buy. Or paste the setup prompt into your coding agent.

Request
POST api.spider.cloud/scrape
Try

  • api.spider.cloud POST
    100K+ pages per second

    Crawl, scrape and search on one host. Whole sites stream back as markdown, in order.

  • browser.spider.cloud WS
    100% browser fidelity bench v1

    Spider, 80 tasks, Sep 18, 2026. Real production sites. See the results

  • proxy.spider.cloud HTTP
    215M+ exits in 199 countries

    Residential addresses, rotated per request. The same network runs the crawler and the browser.

One key 3 hosts

Integrations

Use the whole web like a dataset.

Spider is a research lab for web data quality. Most websites have no API, so we ship one for your RAG pipeline or agent.

  1. Renders like a user

    Scripts run, lazy images load, and infinite scroll reaches the end. You get the page a person sees, not the one the server sends.

    request: "browser" See the browser

  2. Loads the protected page

    A protected page loads for Spider the way it loads for a person, geo checks included. What comes back is the page, with a 200.

    stealth: true See the scrape endpoint

  3. Extracts the content

    Navigation, ads and footers drop away. What is left is markdown or JSON that reads the way the page was written.

    return_format: "markdown" See the formats

  4. Streams in order

    Each page in a crawl arrives as one JSONL line the moment it is done, in order. Nothing waits for the last URL.

    application/jsonl See streaming

Pay for usage, or for concurrency.

pay as you go

no minimum

$1 per GB

CPU time
$0.0001 per minute
Blocked page minimum
0.01 credits
Start free

unlimited

flat rate

$40 a month

In flight
from 2
Requests
unlimited
Per concurrent request
$20 down to $13
See unlimited plans

Make your first request.

Pick your client and paste.

01 Get a key

Create API key

Your new account gets 2,500 free credits, worth $0.25, if your signup passes our checks.

export SPIDER_API_KEY="sk-..."

02 Connect Claude Code

claude mcp add spider --transport http \
  https://mcp.spider.cloud/mcp \
  -H "Authorization: Bearer $SPIDER_API_KEY"
CLAUDE.md 
For web pages use spider-agent. Read https://spider.cloud/agent-skill/spider-agent/SKILL.md first.

Or the CLI. It picks HTTP or a browser per page, stops at your budget, and returns 187 bytes for named fields where asking for everything returns 9021.

Client configs in the docs

03 Read the data

example output example data after a scrape 
[{
  "url": "https://example.com",
  "status": 200,
  "content": "# Example Domain\n\n..."
}]
Quickstart in the docs

Start with one URL.

Get started free