Give your agents the whole web, as data.

One API to crawl, scrape and search any site and get back clean markdown or JSON. It runs on a browser we wrote in Rust and 215M+ residential exits, at 100K+ pages per second.

Get started free

Request POST api.spider.cloud/scrape

{
  "url": "https://news.ycombinator.com",
  "return_format": "markdown"
}

Response 200 · 1 page · markdown

## Hacker News
1. [Show HN: A crawler written in Rust](…)  412 points
2. [Ask HN: How do you keep docs current?](…)  301 points
3. [The state of headless browsers](…)  188 points

pages per second: 100K+ pages per second
browser, ours: 1 browser, ours
exits: 215M+ exits
on failures: $0 on failures

Run it in the playground

Built into frameworks agents run on

Three products. One key.

Read the web as data, drive it with a real browser, or route through our network. Mix them in the same request.

api.spider.cloud

POST /crawl  news.ycombinator.com

# Hacker News front page
/          200  18 KB  markdown
/newest    200  21 KB  markdown
/ask       200  14 KB  markdown
27 more pages, streamed in order

200 OK · 30 pages · 612 ms

Crawl, scrape, search

Any URL in, markdown or JSON out through one scraping API. Whole sites streamed in order, plus links, screenshots and web search from the same key.

See the API

browser.spider.cloud
```
wss://browser.spider.cloud

session     connected
navigate    https://app.example.com
click       "Sign in"
fill        #email
screenshot  1280x800  png

session     closed
```
A browser we wrote
Headless and written from scratch in Rust. Stealth, proxy rotation and CAPTCHA solving are on by default, with up to 100 concurrent sessions.
See the browser
proxy.spider.cloud
```
http://proxy.spider.cloud

country    DE
type       ISP
rotate     per request

215M+ IPs · 199 countries
from $1 / GB · failed requests $0
```
The network underneath
215M+ residential and ISP exits in 199 countries, rotated per request. Use it with our browser or your own.
See proxy locations

Use the whole web like a dataset.

Most of the web has no API. Spider is an AI web crawler that gives your agent the page as it renders, then hands back only the part worth reading. See how that feeds a RAG pipeline or an AI agent.

Renders like a user
JavaScript, infinite scroll, lazy media. The page as a person sees it.
request: "browser"
Gets past the blocks
Anti-bot checks, CAPTCHAs and geo walls, handled by the browser and the network together.
stealth: true
Returns the signal
Readability-cleaned markdown or structured JSON, with source URL, title and timestamp on every record.
return_format: "markdown"
Streams, in order
Whole sites arrive page by page while the crawl runs. Nothing waits for the last URL.
application/jsonl
Bills what came back
A blocked, timed-out or failed page costs $0. You pay for data, not attempts.
costs: null

Pay as you go

$1 per GB

Bandwidth: $1 per GB
Compute: $0.001 per minute
Blocked, timed out, failed: $0
Minimum: none

Start free

Unlimited

$40 per month, from

Requests: unlimited
From: 2 units, $40 a month
Per unit: $20 down to $13
Failed requests: $0

Go unlimited

Measured in the open

Benchmarks are public. Rerun StealthBench V1 or BENCHMARKS.md yourself.

pages per second: 100K+; Whole sites crawled and streamed back as markdown, in order.
browser, ours: 1; Headless, written from scratch in Rust. Up to 100 concurrent sessions.
exits: 215M+; Residential and ISP proxies in 199 countries, rotated per request.
on failures: $0; If a page does not come back, it is not on the bill.

Three steps to your first crawl.

Get a key

Create API key

Connect Claude Code Client configs in the docs

claude mcp add spider --transport http \
  https://mcp.spider.cloud/mcp \
  -H "Authorization: Bearer $SPIDER_API_KEY"

Read and follow https://spider.cloud/agent-skill/SKILL.md

Or skip MCP and let the agent onboard itself

Connect Codex Client configs in the docs

Read and follow https://spider.cloud/agent-skill/SKILL.md

Paste into a Codex session. The skill covers the whole API.

Connect your MCP client Client configs in the docs

Endpoint:  https://mcp.spider.cloud/mcp
Transport: Streamable HTTP
Auth:      Authorization: Bearer sk-...

Works in Claude Desktop, Cursor, Windsurf, or any MCP client.

Make the first request Full example in the docs

curl -X POST https://api.spider.cloud/scrape \
  -H "Authorization: Bearer $SPIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "return_format": "markdown"}'

Install the SDK and run it Full example in the docs

pip install spider_client

from spider import Spider
app = Spider()  # reads SPIDER_API_KEY
page = app.scrape_url("https://example.com")

Install the SDK and run it Full example in the docs

npm install @spider-cloud/spider-client

import { Spider } from "@spider-cloud/spider-client";
const app = new Spider(); // reads SPIDER_API_KEY
const page = await app.scrapeUrl("https://example.com");

Read the data

[{
  "url": "https://example.com",
  "status": 200,
  "content": "# Example Domain\n\n..."
}]

Markdown or JSON, streamed. Drop it into your RAG store, agent context or database.

The whole web,
at agent speed.

Get started free

Give your agents the whole web, as data.

Three products. One key.

Crawl, scrape, search

A browser we wrote

The network underneath

Use the whole web like a dataset.

Renders like a user

Gets past the blocks

Returns the signal

Streams, in order

Bills what came back

Pricing

Pay as you go

Unlimited

By the numbers

Three steps to your first crawl.

Get a key

Connect Claude Code Client configs in the docs →

Connect Codex Client configs in the docs →

Connect your MCP client Client configs in the docs →

Make the first request Full example in the docs →

Install the SDK and run it Full example in the docs →

Install the SDK and run it Full example in the docs →

Read the data

The whole web,at agent speed.

Connect Claude Code Client configs in the docs

Connect Codex Client configs in the docs

Connect your MCP client Client configs in the docs

Make the first request Full example in the docs

Install the SDK and run it Full example in the docs

Install the SDK and run it Full example in the docs

The whole web,
at agent speed.