The search API that returns pages, not links.
Spider runs your query across the major engines, crawls every result, and returns each page's content as markdown, text, or HTML. Ten links are where other search APIs stop, and where this one starts.
Type a query and hit Search to see live results from Spider's Search API.
No sign-in required. 25 free searches per day.
Through the API, $1 buys 10,000 credits and pages are billed as they come back. Failed pages cost nothing.
The difference is one boolean.
Other search APIs sell the left panel. The right panel is the same call with fetch_page_content turned on.
{
"content": [
{
"title": "Web crawler",
"url": "https://en.wikipedia.org/wiki/Web_crawler",
"description": "A web crawler, sometimes called a spider or spiderbot, is an Internet bot that systematically browses…"
}
// 9 more links
]
}Stop here and fetching, rendering, unblocking, and cleanup are still your backlog.
[
{
"url": "https://en.wikipedia.org/wiki/Web_crawler",
"status": 200,
"content": "# Web crawler\n\nA web crawler, sometimes called a spider or spiderbot, systematically browses the World Wide Web…",
"costs": { "total_cost": 0.00036 }
}
// one object per result
]Rendered in a real browser when the page needs one, cleaned into your format, priced per page in the response.
RAG without the stale index.
The question becomes the query, the crawled results become the context, and the answer stops aging. Nothing to embed, sync, or re-crawl on a schedule.
from spider import Spider
client = Spider()
# The user's question is the search query
pages = client.search(
"what changed in the EU AI Act this month",
params={
"search_limit": 5,
"fetch_page_content": True,
"return_format": "markdown",
"tbs": "qdr:m", # past month only
},
)
# Crawled pages become the model's context
context = "\n\n".join(p["content"] for p in pages)Ten parameters cover the job.
- search_limit
- How many results come back, and with content fetching on, how many get crawled. More pages cost more credits and take longer.
- fetch_page_content
- Off, you get the ranked list. On, Spider visits every result and returns the body.
- return_format
- markdown, text, html, or raw bytes. The same cleaning as crawl and scrape.
- limit
- Pages crawled per result site. Raise it above 1 to go deeper than the page that ranked.
- tbs
- Time window, qdr:h through qdr:y. Past hour, day, week, month, or year.
- country, location, language
- Results as a user in that region sees them. location takes a city, like "San Diego, CA".
- metadata
- Adds titles, descriptions, and keywords next to the content.
- page
- Paginates deeper into the results.
- array request body
- POST an array of query objects and each search runs independently in the same call. Streaming is not supported for batch.
- jsonl streaming
- Ask for JSONL and pages arrive as each one finishes instead of after the slowest.
Worked examples for every one of these live in the realtime search guide .
Checked against the alternatives.
Every competitor cell verified on public pricing and rate limit pages, August 2026.
| Feature | Spider | Firecrawl | Jina | SerpAPI |
|---|---|---|---|---|
| A search returns | Crawled page content in your format | Links, scraping billed per page on top | Top results as markdown | Links and snippets |
| 10 results with content | About $0.003 | About $0.01 in plan credits | Token-billed by content length | Not offered |
| Billing | Pay as you go, $1 per 10,000 credits | Monthly credit plans | Prepaid token bundles | Subscription, from $25 per 1,000 searches |
| Throughput | 10,000 requests per minute | 2 to 100 concurrent, by plan | 500 requests per minute on paid keys | Hourly cap, 20% of monthly volume |
Spider's per-search figure is derived from the per-page costs in the docs example response, about $0.00035 per crawled page.
Or let a model write the query.
POST /ai/search takes a plain prompt, generates the search, and returns structured results in the response metadata.
curl -X POST https://api.spider.cloud/ai/search \
-H "Authorization: Bearer $SPIDER_API_KEY" \
-d '{
"prompt": "Find recent benchmarks comparing Rust web crawlers on throughput.",
"num": 10,
"fetch_page_content": true,
"return_format": "markdown"
}'Full reference in the AI Studio docs .
Cost, blocking, speed, scale.
What does it cost?
$1 buys 10,000 credits, pay as you go. In the docs example a crawled result runs about $0.00035 per page, so a 10-result search with full content lands near a third of a cent. Failed, blocked, and timed-out pages cost nothing.
What about JavaScript-heavy results?
Results that need a browser get one. Spider renders the page before extracting, so client-side content shows up in the markdown like everything else.
How fast is it?
Run the demo at the top of this page. It hits the production endpoint and the timer next to the results is real, so you get your own latency number instead of ours.
Will it scale?
The engine takes 10,000 requests a minute. Stream JSONL to read pages as each finishes, and batch several queries into one call when an agent has more than one question.
More from the API.
Skip the crawler you were about to write.
One request searches, crawls, and returns content in your format. Pay as you go, $1 for 10,000 credits, nothing charged for failed pages.