Realtime search

/search returns live web results, with optional full-page content extraction on the same call. Discover and collect in one step.

Basic web search

POST a search query and you get ranked results from across the web. search_limit caps how many come back; page drives pagination. Each result carries the URL, title, and description.

import requests, os headers = { 'Authorization': f'Bearer {os.getenv("SPIDER_API_KEY")}', 'Content-Type': 'application/json', } params = { "search": "spider web crawling", # query "search_limit": 10, # results to return "page": 1, # pagination page } response = requests.post( 'https://api.spider.cloud/search', headers=headers, json=params, ) print(response.json())

{ "content": [ { "description": "Spider is a leading web crawling tool designed for speed and cost-effectiveness, supporting various data formats including LLM-ready markdown.", "title": "Spider: The Web Crawler for AI", "url": "https://spider.cloud/" }, { "description": "Web crawler, sometimes called a spider or spiderbot, is an Internet bot that systematically browses the World Wide Web …", "title": "Web crawler", "url": "https://en.wikipedia.org/wiki/Web_crawler" } // more ] }

Search and scrape in one call

Set fetch_page_content: true and Spider visits every result URL and returns the body in your chosen format. Push limit above 1 to crawl deeper from each result.

import requests, os headers = { 'Authorization': f'Bearer {os.getenv("SPIDER_API_KEY")}', 'Content-Type': 'application/json', } params = { "search": "spider web crawling", "return_format": "raw", # also: markdown, text, html "fetch_page_content": True, "search_limit": 10, "page": 1, "limit": 1, # 1 = scrape only the top hit } response = requests.post( 'https://api.spider.cloud/search', headers=headers, json=params, ) print(response.json())

[ { "error": null, "status": 200, "duration_elasped_ms": 120, "costs": { "file_cost": 0.000363, "ai_cost": 0, "compute_cost": 7e-8, "transform_cost": 0, "total_cost": 0.00036307, "bytes_transferred_cost": 0 }, "url": "https://en.wikipedia.org/wiki/Web_crawler", "content": "<!DOCTYPE html><html><body>content...</body></html>" }, { "error": null, "costs": { "total_cost": 0.00033178 }, "status": 200, "duration_elasped_ms": 160, "url": "https://spider.cloud/", "content": "<!DOCTYPE html><html><body>content...</body></html>" } // more ]

Batch

Multiple queries in one request

POST an array of query objects to run several searches in a single API call. Each query runs independently and carries its own result set. Useful in agentic workflows that need answers to a handful of questions at once. Streaming isn't supported for batch search.

import requests, os headers = { 'Authorization': f'Bearer {os.getenv("SPIDER_API_KEY")}', 'Content-Type': 'application/json', } params = [ {"search": "latest sports news united states", "search_limit": 5}, {"search": "latest news around the globe", "search_limit": 5}, ] response = requests.post( 'https://api.spider.cloud/search', headers=headers, json=params, ) print(response.json())

[ { "content": [ { "title": "ESPN", "url": "https://www.espn.com/", "description": "Live scores, highlights, sports news…" }, { "title": "Yahoo Sports", "url": "https://sports.yahoo.com/", "description": "News, scores, video, fantasy games…" } // 3 more ] }, { "content": [ { "title": "World news | BBC", "url": "https://www.bbc.com/news/world", "description": "Latest world headlines…" }, { "title": "World news | CNN", "url": "https://www.cnn.com/world", "description": "Breaking world news…" } // 3 more ] } ]

Filters

Geo-targeted search

Pin where results originate with location, language, and country. Returns results as a user in that region would see them.

import requests, os headers = { 'Authorization': f'Bearer {os.getenv("SPIDER_API_KEY")}', 'Content-Type': 'application/json', } params = { "search": "latest sports news", "search_limit": 5, "language": "en", # result language "country": "us", # prioritize US results "location": "San Diego, CA", # search origin } response = requests.post( 'https://api.spider.cloud/search', headers=headers, json=params, ) print(response.json())

Time-based filters

Use tbs to restrict results to a recent window — past hour through past year.

import requests, os headers = { 'Authorization': f'Bearer {os.getenv("SPIDER_API_KEY")}', 'Content-Type': 'application/json', } params = { "search": "latest sports news", "search_limit": 5, "tbs": "qdr:w", # past week } response = requests.post( 'https://api.spider.cloud/search', headers=headers, json=params, ) print(response.json())

tbs value	Window
`qdr:h`	Past hour
`qdr:d`	Past 24 hours
`qdr:w`	Past week
`qdr:m`	Past month
`qdr:y`	Past year