Skip to main content

Quickstart

Install a client, export an API key, run the first request. Three steps — or hand the setup to your AI agent.

01

Install a client

Pick the language you'll call the API from. The Python and Node SDKs wrap the REST endpoints; cURL hits the bare API.

pip install spider_client
02

Export an API key

Create an API keyin the dashboard, then export it so the SDK and cURL examples can read it.

export SPIDER_API_KEY="your_api_key_here"
03

Make your first request

Pick an endpoint and run the example. All three accept the same JSON body shape.

Request
import requests

headers = {
    'Authorization': 'Bearer $SPIDER_API_KEY',
    'Content-Type': 'application/json',
}

json_data = {"limit":5,"url":"https://example.com"}

response = requests.post('https://api.spider.cloud/crawl', 
  headers=headers, json=json_data)

print(response.json())
Handle the response

Every response is a JSON array. Each element carries the page URL, content in the requested format, an HTTP status code, and a costs breakdown. Check status before reading content.

Parse the response

import requests, os headers = { 'Authorization': f'Bearer {os.getenv("SPIDER_API_KEY")}', 'Content-Type': 'application/json', } response = requests.post( 'https://api.spider.cloud/crawl', headers=headers, json={"url": "https://example.com", "limit": 5, "return_format": "markdown"} ) data = response.json() for page in data: if page.get('status') == 200: print(f"URL: {page['url']}") print(f"Content length: {len(page.get('content', ''))} chars") print(f"Cost: {page['costs']['total_cost_formatted']}") else: print(f"Failed: {page['url']} - {page.get('error', 'Unknown error')}")
Next steps
ConceptsRequest modes, output formats, streaming, and the credits system.Scraping and crawlingDepth control, response fields, and cost breakdowns.Efficient scrapingBatch requests, retries, timeouts, and multi-URL patterns.JSON scrapingExtract structured data from JSON-LD, Next.js SSR, and other embedded formats.RecipesCopy-paste code for crawling, scraping, search, screenshots, streaming, and webhooks.