MCP server

Connect an AI agent to Spider over the Model Context Protocol. Crawl, scrape, search and extract web data from Claude, Cursor, Windsurf or any MCP client.

Connect

Point your MCP client at https://mcp.spider.cloud/mcp with your API key as a Bearer token. Nothing to install. To run it locally instead, use npx spider-cloud-mcp.

Hosted server (recommended)

claude mcp add spider \
  --transport http https://mcp.spider.cloud/mcp \
  -H "Authorization: Bearer your-api-key"

Available tools

The server exposes 21 tools in three groups.

Core tools

Core tools
Tool	What it does
spider_crawl	Crawl a site and return content from many pages.
spider_scrape	Scrape one page and return its content.
spider_search	Search the web, with an option to crawl the results.
spider_links	Return every link on a page.
spider_screenshot	Capture a screenshot of a page.
spider_transform	Turn HTML into markdown or text.
spider_get_credits	Read the credit balance on the key.

AI tools

These need an AI Studio subscription.

AI tools
Tool	What it does
spider_ai_crawl	Crawl guided by a natural language prompt.
spider_ai_scrape	Extract structured data from a page.
spider_ai_search	Semantic web search.
spider_ai_browser	Drive a browser from a prompt.
spider_ai_links	Pick the links on a page that match a prompt.

Browser tools

Browser tools
Tool	What it does
spider_browser_open	Open a remote session in the native Spider browser.
spider_browser_navigate	Go to a URL and wait for the load.
spider_browser_click	Click an element by CSS selector.
spider_browser_fill	Type text into a form field.
spider_browser_screenshot	Capture the current page.
spider_browser_content	Return the page HTML or its visible text.
spider_browser_evaluate	Run JavaScript in the page.
spider_browser_wait_for	Wait for a selector, a navigation or network idle.
spider_browser_close	Close the session and release its resources.

Usage examples

Once connected, the agent calls Spider tools on its own. Two common runs, as the agent sees them.

Scraping and crawling

User: "Scrape spider.cloud and give me the pricing details"

Agent uses spider_scrape with:
  url: "https://spider.cloud"
  return_format: "markdown"

User: "Crawl the docs and find all API endpoints"

Agent uses spider_crawl with:
  url: "https://spider.cloud/docs"
  limit: 25
  return_format: "markdown"

Browser automation workflow

User: "Log into my dashboard and screenshot the analytics page"

Agent uses spider_browser_open
  → returns session_id: "abc-123"

Agent uses spider_browser_navigate with:
  session_id: "abc-123"
  url: "https://app.example.com/login"

Agent uses spider_browser_fill with:
  session_id: "abc-123"
  selector: "input[name='email']"
  value: "user@example.com"

Agent uses spider_browser_click with:
  session_id: "abc-123"
  selector: "button[type='submit']"

Agent uses spider_browser_wait_for with:
  session_id: "abc-123"
  selector: ".dashboard-loaded"

Agent uses spider_browser_navigate with:
  session_id: "abc-123"
  url: "https://app.example.com/analytics"

Agent uses spider_browser_screenshot with:
  session_id: "abc-123"
  → returns base64 PNG image

Agent uses spider_browser_close with:
  session_id: "abc-123"

Browser session lifecycle

The browser tools drive a remote browser. Each session is isolated per user and comes with the native browser engine, proxy rotation and cross-browser support.

How it works

01Call spider_browser_open to start a session. It returns a session_id.
02Pass that session_id to any browser tool: navigate, click, fill, screenshot, evaluate, wait or content.
03Call spider_browser_close when you are done. Billing stops there.

Key parameters

Browser tool parameters
Parameter	Tools	Description
session_id	all but open	Which session to drive.
selector	click, fill, wait_for	CSS selector for the target element, such as `"button.submit"` or `"#login-btn"`.
timeout	click, fill, wait_for	Longest wait in ms before the call fails. Default 10,000ms for click and fill, 30,000ms for wait_for.
expression	evaluate	JavaScript to run in the page, such as `"document.title"`.
format	content	`"html"` for the full DOM or `"text"` for visible text only.
stealth	open	0 auto, 1 standard, 2 residential proxy, 3 premium proxy.

Limits

Up to 5 concurrent sessions per MCP connection.
A session closes on its own after 5 minutes of inactivity.
Close a session as soon as you are done with it. An open session keeps billing.

Parameters

Core and AI tools take the same parameters as the matching Spider API endpoint. The common ones are url, return_format, limit, depth, proxy and request. The API reference has the full list.

Hosted server

Nothing runs on your machine. Point any MCP client at https://mcp.spider.cloud/mcp over Streamable HTTP with your API key as a Bearer token. The Connect page sets it up in one click. No Node.js. Same 21 tools, same billing.

Add hosted Spider MCP server to Claude Code

claude mcp add spider --transport http https://mcp.spider.cloud/mcp \
  -H "Authorization: Bearer your-api-key"

npm package

The server ships as spider-cloud-mcp on npm. The source is on GitHub.