Skip to main content

Spider integrations for the tools you already use.

Native document loaders for the AI frameworks, ready-made nodes for n8n and Zapier, and official SDKs for Python, Node, and Rust.

LangChain LlamaIndex CrewAI AutoGen MCP +14more

AI frameworks with native loaders.

Pull Spider into LangChain, LlamaIndex, CrewAI, and the other frameworks shipping today. No glue code.

Browser New

Spider Browser

Custom Rust browser engine with AI agents, stealth, and retry engine. Connect to pre-warmed cloud browsers via WebSocket.

npm install spider-browser
Docs → GitHub ↗
MCP Popular

MCP Server

22 tools for Claude, Cursor, Windsurf, and any MCP client. Run locally via npx or connect to the hosted server at mcp.spider.cloud.

npx spider-cloud-mcp
Docs → GitHub ↗
AI Framework

LangChain

Use Spider as a document loader in your LangChain applications. Perfect for building RAG systems and AI agents.

Docs → Guide →
AI Framework

LlamaIndex

Spider Reader for LlamaIndex loads pages into your index as clean documents.

Docs →
AI Agents

CrewAI

Build AI agent crews with web crawling capabilities using the SpiderTool integration.

Docs → Guide →
AI Agents

AutoGen

Microsoft's AutoGen framework integration for multi-agent web crawling and scraping workflows.

Guide →
AI Framework

Agno

Web scraping toolkit integration for the Agno AI platform.

Docs →
No-Code

FlowiseAI

No-code integration for building LLM workflows with Spider as a document loader.

Docs →
No-Code

Langflow

Spider Web Crawler and Scraper component, ready to drop into any Langflow graph.

GitHub ↗
No-Code

Dify

Dify-verified plugin that gives workflows and agents scrape and crawl tools.

GitHub ↗
AI Agents

Julep

Crawl, links, search, and screenshot methods available to Julep tasks.

GitHub ↗

Route data wherever it needs to go.

Ready-made nodes for n8n and Zapier, first-party connectors for S3, GCS, Supabase, and Sheets, and webhooks for everything else.

Data Pipeline

Data Connectors

Pipe crawl results directly to S3, GCS, Azure Blob, Supabase, or Google Sheets as pages arrive.

Learn more →
Automation

Zapier

Connect Spider to 5,000+ apps. Trigger crawls from any event and send data anywhere.

Learn more →
Automation

Webhooks

Receive crawl results in real-time via HTTP webhooks to any endpoint.

Learn more →
Automation

n8n

Community node for n8n workflow automation. Crawl, scrape, search, and transform web content in your workflows.

Learn more →

Official SDKs for every major language.

Python, Node, and Rust all share the same surface. View on GitHub ↗

client libraries

  • Python

    Official Python client with async support and type hints.

    pip install spider-client
    Docs →

  • Node.js

    TypeScript-ready Node.js client for server and edge environments.

    npm install @spider-cloud/spider-client
    Docs →

  • Rust

    Native Rust client for maximum performance in Rust applications.

    cargo add spider-client
    Docs →

  • cURL / REST

    Direct REST API access for any language or platform.

    Docs →

Don't see your tool?

Spider's REST API works with any language or platform. Build your own integration, or open a pull request on the open-source clients.

API reference Contribute on GitHub ↗