Connect Spider to the tools you already use.

Spider's web scraping API plugs into LangChain, LlamaIndex and CrewAI as a loader, into Zapier and Pipedream as a node, and streams crawl results to S3, Google Sheets or Supabase.

AI frameworks

LangChain

Document loader for LangChain chains and RAG pipelines.

Read docs

LlamaIndex

Web reader that loads crawled pages into a LlamaIndex index.

Read docs

CrewAI

SpiderTool for CrewAI agents that scrape and crawl.

Read docs

FlowiseAI

Document loader node for the visual builder.

Read docs

Autogen

Web crawl agents inside Microsoft Autogen.

Read docs

Composio

Scrape and crawl tool in the Composio catalog.

Read docs

Agno

SpiderTools for Agno agents: scrape, crawl and search.

Read docs

Langflow

Crawler and scraper node for Langflow graphs.

Read docs

Dify

Verified plugin for Dify workflows and agents.

Read docs

Julep

Scrape and crawl inside Julep workflows.

Read docs

Data

Data connectors

Stream crawl results to S3, GCS, Azure Blob, Google Sheets or Supabase.

Read docs

Command line

Agent CLI

spider-agent picks HTTP or a browser per page, stops at a budget, and streams NDJSON.

Read docs

Agent examples

spider-agent at work in Claude Code, Codex, a cron job and a Rust program.

Read docs

No code

Zapier

Scrape a URL as a step in any Zap.

Read docs

Pipedream

Run Spider as a step in a Pipedream workflow.

Read docs