# AI web scraping integrations showcase | Spider

Web scraping integrations for LangChain, LlamaIndex, CrewAI, AutoGen, Dify, Flowise and MCP. See the AI tools that crawl the web with Spider in production.

Canonical: https://spider.cloud/showcase/

## Built with Spider.

Integrations that use Spider for web crawling and scraping in production, inside AI frameworks like LangChain, LlamaIndex and CrewAI, no-code builders, and an accessibility auditor.

AI Framework Accessibility AI Agents LLM Platform No-Code MCP

[Get an API key](https://spider.cloud/register/) [Read the docs](https://spider.cloud/docs/overview/)

## Spider integrations in production.

[AI Framework](https://reference.langchain.com/python/langchain-community/document_loaders/spider/SpiderLoader)

[LangChain](https://reference.langchain.com/python/langchain-community/document_loaders/spider/SpiderLoader)

### [LangChain SpiderLoader](https://reference.langchain.com/python/langchain-community/document_loaders/spider/SpiderLoader)

[First-class document loader in langchain_community. Feed it any URL, get back LLM-ready markdown. Supports scrape and crawl modes with full Spider API parameters. Used by teams building RAG pipelines who need clean, structured web content without managing scrapers.](https://reference.langchain.com/python/langchain-community/document_loaders/spider/SpiderLoader)

[Python LangChain spider-client](https://reference.langchain.com/python/langchain-community/document_loaders/spider/SpiderLoader)

[Accessibility](https://a11ywatch.com)

[A11yWatch](https://a11ywatch.com)

### [A11yWatch](https://a11ywatch.com)

[Automated WCAG accessibility auditing platform. Spider replaced an entire crawling infrastructure of dedicated servers, headless browsers, and proxy rotation. Monthly cost went from $2,000 to $13 for the same workload. The engineering time freed up went back into building better audits.](https://a11ywatch.com)

[Rust Spider API WCAG](https://a11ywatch.com)

## Web scraping in the AI tools you already use.

[AI Framework](https://spider.cloud/docs/integrations/llamaindex)

[LlamaIndex](https://spider.cloud/docs/integrations/llamaindex)

### [LlamaIndex SpiderWebReader](https://spider.cloud/docs/integrations/llamaindex)

[Web reader integration for LlamaIndex. Crawls and scrapes URLs into LlamaIndex Document objects with configurable return formats, readability mode, and metadata extraction. Ships as part of llama-index-readers-web.](https://spider.cloud/docs/integrations/llamaindex)

[Python LlamaIndex spider-client](https://spider.cloud/docs/integrations/llamaindex)

[AI Agents](https://docs.crewai.com/tools/spidertool)

[CrewAI](https://docs.crewai.com/tools/spidertool)

### [CrewAI SpiderTool](https://docs.crewai.com/tools/spidertool)

[Official tool in the CrewAI multi-agent framework. Gives CrewAI agents the ability to scrape and crawl websites as part of their task execution. Agents can pull live web data mid-workflow without custom code.](https://docs.crewai.com/tools/spidertool)

[Python CrewAI spider-client](https://docs.crewai.com/tools/spidertool)

[LLM Platform](https://github.com/langgenius/dify)

[Dify](https://github.com/langgenius/dify)

### [Dify Spider Tool](https://github.com/langgenius/dify)

[Built-in scraper and crawler tool inside Dify, the open-source LLM app platform. Available in both Workflow and Agent modes. Configure your Spider API key in the Dify UI and any workflow can pull live web content.](https://github.com/langgenius/dify)

[Python Dify Spider API](https://github.com/langgenius/dify)

[No-Code](https://docs.flowiseai.com/integrations/langchain/document-loaders/spider-web-scraper-crawler)

[FlowiseAI](https://docs.flowiseai.com/integrations/langchain/document-loaders/spider-web-scraper-crawler)

### [Flowise Spider Node](https://docs.flowiseai.com/integrations/langchain/document-loaders/spider-web-scraper-crawler)

[Visual document loader node in Flowise, the low-code LLM flow builder. Drag and drop Spider into any workflow to scrape or crawl URLs. No code required. Connects directly to vector stores, text splitters, and LLM chains.](https://docs.flowiseai.com/integrations/langchain/document-loaders/spider-web-scraper-crawler)

[TypeScript Flowise LangChain](https://docs.flowiseai.com/integrations/langchain/document-loaders/spider-web-scraper-crawler)

[AI Agents](https://microsoft.github.io/autogen/0.2/docs/notebooks/agentchat_webcrawling_with_spider/)

[Microsoft](https://microsoft.github.io/autogen/0.2/docs/notebooks/agentchat_webcrawling_with_spider/)

### [AutoGen Spider Notebook](https://microsoft.github.io/autogen/0.2/docs/notebooks/agentchat_webcrawling_with_spider/)

[Official AutoGen example showing how to wire Spider into multi-agent conversations. A scraper agent and crawler agent work together to extract and summarize web content. Part of the AutoGen 0.2 documentation.](https://microsoft.github.io/autogen/0.2/docs/notebooks/agentchat_webcrawling_with_spider/)

[Python AutoGen GPT-4](https://microsoft.github.io/autogen/0.2/docs/notebooks/agentchat_webcrawling_with_spider/)

[MCP](https://github.com/AshDevFr/spider-mcp)

[Spider](https://github.com/AshDevFr/spider-mcp)

### [Spider MCP Server](https://github.com/AshDevFr/spider-mcp)

[Model Context Protocol server that gives any MCP-compatible AI (Claude, etc.) the ability to crawl and scrape the web through Spider. Install it once, and your AI assistant can pull live web data during conversations.](https://github.com/AshDevFr/spider-mcp)

[TypeScript MCP Spider SDK](https://github.com/AshDevFr/spider-mcp)

## Free account, single API call.

Every project above started the same way.

[Get an API key](https://spider.cloud/register/) [Quickstart](https://spider.cloud/docs/quickstart/)

submit

### Built something with Spider?

Open source or commercial, big or small. Send us the link.

[Submit your project →](mailto:sales@spider.cloud?subject=Showcase%20Submission)
