# AI data pipelines and use cases | Spider

How teams build AI data pipelines on Spider: site indexing, LLM ingestion, agent data collection, RAG refresh, price monitoring and compliance tracking.

Canonical: https://spider.cloud/use-cases/

## Use cases for teams building AI and data pipelines.

Five pages cover feeding models and agents. Seven cover tracking prices, rankings, leads and pages as they change. All twelve run on the same API.

[Get an API key](https://spider.cloud/register/) [Read the docs](https://spider.cloud/docs/quickstart/)

## For models and agents

5 pages

- [Agent data collection Give an agent crawl, search and browser tools through the MCP server or a framework SDK. MCP server Web search Native browser](https://spider.cloud/use-cases/ai-agents/)
- [Coding agents Let a coding agent read current docs, API references and changelogs before it writes code. Live docs MCP server](https://spider.cloud/use-cases/coding-agents/)
- [RAG refresh APIs Re-crawl docs and knowledge bases as markdown so the vector store matches the source. Streaming Incremental updates](https://spider.cloud/use-cases/rag/)
- [LLM knowledge ingestion Collect markdown with metadata, chunking and deduplication for a training dataset. Chunking Deduplication](https://spider.cloud/use-cases/ai-training/)
- [AI platforms Offer web data inside your product without running proxies, browsers or rendering yourself. Client SDKs Webhooks](https://spider.cloud/use-cases/ai-platforms/)

## For tracking the web over time

7 pages

- [Lead generation Pull contact details from company websites and export them to a CRM. Contact extraction CRM export](https://spider.cloud/use-cases/lead-generation/)
- [SEO and SERP tracking Track search positions by location in 199+ countries, returned as JSON. Rank tracking Bulk keywords](https://spider.cloud/use-cases/seo-tracking/)
- [Price monitoring Watch prices and stock on product pages and get a webhook when they change. JSON extraction Webhook alerts](https://spider.cloud/use-cases/price-monitoring/)
- [Market research Crawl competitor sites, news and industry publications in parallel, with AI extraction. Competitor monitoring](https://spider.cloud/use-cases/market-research/)
- [Content aggregation Turn many sites into one feed of clean content, with or without RSS. Readability extraction Metadata](https://spider.cloud/use-cases/content-aggregation/)
- [Website archiving Keep timestamped copies of a site, re-crawling only the pages that changed. Incremental crawling Full resources](https://spider.cloud/use-cases/website-archiving/)
- [Compliance monitoring Check terms and policy pages on a schedule and review the diff when one changes. Change detection Scheduled crawls](https://spider.cloud/use-cases/compliance-monitoring/)

See also

[Pricing Calculator](https://spider.cloud/compare/) [Why Spider](https://spider.cloud/why-spider/) [Integrations](https://spider.cloud/integrations/) [MCP Server](https://spider.cloud/mcp/)
