Extract web data with natural language.
Describe what you need in plain English. Spider's AI endpoints crawl, scrape, search, and automate the web — structured JSON back, ready for your application.
Prompt in. JSON out.
Send a prompt
Pass a URL and describe what you want to extract in plain English. No selectors, no XPath, no parsers.
AI extracts
Spider renders the page in a real browser, then runs your prompt against the DOM to find matching data.
Get JSON
Receive clean structured output, ready for your database, API, or AI pipeline.
Five ways to ask.
Each endpoint targets one extraction pattern. All five share the same prompt-driven request shape and your existing API key.
AI Crawl
POST /ai/crawlCrawl a site with a natural-language goal. The crawler follows links toward whatever you described, not whatever a sitemap says.
AI Scrape
POST /ai/scrapeHand the page a prompt instead of selectors. Spider renders it, runs your prompt over the DOM, and returns the fields you asked for as JSON.
AI Search
POST /ai/searchSearch the web with an intent, not a keyword. Results are ranked by relevance to the question, not just lexical match.
AI Browser
POST /ai/browserDrive a real browser with English instructions. Spider figures out the clicks, fills, and waits without you writing selectors.
AI Links
POST /ai/linksPull only the links you want — product pages, docs, articles — by describing them, not by writing a CSS selector.
One field, structured results.
Add a prompt field to any AI request. Works with every Spider SDK and the same API key you already issued.
$ curl -X POST \
https://api.spider.cloud/ai/scrape \
-H "Authorization: Bearer $API_KEY" \
-d '{
"url": "https://example.com",
"prompt": "Extract product names and prices"
}'[
{ "name": "Widget Pro", "price": "$49.99" },
{ "name": "Gadget Plus", "price": "$29.99" },
{ "name": "Tool Max", "price": "$79.99" }
]Built for the people writing the code.
No selectors needed
Drop XPath, CSS selectors, and brittle scraper scripts. Describe your data in English and let the model do the parsing.
Works with any site
JavaScript-rendered pages, SPAs, dynamic content. Spider drives a real browser before AI extraction runs.
Schema-shaped output
Get JSON with the exact fields you asked for. Pin a schema, or let the model infer the shape from the prompt.
Scales on demand
From a single page to a million. Same API, built-in concurrency, rate limiting, and retries.
Ship your first prompt in minutes.
Sign up, grab an API key, and send your first AI extraction request. No selectors to maintain, no infrastructure to run.