AI Scrape
Extract structured data using plain English
AI Scrape transforms any webpage into structured data using natural language. Simply describe what data you need, and the AI identifies elements, handles variations, and returns clean, structured results.
POST /ai/scrapeFull API DocsFeatures
Plain language data extraction
Automatic schema detection
Handles page variations
Multiple output formats
Context-aware parsing
Nested data extraction
Example Use Cases
Product details
"Extract product name, price, rating, and description"
Article content
"Get the main article text, author, and publish date"
Contact info
"Find all email addresses and phone numbers"
Table data
"Extract the pricing table as structured data"
Parameters
| Name | Type | Required | Description |
|---|---|---|---|
| url | string | required | URL to scrape |
| prompt | string | required | Description of data to extract |
| return_format | string | optional | Output format: json, markdown, csv |
Code Examples
cURL
curl -X POST https://api.spider.cloud/ai/scrape \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://example.com/product",
"prompt": "Extract product name, price, and all reviews"
}'Python
from spider import Spider
spider = Spider()
result = spider.ai_scrape(
url="https://example.com/product",
prompt="Extract product name, price, and all reviews"
)
print(result)JavaScript
import { Spider } from '@spider-cloud/spider-client';
const spider = new Spider();
const result = await spider.aiScrape({
url: 'https://example.com/product',
prompt: 'Extract product name, price, and all reviews'
});
console.log(result);