Extract structured JSON from a web page with one call
Send a URL and a map of CSS selectors to /scrape and get named fields back as JSON. One call returned 250 rows of four fields in about 1.5 seconds on two days.
How do I scrape a web page to JSON?
Send the URL and a css_extraction_map to /scrape. The map names each field and gives the CSS selectors that find it. The fields come back under css_extracted as JSON, and no model runs. On 2026-11-11 we ran this on two public practice pages, and we ran the same two calls again on 2026-11-17. One call returned all 250 rows of a listing page, four fields each, in 1.4 seconds for $0.00031.
Both pages are sandboxes built for scraping practice, with tidy class names. Your pages will be messier. The requests have the same shape either way, and to see what a plain fetch of your own page returns before you write selectors, paste the URL into the web scraper. It needs no key.
How do I get every row of a listing page as JSON?
Give each column a name and a selector. This call reads the country list on scrapethissite.com.
curl -s https://api.spider.cloud/scrape \
-H "Authorization: Bearer $SPIDER_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://www.scrapethissite.com/pages/simple/",
"css_extraction_map": {
"/": [
{ "name": "name", "selectors": [".country-name"] },
{ "name": "capital", "selectors": [".country-capital"] },
{ "name": "population", "selectors": [".country-population"] },
{ "name": "area_km2", "selectors": [".country-area"] }
]
}
}'The fields arrive under css_extracted, one array per name, in page order. Each array below has 250 entries and we show the first three. This is the response of 2026-11-11, cut to the fields this guide reads.
[
{
"url": "https://www.scrapethissite.com/pages/simple/",
"status": 200,
"css_extracted": {
"area_km2": ["468.0", "82880.0", "647500.0"],
"capital": ["Andorra la Vella", "Abu Dhabi", "Kabul"],
"name": ["Andorra", "United Arab Emirates", "Afghanistan"],
"population": ["84000", "4975593", "29121286"]
},
"costs": { "ai_cost": 0, "total_cost": 0.00030837 }
}
]That is 250 rows and 1,000 values from one call. The raw HTML in the same response has 250 country blocks, so the count is the page’s own. The call took 1.4 seconds and cost $0.00031, which is $0.31 per 1,000 pages of this size. ai_cost is 0 because no model ran.
Save the curl output with -o listing.json and a few lines of Python turn the arrays into rows.
import json
response = json.load(open("listing.json"))
data = response[0]["css_extracted"]
rows = [dict(zip(data, values)) for values in zip(*data.values())]
print(len(rows), rows[0])
# 250 {'area_km2': '468.0', 'capital': 'Andorra la Vella', 'name': 'Andorra', 'population': '84000'}How do I get a product page as JSON?
Use the same call with selectors for one product. This one reads a book on books.toscrape.com, a sandbox store.
curl -s https://api.spider.cloud/scrape \
-H "Authorization: Bearer $SPIDER_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html",
"css_extraction_map": {
"/": [
{ "name": "title", "selectors": [".product_main h1"] },
{ "name": "price", "selectors": [".product_main .price_color"] },
{ "name": "availability", "selectors": [".product_main .availability"] },
{ "name": "upc", "selectors": ["table.table-striped tr:nth-child(1) td"] }
]
}
}'[
{
"url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html",
"status": 200,
"css_extracted": {
"availability": ["In stock (22 available)"],
"price": ["£51.77"],
"title": ["A Light in the Attic"],
"upc": ["a897fe39b1053632"]
},
"costs": { "ai_cost": 0, "total_cost": 0.00002734 }
}
]That is the response of 2026-11-17, cut the same way. All four values match what the page prints. They are also only what the page prints. The price is the string "£51.77" and the stock count sits inside a sentence, so you still cast them. With the output saved to product.json:
import json
fields = json.load(open("product.json"))[0]["css_extracted"]
price = float(fields["price"][0].lstrip("£"))
stock = int(fields["availability"][0].split("(")[1].split()[0])
print(price, stock)
# 51.77 22What did each call cost, and how long did it take?
| Page | Values returned | Time, 2026-11-11 | Time, 2026-11-17 | Cost | Source |
|---|---|---|---|---|---|
| Listing, scrapethissite.com | 1,000 strings in 250 rows | 1.4 s | 1.5 s | $0.00031 | costs.total_cost in the response and the wall-clock time of the call |
| Product, books.toscrape.com | 4 strings | 0.6 s | 1.5 s | $0.00003 | costs.total_cost in the response and the wall-clock time of the call |
We timed each call once on each day, from a laptop, so read the times as two samples. The product call took 0.6 seconds the first day and 1.5 the second, and we don’t know which is typical. The listing call cost $0.00030837 the first day and $0.00030814 the second. The product call rounded to $0.00003 on both. At these costs 1,000 product pages are $0.03.
The response carries the page’s HTML in content beside the fields. On 2026-11-17 that was 203,705 characters for the listing page and 9,275 for the product page, about 22 times the size, and the listing call cost about 11 times as much. The printed responses keep two keys of costs. The full object also has bytes_transferred_cost, compute_cost, file_cost and transform_cost, and your own calls print theirs.
What can go wrong with selectors?
Check these before you trust a selector map on a site you don’t control.
An array can come back shorter than the row count. On 2026-11-17 we sent three selectors to the books.toscrape.com front page, which lists 20 books. title and price came back with 20 entries each. availability came back with one. All 20 books read “In stock”. Zip those three and you get one row, so check every array against the row count you expect. On the country page all four arrays had 250 entries and the HTML has 250 country blocks. Equal lengths still do not prove the rows line up, because two fields can each miss a different row. When a field repeats or can be missing, read the HTML in content and select inside each row yourself.
Selectors follow the markup. .country-name works until the site renames the class, so have your code fail when an array comes back empty.
The map key is a URL path, and it is stricter than it looks. "/" matches every page. Any other key matches only the page whose path is exactly that key, and a prefix such as "/products" matches nothing today. Key by "/" unless you mean one page. The scrape reference has the rule.
What if I have no selectors?
/ai/scrape takes a prompt and a JSON schema instead of selectors. A model reads the page and fills the fields, and it can cast "£51.77" to a number on the way. It needs an AI Studio subscription, and the AI docs show the request. A model can also return a value the page does not print. Selectors cannot, so check a model’s fields against the page before you store them. This guide measures the selector route only.
Run it on your page
Change the URL in either call, rename the fields and read css_extracted in the response. How a fetch works before any field is pulled out is in the web scraping guide. The calls need a key, and you can get one here.
Run this on a page you care about
The playground sends the request this page describes and shows you the response. Keyless runs work without an account and are rate limited.