Fetch directory

One endpoint per domain.

Alpha

Per-domain fetch endpoints. The CSS selectors and extraction schema are AI-resolved on the first request, then cached and reused.

API docs All scrapers

01 · How it works

AI resolves the config once. Cache serves the rest.

Request a domain

POST to /fetch/<domain>/<path> with your API key. No per-site configuration.

AI discovers config

On the first request, AI analyses the page and resolves the CSS selectors, extraction schema, and request settings.

Cached & reused

The resolved config is validated, cached in memory + the database, and reused on every subsequent request.

02 · Quick start

One `POST`, structured JSON back.

news.ycombinator.com

curl -X POST https://api.spider.cloud/fetch/news.ycombinator.com/ \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"return_format": "json"}'

returns structured JSON · fetch API

03 · Directory

Available endpoints.

Start

Start extracting structured data.

Get an API key and call any directory endpoint. The first request discovers; every request after that hits cache.

Get started free All scrapers