POST/unlimited/{scrape,crawl,links}

Unlimited API reference

Scrape, Crawl and Links for one monthly price. You buy how many requests can run at once instead of paying per request, and the price per concurrent request falls from $20 to $13 as you add more. See Unlimited pricing for the full table.

Get an API key

Routes

api.spider.cloud

POST https://api.spider.cloud/unlimited/scrape

alias /v1/unlimited/scrape, same parameters and response as /scrape

Same API key as every other endpoint. No per-request credits.

curl -X POST https://api.spider.cloud/unlimited/scrape \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "return_format": "markdown"
  }'

Concurrency

Concurrency is how many requests can be in flight at once across /unlimited/scrape, /unlimited/crawl and /unlimited/links. Buy 25 and 25 can run at any moment.

One crawl fetches up to 10 pages in parallel at every plan size. More concurrency means more crawls at once, not faster crawls.

Over the limit

Requests are not queued. When every slot is in use the API returns 429 Too Many Requests at once. Wait for the Retry-After interval, then retry with backoff.

Concurrency0 of 4 active

Each request holds one slot while it runs. One past the limit is refused, not queued.

Responseidle

Send a request to fill a slot.

Every response, 200 or 429, carries X-Concurrency-Limit and X-Concurrency-Active.

Proxy quality

Unlimited runs on a standard proxy tier. Pay as you go uses our highest quality proxies and gets through more often on the hardest targets, so use pay as you go where success rate matters most.

Restrictions

Pay as you go only

403

Not supported: social media automation (LinkedIn, Facebook), Google search and multi-account. Run these on pay as you go.

These return 403 with {"error":"payg_only_domain"} and use no concurrency. Send them to /scrape, /crawl or /links.

Subscription

403

Use the same API key as every other endpoint. The account needs an active Unlimited subscription, or the routes return 403 with {"error":"unlimited_plan_required"} or {"error":"unlimited_plan_inactive"}.

Unlimited is subject to fair use.