Skip to main content

Built-in proxies for web scraping: median $0.0025 a complete page

Proxies for web scraping come built into every Spider request. On November 17, 2026, a complete category or listing page cost a median of $0.0025, proxy included.

Jeff Mendez 4 min read

Proxies for web scraping come built into every Spider request, and they rotate on their own. On November 17, 2026, a complete category or listing page from a large site cost a median of $0.0025 through the Spider scrape endpoint, proxy included. The calls carried no proxy settings. There is no proxy pool to buy and no separate proxy invoice. Complete pages came back in a median of 3.3 seconds, and each cost less than a cent.

The $0.0025 is the median costs.total_cost over complete pages, on the pay as you go meter. The whole run, every call included, cost $3.17 per 1,000 complete pages. The Unlimited plan charges by concurrency instead, from 2 requests in flight for $40 a month.

Do you need to buy proxies for web scraping?

Not with Spider. Every request gets a fresh IP by default, whether Spider fetches the page over plain HTTP or renders it in a browser. Each call in the run sent three fields, url, return_format and cache, and none of them is a proxy setting.

Rotating proxies explains rotation and sticky sessions, and residential proxies gives the price per GB. The web scraping guide covers the rest of a scrape, such as when a page needs a browser and what robots.txt allows.

What does a scrape call with a proxy look like?

The same as any other scrape call. The body names the page and the output format and nothing else, because the proxy needs no field.

curl -s https://api.spider.cloud/scrape \
  -H "Authorization: Bearer $SPIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "return_format": "markdown"}' \
  | jq '.[0] | {status, chars: (.content | length),
      costs: (.costs | with_entries(select(.key | endswith("_formatted") | not)))}'

Here is the same filter’s output for the run’s median page by cost, a marketplace category page:

{
  "status": 200,
  "chars": 63690,
  "costs": {
    "ai_cost": 0.0,
    "bytes_transferred_cost": 0.0000027746472,
    "compute_cost": 0.00001199608,
    "file_cost": 0.002433388104,
    "total_cost": 0.0024662388312,
    "transform_cost": 0.000018080000000000003
  }
}

The run also turned the cache off, so every page was a fresh fetch. The call above leaves the cache at its default.

There is no proxy line in costs. file_cost charges the page body at $1 per GB as fetched. bytes_transferred_cost charges response headers, and the scripts and images a browser loads, at the same rate. Spider prices the proxy into that rate, and the proxy a request goes through can raise it. Across the complete pages in the run, file_cost was 98% of the total.

What did each kind of page cost?

By median, retail category pages cost the least, $0.0021, and travel and property listings the most, $0.0036.

Kind of pagePagesMedianLowestHighestMedian length, characters
Retail category pages11$0.0021$0.0008$0.007621,837
Travel and property listings4$0.0036$0.0010$0.005537,417
Jobs, reviews and other listings6$0.0035$0.0028$0.006422,106
All complete pages21$0.0025$0.0008$0.007622,052

Run of 2026-11-17, starting at 16:02:49 UTC: one call to /scrape per page, with return_format set to markdown, the cache off and no proxy settings, five calls at a time. A page counts as complete when its status is 200, its markdown runs to at least 1,000 characters, it contains a word we chose for that page before the run, and no challenge text appears. respect_robots stayed at its default of true. Costs are costs.total_cost from each response. Medians are over complete pages, and the per 1,000 figure divides the whole run’s spend by the complete pages. Times run from sending the request to reading the response, on our client.

Two kinds of page had about the same median markdown length and different median costs. Retail category pages came to 21,837 characters and $0.0021. The jobs, reviews and other listings row came to 22,106 characters and $0.0035. file_cost counts the bytes of the page as fetched, before conversion to markdown, and some sites have a minimum price per page.

The run fetched each page once, on the afternoon of November 17, 2026, so the medians describe these 21 pages on that day.

How do you pick the country a request exits from?

Set country_code to the two-letter ISO code of the country you want the proxy to exit from, for example gb. The proxy locations page lists the countries by continent. locale sets the locale for the request, for example en-US.

Can you use your own proxies?

Yes. Pass your proxy in remote_proxy, as http://user:pass@host:port, and Spider halves file_cost and bytes_transferred_cost.

Can you use Spider’s proxy without the scrape API?

Yes. Point your client at proxy.spider.cloud on port 8888 for HTTP, 8889 for HTTPS or 8887 for SOCKS5. Your API key is the username, and parameters such as country_code=us go in the password. It costs $1 per GB transferred. Proxy mode does not render JavaScript, so send pages that need a browser through the scrape call. The proxy mode guide has the setup.

Create an API key and send the call above with one of your own pages in url.

Frequently asked questions

Do I need to buy proxies to scrape with Spider?

No. Every Spider request goes out through a proxy that rotates on its own, and there is no separate proxy invoice. In a run on November 17, 2026, a complete category or listing page cost a median of $0.0025 with the proxy included.

How much do proxies for web scraping cost on Spider?

The proxy has no line of its own in a response's costs. Its price is in the per GB rate of file_cost and bytes_transferred_cost. That rate is $1 per GB as fetched, and the proxy a request goes through can raise it. In the November 17, 2026 run, a complete page cost a median of $0.0025, and the whole run, every call included, came to $3.17 per 1,000 complete pages. The Unlimited plan charges by concurrency instead, from 2 requests in flight for $40 a month.

Can I choose the country the proxy exits from?

Yes. Set country_code to the two-letter ISO code of the country you want the proxy to exit from, for example gb. The proxy locations page lists the countries by continent.

Can I use my own proxies with Spider?

Yes. Send your proxy in remote_proxy as http://user:pass@host:port, and Spider halves file_cost and bytes_transferred_cost.

Can I use Spider's proxy without the scrape API?

Yes. Point your client at proxy.spider.cloud on port 8888 for HTTP, 8889 for HTTPS or 8887 for SOCKS5, with your API key as the username. It costs $1 per GB transferred and does not render JavaScript.

Sources

  1. Measured run, 2026-11-17, 16:02 UTC: 21 complete category and listing pages from large sites, most of them stores, travel sites and job boards, one call each to the Spider scrape endpoint with markdown output, the cache off and no proxy settings
  2. Spider OpenAPI file, the costs fields and the country_code, locale, remote_proxy and respect_robots parameters
  3. Spider pricing page, the pay as you go meter and the Unlimited plan
  4. Rotating proxies, how rotation and sticky sessions work
  5. Residential proxies, the price per GB
  6. Proxy locations, the countries by continent
  7. Proxy mode guide, proxy.spider.cloud over HTTP, HTTPS and SOCKS5

Keep reading

Web scraping to markdown: a 63-page docs site in one call

Web scraping to markdown, measured on November 17, 2026. One crawl call returned a 63-page docs site as markdown in 3.5 seconds for $0.0138.

How web scraping works: what a plain fetch gets and misses

How web scraping works, measured on November 16, 2026. A plain HTTP fetch returned 90% or more of a browser's words on 26 of 29 docs, news and product pages.

Is web scraping legal? What 67 top sites ask for in robots.txt

We asked 100 top-ranked domains for robots.txt on 2026-11-16. Of the 67 that served one, 13 hold Disallow: / for unnamed crawlers and 29 name an AI crawler.

Run this on a page you care about

The playground sends the request this page describes and shows you the response. Keyless runs work without an account and are rate limited.