Web scraping cost per 1,000 pages, by kind of page

Note: This report is indexed at Spider Research. It first ran on February 11, 2026 and was rebuilt in November 2026. The figures are day 1 of a seven-day run taken on November 6, 2026. The tables will be refilled with the median and the daily range when all seven days are in.

Spider charges for what a page weighs. The pages that agents and retrieval pipelines read most are the light ones, and on day 1 of this run they were the cheap ones: $0.27 per 1,000 documentation pages, $0.33 per 1,000 articles, $0.37 per 1,000 pages fetched straight from a search.

There is no plan underneath those numbers. The first page and the millionth bill on the same two meters, so a prototype, a job that runs hard for a week and stops, or a workload that mixes search with reading never pays for an allowance it did not use.

I measured five kinds of page, each one a job that agents and RAG pipelines do, with the pages picked by a rule I wrote down before measuring anything. Here is what they cost, what the meter does for each, and what the setup looks like when the volume gets serious.

What each kind of page costs

Smart mode, which is the default, with markdown output, on November 6, 2026:

The spread from the first row to the last is more than fivefold, and it is page weight. Compute was under 2% of the charge in every run. The rest is bytes.

Reference surprised me. An element article on Wikipedia costs five times a documentation page because its HTML is that much larger, and the markdown that came back was about eleven times longer. You are paying for a page that holds more.

You pay for what the page weighs

The pricing guide lists two meters: $1 per GB of content transferred, counted before any transformation, and $0.0001 per minute of compute. That is the whole price list for /scrape.

A credit plan charges a short documentation page and a heavy storefront the same credit. A meter does not. On this run a documentation page cost a fifth of a product page, and for an agent that lands the right way round: documentation, articles and search results are what a model reads to answer a question, and they are the three cheapest rows.

No plan floor and no step

At 10,000 documentation pages a month the bill is $2.75. At 10,000 articles it is $3.30. At 10 million pages a month the same rates come to $2,700 for documentation and $14,800 for reference, and the volume averages about 230 requests a minute, well inside the 10,000 every account starts with. There is no allowance to size, no tier to outgrow, and no month where page 250,001 costs more than page 250,000. Plans step when you outgrow them; a meter has no step.

It holds in the other direction too. Spider credits never expire, so a month with a tenth of the traffic costs a tenth.

Search and fetch in one call

One /search request with fetch_page_content: true returns the result pages as markdown, ready for a model. I sent five developer questions, each asking for 5 results, and the pages came back at $0.37 per 1,000, counting the page charges in the response. A research agent does not need a second vendor for the reading step.

The smart default costs the same as plain HTTP

The request field has three values. http fetches the page and runs no JavaScript. browser renders it. smart, the default, tries HTTP and moves to a browser only when the page needs it. There is no multiplier for JavaScript rendering, and the run shows it:

Same price on every set, within a cent. I checked that this is the meter and not a cached repeat: ten pages sent with cache: false cost $0.85 per 1,000 in both modes. So there is no rendering surcharge to manage and no reason to pick http by hand. Leave the default on and the meter does the choosing per page.

Forcing browser on every page is the one setting that changes the bill, because a browser fetches the scripts and data calls a page pulls in and those bytes are on the meter. Use it when you want a page rendered no matter what, and let smart handle the rest.

Two output and blocking options, filter_output_main_only and the first-party blocking flags, did not change the bill. The meter follows bytes fetched, so what you drop from the output after the fetch is free either way.

Markdown out, at a fraction of the tokens

Every page in this run came back as markdown. In a separate measurement on ten public pages, markdown cut the token count 15x at the median against raw HTML, with a range of 2.9x to 47x. So the bytes you pay Spider for are the bytes of the page, and the tokens you pay your model for are the markdown. The second bill is the one that usually dominates, and it is the one the format shrinks.

One key, four routes

Every number here came from one API key and two routes, /scrape and /search. The same key and the same balance cover /crawl for a whole site with a limit, and /screenshot for a page image, per the API reference. One balance, one meter, markdown from each.

What that makes possible

Three scenarios, computed from the measured cost per page above. They are not customer results.

None of the three needs a plan picked in advance, and each one can double or halve next month and the bill follows.

What the best looks like at scale

The meter prices a page. The limits around it decide how many pages you can move at once, and they are on the public pages, so here they are with their numbers.

Every account starts at 10,000 requests a minute, from the first paid request. The pricing page states the limit, and llms.txt lists the levels above it: pay as you go rises through four levels to 80,000 requests a minute as the account uses more. You do not ask for it and you do not move to another plan for it.

For steady high volume, Unlimited bills by concurrency instead of pages. Per the pricing page, you pick how many requests may be in flight at once, from 2 at $40 a month up to 1,538, and run unlimited requests through them for a flat price. When every seat is busy the next request gets an immediate 429 with a Retry-After header. Spider queues nothing and bills nothing extra, so a pipeline that knows its parallelism knows its bill before the month starts.

The browser product scales the same way, by sessions. Per the browser page, an account holds up to 1,000 open sessions and opens up to 100 new ones a minute; each session takes 500 commands a second sustained, with bursts to 1,000, and the 999-URL run held 25 sessions open for 19 minutes with no failures.

All of it runs on spider, the open source engine, written in Rust under the MIT licence, at 2.8k stars when I read the page on November 6, 2026. The engine that priced these 125 pages is the one you can read.

How the pages were picked

Each case has 25 pages, chosen by one rule and applied once. I did not look at page size or cost before fixing the lists, and I did not swap a page out afterwards.

The rules have side effects, and I kept them. Astro’s sitemap lists its Arabic pages first, so those are the five Astro pages. Three of the 25 product pages are gift cards.

Every request went to the API once per run with markdown output, and respect_robots stayed at its default of true. The cost is the costs.total_cost field on each response. A page counts when the response is a 200 with at least 1,000 characters of markdown, and every page in the four fixed lists came back on day 1, 25 of 25 each. Cost per 1,000 is the charge for the pages divided by how many. The same case file runs every day for seven days.

Price your own pages

My 25 pages per case are not yours. Send a hundred of your own URLs in smart mode, add up the costs field, and you have your own row for the table above in about a minute. Create an API key to run it.