Spider vs. ScrapFly: Understanding What Credits Really Cost

ScrapFly is a mature scraping API with a solid feature set: anti-bot bypass, browser rendering, proxy rotation, and extraction tools. It competes with Spider on most capabilities.

The catch is how you pay for those capabilities. ScrapFly uses a credit multiplier system where a single request can consume anywhere from 1 to 30+ credits depending on which features you enable. Spider charges flat bandwidth + compute regardless of features. This post digs into the pricing math, compares features, and lays out where each tool fits.

How ScrapFly’s credit system works

Every ScrapFly request starts with a base cost determined by proxy tier, with add-ons stacking:

These costs stack. A request using residential proxies with browser rendering costs 25 + 5 = 30 credits. ASP doesn’t carry a fixed surcharge; it dynamically upgrades your proxy pool and may trigger retries, so the per-request cost varies by target site difficulty.

That 30x multiplier matters. On a plan that advertises a million credits, residential + render requests give you roughly 33,000 actual page fetches. For datacenter-only requests (unprotected sites), the 1-credit base rate applies and you get the full credit count.

ScrapFly’s current plans

Source: scrapfly.io/pricing

Credits expire monthly. Unused allocation doesn’t carry over. When you exhaust your quota, ScrapFly switches to pay-as-you-go overage pricing.

The gap between “advertised credits” and “actual requests” depends entirely on your targets. If your workload is unprotected sites using datacenter proxies, you get close to the headline credit count. If you’re hitting protected sites with residential proxies and rendering, a $250 Startup plan’s 2.5 million credits becomes roughly 83,000 actual requests.

How Spider’s pricing works

Spider charges two things:

• Bandwidth: $1 per GB transferred
• Compute: $0.001 per minute of processing time

No multipliers. Anti-bot bypass, browser rendering, proxy rotation, and every output format are all included in the base cost. The production average works out to about $0.65 per 1,000 pages whether you’re scraping static HTML or heavily protected e-commerce sites. Spider credits never expire.

Running the numbers side by side

Assuming production scraping on protected sites (the common case that triggers ScrapFly’s residential + render tier):

For workloads hitting protected sites, Spider is cheaper at every volume level. For datacenter-only workloads on unprotected sites, ScrapFly’s per-page cost can be lower (200K datacenter requests for $30 = ~$0.15/1K pages vs. Spider’s ~$0.65/1K). The gap favors Spider most at lower volumes where ScrapFly’s subscription floor dominates.

A real-world example

A price monitoring system checking 50,000 product pages weekly (200K/month) across Cloudflare-protected e-commerce sites:

ScrapFly: Residential proxies + rendering needed → 30 credits per request → 200K × 30 = 6M credits/month → Enterprise plan ($500/month) covers 5.5M, so you’d need overage. Minimum $500+/month.

Spider: 200K pages × ~$0.00065/page = ~$130/month. Same price whether you need rendering, proxies, or both.

The inability to predict which multiplier tier ScrapFly will apply per request makes budgeting harder. Spider’s linear pricing removes that variable.

Performance

ScrapFly reports a 99.99% success rate on their site. Independent benchmarks from Scrapeway measure closer to 98% on protected sites with default settings. Spider’s measured success rate from our 1,000-URL benchmark was 99.9%, though that’s our own benchmark, not independently verified. Take all vendor-reported success rates with appropriate skepticism, including ours.

On a 100,000-page workload, even small gaps in success rate translate to hundreds or thousands of missing pages in your data pipeline.

Spider’s Rust engine processes static HTML at 182 pages/second and sustains 74 pages/second across a mixed corpus of static, JS-rendered, and anti-bot-protected sites. ScrapFly doesn’t publish equivalent throughput benchmarks, so run your own workload on both if throughput is a deciding factor.

Feature-by-feature

Where ScrapFly has a distinct advantage

CSS/XPath extraction rules. ScrapFly’s extraction API supports CSS selectors, XPath, regex, JMESPath, and specialized extractors (price, email, links), all running server-side. For structured data extraction on targets with stable HTML, this is powerful and doesn’t require an LLM.

Session management. Explicit session support for maintaining cookies and state across multiple requests to the same site. Spider handles this implicitly through browser automation, but ScrapFly’s approach is simpler for basic cookie persistence.

Broader SDK coverage for scraping. ScrapFly has a dedicated Scrapy integration, which matters if your team already uses the Scrapy ecosystem.

A quick code comparison

Spider

import requests

response = requests.post(
    "https://api.spider.cloud/crawl",
    headers={
        "Authorization": "Bearer YOUR_API_KEY",
        "Content-Type": "application/json",
    },
    json={
        "url": "https://example.com",
        "return_format": "markdown",
        "country_code": "us",
    },
)
# Flat cost. Same price with or without anti-bot.
content = response.json()[0]["content"]

ScrapFly

import requests

response = requests.get(
    "https://api.scrapfly.io/scrape",
    params={
        "key": "YOUR_API_KEY",
        "url": "https://example.com",
        "asp": "true",
        "render_js": "true",
        "country": "us",
    },
)
# 30+ credits consumed (residential + render)
html = response.json()["result"]["content"]

The API calls look similar. The billing behind them is where they diverge: Spider returns markdown at flat cost; ScrapFly returns content inside a JSON wrapper, with cost determined by the features you enabled.

Choosing between them

ScrapFly is a reasonable choice if your workload is mostly unprotected sites where datacenter proxies suffice (1-credit requests keep costs low), you need CSS/XPath extraction rules on targets with stable HTML, or you need the Scrapy integration.

Spider is the better fit if your budget needs to be predictable (no multipliers to estimate), your targets include protected sites (anti-bot bypass at no extra credit cost), you need to crawl multi-page sites rather than scrape individual URLs, you want non-expiring credits, or you need live browser automation beyond JavaScript injection.

For teams where both scraping and crawling are in scope, Spider’s native multi-page crawling and flat pricing model mean fewer surprises as your workload grows.

Spider pricing breakdown