Spider vs. ScraperAPI: What Credit Multipliers Actually Cost You

ScraperAPI has been around for years and does one thing: you send a URL, it rotates proxies and renders JavaScript, you get HTML back. Straightforward, but two things start hurting at scale: the credit multiplier system, and the fact that HTML is the only output you get.

ScraperAPI’s credit multiplier system

Each API request eats a different number of credits depending on the features you turn on:

Source: ScraperAPI docs — credits and request costs

Most production scraping targets sites with bot protection. If you need Cloudflare bypass with rendering, that’s 75 credits per request:

Source: scraperapi.com/pricing

ScraperAPI has a free plan, but it’s 1,000 credits per month with 5 concurrent connections, enough to test the API, not enough to run a workload. Credits expire at the end of each billing cycle.

Spider’s pricing is flat: you pay for what you transfer and compute, regardless of what anti-bot system a site uses. The production average is about $0.48 per 1,000 pages (pricing). No multipliers, no expiring credits.

At 100,000 pages per month scraping protected sites:

• ScraperAPI (Business): ~$748
• Spider: ~$65

The output format gap

This is where ScraperAPI falls shortest for modern workflows. It returns HTML. Always. You parse it yourself.

That was fine five years ago when the scraping pipeline ended at a database. Today, most teams are feeding scraped content into LLM pipelines: RAG systems, fine-tuning datasets, AI agents. HTML is the worst possible format for that. You end up bolting on a parsing step, a cleaning step, and a chunking step before the data is usable.

Spider returns the format you need directly from the API:

import requests

# Get markdown — ready for LLM consumption
response = requests.post(
    "https://api.spider.cloud/crawl",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={
        "url": "https://docs.example.com/api-reference",
        "return_format": "markdown",
    },
)
pages = response.json()

# Each page comes back as clean markdown
for page in pages:
    # Feed directly into your embedding pipeline
    embeddings = embed(page["content"])
    vector_db.upsert(page["url"], embeddings)

That same workflow with ScraperAPI requires BeautifulSoup or Cheerio to parse the HTML, a text extraction step to strip tags and scripts, a markdown converter if your LLM works better with structured text, and a chunking strategy. Four extra steps before you get to the same place.

Spider also supports JSON, plain text, XML, CSV, and JSONL. Pick the format that fits your pipeline.

Browser automation

ScraperAPI is a URL-in, HTML-out pipe. There’s no way to log into a site, click through pagination, fill forms, or execute JavaScript against the live DOM.

Spider Browser gives you a remote browser session over WebSocket. The use case that comes up most often is scraping behind authentication:

import { SpiderBrowser } from "spider-browser";

const spider = new SpiderBrowser({
  apiKey: process.env.SPIDER_API_KEY,
});

await spider.init();

await spider.page.goto("https://app.example.com/login");
await spider.page.fill("#email", "user@example.com");
await spider.page.fill("#password", "password");
await spider.page.click("#submit");
await spider.page.waitForSelector(".dashboard");

// Use natural language to extract — no CSS selectors needed
const data = await spider.page.extract(
  "Get the user's name, account balance, and most recent 5 orders"
);

await spider.close();

With ScraperAPI, you’d need to run your own Puppeteer or Playwright instance, manage your own anti-detection, and route it through their proxy, rebuilding half the infrastructure you’re paying them to provide.

ScraperAPI’s structured data endpoints

ScraperAPI does offer pre-built JSON endpoints for Amazon and Google. If those two domains cover your use case, those endpoints save parsing work and they’re well-maintained.

Spider’s approach is broader. The extract() method works on any site: describe what you want in English, get JSON back. Not limited to specific domains, and the extraction runs on the page’s live DOM, so it works with dynamically loaded content that a static HTML parse would miss.

Throughput and reliability

ScraperAPI response times vary widely. Independent tests report ~80% success rates on search engine results pages with response times sometimes exceeding 40 seconds.

Spider’s production numbers: 99.9% success rate, sub-second median response time, no per-plan concurrency caps.

The bottom line

ScraperAPI is a clean, simple API that does what it says. If your workload is unprotected pages at 1 credit each, it works fine.

The two gaps that matter are the 75x credit multiplier on protected sites and the HTML-only output. If you’re scraping protected sites at volume, the multiplier makes Spider roughly 11x cheaper on ScraperAPI’s best plan. And if you’re building an LLM pipeline, Spider’s native markdown output cuts out the parsing and cleaning steps that HTML-only APIs force you to build.

• Spider pricing. Free credits to start
• Spider API docs