Libraries

Official SDKs for Python, JavaScript, and Rust. Each library wraps the Spider API with typed methods for crawling, scraping, and search.

Python

Install the Python SDK with pip:

pip install spider_client

Crawl website with Python

from spider import Spider
app = Spider()
params = {"limit":5}
crawl_result = app.crawl_url('https://example.com', params=params)

JavaScript / TypeScript

Install the JS/TS SDK — works with Node.js, Deno, and Bun:

npm install @spider-cloud/spider-client

Crawl website with Javascript

import { Spider } from "@spider-cloud/spider-client";
const app = new Spider();
const params = {"limit":5};
const result = await app.crawlUrl("https://example.com", params)

Rust

Add the Rust SDK with cargo:

cargo add spider-client

Crawl website with Rust

let spider = spider_client::Spider::new(None).expect("API required");
let crawler_params = RequestParams { limit: Some(5), ..Default::default() };
let crawl_result = spider.crawl_url("https://example.com", Some(crawler_params), false, "application/json", None).await.expect("Failed to crawl the URL");

CLI

Install the CLI with cargo:

cargo install spider-cloud-cli

spider-cloud-cli auth --api_key YOUR_API_KEY

Now after authentication run any of the commands:

spider-cloud-cli crawl --url http://example.com --limit 5

Use spider-cloud-cli --help in terminal to see all available commands.