Developer Quickstart
The Spider Cloud API provides a simple interface for web crawling, web scraping, and web screenshots to support your data-driven applications. Follow this quick start guide to get up and running quickly.
Create and Export an API Key
Create an API key in the dashboard, then store it securely as an environment variable. This key will allow you to access the Spider Cloud API securely. Store the key in a safe location, like a .zshrc
file or another text file on your computer. Once you’ve generated an API key, export it as an environment variable
in your terminal.
Export an envrionment variable on *nix systems
export SPIDER_API_KEY="your_api_key_here"
Make Your First API Request
Once you have your API key, you can start making requests to Spider Cloud API. Here are examples for web crawling, web scraping, and capturing web screenshots:
from spider import Spider
app = Spider()
params = {"limit":50}
crawl_result = app.crawl_url('https://mywebsite.com', params=params)