Getting started with Proxy Mode

What is Proxy Mode?

Spider offers a proxy front-end to the API. Instead of making POST requests, you route standard HTTP traffic through Spider’s proxy endpoint. This makes it easy to integrate with tools that support HTTP proxies. Costs and parameters are the same as the standard API.

JavaScript rendering is enabled by default. Disable it with render_js=False unless you need it. Connection details:

• HTTP address: proxy.spider.cloud:8888
• HTTPS address: proxy.spider.cloud:8889
• Username: YOUR-API-KEY
• Password: PARAMETERS

Important: Replace PARAMETERS with our supported API parameters. If you don’t know what to use, you can begin by using render_js=False. If you want to use multiple parameters, use & as a delimiter, example: render_js=False&premium_proxy=True.

As an alternative, you can use URLs like the following:

{
	"url": "http://proxy.spider.cloud:8888",
	"username": "YOUR-API-KEY",
	"password": "render_js=False&premium_proxy=True"
}

Features

• Proxy rotation: Datacenter and residential proxies with automatic failover
• Pricing: $1 per GB transferred
• Concurrency: Thousands of concurrent requests
• Caching: Repeated page requests return faster
• Anti-bot protection: Built-in measures to reduce blocks

See the full API reference for all supported parameters.

Usage

Create an API key, then configure your HTTP client to use Spider as a proxy. Pass API parameters in the password field:

import requests, os

# Proxy configuration
proxies = {
    'http': f"http://{os.getenv('SPIDER_API_KEY')}:country_code=na&premium_proxy=False@proxy.spider.cloud:8888",
    'https': f"https://{os.getenv('SPIDER_API_KEY')}:country_code=na&premium_proxy=False@proxy.spider.cloud:8889"
}

# Function to make a request through the proxy
def get_via_proxy(url):
    try:
        response = requests.get(url, proxies=proxies)
        response.raise_for_status()
        print('Response HTTP Status Code: ', response.status_code)
        print('Response HTTP Response Body: ', response.content)
        return response.text
    except requests.exceptions.RequestException as e:
        print(f"Error: {e}")
        return None

# Example usage
if __name__ == "__main__":
     get_via_proxy("https://www.example.com")
     get_via_proxy("https://www.example.com/community")

Note: The proxies available in Proxy Mode differ from those used in standard API crawling. The standard API proxies include AI-powered anti-bot capabilities and more advanced rotation logic.