To help you get started with Spider, we’ll give you $200 in credits when you spend $100.
Pricing
Spider operates on a pay-per-use model for pricing. You only pay for the data you consume, with no extra costs once you stop using the service.
Need unique scraped data?
Contact us at support@spider.cloud to talk about what we can do.
Are you Enterprise?
Reach out to us at support@spider.cloud to learn more.
FAQ
Frequently asked questions about Spider
What is Spider?
Spider is a leading web crawling tool designed for speed and cost-effectiveness, supporting various data formats including LLM-ready markdown.
Why is my website not crawling?
Your crawl may fail if it requires JavaScript rendering. Try setting your request to 'chrome' to solve this issue.
Can you crawl all pages?
Yes, Spider accurately crawls all necessary content without needing a sitemap.
What formats can Spider convert web data into?
Spider outputs HTML, raw, text, and various markdown formats. It supports JSON, JSONL, CSV, and XML for API responses.
Is Spider suitable for large scraping projects?
Absolutely, Spider is ideal for large-scale data collection and offers a cost-effective dashboard for data management.
How can I try Spider?
Purchase credits for our cloud system or test the Open Source Spider engine to explore its capabilities.
Does it respect robots.txt?
Yes, compliance with robots.txt is default, but you can disable this if necessary.
Unable to get dynamic content?
If you are having trouble getting dynamic pages, try setting the request parameter to "chrome" or "smart." You may also need to set `disable_intercept` to allow third-party or external scripts to run.
Why is my crawl going slow?
If you are experiencing a slow crawl, it is most likely due to the robots.txt file for the website. The robots.txt file may have a crawl delay set, and we respect the delay up to 60 seconds.
Do you offer a Free Trial?
Yes, you can try out the service before being charged for free at checkout.