Discover every URL on a website without downloading full page content. The Links endpoint is faster and cheaper than a full crawl because it only collects URLs. Use it for sitemap generation, broken link detection, site structure analysis, or as a first pass before targeted scraping.
Key capabilities
- URL discovery without full content extraction
- Lower cost per page than Crawl or Scrape
- Faster response times for large sites
- Recursive link following with depth control
- Great for building sitemaps and link graphs
- Streaming output as URLs are discovered