FlowiseAI Integration

Use Spider as a document loader inside FlowiseAI to scrape and crawl pages directly from the visual builder.

Setup

Connect your Spider API key to FlowiseAI.

  1. Go to your Spider dashboard.
  2. Then go to the API Keys and create a new API key. You must add credits first before creating an API key.
  3. Copy the API key and paste it into the "Credential" field in the Spider node.
FlowiseAI Spider document loader form

Scrape and Crawl

Configure the Spider node in your FlowiseAI workflow.

  1. Choose either "Scrape" or "Crawl" in the mode dropdown.
  2. Enter the URL you want to scrape or crawl in the "Web Page URL" field.
  3. If you chose "Crawl", enter the maximum amount of pages you want to crawl in the "Limit" field. If no value or 0 is entered, the crawler will crawl all pages.
FlowiseAI Spider Integration builder workflow.
FlowiseAI Spider integration workflow