PhiData Integration

Phidata is a framework for creating multi-modal agents with memory, tools, and reasoning capabilities, designed to collaborate in teams to solve problems. It features a user-friendly Agent UI for seamless interaction.

Install Spider client

Install the Python Spider client

pip install spider-client

Create an API key, then store it as an environment variable. This key will allow you to access the Spider API securely. If no API key is provided it looks for SPIDER_API_KEY in the env.

Then make sure to install Phidata, see instructions here.

Simple Scrape Example

The following agent will simply scrape a single URL passed on to the agent or whatever URL the agent comes across in its session. The agent will return the scraped data in markdown format.

Simple Scrape Example in Markdown

from phi.agent import Agent from phi.tools.spider import SpiderTools agent = Agent(tools=[SpiderTools()]) agent.print_response('Scrape https://www.cnn.com for the latest news"', markdown=True)
PhiData Agent Result

Example Usage for Search and Scrape

The following agent will run a search query to get the latest news in the USA and scrape the first search result. The agent will return the scraped data in markdown format.

Search and Scrape Results Example

from phi.agent import Agent from phi.tools.spider import SpiderTools agent = Agent(tools=[SpiderTools()]) agent.print_response('Scrape the first result from a search on "current news in the USA"', markdown=True)
PhiData search and scrape results

Toolkit Parameters

  • max_results (int): The maximum number of search results to return
  • url (str): The URL to be scraped or crawled

Toolkit Functions

  • search: Searches the web for the given query
  • scrape: Scrapes the given URL
  • crawl: Crawls the given URL