Hex Packages Scraper

A directory listing of all packages available on the Hex package manager for the Erlang ecosystem. Built on spider-browser .

Get started Docs

target
hex.pm: success rate
99.9%: latency
~4ms

POST /fetch/hex.pm/

return_format

curl -X POST https://api.spider.cloud/fetch/hex.pm/ \
  -H "Authorization: Bearer $SPIDER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"return_format": "json"}'

200 OK · cache hit · response shape

{
  "url": "https://hex.pm/",
  "status": 200,
  "data": {
    "about_hex": "string",
    "about_hex_link": "string",
    "about_hex_title": "string",
    "help": "string",
    "help_link": "string",
    "help_title": "string",
    "new_package_link": "string",
    "new_package_title": "string"
  }
}

# Hex Packages Scraper

**About Hex**: string
**About Hex Link**: string
**About Hex Title**: string
**Help**: string
**Help Link**: string
**Help Title**: string
**New Package Link**: string
**New Package Title**: string

Quick start

Extract data in minutes.

Structured JSON from hex.pm with a single POST. AI-resolved selectors, cached on the first call.

hex-pm-scraper.ts

import { SpiderBrowser } from "spider-browser";

const spider = new SpiderBrowser({
  apiKey: process.env.SPIDER_API_KEY!,
});

await spider.connect();
const page = spider.page!;
await page.goto("https://www.hex.pm");

const data = await page.extractFields({
  about_hex: "div.footer-nav ul li",
  about_hex_link: "div.footer-nav ul li a",
  about_hex_title: "div.footer-nav h4",
  help: "div.footer-nav ul li",
  help_link: "div.footer-nav ul li a",
  help_title: "div.footer-nav h4",
});

console.log(data);
await spider.close();

curl -X POST https://api.spider.cloud/fetch/hex.pm/ \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"return_format": "json"}'

import requests

resp = requests.post(
    "https://api.spider.cloud/fetch/hex.pm/",
    headers={
        "Authorization": "Bearer YOUR_API_KEY",
        "Content-Type": "application/json",
    },
    json={"return_format": "json"},
)
print(resp.json())

const resp = await fetch("https://api.spider.cloud/fetch/hex.pm/", {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({ return_format: "json" }),
});
const data = await resp.json();
console.log(data);

ready to run · spider-browser · TypeScript

Fetch docs →

Extraction

Fields you can pull.

About HexAbout Hex LinkAbout Hex TitleHelpHelp LinkHelp TitleNew Package LinkNew Package TitlePackage LinkPackage TitlePolicies And TermsPolicies And Terms LinkPolicies And Terms TitleRecently Updated LinkRecently Updated TitleSearch ButtonSearch LabelTitleUsing With Box ElixirUsing With Box Erlang

Rendering