Science & Research

Connected Papers Scraper

Extract paper similarity graphs, citation connections, prior and derivative works from Connected Papers visual research explorer. Powered by spider-browser .

Get Started Documentation

connectedpapers.com target

99.5% success rate

~4ms latency

Quick Start

Extract data in minutes

connected-papers-scraper.ts

import { SpiderBrowser } from "spider-browser";

const spider = new SpiderBrowser({
  apiKey: process.env.SPIDER_API_KEY!,
  stealth: 2,
});

await spider.connect();
const page = spider.page!;
await page.goto("https://www.connectedpapers.com/search?q=attention+mechanism");
await page.content(10000);

const data = await page.evaluate(`(() => {
  const papers = [];
  document.querySelectorAll("[data-testid='search-result'], .paper-search-result").forEach(el => {
    const title = el.querySelector("h3, .paper-title")?.textContent?.trim();
    const authors = el.querySelector(".paper-authors")?.textContent?.trim();
    const year = el.querySelector(".paper-year")?.textContent?.trim();
    const citations = el.querySelector(".paper-citations")?.textContent?.trim();
    const venue = el.querySelector(".paper-venue")?.textContent?.trim();
    if (title) papers.push({ title, authors, year, citations, venue });
  });
  return JSON.stringify({ total: papers.length, papers: papers.slice(0, 15) });
})()`);

console.log(JSON.parse(data));
await spider.close();

✓ ready to run | spider-browser | TypeScript

Fetch API

Structured data endpoint

Extract structured JSON from connectedpapers.com with a single POST request. AI-configured selectors, cached for fast repeat calls.

POST /fetch/connectedpapers.com/

Paper titleAuthorsYearCitationsGraph positionRelated papers

Try it Fetch API Docs

curl

curl -X POST https://api.spider.cloud/fetch/connectedpapers.com/ \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"return_format": "json"}'

Python

import requests

resp = requests.post(
    "https://api.spider.cloud/fetch/connectedpapers.com/",
    headers={
        "Authorization": "Bearer YOUR_API_KEY",
        "Content-Type": "application/json",
    },
    json={"return_format": "json"},
)
print(resp.json())

Node.js

const resp = await fetch("https://api.spider.cloud/fetch/connectedpapers.com/", {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({ return_format: "json" }),
});
const data = await resp.json();
console.log(data);

Extraction

Data you can extract

Paper titleAuthorsYearCitationsGraph positionRelated papersPrior worksDerivative works

Content

Paper extraction

Extract abstracts, citations, author data, and metadata from connectedpapers.com.

Parsing

Academic parsing

Clean extraction of LaTeX, references, and structured research data.

Scale

Bulk research

Process thousands of papers and citations for systematic reviews.

More Science & Research scrapers

arxiv.org

arXiv Scraper

Extract preprint papers, abstracts, author lists, and citation metadata from arXiv open-access research repository.

pubmed.ncbi.nlm.nih.gov

PubMed Scraper

Extract biomedical literature, abstracts, MeSH terms, and citation data from PubMed National Library of Medicine database.

researchgate.net

ResearchGate Scraper

Extract researcher profiles, publication lists, citation metrics, and project data from ResearchGate academic network.

Start scraping connectedpapers.com

Get your API key and start extracting data in minutes.

Get Started Free All Scrapers