Verified

Google Scholar Search Scraper

Extract academic papers, citations, author profiles, and publication data from Google Scholar. Built on spider-browser .

Get started Docs

target
scholar.google.com: success rate
99.9%: latency
~4ms

Quick start

Extract data in minutes.

google-scholar-search-scraper.ts

import { SpiderBrowser } from "spider-browser";

const spider = new SpiderBrowser({
  apiKey: process.env.SPIDER_API_KEY!,
  stealth: 2,
});

await spider.connect();
const page = spider.page!;
await page.goto("https://scholar.google.com/scholar?q=machine+learning");
await page.content();

const data = await page.evaluate(`(() => {
  const papers = [];
  document.querySelectorAll("[data-cid]").forEach(el => {
    const title = el.querySelector("h3 a")?.textContent?.trim();
    const authors = (el.querySelector("h3 + div a") ||
      el.querySelector("h3 ~ div:nth-of-type(1)"))?.textContent?.trim();
    const snippet = el.querySelector("h3 ~ div:nth-of-type(2)")?.textContent?.trim();
    const citLink = [...(el.querySelectorAll("a") || [])].find(a => a.textContent?.includes("Cited by"));
    const citations = citLink?.textContent?.trim();
    if (title) papers.push({ title, authors, snippet, citations });
  });
  return JSON.stringify({ total: papers.length, papers: papers.slice(0, 10) });
})()`);

console.log(JSON.parse(data));
await spider.close();

ready to run · spider-browser · TypeScript

Fetch API

One endpoint for scholar.google.com.

Structured JSON from scholar.google.com with a single POST. AI-resolved selectors, cached on the first call.

POST /fetch/scholar.google.com/

Paper titleAuthorsPublicationYearCitationsSnippet

Try it Fetch docs

cURL

curl -X POST https://api.spider.cloud/fetch/scholar.google.com/ \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"return_format": "json"}'

Python

import requests

resp = requests.post(
    "https://api.spider.cloud/fetch/scholar.google.com/",
    headers={
        "Authorization": "Bearer YOUR_API_KEY",
        "Content-Type": "application/json",
    },
    json={"return_format": "json"},
)
print(resp.json())

Node.js

const resp = await fetch("https://api.spider.cloud/fetch/scholar.google.com/", {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({ return_format: "json" }),
});
const data = await resp.json();
console.log(data);

Extraction

Fields you can pull.

Paper titleAuthorsPublicationYearCitationsSnippetPDF linkRelated articles

Geo-Proxy

Location targeting

Access region-specific results from 199+ countries via residential proxies.

Stealth

Rate-limit bypass

Distributed requests with fingerprint rotation to avoid IP-based throttling.

Parsing

Structured SERP data

Clean extraction of rankings, snippets, and knowledge panels into JSON.

More Search scrapers.

google.com

Google Scraper

Extract Google Search results, knowledge panels, featured snippets, People Also Ask, and full SERP data programmatically. Geo-targeted proxies from 195+ countries for localized results.

play.google.com

Google Play Scraper

Extract app listings, ratings, reviews, download counts, and developer info from Google Play Store.

google.com/maps

Google Maps Scraper

Extract business listings, reviews, ratings, contact info, hours, and location data from Google Maps. Full browser rendering handles map interfaces and infinite scroll.

Start

Start scraping scholar.google.com.

Grab an API key and call the endpoint above. The first request resolves the config; every request after hits cache.

Get started free All scrapers