Google Scholar Science Scraper
Extract academic papers, citation counts, author profiles, and related articles from Google Scholar search results. Built on spider-browser .
- target
- scholar.google.com
- success rate
- 99.9%
- latency
- ~4ms
/fetch/scholar.google.com/ curl -X POST https://api.spider.cloud/fetch/scholar.google.com/ \ -H "Authorization: Bearer $SPIDER_API_KEY" \ -H "Content-Type: application/json" \ -d '{"return_format": "json"}'
{
"url": "https://scholar.google.com/",
"status": 200,
"data": {
"paper_title": "string",
"authors": "string",
"snippet": "string",
"citation_count": "string",
"year": "string",
"source": "string",
"pdf_link": "string",
"related_articles": "string"
}
}# Google Scholar Science Scraper
**Paper title**: string
**Authors**: string
**Snippet**: string
**Citation count**: string
**Year**: string
**Source**: string
**PDF link**: string
**Related articles**: stringExtract data in minutes.
Structured JSON from scholar.google.com with a single POST. AI-resolved selectors, cached on the first call.
import { SpiderBrowser } from "spider-browser";
const spider = new SpiderBrowser({
apiKey: process.env.SPIDER_API_KEY!,
stealth: 2,
});
await spider.connect();
const page = spider.page!;
await page.goto("https://scholar.google.com/scholar?q=attention+is+all+you+need");
await page.content(10000);
const data = await page.evaluate(`(() => {
const papers = [];
document.querySelectorAll(".gs_r.gs_or.gs_scl").forEach(el => {
const title = el.querySelector(".gs_rt a")?.textContent?.trim();
const authors = el.querySelector(".gs_a")?.textContent?.trim();
const snippet = el.querySelector(".gs_rs")?.textContent?.trim();
const citations = el.querySelector(".gs_fl a:nth-child(3)")?.textContent?.trim();
const link = el.querySelector(".gs_rt a")?.getAttribute("href");
if (title) papers.push({ title, authors, snippet: snippet?.slice(0, 200), citations, link });
});
return JSON.stringify({ total: papers.length, papers: papers.slice(0, 10) });
})()`);
console.log(JSON.parse(data));
await spider.close();curl -X POST https://api.spider.cloud/fetch/scholar.google.com/ \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"return_format": "json"}'import requests
resp = requests.post(
"https://api.spider.cloud/fetch/scholar.google.com/",
headers={
"Authorization": "Bearer YOUR_API_KEY",
"Content-Type": "application/json",
},
json={"return_format": "json"},
)
print(resp.json())const resp = await fetch("https://api.spider.cloud/fetch/scholar.google.com/", {
method: "POST",
headers: {
"Authorization": "Bearer YOUR_API_KEY",
"Content-Type": "application/json",
},
body: JSON.stringify({ return_format: "json" }),
});
const data = await resp.json();
console.log(data);Fields you can pull.
Paper extraction
Extract abstracts, citations, author data, and metadata from scholar.google.com.
Academic parsing
Clean extraction of LaTeX, references, and structured research data.
Bulk research
Process thousands of papers and citations for systematic reviews.
More Science & Research scrapers.
arXiv Scraper
Extract preprint papers, abstracts, author lists, and citation metadata from arXiv open-access research repository.
PubMed Scraper
Extract biomedical literature, abstracts, MeSH terms, and citation data from PubMed National Library of Medicine database.
ResearchGate Scraper
Extract researcher profiles, publication lists, citation metrics, and project data from ResearchGate academic network.
Start scraping scholar.google.com.
Grab an API key and call the endpoint above. The first request resolves the config; every request after hits cache.