arxiv.org · HTTP 200

Scrape arxiv.org

Get any arxiv.org page back as clean markdown, no account needed.

Measured on this page: 1,078 ms · $0.000075 · 186 lines

Get a free API key Free balance on signup. No card.

Last capture arxiv.org/html/2403.04665v2.md markdown · 186 lines

Tables should be formatted as shown in the example below with no column lines, unless these are essential for clarifying the content of the table. Row lines can be used to distinguish the column headings from the content of the table.tables must be numbered and cited within the text in strict numerical order. Table captions must be above the table and in 10 pt font.Equations should fit into a two-column print format and be single spaced.When writing mathematics, avoid confusion between characters that could be mistaken for one another, e.g. the letter ‘l’ and the number one.Vectors and matrices should be in bold italic and variables in italic.If your paper contains superscripts or subscripts, take special care to ensure that the positioning of the characters is unambiguous.Exponential expressions should be written using superscript notation, i.e. 5×1035superscript1035\times 10^{3}5 × 10 start_POSTSUPERSCRIPT 3 end_POSTSUPERSCRIPT not 5E03.A multiplication symbol should be used, not a dot.Refer to equations using round brackets e.g. (1)Use SI (MKS) units only and avoid spelling the unit in full instead of using the shortened notation e.g. use kJ not kilo Joules.If for any reason you must use mixed units, the units used for each quantity in an equation must be stated.Place a zero before decimal points: “0.10” do not put “.10”.Submissions should always include the following sections: an abstract; an introduction; a conclusion and a references section. If any of the above sections are not included the paper may be asked to add the relevant section or be rejected.Acknowledgements should be placed after the conclusion and before the references section. Details of grants, financial aid and other special assistance should be noted.You should number your references sequentially throughout the text, and eachreference should be individually numbered and enclosed in square brackets

The same call, in code

The run above is the whole API. A key adds browser rendering, proxies, and concurrency on arxiv.org.

arxiv-scraper.ts

import { SpiderBrowser } from "spider-browser";

const spider = new SpiderBrowser({
  apiKey: process.env.SPIDER_API_KEY!,
});

await spider.connect();
const page = spider.page!;
await page.goto("https://arxiv.org/list/cs.AI/recent");

// No selectors, no schema. Spider reads the page and names the fields.
const data = await page.scrape();

console.log(data);
await spider.close();

ready to run · spider-browser, no selectors npm i spider-browser

Fields and cost

The capture above cost $0.000075 to fetch. You pay $1 per GB plus $0.001 per CPU minute, and failed requests cost $0.

Paper titleAuthorsAbstractCategoriesSubmission datePDF linkDOIComments

Spider names these fields from the page. The keyless call returns the page as markdown. Structured JSON comes from the /fetch lane, which needs a key.