The Verge News Scraper
Spider read theverge.com in 113 ms without a browser and returned 410 lines of clean markdown.
AI bots started a religion — humans immediately followedCameron Faulkner, Brad Bourque and 1 moreSkylight Calendar Max review: a game-changer for busy parentsAn AI-powered Sidekick kicks this smart calendar display up a gear.I tested three Windows laptops in the MacBook Neo’s price range — there’s no contestThe cut corners of these Asus, Lenovo, and Acer laptops are glaringly obvious next to the Neo.The Fitbit Ace LTE is like a Nintendo smartwatch for kidsThis smartwatch for kids offers a fresher take on gamifying activity, while also enabling GPS tracking for parents.Anthropic is hiring an “Insider Risk Investigator.”The job’s responsibilities include conducting insider risk investigations, monitoring and triaging “external threats targeting employees,” and conducting “sensitive interviews of employees or other involved parties.”Trevor Noah is hosting Google’s Pixel 11 launch eventGoogle’s next hardware show will feature new gadgets and a lot of celebrities.Jony Ive’s first OpenAI gadget is reportedly a hockey puck-sized smart speakerThe “doughnut-shaped” device could cost more than $300 when it launches next year, according to Bloomberg.Two men convicted in France over the death of a Kick ‘humiliation streamer.’The *New York Times* reports that Owen Cenazandotti and Safine Hamadi were convicted of aggravated violence, incitement to hatred, and violence online related to Raphaël Graven’s death during a Kick livestream last year. Cenazandotti and Hamadi both face suspended prison terms, fines, and a six-month ban from posting on Kick.2 French Men Convicted in Case Related to Livestream DeathJennifer Pattison TuohyAug 6Today’s Vergecast: The next big thing in movies is 60-year-old IMAX tech.The same call, in code.
The capture above came back as markdown. These examples add a key, so you get browser rendering, proxies, and concurrency on theverge.com.
import { SpiderBrowser } from "spider-browser";
const spider = new SpiderBrowser({
apiKey: process.env.SPIDER_API_KEY!,
});
await spider.connect();
const page = spider.page!;
await page.goto("https://www.theverge.com/reviews");
// No selectors, no schema. Spider reads the page and names the fields.
const data = await page.scrape();
console.log(data);
await spider.close();import { SpiderBrowser } from "spider-browser";
const spider = new SpiderBrowser({
apiKey: process.env.SPIDER_API_KEY!,
stealth: 2,
});
await spider.connect();
const page = spider.page!;
await page.goto("https://www.theverge.com/reviews");
await page.content();
const data = await page.extractFields({
headline: "h2 a, h1",
author: "[rel='author'], a[href*='/authors/']",
date: "time[datetime]",
summary: "p:first-of-type, [class*='dek']",
score: "[class*='score'] span, [data-testid*='score']",
image: { selector: "figure img, picture source", attribute: "srcset" },
});
console.log(data);
await spider.close();Ready for volume? Get an API key →
Fields you can pull.
Spider names these from the page. The capture above came back as markdown; the same
call with return_format: "json" returns them as keys.
What theverge.com costs to scrape.
The capture above cost $0.001494 to fetch. Pricing is $1 per GB of pre-transformation bandwidth plus $0.001 per CPU minute, so a page like this one lands at a fraction of a cent. Failed requests are billed at $0.
- Free balance on signup
- No card required to test
- Balance never expires
Run it keyless, no account
More News scrapers.
Google News Scraper
Extract news articles, headlines, publication sources, and trending stories from Google News.
BBC News Scraper
Extract news articles, headlines, and publication data from BBC News.
CNN Scraper
Extract news articles, headlines, and video content data from CNN.
Start scraping theverge.com.
You already have the call. A key raises the rate limit and turns on browser rendering, proxies, and concurrency. Balance never expires, and top-ups go through secure checkout.