Twitch Scraper
Spider read twitch.tv in 1.4 s without a browser and returned 118 lines of clean markdown, including sections like "Clips", "Getting a game’s clips" and "Getting a specific clip".
# ClipsThe clips are returned in descending order of views. Because the request could return thousands of results to page through, you could use the *first* query parameter to get the top 10 clips with the most views. Or, to get clips captured within a specific date range, use the *started_at* and *ended_at* query parameters (see Getting clips captured within a specific date range).### Getting a game’s clipsTo get clips captured from a specific game, use the *game_id* query parameter. The following example shows how to get all clips captured from broadcasters that were playing Minecraft.`curl -X GET 'https://api.twitch.tv/helix/clips?game_id=27471' \To get a game’s ID, use the Search Categories API.### Getting a specific clipTo get specific clips, use the *id* query parameter. Include the *id* query parameter for each ID that you specify. You may specify a maximum of 100 IDs.`curl -X GET 'https://api.twitch.tv/helix/clips?id=SpillYummyPigeonPieHuhu&id=DillDiligentTireOneHand' \If you specify the *id* query parameter, the API ignores the pagination query parameters and date range parameters.### Getting clips captured within a specific date rangeBy default, Get Clips returns all clips that were captured for the specified game or broadcaster. If you’re only interested in clips for a specific date range like the last week or yesterday, use the *started_at* and *ended_at* query parameters. Dates are UTC.The following example shows how to get clips that were captured in the last week from the specified broadcaster’s streams.`curl -X GET 'https://api.twitch.tv/helix/clips?broadcaster_id=123456&started_at=2022-07-03T00:00:00Z&ended_at=2022-07-09T00:00:00Z' \The *ended_at* parameter is optional. If you don’t specify it, the date range is one week from the start date. The following example shows how the above request could have been written:`curl -X GET 'https://api.twitch.tv/helix/clips?broadcaster_id=123456&started_at=2022-07-03T00:00:00Z' \The same call, in code.
The capture above came back as markdown. These examples add a key, so you get browser rendering, proxies, and concurrency on twitch.tv.
import { SpiderBrowser } from "spider-browser";
const spider = new SpiderBrowser({
apiKey: process.env.SPIDER_API_KEY!,
});
await spider.connect();
const page = spider.page!;
await page.goto("https://www.twitch.tv/directory");
// No selectors, no schema. Spider reads the page and names the fields.
const data = await page.scrape();
console.log(data);
await spider.close();import { SpiderBrowser } from "spider-browser";
const spider = new SpiderBrowser({
apiKey: process.env.SPIDER_API_KEY!,
});
await spider.connect();
const page = spider.page!;
await page.goto("https://www.twitch.tv/directory");
await page.content(10000);
const data = await page.evaluate(`(() => {
const categories = [];
document.querySelectorAll("[data-a-target*='card'], a[href*='/directory/category/'], article").forEach(el => {
const name = el.querySelector("h2, h3, [data-a-target*='Name'], p")?.textContent?.trim();
const viewerEl = el.querySelector("[aria-label*='viewer' i]");
let viewers = viewerEl?.textContent?.trim();
if (!viewers) {
el.querySelectorAll("p, span").forEach(s => {
const t = s.textContent?.trim();
if (t && /viewer/i.test(t)) viewers = t;
});
}
if (name) categories.push({ name, viewers });
});
return JSON.stringify({ total: categories.length, categories: categories.slice(0, 10) });
})()`);
console.log(JSON.parse(data));
await spider.close();Ready for volume? Get an API key →
Fields you can pull.
Spider names these from the page. The capture above came back as markdown; the same
call with return_format: "json" returns them as keys.
What twitch.tv costs to scrape.
The capture above cost $0.000079 to fetch. Pricing is $1 per GB of pre-transformation bandwidth plus $0.001 per CPU minute, so a page like this one lands at a fraction of a cent. Failed requests are billed at $0.
- Free balance on signup
- No card required to test
- Balance never expires
Run it keyless, no account
More Media scrapers.
YouTube Scraper
Extract video metadata, channel statistics, view counts, comments, playlist data, and trending content from YouTube. Full rendering for dynamic content and infinite scroll.
Spotify Scraper
Extract playlist data, track listings, artist info, and album metadata from Spotify.
IMDb Scraper
Extract movie ratings, cast info, box office data, and reviews from IMDb.
Start scraping twitch.tv.
You already have the call. A key raises the rate limit and turns on browser rendering, proxies, and concurrency. Balance never expires, and top-ups go through secure checkout.