JetBrains Marketplace Scraper
Spider read plugins.jetbrains.com in 1.1 s without a browser and returned 26 lines of clean markdown, including sections like "Staff-picked plugins", "Top downloaded plugins" and "New plugins".
When you visit JetBrains Marketplace, you can see several pre-selected plugin categories. You can use the product selection filter to search the homepage for a specific product. By default, when no product is selected on the website, you’ll see a list of plugins for all supported IDEs.### Staff-picked pluginsPlugins are selected by the product teams of each IDE based on a combination of factors, primarily the plugin’s general usefulness and popularity, as determined by plugin downloads and ratings, among other factors. Learn more about the staff picks selection process.### Top downloaded pluginsPlugins are ranked by the number of downloads for each plugin, starting from the plugin with the greatest number of overall downloads.### New pluginsPlugins are ranked by the date when the last plugin update was uploaded to JetBrains servers, with the most recent date at the top of the list.### Top-rated pluginsPlugins are ranked by rating, starting with the plugin with the highest rating.## Search resultsWhen you search for a particular plugin, we take the following factors into consideration:1. Relevance of the search term:Firstly, we determine whether the plugin's text fields that have been input by the vendor match the user’s request. These text fields include the plugin ID, name, description, tags, and the plugin’s vendor name. Matches in different fields are scored differently. We account for declensions, conjugations, articles, different spellings, etc. For every matching plugin, we compute an initial numerical score that shows how relevant this plugin is to a search query.If the list of plugins contains staff-picked plugins, then we add a coefficient to their initial score to boost those plugins in the search results.The same call, in code.
The capture above came back as markdown. These examples add a key, so you get browser rendering, proxies, and concurrency on plugins.jetbrains.com.
import { SpiderBrowser } from "spider-browser";
const spider = new SpiderBrowser({
apiKey: process.env.SPIDER_API_KEY!,
});
await spider.connect();
const page = spider.page!;
await page.goto("https://plugins.jetbrains.com/search?orderBy=downloads");
// No selectors, no schema. Spider reads the page and names the fields.
const data = await page.scrape();
console.log(data);
await spider.close();import { SpiderBrowser } from "spider-browser";
const spider = new SpiderBrowser({
apiKey: process.env.SPIDER_API_KEY!,
});
await spider.connect();
const page = spider.page!;
await page.goto("https://plugins.jetbrains.com/search?orderBy=downloads");
await page.content(8000);
const data = await page.evaluate(`(() => {
const plugins = [];
document.querySelectorAll("[class*='PluginCard'], [class*='plugin-card']").forEach(el => {
const name = el.querySelector("[class*='name'] a, h3 a")?.textContent?.trim();
const downloads = el.querySelector("[class*='downloads']")?.textContent?.trim();
const rating = el.querySelector("[class*='rating']")?.textContent?.trim();
const vendor = el.querySelector("[class*='vendor']")?.textContent?.trim();
const desc = el.querySelector("[class*='description'], p")?.textContent?.trim();
if (name) plugins.push({ name, downloads, rating, vendor, desc });
});
return JSON.stringify({ total: plugins.length, plugins: plugins.slice(0, 15) });
})()`);
console.log(JSON.parse(data));
await spider.close();Ready for volume? Get an API key →
Fields you can pull.
Spider names these from the page. The capture above came back as markdown; the same
call with return_format: "json" returns them as keys.
What plugins.jetbrains.com costs to scrape.
The capture above cost $0.000028 to fetch. Pricing is $1 per GB of pre-transformation bandwidth plus $0.001 per CPU minute, so a page like this one lands at a fraction of a cent. Failed requests are billed at $0.
- Free balance on signup
- No card required to test
- Balance never expires
Run it keyless, no account
More AI & Developer scrapers.
ChatGPT Scraper
Extract shared ChatGPT conversations, prompts, and AI-generated content from public links.
Hugging Face Scraper
Extract ML model cards, dataset info, leaderboard data, and paper metadata from Hugging Face.
GitHub Scraper
Extract trending repositories, star counts, contributor data, and code snippets from GitHub.
Start scraping plugins.jetbrains.com.
You already have the call. A key raises the rate limit and turns on browser rendering, proxies, and concurrency. Balance never expires, and top-ups go through secure checkout.