Icio Scraper
Spider read icio.us in 170 ms without a browser and returned 160 lines of clean markdown.
Some Pinboard updates as we careen forward into the New Year:What does the future hold for Pinboard? Death! The bus that one day comes for us all! The skeletal, icy hand on an unprepared shoulder! Pain, a flash of light, then numbing darkness. So back up your bookmarks.On this happy day, there are three people I would like to especially thank:* **Joshua Schachter**, for giving me the moral blessing to clone Delicious while it was still a going concern, and for contributing invaluable ideas (like the URL schema and pricing model).* **Peter Gadjokov**, who co-founded the site with me, came up with the name, and helped me through the insanity of December 2010.* And **Britta Gustafson**, who introduced me to fandom and showed me how they had created an entire world of their own on Delicious.Not coincidentally, all three of these people are from the earliest days of Delicious, and I am happy that the community they created in 2003 can in some way live on in the very different internet of 2019, biding its time until things can be fun again. I am grateful to them all, and grateful to the people who use this website, hopefully for many decades to come.The big story this year was last month's surprise acquisition of Pinboard's long-time nemesis Delicious. This illustrates the importance of always having a backup nemesis, an area where Pinboard leads the industry.But it's also been a rough year for Pinboard users! In November I began traveling extensively in support of Tech Solidarity, an attempt to mobilize tech workers after the disastrous US election.All the travel meant I sometimes ignored support emails for weeks at a time. In the last couple of months, I've taken a bit of a reverse sabbatical to try to stabilize the site, make it easier for me to monitor and run, and catch up with a backlog of very, very, very irate messages.I also baked this delicious pie:As every year, I'd like to thank all Pinboard users, old and new, for their support and their custom. I know there are lots of rival bookmarking services out there.The same call, in code.
The capture above came back as markdown. These examples add a key, so you get browser rendering, proxies, and concurrency on icio.us.
import { SpiderBrowser } from "spider-browser";
const spider = new SpiderBrowser({
apiKey: process.env.SPIDER_API_KEY!,
});
await spider.connect();
const page = spider.page!;
await page.goto("https://icio.us");
// No selectors, no schema. Spider reads the page and names the fields.
const data = await page.scrape();
console.log(data);
await spider.close();import { Spider } from "@spider-cloud/spider-client";
const spider = new Spider({ apiKey: process.env.SPIDER_API_KEY! });
const result = await spider.scrapeUrl("https://www.icio.us", {
return_format: "markdown",
});
console.log(result);Ready for volume? Get an API key →
Fields you can pull.
Spider names these from the page. The capture above came back as markdown; the same
call with return_format: "json" returns them as keys.
What icio.us costs to scrape.
The capture above cost $0.000068 to fetch. Pricing is $1 per GB of pre-transformation bandwidth plus $0.001 per CPU minute, so a page like this one lands at a fraction of a cent. Failed requests are billed at $0.
- Free balance on signup
- No card required to test
- Balance never expires
Run it keyless, no account
More International scrapers.
Mercado Libre Scraper
Extract product listings, seller ratings, pricing in local currencies, and shipping data from Mercado Libre.
Rakuten Scraper
Extract product listings, store ratings, cashback offers, and pricing data from Rakuten Japan marketplace.
Flipkart Scraper
Extract product listings, seller data, pricing in INR, and delivery estimates from Flipkart India store.
Start scraping icio.us.
You already have the call. A key raises the rate limit and turns on browser rendering, proxies, and concurrency. Balance never expires, and top-ups go through secure checkout.