Google Scraper
Spider read google.cn in 1.6 s without a browser and returned 173 lines of clean markdown, including sections like "Google 支持的 meta 标记和属性", "meta 代码" and "HTML 标记属性".
# Google 支持的 `meta` 标记和属性## `meta` 代码`meta` 标记是一种 HTML 标记，用于向搜索引擎和其他客户端提供有关网页的其他信息。客户端会处理 `meta` 标记，并忽略不受支持的元标记。`meta` 标记应添加到 HTML 网页的 `<head>` 部分，通常如下所示：**<meta name="description" content="Author: A.N. Author, Illustrator: P. Picture, Category: Books, Price: £9.24, Length: 784 pages">****<meta name="google-site-verification" content="+nxGUDJ4QpAZ5l9Bsjdi102tLVC21AIh5d1Nl23908vVuFHs34=">**<title>Example Books - high-quality used books for children</title>**<meta name="robots" content="noindex,nofollow">****如果您使用 Wix、WordPress 或 Blogger 等 CMS**，则可能无法直接修改 HTML，也可能不希望修改 HTML。实际上，您的 CMS 可能具有搜索引擎设置页面或其他某种机制，能够将 `meta` 标记告知搜索引擎。如果您要向网站添加 `meta` 标记，请在您的 CMS 上搜索有关修改网页 `<head>` 的说明（例如，搜索“wix add meta tags”）。<meta name="description" content="A description of the page"><meta name="googlebot" content="..., ...">`<meta name="robots" ...` 标记适用于所有搜索引擎，但 `<meta name="googlebot ...` 标记专用于 Google。如果 `robots`（或 `googlebot`）`meta` 标记之间存在冲突，那么系统会应用限制较严的标记。例如，如果某个网页同时包含 `max-snippet:50` 和 `nosnippet` 标记，那么会应用 `nosnippet` 标记。<meta name="googlebot" content="notranslate"><meta name="google" content="nopagereadaloud"><meta name="google-site-verification" content="..."><meta http-equiv="Content-Type" content="...; charset=...">这两个标记分别用于定义网页的内容类型和字符集。请务必使用引号引住 `http-equiv``meta` 标记中 `content` 属性的值，否则 `charset` 属性可能会被错误地解读。我们建议尽可能使用 Unicode/UTF-8。<meta http-equiv="refresh" content="...;url=..."><meta name="rating" content="adult"><meta name="rating" content="RTA-5042-1996-1400-1577-RTA">## HTML 标记属性HTML 标记属性是 HTML 标记的额外值，用于配置父标记。例如，`<a>` 标记的 `href` 属性会配置锚标记所指向的资源：`<a **href="https://example.com/"**...>`。对于索引编制来说，Google 搜索支持的 HTML 属性数量有限。`src` 和 `href` 等属性用于发现图片和网址等资源。Google 还支持各种 `rel` 属性，可让网站所有者限定出站链接。通过 `div`、`span` 和 `section` 标记的 `data-nosnippet` 属性，您可以从摘要中排除 HTML 网页的某些部分。## 需要注意的其他事项除了 `google-site-verification` 外，其他 `meta` 标记的大小写通常无关紧要。如果您考虑使用 JavaScript 注入或更改 `meta` 标记，请谨慎操作。我们建议您尽可能避免使用 JavaScript 注入或更改 `meta` 标记。如果确有必要这么做，请全面测试您的实现。## 不受支持的标记和属性The same call, in code.
The capture above came back as markdown. These examples add a key, so you get browser rendering, proxies, and concurrency on google.cn.
import { SpiderBrowser } from "spider-browser";
const spider = new SpiderBrowser({
apiKey: process.env.SPIDER_API_KEY!,
});
await spider.connect();
const page = spider.page!;
await page.goto("https://google.cn");
// No selectors, no schema. Spider reads the page and names the fields.
const data = await page.scrape();
console.log(data);
await spider.close();import { Spider } from "@spider-cloud/spider-client";
const spider = new Spider({ apiKey: process.env.SPIDER_API_KEY! });
const result = await spider.scrapeUrl("https://www.google.cn", {
return_format: "markdown",
});
console.log(result);Ready for volume? Get an API key →
Fields you can pull.
Spider names these from the page. The capture above came back as markdown; the same
call with return_format: "json" returns them as keys.
What google.cn costs to scrape.
The capture above cost $0.000297 to fetch. Pricing is $1 per GB of pre-transformation bandwidth plus $0.001 per CPU minute, so a page like this one lands at a fraction of a cent. Failed requests are billed at $0.
- Free balance on signup
- No card required to test
- Balance never expires
Run it keyless, no account
More International scrapers.
Mercado Libre Scraper
Extract product listings, seller ratings, pricing in local currencies, and shipping data from Mercado Libre.
Rakuten Scraper
Extract product listings, store ratings, cashback offers, and pricing data from Rakuten Japan marketplace.
Flipkart Scraper
Extract product listings, seller data, pricing in INR, and delivery estimates from Flipkart India store.
Start scraping google.cn.
You already have the call. A key raises the rate limit and turns on browser rendering, proxies, and concurrency. Balance never expires, and top-ups go through secure checkout.