Xfyun Scraper
Spider read xfyun.cn in 2.3 s without a browser and returned 322 lines of clean markdown, including the section "通用文档识别(OCR大模型) API 文档".
# # 通用文档识别(OCR大模型) API 文档3）signature的原始字段(signature_origin)规则如下。signature原始字段由 host，date，request-line三个参数按照格式拼接成，`host: $host\ndate: $date\n$request-line`请求url = "https://cbm01.cn-huabei-1.xf-yun.com/v1/private/se75ocrbm"date = "Wed, 11 Aug 2021 06:55:18 GMT"那么 signature原始字段(signature_origin)则为：date: Wed, 11 Aug 2021 06:55:18 GMT4）使用hmac-sha256算法结合apiSecret对signature_origin签名，获得签名后的摘要signature_sha。`signature_sha=hmac-sha256(signature_origin,$apiSecret)5）使用base64编码对signature_sha进行编码获得最终的signature。`signature=base64(signature_sha)`APISecret = "apisecretXXXXXXXXXXXXXXXXXXXXXXX"`signature="/mg2h9BCkespilZ94HUBaQVPq2v7PxYF90teTBlaxd8="6）根据以上信息拼接authorization base64编码前（authorization_origin）的字符串，示例如下。`api_key="apikeyXXXXXXXXXXXXXXXXXXXXXXXXXX", algorithm="hmac-sha256", headers="host date request-line", signature="/mg2h9BCkespilZ94HUBaQVPq2v7PxYF90teTBlaxd8="*注：* headers是参与签名的参数，请注意是固定的参数名（"host date request-line"），而非这些参数的值。7）最后再对authorization_origin进行base64编码获得最终的authorization参数。`authorization = base64(authorization_origin)authorization=YXBpX2tleT0iYXBpa2V5WFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFgiLCBhbGdvcml0aG09ImhtYWMtc2hhMjU2IiwgaGVhZGVycz0iaG9zdCBkYXRlIHJlcXVlc3QtbGluZSIsIHNpZ25hdHVyZT0iL21nMmg5QkNrZXNwaWxaOTRIVUJhUVZQcTJ2N1B4WUY5MHRlVEJsYXhkOD0i### # 鉴权结果检查是否有authorization参数，详情见authorization参数详细生成规则{“message”:”HMAC signature cannot be verified”}{“message”:”HMAC signature does not match”}{“message”:”HMAC signature cannot be verified, a valid date or x-date header is required for HMAC Authentication”}Date: Mon, 30 Nov 2020 02:34:33 GMTContent-Type: text/plain; charset=utf-8"message": "HMAC signature does not match, a valid date or x-date header is required for HMAC Authentication"## # 请求`https://cbm01.cn-huabei-1.xf-yun.com/v1/private/se75ocrbm"markdown_element_option": "watermark=0,page_header=0,page_footer=0,page_number=0,graph=0","sed_element_option": "watermark=0,page_header=0,page_footer=0,page_number=0,graph=0",The same call, in code.
The capture above came back as markdown. These examples add a key, so you get browser rendering, proxies, and concurrency on xfyun.cn.
import { SpiderBrowser } from "spider-browser";
const spider = new SpiderBrowser({
apiKey: process.env.SPIDER_API_KEY!,
});
await spider.connect();
const page = spider.page!;
await page.goto("https://xfyun.cn");
// No selectors, no schema. Spider reads the page and names the fields.
const data = await page.scrape();
console.log(data);
await spider.close();import { Spider } from "@spider-cloud/spider-client";
const spider = new Spider({ apiKey: process.env.SPIDER_API_KEY! });
const result = await spider.scrapeUrl("https://www.xfyun.cn", {
return_format: "markdown",
});
console.log(result);Ready for volume? Get an API key →
Fields you can pull.
Spider names these from the page. The capture above came back as markdown; the same
call with return_format: "json" returns them as keys.
What xfyun.cn costs to scrape.
The capture above cost $0.000184 to fetch. Pricing is $1 per GB of pre-transformation bandwidth plus $0.001 per CPU minute, so a page like this one lands at a fraction of a cent. Failed requests are billed at $0.
- Free balance on signup
- No card required to test
- Balance never expires
Run it keyless, no account
More International scrapers.
Mercado Libre Scraper
Extract product listings, seller ratings, pricing in local currencies, and shipping data from Mercado Libre.
Rakuten Scraper
Extract product listings, store ratings, cashback offers, and pricing data from Rakuten Japan marketplace.
Flipkart Scraper
Extract product listings, seller data, pricing in INR, and delivery estimates from Flipkart India store.
Start scraping xfyun.cn.
You already have the call. A key raises the rate limit and turns on browser rendering, proxies, and concurrency. Balance never expires, and top-ups go through secure checkout.