Sensorsdata Scraper
Spider read sensorsdata.cn in 3.7 s without a browser and returned 113 lines of clean markdown, including sections like "用户表 (users)", "Items 表" and "Session 表".
# 数据表及数据类型神策系统内预置的业务 ID 字段，不会自动创建，起到占位符的作用，是配合 SDK 自动采集一起使用的。在实际采集过程中，需要提前在神策分析产品界面中手动添加才可生效。如$identity_android_id、$identity_mp_id等。需要特别注意的是，事件表的 user_id 字段并不是 track 时传入的 distinct_id，而是由神策分析为该用户分配的内部 ID，具体的机制见 标识用户——全域用户关联。### 用户表 (users)cookie_id+15ffdb0a3f898-02045d1cb7be78-31126a5d-250125-15ffdb0a3fa40a 或 15ffdb0a3f898-02045d1cb7be78-31126a5d-250125-15ffdb0a3fa40a当前 first_id 取值是 $identity_anonymous_id 还是 $identity_login_id$identity_anonymous_id 或 $identity_login_id当前 second_id 取值是 $identity_anonymous_id 还是 $identity_login_id老架构下，用户可以通过匿名 id 和登录 id 任意一个找到用户（ 对应老架构下 users 表中的 first_id / second_id / $device_id_list 字段 ），在当前新架构下$identity_distinct_id 字段中就包含了匿名 id 和登录 id， 用来实现替代老架构中 first_id 、second_id 和$device_id_list 字段的能力0f485d4daaadedae5f id123456774d56d682e549c3 n5436k65 oWDMZ0WHqfsjIz7A9B2XNQOWmN3E1570230586048:0c476090a0b2940a；1570230591000:65A71299-7139-4B4C-9B71-23A0AC9AAF7D### Items 表### Session 表每张 Session 表都对应一个 Session 的配置，命名规则为：Sessions_${Session_name}。Session 表是对 events 表做了扩展，除了包含 events 表包含的字段，还包含 Session 属性和 Session 相关的特殊字段，Session 属性的命名规则是原始的属性名加上后缀 $Session，表示 Session 中初始事件的属性。其中特殊字段说明如下：标示一个 Session 中事件的索引，从 0 开始，依次递增。1.14 及之前版本 Session 中最后一个事件的索引是 -1，如果 Session 中只有 1 个事件，则索引值是 -2。1.15 及之后版本，不再有特殊的 -1、-2 索引值。`SELECT event, user_id, distinct_id, date FROM Sessions_default/*SESSION_TABLE_DATE_RANGE=[2018-01-01,2018-01-05]*/`### 用户分群 / 标签表`SELECT * FROM user_group_fenqun9 WHERE base_time=unix_timestamp_ms('2019-01-17 00:00:00')`## 数据类型### **Number**### **String**### **Date**如果有需要，可以使用 EPOCH_TO_TIMESTAMP 函数转换为 Timestamp 类型，例如：`SELECT EPOCH_TO_TIMESTAMP($signup_time / 1000) FROM users LIMIT 100;``SELECT COUNT(*) AS cnt FROM users WHERE EPOCH_TO_TIMESTAMP($signup_time / 1000) > '2017-01-01';`### **Datetime**日期时间类型，和 Date 类型一样，也使用毫秒级的 Timestamp 表示，例如：1442592138000。同样也可以使用 EPOCH_TO_TIMESTAMP 类型进行类型转换。### **Bool**### **List**`SELECT FavoriteFruits FROM users WHERE CONTAINS('橘子', FavoriteFruits);`The same call, in code.
The capture above came back as markdown. These examples add a key, so you get browser rendering, proxies, and concurrency on sensorsdata.cn.
import { SpiderBrowser } from "spider-browser";
const spider = new SpiderBrowser({
apiKey: process.env.SPIDER_API_KEY!,
});
await spider.connect();
const page = spider.page!;
await page.goto("https://sensorsdata.cn");
// No selectors, no schema. Spider reads the page and names the fields.
const data = await page.scrape();
console.log(data);
await spider.close();import { Spider } from "@spider-cloud/spider-client";
const spider = new Spider({ apiKey: process.env.SPIDER_API_KEY! });
const result = await spider.scrapeUrl("https://www.sensorsdata.cn", {
return_format: "markdown",
});
console.log(result);Ready for volume? Get an API key →
Fields you can pull.
Spider names these from the page. The capture above came back as markdown; the same
call with return_format: "json" returns them as keys.
What sensorsdata.cn costs to scrape.
The capture above cost $0.001855 to fetch. Pricing is $1 per GB of pre-transformation bandwidth plus $0.001 per CPU minute, so a page like this one lands at a fraction of a cent. Failed requests are billed at $0.
- Free balance on signup
- No card required to test
- Balance never expires
Run it keyless, no account
More International scrapers.
Mercado Libre Scraper
Extract product listings, seller ratings, pricing in local currencies, and shipping data from Mercado Libre.
Rakuten Scraper
Extract product listings, store ratings, cashback offers, and pricing data from Rakuten Japan marketplace.
Flipkart Scraper
Extract product listings, seller data, pricing in INR, and delivery estimates from Flipkart India store.
Start scraping sensorsdata.cn.
You already have the call. A key raises the rate limit and turns on browser rendering, proxies, and concurrency. Balance never expires, and top-ups go through secure checkout.