Twitter/X Scraper API
Extract tweets, user profiles, engagement metrics, and trending topics from X (formerly Twitter). Full stealth browsing with residential proxies to handle X's aggressive bot detection. Built on spider-browser .
X (formerly Twitter) has some of the most aggressive anti-scraping defenses on the web. Spider uses browser fingerprint rotation, residential proxies, and real browser rendering to reliably extract tweets, profiles, and engagement data at scale.
- target
- x.com
- anti-bot bypass
- Stealth mode
- data freshness
- Real-time
Extract data in minutes.
import { SpiderBrowser } from "spider-browser";
const spider = new SpiderBrowser({
apiKey: process.env.SPIDER_API_KEY!,
stealth: 2,
});
await spider.connect();
const page = spider.page!;
await page.goto("https://x.com/OpenAI");
await page.content(12000);
const data = await page.evaluate(`(() => {
const tweets = [];
document.querySelectorAll("article[data-testid='tweet']").forEach(el => {
const text = el.querySelector("[data-testid='tweetText']")?.textContent?.trim();
const time = el.querySelector("time")?.getAttribute("datetime");
const likes = el.querySelector("[data-testid='like'] span")?.textContent?.trim();
const retweets = el.querySelector("[data-testid='retweet'] span")?.textContent?.trim();
if (text) tweets.push({ text: text.slice(0, 280), time, likes, retweets });
});
return JSON.stringify({ total: tweets.length, tweets: tweets.slice(0, 10) });
})()`);
console.log(JSON.parse(data));
await spider.close();Start extracting x.com data in three steps.
Get an API key
Sign up free and a balance is credited to your account.
Configure the call
Set the target URL, output format, and rendering mode.
Get structured data
Clean JSON back in seconds, ready for your pipeline.
One endpoint for x.com.
Structured JSON from x.com with a single POST. AI-resolved selectors, cached on the first call.
curl -X POST https://api.spider.cloud/fetch/x.com/ \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"return_format": "json"}'import requests
resp = requests.post(
"https://api.spider.cloud/fetch/x.com/",
headers={
"Authorization": "Bearer YOUR_API_KEY",
"Content-Type": "application/json",
},
json={"return_format": "json"},
)
print(resp.json())const resp = await fetch("https://api.spider.cloud/fetch/x.com/", {
method: "POST",
headers: {
"Authorization": "Bearer YOUR_API_KEY",
"Content-Type": "application/json",
},
body: JSON.stringify({ return_format: "json" }),
});
const data = await resp.json();
console.log(data);What teams build with x.com data.
Social Listening
Monitor brand mentions, competitor activity, and industry conversations across public tweets and threads.
Trend Analysis
Track trending topics, hashtags, and viral content to spot emerging narratives before they peak.
Influencer Research
Analyze engagement rates, follower counts, and posting patterns to identify high-value accounts for partnerships.
News Monitoring
Capture breaking news and real-time updates from journalist accounts, news outlets, and official sources.
Fields you can pull.
Anti-bot bypass
Residential proxy rotation and fingerprint randomization to reliably access X pages despite aggressive bot detection.
Full React SPA
Complete JavaScript execution renders X's React interface, including lazy-loaded tweets and infinite scroll.
Rich tweet data
Extract text, engagement metrics, media, threads, and profile data in clean structured JSON.
Why teams pick Spider for x.com.
Free balance on sign-up. No credit card required to test.
About $0.03 per 1,000 pages on a typical workload. No monthly minimum.
Top up once, use it whenever. Unused balance carries over forever.
Scraping x.com.
Can I scrape X/Twitter without API access?
Yes. Spider renders X pages in a real browser and extracts data from the DOM. No Twitter/X API keys or paid API tier required.
How does Spider handle X's anti-bot detection?
Spider uses residential proxy rotation, browser fingerprint randomization, and stealth browsing with real Chrome instances to avoid detection.
What data can I extract from X?
Tweet text, author info, likes, retweets, replies, timestamps, media attachments, hashtags, and full user profiles.
Can I scrape X search results?
Yes. Spider can render X search results pages and extract tweets matching any query, hashtag, or advanced search filter.
Does scraping X violate their terms?
Spider accesses only publicly visible content. The legality of scraping public social media data is supported by court precedent (hiQ v. LinkedIn).
More Social scrapers.
Reddit Scraper
Extract subreddit posts, comments, vote counts, and user data from Reddit. The fastest Reddit scraping API with full JavaScript rendering for new Reddit and old Reddit layouts.
LinkedIn Scraper
Extract public company profiles, employee counts, job listings, and professional data from LinkedIn. The most reliable LinkedIn scraping API with residential proxy rotation and stealth browsing to handle aggressive anti-bot protection.
Pinterest Scraper
Extract pins, board data, user profiles, and visual content from Pinterest.
Start scraping x.com.
Grab an API key and call the endpoint above. The first request resolves the config; every request after hits cache.