bggg-data-x

Collect public X posts from logged-in Chrome DOM into normalized JSONL.

588|92|Updated May 2, 2026
One-click install
npx skills add https://github.com/binggandata/bggg-skills --skill bggg-data-x
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: bggg-data-x
Source: https://github.com/binggandata/bggg-skills/tree/main/bggg-data-x
Command: npx skills add https://github.com/binggandata/bggg-skills --skill bggg-data-x

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

X/Twitter blocks most automated access behind login walls, paid APIs, and internal GraphQL endpoints, making it hard to gather auditable public post data for voice-of-customer research. This Skill collects visible posts through the user's own logged-in Chrome session and normalizes them into a consistent, auditable JSONL dataset.

Core Features & Use Cases

  • Deterministic Query Planning: Builds a validated request plan from a TSV of queries with language, round, row cap, and sort mode, generating encoded X search URLs.
  • Visible-DOM Collection: Extracts post text, Tweet IDs, canonical URLs, timestamps, author handles, and engagement labels using a fixed selector contract, with scroll-loop checkpointing and per-query source packages.
  • Normalization and Audit: Merges per-query packages, deduplicates by Tweet ID, parses engagement metrics, matches keywords, and emits a summary with missing-data and failure counts.
  • Use Case: A researcher running multilingual VOC analysis prepares queries in English and Japanese, collects up to 250 posts per query from the Latest tab, and receives a deduplicated JSONL corpus plus an audit summary for downstream reporting.

Quick Start

Use bggg-data-x to search X in my logged-in Chrome for these keyword queries and save the collected posts as normalized JSONL with an audit summary.

Frequently Asked Questions about bggg-data-x

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I collect X/Twitter posts without the paid API?

Drive the user's already logged-in Chrome session, open X search URLs, and extract rendered tweet cards from the visible DOM using selectors like article[data-testid="tweet"]. Results are saved per query and normalized into JSONL without touching internal GraphQL or cookies.

How to scrape X search results with a logged-in Chrome session?

Build a query plan from a TSV file, open each generated search URL in one Chrome tab, validate the selector contract, then scroll and accumulate posts keyed by Tweet ID. Stop after six stale scrolls or a 180-scroll hard cap and save one source package per query.

Does this approach export cookies or session tokens from Chrome?

No. The collection reads only visible rendered DOM content and explicitly forbids inspecting or exporting cookies, local storage, credentials, or profile data. Authentication relies entirely on the user's existing signed-in browser session.

What data fields are captured for each X post?

Each row preserves the numeric Tweet ID, canonical status URL, timestamp, exact post text, author handle and display block, engagement aria-label, full card text, query, language hint, and collection time. Normalization also parses replies, reposts, likes, bookmarks, and views.

What are the limitations of collecting X data via DOM scraping?

X search is a visible sample, not a complete census, so results depend on sort mode, query syntax, and date slicing. Collection stops on challenge pages, rate limits, or blank timelines, and search-engine snippets must never substitute for original post text.