skills.rest
brightdata
brightdataOFFICIAL
Explore all skills by this author →
WebsiteGitHub

data-feeds

Extract structured JSON data from major websites via Bright Data's Web Data APIs.

245|35|Updated Jan 28, 2026
Search Tags:#automation#polling#data-extraction#structured-data#brightdata#web-data-api
One-click install
npx skills add https://github.com/brightdata/skills --skill data-feeds-brightdata
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-feeds
Source: https://github.com/brightdata/skills/tree/main/skills/data-feeds
Command: npx skills add https://github.com/brightdata/skills --skill data-feeds-brightdata

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires curl, jq, and includes scripts (resource) components.

What problem does it solve?

This skill automates the extraction of structured data from 40+ websites using Bright Data's Web Data APIs, eliminating manual scraping and data wrangling.

Core Features & Use Cases

  • Automated data extraction from major websites via Bright Data's Web Data APIs with automatic polling to monitor progress.
  • Clean JSON outputs including product details, profiles, reviews, posts, and comments for easy ingestion into analytics pipelines.
  • Use cases include product analytics, competitive intelligence, and data enrichment across ecommerce, social media, and news domains.

Quick Start

Provide a target URL and run the datasets.sh script to retrieve a structured JSON feed.

Frequently Asked Questions about data-feeds

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract structured data from major websites into JSON?▼

This Skill automates web data extraction via Bright Data's Web Data APIs, returning clean JSON outputs containing product details, profiles, reviews, posts, and comments for easy ingestion into analytics pipelines.

How do I automate polling for web data extraction tasks?▼

Automated polling is built into the extraction scripts, allowing the datasets.sh script to automatically monitor progress and retrieve completed structured data from Bright Data's APIs without manual intervention.

Do I need an API key to extract web data using Bright Data?▼

Yes, you need to provide your BRIGHTDATA_API_KEY to authenticate requests. The provided scripts require this environment variable to access Bright Data's Web Data APIs for structured data extraction.

What's the best way to get clean JSON for competitive intelligence across ecommerce sites?▼

Using Bright Data's Web Data APIs through the datasets.sh script is the best way to get clean JSON for competitive intelligence, as it automatically handles data extraction from 40+ sites without manual scraping or data wrangling.

Does this web data extraction approach work with curl and jq?▼

Yes, the scripts depend on curl and jq to execute API requests and parse the returned JSON, ensuring structured data from Bright Data's APIs is properly formatted for your analytics pipelines.

What are the limitations of using Bright Data APIs for data extraction?▼

Data extraction is limited to the 40+ websites supported by Bright Data's Web Data APIs, and you must supply a valid BRIGHTDATA_API_KEY, but it eliminates manual scraping and data wrangling for supported domains.

Related Skills

Explore More Data & Analytics Skills→
semaes111semaes111

search

Retrieve Google search results via Bright Data's SERP API with pagination.

Community
Intermediate
farzanmrzfarzanmrz

data-feeds

Extract structured JSON from platform URLs using Bright Data pipelines.

Community
Advanced
weirdgmeweirdgme
4

platform-engineering

Design and implement scalable platform engineering patterns for internal developer platforms.

Community
Advanced
semaes111semaes111

data-feeds

Fetch structured website data via Bright Data APIs and return normalized JSON.

Community
Intermediate