skills.rest
Wakewell-Sleep-Solutions
Wakewell-Sleep-SolutionsOFFICIAL
Explore all skills by this author →
GitHub

data-scraper-agent

Automate public data collection, enrichment, and storage via scheduled pipelines.

Updated Aug 23, 2026
Search Tags:#automation#workflow#ai#data#notion#scheduling#scrape
One-click install
npx skills add https://github.com/Wakewell-Sleep-Solutions/claude-skills-ecosystem --skill data-scraper-agent-wakewell-sleep-solutions
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-scraper-agent
Source: https://github.com/Wakewell-Sleep-Solutions/claude-skills-ecosystem/tree/main/data-scraper-agent
Command: npx skills add https://github.com/Wakewell-Sleep-Solutions/claude-skills-ecosystem --skill data-scraper-agent-wakewell-sleep-solutions

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

The Data Scraper Agent solves the pain of manual public data collection by providing an automated, AI-assisted pipeline that collects, enriches, and stores results with minimal hosting requirements.

Core Features & Use Cases

  • Three-layer pipeline: Collect → Enrich → Store, with a deterministic flow that runs on schedules.
  • Free/open-stack approach: lightweight scraping with requests/BeautifulSoup, optional Playwright for JS-rendered pages, Gemini Flash for AI enrichment, and storage in Notion / Sheets / Supabase, all designed to run 100% free in public repos via GitHub Actions.
  • Common scenarios: monitor jobs, track prices, gather news, or watch repositories and events by regularly scraping public sources, enriching items with AI, and persisting results for review.

Quick Start

Configure your environment and run the agent to start harvesting data from your public sources.

Frequently Asked Questions about data-scraper-agent

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate web scraping and store the results directly in Notion or Google Sheets?▼

Automated web scraping pipelines collect public data on a schedule, enrich items with AI, and persist results directly into Notion, Google Sheets, or Supabase for automated review and storage.

Can I run an automated data collection workflow for free using GitHub Actions?▼

Automated data collection workflows can run 100% free in public repositories via GitHub Actions, using a lightweight open-stack approach that eliminates dedicated hosting costs for scheduled scraping.

What is the best way to enrich scraped web data with AI before saving it?▼

Enriching scraped web data with AI is best handled through a multi-layer pipeline that executes batch AI calls via Gemini Flash, enhancing and deduplicating collected items before final storage.

Does this scraping pipeline support JavaScript-rendered pages that require Playwright?▼

The scraping pipeline supports JavaScript-rendered pages by optionally integrating Playwright alongside default lightweight requests and BeautifulSoup scraping for dynamic public web sources.

How do I schedule a web scraper to monitor public sources continuously?▼

Scheduling a web scraper to monitor public sources continuously involves configuring a deterministic workflow that runs on a schedule to regularly collect, enrich with AI, and persist results.

What are the limitations of using a free hosted scraping workflow in public repositories?▼

Limitations of free hosted scraping workflows in public repositories include execution time limits and rate constraints imposed by GitHub Actions, alongside public visibility of scraping logic and configurations.

Related Skills

Explore More Data & Analytics Skills→
heyAyushhheyAyushh
2

kafka-topic-setup

Creates and configures Kafka topics for the hook event pipeline.

Community
Intermediate
abhishekdutta18abhishekdutta18

data-scraper-agent

Scrape public data sources with Python and store results in Notion, Sheets, or Supabase.

Community
Intermediate
MomoDavilukeMomoDaviluke
2

data-scraper-agent

Automate public data collection with scheduled scrapes and AI enrichment.

Community
Advanced
zeronx798zeronx798

scrapling

Scrape static, dynamic, and protected websites using the scrapling library.

Community
Advanced