skills.rest
sakamoto-family-smile
sakamoto-family-smileCOMMUNITY
Explore all skills by this author →
GitHub

data-scraper-agent

Scrape public sources via REST, HTML, or RSS and store enriched data.

Updated Apr 13, 2026
Search Tags:#automation#supabase#data#notion#scraping#github-actions#ai-enrichment
One-click install
npx skills add https://github.com/sakamoto-family-smile/agent_monorepo --skill data-scraper-agent-sakamoto-family-smile
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-scraper-agent
Source: https://github.com/sakamoto-family-smile/agent_monorepo/tree/main/.claude/skills/ecc/data-scraper-agent
Command: npx skills add https://github.com/sakamoto-family-smile/agent_monorepo --skill data-scraper-agent-sakamoto-family-smile

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Collecting data from public sources is repetitive, error-prone, and time-consuming. This skill automates scraping, AI enrichment, and storage to centralize insights.

Core Features & Use Cases

  • Automated data collection: schedule scraping from public sources (REST, HTML, RSS) and aggregate results.
  • AI enrichment: apply free Gemini or other LLMs to score, summarize, and classify items.
  • Centralized storage: push results to Notion, Google Sheets, or Supabase for review and collaboration.
  • Use Case: Monitor job postings, product prices, news, GitHub repos, or sports data over time with learning from user feedback.

Quick Start

Configure the agent by setting up GEMINI_API_KEY, storage backend credentials, and a GitHub Actions workflow to run the agent on a schedule.

Frequently Asked Questions about data-scraper-agent

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate web scraping and store the results in Notion or Supabase on a schedule?▼

Automated web scraping on a schedule requires a GitHub Actions workflow to trigger the agent, a Gemini API key for AI enrichment, and credentials for your storage backend like Notion, Supabase, or Google Sheets. The agent handles REST, HTML, and RSS sources automatically.

What is AI enrichment for scraped data and how does it work with free LLMs?▼

AI enrichment for scraped data uses free-tier Gemini LLMs to score, summarize, and classify raw items collected from public sources. This process transforms raw REST, HTML, or RSS payloads into centralized, structured insights ready for review.

Do I need Python and a Gemini API key to run scheduled data scraping jobs?▼

Yes, running scheduled data scraping jobs requires Python and a Gemini API key with access to free tiers. You also need a GitHub Actions workflow for continuous scheduling and a storage backend like Supabase or Google Sheets to save the results.

Can I use GitHub Actions to continuously monitor prices and job postings from public sources?▼

Yes, you can use GitHub Actions to continuously monitor prices, job postings, news, or sports data. The agent automates scraping these public sources via REST, HTML, or RSS feeds and pushes the AI-enriched results to your centralized storage backend.

What are the limitations of using free Gemini API tiers for scraping and AI enrichment?▼

The limitations of using free Gemini API tiers for AI enrichment involve rate limits and quota constraints within your GitHub Actions workflow. While the agent handles automated scraping and classification, heavy workloads might exceed free-tier boundaries, requiring careful schedule management.

Related Skills

Explore More Data & Analytics Skills→
psyducklerpsyduckler
9

aeo-content-free

Create or refresh citation-worthy content for AI assistants using free tools.

Community
Advanced
camoneartcamoneart
5

grafana-dashboards

Design and deploy Grafana dashboards for real-time system and application metrics.

Community
Intermediate
RepairYourTechRepairYourTech
6

github-actions

Create and manage CI/CD pipelines with GitHub Actions workflows.

Community
Advanced
BagelHoleBagelHole
46

gcp-cloud-functions

Deploy and configure Python serverless functions on Google Cloud Functions.

Community
Basic