skills.rest
Yog-Sotho
Yog-SothoCOMMUNITY
Explore all skills by this author →
Website𝕏@YogSoth0GitHub

data-scraper-agent

Schedule public data crawls with AI enrichment and storage to Notion, Sheets, or Supabase.

4|Updated Mar 21, 2026
Search Tags:#automation#supabase#notion#sheets#data-collection#data-scraping#ai-enrichment
One-click install
npx skills add https://github.com/Yog-Sotho/claude-skills --skill data-scraper-agent-yog-sotho
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-scraper-agent
Source: https://github.com/Yog-Sotho/claude-skills/tree/main/data-scraper-agent
Command: npx skills add https://github.com/Yog-Sotho/claude-skills --skill data-scraper-agent-yog-sotho

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Build a production-ready AI-powered data collection agent that continuously gathers public data sources, enriches results with a free LLM, and stores them for review.

Core Features & Use Cases

  • Automated data collection: schedule scrapes across public sources (job boards, prices, news, GitHub, etc.)
  • AI enrichment & storage: enrich results with LLM responses and store to Notion/Sheets/Supabase with deduplication and feedback learning
  • End-to-end automation: GitHub Actions-based free stack, learn from user feedback to improve future runs.

Quick Start

Configure sources and secrets, then run the agent to start automated data collection on schedule.

Frequently Asked Questions about data-scraper-agent

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate public data scraping and store results in Notion or Google Sheets?▼

Automated data scraping is handled by scheduling crawls across public sources and configuring storage destinations. The agent enriches results with AI and saves them directly to Notion, Sheets, or Supabase with automatic deduplication.

Can I enrich scraped data with AI without paying for a large language model?▼

Yes, AI enrichment uses a free LLM via a Gemini AI API key. The agent automatically processes scraped public data, appends AI-generated insights, and stores the enriched output for review.

How do I set up scheduled web scraping to run continuously for free?▼

Scheduled web scraping runs continuously using GitHub Actions as the automation stack. You configure source URLs and secrets, then the agent executes crawls on a defined schedule without requiring paid infrastructure.

Does this data collection agent require Python and specific API keys to run?▼

Yes, the data collection agent requires Python for execution, a public storage backend for saving results, and a Gemini AI API key to enable AI enrichment and automation features.

How does the agent handle duplicate data entries during automated collection?▼

Automated collection includes built-in deduplication to prevent redundant data entries. The agent checks scraped results against existing records before storing them in your connected Notion, Sheets, or Supabase backend.

Can the automated scraping workflow learn and improve from user feedback over time?▼

Yes, the automated scraping workflow incorporates user feedback to refine future runs. It learns from your input during the review process to improve the accuracy of subsequent scheduled data collection and AI enrichment.

Related Skills

Explore More Data & Analytics Skills→
sakamoto-family-smilesakamoto-family-smile

data-scraper-agent

Scrape public sources via REST, HTML, or RSS and store enriched data.

Community
Advanced
samymitysamymity
1

data-scraper-agent

Scrape public websites, APIs, and RSS feeds on a schedule.

Community
Advanced
multiplex-aimultiplex-ai
2

data-scraper-agent

Scrape, enrich, and store public data on a schedule.

Official
Advanced
arbisoftarbisoft
4

data-scraper-agent

Automate scraping, enrichment, and storage of public data sources.

Official
Advanced