data-scraper-agent

Automate scheduled public data collection with Python and Gemini Flash enrichment.

Updated Mar 1, 2026
One-click install
npx skills add https://github.com/derekhu0002/ai4pb-orchestrator --skill data-scraper-agent-derekhu0002
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: data-scraper-agent
Source: https://github.com/derekhu0002/ai4pb-orchestrator/tree/main/skills/data-scraper-agent
Command: npx skills add https://github.com/derekhu0002/ai4pb-orchestrator --skill data-scraper-agent-derekhu0002

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Build a production-ready, AI-powered data collection agent for any public data source. Runs on a schedule, enriches data with a free LLM, stores results in Notion/Sheets/Supabase, and learns from user feedback.

Core Features & Use Cases

  • Automated data collection: scrape public sources on a schedule.
  • AI enrichment: score, summarize, classify using Gemini Flash.
  • Storage & learning: persists results to Notion/Sheets/Supabase and adapts from user feedback.

Quick Start

Configure your data sources and start the agent to automatically collect, enrich, and store data.

Frequently Asked Questions about data-scraper-agent

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate public data scraping and store results in Supabase on a schedule?

Automated data scraping on a schedule persists enriched results directly to Supabase. You configure Python-based scraping scripts to run as scheduled GitHub Actions, collecting public data and storing it in your Supabase database without manual intervention.

Can I enrich scraped data with AI before saving it to Notion or Google Sheets?

AI enrichment of scraped data is supported before saving to Notion or Sheets. The agent uses Gemini Flash to score, summarize, and classify collected records, ensuring your persisted data in Notion or Google Sheets is processed and structured.

Does GitHub Actions work for running automated web scraping jobs for free?

GitHub Actions supports running automated web scraping jobs entirely for free. The agent's Python-based scraping processes are designed to execute within GitHub Actions, enabling scheduled monitoring, price tracking, and data collection without infrastructure costs.

What is the best way to track prices and monitor listings without paying for hosting?

The best way to track prices and monitor listings without hosting fees is running Python scrapers on GitHub Actions. This approach automates public data collection on a schedule, applies AI enrichment via Gemini Flash, and stores results in Notion or Supabase.

Can I scrape GitHub repositories and sports scores automatically with Python?

Scraping GitHub repositories and sports scores automatically is supported through Python-based collection. The agent automates end-to-end public data collection from diverse sources, including repository data and sports scores, on scheduled runs.

How do I set up a data collection agent that learns from user feedback?

A data collection agent that learns from user feedback adapts its enrichment logic over time. After scraping and storing results in Notion, Sheets, or Supabase, the agent incorporates user feedback to refine future AI-powered data collection runs.