skills.rest
RambleRainbow
RambleRainbowCOMMUNITY
Explore all skills by this author →
GitHub

data-scraper-agent

Automate scheduled collection, AI enrichment, and storage of public web data.

Updated May 9, 2026
Search Tags:#github actions#price tracking#data scraping#notion integration#job monitoring#ai enrichment#automated collection
One-click install
npx skills add https://github.com/RambleRainbow/jd --skill data-scraper-agent-ramblerainbow
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-scraper-agent
Source: https://github.com/RambleRainbow/jd/tree/main/.claude/skills/data-scraper-agent
Command: npx skills add https://github.com/RambleRainbow/jd --skill data-scraper-agent-ramblerainbow

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Manually collecting and tracking public data from job boards, e-commerce sites, news feeds, and other sources is time-consuming, difficult to schedule, and often requires paid hosting. This Skill eliminates that overhead by enabling you to build fully automated, AI-powered data collection agents that run for free on GitHub Actions.

Core Features & Use Cases

  • Scheduled Scraping: Automatically collect data from any public website, API, or RSS feed on a custom schedule (hourly, daily, weekly etc.)
  • AI Enrichment: Use free Gemini Flash to score, summarise, and classify collected items based on your custom priorities and context
  • Flexible Storage: Push results directly to Notion, Google Sheets, or Supabase for easy review and analysis
  • Feedback Learning: The agent improves over time by learning from your accept/reject decisions on collected items
  • Common Use Cases: Monitor job listings for relevant roles, track product prices for drops, aggregate industry news, summarise new GitHub releases, and more

Quick Start

Use the data-scraper-agent skill to build an automated agent that monitors Hacker News for AI startup funding news and stores scored results in your Notion workspace.

Frequently Asked Questions about data-scraper-agent

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate scheduled web scraping without paying for hosting?▼

Automated data scraping can run for free by using GitHub Actions to schedule Python scripts that collect public data from websites, APIs, or RSS feeds without requiring paid hosting or servers.

Can I use AI to enrich and summarize scraped data automatically?▼

Yes, AI enrichment processes collected scraping data by using Gemini Flash to automatically score, summarize, and classify items based on your custom priorities and context.

How do I save scraped data directly to Notion or Google Sheets?▼

Scraped data can be pushed directly to Notion, Google Sheets, or Supabase storage backends, enabling easy review and persistent tracking of newly collected items in your preferred workspace.

Does this data scraping approach work for tracking product prices and job listings?▼

Yes, the automated collection supports job board monitoring, product price tracking, news aggregation, and GitHub repository tracking by scheduling regular checks on public web sources.

Can the scraping agent learn from my feedback to improve data relevance?▼

Yes, the agent improves over time through feedback learning, adjusting its scoring based on your accept or reject decisions on previously collected items to refine future data relevance.

Related Skills

Explore More Data & Analytics Skills→
sakamoto-family-smilesakamoto-family-smile

data-scraper-agent

Scrape public sources via REST, HTML, or RSS and store enriched data.

Community
Advanced
OneWave-AIOneWave-AI
245

contact-hunter

Find and enrich contact information from public sources with structured outputs.

Official
Basic
yunseo-kimyunseo-kim
1

github-triage

Classify GitHub issues and pull requests via gh CLI and Python scripts.

Community
Advanced
idiaz01idiaz01
3

data-scraper-agent

Automates scraping, enriching, and storing web data to Notion, Google Sheets, or Supabase via GitHub Actions.

Community
Advanced