skills.rest
sumeetonline90
sumeetonline90COMMUNITY
Explore all skills by this author →
GitHub

data-scraper-agent

Collect, enrich, and store public data from web sources, APIs, and RSS feeds.

Updated Jun 25, 2026
Search Tags:#github-actions#web-scraping#batch-processing#data-automation#notion-integration#public-data#gemini-ai
One-click install
npx skills add https://github.com/sumeetonline90/fitup_all --skill data-scraper-agent-sumeetonline90
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-scraper-agent
Source: https://github.com/sumeetonline90/fitup_all/tree/main/.cursor/skills/data-scraper-agent
Command: npx skills add https://github.com/sumeetonline90/fitup_all --skill data-scraper-agent-sumeetonline90

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Manually scraping, monitoring, and collecting public data from job boards, price trackers, news sites, and other sources is tedious, requires paid hosting for automation, and lacks intelligent enrichment to prioritize relevant results.

Core Features & Use Cases

  • Automated Scheduled Scraping: Runs on a free GitHub Actions schedule to collect data from any public website, API, or RSS feed without paid hosting.
  • AI-Powered Enrichment: Uses free Gemini Flash to score, summarize, and classify collected items based on your custom priorities and context.
  • Adaptive Learning: Improves scoring over time by learning from your feedback on which items you save, skip, or reject.
  • Use Case Example: Track new GitHub repositories tagged with "llm-agents", summarize each new release, and store prioritized results in Supabase for your team to review.

Quick Start

Use the data-scraper-agent skill to build a custom agent that monitors Hacker News for AI startup funding news, scores each post for relevance to your machine learning interests, and saves results to your Notion workspace.

Frequently Asked Questions about data-scraper-agent

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate web scraping for public data without paying for hosting?▼

You can automate web scraping without paid hosting by running scheduled data collection jobs on GitHub Actions, which fetches public data from websites, APIs, and RSS feeds for free.

Can I use Gemini AI to summarize and score scraped RSS feed data?▼

Yes, Gemini Flash AI can score, summarize, and classify scraped RSS feed data based on your custom priorities, using an adaptive model fallback chain to avoid rate limits.

Does batch processing help avoid rate limits when collecting data from job boards?▼

Batch processing helps avoid API rate limits when collecting data from job boards by grouping data requests, allowing the scraper to process large volumes of public data efficiently.

How do I save scraped and enriched data to Notion or Google Sheets?▼

You can save scraped and enriched data to Notion or Google Sheets by integrating these platforms as storage endpoints, automatically pushing collected and classified items directly into your workspace.

What are the limitations of using free AI models for data enrichment in web scraping?▼

Limitations of using free AI models for data enrichment include strict rate limits and quota constraints, which are mitigated by batch processing and implementing model fallback chains during data collection.

Related Skills

Explore More Data & Analytics Skills→
davpatel605-beepdavpatel605-beep

watchers

Poll RSS feeds, JSON APIs, and GitHub repositories for new updates with deduplication.

Community
Intermediate
richardnpaulrichardnpaul

data-scraper-agent

Collect public data from sources into Notion, Google Sheets, or Supabase.

Community
Advanced
Michae2xlMichae2xl

data-scraper-agent

Automate public data collection and AI enrichment with Python and Gemini Flash.

Community
Advanced
ShaivpidadiShaivpidadi
231

freeride

Automate model switching and fallback configuration for OpenClaw AI services.

Community
Advanced