skills.rest
riftzen-bit
riftzen-bitCOMMUNITY
Explore all skills by this author →
GitHub

data-scraper-agent

Collect and monitor public web and API data on a schedule.

1|Updated Apr 7, 2026
Search Tags:#automation#notion#gemini#scraping#github-actions#data-enrichment#web-scraper
One-click install
npx skills add https://github.com/riftzen-bit/gemini-setup --skill data-scraper-agent-riftzen-bit
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-scraper-agent
Source: https://github.com/riftzen-bit/gemini-setup/tree/main/skills/data-scraper-agent
Command: npx skills add https://github.com/riftzen-bit/gemini-setup --skill data-scraper-agent-riftzen-bit

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

The Data Scraper Agent automates monitoring and collection of public web and API data so users no longer manually check sources, extract fields, or aggregate results across sites and feeds.

Core Features & Use Cases

  • Scheduled Collection: Run scrapers on a cron schedule via GitHub Actions to continuously gather data from job boards, marketplaces, news sites, RSS feeds, and APIs.
  • AI Enrichment & Filtering: Batch-enrich and score items using Gemini Flash with model fallback, then apply configurable priorities and deduplication rules.
  • Flexible Storage & Learning: Persist results to Notion, Google Sheets, Supabase (or local files), and store feedback history to improve future scoring and relevance.
  • Use Case: Monitor multiple job boards, score new listings against a resume, and push high-scoring entries into a Notion database automatically.

Quick Start

Build a GitHub Actions-powered scraper that monitors https://example.com/jobs every 3 hours, extracts title, url, and date, enriches items with Gemini Flash, and saves new enriched items to my Notion database.

Frequently Asked Questions about data-scraper-agent

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate continuous web scraping on a schedule?▼

Automate continuous web scraping by running scheduled cron tasks via GitHub Actions to collect public web and API data, enabling continuous monitoring of job boards, price trackers, and news feeds.

Can I enrich scraped data using Gemini Flash before saving it to Notion?▼

Yes, you can batch-enrich scraped data using Gemini Flash with model fallback, apply configurable priorities and deduplication rules by URL, and save the enriched items directly to Notion.

What is the best way to store scraped data from multiple sources?▼

The best way to store scraped data is to persist results to flexible storage destinations like Notion, Google Sheets, Supabase, or local files, while storing feedback history to improve future relevance scoring.

Does GitHub Actions work for scheduled scraping tasks with rate limiting?▼

Yes, GitHub Actions works for scheduled scraping tasks by running cron schedules with built-in rate limiting and model fallback to continuously gather public web and API data safely.

How do I deduplicate scraped items by URL during data collection?▼

Deduplicate scraped items by URL during data collection by applying configurable deduplication rules during the AI enrichment and filtering phase, ensuring only new entries are stored.

Limitations of using AI enrichment for monitoring public API data?▼

Limitations of using AI enrichment include managing model fallback and rate limiting constraints, requiring scheduled GitHub Actions configurations to safely monitor public API data without exceeding quotas.

Related Skills

Explore More Data & Analytics Skills→
woxiangyangzhimaowoxiangyangzhimao
3

data-scraper-agent

Automates scheduled scraping of public sources with AI enrichment into databases like Notion.

Community
Advanced
kk20300113-pngkk20300113-png

data-scraper-agent

Automate scheduled scraping, enrichment, and storage of public data.

Community
Advanced
sumeetonline90sumeetonline90

data-scraper-agent

Collect, enrich, and store public data from web sources, APIs, and RSS feeds.

Community
Advanced
EhsaanArkEhsaanArk

setup-deploy

Detect hosting platforms and write a Deploy Configuration to CLAUDE.md.

Community
Advanced