skills.rest
Sake-Team
Sake-TeamOFFICIAL
Explore all skills by this author →
GitHub

data-scraper-agent

Automate Python-based scraping, AI enrichment, and storage to Notion, Sheets, or Supabase.

Updated Nov 19, 2025
Search Tags:#data collection#AI analysis#data scraping#Automation#public data
One-click install
npx skills add https://github.com/Sake-Team/SmartSake --skill data-scraper-agent-sake-team
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: data-scraper-agent
Source: https://github.com/Sake-Team/SmartSake/tree/main/backup/skills/data-scraper-agent
Command: npx skills add https://github.com/Sake-Team/SmartSake --skill data-scraper-agent-sake-team

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires requests, beautifulsoup4, playwright, notion-client, gspread, supabase, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill automates the collection, analysis, and storage of public data from various sources, saving time and reducing manual labor.

Core Features & Use Cases

  • Data Collection: Gather data from job boards, prices, news, GitHub, sports, and more using scheduled scraping.
  • AI Enrichment: Utilize a free LLM for data summarization, classification, and scoring.
  • Storage: Store results in Notion, Google Sheets, or Supabase.
  • Use Case: Monitor price changes for products over time or track job openings in a specific field.

Quick Start

Run the data-scraper-agent to scrape and analyze data from the 'example-website.com' domain every day.

Frequently Asked Questions about data-scraper-agent

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate data scraping from public websites on a schedule?▼

Automated data scraping uses Python requests and BeautifulSoup to gather public data, while scheduled tasks ensure the collection runs periodically and saves results to Notion, Google Sheets, or Supabase.

Does this data collection approach work with JavaScript-rendered pages?▼

JavaScript-rendered pages are supported through the optional Playwright dependency, enabling the Python data collection scripts to scrape dynamically loaded content alongside standard requests.

Can I use AI analysis to enrich scraped public data?▼

AI analysis enriches scraped public data by leveraging a free LLM to perform automated text summarization, data classification, and relevance scoring during the processing workflow.

Do I need a Python environment to run this data scraping automation?▼

A Python environment is required to execute the data scraping automation, as the processing logic depends on Python libraries like requests and beautifulsoup4 to function effectively.

What is the best way to store scraped data in Google Sheets and Supabase?▼

Storing scraped data in Google Sheets and Supabase is achieved through API integration, sending processed public data directly from Python scripts using gspread and supabase-client libraries.

What are the limitations of using BeautifulSoup for public data scraping?▼

A key limitation of BeautifulSoup for public data scraping is its inability to execute JavaScript or render dynamic content, requiring the optional Playwright integration for such pages.

Related Skills

Explore More Data & Analytics Skills→
rlagycksrlagycks
3

data-scraper-agent

Scrape, process, and store public data from websites, APIs, and RSS feeds.

Community
Advanced
neuron-oneneuron-one
25

data-analysis

Analyze Google Sheets, Notion and HubSpot data via Composio to generate charts and insights.

Community
Intermediate
Michae2xlMichae2xl

data-scraper-agent

Automate public data collection and AI enrichment with Python and Gemini Flash.

Community
Advanced
LuminairPrimeLuminairPrime

mcp-firecrawl

Scrape web pages and extract structured content using Firecrawl.

Community
Advanced