skills.rest
Adelie-Squad
Adelie-SquadOFFICIAL
Explore all skills by this author →
GitHub

Data Collector Agent

Collect structured telemetry, API responses, and scraped content via Python and Playwright.

17|1|Updated Feb 3, 2026
Search Tags:#playwright#airflow#data-pipeline#web-scraping#data-collection#event-tracking#api-ingestion
One-click install
npx skills add https://github.com/Adelie-Squad/solopreneur-team-agents --skill data-collector-agent
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: Data Collector Agent
Source: https://github.com/Adelie-Squad/solopreneur-team-agents/tree/main/agents/engineering/data-collector
Command: npx skills add https://github.com/Adelie-Squad/solopreneur-team-agents --skill data-collector-agent

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Manual data collection across APIs, web pages, and event schemas is fragmented and technically detail-heavy, and this Skill layers in a consistent strategy so founders can rely on a specialist agent instead of stitching workflows themselves.

Core Features & Use Cases

  • API & Web Collection: Designs authenticated, paginated API requests and Playwright-powered scrapes while honoring rate limits and retries.
  • Event Tracking Design: Catalogs telemetry, defines event schemas, and aligns client instrumentation with backend storage expectations.
  • Pipeline Orchestration: Recommends Airflow/DAG, scheduling, and artifact handoff patterns so the collected data reliably lands with engineers or analysts.

Quick Start

Tell the Data Collector Agent to design the API, scraping, and event tracking strategy for capturing the next product launch metrics.

Frequently Asked Questions about Data Collector Agent

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I design a data collection strategy for paginated API responses and web scraping?▼

To design a data collection strategy, this Skill builds asynchronous Python collectors that handle authenticated, paginated API requests and Playwright browser automation for web scraping, honoring rate limits and implementing retries to reliably gather structured telemetry and scraped content.

What is the best way to orchestrate a web scraping and API ingestion pipeline with Airflow?▼

The best way to orchestrate an API ingestion and web scraping pipeline is by defining Airflow DAGs. This Skill recommends scheduling and artifact handoff patterns so collected data reliably lands with data engineers or analysts for downstream use.

How do I catalog event tracking schemas for telemetry data collection?▼

To catalog event tracking schemas for telemetry data collection, this Skill aligns client instrumentation with backend storage expectations. It defines JSON schemas to ensure structured telemetry is consistently captured across products.

Can I use Playwright for browser automation in an asynchronous Python data pipeline?▼

Yes, you can use Playwright for browser automation within an asynchronous Python data pipeline. This Skill specifically applies Playwright to execute web scraping tasks while managing rate limits and retries for reliable content collection.

Do I need asynchronous Python collectors to handle rate limits and retries for API ingestion?▼

Yes, asynchronous Python collectors are required. This Skill uses them to manage the complexities of API ingestion, specifically handling rate limits and implementing retries during authenticated and paginated data requests.

Related Skills

Explore More Data & Analytics Skills→
JuanMS20JuanMS20

github-auth

Configure GitHub authentication for Hermes Agent using git, gh CLI, or SSH credentials.

Community
Intermediate
shaharKeisarAppsshaharKeisarApps

store5-room-bridge

Implement offline-first data pipelines with Store5 and Room as SourceOfTruth.

Community
Advanced
pengfitpengfit

playwright-cli

Execute Playwright CLI commands for browser automation and web scraping.

Community
Intermediate
dallenpyrahdallenpyrah

writing-shape

Transform raw notes and drafts into structured articles through interactive text shaping.

Community
Advanced