skills.rest
ComposioHQ
ComposioHQOFFICIAL
Explore all skills by this author →
Website𝕏@composioGitHub

scrapfly-automation

Automate web scraping workflows and data extraction tasks using the Scrapfly API.

74.2k|8.5k|Updated Oct 17, 2025
Search Tags:#api#mcp#scraping#Automation#data-extraction#scrapfly
One-click install
npx skills add https://github.com/ComposioHQ/awesome-claude-skills --skill scrapfly-automation
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: scrapfly-automation
Source: https://github.com/ComposioHQ/awesome-claude-skills/tree/main/composio-skills/scrapfly-automation
Command: npx skills add https://github.com/ComposioHQ/awesome-claude-skills --skill scrapfly-automation

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill removes the complexity of managing web scraping infrastructure and authentication, allowing you to focus on data collection rather than connection maintenance.

Core Features & Use Cases

  • Automated Tool Discovery: Dynamically fetch current Scrapfly tool schemas to ensure compatibility with the latest API updates.
  • Connection Management: Handle authentication and session state for Scrapfly toolkits via the Rube MCP gateway.
  • Use Case: Automate the extraction of product pricing data from e-commerce websites by discovering available scraping tools and executing them within a managed session.

Quick Start

Use the scrapfly-automation skill to discover available tools and execute a scraping task for the target website.

Frequently Asked Questions about scrapfly-automation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I discover available Scrapfly tools?▼

Call the RUBE_SEARCH_TOOLS function with a query related to your specific scraping task. This returns the latest tool slugs, input schemas, and execution plans required for your workflow.

How do I verify my Scrapfly connection status?▼

Use the RUBE_MANAGE_CONNECTIONS function with the 'scrapfly' toolkit parameter. This will return the current connection status and provide an authentication link if the connection is not active.

Can I reuse session IDs for multiple scraping tasks?▼

Yes, you should reuse session IDs within a single logical workflow to maintain state. However, generate new session IDs when starting entirely different scraping projects to avoid data contamination.

What should I do if my scraping tool execution fails?▼

Verify that your connection status is ACTIVE using RUBE_MANAGE_CONNECTIONS and ensure your arguments strictly match the schema returned by RUBE_SEARCH_TOOLS. Always include the memory parameter in your execution calls.

Related Skills

Explore More Software Engineering Skills→
NiravenNiraven

bright-data

Automates web scraping and data collection through the Bright Data proxy network.

Community
Intermediate
KrismithReddy12KrismithReddy12

scrape

Extract structured data from web pages into JSON outputs.

Community
Intermediate
MadAppGangMadAppGang
7

web-scraping

Extract structured data from web pages with pagination and session handling.

Official
Advanced
npc-chrisnpc-chris
1

browse

Automate headless browser navigation, form interaction, and screenshot comparison for QA testing.

Community
Advanced