skills.rest
mofa-org
mofa-orgOFFICIAL
Explore all skills by this author →
WebsiteGitHub

mofa-crawlee-python

Guide AI to set up and run Crawlee-Python crawlers for web scraping.

11|12|Updated Feb 28, 2026
Search Tags:#automation#python#playwright#crawler#web-scraping#beautifulsoup#crawlee
One-click install
npx skills add https://github.com/mofa-org/mofa-skills --skill mofa-crawlee-python
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: mofa-crawlee-python
Source: https://github.com/mofa-org/mofa-skills/tree/main/_unpublished/mofa-crawlee-python
Command: npx skills add https://github.com/mofa-org/mofa-skills --skill mofa-crawlee-python

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

AI in development environments often struggles to learn and apply Crawlee-Python for robust web scraping and browser automation. This skill provides structured guidance to teach AI how to set up, compose, and execute Crawlee-powered crawlers across multiple engines (BeautifulSoup, Playwright, HttpCrawler), manage data extraction, and export results.

Core Features & Use Cases

  • Onboarding and environment setup for Crawlee-Python (Python 3.9+ required, dependencies).
  • Multi-engine crawler patterns (BeautifulSoup, Playwright, HttpCrawler, Parsel).
  • End-to-end data extraction, handling, and export for JSON/CSV.
  • Practical examples, templates, and configuration guidance for real-world crawling tasks.

Quick Start

Install Crawlee-Python, create a basic crawler, and run it to verify the setup.

Frequently Asked Questions about mofa-crawlee-python

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I build a Python web scraper for dynamic sites?▼

Crawlee-Python supports building robust web crawlers using Playwright to automate browsers, extract data, and export results to JSON or CSV.

What is the best way to export scraped data to JSON or CSV in Python?▼

Exporting scraped data to JSON or CSV is handled natively by Crawlee-Python, which provides end-to-end data extraction, handling, and export configurations for your web scraping tasks.

Does Crawlee-Python work with BeautifulSoup and Playwright?▼

Crawlee-Python works with BeautifulSoup for static sites and Playwright for dynamic sites, providing multi-engine crawler patterns to handle various web scraping and automation tasks.

Do I need Python 3.9 to use Crawlee-Python for web scraping?▼

Python 3.9 or higher is required to set up the environment for Crawlee-Python, ensuring compatibility for building and executing robust web crawlers and browser automation scripts.

When should I use HttpCrawler vs Playwright for web scraping?▼

Use HttpCrawler for static content and Playwright for dynamic sites, applying Crawlee-Python's multi-engine patterns to manage data extraction based on target website complexity.

What are the safety best practices for browser automation and crawling?▼

Safety best practices for browser automation are built into Crawlee-Python guidance, ensuring reliable and ethical crawling through proper configuration, runtime management, and structured data extraction.

Related Skills

Explore More Software Engineering Skills→
mitsuhikomitsuhiko
2.8k

uv

Execute Python scripts with inline metadata and per-script dependencies.

Community
Advanced
bendruckerbendrucker
16

json

Process JSON data with jq filters for parsing, filtering, and transforming.

Community
Basic
MonichreMonichre

open-researcher

Gather credible open-source research sources with ranked summaries and citations.

Community
Basic
spideynolovespideynolove
2

lightpanda

Fetch HTML content from web pages using a lightweight headless browser.

Community
Basic