web-scraper

Web scraping dynamic content involves rendering JavaScript via Playwright before parsing the HTML structure. This skill handles Single Page Applications by executing scripts and waiting for dynamic elements to load before applying CSS selectors or LLM extraction.

LLM extraction allows you to pull unstructured data from web pages using natural language instructions instead of predefined CSS selectors. This skill uses AI to interpret page content and extract specific information based on your descriptive prompts.

Deep crawling a documentation site involves following internal links to fetch multiple pages and converting the HTML content into Markdown. This skill supports deep crawling to systematically extract and compile entire documentation sites into searchable Markdown files.

Batch processing multiple URLs allows you to scrape a list of links from a file efficiently. This skill automates fetching and extracting structured data across numerous URLs simultaneously, streamlining large-scale data extraction and full page scraping tasks.

HTML parsing uses Beautiful Soup to navigate the document tree and apply CSS selectors for targeted data extraction. For complex scenarios, this skill escalates to JavaScript rendering or LLM-based extraction to handle dynamic content and unstructured data.