Extract web content using multi-strategy scraping with anti-bot bypass.
npx skills add https://github.com/KuaaMU/omnihive --skill web-scraping-kuaamu
Please help me install this Agent Skill. Skill: web-scraping Source: https://github.com/KuaaMU/omnihive/tree/main/library/real-skills/web-scraping Command: npx skills add https://github.com/KuaaMU/omnihive --skill web-scraping-kuaamu
This Skill automates the process of extracting valuable content from websites, even when faced with anti-bot measures, paywalls, or dynamic JavaScript rendering.
Use the web-scraping skill to extract the main content from the URL 'https://example.com'.
High-intent search queries and answers about installing and using this skill.
To scrape web content protected by anti-bot measures, this Skill uses Playwright with stealth mode to mimic human browsing and bypass detection. It also rotates user agents and implements polite scraping delays to ensure reliable data extraction.
To extract data from YouTube, Instagram, or TikTok, this Skill provides specific patterns using dedicated tools like yt-dlp and instaloader. These integrations handle platform-specific structures to reliably retrieve social media content.
The best way to scrape websites with dynamic JavaScript rendering is using a multi-strategy scraping cascade. This Skill automatically falls back from basic requests to Trafilatura and ultimately Playwright to ensure all dynamically loaded content is captured.
Poison pill detection helps web scraping by identifying and flagging soft paywalls, CAPTCHAs, and rate limits. This prevents wasted scraping efforts on blocked content and ensures the extraction process stops before triggering anti-bot alarms.
Yes, you can identify and use undocumented APIs for data extraction. This Skill provides methods to discover hidden APIs used by websites, allowing you to retrieve structured data directly without parsing HTML or rendering JavaScript.
You do not always need Playwright to extract main content from a standard URL. This Skill employs a fallback cascade starting with Trafilatura and Requests, only escalating to Playwright for dynamic pages or anti-bot bypass.
MaxMiksaAutomate web extraction with Trafilatura, Requests, Playwright stealth, and yt-dlp or instaloader for sites and social media.
jesseotremblayTransform, clean, and extract structured information from CSV and JSON data.
luis2404123Bypass CAPTCHA challenges during web scraping with proxies and automation.
GrassHopper42Scrape web pages and codify data into browser skills.
