Retrieve and extract text or raw HTML from URLs with robots.txt compliance.
npx skills add https://github.com/caoronglin/copaw-skills --skill mcp-fetch
Please help me install this Agent Skill. Skill: MCP Fetch Source: https://github.com/caoronglin/copaw-skills/tree/main/skills/mcp-fetch Command: npx skills add https://github.com/caoronglin/copaw-skills --skill mcp-fetch
This Skill automates the process of retrieving content from web pages, making it easy to access and process information from the internet without manual browsing.
Fetch the content from the URL https://example.com/article.
High-intent search queries and answers about installing and using this skill.
To extract content for information retrieval, provide the target web page URL to fetch the main text, titles, and links. The skill handles character encoding automatically and returns simplified text or raw HTML.
Yes, this web scraping tool adheres to robots.txt rules by default. It ensures your content extraction and web crawler activities respect website crawling restrictions without requiring manual configuration.
Yes, you can retrieve raw HTML instead of simplified text when fetching URL content. The format flexibility allows you to specify raw HTML output for tasks requiring original page structure and link extraction.
The best way to automate web page scraping for content aggregation is to supply the target URLs to the skill. It automatically handles character encoding and extracts titles, links, and text without manual browsing.
No, character encoding issues will not affect web content extraction. The skill handles character encoding automatically, ensuring accurate text retrieval and content aggregation from international online sources.
You should not use this URL fetch approach if a website strictly prohibits web scraping in its robots.txt file. The tool adheres to crawling rules by default, which may block extraction from restricted sources.
olostepScrape up to 10,000 URLs in parallel with configurable output formats.
markacianfraniFetch web pages and convert HTML into markdown, text, or raw HTML.
MaxServDevExtract structured text, tables, and insights from PDF, DOCX, XLSX, CSV, TXT, and Markdown files.
zeronx798Scrape static, dynamic, and protected websites using the scrapling library.
