skills.rest
gebruder
gebruderCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

web-fetch

Fetch web content from URLs with curl and parse JSON APIs using jq.

166|4|Updated Mar 25, 2026
Search Tags:#data retrieval#curl#api calls#json parsing#url extraction
One-click install
npx skills add https://github.com/gebruder/wirken --skill web-fetch-gebruder
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: web-fetch
Source: https://github.com/gebruder/wirken/tree/main/skills/web-fetch
Command: npx skills add https://github.com/gebruder/wirken --skill web-fetch-gebruder

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires curl, jq, and includes scripts (resource) and assets (resource) components.

What problem does it solve?

Streamlines the process of retrieving and parsing online content from various websites, reducing the manual work involved in scraping data.

Core Features & Use Cases

  • URL Content Extraction: Automatically fetch HTML, plain text, and headers from URLs.
  • Redirect Following: Follow HTTP redirects without manual intervention.
  • File Downloading: Supports downloading files and custom-naming of downloads.
  • API Integration: Capable of handling JSON APIs using jq.
  • Use Case: Efficiently pull the latest news updates or fetch data from an API endpoint.

Quick Start

Fetch all data from 'https://example.com/api/news' using the 'web-fetch' skill.

Frequently Asked Questions about web-fetch

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I extract web content from a URL using curl?▼

To extract web content from a URL, web-fetch uses curl to retrieve HTML, plain text, and headers. It automates redirection handling to streamline fetching data from specified web pages without manual intervention.

How do I parse JSON API responses with jq?▼

You can parse JSON API responses with jq by using this skill to handle API integration. It fetches data from API endpoints and processes JSON payloads using jq for structured data extraction.

Does web scraping with this skill require manual intervention for HTTP redirects?▼

No, web scraping does not require manual intervention for HTTP redirects. The skill performs automated redirect following, allowing seamless access to target resources even when URLs change location.

Can I download files and extract text from web pages?▼

Yes, you can download files and extract text from web pages. The skill supports file downloading with custom-naming capabilities and automatically fetches plain text or HTML content from target URLs.

What is the best way to automate data retrieval from web pages?▼

The best way to automate data retrieval is using a skill that combines curl and jq. It streamlines scraping data online, reducing manual work by fetching HTML, handling redirects, and parsing JSON APIs.

Related Skills

Explore More Software Engineering Skills→
vm0-aivm0-ai
76

qiita

Search, read, and publish Qiita articles via CLI with curl and jq.

Official
Advanced
1thirteeng31thirteeng3

youtube-content

Fetch YouTube transcripts and generate structured chapters, summaries, threads, and blog posts.

Community
Intermediate
thimsluggathimslugga

jq-json-processor

Filter, map, and transform JSON data using jq syntax.

Community
Intermediate
zeronx798zeronx798

scrapling

Scrape static, dynamic, and protected websites using the scrapling library.

Community
Advanced