skills.rest
Aradotso
AradotsoOFFICIAL
Explore all skills by this author →
Website𝕏@aradotsoGitHub

autocli-web-scraping

Fetch web data from 55+ sites via a single CLI workflow.

4|Updated May 16, 2026
Search Tags:#browser automation#web scraping#json output#cli automation#social media data#adapter generation#multi-site data collection
One-click install
npx skills add https://github.com/Aradotso/devtools-skills --skill autocli-web-scraping
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: autocli-web-scraping
Source: https://github.com/Aradotso/devtools-skills/tree/main/skills/autocli-web-scraping
Command: npx skills add https://github.com/Aradotso/devtools-skills --skill autocli-web-scraping

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

You need quick, repeatable access to trending and structured data across many websites, but manual browsing and custom scraping scripts are slow, brittle, and hard to maintain.

Core Features & Use Cases

  • One-command web data fetching: Pull information from 55+ supported sites (Twitter/X, Reddit, YouTube, Bilibili, Zhihu, Xiaohongshu, and more) using a single Rust-based CLI.
  • Public APIs and browser automation: Use fast public commands without a browser for supported sites, and enable extension-based browser commands when the content requires an authenticated session.
  • AI-powered adapter generation: Generate and share custom site adapters with autocli.ai for scraping new or hard-to-access pages via declarative YAML pipelines.
  • Multi-format outputs: Emit results as table, JSON, YAML, CSV, or Markdown to integrate with downstream tooling.

Quick Start

Run autocli twitter trending to fetch current Twitter/X trending topics from the supported dataset.

Frequently Asked Questions about autocli-web-scraping

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape multiple social media sites using a single CLI workflow?▼

You can scrape social media sites using a single Rust-based CLI to fetch trending topics, search results, and structured data. It supports 55+ platforms like Twitter, Reddit, and YouTube, outputting data in JSON, CSV, or Markdown formats.

Do I need a logged-in browser session to scrape content from public APIs?▼

You do not need a browser session for public API scraping. However, accessing content requiring authentication needs the Chrome extension configured with a logged-in session to correctly reuse the browser access.

What is the best way to extract web data from sites without custom scraping scripts?▼

The best way to extract web data without brittle custom scripts is using an AI-powered adapter generation pipeline. It builds declarative YAML configurations to scrape new or hard-to-access pages for multi-format output.

Can I download transcripts and media metadata from YouTube and Bilibili?▼

Yes, you can download transcripts and extract media metadata from platforms like YouTube and Bilibili. The CLI fetches this structured data directly through public commands or extension-based browser session reuse.

Does this web scraping tool work for multi-site data collection at scale?▼

Yes, the tool is designed for multi-site data collection at scale, supporting 55+ sites in a blazing-fast CLI workflow. It applies Rust-based performance to fetch trending discovery and structured scraping results efficiently.

Related Skills

Explore More Data & Analytics Skills→
nashsunashsu
909

autocli

Route natural language commands to 55+ websites through Chrome login state.

Community
Advanced
bendruckerbendrucker
16

json

Process JSON data with jq filters for parsing, filtering, and transforming.

Community
Basic
vm0-aivm0-ai
1.1k

agent-browser

Automate browser navigation, form filling, screenshots, and data extraction.

Official
Intermediate
RanaPriyanshRanaPriyansh

twitter-scraping

Scrape Twitter profiles, followers, tweets, and media via Node.js or browser scripts.

Community
Intermediate