skills.rest
Mysense775
Mysense775COMMUNITY
Explore all skills by this author →
GitHub

web-hunter

Search, scrape, and aggregate data from Reddit, HackerNews, and arXiv.

Updated Feb 27, 2026
Search Tags:#information retrieval#search#web scraping#web crawling#data aggregation#fact checking
One-click install
npx skills add https://github.com/Mysense775/deya-openclaw --skill web-hunter
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: web-hunter
Source: https://github.com/Mysense775/deya-openclaw/tree/main/web-hunter
Command: npx skills add https://github.com/Mysense775/deya-openclaw --skill web-hunter

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires aiohttp, playwright, python-dateutil, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill streamlines the process of gathering information from the web, overcoming the challenges of scattered data sources, dynamic content, and the need for factual verification.

Core Features & Use Cases

  • Multi-Source Aggregation: Search across platforms like Reddit, HackerNews, and arXiv simultaneously for comprehensive information gathering.
  • Dynamic Web Scraping: Parse JavaScript-rendered websites and extract specific data points using advanced browser automation.
  • Fact Verification: Cross-reference claims against reputable sources to ensure accuracy and identify misinformation.
  • Use Case: A marketing analyst needs to understand public sentiment about a new product. They can use this Skill to aggregate discussions from Reddit and Twitter, parse competitor websites for pricing, and verify any bold claims made in press releases.

Quick Start

Use the web-hunter skill to find the latest news about AI breakthroughs on Reddit and HackerNews.

Frequently Asked Questions about web-hunter

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I scrape dynamic JavaScript-rendered websites for data aggregation?▼

Scraping dynamic JavaScript-rendered websites requires advanced browser automation using Playwright to execute scripts and extract specific data points. This approach systematically collects content from diverse online sources to overcome information overload.

What is the best way to aggregate information across Reddit, HackerNews, and arXiv?▼

Multi-source aggregation allows you to search across platforms like Reddit, HackerNews, and arXiv simultaneously. This provides comprehensive information gathering and systematically solves the problem of scattered data sources.

Can I use web crawling to verify facts and cross-reference claims against reputable sources?▼

Yes, you can use web crawling for fact verification to cross-reference claims against reputable sources. This ensures accuracy and helps identify misinformation by systematically collecting and analyzing online content.

Does web scraping for fact checking require specific environment dependencies?▼

Fact checking and dynamic website parsing require specific environment dependencies including aiohttp, playwright, and python-dateutil. These libraries enable advanced browser automation and asynchronous data retrieval.

How do I monitor price changes through web scraping and script execution?▼

Monitoring price changes through web scraping involves executing scripts to retrieve and parse dynamic website content. This deterministic task allows you to systematically extract specific data points and track fluctuations over time.

Related Skills

Explore More Marketing & Sales Skills→
gatovillanogatovillano

kai_measurement_pipeline

Measure KAI hallucination rate, Recall@5, and tool success rate via API queries.

Community
Advanced
EthanAlgoXEthanAlgoX
65

market-monitor

Monitor market feeds and generate prioritized alerts with human-readable summaries.

Community
Advanced
liangdabiaoliangdabiao
264

content-analysis

Extract sentiment, topics, keywords, and insights from text content.

Community
Advanced
olostepolostep

batch

Scrape up to 10,000 URLs in parallel with configurable output formats.

Community
Intermediate