data-scraper-agent

Name: data-scraper-agent
Availability: InStock
Author: brucesongs

Collect unstructured web data from CVE databases and threat feeds into structured JSON/CSV records.

60|14|Updated Apr 27, 2026

Search Tags:#automation #cve #osint #threat-intel #data-scraper

Ready to run

npx skills add https://github.com/brucesongs/kali-claw --skill data-scraper-agent-brucesongs

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: data-scraper-agent
Source: https://github.com/brucesongs/kali-claw/tree/main/skills/data-scraper-agent
Command: npx skills add https://github.com/brucesongs/kali-claw --skill data-scraper-agent-brucesongs

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Aggregates unstructured data from CVE databases, threat feeds, and advisories into structured records to accelerate vulnerability research.

Core Features & Use Cases

Automated data collection from CVE databases, threat intel feeds, exploit databases, and vendor advisories.
Structured normalization and deduplication to unify disparate sources.
Incremental syncing with caching and JSON/CSV output for easy integration into existing pipelines.
Use Case: Security teams track CVEs affecting assets, correlate with internal inventory, and generate actionable reports.

Quick Start

Run the data-scraper-agent to fetch CVE data and threat intel feeds and output structured records.

Frequently Asked Questions about data-scraper-agent

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I automate CVE data collection from NVD and Exploit-DB?▼

Automate CVE data collection by running an API-first extraction pipeline that fetches from NVD, Exploit-DB, MITRE, and GitHub advisories. The pipeline uses pluggable source adapters to collect, normalize, and deduplicate unstructured web data into structured records.

What is the best way to aggregate threat intelligence feeds into structured JSON or CSV?▼

Aggregate threat intelligence feeds into structured JSON or CSV using an extraction pipeline with Extractor, Parser, Transformer, and Loader stages. It normalizes disparate sources, deduplicates entries, and outputs files for easy integration into existing security pipelines.

Can I use this data scraper to monitor vendor advisories with rate limiting?▼

Yes, you can monitor vendor advisories with configurable rate limiting and caching. The scraper enforces ethical scraping guidelines while incrementally syncing security advisories to prevent duplicate records and avoid overwhelming source APIs.

Does the threat intel scraper support incremental syncing for vulnerability research?▼

Yes, the threat intel scraper supports incremental syncing with built-in caching for vulnerability research. This allows security teams to track newly published CVEs affecting assets, correlate findings with internal inventory, and generate actionable reports without rescanning entire databases.

How do I normalize unstructured security advisory data across multiple sources?▼

Normalize unstructured security advisory data by applying the pipeline's Transformer stage to unify records from NVD, MITRE, and Exploit-DB. The system automatically handles data normalization and deduplication to consolidate disparate threat intel feeds into cohesive knowledge units.

What are the limitations of automated web scraping for threat intelligence?▼

Automated web scraping for threat intelligence is limited by enforced ethical scraping guidelines and configurable rate limits. Users must respect source API constraints and rely on pluggable adapters for supported sources like NVD, Exploit-DB, MITRE, and GitHub advisories rather than arbitrary web pages.