brave-search

Web content extraction as Markdown is achieved by fetching specific URLs and parsing the HTML using jsdom and @mozilla/readability. This process strips out non-essential page elements to deliver clean, readable text directly into your notes or documentation.

Automated web searches for technical documentation are performed by querying the Brave Search API. The Skill retrieves up to 10 search results per query, allowing you to quickly find relevant API references or facts without manual browser interaction.

A Brave Search API key is required to use this web scraping functionality. You must have an active BRAVE_API_KEY configured in your environment to authenticate requests and retrieve search results from the Brave Search API.

This content extraction tool supports GitHub Flavored Markdown through the turndown-plugin-gfm dependency. It works alongside turndown to convert parsed HTML content from web pages into structured Markdown format for research and summarization.

The best way to retrieve information and summarize web pages for research is to combine the Brave Search API for finding sources with @mozilla/readability for content parsing. This approach isolates the main text from web pages, making fact-checking and documentation lookup highly efficient.

Headless browser techniques for web scraping are utilized for content extraction, but the metadata does not specify deep JavaScript rendering capabilities. It relies on jsdom to parse fetched HTML, which may limit extraction from dynamically loaded content.