defuddle

To extract clean markdown from a web page, you need a tool that strips away navigation and ads, leaving only core article text. This process significantly reduces token usage for language models by outputting just the essential content.

Stripping website clutter before processing article text involves removing sidebars, advertisements, and navigation menus. This ensures the AI assistant receives a token-light text representation containing only the core article content.

Yes, you need npm to install the required CLI tool for converting web pages to markdown. The extraction process relies on this command line interface utilizing a specific markdown flag to output the clean text.

Yes, you can extract metadata like titles and descriptions alongside the main article text. The extraction process optionally provides title, description, and domain information while generating the clean markdown content.

Markdown content extraction works best on standard web pages, articles, blogs, and online documentation. These page types benefit from having their core content cleanly isolated from surrounding structural clutter.

Processing raw web page HTML inflates token usage because it includes unnecessary navigation menus, sidebars, and advertisements. Extracting only the core article content as markdown avoids this overhead and makes downstream prompting more efficient.