Verified:GitHub

doc2kb

Convert mixed document corpora into an LLM-optimized knowledge base with manifest indexing.

11|2|Updated Apr 2, 2026

Search Tags:#knowledge base #document ingestion #local-first #RAG preparation #PDF conversion #DOCX/PPTX parsing #LLM indexing

Ready to run

npx skills add https://github.com/zevtos/agentpipe --skill doc2kb

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: doc2kb
Source: https://github.com/zevtos/agentpipe/tree/main/skills/doc2kb
Command: npx skills add https://github.com/zevtos/agentpipe --skill doc2kb

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pymupdf4llm, pdfplumber, pypdf, pikepdf, python-magic, python-docx, mammoth, python-pptx, openpyxl, trafilatura, markdownify, charset-normalizer, Pillow, tiktoken, lxml, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

doc2kb turns a folder of heterogeneous documents into an LLM-optimized knowledge base that can be ingested in a separate Claude/Codex session, without losing content via summarization.

Core Features & Use Cases

Structured, per-source extraction (no summarization): Keeps document content verbatim and stores each source as its own Markdown file with YAML frontmatter for indexing and citation.
Mixed-format corpus support: Converts PDFs, DOCX, PPTX, HTML, Markdown, TXT, and Jupyter notebooks into a unified KB layout (with manifests and navigation).
PDF robustness for math-heavy/scanned cases: Detects problematic PDFs (ligature issues, mangled visual math, dropped figures) and can optionally use a VLM-grade MinerU tier for recovery.
Local-first ingestion workflow: Produces manifest.json, INDEX.md, llms.txt, and AGENTS.md so an AI can selectively read only relevant documents via filenames/headings.

Quick Start

Ask the agent to ingest and index a mixed document folder by running doc2kb on your corpus path so it outputs a ready-to-ingest knowledge base.

Frequently Asked Questions about doc2kb

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I prepare mixed documents for RAG ingestion without losing content?▼

To prepare mixed documents for RAG ingestion without losing content, you need a knowledge base preprocessing tool that extracts text verbatim from heterogeneous formats. This skill converts PDFs, DOCX, and HTML into per-source Markdown files with YAML frontmatter, ensuring strict verbatim preservation for downstream LLM indexing.

What is the best way to convert PDF, DOCX, and PPTX files into a unified knowledge base for LLM indexing?▼

The best way to convert mixed-format corpora into a unified knowledge base for LLM indexing is through structured per-source extraction. This process parses PDF, DOCX, PPTX, and HTML files into individual Markdown files, generating a manifest.json and INDEX.md for selective AI retrieval.

Can I extract text from scanned PDFs or math-heavy documents for a local knowledge base?▼

Yes, you can extract text from scanned or math-heavy PDFs for a local knowledge base using optional VLM-grade recovery. The system detects problematic PDFs with ligature issues or mangled visual math and applies an opt-in VLM tier to recover content safely during ingestion.

Does RAG preparation work with local-first document extraction for Jupyter notebooks and TXT files?▼

RAG preparation works with local-first document extraction for Jupyter notebooks, TXT, and Markdown files by applying deterministic local extraction. It processes mixed inputs into a unified KB layout, producing llms.txt and AGENTS.md files so AI can selectively read relevant documents via filenames and headings.

Why does my PDF conversion drop figures and mangle math formulas during LLM indexing?▼

PDF conversion drops figures and mangles math formulas during LLM indexing due to ligature issues and complex visual math structures. The extraction system detects these problematic PDFs and offers an optional VLM-grade MinerU tier to recover dropped figures and mangled visual math safely.

Do I need to summarize documents before adding them to a knowledge base for Claude?▼

No, you do not need to summarize documents before adding them to a knowledge base for Claude. This ingestion workflow applies structured per-source extraction that keeps document content verbatim, allowing an AI to selectively read only relevant documents without losing content through summarization.