document-pipeline

Ingest raw documents into structured fragments with dual Markdown outputs.

Updated Apr 8, 2026

Search Tags:#markdown #webp #survey #knowledge-base #fragments #video-subtitles #document-ingestion

Ready to run

npx skills add https://github.com/rd162/skills --skill document-pipeline

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: document-pipeline
Source: https://github.com/rd162/skills/tree/main/document-pipeline
Command: npx skills add https://github.com/rd162/skills --skill document-pipeline

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires markitdown, docling, pyvips, Pillow, python-pptx, openpyxl, python-docx, docx2pdf, PyMuPDF, py7zr, openai-whisper, scenedetect[opencv], opencv-python-headless, tqdm, and includes scripts (resource) and references (resource) components.

What problem does it solve?

Ingests and transforms a mix of documents (PDF, Word, slides), diagrams, and videos into structured fragments and a knowledge base, enabling efficient downstream analysis and knowledge extraction.

Core Features & Use Cases

Ingestion: convert raw documents, diagrams, and videos into fragments with source-tier metadata and versioned frontmatter.
Survey: generate targeted knowledge bases and structured sections (§1–§12) from fragments with source traceability and gap/ambiguity catalogs.
Workflow orchestration: support incremental changes, resume after interruptions, and provide outputs suitable for LLMs.

Quick Start

Run the pipeline to ingest documents into fragments and generate a targeted survey from those fragments.

Frequently Asked Questions about document-pipeline

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I ingest PDF, Word, and video files to build a knowledge base?▼

Ingesting PDF, Word, and video files to build a knowledge base requires converting raw documents into structured fragments with source-tier metadata and versioned frontmatter. This pipeline supports end-to-end workflows from ingestion to targeted surveys.

Can I extract video subtitles and scene changes for document ingestion?▼

Extracting video subtitles and scene changes for document ingestion is supported using OpenAI Whisper and PySceneDetect. The pipeline transforms video content into structured fragments suitable for downstream LLM analysis.

Does this pipeline support rendering images to WEBP during document ingestion?▼

Rendering images to WEBP during document ingestion is fully supported. The pipeline ensures dual Markdown outputs alongside WEBP image rendering, providing optimized visual data for your structured knowledge base.

What is the best way to resume document ingestion after an interruption?▼

Resuming document ingestion after an interruption is handled through built-in workflow orchestration. The pipeline supports incremental changes and tracks modifications, allowing you to resume processing without restarting the entire workflow.

How do I generate targeted survey sections from extracted document fragments?▼

Generating targeted survey sections from extracted document fragments involves structuring knowledge into defined sections (§1–§12). The pipeline creates a survey-ready knowledge base with source traceability and gap or ambiguity catalogs.